Configure ATS reverse proxy with live HLS origin returning

[ezko]

Hi,
I am trying to configure ATS reverse proxy with a live HLS origin which
returns Cache-Control: no-cache on everything.
By default ATS doesn't cache anything because the origin server objects have
response headers with Cache-Control: no-cache (this is documented).

The only way i found to get it to work was pinning the objects by adding the
following to cache.config:

url_regex=<path to origin> ttl-in-cache=10s

i choose 10 seconds because that is the target duration in the HLS playlist
(#EXT-X-TARGETDURATION)

But this seems like a very manual and verbose method , is there any better
way to configure live content from an origin that returns no-cache ?
Thanks.




--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Miles Libbey]

We tend to use the header_rewrite plugin for this in our remap rules.

map http://inbound.example.com \
http://origin.example.com \
@plugin=header_rewrite.so @pparam=hr.config


With hr.config containing:

cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=604800, public"

(so that we only cache the success not the missing objects).

miles

Post by ezko
Hi,
I am trying to configure ATS reverse proxy with a live HLS origin which
returns Cache-Control: no-cache on everything.
By default ATS doesn't cache anything because the origin server objects have
response headers with Cache-Control: no-cache (this is documented).
The only way i found to get it to work was pinning the objects by adding the
url_regex=<path to origin> ttl-in-cache=10s
i choose 10 seconds because that is the target duration in the HLS playlist
(#EXT-X-TARGETDURATION)
But this seems like a very manual and verbose method , is there any better
way to configure live content from an origin that returns no-cache ?
Thanks.
--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[ezko]

Thanks !!!

But can you explain why you used such a large max-age ?
When i tried it with your max-age value it didn't work (i kept getting the
same old playlist and segments).
So I tried changing max-age to 10 seconds and now it seems to work much
better.

If my HLS playlist has 3 segments and each segment is 10 second each , then
why should i cache anything for more then 30 seconds ?

For live HLS , since everything is short lived and assuming i only have a
small number of streams and enough RAM , is there a way to configure ATS to
work with the in-memory cache only ?

thanks again.

Post by Miles Libbey
We tend to use the header_rewrite plugin for this in our remap rules.
map http://inbound.example.com \
http://origin.example.com \
@plugin=header_rewrite.so @pparam=hr.config
cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=604800, public"
(so that we only cache the success not the missing objects).
miles
On Sun, Dec 3, 2017 at 8:54 AM, ezko &lt;

Post by ezko
Hi,
I am trying to configure ATS reverse proxy with a live HLS origin which
returns Cache-Control: no-cache on everything.
By default ATS doesn't cache anything because the origin server objects have
response headers with Cache-Control: no-cache (this is documented).
The only way i found to get it to work was pinning the objects by adding the
url_regex=

<path to origin>
ttl-in-cache=10s

Post by ezko
i choose 10 seconds because that is the target duration in the HLS playlist
(#EXT-X-TARGETDURATION)
But this seems like a very manual and verbose method , is there any better
way to configure live content from an origin that returns no-cache ?
Thanks.
--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Miles Libbey]

Post by ezko
Thanks !!!
But can you explain why you used such a large max-age ?

It's just an example.

Post by ezko
When i tried it with your max-age value it didn't work (i kept getting the
same old playlist and segments).
So I tried changing max-age to 10 seconds and now it seems to work much
better.
If my HLS playlist has 3 segments and each segment is 10 second each , then
why should i cache anything for more then 30 seconds ?

I think there are 2 distinct HLS cases. For live streaming, the
playlist will change frequently (but usually, the encoded segments
won't). For already recorded content, neither likely change. You might
distinguish between the 2 cases in your remap rule. Or, you could add
extra header_rewrite rules like

cond %{CLIENT-URL:PATH} /\.m3u8/

or perhaps look at the mime type like

cond %{HEADER:Content-Type} /mpegURL/

(though, now re-reading, perhaps you only have the live streaming case.)

Post by ezko
For live HLS , since everything is short lived and assuming i only have a
small number of streams and enough RAM , is there a way to configure ATS to
work with the in-memory cache only ?

No, the ram cache is too intertwined to the disk cache to do that.
miles

Post by ezko
thanks again.

Post by Miles Libbey
We tend to use the header_rewrite plugin for this in our remap rules.
map http://inbound.example.com \
http://origin.example.com \
@plugin=header_rewrite.so @pparam=hr.config
cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=604800, public"
(so that we only cache the success not the missing objects).
miles
On Sun, Dec 3, 2017 at 8:54 AM, ezko &lt;

Post by ezko
Hi,
I am trying to configure ATS reverse proxy with a live HLS origin which
returns Cache-Control: no-cache on everything.
By default ATS doesn't cache anything because the origin server objects have
response headers with Cache-Control: no-cache (this is documented).
The only way i found to get it to work was pinning the objects by adding the
url_regex=

<path to origin>
ttl-in-cache=10s

Post by ezko
i choose 10 seconds because that is the target duration in the HLS playlist
(#EXT-X-TARGETDURATION)
But this seems like a very manual and verbose method , is there any better
way to configure live content from an origin that returns no-cache ?
Thanks.
--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[ezko]

got it.
i will add an extra header_rewrite for .m3u8
thanks again.

Post by ezko
Thanks !!!
But can you explain why you used such a large max-age ?

It's just an example.

Post by ezko
When i tried it with your max-age value it didn't work (i kept getting the
same old playlist and segments).
So I tried changing max-age to 10 seconds and now it seems to work much
better.
If my HLS playlist has 3 segments and each segment is 10 second each , then
why should i cache anything for more then 30 seconds ?

I think there are 2 distinct HLS cases. For live streaming, the
playlist will change frequently (but usually, the encoded segments
won't). For already recorded content, neither likely change. You might
distinguish between the 2 cases in your remap rule. Or, you could add
extra header_rewrite rules like

cond %{CLIENT-URL:PATH} /\.m3u8/

or perhaps look at the mime type like

cond %{HEADER:Content-Type} /mpegURL/

(though, now re-reading, perhaps you only have the live streaming case.)

Post by ezko
For live HLS , since everything is short lived and assuming i only have a
small number of streams and enough RAM , is there a way to configure ATS to
work with the in-memory cache only ?

No, the ram cache is too intertwined to the disk cache to do that.
miles






--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Burak Sarp]

Hi, 
You have to  handle headers on origin servers. In addition to that there are some parameters in records.config ingore no cache both client and servers.
Sarp


Sent from Yahoo Mail for iPhone


On Sunday, December 3, 2017, 8:54 AM, ezko <***@harmonicinc.com> wrote:

Hi,
I am trying to configure ATS reverse proxy with a live HLS origin which
returns Cache-Control: no-cache on everything.
By default ATS doesn't cache anything because the origin server objects have
response headers with Cache-Control: no-cache (this is documented).

The only way i found to get it to work was pinning the objects by adding the
following to cache.config:

url_regex=<path to origin> ttl-in-cache=10s

i choose 10 seconds because that is the target duration in the HLS playlist
(#EXT-X-TARGETDURATION)

But this seems like a very manual and verbose method , is there any better
way to configure live content from an origin that returns no-cache ?
Thanks.




--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/