ATS cache miss if backend service DNS name rotates.

[Jin]

I have a backend server using DNS roundrobin like service.domain.com pointing
to service[1-8].domain.com. They all run the same Ceph Rados gateway service
and serve same data.

Is there a way for ATS to ignore hostname if the file path and names are
same? For example, if http://service1.domain.com/dir1/file1 was cached,
requests for http://service2.domain.com/dir1/file1 should be pulled from
cache instead of from backend server.

Thank you.

Jin.





--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Jeremy Payne]

Maybe use the below plugin to exclude the FQDN as the part of the
cache key or generate the cache key based on the path+file ?

https://docs.trafficserver.apache.org/en/7.1.x/admin-guide/plugins/cachekey.en.html

Post by Jin
I have a backend server using DNS roundrobin like service.domain.com pointing
to service[1-8].domain.com. They all run the same Ceph Rados gateway service
and serve same data.
Is there a way for ATS to ignore hostname if the file path and names are
same? For example, if http://service1.domain.com/dir1/file1 was cached,
requests for http://service2.domain.com/dir1/file1 should be pulled from
cache instead of from backend server.
Thank you.
Jin.
--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Miles Libbey]

I wouldn't recommend excluding the FQDN (in case you get more domains
later, and some portion of their paths have collisions). But, the
prefix section
https://docs.trafficserver.apache.org/en/7.1.x/admin-guide/plugins/cachekey.en.html#prefix-section
does let you normalize them to your liking.

miles

Post by Jeremy Payne
Maybe use the below plugin to exclude the FQDN as the part of the
cache key or generate the cache key based on the path+file ?
https://docs.trafficserver.apache.org/en/7.1.x/admin-guide/plugins/cachekey.en.html

Post by Jin
I have a backend server using DNS roundrobin like service.domain.com pointing
to service[1-8].domain.com. They all run the same Ceph Rados gateway service
and serve same data.
Is there a way for ATS to ignore hostname if the file path and names are
same? For example, if http://service1.domain.com/dir1/file1 was cached,
requests for http://service2.domain.com/dir1/file1 should be pulled from
cache instead of from backend server.
Thank you.
Jin.
--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Leif Hedstrom]

I’m pretty sure that if you enable the pristine host header setting (either globally or per remap rule), the cache key will be based on service.domain.com and not the individual origins. Of course, all origins have to accept that one host header.

— Leif

Post by Jin
I have a backend server using DNS roundrobin like service.domain.com pointing
to service[1-8].domain.com. They all run the same Ceph Rados gateway service
and serve same data.
Is there a way for ATS to ignore hostname if the file path and names are
same? For example, if http://service1.domain.com/dir1/file1 was cached,
requests for http://service2.domain.com/dir1/file1 should be pulled from
cache instead of from backend server.
Thank you.
Jin.
--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Jin]

It worked with pristine host header enabled. However, I am facing an issue
when trying to tspush contents to cache.

The setup has a set of ATS servers cache{01..04} with a DNS roundrobin name
called ats-g1. The backend Ceph storage rados gateway servers are
ceph{01..10} with another DNS roundrobin name ceph-rgw.

A file was pushed into cache using:
tspush -f /path_to/data_file -u http://cache01:8080/path_to/data_file

The file was in place normally and reported by http://{cache} as
http://cache01:8080/path_to/data_file.

When trying to retrieve this file, ATS reported that data_file was not
cached and fetched it from backend Ceph http endpoint. After this attempt,
the following curl shows cache was hit.
curl -x http://cache01:8080 http://ceph-rgw/path_to/data_file

After this, http://{cache} reported two cached data files from two different
source hosts:
http://cache01:8080/path_to/data_file
http://ceph-rgw/path_to/data_file

Is it possible to push data file to the cache server but using another host
name?

Thanks.

Jin.




--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Miles Libbey]

Could you try
curl -x cache01:8080 -s -o /dev/null -X PUSH --data-binary
@/path_to/data_file http://ceph-rgw/the/url/users/would/hit

Post by Jin
It worked with pristine host header enabled. However, I am facing an issue
when trying to tspush contents to cache.
The setup has a set of ATS servers cache{01..04} with a DNS roundrobin name
called ats-g1. The backend Ceph storage rados gateway servers are
ceph{01..10} with another DNS roundrobin name ceph-rgw.
tspush -f /path_to/data_file -u http://cache01:8080/path_to/data_file
The file was in place normally and reported by http://{cache} as
http://cache01:8080/path_to/data_file.
When trying to retrieve this file, ATS reported that data_file was not
cached and fetched it from backend Ceph http endpoint. After this attempt,
the following curl shows cache was hit.
curl -x http://cache01:8080 http://ceph-rgw/path_to/data_file
After this, http://{cache} reported two cached data files from two different
http://cache01:8080/path_to/data_file
http://ceph-rgw/path_to/data_file
Is it possible to push data file to the cache server but using another host
name?
Thanks.
Jin.
--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/

[Jin]

Here is an attempt and failed with below message. Tried to add a header
"Content-Type:application/octet-stream" and saw the first Content-Type
changed but result same error.

1542119404.068 1001 10.40.25.49 ERR_INVALID_REQ/400 539 PUSH
http://ceph-rgw/path_to/data_file - NONE/- text/html


$ curl -v -s -x cache01:8080 -X PUSH --data-binary @/path_to/data_file
http://ceph-rgw/path_to/data_file

PUSH http://ceph-rgw/path_to/data_file HTTP/1.1
User-Agent: curl/7.29.0
Host: ceph-rgw
Accept: */*
Proxy-Connection: Keep-Alive
Content-Length: 584651892
Content-Type: application/x-www-form-urlencoded
Expect: 100-continue

* Done waiting for 100-continue
< HTTP/1.1 400 Malformed Pushed Response Header
< Date: Tue, 13 Nov 2018 14:38:29 GMT
* HTTP/1.1 proxy connection set close!
< Proxy-Connection: close
< Via: http/1.1 cache01 (ApacheTrafficServer/8.0.0 [uEc s f p eH:t cC p s ])
< Server: ATS/8.0.0
< Cache-Control: no-store
< Content-Type: text/html
< Content-Language: en
< Content-Length: 207
<
<HTML>
<HEAD>
<TITLE>Error</TITLE>
</HEAD>

<BODY BGCOLOR="white" FGCOLOR="black">
Error

<HR>


Description: Could not process this request.

<HR>
</BODY>
* we are done reading and this is set to close, stop send
* Closing connection 0


Thank you.

Jin.




--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/