How to purge all cached negative responses

[Veiko Kukk]

Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630
also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache.
It's even good enough if I could get list of objects with certain HTTP
status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects
based on HTTP status code?


Veiko

[David Carlin]

Can you grab the list of objects from the log files? Thats only thing I
can think of.

The cache inspector exists, but I've never had any luck with it. I think
our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

Post by Veiko Kukk
Hi,
We had configuration mistake that enforced Cache-Contro: max-age=157784630
also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from cache.
It's even good enough if I could get list of objects with certain HTTP
status code, then I could write script that purges those objects one by one.
How to do mass purge based on HTTP status code or just get list of objects
based on HTTP status code?
Veiko

[Veiko Kukk]

Thank you for your answer.

Sure I can find those failures from logs, but it's no good because, well,
they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter
based on HTTP status code.

Veiko

Post by David Carlin
Can you grab the list of objects from the log files? Thats only thing I
can think of.
The cache inspector exists, but I've never had any luck with it. I think
https://docs.trafficserver.apache.org/en/latest/admin-
guide/storage/index.en.html#inspecting-the-cache
David

Hi,
max-age=157784630 also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from cache.
It's even good enough if I could get list of objects with certain HTTP
status code, then I could write script that purges those objects one by one.
How to do mass purge based on HTTP status code or just get list of
objects based on HTTP status code?
Veiko

[David Carlin]

I'm suggesting finding the failed requests from the logs, and purging those
URLs so future requests work.

David

Post by Veiko Kukk
Thank you for your answer.
Sure I can find those failures from logs, but it's no good because, well,
they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter
based on HTTP status code.
Veiko

Post by David Carlin
Can you grab the list of objects from the log files? Thats only thing I
can think of.
The cache inspector exists, but I've never had any luck with it. I think
https://docs.trafficserver.apache.org/en/latest/admin-guide/
storage/index.en.html#inspecting-the-cache
David

Hi,
max-age=157784630 also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from cache.
It's even good enough if I could get list of objects with certain HTTP
status code, then I could write script that purges those objects one by one.
How to do mass purge based on HTTP status code or just get list of
objects based on HTTP status code?
Veiko

[Sudheer Vinukonda]

Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).

This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.

Thanks,

Sudheer

I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.
David

Post by Veiko Kukk
Thank you for your answer.
Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.
Veiko

Can you grab the list of objects from the log files? Thats only thing I can think of.
https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache
David

Hi,
We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.
How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?
Veiko

[Miles Libbey]

How about a header_rewrite like
If status>399
Turn off cache.

You might just be better off dropping the entire cache or changing the
cache generation number for that remap ruleset.
miles

On Thursday, December 21, 2017, Sudheer Vinukonda <

Post by Sudheer Vinukonda
Another option is to write a simple plugin that hooks at Cache Lookup
Complete state to overwrite the cache status from hit to stale for negative
responses (status > 399). Unfortunately, I think header rewrite doesn’t
expose the cache lookup Complete hooks, so this has to be written as a
plugin (perhaps, might be simple with Lua?).
This approach won’t purge these objects instantly, but prevents them from
returning to clients from the cache and they eventually get removed
automatically due to cyclic cache.
Thanks,
Sudheer
I'm suggesting finding the failed requests from the logs, and purging
those URLs so future requests work.
David

Post by Veiko Kukk
Thank you for your answer.
Sure I can find those failures from logs, but it's no good because, well,
they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to
filter based on HTTP status code.
Veiko

Can you grab the list of objects from the log files? Thats only thing I can think of.
The cache inspector exists, but I've never had any luck with it. I
https://docs.trafficserver.apache.org/en/latest/admin-guide/
storage/index.en.html#inspecting-the-cache
David

Hi,
max-age=157784630 also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from
cache. It's even good enough if I could get list of objects with certain
HTTP status code, then I could write script that purges those objects one
by one.
How to do mass purge based on HTTP status code or just get list of
objects based on HTTP status code?
Veiko

[Sudheer Vinukonda]

Post by Miles Libbey
How about a header_rewrite like
If status>399
Turn off cache.

It’s too “late” to turn off cache at this point.

Post by Miles Libbey
You might just be better off dropping the entire cache or changing the cache generation number for that remap ruleset.
miles

Post by Sudheer Vinukonda
Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).
This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.
Thanks,
Sudheer

I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.
David

Post by Veiko Kukk
Thank you for your answer.
Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.
Veiko

Can you grab the list of objects from the log files? Thats only thing I can think of.
https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache
David

Hi,
We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.
How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?
Veiko

[Sudheer Vinukonda]

Post by Miles Libbey
How about a header_rewrite like
If status>399
Turn off cache.

It’s too “late” to turn off cache at this point (using header_rewrite with the hooks it currently allows)

Post by Miles Libbey
You might just be better off dropping the entire cache or changing the cache generation number for that remap ruleset.
miles

Post by Sudheer Vinukonda
Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).
This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.
Thanks,
Sudheer

I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.
David

Post by Veiko Kukk
Thank you for your answer.
Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.
Veiko

Can you grab the list of objects from the log files? Thats only thing I can think of.
https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache
David

Hi,
We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.
How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?
Veiko