Discussion:
forward proxy - Restricting domains.
Paul Tader
2015-01-08 17:53:02 UTC
Permalink
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com <http://google.com/>, yahoo.com <http://yahoo.com/>, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.

Thanks everybody,
Paul
Neddy, NH. Nam
2015-01-09 00:00:47 UTC
Permalink
Hi Paul, I'm using DNS server for blacklisting a lot of domains.

~Neddy
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a
internal IP address or IP address range to only be able to proxy certain top
level domains (ie google.com, yahoo.com, etc)? I’ve read a lot on
remapping, but I don’t think that is the correct approach.
Thanks everybody,
Paul
Leif Hedstrom
2015-01-09 02:56:27 UTC
Permalink
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com <http://google.com/>, yahoo.com <http://yahoo.com/>, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).

Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.

— Leif
Paul Tader
2015-01-09 16:00:54 UTC
Permalink
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x

[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!

My remap.conf has these two lines:

.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config

…which is odd because the documentation states:

"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "

http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
James Peach
2015-01-09 16:22:02 UTC
Permalink
Post by Paul Tader
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html
Does your version of ATS match the version of the docs?
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
Paul Tader
2015-01-09 16:33:24 UTC
Permalink
Post by James Peach
Post by Paul Tader
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html
Does your version of ATS match the version of the docs?
Nope and I apologize for that. Time to upgrade.

Thanks everyone.
Post by James Peach
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
Paul Tader
2015-01-09 17:50:28 UTC
Permalink
Post by Paul Tader
Post by James Peach
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html
Does your version of ATS match the version of the docs?
Nope and I apologize for that. Time to upgrade.
Thanks everyone.
Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take. Can something like this be done? (ATS version 3.04)

...
Post by Paul Tader
Post by James Peach
Post by Leif Hedstrom
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
Sudheer Vinukonda
2015-01-09 18:02:57 UTC
Permalink
I think you would need to use named_filters to specify ranges in remap.config.

remap.config — Apache Traffic Server 5.3.0 documentation

|   |
|   |   |   |   |   |
| remap.config — Apache Traffic Server 5.3.0 documentationremap.config The remap.config file (by default, located in/opt/trafficserver/etc/trafficserver/) contains mapping rules that Traffic Serveruses to perform the following actions: |
| |
| View on docs.trafficserver.apache.org | Preview by Yahoo |
| |
|   |

 

On Friday, January 9, 2015 9:50 AM, Paul Tader <***@collectivei.com> wrote:




On Jan 9, 2015, at 10:33 AM, Paul Tader <***@collectivei.com> wrote:


On Jan 9, 2015, at 10:22 AM, James Peach <***@apache.org> wrote:



On Jan 9, 2015, at 8:00 AM, Paul Tader <***@collectivei.com> wrote:

Hmm, I didn’t think about a DNS blackhole.  For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x

[Jan  9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan  9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan  9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan  9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!

My remap.conf has these two lines:

.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config


which is odd because the documentation states:

"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "

http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html


Does your version of ATS match the version of the docs?



Nope and I apologize for that. Time to upgrade.

Thanks everyone.



Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take.  Can something like this be done?  (ATS version 3.04)
...map http://apache.org/   http://apache.org   @action=allow   @src_ip=12.34.56.123map /                    http://127.0.0.1    @action=deny @src_ip=0.0.0.1-254.254.254.254










On Jan 8, 2015, at 8:56 PM, Leif Hedstrom <***@apache.org> wrote:



On Jan 8, 2015, at 10:53 AM, Paul Tader <***@collectivei.com> wrote:

We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)?  I’ve read a lot on remapping, but I don’t think that is the correct approach.



DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).

Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.

— Leif
Paul Tader
2015-01-09 20:30:31 UTC
Permalink
I think this would work, and I think I’m close but I tried this (ver 3 uses .useflt and .defflt instead of .activatefilter and .deactivatefilter):


.defflt disable_all @action=deny
.defflt internal_only @action=allow @src_ip=10.0.0.0-255.255.255.255

.useflt internal_only
map https://www.facebook.com https://www.facebook.com
map https://www.yahoo.com https://www.yahoo.com
map http://finance.yahoo.com http://finance.yahoo.com
.unuseflt internal_only

.useflt disable_all


But going to a site not listed (www.oracle.com) is still allowed. ?
1420835169.093 134 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -

I’ve also tried placing ".useflt disable_all” before the “.useflt internal_only” filter with no luck, sites not on the list are still allowed out.
Post by Sudheer Vinukonda
I think you would need to use named_filters to specify ranges in remap.config.
remap.config — Apache Traffic Server 5.3.0 documentation <https://docs.trafficserver.apache.org/en/latest/reference/configuration/remap.config.en.html#named-filters>
remap.config — Apache Traffic Server 5.3.0 documentation
View on docs.trafficserver.apache.org <https://docs.trafficserver.apache.org/en/latest/reference/configuration/remap.config.en.html#named-filters>
Preview by Yahoo
Post by Paul Tader
Post by James Peach
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html <http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html>
Does your version of ATS match the version of the docs?
Nope and I apologize for that. Time to upgrade.
Thanks everyone.
Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take. Can something like this be done? (ATS version 3.04)
...
Post by Paul Tader
Post by James Peach
Post by Leif Hedstrom
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com <http://google.com/>, yahoo.com <http://yahoo.com/>, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
Sudheer Vinukonda
2015-01-09 20:47:19 UTC
Permalink
You will also need to enable the config proxy.config.url_remap.remap_required (like Leif suggested earlier).


On Friday, January 9, 2015 12:30 PM, Paul Tader <***@collectivei.com> wrote:


I think this would work, and I think I’m close but I tried this (ver 3 uses .useflt and .defflt instead of .activatefilter and .deactivatefilter):

.defflt  disable_all @action=deny.defflt  internal_only @action=allow  @src_ip=10.0.0.0-255.255.255.255
.useflt internal_onlymap https://www.facebook.com    https://www.facebook.commap https://www.yahoo.com       https://www.yahoo.commap http://finance.yahoo.com    http://finance.yahoo.com.unuseflt internal_only
.useflt disable_all

But going to a site not listed (www.oracle.com) is still allowed.  ?1420835169.093 134 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -
I’ve also tried placing ".useflt disable_all” before the “.useflt internal_only” filter with no luck, sites not on the list are still allowed out.


On Jan 9, 2015, at 12:02 PM, Sudheer Vinukonda <***@yahoo-inc.com> wrote:
I think you would need to use named_filters to specify ranges in remap.config.

remap.config — Apache Traffic Server 5.3.0 documentation

|   |
|   |   |   |   |   |
| remap.config — Apache Traffic Server 5.3.0 documentationremap.config The remap.config file (by default, located in/opt/trafficserver/etc/trafficserver/) contains mapping rules that Traffic Serveruses to perform the following actions: |
| |
| View on docs.trafficserver.apache.org | Preview by Yahoo |
| |
|   |

 

On Friday, January 9, 2015 9:50 AM, Paul Tader <***@collectivei.com> wrote:




On Jan 9, 2015, at 10:33 AM, Paul Tader <***@collectivei.com> wrote:


On Jan 9, 2015, at 10:22 AM, James Peach <***@apache.org> wrote:



On Jan 9, 2015, at 8:00 AM, Paul Tader <***@collectivei.com> wrote:

Hmm, I didn’t think about a DNS blackhole.  For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x

[Jan  9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan  9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan  9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan  9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!

My remap.conf has these two lines:

.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config


which is odd because the documentation states:

"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "

http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html


Does your version of ATS match the version of the docs?



Nope and I apologize for that. Time to upgrade.

Thanks everyone.



Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take.  Can something like this be done?  (ATS version 3.04)
...map http://apache.org/   http://apache.org   @action=allow   @src_ip=12.34.56.123map /                    http://127.0.0.1    @action=deny @src_ip=0.0.0.1-254.254.254.254










On Jan 8, 2015, at 8:56 PM, Leif Hedstrom <***@apache.org> wrote:



On Jan 8, 2015, at 10:53 AM, Paul Tader <***@collectivei.com> wrote:

We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)?  I’ve read a lot on remapping, but I don’t think that is the correct approach.



DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).

Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.

— Leif
Paul Tader
2015-01-09 21:29:54 UTC
Permalink
Doesn’t this break the forward proxy then?

# To enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT 1
Post by Sudheer Vinukonda
You will also need to enable the config proxy.config.url_remap.remap_required (like Leif suggested earlier).
.useflt internal_only
map https://www.facebook.com <https://www.facebook.com/> https://www.facebook.com <https://www.facebook.com/>
map https://www.yahoo.com <https://www.yahoo.com/> https://www.yahoo.com <https://www.yahoo.com/>
map http://finance.yahoo.com <http://finance.yahoo.com/> http://finance.yahoo.com <http://finance.yahoo.com/>
.unuseflt internal_only
.useflt disable_all
But going to a site not listed (www.oracle.com <http://www.oracle.com/>) is still allowed. ?
1420835169.093 134 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html <http://www.oracle.com/index.html> - DIRECT/www.oracle.com <http://www.oracle.com/> text/html -
I’ve also tried placing ".useflt disable_all” before the “.useflt internal_only” filter with no luck, sites not on the list are still allowed out.
Post by Sudheer Vinukonda
I think you would need to use named_filters to specify ranges in remap.config.
remap.config — Apache Traffic Server 5.3.0 documentation <https://docs.trafficserver.apache.org/en/latest/reference/configuration/remap.config.en.html#named-filters>
remap.config — Apache Traffic Server 5.3.0 documentation
View on docs.trafficserver.apache.org <https://docs.trafficserver.apache.org/en/latest/reference/configuration/remap.config.en.html#named-filters>
Preview by Yahoo
Post by Paul Tader
Post by James Peach
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html <http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html>
Does your version of ATS match the version of the docs?
Nope and I apologize for that. Time to upgrade.
Thanks everyone.
Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take. Can something like this be done? (ATS version 3.04)
...
Post by Paul Tader
Post by James Peach
Post by Leif Hedstrom
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com <http://google.com/>, yahoo.com <http://yahoo.com/>, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
Leif Hedstrom
2015-01-09 21:38:36 UTC
Permalink
Post by Paul Tader
Doesn’t this break the forward proxy then?
# To enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT 1
That’s somewhat confusing. remap_required disables “open forward proxying”. ATS actually doesn’t know / care about forward vs reverse proxy, it’s just a matter of what requests you allow through. What this setting is saying “Without an explicit rule matching in remap.config, deny the request”. There’s a similar one for reverse proxy.

— Leif
Post by Paul Tader
Post by Sudheer Vinukonda
You will also need to enable the config proxy.config.url_remap.remap_required (like Leif suggested earlier).
.useflt internal_only
map https://www.facebook.com <https://www.facebook.com/> https://www.facebook.com <https://www.facebook.com/>
map https://www.yahoo.com <https://www.yahoo.com/> https://www.yahoo.com <https://www.yahoo.com/>
map http://finance.yahoo.com <http://finance.yahoo.com/> http://finance.yahoo.com <http://finance.yahoo.com/>
.unuseflt internal_only
.useflt disable_all
But going to a site not listed (www.oracle.com <http://www.oracle.com/>) is still allowed. ?
1420835169.093 134 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html <http://www.oracle.com/index.html> - DIRECT/www.oracle.com <http://www.oracle.com/> text/html -
I’ve also tried placing ".useflt disable_all” before the “.useflt internal_only” filter with no luck, sites not on the list are still allowed out.
Post by Sudheer Vinukonda
I think you would need to use named_filters to specify ranges in remap.config.
remap.config — Apache Traffic Server 5.3.0 documentation <https://docs.trafficserver.apache.org/en/latest/reference/configuration/remap.config.en.html#named-filters>
remap.config — Apache Traffic Server 5.3.0 documentation
View on docs.trafficserver.apache.org <https://docs.trafficserver.apache.org/en/latest/reference/configuration/remap.config.en.html#named-filters>
Preview by Yahoo
Post by Paul Tader
Post by James Peach
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html <http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html>
Does your version of ATS match the version of the docs?
Nope and I apologize for that. Time to upgrade.
Thanks everyone.
Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take. Can something like this be done? (ATS version 3.04)
...
Post by Paul Tader
Post by James Peach
Post by Leif Hedstrom
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com <http://google.com/>, yahoo.com <http://yahoo.com/>, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
Paul Tader
2015-01-09 21:51:45 UTC
Permalink
Doesn’t this break the forward proxy then?
# To enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT 1
That’s somewhat confusing. remap_required disables “open forward proxying”. ATS actually doesn’t know / care about forward vs reverse proxy, it’s just a matter of what requests you allow through. What this setting is saying “Without an explicit rule matching in remap.config, deny the request”. There’s a similar one for reverse proxy.
— Leif
Ok, thanks for clearing that up. What that said, I kept the setting at “1” and changed the remap.config file to what’s listed below. Unfortunately I was still able to to connect to sites not listed in remap.config.

.defflt internal_only @action=allow @src_ip=10.0.0.0-255.255.255.255

.useflt internal_only
map https://www.facebook.com https://www.facebook.com
map https://www.yahoo.com https://www.yahoo.com
map http://finance.yahoo.com http://finance.yahoo.com


1420840183.867 126 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -

Not sure it matters, but I also have our networks IP’s listed in ip_allow.config.
Post by Sudheer Vinukonda
You will also need to enable the config proxy.config.url_remap.remap_required (like Leif suggested earlier).
.useflt internal_only
map https://www.facebook.com https://www.facebook.com
map https://www.yahoo.com https://www.yahoo.com
map http://finance.yahoo.com http://finance.yahoo.com
.unuseflt internal_only
.useflt disable_all
But going to a site not listed (www.oracle.com) is still allowed. ?
1420835169.093 134 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -
I’ve also tried placing ".useflt disable_all” before the “.useflt internal_only” filter with no luck, sites not on the list are still allowed out.
Post by Sudheer Vinukonda
I think you would need to use named_filters to specify ranges in remap.config.
remap.config — Apache Traffic Server 5.3.0 documentation
remap.config — Apache Traffic Server 5.3.0 documentation
View on docs.trafficserver.apache.org
Preview by Yahoo
Post by Paul Tader
Post by James Peach
Post by Paul Tader
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html
Does your version of ATS match the version of the docs?
Nope and I apologize for that. Time to upgrade.
Thanks everyone.
Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take. Can something like this be done? (ATS version 3.04)
...
Post by Paul Tader
Post by James Peach
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
Paul Tader
2015-01-12 17:23:49 UTC
Permalink
Post by Paul Tader
Doesn’t this break the forward proxy then?
# To enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT 1
That’s somewhat confusing. remap_required disables “open forward proxying”. ATS actually doesn’t know / care about forward vs reverse proxy, it’s just a matter of what requests you allow through. What this setting is saying “Without an explicit rule matching in remap.config, deny the request”. There’s a similar one for reverse proxy.
— Leif
Ok, thanks for clearing that up. What that said, I kept the setting at “1” and changed the remap.config file to what’s listed below. Unfortunately I was still able to to connect to sites not listed in remap.config.
.useflt internal_only
map https://www.facebook.com https://www.facebook.com
map https://www.yahoo.com https://www.yahoo.com
map http://finance.yahoo.com http://finance.yahoo.com
1420840183.867 126 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -
Not sure it matters, but I also have our networks IP’s listed in ip_allow.config.
Is there an equivilent to .deactivatefilter in ATS 3?

Paul
Post by Paul Tader
Post by Sudheer Vinukonda
You will also need to enable the config proxy.config.url_remap.remap_required (like Leif suggested earlier).
.useflt internal_only
map https://www.facebook.com https://www.facebook.com
map https://www.yahoo.com https://www.yahoo.com
map http://finance.yahoo.com http://finance.yahoo.com
.unuseflt internal_only
.useflt disable_all
But going to a site not listed (www.oracle.com) is still allowed. ?
1420835169.093 134 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -
I’ve also tried placing ".useflt disable_all” before the “.useflt internal_only” filter with no luck, sites not on the list are still allowed out.
Post by Sudheer Vinukonda
I think you would need to use named_filters to specify ranges in remap.config.
remap.config — Apache Traffic Server 5.3.0 documentation
remap.config — Apache Traffic Server 5.3.0 documentation
View on docs.trafficserver.apache.org
Preview by Yahoo
Post by Paul Tader
Post by James Peach
Post by Paul Tader
Hmm, I didn’t think about a DNS blackhole. For now I’m looking into additional remap files using the “.include” directive in remap.config but I get these errors after running traffic_line -x
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at line #126; Aborting!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown directive ".include" at line 126
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: something failed during BuildTable() -- check your remap plugins!
[Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to reload remap.config, not replacing!
.include /etc/trafficserver/filters.config
.include /etc/trafficserver/set1.remap.config
"The .include directive allows mapping rules to be spread across multiple files. The argument to the .include directive is a list of file names to be parsed for additional mapping rules. "
http://trafficserver.readthedocs.org/en/latest/reference/configuration/remap.config.en.html
Does your version of ATS match the version of the docs?
Nope and I apologize for that. Time to upgrade.
Thanks everyone.
Before I upgrade, I’ve tried a “deny all” map as the last line in remap.conf and listing all the allowed sites before this deny line, but it doesn’t take. Can something like this be done? (ATS version 3.04)
...
Post by Paul Tader
Post by James Peach
Post by Paul Tader
We have a forward only proxy server configured. How can I restrict a internal IP address or IP address range to only be able to proxy certain top level domains (ie google.com, yahoo.com, etc)? I’ve read a lot on remapping, but I don’t think that is the correct approach.
DNS blackholing as suggested seems like a reasonable solution. If your list of domains is smallish, then something in remap.config might work as well. I’ve done this in the past, blocking all but a few HTTPS sites (via setting remap.required to 1 in records.config). The other option is to allow all sites, but list the ones that you intend to block (map them to some nonexistent domain or IP, e.g. 10.0.0.0).
Fwiw, remap rules like this with CONNECT methods only works in 5.0.0 and later.
— Leif
James Peach
2015-01-12 17:42:10 UTC
Permalink
Post by Paul Tader
Post by Paul Tader
Doesn’t this break the forward proxy then?
# To enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT 1
That’s somewhat confusing. remap_required disables “open forward proxying”. ATS actually doesn’t know / care about forward vs reverse proxy, it’s just a matter of what requests you allow through. What this setting is saying “Without an explicit rule matching in remap.config, deny the request”. There’s a similar one for reverse proxy.
— Leif
Ok, thanks for clearing that up. What that said, I kept the setting at “1” and changed the remap.config file to what’s listed below. Unfortunately I was still able to to connect to sites not listed in remap.config.
.useflt internal_only
map https://www.facebook.com https://www.facebook.com
map https://www.yahoo.com https://www.yahoo.com
map http://finance.yahoo.com http://finance.yahoo.com
1420840183.867 126 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -
Not sure it matters, but I also have our networks IP’s listed in ip_allow.config.
Is there an equivilent to .deactivatefilter in ATS 3?
"unusefilter", "deactivatefilter", "unactivefilter", deuseflt", and "unuseflt" are all synonyms. I thought that they had all been there forever, but maybe some synonyms were not present in 3 ...

J
Paul Tader
2015-01-12 20:28:35 UTC
Permalink
Post by James Peach
Post by Paul Tader
Post by Leif Hedstrom
Post by Paul Tader
Doesn’t this break the forward proxy then?
# To enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT 1
That’s somewhat confusing. remap_required disables “open forward proxying”. ATS actually doesn’t know / care about forward vs reverse proxy, it’s just a matter of what requests you allow through. What this setting is saying “Without an explicit rule matching in remap.config, deny the request”. There’s a similar one for reverse proxy.
— Leif
Ok, thanks for clearing that up. What that said, I kept the setting at “1” and changed the remap.config file to what’s listed below. Unfortunately I was still able to to connect to sites not listed in remap.config.
.useflt internal_only
map https://www.facebook.com https://www.facebook.com
map https://www.yahoo.com https://www.yahoo.com
map http://finance.yahoo.com http://finance.yahoo.com
1420840183.867 126 10.1.2.3 TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -
Not sure it matters, but I also have our networks IP’s listed in ip_allow.config.
Is there an equivilent to .deactivatefilter in ATS 3?
"unusefilter", "deactivatefilter", "unactivefilter", deuseflt", and "unuseflt" are all synonyms. I thought that they had all been there forever, but maybe some synonyms were not present in 3 ...
J
I was hoping, but I don’t see those directives in the documentation and when adding that text I’m met with an error:
[Jan 12 18:23:54.607] Server {47752783210240} WARNING: Could not add rule at line #151; Aborting!

I was hoping to replicate what is in later ATS versions, for example:

[remap.config]
.defflt disable_all @action=deny
.defflt internal_only @action=allow

.useflt disable_all
.useflt internal_only
map http://url.com <http://url.com/> http://url.com <http://url.com/>
map http://url2.com <http://url2.com/> http://url2.com <http://url2.com/>
map http://url3.com <http://url3.com/> http://url3.com <http://url3.com/>
.disableflt internal_only

(all other sites should be blocked).
[remap.config end]


kind of replicating what iptables does, fall through until you match a rule.
Loading...