Attempting to Reverse Proxy HTTP to HTTPS via a parent proxy

[Jim Groffen]

Hello,

I have written a simple proxy in Java to forward requests through the
corporate proxy and access an external site using HTTPS. There is no
internet access unless we go via this corporate proxy.

I'm now trying to replace my simple proxy with TrafficServer. My setup is
as follows:

User -> TrafficServer -> Corporate Proxy -> External Site

I have configured two remap rules (using the NPMJS Registry for testing):

regex_map http://(.*):8080/npm http://registry.npmjs.org:80/
<http://registry.npmjs.org/>
regex_map http://(.*):8080/snpm https://registry.npmjs.org:443/
<https://registry.npmjs.org/>

I also have configured the corporate proxy as a parent proxy:

dest_domain=. parent=[corporate proxy ip]:[port]

With this configuration I can hit the /npm path and traffic comes through
correctly, but when I try the /snpm path I get a "Could Not Connect" error.

Monitoring the traffic shows that some tcp packets are sent between the
TrafficServer and the corporate proxy, then the corporate proxy returns an
HTTP 400 response.

Monitoring the Java based simple proxy for comparison, I see the TCP
traffic, then a valid HTTP 200 response.

I've tried a few things around the SSL settings like relaxing the allowed
ciphers, but it's hard to know what's wrong because the TCP traffic is
likely the encrypted SSL traffic! One interesting point though, I can see
registry.npmjs.org when using TrafficServer - I assume this is due to SNI
(Server Name Indication) - but my Java based proxy doesn't do this. This
made me wonder if SNI isn't supported on my corporate proxy.

Another possible wrinkle is I need to auth to the Corporate Proxy - I've
enabled proxy.config.http.forward.proxy_auth_to_parent and am using the
headers plugin to provide valid login details - which is all working for
non-https traffic.

Thank you in advance, any help would be greatly appreciated!

Cheers,

Jim

[Jeremy Payne]

can your corporate proxy terminate requests for https://registry.npmjs.org
?
usually corp proxies do not unless they support something like squid's ssl
bump/peak/splice...
if not, then ATS will have to send a CONNECT request to your corp proxy(not
sure if ATS supports issuing CONNECT to upstream servers)..

try the below curl commands to test your corp proxy.

TLS termination..

curl -k -o /dev/null -v --resolve registry.npmjs.org:443:<CORP PROXY IP>
https://registry.npmjs.org

CONNECT

curl -o /dev/null -v -x <CORP PROXY IP>:<PROXY PORT>
https://registry.npmjs.org/


does your existing java based proxy support requests for
https://registry.npmjs.org ?

btw.. you can still inspect the TLS handshake within a pcap. you should
also be able to
view the TLS alert issued by either side within the same pcap.
you can also try placing ATS into debug mode to see whats going on.

Post by Jim Groffen
Hello,
I have written a simple proxy in Java to forward requests through the
corporate proxy and access an external site using HTTPS. There is no
internet access unless we go via this corporate proxy.
I'm now trying to replace my simple proxy with TrafficServer. My setup is
User -> TrafficServer -> Corporate Proxy -> External Site
regex_map http://(.*):8080/npm http://registry.npmjs.org:80/
<http://registry.npmjs.org/>
regex_map http://(.*):8080/snpm https://registry.npmjs.org:443/
<https://registry.npmjs.org/>
dest_domain=. parent=[corporate proxy ip]:[port]
With this configuration I can hit the /npm path and traffic comes through
correctly, but when I try the /snpm path I get a "Could Not Connect" error.
Monitoring the traffic shows that some tcp packets are sent between the
TrafficServer and the corporate proxy, then the corporate proxy returns an
HTTP 400 response.
Monitoring the Java based simple proxy for comparison, I see the TCP
traffic, then a valid HTTP 200 response.
I've tried a few things around the SSL settings like relaxing the allowed
ciphers, but it's hard to know what's wrong because the TCP traffic is
likely the encrypted SSL traffic! One interesting point though, I can see
registry.npmjs.org when using TrafficServer - I assume this is due to SNI
(Server Name Indication) - but my Java based proxy doesn't do this. This
made me wonder if SNI isn't supported on my corporate proxy.
Another possible wrinkle is I need to auth to the Corporate Proxy - I've
enabled proxy.config.http.forward.proxy_auth_to_parent and am using the
headers plugin to provide valid login details - which is all working for
non-https traffic.
Thank you in advance, any help would be greatly appreciated!
Cheers,
Jim

[Jim Groffen]

Hello again,

I have made a few steps forward and a few back.

First I decided to move from ATS 5.3 (latest rpm available on RHEL by
default) to 7.0.0 (latest RHEL rpm build I could find). I did this becuase
I found a few settings that led me to beleive this version was more CONNECT
method aware (proxy.config.http.forward_connect_method,
proxy.local.http.parent_proxy.disable_connect_tunneling, etc).

I hit a snag here though. HTTP works fine but HTTPS reports an Unknown Host
error due to DNS lookup failure of the https host. The issue is I expect
the DNS lookup that ATS performs to fail and have set
'no_dns_just_forward_to_parent' to 1.

I have enabled debugging and it looks like the requests are both mapping
correctly. Looking at the logs and GitHub source - I can see where the
request processing diverges:

HTTP Request:

<HttpTransact.cc:1812 (HandleRequestAuthorized)> (http_trans) Next
action SM_ACTION_API_OS_DNS; HttpTransact::DecideCacheLookup

HTTPS Request:

<HttpTransact.cc:1350 (HandleRequest)> (http_trans) Next action
SM_ACTION_DNS_LOOKUP; HttpTransact::OSDNLookup

... looking in GitHub I can see the following line decides which code path
is taken:


https://github.com/apache/trafficserver/blob/7.0.x/proxy/http/HttpTransact.cc#L1331

This made me think maybe how I'm defining parent hosts wasn't causing the
parent proxy to match to https requests. Playing with the parent hosts
settings didn't seem to change anything - I have tried the following:

dest_domain=. parent=<corp proxy host>:<port>
dest_domain=. parent=<corp proxy host>:<port> go_direct=false
dest_domain=. scheme=https parent=<corp proxy host>:<port>
dest_host=registry.npmjs.org scheme=https parent=<corp proxy
host>:<port>

If I can overcome the problem above then I can try another PCAP to see what
the parent proxy is sent.

Finally, I'm not sure ATS supports initiating a HTTP CONNECT tunnel to a
parent proxy, only that ATS can be the proxy that does the tunneling.

Any suggestions you might have will be very helpful! I'm a bit stuck again
now.

Thanks again,

Thank you for this response Jeremy, I've only now had a chance to try your
suggestions.
As you suspect the corporate proxy does not support TLS Termination but
does support HTTP CONNECT Tunnelling.
My existing Java based proxy can connect to https://registry.npmjs.org and
is using HTTP CONNECT tunelling. I can see the TLS handshakes etc in the
PCAP now.
I haven't worked out what ATS is attempting to do yet - I'm having
unrelated issues getting to my test machines. Once I've got a handle on
that I'll send another update.
Thanks again for such a fast response, and apologies for my tardy reply!

Post by Jeremy Payne
can your corporate proxy terminate requests for
https://registry.npmjs.org ?
usually corp proxies do not unless they support something like squid's
ssl bump/peak/splice...
if not, then ATS will have to send a CONNECT request to your corp
proxy(not sure if ATS supports issuing CONNECT to upstream servers)..
try the below curl commands to test your corp proxy.
TLS termination..
curl -k -o /dev/null -v --resolve registry.npmjs.org:443:<CORP PROXY IP>
https://registry.npmjs.org
CONNECT
curl -o /dev/null -v -x <CORP PROXY IP>:<PROXY PORT>
https://registry.npmjs.org/
does your existing java based proxy support requests for
https://registry.npmjs.org ?
btw.. you can still inspect the TLS handshake within a pcap. you should
also be able to
view the TLS alert issued by either side within the same pcap.
you can also try placing ATS into debug mode to see whats going on.

Post by Jim Groffen
Hello,
I have written a simple proxy in Java to forward requests through the
corporate proxy and access an external site using HTTPS. There is no
internet access unless we go via this corporate proxy.
I'm now trying to replace my simple proxy with TrafficServer. My setup
User -> TrafficServer -> Corporate Proxy -> External Site
regex_map http://(.*):8080/npm http://registry.npmjs.org:80/
<http://registry.npmjs.org/>
regex_map http://(.*):8080/snpm https://registry.npmjs.org:443/
<https://registry.npmjs.org/>
dest_domain=. parent=[corporate proxy ip]:[port]
With this configuration I can hit the /npm path and traffic comes
through correctly, but when I try the /snpm path I get a "Could Not
Connect" error.
Monitoring the traffic shows that some tcp packets are sent between the
TrafficServer and the corporate proxy, then the corporate proxy returns an
HTTP 400 response.
Monitoring the Java based simple proxy for comparison, I see the TCP
traffic, then a valid HTTP 200 response.
I've tried a few things around the SSL settings like relaxing the
allowed ciphers, but it's hard to know what's wrong because the TCP traffic
is likely the encrypted SSL traffic! One interesting point though, I can
see registry.npmjs.org when using TrafficServer - I assume this is due
to SNI (Server Name Indication) - but my Java based proxy doesn't do this.
This made me wonder if SNI isn't supported on my corporate proxy.
Another possible wrinkle is I need to auth to the Corporate Proxy - I've
enabled proxy.config.http.forward.proxy_auth_to_parent and am using the
headers plugin to provide valid login details - which is all working for
non-https traffic.
Thank you in advance, any help would be greatly appreciated!
Cheers,
Jim