Discussion:
Openssl 1.1.0f Support
Jeremy Payne
2017-09-19 20:20:31 UTC
Permalink
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Leif Hedstrom
2017-09-19 22:55:47 UTC
Permalink
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on docs.trafficserver.apache.org <http://docs.trafficserver.apache.org/>. Maybe you have some mismatch / issues between headers (when compiling ATS) and runtime?

Cheers,

— Leif
Alan Carroll
2017-09-19 23:31:05 UTC
Permalink
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues
between headers (when compiling ATS) and runtime?
Cheers,
— Leif
Jeremy Payne
2017-09-20 14:44:28 UTC
Permalink
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.

Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.

Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Alan Carroll
2017-09-20 15:08:20 UTC
Permalink
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides (
https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.

* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues
between
Post by Alan Carroll
Post by Jeremy Payne
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Dave Thompson
2017-09-20 16:26:34 UTC
Permalink
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in
their current forms.

Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides (https://cwiki.apache.org/confluence/display/TS/
Presentations+-+2016). I'd start there and see if you can bug Susan or
Good Dave*. Although that work was designed to use an off box crypto
engine, the implementation from the ATS point of view is identical to what
you're writing about. Susan will be at the Fall 2017 Summit, I'd look her
up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues
between
Post by Alan Carroll
Post by Jeremy Payne
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Jeremy Payne
2017-09-20 20:22:34 UTC
Permalink
Dave,

Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)

1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Dave Thompson
2017-09-20 21:16:37 UTC
Permalink
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by
now at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework.
This was all static code analysis. Integration looked like a non-starter,
so no performance test done.

Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured
order of magnitude improvement over with the same using Openssl 1.0.1(x)
and small packet sizes... Improvement attributed to lock contention issues
in the older OpenSSL 1.0.1(x).

Dave
Post by Jeremy Payne
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long
jumping
Post by Dave Thompson
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in
their
Post by Dave Thompson
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that,
"crypto
Post by Dave Thompson
Post by Alan Carroll
proxy". There's a small mention of it by Susan at the Fall 2016 summit
in
Post by Dave Thompson
Post by Alan Carroll
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016).
I'd
Post by Dave Thompson
Post by Alan Carroll
start there and see if you can bug Susan or Good Dave*. Although that
work
Post by Dave Thompson
Post by Alan Carroll
was designed to use an off box crypto engine, the implementation from
the
Post by Dave Thompson
Post by Alan Carroll
ATS point of view is identical to what you're writing about. Susan will
be
Post by Dave Thompson
Post by Alan Carroll
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
On Tue, Sep 19, 2017 at 6:31 PM, Alan Carroll <
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully
been
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what
about
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch /
issues
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Bryan Call
2017-09-20 22:54:33 UTC
Permalink
I was see something like 2x the performance in my benchmarks with OpenSSL 1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since May, when I upgraded to Fedora 26.

-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best. The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively. I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework. This was all static code analysis. Integration looked like a non-starter, so no performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes... Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016 <https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016>). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org <http://docs.trafficserver.apache.org/>. Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Bryan Call
2017-09-21 00:38:40 UTC
Permalink
I meant to say 1.1.0.

-Bryan
Post by Bryan Call
I was see something like 2x the performance in my benchmarks with OpenSSL 1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since May, when I upgraded to Fedora 26.
-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best. The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively. I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework. This was all static code analysis. Integration looked like a non-starter, so no performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes... Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016 <https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016>). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org <http://docs.trafficserver.apache.org/>. Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Kees Spoelstra
2017-09-21 07:49:04 UTC
Permalink
Hi Dave and Jeremy,



We were also looking into using Intel QAT, the AES was not of interest to us , mostly improving the RSA handshake phase.



Not having looked at the API, I wonder if we would able to offload the handshake part to threads which handle the openssl-async stuff, and after the handshake go back to normal processing. Performance of SSL handshakes is bound by raw CPU and pretty low in rq/s, so the overhead in thread sync could be negligible. Any thoughts about that?



We’re pretty busy here, but I’m going to check here if we can burn some cycles on looking into this. Any other insights from the tests at yahoo are welcome.



Kees



From: Dave Thompson [mailto:***@oath.com]
Sent: Wednesday, September 20, 2017 23:17
To: ***@trafficserver.apache.org
Subject: Re: Openssl 1.1.0f Support



Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best. The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively. I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework. This was all static code analysis. Integration looked like a non-starter, so no performance test done.

Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes... Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).


Dave



On Wed, Sep 20, 2017 at 3:22 PM, Jeremy Payne <***@gmail.com <mailto:***@gmail.com> > wrote:

Dave,

Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)

1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org <http://docs.trafficserver.apache.org> . Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Alan Carroll
2017-09-21 13:12:58 UTC
Permalink
Kees - I think Dave and/or Susan tried the thread off loading approach. I
think it is mentioned in the presentation cited above. IIRC that ended up
not working well for some reason.
Post by Kees Spoelstra
Hi Dave and Jeremy,
We were also looking into using Intel QAT, the AES was not of interest to
us , mostly improving the RSA handshake phase.
Not having looked at the API, I wonder if we would able to offload the
handshake part to threads which handle the openssl-async stuff, and after
the handshake go back to normal processing. Performance of SSL handshakes
is bound by raw CPU and pretty low in rq/s, so the overhead in thread sync
could be negligible. Any thoughts about that?
We’re pretty busy here, but I’m going to check here if we can burn some
cycles on looking into this. Any other insights from the tests at yahoo are
welcome.
Kees
*Sent:* Wednesday, September 20, 2017 23:17
*Subject:* Re: Openssl 1.1.0f Support
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by
now at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework.
This was all static code analysis. Integration looked like a non-starter,
so no performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured
order of magnitude improvement over with the same using Openssl 1.0.1(x)
and small packet sizes... Improvement attributed to lock contention issues
in the older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long
jumping
Post by Dave Thompson
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in
their
Post by Dave Thompson
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that,
"crypto
Post by Dave Thompson
Post by Alan Carroll
proxy". There's a small mention of it by Susan at the Fall 2016 summit
in
Post by Dave Thompson
Post by Alan Carroll
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016).
I'd
Post by Dave Thompson
Post by Alan Carroll
start there and see if you can bug Susan or Good Dave*. Although that
work
Post by Dave Thompson
Post by Alan Carroll
was designed to use an off box crypto engine, the implementation from
the
Post by Dave Thompson
Post by Alan Carroll
ATS point of view is identical to what you're writing about. Susan will
be
Post by Dave Thompson
Post by Alan Carroll
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
On Tue, Sep 19, 2017 at 6:31 PM, Alan Carroll <
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully
been
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what
about
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch /
issues
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Kees Spoelstra
2017-09-21 14:32:33 UTC
Permalink
Ok,



Looking at the presentation it seems that the openssl async jobs approach was not tested, but planned. Dave mentioned static code analysis and no performance tests.



Within openssl+QAT you get the performance from running the openssl calls in an async job which becomes a sort of coroutine, which is then run in openssl’s threadpool. When offloading, it yields control to the scheduler, freeing the CPU for other jobs. Ofcourse it is a bit harder than this :)

Building the jobs could be easy, the whole song and dance around synchronization, thread alignment, could be too cumbersome to bother with.



Next to that it seems that the async jobs are a bit slower when only using the CPU, so you would end up with two separate flows to ensure the same performance for 99.99% of the users.



Anybody willing to donate an Intel QAT PCI card :) Our address is 
.



Note: It seems that the intel QAT cards have an unexpected scaling problem with ECDHE-RSA, which they (intel) were looking into, and with the >32 thread CPUs you will get a break even situation. So for HTTP/2 the benefit would be much smaller than normal RSA if the machine has enough cores. Ofcourse the Intel QAT card should still offload the CPU.



Kees









From: Alan Carroll [mailto:***@oath.com]
Sent: Thursday, September 21, 2017 15:13
To: ***@trafficserver.apache.org
Subject: Re: Openssl 1.1.0f Support



Kees - I think Dave and/or Susan tried the thread off loading approach. I think it is mentioned in the presentation cited above. IIRC that ended up not working well for some reason.



On Thu, Sep 21, 2017 at 2:49 AM, Kees Spoelstra <***@we-amp.com <mailto:***@we-amp.com> > wrote:

Hi Dave and Jeremy,



We were also looking into using Intel QAT, the AES was not of interest to us , mostly improving the RSA handshake phase.



Not having looked at the API, I wonder if we would able to offload the handshake part to threads which handle the openssl-async stuff, and after the handshake go back to normal processing. Performance of SSL handshakes is bound by raw CPU and pretty low in rq/s, so the overhead in thread sync could be negligible. Any thoughts about that?



We’re pretty busy here, but I’m going to check here if we can burn some cycles on looking into this. Any other insights from the tests at yahoo are welcome.



Kees



From: Dave Thompson [mailto:***@oath.com <mailto:***@oath.com> ]
Sent: Wednesday, September 20, 2017 23:17
To: ***@trafficserver.apache.org <mailto:***@trafficserver.apache.org>
Subject: Re: Openssl 1.1.0f Support



Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best. The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively. I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework. This was all static code analysis. Integration looked like a non-starter, so no performance test done.

Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes... Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).


Dave



On Wed, Sep 20, 2017 at 3:22 PM, Jeremy Payne <***@gmail.com <mailto:***@gmail.com> > wrote:

Dave,

Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)

1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org <http://docs.trafficserver.apache.org> . Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Dave Thompson
2017-09-21 15:02:42 UTC
Permalink
I think Alan was thinking of an effort we called CryptoProxy. Right, QAT
was static code analysis considered but ultimately put aside.
Cryptoproxy focus was on keeping the private keys off the ATS box. It was
done for security, not for CPU savings as we're not typically CPU bound,
despite ~80% of traffic over TLS. CryptoProxy project sent RSA
encrypt/decrypt requests from handshake to a separate box. FWIW, I recall
our old out dated test Xeon X5650 boxes taking like 3msec RSA decrypt or 4
msec RSA encrypt (key exchange dependent), and that of course is only for
non-resumption/new TLS connections.

Dave
Post by Kees Spoelstra
Ok,
Looking at the presentation it seems that the openssl async jobs approach
was not tested, but planned. Dave mentioned static code analysis and no
performance tests.
Within openssl+QAT you get the performance from running the openssl calls
in an async job which becomes a sort of coroutine, which is then run in
openssl’s threadpool. When offloading, it yields control to the scheduler,
freeing the CPU for other jobs. Ofcourse it is a bit harder than this :)
Building the jobs could be easy, the whole song and dance around
synchronization, thread alignment, could be too cumbersome to bother with.
Next to that it seems that the async jobs are a bit slower when only using
the CPU, so you would end up with two separate flows to ensure the same
performance for 99.99% of the users.
Anybody willing to donate an Intel QAT PCI card :) Our address is 
.
Note: It seems that the intel QAT cards have an unexpected scaling problem
with ECDHE-RSA, which they (intel) were looking into, and with the >32
thread CPUs you will get a break even situation. So for HTTP/2 the benefit
would be much smaller than normal RSA if the machine has enough cores.
Ofcourse the Intel QAT card should still offload the CPU.
Kees
*Sent:* Thursday, September 21, 2017 15:13
*Subject:* Re: Openssl 1.1.0f Support
Kees - I think Dave and/or Susan tried the thread off loading approach. I
think it is mentioned in the presentation cited above. IIRC that ended up
not working well for some reason.
Hi Dave and Jeremy,
We were also looking into using Intel QAT, the AES was not of interest to
us , mostly improving the RSA handshake phase.
Not having looked at the API, I wonder if we would able to offload the
handshake part to threads which handle the openssl-async stuff, and after
the handshake go back to normal processing. Performance of SSL handshakes
is bound by raw CPU and pretty low in rq/s, so the overhead in thread sync
could be negligible. Any thoughts about that?
We’re pretty busy here, but I’m going to check here if we can burn some
cycles on looking into this. Any other insights from the tests at yahoo are
welcome.
Kees
*Sent:* Wednesday, September 20, 2017 23:17
*Subject:* Re: Openssl 1.1.0f Support
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by
now at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework.
This was all static code analysis. Integration looked like a non-starter,
so no performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured
order of magnitude improvement over with the same using Openssl 1.0.1(x)
and small packet sizes... Improvement attributed to lock contention issues
in the older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long
jumping
Post by Dave Thompson
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in
their
Post by Dave Thompson
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that,
"crypto
Post by Dave Thompson
Post by Alan Carroll
proxy". There's a small mention of it by Susan at the Fall 2016 summit
in
Post by Dave Thompson
Post by Alan Carroll
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016).
I'd
Post by Dave Thompson
Post by Alan Carroll
start there and see if you can bug Susan or Good Dave*. Although that
work
Post by Dave Thompson
Post by Alan Carroll
was designed to use an off box crypto engine, the implementation from
the
Post by Dave Thompson
Post by Alan Carroll
ATS point of view is identical to what you're writing about. Susan will
be
Post by Dave Thompson
Post by Alan Carroll
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
On Tue, Sep 19, 2017 at 6:31 PM, Alan Carroll <
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1 vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully
been
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what
about
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch /
issues
Post by Dave Thompson
Post by Alan Carroll
Post by Jeremy Payne
Post by Alan Carroll
Post by Jeremy Payne
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Jeremy Payne
2017-09-22 14:02:36 UTC
Permalink
issue was this..

i was sending a request to the listening IP address without sending
the right SNI value.
i didnt have a 'default' certificate defined so ATS 'rejected' the
request. hence giving the impression
no TLS session was established.
i then defined a default certificate and was able to send a request to
the listening IP.
so pebcak error..
problem exists between chair and keyboard..
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
i***@sina.cn
2017-09-21 06:52:13 UTC
Permalink
The following traffic server patch can improve openssl 1.0.1 performance as openssl 1.1.0:


diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}

@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}

ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}

ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}

From: Bryan Call <***@apache.org>
Reply-To: "***@trafficserver.apache.org" <***@trafficserver.apache.org>

Date: Thursday, September 21, 2017 at 8:38 AM

To: "***@trafficserver.apache.org" <***@trafficserver.apache.org>

Subject: Re: Openssl 1.1.0f Support

I meant to say 1.1.0.

-Bryan

On Sep 20, 2017, at 3:54 PM, Bryan Call <***@apache.org> wrote:

I was see something like 2x the performance in my benchmarks with OpenSSL 1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since May, when I upgraded to Fedora 26.

-Bryan

On Sep 20, 2017, at 2:16 PM, Dave Thompson <***@oath.com> wrote:

Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best. The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively. I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework. This was all static code analysis. Integration looked like a non-starter, so no performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes... Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).

Dave

On Wed, Sep 20, 2017 at 3:22 PM, Jeremy Payne <***@gmail.com> wrote:
Dave,


Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)


1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
haha
2017-09-21 07:06:46 UTC
Permalink
Can you push your patch against master on github ?


scw00
------------------ Ô­ÊŒÓÊŒþ ------------------
·¢ŒþÈË: "iloveperl";<***@sina.cn>;
·¢ËÍʱŒä: 2017Äê9ÔÂ21ÈÕ(ÐÇÆÚËÄ) ÏÂÎç2:52
ÊÕŒþÈË: "users"<***@trafficserver.apache.org>;"bcall"<***@apache.org>;

Ö÷Ìâ: Re: Openssl 1.1.0f Support





The following traffic server patch can improve openssl 1.0.1 performance as openssl 1.1.0:


diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}

@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}

ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}

ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}





From: Bryan Call <***@apache.org>
Reply-To: "***@trafficserver.apache.org" <***@trafficserver.apache.org>
Date: Thursday, September 21, 2017 at 8:38 AM
To: "***@trafficserver.apache.org" <***@trafficserver.apache.org>
Subject: Re: Openssl 1.1.0f Support




I meant to say 1.1.0.



-Bryan



On Sep 20, 2017, at 3:54 PM, Bryan Call <***@apache.org> wrote:



I was see something like 2x the performance in my benchmarks with OpenSSL 1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since May, when I upgraded to Fedora 26.



-Bryan



On Sep 20, 2017, at 2:16 PM, Dave Thompson <***@oath.com> wrote:



Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best. The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively. I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework. This was all static code analysis. Integration looked like a non-starter, so no performance test done.

Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes... Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).



Dave



On Wed, Sep 20, 2017 at 3:22 PM, Jeremy Payne <***@gmail.com> wrote:

Dave,




Did you run any comparison performance tests using the QAT engine ?

Specifically around these configurations(or similar)




1. ATS + Openssl 1.1.0(x) + QAT engine(sync)

2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we¡¯re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
¡ª Leif
Bryan Call
2017-09-21 15:37:21 UTC
Permalink
This only changes the order of the calls. There is still going to be lock contention inside OpenSSL 1.0.1.

-Bryan
Post by i***@sina.cn
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}
@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
Date: Thursday, September 21, 2017 at 8:38 AM
Subject: Re: Openssl 1.1.0f Support
I meant to say 1.1.0.
-Bryan
I was see something like 2x the performance in my benchmarks with OpenSSL 1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since May, when I upgraded to Fedora 26.
-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best. The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively. I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework. This was all static code analysis. Integration looked like a non-starter, so no performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes... Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
Post by Dave Thompson
July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in their
current forms.
Dave
Post by Alan Carroll
Susan and Dave Thompson were working on something related to that, "crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016 <https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016>). I'd
start there and see if you can bug Susan or Good Dave*. Although that work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.
Post by Jeremy Payne
Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!
Post by Alan Carroll
Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.
Post by Jeremy Payne
I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org <http://docs.trafficserver.apache.org/>. Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif
Continue reading on narkive:
Loading...