回复：回复：Re: Openssl 1.1.0f Support

i***@sina.cn

2017-09-22 03:05:55 UTC

I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 + openssl 1.0.1 + patch respectively, and they almost have the same performance.

----- åå§é®ä»¶ -----
åä»¶äººïŒ<***@sina.cn>
æ¶ä»¶äººïŒ"bcall" <***@apache.org>
æéäººïŒ"users" <***@trafficserver.apache.org>
äž»é¢ïŒåå€ïŒRe: Openssl 1.1.0f Support
æ¥æïŒ2017å¹Ž09æ22æ¥ 10ç¹55å

With the patch, the ERR_clear_error() will only be called when the error occurs. In the normal situation, ERR_clear_error() will not be called, so the Err_get_state() will not be called and no lock contention in openssl 1.0.1 with the patch.

----- åå§é®ä»¶ -----åä»¶äººïŒBryan Call <***@apache.org>
æ¶ä»¶äººïŒ***@sina.cn
æéäººïŒusers <***@trafficserver.apache.org>
äž»é¢ïŒRe: Openssl 1.1.0f Support
æ¥æïŒ2017å¹Ž09æ21æ¥ 23ç¹37å

This only changes the order of the calls. There is still going to be lock contention inside OpenSSL 1.0.1.-BryanOn Sep 20, 2017, at 11:37 PM, ***@sina.cn wrote:The following traffic server patch can improve openssl 1.0.1 performance as openssl 1.1.0:

diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}

@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}

ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
}

ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}

+ ERR_clear_error();
+
return ssl_error;
} From: Bryan Call <***@apache.org>
Reply-To: "***@trafficserver.apache.org" <***@trafficserver.apache.org>
Date: Thursday, September 21, 2017 at 8:38 AM

docs.trafficserver.apache.org. Maybe you have some mismatch / issues>>> >> between>>> >> headers (when compiling ATS) and runtime?>>> >>>>> >> Cheers,>>> >>>>> >> â Leif

Jeremy Payne

2017-09-22 13:44:09 UTC

Permalink

Did you have to make any changes to 6.2 for it to compile cleanly
against openssl 1.1 ?
I can only get ATS 7+ to compile without producing any errors.

Post by i***@sina.cn
I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 + openssl
1.0.1 + patch respectively, and they almost have the same performance.
----- 原始邮件 -----
主题：回复：Re: Openssl 1.1.0f Support
日期：2017年09月22日 10点55分
With the patch, the ERR_clear_error() will only be called when the error
occurs. In the normal situation, ERR_clear_error() will not be called, so
the Err_get_state() will not be called and no lock contention in openssl
1.0.1 with the patch.
----- 原始邮件 -----
主题：Re: Openssl 1.1.0f Support
日期：2017年09月21日 23点37分
This only changes the order of the calls. There is still going to be lock
contention inside OpenSSL 1.0.1.
-Bryan
The following traffic server patch can improve openssl 1.0.1 performance as
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}
@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
Date: Thursday, September 21, 2017 at 8:38 AM
Subject: Re: Openssl 1.1.0f Support
I meant to say 1.1.0.
-Bryan
I was see something like 2x the performance in my benchmarks with OpenSSL
1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since
May, when I upgraded to Fedora 26.
-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now
at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework. This
was all static code analysis. Integration looked like a non-starter, so no
performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured order
of magnitude improvement over with the same using Openssl 1.0.1(x) and small
packet sizes... Improvement attributed to lock contention issues in the
older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration

July 2016, I was evaluating the async Quick Assist in the context of ATS,
and came away with the opinion it's value comes into play with a much
simpler application. It's effectively it's own async engine, long
jumping
across the stack, and doesn't play well or add value to ATS's more
extensive model to do similar.... not to mention mutually exclusive in
their
current forms.
Dave

Susan and Dave Thompson were working on something related to that,
"crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016).
I'd
start there and see if you can bug Susan or Good Dave*. Although that
work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will
be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.

Thanks guys.. Thats all I needed to know.. Now I can look closer at my
end. Will let you know what I find.
Also, any plans on supporting openssl async, which then allows for
taking full advantage of the Intel QAT engine?
Understood patches/commits are welcome, but just figured there may be
some behind the scene works already started.
Thanks!

Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.

I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some
reason I can't establish a SSL/TLS connection. Has anyone
successfully linked ATS against openssl 1.1.0f and successfully been
able to establish a SSL/TLS session?
In other words, is openssl 1.1.0f supported by ATS? If not, what
about
an earlier version of 1.1.0(x)??
Yeh, we’re running current master with OpenSSL v1.1.0f on
docs.trafficserver.apache.org. Maybe you have some mismatch / issues
between
headers (when compiling ATS) and runtime?
Cheers,
— Leif

Jack Bates

2017-09-22 16:28:45 UTC

Permalink

I remember backporting some fixes for building with OpenSSL 1.1 to 6.2
[1] ... Which errors are you still getting?

I tried just now and successfully built the 6.2 branch with OpenSSL
1.1.0f -- I did get warnings but no errors:

$ git clone -b 6.2.x https://github.com/apache/trafficserver.git
$ cd trafficserver
$ autoreconf -i
$ ./configure
$ make

[1] https://github.com/apache/trafficserver/pull/1321

Post by Jeremy Payne
Did you have to make any changes to 6.2 for it to compile cleanly
against openssl 1.1 ?
I can only get ATS 7+ to compile without producing any errors.

Post by i***@sina.cn
I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 + openssl
1.0.1 + patch respectively, and they almost have the same performance.
----- 原始邮件 -----
主题：回复：Re: Openssl 1.1.0f Support
日期：2017年09月22日 10点55分
With the patch, the ERR_clear_error() will only be called when the error
occurs. In the normal situation, ERR_clear_error() will not be called, so
the Err_get_state() will not be called and no lock contention in openssl
1.0.1 with the patch.
----- 原始邮件 -----
主题：Re: Openssl 1.1.0f Support
日期：2017年09月21日 23点37分
This only changes the order of the calls. There is still going to be lock
contention inside OpenSSL 1.0.1.
-Bryan
The following traffic server patch can improve openssl 1.0.1 performance as
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}
@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
Date: Thursday, September 21, 2017 at 8:38 AM
Subject: Re: Openssl 1.1.0f Support
I meant to say 1.1.0.
-Bryan
I was see something like 2x the performance in my benchmarks with OpenSSL
1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since
May, when I upgraded to Fedora 26.
-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now
at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework. This
was all static code analysis. Integration looked like a non-starter, so no
performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured order
of magnitude improvement over with the same using Openssl 1.0.1(x) and small
packet sizes... Improvement attributed to lock contention issues in the
older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration

Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.

Jeremy Payne

2017-09-22 17:51:56 UTC

Permalink

openssl lib and include directories werent matching versions.
fixed that, and was able to compile without issue.

I remember backporting some fixes for building with OpenSSL 1.1 to 6.2 [1]
... Which errors are you still getting?
I tried just now and successfully built the 6.2 branch with OpenSSL 1.1.0f
$ git clone -b 6.2.x https://github.com/apache/trafficserver.git
$ cd trafficserver
$ autoreconf -i
$ ./configure
$ make
[1] https://github.com/apache/trafficserver/pull/1321

Post by Jeremy Payne
Did you have to make any changes to 6.2 for it to compile cleanly
against openssl 1.1 ?
I can only get ATS 7+ to compile without producing any errors.

Post by i***@sina.cn
I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 + openssl
1.0.1 + patch respectively, and they almost have the same performance.
----- 原始邮件 -----
主题：回复：Re: Openssl 1.1.0f Support
日期：2017年09月22日 10点55分
With the patch, the ERR_clear_error() will only be called when the error
occurs. In the normal situation, ERR_clear_error() will not be called, so
the Err_get_state() will not be called and no lock contention in openssl
1.0.1 with the patch.
----- 原始邮件 -----
主题：Re: Openssl 1.1.0f Support
日期：2017年09月21日 23点37分
This only changes the order of the calls. There is still going to be lock
contention inside OpenSSL 1.0.1.
-Bryan
The following traffic server patch can improve openssl 1.0.1 performance as
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}
@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
Date: Thursday, September 21, 2017 at 8:38 AM
Subject: Re: Openssl 1.1.0f Support
I meant to say 1.1.0.
-Bryan
I was see something like 2x the performance in my benchmarks with OpenSSL
1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since
May, when I upgraded to Fedora 26.
-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now
at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework.
This
was all static code analysis. Integration looked like a non-starter, so no
performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured order
of magnitude improvement over with the same using Openssl 1.0.1(x) and small
packet sizes... Improvement attributed to lock contention issues in the
older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration

Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.

Chou, Peter

2017-09-25 23:56:20 UTC

Permalink

Jack,

Regarding compiling 6.2.x with OpenSSL 1.1.0f, I submitted PR #2570 which back-ports your fix for the sslheaders plugin that was required due to X509 being made opaque in OpenSSL 1.1. There was also a linking issue with Ubuntu when compiling with the 'with-openssl' configure option. Apparently, this might not occur on Red Hat which is what Jeremy used below.

Thanks,
Peter

-----Original Message-----
From: Jeremy Payne [mailto:***@gmail.com]
Sent: Friday, September 22, 2017 10:52 AM
To: ***@trafficserver.apache.org
Subject: Re: 回复：Re: Openssl 1.1.0f Support

openssl lib and include directories werent matching versions.
fixed that, and was able to compile without issue.

I remember backporting some fixes for building with OpenSSL 1.1 to 6.2 [1]
... Which errors are you still getting?
I tried just now and successfully built the 6.2 branch with OpenSSL 1.1.0f
$ git clone -b 6.2.x https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver.git&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=8HrQiBnIzItF-b6piI1aWqjSdcpsW9r59uuAoKA_sIQ&e=
$ cd trafficserver
$ autoreconf -i
$ ./configure
$ make
[1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver_pull_1321&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=nr0xsT0XCw7uLhrHvYwBhmQd5BbnSAKC2RfefCTpHgI&e=

Post by Jeremy Payne
Did you have to make any changes to 6.2 for it to compile cleanly
against openssl 1.1 ?
I can only get ATS 7+ to compile without producing any errors.

Post by i***@sina.cn
I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 + openssl
1.0.1 + patch respectively, and they almost have the same performance.
----- 原始邮件 -----
主题：回复：Re: Openssl 1.1.0f Support
日期：2017年09月22日 10点55分
With the patch, the ERR_clear_error() will only be called when the error
occurs. In the normal situation, ERR_clear_error() will not be called, so
the Err_get_state() will not be called and no lock contention in openssl
1.0.1 with the patch.
----- 原始邮件 -----
主题：Re: Openssl 1.1.0f Support
日期：2017年09月21日 23点37分
This only changes the order of the calls. There is still going to be lock
contention inside OpenSSL 1.0.1.
-Bryan
The following traffic server patch can improve openssl 1.0.1 performance as
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}
@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
Date: Thursday, September 21, 2017 at 8:38 AM
Subject: Re: Openssl 1.1.0f Support
I meant to say 1.1.0.
-Bryan
I was see something like 2x the performance in my benchmarks with OpenSSL
1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since
May, when I upgraded to Fedora 26.
-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now
at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework.
This
was all static code analysis. Integration looked like a non-starter, so no
performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured order
of magnitude improvement over with the same using Openssl 1.0.1(x) and small
packet sizes... Improvement attributed to lock contention issues in the
older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration

Susan and Dave Thompson were working on something related to that,
"crypto
proxy". There's a small mention of it by Susan at the Fall 2016 summit in
the TLS state slides
(https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_TS_Presentations-2B-2D-2B2016&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=zhE5_TgdGKvMRHbD7pN1VeJPtK8Zzlu8mGsIu7xpNVM&e= ).
I'd
start there and see if you can bug Susan or Good Dave*. Although that
work
was designed to use an off box crypto engine, the implementation from the
ATS point of view is identical to what you're writing about. Susan will
be
at the Fall 2017 Summit, I'd look her up then as well.
* To distinguish from "Evil Dave" Carlin.

Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.

Jack Bates

2017-09-26 22:44:18 UTC

Permalink

Cool -- I replied to you pull request. Thanks!

Post by Chou, Peter
Jack,
Regarding compiling 6.2.x with OpenSSL 1.1.0f, I submitted PR #2570 which back-ports your fix for the sslheaders plugin that was required due to X509 being made opaque in OpenSSL 1.1. There was also a linking issue with Ubuntu when compiling with the 'with-openssl' configure option. Apparently, this might not occur on Red Hat which is what Jeremy used below.
Thanks,
Peter
-----Original Message-----
Sent: Friday, September 22, 2017 10:52 AM
Subject: Re: 回复：Re: Openssl 1.1.0f Support
openssl lib and include directories werent matching versions.
fixed that, and was able to compile without issue.

I remember backporting some fixes for building with OpenSSL 1.1 to 6.2 [1]
... Which errors are you still getting?
I tried just now and successfully built the 6.2 branch with OpenSSL 1.1.0f
$ git clone -b 6.2.x https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver.git&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=8HrQiBnIzItF-b6piI1aWqjSdcpsW9r59uuAoKA_sIQ&e=
$ cd trafficserver
$ autoreconf -i
$ ./configure
$ make
[1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver_pull_1321&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=nr0xsT0XCw7uLhrHvYwBhmQd5BbnSAKC2RfefCTpHgI&e=

Post by Jeremy Payne
Did you have to make any changes to 6.2 for it to compile cleanly
against openssl 1.1 ?
I can only get ATS 7+ to compile without producing any errors.

Post by i***@sina.cn
I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 + openssl
1.0.1 + patch respectively, and they almost have the same performance.
----- 原始邮件 -----
主题：回复：Re: Openssl 1.1.0f Support
日期：2017年09月22日 10点55分
With the patch, the ERR_clear_error() will only be called when the error
occurs. In the normal situation, ERR_clear_error() will not be called, so
the Err_get_state() will not be called and no lock contention in openssl
1.0.1 with the patch.
----- 原始邮件 -----
主题：Re: Openssl 1.1.0f Support
日期：2017年09月21日 23点37分
This only changes the order of the calls. There is still going to be lock
contention inside OpenSSL 1.0.1.
-Bryan
The following traffic server patch can improve openssl 1.0.1 performance as
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_write(ssl, buf, (int)nbytes);
if (ret > 0) {
nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
nbytes, int64_t &nwritten)
ERR_error_string_n(e, buf, sizeof(buf));
Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+
+ ERR_clear_error();
+
return ssl_error;
}
@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
int64_t &nread)
if (unlikely(nbytes == 0)) {
return SSL_ERROR_NONE;
}
- ERR_clear_error();
+
int ret = SSL_read(ssl, buf, (int)nbytes);
if (ret > 0) {
nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
int64_t &nread)
Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLAccept(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_accept(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
ssl_error_t
SSLConnect(SSL *ssl)
{
- ERR_clear_error();
int ret = SSL_connect(ssl);
if (ret > 0) {
return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d,
ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
}
+ ERR_clear_error();
+
return ssl_error;
}
Date: Thursday, September 21, 2017 at 8:38 AM
Subject: Re: Openssl 1.1.0f Support
I meant to say 1.1.0.
-Bryan
I was see something like 2x the performance in my benchmarks with OpenSSL
1.0.1. I have been doing all my development with OpenSSL 1.0.1 ATS since
May, when I upgraded to Fedora 26.
-Bryan
Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now
at best. The gist of my recollection is that QAT is an IO based async
engine, which of course ATS already has done extensively. I recall the
under-the-hood QAT longjumping was a non-starter in an ATS framework.
This
was all static code analysis. Integration looked like a non-starter, so no
performance test done.
Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni
acceleration", Susan (?Bryan?) was just telling me today of a measured order
of magnitude improvement over with the same using Openssl 1.0.1(x) and small
packet sizes... Improvement attributed to lock contention issues in the
older OpenSSL 1.0.1(x).
Dave
Dave,
Did you run any comparison performance tests using the QAT engine ?
Specifically around these configurations(or similar)
1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration

Susan has also run some performance tests with 7.1.x and openSSL 1.1
vs.
openSSL 1.0.2.