[
https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191824#comment-15191824
]
Andy Tolbert edited comment on CASSANDRA-10735 at 3/12/16 9:36 PM:
-------------------------------------------------------------------
I went ahead and updated my
[branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] today to
rebase it off of trunk and add a commit that enables using netty openssl in
cassandra stress, example usage:
{code}
tools/bin/cassandra-stress write n=10000000 -rate threads=250 -transport
ssl-openssl ssl-private-key=client.pem ssl-cert=client.crt
truststore=client.truststore truststore-password=cassandra
{code}
The performance characteristics were pretty much exactly the same as I had
observed previously, with exception to the no ssl run which had a few pauses
(didn't read too much into it as I was focusing on the SSL configs).
There are a few caveats / things I wanted to mention:
# The cipher suite format for openssl is different than the format jdk ssl
expects. Netty appropriately does translation when you provide the ciphers to
convert the jdk-style cipher names into what openssl expects, but there is no
way to convert it going back. {{SSLServerSocket.getSupportedCipherSuites()}}
appears to return the open ssl style. This makes things tricky as
{{SSLFactory.filterCipherSuites}} compares cipher's on the socket to those in
our configuration, which currently use the JDK format. I think this becomes a
non-issue if [~spo...@gmail.com]'s [CASSANDRA-10508] is implemented. You can
currently work around it by providing openssl cipher suites in your
cassandra.yaml under client_encryption_options (see
[this|https://gist.github.com/tolbertam/87267fa5708d998b57ee] for a list).
# One thing that is interesting to me is that netty's openssl builder will
allow you to use a truststore for establishing trust, but not a keystore for
your private keys. It looks like it does some work extracting items from the
truststore into X509 certs. Because of this we need separate options for
private key & cert (for openssl) and keystore (for jdk). I wonder if something
similar can be done for keys in netty, will pursue this.
# The transport encryption options do not let you override the keystore and
keystore password used for jdk ssl, it just uses defaults (conf/.keystore and
cassandra for password). Would be good to add those as extra options, will add
commit for that.
# With the version of netty we are using here (4.0.33) it seems the most recent
version of netty-tcnative that works is
[1.1.33.Fork11|http://central.maven.org/maven2/io/netty/netty-tcnative/1.1.33.Fork11/].
I ran into problems using Fork14 on the client side
([issue|https://github.com/netty/netty-tcnative/issues/124]), but it seems to
work well on the server side. For openssl to work, download the appropriate
platform jar and dump it in cassandra's lib directory.
# We could consider including the netty-tcnative jars in the lib directory as
part of the distribution. The only problem would be where/if we have multiple
jars that could match the same platform. For example, in Fork11 there are
separate jars for fedora x64 and linux x64 and the presence of both in the
classpath can be a problem.
Attached [^netty-ssl-trunk.tgz] which includes stress logs, example
keystores/certs/pems/etc., and the html benchmark report (doesn't work in
chrome for some reason).
!sslbench12-03.png!
(Red: no encryption, Orange: openssl on both ends, Purple: openssl client, jdk
server, Green: jdk client jdk server, Blue: jdk client, openssl server
was (Author: andrew.tolbert):
I went ahead and updated my
[branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] today to
rebase it off of trunk and add a commit that enables using netty openssl in
cassandra stress, example usage:
{code}
tools/bin/cassandra-stress write n=10000000 -rate threads=250 -transport
ssl-openssl ssl-private-key=client.pem ssl-cert=client.crt
truststore=client.truststore truststore-password=cassandra
{code}
The performance characteristics were pretty much exactly the same as I had
observed previously, with exception to the no ssl run which had a few pauses
(didn't read too much into it as I was focusing on the SSL configs).
There are a few caveats / things I wanted to mention:
# The cipher suite format for openssl is different than the format jdk ssl
expects. Netty appropriately does translation when you provide the ciphers to
convert the jdk-style cipher names into what openssl expects, but there is no
way to convert it going back. {{SSLServerSocket.getSupportedCipherSuites()}}
appears to return the open ssl style. This makes things tricky as
{{SSLFactory.filterCipherSuites}} compares cipher's on the socket to those in
our configuration, which currently use the JDK format. I think this becomes a
non-issue if [~spo...@gmail.com]'s [CASSANDRA-10508] is implemented. You can
currently work around it by providing openssl cipher suites in your
cassandra.yaml under client_encryption_options (see
[this|https://gist.github.com/tolbertam/87267fa5708d998b57ee] for a list).
# One thing that is interesting to me is that netty's openssl builder will
allow you to use a truststore for establishing trust, but not a keystore for
your private keys. It looks like it does some work extracting items from the
truststore into X509 certs. Because of this we need separate options for
private key & cert (for openssl) and keystore (for jdk). I wonder if something
similar can be done for keys in netty, will pursue this.
# The transport encryption options do not let you override the keystore and
keystore password used for jdk ssl, it just uses defaults (conf/.keystore and
cassandra for password). Would be good to add those as extra options, will add
commit for that.
# With the version of netty we are using here (4.0.33) it seems the most recent
version of netty-tcnative that works is
[1.1.33.Fork11|http://central.maven.org/maven2/io/netty/netty-tcnative/1.1.33.Fork11/].
I ran into problems using Fork14 on the client side
([issue|https://github.com/netty/netty-tcnative/issues/124]), but it seems to
work well on the server side. For openssl to work, download the appropriate
platform jar and dump it in cassandra's lib directory.
# We could consider including the netty-tcnative jars in the lib directory as
part of the distribution. The only problem would be where/if we have multiple
jars that could match the same platform. For example, in Fork11 there are
separate jars for fedora x64 and linux x64 and the presence of both in the
classpath can be a problem.
Attached [^netty-ssl-trunk.tgz] which includes stress logs, example
keystores/certs/pems/etc., and the html benchmark report (doesn't work in
chrome for some reason).
!sslbench12-03.png|thumbnail!
(Red: no encryption, Orange: openssl on both ends, Purple: openssl client, jdk
server, Green: jdk client jdk server, Blue: jdk client, openssl server
> Support netty openssl (netty-tcnative) for client encryption
> ------------------------------------------------------------
>
> Key: CASSANDRA-10735
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10735
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Andy Tolbert
> Assignee: Aleksey Yeschenko
> Fix For: 3.x
>
> Attachments: netty-ssl-trunk.tgz, nettyssl-bench.tgz,
> nettysslbench.png, nettysslbench_small.png, sslbench12-03.png
>
>
> The java-driver recently added support for using netty openssl via
> [netty-tcnative|http://netty.io/wiki/forked-tomcat-native.html] in
> [JAVA-841|https://datastax-oss.atlassian.net/browse/JAVA-841], this shows a
> very measured improvement (numbers incoming on that ticket). It seems
> likely that this can offer improvement if implemented C* side as well.
> Since netty-tcnative has platform specific requirements, this should not be
> made the default, but rather be an option that one can use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
