[jira] [Comment Edited] (CASSANDRA-10735) Support netty openssl (netty-tcnative) for client encryption
[
https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191824#comment-15191824
]
Andy Tolbert edited comment on CASSANDRA-10735 at 3/12/16 9:36 PM:
---
I went ahead and updated my
[branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] today to
rebase it off of trunk and add a commit that enables using netty openssl in
cassandra stress, example usage:
{code}
tools/bin/cassandra-stress write n=1000 -rate threads=250 -transport
ssl-openssl ssl-private-key=client.pem ssl-cert=client.crt
truststore=client.truststore truststore-password=cassandra
{code}
The performance characteristics were pretty much exactly the same as I had
observed previously, with exception to the no ssl run which had a few pauses
(didn't read too much into it as I was focusing on the SSL configs).
There are a few caveats / things I wanted to mention:
# The cipher suite format for openssl is different than the format jdk ssl
expects. Netty appropriately does translation when you provide the ciphers to
convert the jdk-style cipher names into what openssl expects, but there is no
way to convert it going back. {{SSLServerSocket.getSupportedCipherSuites()}}
appears to return the open ssl style. This makes things tricky as
{{SSLFactory.filterCipherSuites}} compares cipher's on the socket to those in
our configuration, which currently use the JDK format. I think this becomes a
non-issue if [~spo...@gmail.com]'s [CASSANDRA-10508] is implemented. You can
currently work around it by providing openssl cipher suites in your
cassandra.yaml under client_encryption_options (see
[this|https://gist.github.com/tolbertam/87267fa5708d998b57ee] for a list).
# One thing that is interesting to me is that netty's openssl builder will
allow you to use a truststore for establishing trust, but not a keystore for
your private keys. It looks like it does some work extracting items from the
truststore into X509 certs. Because of this we need separate options for
private key & cert (for openssl) and keystore (for jdk). I wonder if something
similar can be done for keys in netty, will pursue this.
# The transport encryption options do not let you override the keystore and
keystore password used for jdk ssl, it just uses defaults (conf/.keystore and
cassandra for password). Would be good to add those as extra options, will add
commit for that.
# With the version of netty we are using here (4.0.33) it seems the most recent
version of netty-tcnative that works is
[1.1.33.Fork11|http://central.maven.org/maven2/io/netty/netty-tcnative/1.1.33.Fork11/].
I ran into problems using Fork14 on the client side
([issue|https://github.com/netty/netty-tcnative/issues/124]), but it seems to
work well on the server side. For openssl to work, download the appropriate
platform jar and dump it in cassandra's lib directory.
# We could consider including the netty-tcnative jars in the lib directory as
part of the distribution. The only problem would be where/if we have multiple
jars that could match the same platform. For example, in Fork11 there are
separate jars for fedora x64 and linux x64 and the presence of both in the
classpath can be a problem.
Attached [^netty-ssl-trunk.tgz] which includes stress logs, example
keystores/certs/pems/etc., and the html benchmark report (doesn't work in
chrome for some reason).
!sslbench12-03.png!
(Red: no encryption, Orange: openssl on both ends, Purple: openssl client, jdk
server, Green: jdk client jdk server, Blue: jdk client, openssl server
was (Author: andrew.tolbert):
I went ahead and updated my
[branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] today to
rebase it off of trunk and add a commit that enables using netty openssl in
cassandra stress, example usage:
{code}
tools/bin/cassandra-stress write n=1000 -rate threads=250 -transport
ssl-openssl ssl-private-key=client.pem ssl-cert=client.crt
truststore=client.truststore truststore-password=cassandra
{code}
The performance characteristics were pretty much exactly the same as I had
observed previously, with exception to the no ssl run which had a few pauses
(didn't read too much into it as I was focusing on the SSL configs).
There are a few caveats / things I wanted to mention:
# The cipher suite format for openssl is different than the format jdk ssl
expects. Netty appropriately does translation when you provide the ciphers to
convert the jdk-style cipher names into what openssl expects, but there is no
way to convert it going back. {{SSLServerSocket.getSupportedCipherSuites()}}
appears to return the open ssl style. This makes things tricky as
{{SSLFactory.filterCipherSuites}} compares cipher's on the socket to those in
our configuration, which currently use the JDK format. I think this becomes a
non-issue if [
[jira] [Comment Edited] (CASSANDRA-10735) Support netty openssl (netty-tcnative) for client encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025970#comment-15025970 ] Andy Tolbert edited comment on CASSANDRA-10735 at 11/25/15 2:03 AM: I made an attempt at implementing this just to get an idea of what kind of performance improvement would be seen. My changes are on a [personal branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] in github. Summary of changes: * Update from netty 4.0.23 to 4.0.33, primarily for SslContextBuilder and I think some API calls may be missing at 4.0.23 (need to confirm). * Add support for using netty openssl. This requires platform-specific netty-tcnative jar in classpath (did not put in build.xml), openssl, and libapr1. * Netty's SslContext is used for both openssl and jdk ssl. * New config options under client_encryption_options: ** provider (default jdk): What ssl provider to use (openssl|jdk) ** key_cert_chain: certificate to use for server. ** key: private key file for cert. ** key_password (default: null (unprotected): password for private key file if encrypted. ** trust_cert_chain: certificate chain file for trust (used only if use_client_auth is true). * I also have some changes to use netty ssl with the java driver in cassandra-stress, but these aren't included (but will add them). stress output including graph data from [CASSANDRA-7918]: [^nettyssl-bench.tgz] The throughput improvement is very apparent (313.68% increase in op/s for write) when using netty openssl in both C* and the java driver, but not so much if one of the two. The total gc'd mb is 50.58% less with openssl vs. jdk ssl. Legend (highest write throughput to worst): ||Color||Client Config||Server Config||op/s rate||% op/s diff existing SSL||total gc mb|| ||Purple|None|None|109047|--|61097| ||Blue|OpenSSL|OpenSSL|76849|+313.68%|75400| ||Green|OpenSSL|JDK SSL|26331|+41.7%|146380| ||Orange|JDK SSL|OpenSSL|25450|+37.0%|112929| ||Red|JDK SSL|JDK SSL|18577|0%|152548| !nettysslbench_small.png! See html file in [^nettyssl-bench.tgz] for more graphical data (doesn't work in chrome for some reason, but works for me in safari). I believe the variance between no ssl and using netty openssl would be less pronounced if system cpu was not saturated during the test. Hardware configuration (both client and server): * 2 x Xeon 12 cores CPU * 128 GB RAM * 2 x Samsung 1 TB SSD (850 EVO) * 2 x Gigabit Ethernet was (Author: andrew.tolbert): I made an attempt at implementing this just to get an idea of what kind of performance improvement would be seen. My changes are on a [personal branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] in github. Summary of changes: * Update from netty 4.0.23 to 4.0.33, primarily for SslContextBuilder and I think some API calls may be missing at 4.0.23 (need to confirm). * Add support for using netty openssl. This requires platform-specific netty-tcnative jar in classpath (did not put in build.xml), openssl, and libapr1. * Netty's SslContext is used for both openssl and jdk ssl. * New config options under client_encryption_options: ** provider (default jdk): What ssl provider to use (openssl|jdk) ** key_cert_chain: certificate to use for server. ** key: private key file for cert. ** key_password (default: null (unprotected): password for private key file if encrypted. ** trust_cert_chain: certificate chain file for trust (used only if use_client_auth is true). * I also have some changes to use netty ssl with the java driver in cassandra-stress, but these aren't included (but will add them). stress output including graph data from [CASSANDRA-7918]: [^nettyssl-bench.tgz] The throughput improvement is very apparent when using netty openssl in both C* and the java driver, but not so much if one of the two is not using it as exhibited below: Legend (highest write throughput to worst): ||Color||Client Config||Server Config||op/s rate||+/- ops nossl||total gc mb|| ||Purple|None|None|109047|0%|61097| ||Blue|OpenSSL|OpenSSL|76849|-29.5%|75400| ||Green|OpenSSL|JDK SSL|26331|-75.9%|146380| ||Orange|JDK SSL|OpenSSL|25450|-76.7%|112929| ||Red|JDK SSL|JDK SSL|18577|-82.3%|152548| !nettysslbench_small.png! See html file in [^nettyssl-bench.tgz] for more graphical data (doesn't work in chrome for some reason, but works for me in safari). I believe the variance between no ssl and using netty openssl would be less pronounced if system cpu was not saturated during the test. Hardware configuration (both client and server): * 2 x Xeon 12 cores CPU * 128 GB RAM * 2 x Samsung 1 TB SSD (850 EVO) * 2 x Gigabit Ethernet > Support netty openssl (netty-tcnative) for client encryption > > > Key: CASSANDRA-10735 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10735 >
[jira] [Comment Edited] (CASSANDRA-10735) Support netty openssl (netty-tcnative) for client encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025970#comment-15025970 ] Andy Tolbert edited comment on CASSANDRA-10735 at 11/25/15 1:50 AM: I made an attempt at implementing this just to get an idea of what kind of performance improvement would be seen. My changes are on a [personal branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] in github. Summary of changes: * Update from netty 4.0.23 to 4.0.33, primarily for SslContextBuilder and I think some API calls may be missing at 4.0.23 (need to confirm). * Add support for using netty openssl. This requires platform-specific netty-tcnative jar in classpath (did not put in build.xml), openssl, and libapr1. * Netty's SslContext is used for both openssl and jdk ssl. * New config options under client_encryption_options: ** provider (default jdk): What ssl provider to use (openssl|jdk) ** key_cert_chain: certificate to use for server. ** key: private key file for cert. ** key_password (default: null (unprotected): password for private key file if encrypted. ** trust_cert_chain: certificate chain file for trust (used only if use_client_auth is true). * I also have some changes to use netty ssl with the java driver in cassandra-stress, but these aren't included (but will add them). stress output including graph data from [CASSANDRA-7918]: [^nettyssl-bench.tgz] The throughput improvement is very apparent when using netty openssl in both C* and the java driver, but not so much if one of the two is not using it as exhibited below: Legend (highest write throughput to worst): ||Color||Client Config||Server Config||op/s rate||+/- ops nossl||total gc mb|| ||Purple|None|None|109047|0%|61097| ||Blue|OpenSSL|OpenSSL|76849|-29.5%|75400| ||Green|OpenSSL|JDK SSL|26331|-75.9%|146380| ||Orange|JDK SSL|OpenSSL|25450|-76.7%|112929| ||Red|JDK SSL|JDK SSL|18577|-82.3%|152548| !nettysslbench_small.png! See html file in [^nettyssl-bench.tgz] for more graphical data (doesn't work in chrome for some reason, but works for me in safari). I believe the variance between no ssl and using netty openssl would be less pronounced if system cpu was not saturated during the test. Hardware configuration (both client and server): * 2 x Xeon 12 cores CPU * 128 GB RAM * 2 x Samsung 1 TB SSD (850 EVO) * 2 x Gigabit Ethernet was (Author: andrew.tolbert): I made an attempt at implementing this just to get an idea of what kind of performance improvement would be seen. My changes are on a [personal branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] in github. Summary of changes: * Update from netty 4.0.23 to 4.0.33, primarily for SslContextBuilder and I think some API calls may be missing at 4.0.23 (need to confirm). * Add support for using netty openssl. This requires platform-specific netty-tcnative jar in classpath (did not put in build.xml), openssl, and libapr1. * Netty's SslContext is used for both openssl and jdk ssl. * New config options under client_encryption_options: ** provider (default jdk): What ssl provider to use (openssl|jdk) ** key_cert_chain: certificate to use for server. ** key: private key file for cert. ** key_password (default: null (unprotected): password for private key file if encrypted. ** trust_cert_chain: certificate chain file for trust (used only if use_client_auth is true). * I also have some changes to use netty ssl with the java driver in cassandra-stress, but these aren't included (but will add them). stress output including graph data from [CASSANDRA-7918]: [^nettyssl-bench.tgz] The throughput improvement is very apparent when using netty openssl in both C* and the java driver, but not so much if one of the two is not using it as exhibited below: Legend (highest write throughput to worst): ||Color||Client Config||Server Config||op/s rate||+/- ops nossl||total gc mb|| ||Purple|None|None|109047|0%|61097| ||Blue|OpenSSL|OpenSSL|76849|-29.5%|75400| ||Green|OpenSSL|JDK SSL|26331|-75.9%|146380| ||Orange|JDK SSL|OpenSSL|25450|-76.7%|112929| ||Red|JDK SSL|JDK SSL|18577|-82.3%|152548| !nettysslbench.png! See html file in [^nettyssl-bench.tgz] for more graphical data (doesn't work in chrome for some reason, but works for me in safari). I believe the variance between no ssl and using netty openssl would be less pronounced if system cpu was not saturated during the test. Hardware configuration (both client and server): * 2 x Xeon 12 cores CPU * 128 GB RAM * 2 x Samsung 1 TB SSD (850 EVO) * 2 x Gigabit Ethernet > Support netty openssl (netty-tcnative) for client encryption > > > Key: CASSANDRA-10735 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10735 > Project: Cassandra > Issue Type: Improvement >R
[jira] [Comment Edited] (CASSANDRA-10735) Support netty openssl (netty-tcnative) for client encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025970#comment-15025970 ] Andy Tolbert edited comment on CASSANDRA-10735 at 11/25/15 1:47 AM: I made an attempt at implementing this just to get an idea of what kind of performance improvement would be seen. My changes are on a [personal branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] in github. Summary of changes: * Update from netty 4.0.23 to 4.0.33, primarily for SslContextBuilder and I think some API calls may be missing at 4.0.23 (need to confirm). * Add support for using netty openssl. This requires platform-specific netty-tcnative jar in classpath (did not put in build.xml), openssl, and libapr1. * Netty's SslContext is used for both openssl and jdk ssl. * New config options under client_encryption_options: ** provider (default jdk): What ssl provider to use (openssl|jdk) ** key_cert_chain: certificate to use for server. ** key: private key file for cert. ** key_password (default: null (unprotected): password for private key file if encrypted. ** trust_cert_chain: certificate chain file for trust (used only if use_client_auth is true). * I also have some changes to use netty ssl with the java driver in cassandra-stress, but these aren't included (but will add them). stress output including graph data from [CASSANDRA-7918]: [^nettyssl-bench.tgz] The throughput improvement is very apparent when using netty openssl in both C* and the java driver, but not so much if one of the two is not using it as exhibited below: Legend (highest write throughput to worst): ||Color||Client Config||Server Config||op/s rate||+/- ops nossl||total gc mb|| ||Purple|None|None|109047|0%|61097| ||Blue|OpenSSL|OpenSSL|76849|-29.5%|75400| ||Green|OpenSSL|JDK SSL|26331|-75.9%|146380| ||Orange|JDK SSL|OpenSSL|25450|-76.7%|112929| ||Red|JDK SSL|JDK SSL|18577|-82.3%|152548| !nettysslbench.png! See html file in [^nettyssl-bench.tgz] for more graphical data (doesn't work in chrome for some reason, but works for me in safari). I believe the variance between no ssl and using netty openssl would be less pronounced if system cpu was not saturated during the test. Hardware configuration (both client and server): * 2 x Xeon 12 cores CPU * 128 GB RAM * 2 x Samsung 1 TB SSD (850 EVO) * 2 x Gigabit Ethernet was (Author: andrew.tolbert): I made an attempt at implementing this just to get an idea of what kind of performance improvement would be seen. My changes are on a [personal branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] in github. Summary of changes: * Update from netty 4.0.23 to 4.0.33, primarily for SslContextBuilder and I think some API calls may be missing at 4.0.23 (need to confirm). * Add support for using netty openssl. This requires platform-specific netty-tcnative jar in classpath (did not put in build.xml), openssl, and libapr1. * Netty's SslContext is used for both openssl and jdk ssl. * New config options under client_encryption_options: ** provider (default jdk): What ssl provider to use (openssl|jdk) ** key_cert_chain: certificate to use for server. ** key: private key file for cert. ** key_password (default: null (unprotected): password for private key file if encrypted. ** trust_cert_chain: certificate chain file for trust (used only if use_client_auth is true). * I also have some changes to use netty ssl with the java driver in cassandra-stress, but these aren't included (but will add them). stress output including graph data from [CASSANDRA-7918]: [^nettyssl-bench.tgz] The throughput improvement is very apparent when using netty openssl in both C* and the java driver, but not so much if one of the two is not using it as exhibited below: Legend (highest write throughput to worst): ||Color||Client Config||Server Config||op/s rate||+/- ops nossl||total gc mb|| ||Purple|None|None|109047|0%|61097| ||Blue|OpenSSL|OpenSSL|76849|-29.5%|75400| ||Green|OpenSSL|JDK SSL|26331|-75.9%|146380| ||Orange|JDK SSL|OpenSSL|25450|-76.7%|112929| ||Red|JDK SSL|JDK SSL|18577|-82.3%|152548| !nettysslbench.png|thumbnail! See html file in [^nettyssl-bench.tgz] for more graphical data (doesn't work in chrome for some reason, but works for me in safari). I believe the variance between no ssl and using netty openssl would be less pronounced if system cpu was not saturated during the test. Hardware configuration (both client and server): * 2 x Xeon 12 cores CPU * 128 GB RAM * 2 x Samsung 1 TB SSD (850 EVO) * 2 x Gigabit Ethernet > Support netty openssl (netty-tcnative) for client encryption > > > Key: CASSANDRA-10735 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10735 > Project: Cassandra > Issue Type: Improvement >
