[
https://issues.apache.org/jira/browse/HADOOP-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477489#comment-16477489
]
Wei-Chiu Chuang edited comment on HADOOP-10768 at 5/16/18 2:26 PM:
-------------------------------------------------------------------
!cpu_profile_RPC_encryption_AES.png|thumbnai!
OpenSSLCipher.update() uses 22.9% CPU
SaslCryptoCodec$Integrity.calculateHMAC() uses 13.% CPU
The microbenchmarks I ran show the performance of RPC encryption improved by
almost 3.7x, though still around 35% slower than RPC authentication/no SASL.
(BTW: RPC integral is a very rarely used configuration. RPC encryption/RPC
authentication is more common)
||Configuration||RPC calls per second||
|No SASL|44,188|
|AUTHENTICATION|44,473|
|INTEGRITY|29,877|
|PRIVACY+AES/CTR/NoPadding|28,650|
|PRIVACY w/o AES|7,770|
The microbenchmark runs on a Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz in AWS,
8 cores.
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024 -a
-q AUTHENTICATION
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024 -a
-q INTEGRITY
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 1 -s 1 -w 60 -t 60 -m 1024 -a -q
PRIVACY -f AES/CTR/NoPadding
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024 -a
-q PRIVACY
[~mmokhtar] is also helping to evaluate the performance using Impala
benchmarks. However, so far we don't see much performance improvement at
Impala's level.
was (Author: jojochuang):
!cpu_profile_RPC_encryption_AES.png|thumbnai!
OpenSSLCipher.update() uses 22.9% CPU
SaslCryptoCodec$Integrity.calculateHMAC() uses 13.% CPU
The microbenchmarks I ran shows the performance of RPC encryption has improved
by almost 3.7x, though still around 35% slower than RPC authentication/no SASL.
(BTW: RPC integral is a very rarely used configuration. RPC encryption/RPC
authentication is more common)
No SASL: 44,188 calls/s
AUTHENTICATION: 44,473 calls/s
INTEGRITY: 29,877 calls/s
PRIVACY+AES/CTR/NoPadding: 28,650 calls/s
PRIVACY (no AES-NI): 7,770 calls/s
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024 -a
-q AUTHENTICATION
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024 -a
-q INTEGRITY
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 1 -s 1 -w 60 -t 60 -m 1024 -a -q
PRIVACY -f AES/CTR/NoPadding
hadoop jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-3.0.0-cdh6.0.0-beta1-tests.jar
org.apache.hadoop.ipc.RPCCallBenchmark -r 4 -c 30 -s 30 -w 60 -t 60 -m 1024 -a
-q PRIVACY
[~mmokhtar] is also helping to evaluate the performance using Impala
benchmarks. However, so far we don't see much performance improvement at
Impala's level.
> Optimize Hadoop RPC encryption performance
> ------------------------------------------
>
> Key: HADOOP-10768
> URL: https://issues.apache.org/jira/browse/HADOOP-10768
> Project: Hadoop Common
> Issue Type: Improvement
> Components: performance, security
> Affects Versions: 3.0.0-alpha1
> Reporter: Yi Liu
> Assignee: Dapeng Sun
> Priority: Major
> Attachments: HADOOP-10768.001.patch, HADOOP-10768.002.patch,
> HADOOP-10768.003.patch, HADOOP-10768.004.patch, HADOOP-10768.005.patch,
> HADOOP-10768.006.patch, HADOOP-10768.007.patch, HADOOP-10768.008.patch,
> HADOOP-10768.009.patch, Optimize Hadoop RPC encryption performance.pdf,
> cpu_profile_RPC_encryption_AES.png
>
>
> Hadoop RPC encryption is enabled by setting {{hadoop.rpc.protection}} to
> "privacy". It utilized SASL {{GSSAPI}} and {{DIGEST-MD5}} mechanisms for
> secure authentication and data protection. Even {{GSSAPI}} supports using
> AES, but without AES-NI support by default, so the encryption is slow and
> will become bottleneck.
> After discuss with [~atm], [~tucu00] and [~umamaheswararao], we can do the
> same optimization as in HDFS-6606. Use AES-NI with more than *20x* speedup.
> On the other hand, RPC message is small, but RPC is frequent and there may be
> lots of RPC calls in one connection, we needs to setup benchmark to see real
> improvement and then make a trade-off.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
