[ https://issues.apache.org/jira/browse/HADOOP-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487689#comment-16487689 ]
Wei-Chiu Chuang commented on HADOOP-10768: ------------------------------------------ Sorry for the delay on my part -- uploaded v010 that added the thread local Mac instance, without addressing the code review for rev009: (here's the diff comparing v010 and v009) diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslCryptoCodec.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslCryptoCodec.java index 12dd436..e49f9c5 100644 --- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslCryptoCodec.java +++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslCryptoCodec.java @@ -48,6 +48,26 @@ private final Integrity integrity; + /** + * The name of the hashing algorithm. + */ + private static final String DEFAULT_HMAC_ALGORITHM = "HmacMD5"; + /** + * A thread local store for the Macs. + */ + private static final ThreadLocal<Mac> threadLocalMac = + new ThreadLocal<Mac>(){ + @Override + protected Mac initialValue() { + try { + return Mac.getInstance(DEFAULT_HMAC_ALGORITHM); + } catch (NoSuchAlgorithmException nsa) { + throw new RuntimeException( + "Error creating instance of MD5 MAC algorithm", nsa); + } + } + }; + public SaslCryptoCodec(Configuration conf, CipherOption cipherOption, boolean isServer) throws IOException { CryptoCodec codec = CryptoCodec.getInstance(conf, @@ -174,9 +194,9 @@ void incPeerSeqNum() { System.arraycopy(seqNum, 0, seqAndMsg, 0, SEQ_NUM_LENGTH); System.arraycopy(msg, start, seqAndMsg, SEQ_NUM_LENGTH, len); + Mac m = threadLocalMac.get(); try { - SecretKey keyKi = new SecretKeySpec(key, "HmacMD5"); - Mac m = Mac.getInstance("HmacMD5"); + SecretKey keyKi = new SecretKeySpec(key, DEFAULT_HMAC_ALGORITHM); m.init(keyKi); m.update(seqAndMsg); byte[] hMacMd5 = m.doFinal(); @@ -189,9 +209,6 @@ void incPeerSeqNum() { } catch (InvalidKeyException e) { throw new SaslException("Invalid bytes used for key of HMAC-MD5 hash.", e); - } catch (NoSuchAlgorithmException e) { - throw new SaslException("Error creating instance of MD5 MAC algorithm", - e); } } diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java index 2393f49..93329fd 100644 --- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java +++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java @@ -35,7 +35,10 @@ public void testBenchmarkWithProto() throws Exception { "--time", "5", "--serverReaderThreads", "4", "--messageSize", "1024", - "--engine", "protobuf"}); + "--engine", "protobuf", + "--sasl", + "--qop", "PRIVACY", + "--cipher", "AES/CTR/NoPadding"}); assertEquals(0, rc); } } > Optimize Hadoop RPC encryption performance > ------------------------------------------ > > Key: HADOOP-10768 > URL: https://issues.apache.org/jira/browse/HADOOP-10768 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, security > Affects Versions: 3.0.0-alpha1 > Reporter: Yi Liu > Assignee: Dapeng Sun > Priority: Major > Attachments: HADOOP-10768.001.patch, HADOOP-10768.002.patch, > HADOOP-10768.003.patch, HADOOP-10768.004.patch, HADOOP-10768.005.patch, > HADOOP-10768.006.patch, HADOOP-10768.007.patch, HADOOP-10768.008.patch, > HADOOP-10768.009.patch, HADOOP-10768.010.patch, Optimize Hadoop RPC > encryption performance.pdf, cpu_profile_RPC_encryption_AES.png, > cpu_profile_rpc_encryption_optimize_calculateHMAC.png > > > Hadoop RPC encryption is enabled by setting {{hadoop.rpc.protection}} to > "privacy". It utilized SASL {{GSSAPI}} and {{DIGEST-MD5}} mechanisms for > secure authentication and data protection. Even {{GSSAPI}} supports using > AES, but without AES-NI support by default, so the encryption is slow and > will become bottleneck. > After discuss with [~atm], [~tucu00] and [~umamaheswararao], we can do the > same optimization as in HDFS-6606. Use AES-NI with more than *20x* speedup. > On the other hand, RPC message is small, but RPC is frequent and there may be > lots of RPC calls in one connection, we needs to setup benchmark to see real > improvement and then make a trade-off. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org