[ https://issues.apache.org/jira/browse/HADOOP-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480189#comment-16480189 ]
Dapeng Sun commented on HADOOP-10768: ------------------------------------- {quote} * CryptoInputStream the new readFully() method is a public method but lacks javadoc. is it different from the existing readFully() methods? Can you reuse the existing readFully() method? {quote} Currently, readFully is position read. the under layer FS must be PositionInputStream, so adding a new readFully here. {quote} * Client Why are we catching exceptions in Client.shouldAuthenticateOverKrb()? That seems unnecessary. If not catching exception causes a problem, please have a test case for it. {quote} Yes, shouldAuthenticateOverKrb() would throw NPE in some cases, we should catch it. {quote} * SaslUtil ** negotiateCipherOption() can you please throw a non IOException? I’m not in favor of making every method throwing a generic IOExceptions. Similarly, update the method signature for the code path (getCipherOption, processSaslToken) This method is very similar to DataTransferSaslUtil#negotiateCipherOption() except for the configuration keys. I see no reason to duplicate the code, especially it involves some coding/decoding, which is not that easy to comprehend. ** Also, every time this method is called, it returns a new List<>. I feel like this is too much of a cost. Can we reduce the memory footprint? {quote} I'm argee with you. {quote} * SaslRpcClient ** saslConnect() LOG.debug( "Get SASL RPC CipherOption from Conf" + cipherOptions); —> missing a space after Conf the method is well over 100 lines in length now. Some code refactor will greatly improve readability. ** handleSaslCipherOptions() throw new SaslException(e.getMessage(), e); the exception message should be more descriptive. It could be something like “Unable to initialize SaslCryptoCodec” {quote} Good suggestions, thanks! > Optimize Hadoop RPC encryption performance > ------------------------------------------ > > Key: HADOOP-10768 > URL: https://issues.apache.org/jira/browse/HADOOP-10768 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, security > Affects Versions: 3.0.0-alpha1 > Reporter: Yi Liu > Assignee: Dapeng Sun > Priority: Major > Attachments: HADOOP-10768.001.patch, HADOOP-10768.002.patch, > HADOOP-10768.003.patch, HADOOP-10768.004.patch, HADOOP-10768.005.patch, > HADOOP-10768.006.patch, HADOOP-10768.007.patch, HADOOP-10768.008.patch, > HADOOP-10768.009.patch, Optimize Hadoop RPC encryption performance.pdf, > cpu_profile_RPC_encryption_AES.png, > cpu_profile_rpc_encryption_optimize_calculateHMAC.png > > > Hadoop RPC encryption is enabled by setting {{hadoop.rpc.protection}} to > "privacy". It utilized SASL {{GSSAPI}} and {{DIGEST-MD5}} mechanisms for > secure authentication and data protection. Even {{GSSAPI}} supports using > AES, but without AES-NI support by default, so the encryption is slow and > will become bottleneck. > After discuss with [~atm], [~tucu00] and [~umamaheswararao], we can do the > same optimization as in HDFS-6606. Use AES-NI with more than *20x* speedup. > On the other hand, RPC message is small, but RPC is frequent and there may be > lots of RPC calls in one connection, we needs to setup benchmark to see real > improvement and then make a trade-off. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org