[ https://issues.apache.org/jira/browse/HDFS-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14145832#comment-14145832 ]
Aaron T. Myers commented on HDFS-6606: -------------------------------------- The latest patch looks pretty good to me. I have one question and one small suggestion. Question: Am I reading this correctly that after this patch if both the client and server support AES that we have no way for clients to continue to use 3des, rc4, or des for data encryption? That may be acceptable if we think that AES is in all cases strictly superior to those other algorithms, but if so we should definitely call this out in the hdfs-default.xml description of "dfs.encrypt.data.transfer.algorithm". I'm thinking something along the lines of "note that if AES is supported by both the client and server then this encryption algorithm will only be used to initially transfer keys for AES." Suggestion: Now that {{DataTransferSaslUtil#performSaslStep1}} is only used in one place in the code, might just want to get rid of that function and inline its functionality. Thanks a lot, Yi. This is great work. > Optimize HDFS Encrypted Transport performance > --------------------------------------------- > > Key: HDFS-6606 > URL: https://issues.apache.org/jira/browse/HDFS-6606 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, hdfs-client, security > Reporter: Yi Liu > Assignee: Yi Liu > Attachments: HDFS-6606.001.patch, HDFS-6606.002.patch, > HDFS-6606.003.patch, HDFS-6606.004.patch, HDFS-6606.005.patch, > HDFS-6606.006.patch, OptimizeHdfsEncryptedTransportperformance.pdf > > > In HDFS-3637, [~atm] added support for encrypting the DataTransferProtocol, > it was a great work. > It utilizes SASL {{Digest-MD5}} mechanism (use Qop: auth-conf), it supports > three security strength: > * high 3des or rc4 (128bits) > * medium des or rc4(56bits) > * low rc4(40bits) > 3des and rc4 are slow, only *tens of MB/s*, > http://www.javamex.com/tutorials/cryptography/ciphers.shtml > http://www.cs.wustl.edu/~jain/cse567-06/ftp/encryption_perf/ > I will give more detailed performance data in future. Absolutely it’s > bottleneck and will vastly affect the end to end performance. > AES(Advanced Encryption Standard) is recommended as a replacement of DES, > it’s more secure; with AES-NI support, the throughput can reach nearly > *2GB/s*, it won’t be the bottleneck any more, AES and CryptoCodec work is > supported in HADOOP-10150, HADOOP-10603 and HADOOP-10693 (We may need to add > a new mode support for AES). > This JIRA will use AES with AES-NI support as encryption algorithm for > DataTransferProtocol. -- This message was sent by Atlassian JIRA (v6.3.4#6332)