----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33956/#review84671 -----------------------------------------------------------
Thank you for this patch. I have some questions and will have another round of review after understanding these questions. Thank you! common/src/java/org/apache/hive/common/util/HdfsEncryptionUtilities.java <https://reviews.apache.org/r/33956/#comment136026> Why not use the isPathEncrypted from HdfsEncryptionShim directly? common/src/java/org/apache/hive/common/util/HdfsEncryptionUtilities.java <https://reviews.apache.org/r/33956/#comment136027> The same as above. ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java <https://reviews.apache.org/r/33956/#comment136025> Is it possible to get the FsPermission from org.apache.hadoop.fs.FileContext? ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java <https://reviews.apache.org/r/33956/#comment136022> I am a little confused here. How can a local path be converted to a hdfs path? The original code is trying to create a tar file from a local path and uploading it to the hdfs with replication information. The new code path will lose the replication information. And the previous code path will only be executed in a local file or pfile schema in test. ql/src/test/queries/clientpositive/encryption_map_join_select.q <https://reviews.apache.org/r/33956/#comment136021> drop table encryptedTable PURGE; - cheng xu On May 7, 2015, 9:23 p.m., Sergio Pena wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33956/ > ----------------------------------------------------------- > > (Updated May 7, 2015, 9:23 p.m.) > > > Review request for hive, Brock Noland and cheng xu. > > > Bugs: HIVE-9614 > https://issues.apache.org/jira/browse/HIVE-9614 > > > Repository: hive-git > > > Description > ------- > > The security issue here is that encrypted tables used on MAP-JOIN queries, > and stored on the distribute cache, are first copied to the client local > filesystem in an unencrypted form in order to compress it there. > > This patch avoids the local copy if the table is encrypted on HDFS. It keeps > the hash table on HDFS, compresses the table in HDFS, and then adds it to the > distributed cache. > > Files that are copied to the datanodes by the distributed cache are still > unencrypted. This is a limitation we have from HDFS. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/common/CompressionUtils.java > 0e0d538c2faf1c52c4d8378df013294ae4efa41c > common/src/java/org/apache/hive/common/util/HdfsEncryptionUtilities.java > PRE-CREATION > itests/src/test/resources/testconfiguration.properties > 3eff7d010923a4e07d5024904f1531ca52473aa2 > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java > ad5c8f8302de2a15b1703161799f71cd81a94475 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java > d7a08ecf1c183fe56b5ca41c2c69d413874418bb > ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java > 4d84f0f76ce17711077ceadf23e6b9ed12e6a414 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MapJoinResolver.java > c0a72b69df3871bbcc870af286774aee5269668b > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java > cbc5466261f749fe7b84d7533dc0ff3274b6777f > ql/src/java/org/apache/hadoop/hive/ql/plan/MapredLocalWork.java > 82143a64db163da766dcc138231b4d4174603470 > ql/src/test/queries/clientpositive/encryption_map_join_select.q > PRE-CREATION > > ql/src/test/results/clientpositive/encrypted/encryption_map_join_select.q.out > PRE-CREATION > > Diff: https://reviews.apache.org/r/33956/diff/ > > > Testing > ------- > > > Thanks, > > Sergio Pena > >
