> On Jan. 7, 2020, 10:21 a.m., Peter Vary wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SplitGrouper.java > > Lines 282 (patched) > > <https://reviews.apache.org/r/71949/diff/1/?file=2193164#file2193164line283> > > > > I do not see this in the original code. What is this for?
I made a bit of simplification here. The OrcSplit.parse() call does some metadata preparation steps, like fetching the writeID, transactionID and bucketID and setting it to the split. Originally, this was done in ComparatorCompactor, but my point of view is that the comparator should be used only for comparing OrcSplits and not to prepare data. Since, we are already iterating over the splits, it made sense to move the call here. > On Jan. 7, 2020, 10:21 a.m., Peter Vary wrote: > > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java > > Lines 1228 (patched) > > <https://reviews.apache.org/r/71949/diff/1/?file=2193166#file2193166line1228> > > > > Do we need this? With delete_delta we do not supposed to have > > syntehetic rowIDs... That is correct, but I had to introduce this condition as well, to avoid IllegalStateException. Without this check, the splits in a delete delta directory would be considered as original files but without the correct parent. - Laszlo ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71949/#review219140 ----------------------------------------------------------- On Jan. 4, 2020, 9:06 a.m., Laszlo Pinter wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71949/ > ----------------------------------------------------------- > > (Updated Jan. 4, 2020, 9:06 a.m.) > > > Review request for hive, Denys Kuzmenko, Karen Coppage, and Peter Vary. > > > Repository: hive-git > > > Description > ------- > > HIVE-20934: ACID: Query based compactor for minor compaction > > > Diffs > ----- > > itests/hive-unit/pom.xml bc20cd6168dd61222c75fb866deada26328986dd > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorTestUtil.java > PRE-CREATION > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java > 445e39c260edc68f511550271a7ac471fae908fe > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCrudCompactorOnTez.java > b7245e2c3570b362a00b65b23f3f84616d0a3d1e > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SplitGrouper.java > 33d723a02e28d69a69b88281038f69b5aecfe6a2 > ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java > 3c508ec6cf620aee6a7791c6ab52c331ad5ec6bd > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java > 2ac6232460fedb8351b5f0cfae2ce2d0f2e2d948 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java > 0a96fc30b359043293017b235a36cd044ddb176e > ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java > 20b0ccd94b5f08aa2c1dace1301a8315bd202bf7 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > 2b2cc1a2ba8377aa3681b1a3454a0d64369eef64 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java > 7a0e32463d28007cff5526ae037cc1447e50a50b > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java > 38689ef86c607a36f8ec961a88578c13bfcd5b01 > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MinorQueryCompactor.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java > 9b8420902fb688b218fa432d70f71302f9f180e6 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java > 1eab5b888deef2d0fb5c097941a1dafa51c7d46b > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java > 41cb4b64fbc79dcf81919769c567b26a2e18cfe5 > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommandsForMmTable.java > d4c9121c9f17f8d083f1e1af1caf840678a3559d > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommandsForOrcMmTable.java > d6435342aa1f56ba5495a657b4a43327fdc49645 > > > Diff: https://reviews.apache.org/r/71949/diff/1/ > > > Testing > ------- > > > Thanks, > > Laszlo Pinter > >
