-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71904/
-----------------------------------------------------------
Review request for hive, Gopal V and Peter Vary.
Bugs: HIVE-21164
https://issues.apache.org/jira/browse/HIVE-21164
Repository: hive-git
Description
-------
Extended the original patch with saving the task attempt ids in the file names
and also fixed some bugs in the original patch.
With this fix, inserting into an ACID table would not use move task to place
the generated files into the final directory. It will inserts every files to
the final directory and then clean up the files which are not needed (like
written by failed task attempts).
Also fixed the replication tests which failed for the original patch as well.
Diffs
-----
hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java
da677c7
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestAcidOnTez.java
2868427
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
31d15fd
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java
445e39c
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCrudCompactorOnTez.java
b7245e2
ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractFileMergeOperator.java
9a32581
ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 9ad4e71
ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 06e4ebe
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 3d30d09
ql/src/java/org/apache/hadoop/hive/ql/io/AcidInputFormat.java bba3960
ql/src/java/org/apache/hadoop/hive/ql/io/AcidOutputFormat.java 1e8bb22
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 3c508ec
ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 8980a62
ql/src/java/org/apache/hadoop/hive/ql/io/RecordUpdater.java 737e677
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 76984ab
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat.java c4c56f8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java 2ac6232
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 3fa61d3
ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java
2543dc6
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java f4bd0f9
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 73ca658
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 90549f9
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java c102a69
ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java ecc7bde
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java bed0581
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 2b2cc1a
ql/src/java/org/apache/hadoop/hive/ql/util/UpgradeTool.java 58e6289
ql/src/test/org/apache/hadoop/hive/ql/TestTxnAddPartition.java c9cb669
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 8421408
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java 88ca683
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands3.java 908ceb4
ql/src/test/org/apache/hadoop/hive/ql/TestTxnConcatenate.java 8676e0d
ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java 66b2b27
ql/src/test/org/apache/hadoop/hive/ql/TestTxnLoadData.java bb55d9f
ql/src/test/org/apache/hadoop/hive/ql/TestTxnNoBuckets.java ea6b1d9
ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java af14e62
ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java dd70524
ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 2c4b69b
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java c033a94
ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/CompactorTest.java
cfd7290
ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestWorker.java 70ae85c
ql/src/test/results/clientpositive/acid_subquery.q.out 1dc1775
ql/src/test/results/clientpositive/create_transactional_full_acid.q.out
e324d5e
ql/src/test/results/clientpositive/encrypted/encryption_insert_partition_dynamic.q.out
61b0057
ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 5571c53
ql/src/test/results/clientpositive/llap/insert_overwrite.q.out fbc3326
ql/src/test/results/clientpositive/llap/mm_all.q.out 7542a6a
ql/src/test/results/clientpositive/mm_all.q.out 1377856
streaming/src/test/org/apache/hive/streaming/TestStreaming.java 58b3ae2
Diff: https://reviews.apache.org/r/71904/diff/1/
Testing
-------
Had to modify some tests because of the file name changes. Also added some
specific tests.
In the pre-commit run all tests passed successfully.
Thanks,
Marta Kuczora
