----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49766/ -----------------------------------------------------------
(Updated July 7, 2016, 10:26 a.m.) Review request for hive and Eugene Koifman. Repository: hive-git Description ------- https://issues.apache.org/jira/browse/HIVE-14035 In current Hive version, delta files created by ACID transactions do not allow predicate pushdown if they contain any update/delete events. This is done to preserve correctness when following a multi-version approach during event collapsing, where an update event overwrites an existing insert event. This JIRA proposes to split an update event into a combination of a delete event followed by a new insert event, that can enable predicate push down to all delta files without breaking correctness. To support backward compatibility for this feature, this JIRA also proposes to add some sort of versioning to ACID that can allow different versions of ACID transactions to co-exist together. Diffs ----- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b13fc65 hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java 14f7316 hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java 0c6b9ea metastore/if/hive_metastore.thrift 4d92b73 metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h ae14bd1 metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp f982bf2 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java 5a666f2 metastore/src/gen/thrift/gen-php/metastore/Types.php f505208 metastore/src/gen/thrift/gen-py/hive_metastore/constants.py d1c07a5 metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb eeccc84 metastore/src/java/org/apache/hadoop/hive/metastore/TransactionalValidationListener.java 3e74675 orc/src/java/org/apache/orc/impl/TreeReaderFactory.java c4a2093 ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java dff1815 ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 57b6c67 ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 23a13d6 ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 36f38f6 ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 227a051 ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 69d58d6 ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat.java b0f8c8b ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java e577961 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 82abd52 ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 8cf261d ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 6caca98 ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java e76c925 ql/src/test/org/apache/hadoop/hive/ql/io/TestAcidUtils.java 5745dee Diff: https://reviews.apache.org/r/49766/diff/ Testing (updated) ------- Tests for the feature are in ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java. These are mostly integration tests that test end-to-end insert/update/delete scenarios followed by compaction and cleaning. Thanks, Saket Saurabh
