[ https://issues.apache.org/jira/browse/HIVE-16832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084858#comment-16084858 ]
Hive QA commented on HIVE-16832: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12876914/HIVE-16832.22.patch {color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10888 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1] (batchId=237) org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite] (batchId=237) org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_queries] (batchId=94) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=143) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=99) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=232) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=232) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=177) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=177) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=177) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5989/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5989/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5989/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12876914 - PreCommit-HIVE-Build > duplicate ROW__ID possible in multi insert into transactional table > ------------------------------------------------------------------- > > Key: HIVE-16832 > URL: https://issues.apache.org/jira/browse/HIVE-16832 > Project: Hive > Issue Type: Bug > Components: Transactions > Affects Versions: 2.2.0 > Reporter: Eugene Koifman > Assignee: Eugene Koifman > Priority: Critical > Attachments: HIVE-16832.01.patch, HIVE-16832.03.patch, > HIVE-16832.04.patch, HIVE-16832.05.patch, HIVE-16832.06.patch, > HIVE-16832.08.patch, HIVE-16832.09.patch, HIVE-16832.10.patch, > HIVE-16832.11.patch, HIVE-16832.14.patch, HIVE-16832.15.patch, > HIVE-16832.16.patch, HIVE-16832.17.patch, HIVE-16832.18.patch, > HIVE-16832.19.patch, HIVE-16832.20.patch, HIVE-16832.20.patch, > HIVE-16832.21.patch, HIVE-16832.22.patch > > > {noformat} > create table AcidTablePart(a int, b int) partitioned by (p string) clustered > by (a) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true'); > create temporary table if not exists data1 (x int); > insert into data1 values (1); > from data1 > insert into AcidTablePart partition(p) select 0, 0, 'p' || x > insert into AcidTablePart partition(p='p1') select 0, 1 > {noformat} > Each branch of this multi-insert create a row in partition p1/bucket0 with > ROW__ID=(1,0,0). > The same can happen when running SQL Merge (HIVE-10924) statement that has > both Insert and Update clauses when target table has > _'transactional'='true','transactional_properties'='default'_ (see > HIVE-14035). This is so because Merge is internally run as a multi-insert > statement. > The solution relies on statement ID introduced in HIVE-11030. Each Insert > clause of a multi-insert is gets a unique ID. > The ROW__ID.bucketId now becomes a bit packed triplet (format version, > bucketId, statementId). > (Since ORC stores field names in the data file we can't rename > ROW__ID.bucketId). > This ensures that there are no collisions and retains desired sort properties > of ROW__ID. > In particular _SortedDynPartitionOptimizer_ works w/o any changes even in > cases where there fewer reducers than buckets. -- This message was sent by Atlassian JIRA (v6.4.14#64029)