[jira] [Commented] (HIVE-14402) Vectorization: Fix Mapjoin overflow deserialization
[ https://issues.apache.org/jira/browse/HIVE-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408819#comment-15408819 ] Hive QA commented on HIVE-14402: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12822192/HIVE-14402.2.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10440 tests executed *Failed tests:* {noformat} TestMsgBusConnection - did not produce a TEST-*.xml file TestQueryLifeTimeHook - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropPartition {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/779/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/779/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-779/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12822192 - PreCommit-HIVE-MASTER-Build > Vectorization: Fix Mapjoin overflow deserialization > > > Key: HIVE-14402 > URL: https://issues.apache.org/jira/browse/HIVE-14402 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 2.1.0, 2.2.0 >Reporter: Gopal V >Assignee: Gopal V > Attachments: HIVE-14402.1.patch, HIVE-14402.2.patch > > > This is in a codepath currently disabled in master, however enabling it > triggers OOB. > {code} > Caused by: java.lang.ArrayIndexOutOfBoundsException: 1024 > at > org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setRef(BytesColumnVector.java:92) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorDeserializeRow.java:415) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorDeserializeRow.java:674) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultLargeMultiValue(VectorMapJoinGenerateResultOperator.java:307) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultMultiValue(VectorMapJoinGenerateResultOperator.java:226) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultRepeatedAll(VectorMapJoinGenerateResultOperator.java:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14402) Vectorization: Fix Mapjoin overflow deserialization
[ https://issues.apache.org/jira/browse/HIVE-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405289#comment-15405289 ] Gopal V commented on HIVE-14402: One of the failures is {code} Exception: Cannot remove data directory: /home/hiveptest/54.177.94.186-hiveptest-1/apache-github-source-source/itests/qtest/target/test-data/0c21fd5c-308e-47e6-88d9-57d05d5998c8/dfscluster_7b68df1f-2301-43b9-8996-c12d41b0cc51/dfs/datapath '/home/hiveptest/54.177.94.186-hiveptest-1/apache-github-source-source/itests/qtest/target/test-data/0c21fd5c-308e-47e6-88d9-57d05d5998c8/dfscluster_7b68df1f-2301-43b9-8996-c12d41b0cc51/dfs/data': {code} > Vectorization: Fix Mapjoin overflow deserialization > > > Key: HIVE-14402 > URL: https://issues.apache.org/jira/browse/HIVE-14402 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 2.1.0, 2.2.0 >Reporter: Gopal V >Assignee: Gopal V > Attachments: HIVE-14402.1.patch > > > This is in a codepath currently disabled in master, however enabling it > triggers OOB. > {code} > Caused by: java.lang.ArrayIndexOutOfBoundsException: 1024 > at > org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setRef(BytesColumnVector.java:92) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorDeserializeRow.java:415) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorDeserializeRow.java:674) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultLargeMultiValue(VectorMapJoinGenerateResultOperator.java:307) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultMultiValue(VectorMapJoinGenerateResultOperator.java:226) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultRepeatedAll(VectorMapJoinGenerateResultOperator.java:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14402) Vectorization: Fix Mapjoin overflow deserialization
[ https://issues.apache.org/jira/browse/HIVE-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405258#comment-15405258 ] Hive QA commented on HIVE-14402: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12821663/HIVE-14402.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 10430 tests executed *Failed tests:* {noformat} TestMsgBusConnection - did not produce a TEST-*.xml file TestQueryLifeTimeHook - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_join21 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cte_5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_where_non_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapreduce2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_acid_mapwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_acidvec_mapwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vecrow_mapwork_part_all_complex org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_between_in org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_cast_constant org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join30 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_string_concat org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.stringifyValidTxns org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.testTxnRange org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/743/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/743/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-743/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 23 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12821663 - PreCommit-HIVE-MASTER-Build > Vectorization: Fix Mapjoin overflow deserialization > > > Key: HIVE-14402 > URL: https://issues.apache.org/jira/browse/HIVE-14402 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 2.1.0, 2.2.0 >Reporter: Gopal V >Assignee: Gopal V > Attachments: HIVE-14402.1.patch > > > This is in a codepath currently disabled in master, however enabling it > triggers OOB. > {code} > Caused by: java.lang.ArrayIndexOutOfBoundsException: 1024 > at > org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setRef(BytesColumnVector.java:92) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorDeserializeRow.java:415) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorDeserializeRow.java:674) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultLargeMultiValue(VectorMapJoinGenerateResultOperator.java:307) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultMultiValue(VectorMapJoinGenerateResultOperator.java:226) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultRepeatedAll(VectorMapJoinGenerateResultOperator.java:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14402) Vectorization: Fix Mapjoin overflow deserialization
[ https://issues.apache.org/jira/browse/HIVE-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404606#comment-15404606 ] Matt McCline commented on HIVE-14402: - +1 LGTM > Vectorization: Fix Mapjoin overflow deserialization > > > Key: HIVE-14402 > URL: https://issues.apache.org/jira/browse/HIVE-14402 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 2.1.0, 2.2.0 >Reporter: Gopal V >Assignee: Gopal V > Attachments: HIVE-14402.1.patch > > > This is in a codepath currently disabled in master, however enabling it > triggers OOB. > {code} > Caused by: java.lang.ArrayIndexOutOfBoundsException: 1024 > at > org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setRef(BytesColumnVector.java:92) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorDeserializeRow.java:415) > at > org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorDeserializeRow.java:674) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultLargeMultiValue(VectorMapJoinGenerateResultOperator.java:307) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultMultiValue(VectorMapJoinGenerateResultOperator.java:226) > at > org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultRepeatedAll(VectorMapJoinGenerateResultOperator.java:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)