[jira] [Commented] (HIVE-12905) Issue with mapjoin in tez under certain conditions

2017-03-17 Thread Vaibhav Gumashta (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930782#comment-15930782
 ] 

Vaibhav Gumashta commented on HIVE-12905:
-

[~sershe] Removing 1.2.2 from target version as this patch will need few others 
(e.g. HIVE-11262, HIVE-11182 etc) and I'm not sure we should backport all. If 
there is a patch for 1.2 before release, we can always incorporate that; let me 
know if you feel otherwise.

> Issue with mapjoin in tez under certain conditions
> --
>
> Key: HIVE-12905
> URL: https://issues.apache.org/jira/browse/HIVE-12905
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 1.0.1, 1.2.1, 2.0.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HIVE-12905.02.patch, HIVE-12905.1.patch
>
>
> In a specific case where we have an outer join followed by another join on 
> the same key and the non-outer side of the outer join is empty, hive-on-tez 
> produces incorrect results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-12905) Issue with mapjoin in tez under certain conditions

2016-01-25 Thread Vikram Dixit K (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116624#comment-15116624
 ] 

Vikram Dixit K commented on HIVE-12905:
---

Latest patch looks good to me. The only failing test of concern is the new test 
under mapreduce (TestCliDriver). I think the results actually match but the 
diff may be in the tab/space.

> Issue with mapjoin in tez under certain conditions
> --
>
> Key: HIVE-12905
> URL: https://issues.apache.org/jira/browse/HIVE-12905
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 1.0.1, 1.2.1, 2.0.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Blocker
> Attachments: HIVE-12905.02.patch, HIVE-12905.1.patch
>
>
> In a specific case where we have an outer join followed by another join on 
> the same key and the non-outer side of the outer join is empty, hive-on-tez 
> produces incorrect results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12905) Issue with mapjoin in tez under certain conditions

2016-01-25 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116612#comment-15116612
 ] 

Hive QA commented on HIVE-12905:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12784217/HIVE-12905.02.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10031 tests 
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_empty_join
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6738/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6738/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6738/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12784217 - PreCommit-HIVE-TRUNK-Build

> Issue with mapjoin in tez under certain conditions
> --
>
> Key: HIVE-12905
> URL: https://issues.apache.org/jira/browse/HIVE-12905
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 1.0.1, 1.2.1, 2.0.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Blocker
> Attachments: HIVE-12905.02.patch, HIVE-12905.1.patch
>
>
> In a specific case where we have an outer join followed by another join on 
> the same key and the non-outer side of the outer join is empty, hive-on-tez 
> produces incorrect results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12905) Issue with mapjoin in tez under certain conditions

2016-01-23 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113691#comment-15113691
 ] 

Hive QA commented on HIVE-12905:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12783664/HIVE-12905.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 51 failed/errored test(s), 9437 tests 
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join11.q-vector_groupby_3.q-smb_mapjoin_8.q-and-3-more 
- did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join18.q-union_remove_23.q-input1_limit.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join30.q-scriptfile1.q-join_reorder3.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join9.q-bucketmapjoin10.q-skewjoinopt19.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join_reordering_values.q-auto_sortmerge_join_7.q-multigroupby_singlemr.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-avro_decimal_native.q-bucketmapjoin12.q-ppd_outer_join2.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-avro_joins.q-list_bucket_dml_2.q-groupby_ppr.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-bucketsortoptimize_insert_7.q-enforce_order.q-join36.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-escape_distributeby1.q-union_remove_7.q-skewjoin_union_remove_2.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-groupby2_noskew_multi_distinct.q-skewjoin_noskew.q-vector_data_types.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not 
produce a TEST-*.xml file
TestSparkCliDriver-groupby4.q-timestamp_null.q-auto_join23.q-and-12-more - did 
not produce a TEST-*.xml file
TestSparkCliDriver-groupby_complex_types.q-vectorization_10.q-join4.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-groupby_grouping_id2.q-bucketmapjoin4.q-groupby7.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-join13.q-auto_join24.q-vector_decimal_aggregate.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-join2.q-script_pipe.q-groupby_map_ppr.q-and-12-more - did 
not produce a TEST-*.xml file
TestSparkCliDriver-join39.q-stats12.q-union27.q-and-12-more - did not produce a 
TEST-*.xml file
TestSparkCliDriver-join_cond_pushdown_3.q-groupby7_noskew.q-auto_join13.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-join_cond_pushdown_unqual4.q-vectorization_16.q-groupby10.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more 
- did not produce a TEST-*.xml file
TestSparkCliDriver-load_dyn_part12.q-nullgroup4_multi_distinct.q-union14.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-load_dyn_part5.q-skewjoinopt8.q-groupby1_noskew.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-parallel_join0.q-union_remove_9.q-smb_mapjoin_21.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-parallel_join1.q-ptf_general_queries.q-ptf_rcfile.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-ppd_gby_join.q-stats2.q-groupby_rollup1.q-and-12-more - did 
not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not 
produce a TEST-*.xml file
TestSparkCliDriver-ppd_transform.q-ptf_seqfile.q-join11.q-and-12-more - did not 
produce a TEST-*.xml file
TestSparkCliDriver-skewjoinopt15.q-bucketmapjoin3.q-auto_join10.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-skewjoinopt16.q-join18.q-union11.q-and-12-more - did not 
produce a TEST-*.xml file
TestSparkCliDriver-skewjoinopt3.q-multi_insert_mixed.q-runtime_skewjoin_mapjoin_spark.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-smb_mapjoin_15.q-auto_sortmerge_join_13.q-auto_join18_multi_distinct.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-smb_mapjoin_4.q-auto_join19.q-mapreduce1.q-and-12-more - did 
not produce a TEST-*.xml file
TestSparkCliDriver-stats13.q-groupby6_map.q-join_casesensitive.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-tez_joins_explain.q-input17.q-union29.q-and-12-more - did 
not produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-union_remove_15.q-union32.q-custom_input_output_format.q-and-12-more
 - did n

[jira] [Commented] (HIVE-12905) Issue with mapjoin in tez under certain conditions

2016-01-22 Thread Sergey Shelukhin (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112779#comment-15112779
 ] 

Sergey Shelukhin commented on HIVE-12905:
-

Nit:  if (writeHelper == null) { check should not be necessary, right?

Also do MR and Spark HashtableLoaders use putRow too? I don't recall, they need 
to be updated if they do.

Otherwise +1 pending tests.


> Issue with mapjoin in tez under certain conditions
> --
>
> Key: HIVE-12905
> URL: https://issues.apache.org/jira/browse/HIVE-12905
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 1.0.1, 1.2.1, 2.0.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Blocker
> Attachments: HIVE-12905.1.patch
>
>
> In a specific case where we have an outer join followed by another join on 
> the same key and the non-outer side of the outer join is empty, hive-on-tez 
> produces incorrect results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)