[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-25 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884458#comment-15884458
 ] 

Pengcheng Xiong commented on HIVE-15955:


pushed to master. Thanks [~ashutoshc] for the review.

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch, HIVE-15955.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879646#comment-15879646
 ] 

Hive QA commented on HIVE-15955:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12854017/HIVE-15955.04.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10253 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=223)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3704/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3704/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3704/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12854017 - PreCommit-HIVE-Build

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch, HIVE-15955.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Jason Dere (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879419#comment-15879419
 ] 

Jason Dere commented on HIVE-15955:
---

DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan()

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch, HIVE-15955.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879415#comment-15879415
 ] 

Pengcheng Xiong commented on HIVE-15955:


[~jdere], could u point me to the place where u added SEL-GBY-RS-GBY-RS branch? 
Thanks. 

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch, HIVE-15955.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878958#comment-15878958
 ] 

Pengcheng Xiong commented on HIVE-15955:


[~ashutoshc], i remove "transient" key words for the outputoperators as 
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_union] 
(batchId=145)
is failing. It seems that TestMiniLlapLocalCliDriver is 
serializing/deserializing the operator trees.

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch, HIVE-15955.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Pengcheng Xiong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878950#comment-15878950
 ] 

Pengcheng Xiong commented on HIVE-15955:


[~jdere], yes i think it should be done in a separate jira as this patch only 
deals with all the RS after logical optimization.

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch, HIVE-15955.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Jason Dere (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878911#comment-15878911
 ] 

Jason Dere commented on HIVE-15955:
---

I think this mostly looks good, though this misses one case - semijoin 
optimizations (min/max/bloomfilter) generate an extra SEL-GBY-RS-GBY-RS branch, 
but the last RS is never directly connected to the destination TableScan/Filter 
which uses the min/max/bloomfilter. Instead there is a separate mapping kept in 
the ParseContext to track these. This may require additional handling to call 
setOutputOperators() for this SEL-GBY-RS-GBY-RS branch (which is also generated 
during TezCompiler, so it might occur after the Optimizer rules have fired).
Should handling this be done in a separate Jira?

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878726#comment-15878726
 ] 

Ashutosh Chauhan commented on HIVE-15955:
-

+1

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-22 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877827#comment-15877827
 ] 

Hive QA commented on HIVE-15955:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853871/HIVE-15955.03.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10253 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_union] 
(batchId=145)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=223)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3688/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3688/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3688/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12853871 - PreCommit-HIVE-Build

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch, 
> HIVE-15955.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-21 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877304#comment-15877304
 ] 

Hive QA commented on HIVE-15955:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853824/HIVE-15955.02.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 111 failed/errored test(s), 10252 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input4] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join0] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parallel_join0] 
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[plan_json] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join3] 
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join4] 
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join6] 
(batchId=38)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2] 
(batchId=137)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[constprog_dpp]
 (batchId=141)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[constprog_semijoin]
 (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_5] 
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_mat_1] 
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_mat_2] 
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_mat_3] 
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_mat_4] 
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_mat_5] 
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[deleteAnalyze]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[empty_join] 
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
 (batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_4]
 (batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_cache] 
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_aggregate_without_gby]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
 (batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_timestamp]
 (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing_gby]
 (batchId=146)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1] 
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=94)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4] 
(batchId=94)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] 
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=93)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[multi_count_distinct]
 (batchId=93)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query12] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query13] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query15] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query16] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query17] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query18] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query19] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query1] (batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query20] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query21] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query22] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query25] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query26]

[jira] [Commented] (HIVE-15955) make explain formatted to include opId and etc

2017-02-21 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877127#comment-15877127
 ] 

Ashutosh Chauhan commented on HIVE-15955:
-

Latest patch looks good. Some minor comments on RB.

> make explain formatted to include opId and etc
> --
>
> Key: HIVE-15955
> URL: https://issues.apache.org/jira/browse/HIVE-15955
> Project: Hive
>  Issue Type: New Feature
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15955.01.patch, HIVE-15955.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)