[jira] [Commented] (HIVE-12391) SkewJoinOptimizer might not kick in if columns are renamed after TS
[
https://issues.apache.org/jira/browse/HIVE-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003023#comment-15003023
]
Hive QA commented on HIVE-12391:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12771975/HIVE-12391.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 9782 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6016/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6016/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6016/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12771975 - PreCommit-HIVE-TRUNK-Build
> SkewJoinOptimizer might not kick in if columns are renamed after TS
> ---
>
> Key: HIVE-12391
> URL: https://issues.apache.org/jira/browse/HIVE-12391
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12391.patch
>
>
> SkewJoinOptimizer will not kick in if the columns are just renamed after the
> TS e.g. by the creation of a derived table.
> To reproduce, consider the following example:
> {code}
> set hive.optimize.skewjoin.compiletime = true;
> CREATE TABLE T1(key STRING, val STRING)
> SKEWED BY (key) ON ((2)) STORED AS TEXTFILE;
> CREATE TABLE T2(key STRING, val STRING)
> SKEWED BY (key) ON ((3)) STORED AS TEXTFILE;
> {code}
> For this query, SkewJoinOptimizer kicks in:
> {code}
> SELECT a.*, b.*
> FROM T1 a JOIN T2 b
> ON a.key = b.key
> {code}
> For this one, it does not:
> {code}
> SELECT a.*, b.*
> FROM
> (SELECT key as k, val as v FROM T1) a
> JOIN
> (SELECT key as k, val as v FROM T2) b
> ON a.k = b.k;
> {code}
> The reason is that SkewJoinOptimizer does not backtrack the origin of the
> column. Instead it just uses its name to know if it is produced by a certain
> TS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12391) SkewJoinOptimizer might not kick in if columns are renamed after TS
[
https://issues.apache.org/jira/browse/HIVE-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002860#comment-15002860
]
Lefty Leverenz commented on HIVE-12391:
---
Excuse my ignorance, but what does TS mean? (Best guess: table scan.)
> SkewJoinOptimizer might not kick in if columns are renamed after TS
> ---
>
> Key: HIVE-12391
> URL: https://issues.apache.org/jira/browse/HIVE-12391
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12391.patch
>
>
> SkewJoinOptimizer will not kick in if the columns are just renamed after the
> TS e.g. by the creation of a derived table.
> To reproduce, consider the following example:
> {code}
> set hive.optimize.skewjoin.compiletime = true;
> CREATE TABLE T1(key STRING, val STRING)
> SKEWED BY (key) ON ((2)) STORED AS TEXTFILE;
> CREATE TABLE T2(key STRING, val STRING)
> SKEWED BY (key) ON ((3)) STORED AS TEXTFILE;
> {code}
> For this query, SkewJoinOptimizer kicks in:
> {code}
> SELECT a.*, b.*
> FROM T1 a JOIN T2 b
> ON a.key = b.key
> {code}
> For this one, it does not:
> {code}
> SELECT a.*, b.*
> FROM
> (SELECT key as k, val as v FROM T1) a
> JOIN
> (SELECT key as k, val as v FROM T2) b
> ON a.k = b.k;
> {code}
> The reason is that SkewJoinOptimizer does not backtrack the origin of the
> column. Instead it just uses its name to know if it is produced by a certain
> TS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12391) SkewJoinOptimizer might not kick in if columns are renamed after TS
[
https://issues.apache.org/jira/browse/HIVE-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002738#comment-15002738
]
Ashutosh Chauhan commented on HIVE-12391:
-
[~jcamachorodriguez] can you create a RB for this ?
> SkewJoinOptimizer might not kick in if columns are renamed after TS
> ---
>
> Key: HIVE-12391
> URL: https://issues.apache.org/jira/browse/HIVE-12391
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12391.patch
>
>
> SkewJoinOptimizer will not kick in if the columns are just renamed after the
> TS e.g. by the creation of a derived table.
> To reproduce, consider the following example:
> {code}
> set hive.optimize.skewjoin.compiletime = true;
> CREATE TABLE T1(key STRING, val STRING)
> SKEWED BY (key) ON ((2)) STORED AS TEXTFILE;
> CREATE TABLE T2(key STRING, val STRING)
> SKEWED BY (key) ON ((3)) STORED AS TEXTFILE;
> {code}
> For this query, SkewJoinOptimizer kicks in:
> {code}
> SELECT a.*, b.*
> FROM T1 a JOIN T2 b
> ON a.key = b.key
> {code}
> For this one, it does not:
> {code}
> SELECT a.*, b.*
> FROM
> (SELECT key as k, val as v FROM T1) a
> JOIN
> (SELECT key as k, val as v FROM T2) b
> ON a.k = b.k;
> {code}
> The reason is that SkewJoinOptimizer does not backtrack the origin of the
> column. Instead it just uses its name to know if it is produced by a certain
> TS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12391) SkewJoinOptimizer might not kick in if columns are renamed after TS
[
https://issues.apache.org/jira/browse/HIVE-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002760#comment-15002760
]
Ashutosh Chauhan commented on HIVE-12391:
-
nvm. LGTM +1
> SkewJoinOptimizer might not kick in if columns are renamed after TS
> ---
>
> Key: HIVE-12391
> URL: https://issues.apache.org/jira/browse/HIVE-12391
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 1.3.0, 2.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-12391.patch
>
>
> SkewJoinOptimizer will not kick in if the columns are just renamed after the
> TS e.g. by the creation of a derived table.
> To reproduce, consider the following example:
> {code}
> set hive.optimize.skewjoin.compiletime = true;
> CREATE TABLE T1(key STRING, val STRING)
> SKEWED BY (key) ON ((2)) STORED AS TEXTFILE;
> CREATE TABLE T2(key STRING, val STRING)
> SKEWED BY (key) ON ((3)) STORED AS TEXTFILE;
> {code}
> For this query, SkewJoinOptimizer kicks in:
> {code}
> SELECT a.*, b.*
> FROM T1 a JOIN T2 b
> ON a.key = b.key
> {code}
> For this one, it does not:
> {code}
> SELECT a.*, b.*
> FROM
> (SELECT key as k, val as v FROM T1) a
> JOIN
> (SELECT key as k, val as v FROM T2) b
> ON a.k = b.k;
> {code}
> The reason is that SkewJoinOptimizer does not backtrack the origin of the
> column. Instead it just uses its name to know if it is produced by a certain
> TS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
