[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839959#action_12839959 ] Mafish commented on HIVE-1202: -- @Zheng With my patch, hive will omit the outmost where clauses while joining. So it doesn't do partition pruning for this case, Which may be a problem in this situation. > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[
https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839940#action_12839940
]
Zheng Shao commented on HIVE-1202:
--
{code}
select * from
(
select name from classes
) a
join classes b
where a.date_partition = '2010-02-01' AND b.date_partition = '2010-03-01';
{code}
It seems with the patch, we won't do partition pruning for this case?
Is that a problem?
> "Unknown exception : null" while join
> -
>
> Key: HIVE-1202
> URL: https://issues.apache.org/jira/browse/HIVE-1202
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 0.4.1
> Environment: hive-0.4.1
> hadoop 0.19.1
>Reporter: Mafish
> Fix For: 0.4.1
>
> Attachments: HIVE-1202.branch-0.4.1.patch
>
>
> Hive throws "Unknown exception : null" with query:
> select * from
> (
> select name from classes
> ) a
> join classes b
> where a.name > b.number
> After tracing the code, I found this bug will occur with following
> conditions:
> 1. It is join operation.
> 2. At least one of the source of join is physical table (right side in
> above case).
> 3. With where condition and condition(s) of where clause must include
> columns from both side of join (a.name and b.number in case)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839606#action_12839606 ] Mafish commented on HIVE-1202: -- @Yongqiang We have a internal hive originate from hive-0.4.2rc2. So we are not able to using the latest hive release. This bug afftects 0.4.x only since hive-0.5.x uses a different pruning strategy. I'm wondering if hive-0.4.x still need bug fixing since hive-0.5.x has been released? > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839555#action_12839555 ] He Yongqiang commented on HIVE-1202: Please try http://svn.apache.org/repos/asf/hadoop/hive/trunk or you can download the latest stable 0.5 version from http://hadoop.apache.org/hive/releases.html#Download > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839552#action_12839552 ] Mafish commented on HIVE-1202: -- Which trunk are you using? I'm using release 0.4.1, which is checkoed out from http://svn.apache.org/repos/asf/hadoop/hive/branches/branch-0.4 $ svn info Path: . URL: http://svn.apache.org/repos/asf/hadoop/hive/branches/branch-0.4 Repository Root: http://svn.apache.org/repos/asf Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68 Revision: 916543 Node Kind: directory Schedule: normal Last Changed Author: nzhang Last Changed Rev: 912061 Last Changed Date: 2010-02-20 09:44:44 +0800 (Sat, 20 Feb 2010) > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839551#action_12839551 ] He Yongqiang commented on HIVE-1202: I tried this query with the trunk code. it works fine. hive> select a.name, b.* from (select name from classes) a join classes b on a.name = b.number where a.name>b.number; Total MapReduce jobs = 1 > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839547#action_12839547 ] Mafish commented on HIVE-1202: -- Error message in hive is as title: Unknown exception : null. And the call stack is the same as my first comment. > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[
https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839546#action_12839546
]
He Yongqiang commented on HIVE-1202:
{quote}
But when I changed one of them to sub-query, error occured again, as:
select a.name, b.* from (select name from classes) a join classes b on a.name =
b.number where a.name > b.number ;
{quote}
what's the error for this query?
> "Unknown exception : null" while join
> -
>
> Key: HIVE-1202
> URL: https://issues.apache.org/jira/browse/HIVE-1202
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 0.4.1
> Environment: hive-0.4.1
> hadoop 0.19.1
>Reporter: Mafish
> Fix For: 0.4.1
>
> Attachments: HIVE-1202.branch-0.4.1.patch
>
>
> Hive throws "Unknown exception : null" with query:
> select * from
> (
> select name from classes
> ) a
> join classes b
> where a.name > b.number
> After tracing the code, I found this bug will occur with following
> conditions:
> 1. It is join operation.
> 2. At least one of the source of join is physical table (right side in
> above case).
> 3. With where condition and condition(s) of where clause must include
> columns from both side of join (a.name and b.number in case)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839515#action_12839515 ] Mafish commented on HIVE-1202: -- @Yongqaing I ran the query: select a.name, b.* from classes a join classes b on a.name = b.number where a.name > b.number It passed. In this case, two tables are physical. But when I changed one of them to sub-query, error occured again, as: select a.name, b.* from (select name from classes) a join classes b on a.name = b.number where a.name > b.number ; Please try this case. > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838946#action_12838946 ] He Yongqiang commented on HIVE-1202: Actually hive does not support this kind of join. It only support equal join. please try sth like this: select a.name, b.* from classes a join classes b on a.name = b.number where a.name > b.number > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > Attachments: HIVE-1202.branch-0.4.1.patch > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1202) "Unknown exception : null" while join
[ https://issues.apache.org/jira/browse/HIVE-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838743#action_12838743 ] Mafish commented on HIVE-1202: -- The call stack is: org.apache.hadoop.hive.ql.session.SessionState$LogHelper.printError(SessionState.java:279) - FAILED: Unknown exception : null java.lang.NullPointerException at org.apache.hadoop.hive.ql.parse.QBMetaData.getTableForAlias(QBMetaData.java:76) at org.apache.hadoop.hive.ql.parse.ASTPartitionPruner.getTableColumnDesc(ASTPartitionPruner.java:298) at org.apache.hadoop.hive.ql.parse.ASTPartitionPruner.genExprNodeDesc(ASTPartitionPruner.java:220) at org.apache.hadoop.hive.ql.parse.ASTPartitionPruner.genExprNodeDesc(ASTPartitionPruner.java:234) at org.apache.hadoop.hive.ql.parse.ASTPartitionPruner.genExprNodeDesc(ASTPartitionPruner.java:234) at org.apache.hadoop.hive.ql.parse.ASTPartitionPruner.addExpression(ASTPartitionPruner.java:397) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPartitionPruners(SemanticAnalyzer.java:624) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:4440) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:76) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:249) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:281) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:165) at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68) This bug occurs while hive tries to prune table b, but it takes columns in where clauses. Bug there also exists columns of table a. Thus, hive fails to find column "name" in table b. > "Unknown exception : null" while join > - > > Key: HIVE-1202 > URL: https://issues.apache.org/jira/browse/HIVE-1202 > Project: Hadoop Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.4.1 > Environment: hive-0.4.1 > hadoop 0.19.1 >Reporter: Mafish > Fix For: 0.4.1 > > > Hive throws "Unknown exception : null" with query: > select * from > ( > select name from classes > ) a > join classes b > where a.name > b.number > After tracing the code, I found this bug will occur with following > conditions: > 1. It is join operation. > 2. At least one of the source of join is physical table (right side in > above case). > 3. With where condition and condition(s) of where clause must include > columns from both side of join (a.name and b.number in case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
