[jira] [Commented] (HIVE-27939) Many UNION ALL throws SemanticException when trying to remove partition predicates: fail to find child from parent

2024-01-11 Thread Ryu Kobayashi (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17805436#comment-17805436
 ] 

Ryu Kobayashi commented on HIVE-27939:
--

I pushed new PR.

> Many UNION ALL throws SemanticException when trying to remove partition 
> predicates: fail to find child from parent
> --
>
> Key: HIVE-27939
> URL: https://issues.apache.org/jira/browse/HIVE-27939
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 4.0.0-beta-1
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
>  Labels: pull-request-available
> Attachments: ddl.sql, query.sql
>
>
> I found that the ticket for HIVE-26779 alone does not resolve when using many 
> UNION ALL. When we create a DDL with [^ddl.sql] and execute a query with 
> [^query.sql], we get a SemanticException similar to HIVE-26779.
> {code:java}
> 23/12/07 18:02:01 ERROR ql.Driver: FAILED: SemanticException Exception when 
> trying to remove partition predicates: fail to find child from parent
> org.apache.hadoop.hive.ql.parse.SemanticException: Exception when trying to 
> remove partition predicates: fail to find child from parent
>         at 
> org.apache.hadoop.hive.ql.exec.Operator.removeChildAndAdoptItsChildren(Operator.java:809)
>         at 
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeUnionOperators(GenTezUtils.java:472)
>         at 
> org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:691)
>         at 
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:301)
>         at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.compilePlan(SemanticAnalyzer.java:13054)
>         at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13272)
>         at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12628)
>         at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
>         at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
>         at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
>         at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
>         at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
>         at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
>         at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
>         at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:509)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:525)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:843)
>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:807)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:721)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:236){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HIVE-27939) Many UNION ALL throws SemanticException when trying to remove partition predicates: fail to find child from parent

2023-12-17 Thread Ryu Kobayashi (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17797910#comment-17797910
 ] 

Ryu Kobayashi commented on HIVE-27939:
--

With PR this problem no longer occurs, but OOM occurs. This issue is probably 
caused by https://issues.apache.org/jira/browse/HIVE-27858.

> Many UNION ALL throws SemanticException when trying to remove partition 
> predicates: fail to find child from parent
> --
>
> Key: HIVE-27939
> URL: https://issues.apache.org/jira/browse/HIVE-27939
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 4.0.0-beta-1
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
>  Labels: pull-request-available
> Attachments: ddl.sql, query.sql
>
>
> I found that the ticket for HIVE-26779 alone does not resolve when using many 
> UNION ALL. When we create a DDL with [^ddl.sql] and execute a query with 
> [^query.sql], we get a SemanticException similar to HIVE-26779.
> {code:java}
> 23/12/07 18:02:01 ERROR ql.Driver: FAILED: SemanticException Exception when 
> trying to remove partition predicates: fail to find child from parent
> org.apache.hadoop.hive.ql.parse.SemanticException: Exception when trying to 
> remove partition predicates: fail to find child from parent
>         at 
> org.apache.hadoop.hive.ql.exec.Operator.removeChildAndAdoptItsChildren(Operator.java:809)
>         at 
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeUnionOperators(GenTezUtils.java:472)
>         at 
> org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:691)
>         at 
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:301)
>         at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.compilePlan(SemanticAnalyzer.java:13054)
>         at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13272)
>         at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12628)
>         at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
>         at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
>         at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
>         at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
>         at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
>         at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
>         at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
>         at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:509)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:525)
>         at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:843)
>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:807)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:721)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:236){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)