[jira] [Updated] (HIVE-27957) Better error message for STORED BY
[
https://issues.apache.org/jira/browse/HIVE-27957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27957:
--
Labels: pull-request-available (was: )
> Better error message for STORED BY
> --
>
> Key: HIVE-27957
> URL: https://issues.apache.org/jira/browse/HIVE-27957
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0-beta-1
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
>
> We'd like to see a little kinder message when an invalid identifier or
> literal is given. It is confusing especially when I mistakenly type STORED BY
> instead of STORED AS.
> {code:java}
> 0: jdbc:hive2://hive-hiveserver2:1/defaul> CREATE TABLE test (a STRING)
> STORED BY ORC TBLPROPERTIES ("orc.compress"="ZSTD");
> Error: Error while compiling statement: FAILED: SemanticException Cannot find
> class '' (state=42000,code=4){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27957) Better error message for STORED BY
okumin created HIVE-27957:
-
Summary: Better error message for STORED BY
Key: HIVE-27957
URL: https://issues.apache.org/jira/browse/HIVE-27957
Project: Hive
Issue Type: Improvement
Affects Versions: 4.0.0-beta-1
Reporter: okumin
Assignee: okumin
We'd like to see a little kinder message when an invalid identifier or literal
is given. It is confusing especially when I mistakenly type STORED BY instead
of STORED AS.
{code:java}
0: jdbc:hive2://hive-hiveserver2:1/defaul> CREATE TABLE test (a STRING)
STORED BY ORC TBLPROPERTIES ("orc.compress"="ZSTD");
Error: Error while compiling statement: FAILED: SemanticException Cannot find
class '' (state=42000,code=4){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-22416) MR-related operation logs missing when parallel execution is enabled
[
https://issues.apache.org/jira/browse/HIVE-22416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-22416:
--
Labels: pull-request-available (was: )
> MR-related operation logs missing when parallel execution is enabled
> -
>
> Key: HIVE-22416
> URL: https://issues.apache.org/jira/browse/HIVE-22416
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1
>
> Attachments: HIVE-22416.01.patch
>
>
> Repro steps:
> 1. Happy path, parallel execution disabled
> {code:java}
> 0: jdbc:hive2://localhost:1> set hive.exec.parallel=false;
> No rows affected (0.023 seconds)
> 0: jdbc:hive2://localhost:1> select count (*) from t1;
> INFO : Compiling
> command(queryId=karencoppage_20191028152610_a26c25e1-9834-446a-9a56-c676cb693e7d):
> select count (*) from t1
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=karencoppage_20191028152610_a26c25e1-9834-446a-9a56-c676cb693e7d);
> Time taken: 0.309 seconds
> INFO : Executing
> command(queryId=karencoppage_20191028152610_a26c25e1-9834-446a-9a56-c676cb693e7d):
> select count (*) from t1
> WARN :
> INFO : Query ID =
> karencoppage_20191028152610_a26c25e1-9834-446a-9a56-c676cb693e7d
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Number of reduce tasks determined at compile time: 1
> INFO : In order to change the average load for a reducer (in bytes):
> INFO : set hive.exec.reducers.bytes.per.reducer=
> INFO : In order to limit the maximum number of reducers:
> INFO : set hive.exec.reducers.max=
> INFO : In order to set a constant number of reducers:
> INFO : set mapreduce.job.reduces=
> DEBUG : Configuring job job_local495362389_0008 with
> file:/tmp/hadoop/mapred/staging/karencoppage495362389/.staging/job_local495362389_0008
> as the submit dir
> DEBUG : adding the following namenodes' delegation tokens:[file:///]
> DEBUG : Creating splits at
> file:/tmp/hadoop/mapred/staging/karencoppage495362389/.staging/job_local495362389_0008
> INFO : number of splits:0
> INFO : Submitting tokens for job: job_local495362389_0008
> INFO : Executing with tokens: []
> INFO : The url to track the job: http://localhost:8080/
> INFO : Job running in-process (local Hadoop)
> INFO : 2019-10-28 15:26:22,537 Stage-1 map = 0%, reduce = 100%
> INFO : Ended Job = job_local495362389_0008
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 SUCCESS
> INFO : Total MapReduce CPU Time Spent: 0 msec
> INFO : Completed executing
> command(queryId=karencoppage_20191028152610_a26c25e1-9834-446a-9a56-c676cb693e7d);
> Time taken: 6.497 seconds
> INFO : OK
> DEBUG : Shutting down query select count (*) from t1
> +-+
> | c0 |
> +-+
> | 0 |
> +-+
> 1 row selected (11.874 seconds)
> {code}
> 2. Faulty path, parallel execution enabled
> {code:java}
> 0: jdbc:hive2://localhost:1> set
> hive.server2.logging.operation.level=EXECUTION;
> No rows affected (0.236 seconds)
> 0: jdbc:hive2://localhost:1> set hive.exec.parallel=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1> select count (*) from t1;
> INFO : Compiling
> command(queryId=karencoppage_20191028155346_4e7b793b-654e-4d69-b588-f3f0d3ae0c77):
> select count (*) from t1
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=karencoppage_20191028155346_4e7b793b-654e-4d69-b588-f3f0d3ae0c77);
> Time taken: 4.707 seconds
> INFO : Executing
> command(queryId=karencoppage_20191028155346_4e7b793b-654e-4d69-b588-f3f0d3ae0c77):
> select count (*) from t1
> WARN :
> INFO : Query ID =
> karencoppage_20191028155346_4e7b793b-654e-4d69-b588-f3f0d3ae0c77
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in parallel
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 SUCCESS
> INFO : Total MapReduce CPU Time Spent: 0 msec
> INFO : Completed executing
> command(queryId=karencoppage_20191028155346_4e7b793b-654e-4d69-b588-f3f0d3ae0c77);
> Time taken: 44.577 seconds
> INFO : OK
> DEBUG : Shutting down query select count (*) from t1
> +-+
> | c0 |
> +-+
> | 0 |
> +-+
> 1 row selected (54.665 seconds)
> {code}
> The issue is that Log4J stores the session ID and que
[jira] [Updated] (HIVE-27956) Query based compactor implementation separation
[ https://issues.apache.org/jira/browse/HIVE-27956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27956: -- Labels: pull-request-available (was: ) > Query based compactor implementation separation > --- > > Key: HIVE-27956 > URL: https://issues.apache.org/jira/browse/HIVE-27956 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Minor > Labels: pull-request-available > > Currently all Query based compactors are based on the CompactionQueryBuilder > class, where the query generation for each implementation is mixed > altogether. This can lead to issues when changing the query as the changes > may affect multiple compactors. Query generation should moved inside the > query compactors, and this class should be a Utility/Helper class to provide > common features. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27948) Wrong results when using materialized views with non-deterministic/dynamic functions
[
https://issues.apache.org/jira/browse/HIVE-27948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17797157#comment-17797157
]
Krisztian Kasa commented on HIVE-27948:
---
The general rule is if there is any error message during materialized view
definition validation the MV can not be used by Calcite to perform query
rewrites but it can be used for text/AST based rewrite algorithm so MV creation
succeeds with a warning message.
As the description shows this logic can not be applied to any type of MVs, so
I submitted a patch to filter out MVs which doesn't support any type of rewrite
algorithm.
Thanks for reporting this bug.
> Wrong results when using materialized views with non-deterministic/dynamic
> functions
>
>
> Key: HIVE-27948
> URL: https://issues.apache.org/jira/browse/HIVE-27948
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0-beta-1
>Reporter: Stamatis Zampetakis
>Assignee: Krisztian Kasa
>Priority: Critical
> Labels: pull-request-available
> Attachments: materialized_view_unix_timestamp.q
>
>
> There are certain SQL functions that return different results across
> different executions. Usually we refer to these functions as
> non-deterministic or dynamic. Some examples are: UNIX_TIMESTAMP(),
> CURRENT_TIMESTAMP, CURRENT_DATE, etc.
> When a materialized view definition contains such functions the queries that
> are using this view may return wrong results.
> Consider the following scenario where we populate the employee table with
> timestamps representing the future. For making this easily reproable in
> self-contained test the timestamps are only a few seconds apart.
> {code:sql}
> CREATE TABLE EMPS (ENAME STRING, BIRTH_EPOCH_SECS INT) STORED AS ORC
> TBLPROPERTIES ('transactional'='true');
> INSERT INTO EMPS
> VALUES ('Victor', UNIX_TIMESTAMP()),
>('Alex', UNIX_TIMESTAMP() + 2),
>('Bob', UNIX_TIMESTAMP() + 5),
>('Alice', UNIX_TIMESTAMP() + 10);
> CREATE MATERIALIZED VIEW v_emp AS SELECT * FROM EMPS WHERE BIRTH_EPOCH_SECS
> <= UNIX_TIMESTAMP();
> {code}
> When the materialized view is created it is populated with only the rows that
> match the timestamp at the given time.
> To demonstrate the problem run the following queries with view based
> rewritting disabled and enabled.
> {code:sql}
> set hive.materializedview.rewriting.sql=false;
> SELECT * FROM EMPS WHERE BIRTH_EPOCH_SECS <= UNIX_TIMESTAMP();
> {code}
> {noformat}
> Victor1702302786
> Alex 1702302788
> Bob 1702302791
> {noformat}
> {code:sql}
> set hive.materializedview.rewriting.sql=true;
> SELECT * FROM EMPS WHERE BIRTH_EPOCH_SECS <= UNIX_TIMESTAMP();
> {code}
> {noformat}
> Victor1702302786
> Alex 1702302788
> {noformat}
> Naturally the second query should return more rows than the first one since
> UNIX_TIMESTAMP is constantly growing. However, when view based rewritting is
> in use the second query will use the results from the materialized view which
> are by now obsolete (missing Bob entry).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27948) Wrong results when using materialized views with non-deterministic/dynamic functions
[
https://issues.apache.org/jira/browse/HIVE-27948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27948:
--
Labels: pull-request-available (was: )
> Wrong results when using materialized views with non-deterministic/dynamic
> functions
>
>
> Key: HIVE-27948
> URL: https://issues.apache.org/jira/browse/HIVE-27948
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 4.0.0-beta-1
>Reporter: Stamatis Zampetakis
>Assignee: Krisztian Kasa
>Priority: Critical
> Labels: pull-request-available
> Attachments: materialized_view_unix_timestamp.q
>
>
> There are certain SQL functions that return different results across
> different executions. Usually we refer to these functions as
> non-deterministic or dynamic. Some examples are: UNIX_TIMESTAMP(),
> CURRENT_TIMESTAMP, CURRENT_DATE, etc.
> When a materialized view definition contains such functions the queries that
> are using this view may return wrong results.
> Consider the following scenario where we populate the employee table with
> timestamps representing the future. For making this easily reproable in
> self-contained test the timestamps are only a few seconds apart.
> {code:sql}
> CREATE TABLE EMPS (ENAME STRING, BIRTH_EPOCH_SECS INT) STORED AS ORC
> TBLPROPERTIES ('transactional'='true');
> INSERT INTO EMPS
> VALUES ('Victor', UNIX_TIMESTAMP()),
>('Alex', UNIX_TIMESTAMP() + 2),
>('Bob', UNIX_TIMESTAMP() + 5),
>('Alice', UNIX_TIMESTAMP() + 10);
> CREATE MATERIALIZED VIEW v_emp AS SELECT * FROM EMPS WHERE BIRTH_EPOCH_SECS
> <= UNIX_TIMESTAMP();
> {code}
> When the materialized view is created it is populated with only the rows that
> match the timestamp at the given time.
> To demonstrate the problem run the following queries with view based
> rewritting disabled and enabled.
> {code:sql}
> set hive.materializedview.rewriting.sql=false;
> SELECT * FROM EMPS WHERE BIRTH_EPOCH_SECS <= UNIX_TIMESTAMP();
> {code}
> {noformat}
> Victor1702302786
> Alex 1702302788
> Bob 1702302791
> {noformat}
> {code:sql}
> set hive.materializedview.rewriting.sql=true;
> SELECT * FROM EMPS WHERE BIRTH_EPOCH_SECS <= UNIX_TIMESTAMP();
> {code}
> {noformat}
> Victor1702302786
> Alex 1702302788
> {noformat}
> Naturally the second query should return more rows than the first one since
> UNIX_TIMESTAMP is constantly growing. However, when view based rewritting is
> in use the second query will use the results from the materialized view which
> are by now obsolete (missing Bob entry).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27939) Many UNION ALL throws SemanticException when trying to remove partition predicates: fail to find child from parent
[
https://issues.apache.org/jira/browse/HIVE-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27939:
--
Labels: pull-request-available (was: )
> Many UNION ALL throws SemanticException when trying to remove partition
> predicates: fail to find child from parent
> --
>
> Key: HIVE-27939
> URL: https://issues.apache.org/jira/browse/HIVE-27939
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 4.0.0-beta-1
>Reporter: Ryu Kobayashi
>Priority: Major
> Labels: pull-request-available
> Attachments: ddl.sql, query.sql
>
>
> I found that the ticket for HIVE-26779 alone does not resolve when using many
> UNION ALL. When we create a DDL with [^ddl.sql] and execute a query with
> [^query.sql], we get a SemanticException similar to HIVE-26779.
> {code:java}
> 23/12/07 18:02:01 ERROR ql.Driver: FAILED: SemanticException Exception when
> trying to remove partition predicates: fail to find child from parent
> org.apache.hadoop.hive.ql.parse.SemanticException: Exception when trying to
> remove partition predicates: fail to find child from parent
> at
> org.apache.hadoop.hive.ql.exec.Operator.removeChildAndAdoptItsChildren(Operator.java:809)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeUnionOperators(GenTezUtils.java:472)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:691)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:301)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.compilePlan(SemanticAnalyzer.java:13054)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13272)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12628)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
> at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:509)
> at
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:525)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:843)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:807)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:721)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:236){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work started] (HIVE-27939) Many UNION ALL throws SemanticException when trying to remove partition predicates: fail to find child from parent
[
https://issues.apache.org/jira/browse/HIVE-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-27939 started by Ryu Kobayashi.
> Many UNION ALL throws SemanticException when trying to remove partition
> predicates: fail to find child from parent
> --
>
> Key: HIVE-27939
> URL: https://issues.apache.org/jira/browse/HIVE-27939
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 4.0.0-beta-1
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
> Labels: pull-request-available
> Attachments: ddl.sql, query.sql
>
>
> I found that the ticket for HIVE-26779 alone does not resolve when using many
> UNION ALL. When we create a DDL with [^ddl.sql] and execute a query with
> [^query.sql], we get a SemanticException similar to HIVE-26779.
> {code:java}
> 23/12/07 18:02:01 ERROR ql.Driver: FAILED: SemanticException Exception when
> trying to remove partition predicates: fail to find child from parent
> org.apache.hadoop.hive.ql.parse.SemanticException: Exception when trying to
> remove partition predicates: fail to find child from parent
> at
> org.apache.hadoop.hive.ql.exec.Operator.removeChildAndAdoptItsChildren(Operator.java:809)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeUnionOperators(GenTezUtils.java:472)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:691)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:301)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.compilePlan(SemanticAnalyzer.java:13054)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13272)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12628)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
> at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:509)
> at
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:525)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:843)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:807)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:721)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:236){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HIVE-27939) Many UNION ALL throws SemanticException when trying to remove partition predicates: fail to find child from parent
[
https://issues.apache.org/jira/browse/HIVE-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi reassigned HIVE-27939:
Assignee: Ryu Kobayashi
> Many UNION ALL throws SemanticException when trying to remove partition
> predicates: fail to find child from parent
> --
>
> Key: HIVE-27939
> URL: https://issues.apache.org/jira/browse/HIVE-27939
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 4.0.0-beta-1
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Major
> Labels: pull-request-available
> Attachments: ddl.sql, query.sql
>
>
> I found that the ticket for HIVE-26779 alone does not resolve when using many
> UNION ALL. When we create a DDL with [^ddl.sql] and execute a query with
> [^query.sql], we get a SemanticException similar to HIVE-26779.
> {code:java}
> 23/12/07 18:02:01 ERROR ql.Driver: FAILED: SemanticException Exception when
> trying to remove partition predicates: fail to find child from parent
> org.apache.hadoop.hive.ql.parse.SemanticException: Exception when trying to
> remove partition predicates: fail to find child from parent
> at
> org.apache.hadoop.hive.ql.exec.Operator.removeChildAndAdoptItsChildren(Operator.java:809)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeUnionOperators(GenTezUtils.java:472)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:691)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:301)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.compilePlan(SemanticAnalyzer.java:13054)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13272)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12628)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
> at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:509)
> at
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:525)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:843)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:807)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:721)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:236){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
