[jira] [Updated] (HIVE-27541) Backport of HIVE-21924, HIVE-22769, HIVE-24224, HIVE-24381,HIVE-21905
[ https://issues.apache.org/jira/browse/HIVE-27541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj updated HIVE-27541: Summary: Backport of HIVE-21924, HIVE-22769, HIVE-24224, HIVE-24381,HIVE-21905 (was: Backport of HIVE-21924, HIVE-22769, HIVE-24224, HIVE-24381) > Backport of HIVE-21924, HIVE-22769, HIVE-24224, HIVE-24381,HIVE-21905 > - > > Key: HIVE-27541 > URL: https://issues.apache.org/jira/browse/HIVE-27541 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27761) Compilation of nested CTEs throws SemanticException
[
https://issues.apache.org/jira/browse/HIVE-27761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27761:
--
Labels: pull-request-available (was: )
> Compilation of nested CTEs throws SemanticException
> ---
>
> Key: HIVE-27761
> URL: https://issues.apache.org/jira/browse/HIVE-27761
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Soumyakanti Das
>Assignee: Soumyakanti Das
>Priority: Major
> Labels: pull-request-available
>
> Currently, nested CTEs are not supported in Hive. Simple repro:
> {code:java}
> with
> test1 as (
> with t1 as (select 1)
> select 1
> )
> select * from test1;
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 5:13 Ambiguous table
> alias 't1'
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processCTE(SemanticAnalyzer.java:1310)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1(SemanticAnalyzer.java:1980)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27653) Iceberg: Add conflictDetectionFilter to validate concurrently added data and delete files
[ https://issues.apache.org/jira/browse/HIVE-27653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27653: -- Labels: pull-request-available (was: ) > Iceberg: Add conflictDetectionFilter to validate concurrently added data and > delete files > - > > Key: HIVE-27653 > URL: https://issues.apache.org/jira/browse/HIVE-27653 > Project: Hive > Issue Type: Improvement >Reporter: Simhadri Govindappa >Assignee: Simhadri Govindappa >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27761) Compilation of nested CTEs throws SemanticException
[
https://issues.apache.org/jira/browse/HIVE-27761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Soumyakanti Das updated HIVE-27761:
---
Summary: Compilation of nested CTEs throws SemanticException (was:
SemanticException for Nested CTEs)
> Compilation of nested CTEs throws SemanticException
> ---
>
> Key: HIVE-27761
> URL: https://issues.apache.org/jira/browse/HIVE-27761
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Soumyakanti Das
>Assignee: Soumyakanti Das
>Priority: Major
>
> Currently, nested CTEs are not supported in Hive. Simple repro:
> {code:java}
> with
> test1 as (
> with t1 as (select 1)
> select 1
> )
> select * from test1;
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 5:13 Ambiguous table
> alias 't1'
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processCTE(SemanticAnalyzer.java:1310)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1(SemanticAnalyzer.java:1980)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27761) SemanticException for Nested CTEs
Soumyakanti Das created HIVE-27761:
--
Summary: SemanticException for Nested CTEs
Key: HIVE-27761
URL: https://issues.apache.org/jira/browse/HIVE-27761
Project: Hive
Issue Type: Bug
Components: Hive
Reporter: Soumyakanti Das
Assignee: Soumyakanti Das
Currently, nested CTEs are not supported in Hive. Simple repro:
{code:java}
with
test1 as (
with t1 as (select 1)
select 1
)
select * from test1;
org.apache.hadoop.hive.ql.parse.SemanticException: Line 5:13 Ambiguous table
alias 't1'
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processCTE(SemanticAnalyzer.java:1310)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.doPhase1(SemanticAnalyzer.java:1980)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27760) Filter on date type partitioning column producing 0 results
[
https://issues.apache.org/jira/browse/HIVE-27760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27760:
--
Labels: pull-request-available (was: )
> Filter on date type partitioning column producing 0 results
> ---
>
> Key: HIVE-27760
> URL: https://issues.apache.org/jira/browse/HIVE-27760
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
> Labels: pull-request-available
>
> Filter on date type partitioning columns producing 0 results.
> {*}Reproduction steps{*}:
> 1. test.q
> {noformat}
> CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate
> DATE) STORED AS ORC;
> INSERT into test(PartitionDate, a,b)
> VALUES('2023-01-01','2023-01-01','2023-01-01');
> INSERT into test(PartitionDate, a,b)
> VALUES('2023-01-02','2023-01-02','2023-01-02');
> select count(*) from test where PartitionDate = '2023-01-01';{noformat}
> 2. Command to execute (pass different timezone than server)
> {noformat}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q
> -Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
>
> *RootCause:* As a part of HIVE-27373 issue fix to parse the string to
> java.sql.Date object, java.text.SimpleDateFormat is replaced with
> java.time.format.DateTimeFormatter using java.time.LocalDate which represents
> a Date without TimeZone. Here this input is passed [here
> |https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
> which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and
> java.sql.Date. Here user timezone is passed different so actual value is
> getting changed to a different value (for example 2023-01-01 is changed to
> 2022-12-31) which is not matching with any partition so nothing gets returned.
> *Solution:* In MetaStoreDirectSql.java, we should use
> java.time.format.DateTimeFormatter with java.time.LocalDate so that it will
> return proper date string.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27649) Support ORDER BY clause in subqueries with set operators
[
https://issues.apache.org/jira/browse/HIVE-27649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HIVE-27649:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Support ORDER BY clause in subqueries with set operators
>
>
> Key: HIVE-27649
> URL: https://issues.apache.org/jira/browse/HIVE-27649
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Nicolas Richard
>Assignee: Nicolas Richard
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> Consider the following query:
> {code:java}
> select key from ((select key from src order by key) union (select key from
> src))subq {code}
> Up until 3.1.2, Hive would parse this query without any problems. However, if
> you try it on the latest versions, you'll get the following exception:
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:60 cannot recognize
> input near 'union' '(' 'select' in subquery source
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:97) {code}
> With the inner exception stack trace being:
> {code:java}
> NoViableAltException(367@[])
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:14006)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:45086)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:5411)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1921)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:2175)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:2110)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:2175)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1750)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1593)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45094)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:38538)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:38831)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:38424)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:37686)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:37574)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2757)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.explainStatement(HiveParser.java:1751)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1614)
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:123)
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:97)
> {code}
> Note that this behavior also happens if the subquery contains a SORT BY,
> CLUSTER BY, DISTRIBUTE BY or LIMIT clause.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27649) Support ORDER BY clause in subqueries with set operators
[
https://issues.apache.org/jira/browse/HIVE-27649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770488#comment-17770488
]
Ayush Saxena commented on HIVE-27649:
-
Committed to master.
Thanx [~nicolasrichard] for the contribution!!!
> Support ORDER BY clause in subqueries with set operators
>
>
> Key: HIVE-27649
> URL: https://issues.apache.org/jira/browse/HIVE-27649
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Nicolas Richard
>Assignee: Nicolas Richard
>Priority: Major
> Labels: pull-request-available
>
> Consider the following query:
> {code:java}
> select key from ((select key from src order by key) union (select key from
> src))subq {code}
> Up until 3.1.2, Hive would parse this query without any problems. However, if
> you try it on the latest versions, you'll get the following exception:
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:60 cannot recognize
> input near 'union' '(' 'select' in subquery source
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:97) {code}
> With the inner exception stack trace being:
> {code:java}
> NoViableAltException(367@[])
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:14006)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:45086)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:5411)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1921)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:2175)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:2110)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:2175)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1750)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1593)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45094)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:38538)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:38831)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:38424)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:37686)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:37574)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2757)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.explainStatement(HiveParser.java:1751)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1614)
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:123)
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:97)
> {code}
> Note that this behavior also happens if the subquery contains a SORT BY,
> CLUSTER BY, DISTRIBUTE BY or LIMIT clause.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27649) Support ORDER BY clause in subqueries with set operators
[
https://issues.apache.org/jira/browse/HIVE-27649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HIVE-27649:
Summary: Support ORDER BY clause in subqueries with set operators (was:
Subqueries with a set operator do not support order by clauses)
> Support ORDER BY clause in subqueries with set operators
>
>
> Key: HIVE-27649
> URL: https://issues.apache.org/jira/browse/HIVE-27649
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 3.1.2, 4.0.0
>Reporter: Nicolas Richard
>Assignee: Nicolas Richard
>Priority: Major
> Labels: pull-request-available
>
> Consider the following query:
> {code:java}
> select key from ((select key from src order by key) union (select key from
> src))subq {code}
> Up until 3.1.2, Hive would parse this query without any problems. However, if
> you try it on the latest versions, you'll get the following exception:
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:60 cannot recognize
> input near 'union' '(' 'select' in subquery source
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:97) {code}
> With the inner exception stack trace being:
> {code:java}
> NoViableAltException(367@[])
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:14006)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:45086)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:5411)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1921)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:2175)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:2110)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:2175)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1750)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1593)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45094)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:38538)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:38831)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:38424)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:37686)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:37574)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2757)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.explainStatement(HiveParser.java:1751)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1614)
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:123)
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:97)
> {code}
> Note that this behavior also happens if the subquery contains a SORT BY,
> CLUSTER BY, DISTRIBUTE BY or LIMIT clause.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27760) Filter on date type partitioning column producing 0 results
[
https://issues.apache.org/jira/browse/HIVE-27760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated HIVE-27760:
---
Description:
Filter on date type partitioning columns producing 0 results.
{*}Reproduction steps{*}:
1. test.q
{noformat}
CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate
DATE) STORED AS ORC;
INSERT into test(PartitionDate, a,b)
VALUES('2023-01-01','2023-01-01','2023-01-01');
INSERT into test(PartitionDate, a,b)
VALUES('2023-01-02','2023-01-02','2023-01-02');
select count(*) from test where PartitionDate = '2023-01-01';{noformat}
2. Command to execute (pass different timezone than server)
{noformat}
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q
-Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
*RootCause:* As a part of HIVE-27373 issue fix to parse the string to
java.sql.Date object, java.text.SimpleDateFormat is replaced with
java.time.format.DateTimeFormatter using java.time.LocalDate which represents a
Date without TimeZone. Here this input is passed [here
|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and
java.sql.Date. Here user timezone is passed different so actual value is
getting changed to a different value (for example 2023-01-01 is changed to
2022-12-31) which is not matching with any partition so nothing gets returned.
*Solution:* In MetaStoreDirectSql.java, we should use
java.time.format.DateTimeFormatter with java.time.LocalDate so that it will
return proper date string.
was:
Filter on date type partitioning columns producing 0 results.
{*}Reproduction steps{*}:
1. test.q
{noformat}
CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate
DATE) STORED AS ORC;
INSERT into test(PartitionDate, a,b)
VALUES('2023-01-01','2023-01-01','2023-01-01');
INSERT into test(PartitionDate, a,b)
VALUES('2023-01-02','2023-01-02','2023-01-02');
select count(*) from test where PartitionDate = '2023-01-01';{noformat}
2. Command to execute (pass different timezone than server)
{noformat}
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q
-Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
*RootCause:* As a part of [HIVE-27373|http://example.com/] issue fix to parse
the string to java.sql.Date object, java.text.SimpleDateFormat is replaced with
java.time.format.DateTimeFormatter using java.time.LocalDate which represents a
Date without TimeZone. Here this input is passed [here
|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and
java.sql.Date. Here user timezone is passed different so actual value is
getting changed to a different value (for example 2023-01-01 is changed to
2022-12-31) which is not matching with any partition so nothing gets returned.
*Solution:* In MetaStoreDirectSql.java, we should use
java.time.format.DateTimeFormatter with java.time.LocalDate so that it will
return proper date string.
> Filter on date type partitioning column producing 0 results
> ---
>
> Key: HIVE-27760
> URL: https://issues.apache.org/jira/browse/HIVE-27760
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
>
> Filter on date type partitioning columns producing 0 results.
> {*}Reproduction steps{*}:
> 1. test.q
> {noformat}
> CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate
> DATE) STORED AS ORC;
> INSERT into test(PartitionDate, a,b)
> VALUES('2023-01-01','2023-01-01','2023-01-01');
> INSERT into test(PartitionDate, a,b)
> VALUES('2023-01-02','2023-01-02','2023-01-02');
> select count(*) from test where PartitionDate = '2023-01-01';{noformat}
> 2. Command to execute (pass different timezone than server)
> {noformat}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q
> -Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
>
> *RootCause:* As a part of HIVE-27373 issue fix to parse the string to
> java.sql.Date object, java.text.SimpleDateFormat is replaced with
> java.time.format.DateTimeFormatter using java.time.LocalDate which represents
> a Date without TimeZone. Here this input is passed [here
> |https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
> which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and
> jav
[jira] [Updated] (HIVE-27663) implement array_position UDF in Hive
[
https://issues.apache.org/jira/browse/HIVE-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27663:
--
Labels: pull-request-available (was: )
> implement array_position UDF in Hive
>
>
> Key: HIVE-27663
> URL: https://issues.apache.org/jira/browse/HIVE-27663
> Project: Hive
> Issue Type: Sub-task
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Taraka Rama Rao Lethavadla
>Priority: Major
> Labels: pull-request-available
>
> array_position(array, element) - Returns the position of the first occurrence
> of element in array.
> Array indexing starts at 1. If the element value is NULL, a NULL is returned.
>
> Example:
> {noformat}
> > SELECT array_position(array(1, 2, 3,4,2), 2) FROM src;
> 2{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27760) Filter on date type partitioning column producing 0 results
Dayakar M created HIVE-27760:
Summary: Filter on date type partitioning column producing 0
results
Key: HIVE-27760
URL: https://issues.apache.org/jira/browse/HIVE-27760
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Dayakar M
Assignee: Dayakar M
Filter on date type partitioning columns producing 0 results.
{*}Reproduction steps{*}:
1. test.q
{noformat}
CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate
DATE) STORED AS ORC;
INSERT into test(PartitionDate, a,b)
VALUES('2023-01-01','2023-01-01','2023-01-01');
INSERT into test(PartitionDate, a,b)
VALUES('2023-01-02','2023-01-02','2023-01-02');
select count(*) from test where PartitionDate = '2023-01-01';{noformat}
2. Command to execute (pass different timezone than server)
{noformat}
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q
-Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
*RootCause:* As a part of [HIVE-27373|http://example.com/] issue fix to parse
the string to java.sql.Date object, java.text.SimpleDateFormat is replaced with
java.time.format.DateTimeFormatter using java.time.LocalDate which represents a
Date without TimeZone. Here this input is passed [here
|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and
java.sql.Date. Here user timezone is passed different so actual value is
getting changed to a different value (for example 2023-01-01 is changed to
2022-12-31) which is not matching with any partition so nothing gets returned.
*Solution:* In MetaStoreDirectSql.java, we should use
java.time.format.DateTimeFormatter with java.time.LocalDate so that it will
return proper date string.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27752) Remove DagUtils duplicate class
[ https://issues.apache.org/jira/browse/HIVE-27752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27752: Resolution: Fixed Status: Resolved (was: Patch Available) > Remove DagUtils duplicate class > --- > > Key: HIVE-27752 > URL: https://issues.apache.org/jira/browse/HIVE-27752 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: Akshat Mathur >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 4.0.0 > > > remove this small orphaned stuff: > https://github.com/apache/hive/blob/57c15936d7a69e215c986d62aa959e70cb352da4/ql/src/java/org/apache/hadoop/hive/ql/exec/DagUtils.java > and place method to > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27752) Remove DagUtils duplicate class
[ https://issues.apache.org/jira/browse/HIVE-27752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27752: Fix Version/s: 4.0.0 > Remove DagUtils duplicate class > --- > > Key: HIVE-27752 > URL: https://issues.apache.org/jira/browse/HIVE-27752 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: Akshat Mathur >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 4.0.0 > > > remove this small orphaned stuff: > https://github.com/apache/hive/blob/57c15936d7a69e215c986d62aa959e70cb352da4/ql/src/java/org/apache/hadoop/hive/ql/exec/DagUtils.java > and place method to > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27752) Remove DagUtils duplicate class
[ https://issues.apache.org/jira/browse/HIVE-27752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770337#comment-17770337 ] László Bodor commented on HIVE-27752: - merged to master, thanks [~akshatm] for the patch! > Remove DagUtils duplicate class > --- > > Key: HIVE-27752 > URL: https://issues.apache.org/jira/browse/HIVE-27752 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: Akshat Mathur >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 4.0.0 > > > remove this small orphaned stuff: > https://github.com/apache/hive/blob/57c15936d7a69e215c986d62aa959e70cb352da4/ql/src/java/org/apache/hadoop/hive/ql/exec/DagUtils.java > and place method to > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27716) Precommit: Save log files for first 10 failures
[
https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770323#comment-17770323
]
László Bodor commented on HIVE-27716:
-
after the patch, there are save files for the failed tests, like:
{code}
./split-01/itests/hive-unit/target/surefire-reports/org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez-output-save.txt
{code}
> Precommit: Save log files for first 10 failures
> ---
>
> Key: HIVE-27716
> URL: https://issues.apache.org/jira/browse/HIVE-27716
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: Screenshot 2023-09-21 at 11.49.16.png
>
>
> This is about to simply applying
> https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master
> [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile]
> current artifacts are simply useless, flaky tests are impossible to
> investigate (no, they don't reproduce in flaky-check, damn)
> looks like the job simply picks up Jenkinsfile from the root:
> http://ci.hive.apache.org/job/hive-precommit/configure
> !Screenshot 2023-09-21 at 11.49.16.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27759) Include docker daemon logs in case of docker issues
[
https://issues.apache.org/jira/browse/HIVE-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27759:
Description:
there is a test failure:
http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
{code}
docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
See 'docker run --help'.
{code}
the root cause of EOF is unknown, there might be further details somewhere
else, here is a github issue for reference (it's for mac but any ideas are
welcome): https://github.com/docker/for-mac/issues/6704
> Include docker daemon logs in case of docker issues
> ---
>
> Key: HIVE-27759
> URL: https://issues.apache.org/jira/browse/HIVE-27759
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> there is a test failure:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
> {code}
> docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
> See 'docker run --help'.
> {code}
> the root cause of EOF is unknown, there might be further details somewhere
> else, here is a github issue for reference (it's for mac but any ideas are
> welcome): https://github.com/docker/for-mac/issues/6704
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27758) Precommit: splits are messed up in the folders
[ https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27758: Summary: Precommit: splits are messed up in the folders (was: Splits are messed up in the folders) > Precommit: splits are messed up in the folders > -- > > Key: HIVE-27758 > URL: https://issues.apache.org/jira/browse/HIVE-27758 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > Attachments: Screenshot 2023-09-29 at 9.15.22.png > > > e.g. in the screenshot below, split-07 folder contains logs for another splits > !Screenshot 2023-09-29 at 9.15.22.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27759) Include docker daemon logs in case of docker issues
László Bodor created HIVE-27759: --- Summary: Include docker daemon logs in case of docker issues Key: HIVE-27759 URL: https://issues.apache.org/jira/browse/HIVE-27759 Project: Hive Issue Type: Sub-task Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27738) SchemaTool picks incorrect schema script after 4.0.0-beta-1 release
[ https://issues.apache.org/jira/browse/HIVE-27738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis resolved HIVE-27738. Fix Version/s: 4.0.0 Resolution: Fixed Fixed in https://github.com/apache/hive/commit/49132a32702131acd280466d093bbb9e34dba7dc. Thanks for the PR [~rkirtir] and [~wechar] for the review! > SchemaTool picks incorrect schema script after 4.0.0-beta-1 release > --- > > Key: HIVE-27738 > URL: https://issues.apache.org/jira/browse/HIVE-27738 > Project: Hive > Issue Type: Bug >Reporter: KIRTI RUGE >Assignee: KIRTI RUGE >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > hive.version.shortname needs to be fixed from / pom.xml and > standalone-metastore/pom.xml so that it should pick up xxx4.0.0-beta-2.xx.sql > file correctly -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27758) Splits are messed up in the folders
[ https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27758: Attachment: Screenshot 2023-09-29 at 9.15.22.png > Splits are messed up in the folders > --- > > Key: HIVE-27758 > URL: https://issues.apache.org/jira/browse/HIVE-27758 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > Attachments: Screenshot 2023-09-29 at 9.15.22.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27758) Splits are messed up in the folders
[ https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27758: Description: e.g. in !Screenshot 2023-09-29 at 9.15.22.png! split-07 folder contains logs > Splits are messed up in the folders > --- > > Key: HIVE-27758 > URL: https://issues.apache.org/jira/browse/HIVE-27758 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > Attachments: Screenshot 2023-09-29 at 9.15.22.png > > > e.g. in !Screenshot 2023-09-29 at 9.15.22.png! > split-07 folder contains logs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27758) Splits are messed up in the folders
[ https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27758: Description: e.g. in the screenshot below, split-07 folder contains logs for another splits !Screenshot 2023-09-29 at 9.15.22.png! was: e.g. in !Screenshot 2023-09-29 at 9.15.22.png! split-07 folder contains logs > Splits are messed up in the folders > --- > > Key: HIVE-27758 > URL: https://issues.apache.org/jira/browse/HIVE-27758 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > Attachments: Screenshot 2023-09-29 at 9.15.22.png > > > e.g. in the screenshot below, split-07 folder contains logs for another splits > !Screenshot 2023-09-29 at 9.15.22.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27738) SchemaTool picks incorrect schema script after 4.0.0-beta-1 release
[ https://issues.apache.org/jira/browse/HIVE-27738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-27738: --- Summary: SchemaTool picks incorrect schema script after 4.0.0-beta-1 release (was: Fix Schematool version so that it can pickup correct schema script file after 4.0.0-beta-1 release) > SchemaTool picks incorrect schema script after 4.0.0-beta-1 release > --- > > Key: HIVE-27738 > URL: https://issues.apache.org/jira/browse/HIVE-27738 > Project: Hive > Issue Type: Bug >Reporter: KIRTI RUGE >Assignee: KIRTI RUGE >Priority: Major > Labels: pull-request-available > > hive.version.shortname needs to be fixed from / pom.xml and > standalone-metastore/pom.xml so that it should pick up xxx4.0.0-beta-2.xx.sql > file correctly -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27758) Splits are messed up in the folders
László Bodor created HIVE-27758: --- Summary: Splits are messed up in the folders Key: HIVE-27758 URL: https://issues.apache.org/jira/browse/HIVE-27758 Project: Hive Issue Type: Sub-task Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.20.10#820010)
