[jira] [Assigned] (HIVE-27768) Mask patterns in q test output to avoid flakiness

2023-10-03 Thread KIRTI RUGE (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

KIRTI RUGE reassigned HIVE-27768:
-

Assignee: KIRTI RUGE

> Mask patterns in q test output to avoid flakiness
> -
>
> Key: HIVE-27768
> URL: https://issues.apache.org/jira/browse/HIVE-27768
> Project: Hive
>  Issue Type: Improvement
>Reporter: KIRTI RUGE
>Assignee: KIRTI RUGE
>Priority: Major
>
> Please mask filesize pattern in below q tests
> Pattern :
> Found 3 items
> drwxr-xr-x - ### USER ### ### GROUP ### 0 ### HDFS DATE ### hdfs://### HDFS 
> PATH ###
> drwxr-xr-x - ### USER ### ### GROUP ### 0 ### HDFS DATE ### hdfs://### HDFS 
> PATH ###
> drwxr-xr-x - ### USER ### ### GROUP ### 0 ### HDFS DATE ### hdfs://### HDFS 
> PATH ###
>  
>  
> Tests
> cascade_dbdrop
> cttl
> orc_merge1
> orc_merge2
> orc_merge3
> orc_merge4
> orc_merge10
> autoColumnStats_6
> flatten_union_subdir
> acid_vectorization_original_tez
> temp_table_external
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (HIVE-26280) Copy more data into COMPLETED_COMPACTIONS for better supportability

2023-10-03 Thread KIRTI RUGE (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-26280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

KIRTI RUGE reassigned HIVE-26280:
-

Assignee: KIRTI RUGE  (was: Karen Coppage)

> Copy more data into COMPLETED_COMPACTIONS for better supportability
> ---
>
> Key: HIVE-26280
> URL: https://issues.apache.org/jira/browse/HIVE-26280
> Project: Hive
>  Issue Type: Improvement
>  Components: Transactions
>Reporter: Karen Coppage
>Assignee: KIRTI RUGE
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> There is some information in COMPACTION_QUEUE that doesn't get copied over to 
> COMPLETED_COMPACTIONS when compaction completes. It would help with 
> supportability if COMPLETED_COMPACTIONS (and especially the view of it in the 
> SYS database) also contained this information.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-27768) Mask patterns in q test output to avoid flakiness

2023-10-03 Thread KIRTI RUGE (Jira)
KIRTI RUGE created HIVE-27768:
-

 Summary: Mask patterns in q test output to avoid flakiness
 Key: HIVE-27768
 URL: https://issues.apache.org/jira/browse/HIVE-27768
 Project: Hive
  Issue Type: Improvement
Reporter: KIRTI RUGE


Please mask filesize pattern in below q tests

Pattern :

Found 3 items
drwxr-xr-x - ### USER ### ### GROUP ### 0 ### HDFS DATE ### hdfs://### HDFS 
PATH ###
drwxr-xr-x - ### USER ### ### GROUP ### 0 ### HDFS DATE ### hdfs://### HDFS 
PATH ###
drwxr-xr-x - ### USER ### ### GROUP ### 0 ### HDFS DATE ### hdfs://### HDFS 
PATH ###

 

 

Tests

cascade_dbdrop

cttl

orc_merge1

orc_merge2

orc_merge3

orc_merge4

orc_merge10

autoColumnStats_6

flatten_union_subdir

acid_vectorization_original_tez

temp_table_external

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27767) Copy more data into HIVE_LOCKS for better supportability

2023-10-03 Thread KIRTI RUGE (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

KIRTI RUGE updated HIVE-27767:
--
Summary: Copy more data into HIVE_LOCKS for better supportability  (was: 
Copy more data into COMPLETED_COMPACTIONS for better supportability)

> Copy more data into HIVE_LOCKS for better supportability
> 
>
> Key: HIVE-27767
> URL: https://issues.apache.org/jira/browse/HIVE-27767
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Reporter: KIRTI RUGE
>Priority: Major
>
> There is some information like ERROR_MESSAGE needs to copy to HIVE_LOCKS . It 
> would help with supportability if HIVE_LOCKS (and especially the view of it 
> in the SYS database) also contained this information.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-27767) Copy more data into COMPLETED_COMPACTIONS for better supportability

2023-10-03 Thread KIRTI RUGE (Jira)
KIRTI RUGE created HIVE-27767:
-

 Summary: Copy more data into COMPLETED_COMPACTIONS for better 
supportability
 Key: HIVE-27767
 URL: https://issues.apache.org/jira/browse/HIVE-27767
 Project: Hive
  Issue Type: Sub-task
  Components: Transactions
Reporter: KIRTI RUGE


There is some information like ERROR_MESSAGE needs to copy to HIVE_LOCKS . It 
would help with supportability if HIVE_LOCKS (and especially the view of it in 
the SYS database) also contained this information.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27760) Filter on date type partitioning column producing 0 results

2023-10-03 Thread Dayakar M (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dayakar M updated HIVE-27760:
-
Description: 
Filter on date type partitioning columns producing 0 results.

{*}Reproduction steps{*}:

1. test.q
{noformat}
CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate 
DATE) STORED AS ORC;
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-01','2023-01-01','2023-01-01');
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-02','2023-01-02','2023-01-02');

select count(*) from test where PartitionDate = '2023-01-01';{noformat}
2. Command to execute (pass different timezone than server)
{noformat}
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q 
-Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
 

*RootCause:* As a part of HIVE-27373 issue fix to parse the string to 
java.sql.Date object, java.text.SimpleDateFormat is replaced with 
java.time.format.DateTimeFormatter using java.time.LocalDate which represents a 
Date without TimeZone.  Here this input is passed [here 
|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
 which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and 
java.sql.Date. Here user timezone is passed different so actual value is 
getting changed to a different value (for example 2023-01-01 is changed to 
2022-12-31) which is not matching with any partition so nothing gets returned.

*Solution:*

1. In MetaStoreDirectSql.java, we should use java.time.format.DateTimeFormatter 
with java.time.LocalDate so that it will return proper date string.

        Or

2. Revert the code changes(only formatter and LocalDate) done for HIVE-27373 
and use SimpleDateFormat and java.sql.Date.

  was:
Filter on date type partitioning columns producing 0 results.

{*}Reproduction steps{*}:

1. test.q
{noformat}
CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate 
DATE) STORED AS ORC;
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-01','2023-01-01','2023-01-01');
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-02','2023-01-02','2023-01-02');

select count(*) from test where PartitionDate = '2023-01-01';{noformat}
2. Command to execute (pass different timezone than server)
{noformat}
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q 
-Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
 

*RootCause:* As a part of HIVE-27373 issue fix to parse the string to 
java.sql.Date object, java.text.SimpleDateFormat is replaced with 
java.time.format.DateTimeFormatter using java.time.LocalDate which represents a 
Date without TimeZone.  Here this input is passed [here 
|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
 which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and 
java.sql.Date. Here user timezone is passed different so actual value is 
getting changed to a different value (for example 2023-01-01 is changed to 
2022-12-31) which is not matching with any partition so nothing gets returned.

*Solution:*

1. In MetaStoreDirectSql.java, we should use java.time.format.DateTimeFormatter 
with java.time.LocalDate so that it will return proper date string.

2. Revert the code changes(only formatter and LocalDate) done for HIVE-27373 
and use SimpleDateFormat and java.sql.Date.


> Filter on date type partitioning column producing 0 results
> ---
>
> Key: HIVE-27760
> URL: https://issues.apache.org/jira/browse/HIVE-27760
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
>  Labels: pull-request-available
>
> Filter on date type partitioning columns producing 0 results.
> {*}Reproduction steps{*}:
> 1. test.q
> {noformat}
> CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate 
> DATE) STORED AS ORC;
> INSERT into test(PartitionDate, a,b) 
> VALUES('2023-01-01','2023-01-01','2023-01-01');
> INSERT into test(PartitionDate, a,b) 
> VALUES('2023-01-02','2023-01-02','2023-01-02');
> select count(*) from test where PartitionDate = '2023-01-01';{noformat}
> 2. Command to execute (pass different timezone than server)
> {noformat}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q 
> -Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
>  
> *RootCause:* As a part of HIVE-27373 issue fix to parse the string to 
> java.sql.Date object, java.text.SimpleDateFormat is replaced with 
> java.time.format.DateTimeFormatter using java.time.LocalDate which represents 
> a Date without TimeZone.  Here this input 

[jira] [Updated] (HIVE-27764) Authentication does not work behind Knox gateway because the "WWW-Authenticate: Negotiate" response header is missing

2023-10-03 Thread Gergely Farkas (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Farkas updated HIVE-27764:
--
Labels: pull-request-available  (was: )

> Authentication does not work behind Knox gateway because the 
> "WWW-Authenticate: Negotiate" response header is missing
> -
>
> Key: HIVE-27764
> URL: https://issues.apache.org/jira/browse/HIVE-27764
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Gergely Farkas
>Assignee: Gergely Farkas
>Priority: Major
>  Labels: pull-request-available
>
> HIVE-27661 refactored the authentication code a bit and also introduced a new 
> bug: 
> Before the modification, if the client was not authenticated and Kerberos 
> authentication was enabled, a response header was included in the HTTP 401 
> response: _WWW-Authenticate: Negotiate_  
> After the modification, this header is not included in the response, so [the 
> reactive authentication flow implemented in the Knox 
> gateway|https://github.com/apache/knox/blob/master/gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/auth/SpnegoAuthInterceptor.java#L105-L113]
>  does not work anymore.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27764) Authentication does not work behind Knox gateway because the "WWW-Authenticate: Negotiate" response header is missing

2023-10-03 Thread Gergely Farkas (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Farkas updated HIVE-27764:
--
Labels:   (was: pull)

> Authentication does not work behind Knox gateway because the 
> "WWW-Authenticate: Negotiate" response header is missing
> -
>
> Key: HIVE-27764
> URL: https://issues.apache.org/jira/browse/HIVE-27764
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Gergely Farkas
>Assignee: Gergely Farkas
>Priority: Major
>
> HIVE-27661 refactored the authentication code a bit and also introduced a new 
> bug: 
> Before the modification, if the client was not authenticated and Kerberos 
> authentication was enabled, a response header was included in the HTTP 401 
> response: _WWW-Authenticate: Negotiate_  
> After the modification, this header is not included in the response, so [the 
> reactive authentication flow implemented in the Knox 
> gateway|https://github.com/apache/knox/blob/master/gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/auth/SpnegoAuthInterceptor.java#L105-L113]
>  does not work anymore.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27764) Authentication does not work behind Knox gateway because the "WWW-Authenticate: Negotiate" response header is missing

2023-10-03 Thread Gergely Farkas (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Farkas updated HIVE-27764:
--
Labels: pull  (was: pull-request-available)

> Authentication does not work behind Knox gateway because the 
> "WWW-Authenticate: Negotiate" response header is missing
> -
>
> Key: HIVE-27764
> URL: https://issues.apache.org/jira/browse/HIVE-27764
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Gergely Farkas
>Assignee: Gergely Farkas
>Priority: Major
>  Labels: pull
>
> HIVE-27661 refactored the authentication code a bit and also introduced a new 
> bug: 
> Before the modification, if the client was not authenticated and Kerberos 
> authentication was enabled, a response header was included in the HTTP 401 
> response: _WWW-Authenticate: Negotiate_  
> After the modification, this header is not included in the response, so [the 
> reactive authentication flow implemented in the Knox 
> gateway|https://github.com/apache/knox/blob/master/gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/auth/SpnegoAuthInterceptor.java#L105-L113]
>  does not work anymore.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27764) Authentication does not work behind Knox gateway because the "WWW-Authenticate: Negotiate" response header is missing

2023-10-03 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-27764:
--
Labels: pull-request-available  (was: )

> Authentication does not work behind Knox gateway because the 
> "WWW-Authenticate: Negotiate" response header is missing
> -
>
> Key: HIVE-27764
> URL: https://issues.apache.org/jira/browse/HIVE-27764
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Gergely Farkas
>Assignee: Gergely Farkas
>Priority: Major
>  Labels: pull-request-available
>
> HIVE-27661 refactored the authentication code a bit and also introduced a new 
> bug: 
> Before the modification, if the client was not authenticated and Kerberos 
> authentication was enabled, a response header was included in the HTTP 401 
> response: _WWW-Authenticate: Negotiate_  
> After the modification, this header is not included in the response, so [the 
> reactive authentication flow implemented in the Knox 
> gateway|https://github.com/apache/knox/blob/master/gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/auth/SpnegoAuthInterceptor.java#L105-L113]
>  does not work anymore.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-27766) Does Alter Table/Partition Concatenate work on ORC file?

2023-10-03 Thread Jira
Zoltán Rátkai created HIVE-27766:


 Summary: Does Alter Table/Partition Concatenate work on ORC file?
 Key: HIVE-27766
 URL: https://issues.apache.org/jira/browse/HIVE-27766
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltán Rátkai


Regarding to this description:
https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl

for ORC files Alter Table/Partition Concatenate should do this:
"In Hive release 0.8.0 RCFile added support for fast block level merging of 
small RCFiles using concatenate command. In Hive release 0.14.0 ORC files added 
support fast stripe level merging of small ORC files using concatenate command."

But in orc_merge12.q.out it seems after the concatenation there are 2 stripes 
instead of one.

I think there should be only one stripe.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27759) Include docker daemon logs in case of docker issues

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771465#comment-17771465
 ] 

Zoltan Haindrich commented on HIVE-27759:
-

hmm.. yes - that used to report more nicely; I think an `EOFException` is kinda 
like a tcp reset happening... I wonder how frequently this is happening?

the other interesting thing is that it happened in 2 different splits; at 
different times:
split22
* 2023-09-28T12:35:05,587 
split7
* 2023-09-28T13:44:09,013 
* 2023-09-28T13:44:59,934  
* the last in s7 is even more odd:
{code}
2023-09-28T13:46:17,934  INFO [Listener at 0.0.0.0/44773] 
externalDB.AbstractExternalDB: Stderr from proc: Unable to find image 
'postgres:9.3' locally
docker: Error response from daemon: received unexpected HTTP status: 503 
Service Unavailable.
{code}


> Include docker daemon logs in case of docker issues
> ---
>
> Key: HIVE-27759
> URL: https://issues.apache.org/jira/browse/HIVE-27759
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> there is a test failure:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
> {code}
> docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
> See 'docker run --help'.
> {code}
> the root cause of EOF is unknown, there might be further details somewhere 
> else, here is a github issue for reference (it's for mac but any ideas are 
> welcome): https://github.com/docker/for-mac/issues/6704



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (HIVE-27759) Include docker daemon logs in case of docker issues

2023-10-03 Thread Jira


[ 
https://issues.apache.org/jira/browse/HIVE-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771459#comment-17771459
 ] 

László Bodor edited comment on HIVE-27759 at 10/3/23 12:18 PM:
---

I already deployed a harbor cache for Cloudera's internal precommit which is 
working well, the same thing might want to be addressed here also
what's really confusing and made me think about docker daemon logs is the EOF, 
I mean, in case of a quota problem, we receive a straightforward message, but 
this EOF is a different beast, and I was hoping to add some logging to 
discover: also I faced "port already in use" kind of error while starting an 
RDBMS docker container, where it was absolutely unexpected (there was no 
running docker containers using that port)...anyway, unfortunately, I'm not 
really deep inside this docker topic at the moment, we can try flaky-check now 
and the harbor cache eventually (anyone is interested, I can show to doc about 
the setup in Cloudera's precommit)


was (Author: abstractdog):
I already deployed a harbor cache for Cloudera's internal precommit which is 
working well, the same thing might want to be addressed here also
what's really confusing and made me think about docker daemon logs is the EOF, 
I mean, in case of a quota problem, we receive a straightforward message, but 
this EOF is a different beast, and I was hoping to add something to discover: 
also I faced "port already in use" kind of error while starting an RDBMS docker 
container, where it was absolutely unexpected (there was no running docker 
containers using that port)...anyway, unfortunately I'm not really deep inside 
this docker topic at the moment, we can try flaky-check now and the harbor 
cache eventually (anyone is interested, I can show to doc about the setup in 
Cloudera's precommit)

> Include docker daemon logs in case of docker issues
> ---
>
> Key: HIVE-27759
> URL: https://issues.apache.org/jira/browse/HIVE-27759
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> there is a test failure:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
> {code}
> docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
> See 'docker run --help'.
> {code}
> the root cause of EOF is unknown, there might be further details somewhere 
> else, here is a github issue for reference (it's for mac but any ideas are 
> welcome): https://github.com/docker/for-mac/issues/6704



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27759) Include docker daemon logs in case of docker issues

2023-10-03 Thread Jira


[ 
https://issues.apache.org/jira/browse/HIVE-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771459#comment-17771459
 ] 

László Bodor commented on HIVE-27759:
-

I already deployed a harbor cache for Cloudera's internal precommit which is 
working well, the same thing might want to be addressed here also
what's really confusing and made me think about docker daemon logs is the EOF, 
I mean, in case of a quota problem, we receive a straightforward message, but 
this EOF is a different beast, and I was hoping to add something to discover: 
also I faced "port already in use" kind of error while starting an RDBMS docker 
container, where it was absolutely unexpected (there was no running docker 
containers using that port)...anyway, unfortunately I'm not really deep inside 
this docker topic at the moment, we can try flaky-check now and the harbor 
cache eventually (anyone is interested, I can show to doc about the setup in 
Cloudera's precommit)

> Include docker daemon logs in case of docker issues
> ---
>
> Key: HIVE-27759
> URL: https://issues.apache.org/jira/browse/HIVE-27759
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> there is a test failure:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
> {code}
> docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
> See 'docker run --help'.
> {code}
> the root cause of EOF is unknown, there might be further details somewhere 
> else, here is a github issue for reference (it's for mac but any ideas are 
> welcome): https://github.com/docker/for-mac/issues/6704



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27759) Include docker daemon logs in case of docker issues

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771455#comment-17771455
 ] 

Zoltan Haindrich commented on HIVE-27759:
-

okay; not sure - but that used to be the case around a year ago - I don't think 
that was fixed :)

whatever causes this issue - if the images used for testing would be hosted 
inside the k8s cluster that would:
* reduce dependency on external service
* reduce external network usage
* and also speed up builds

that's why I think fixing this is not that interesting...


> Include docker daemon logs in case of docker issues
> ---
>
> Key: HIVE-27759
> URL: https://issues.apache.org/jira/browse/HIVE-27759
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> there is a test failure:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
> {code}
> docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
> See 'docker run --help'.
> {code}
> the root cause of EOF is unknown, there might be further details somewhere 
> else, here is a github issue for reference (it's for mac but any ideas are 
> welcome): https://github.com/docker/for-mac/issues/6704



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27758) Precommit: splits are messed up in the folders

2023-10-03 Thread Jira


[ 
https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771450#comment-17771450
 ] 

László Bodor commented on HIVE-27758:
-

okay, cool, thanks for clarifying!

> Precommit: splits are messed up in the folders
> --
>
> Key: HIVE-27758
> URL: https://issues.apache.org/jira/browse/HIVE-27758
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
> Attachments: Screenshot 2023-09-29 at 9.15.22.png
>
>
> e.g. in the screenshot below, split-07 folder contains logs for another 
> splits, maybe I'm getting something wrong
> !Screenshot 2023-09-29 at 9.15.22.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27765) Backport of HIVE-20062: Arrow serde should fill ArrowColumnVector(Decimal) with the given schema precision/scale

2023-10-03 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-27765:
--
Labels: pull-request-available  (was: )

> Backport of HIVE-20062: Arrow serde should fill ArrowColumnVector(Decimal) 
> with the given schema precision/scale
> 
>
> Key: HIVE-27765
> URL: https://issues.apache.org/jira/browse/HIVE-27765
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.2.0
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-27765) Backport of HIVE-20062: Arrow serde should fill ArrowColumnVector(Decimal) with the given schema precision/scale

2023-10-03 Thread Aman Raj (Jira)
Aman Raj created HIVE-27765:
---

 Summary: Backport of HIVE-20062: Arrow serde should fill 
ArrowColumnVector(Decimal) with the given schema precision/scale
 Key: HIVE-27765
 URL: https://issues.apache.org/jira/browse/HIVE-27765
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 3.2.0
Reporter: Aman Raj
Assignee: Aman Raj






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Work started] (HIVE-27764) Authentication does not work behind Knox gateway because the "WWW-Authenticate: Negotiate" response header is missing

2023-10-03 Thread Gergely Farkas (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-27764 started by Gergely Farkas.
-
> Authentication does not work behind Knox gateway because the 
> "WWW-Authenticate: Negotiate" response header is missing
> -
>
> Key: HIVE-27764
> URL: https://issues.apache.org/jira/browse/HIVE-27764
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Gergely Farkas
>Assignee: Gergely Farkas
>Priority: Major
>
> HIVE-27661 refactored the authentication code a bit and also introduced a new 
> bug: 
> Before the modification, if the client was not authenticated and Kerberos 
> authentication was enabled, a response header was included in the HTTP 401 
> response: _WWW-Authenticate: Negotiate_  
> After the modification, this header is not included in the response, so [the 
> reactive authentication flow implemented in the Knox 
> gateway|https://github.com/apache/knox/blob/master/gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/auth/SpnegoAuthInterceptor.java#L105-L113]
>  does not work anymore.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-27764) Authentication does not work behind Knox gateway because the "WWW-Authenticate: Negotiate" response header is missing

2023-10-03 Thread Gergely Farkas (Jira)
Gergely Farkas created HIVE-27764:
-

 Summary: Authentication does not work behind Knox gateway because 
the "WWW-Authenticate: Negotiate" response header is missing
 Key: HIVE-27764
 URL: https://issues.apache.org/jira/browse/HIVE-27764
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Reporter: Gergely Farkas
Assignee: Gergely Farkas


HIVE-27661 refactored the authentication code a bit and also introduced a new 
bug: 
Before the modification, if the client was not authenticated and Kerberos 
authentication was enabled, a response header was included in the HTTP 401 
response: _WWW-Authenticate: Negotiate_  
After the modification, this header is not included in the response, so [the 
reactive authentication flow implemented in the Knox 
gateway|https://github.com/apache/knox/blob/master/gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/auth/SpnegoAuthInterceptor.java#L105-L113]
 does not work anymore.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27759) Include docker daemon logs in case of docker issues

2023-10-03 Thread Jira


[ 
https://issues.apache.org/jira/browse/HIVE-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771419#comment-17771419
 ] 

Zoltán Rátkai commented on HIVE-27759:
--

[~kgyrtkirk] how do you know that? Can we confirm it?

> Include docker daemon logs in case of docker issues
> ---
>
> Key: HIVE-27759
> URL: https://issues.apache.org/jira/browse/HIVE-27759
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> there is a test failure:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
> {code}
> docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
> See 'docker run --help'.
> {code}
> the root cause of EOF is unknown, there might be further details somewhere 
> else, here is a github issue for reference (it's for mac but any ideas are 
> welcome): https://github.com/docker/for-mac/issues/6704



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27661) Auth mode inferred from the Authorization header

2023-10-03 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-27661:
--
Labels: pull-request-available  (was: )

> Auth mode inferred from the Authorization header
> 
>
> Key: HIVE-27661
> URL: https://issues.apache.org/jira/browse/HIVE-27661
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Gergely Farkas
>Assignee: Gergely Farkas
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
>
> In HIVE-27352 we added support for multiple authentication modes and this 
> change introduced the "auth" http header, 
> which broke compatibility with Impala in the following case: The impala-shell 
> client tool is able to connect to HS2 with the authentication mode specified 
> first in the auth mode list, but the other auth options do not work because 
> the impala-shell does not send an "auth" header to HS2. In a discussion with 
> impala devs, it turned out that impala does not need a similar header to 
> identify the authentication mode, because in case of http transport, the 
> content of the Authorization header can be used to infer the auth mode. This 
> improvement aims at avoiding the use of the "auth" header and thus allows us 
> to connect to HS2 via http protocol even if multiple authentication modes are 
> enabled and the client does not use the "auth" header (e.g. impala-shell or 
> older jdbc driver builds).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771402#comment-17771402
 ] 

Zoltan Haindrich commented on HIVE-27718:
-

don't do this in the main job either - extend the debug job instead

> TestMiniTezCliDriver: save application logs for failed tests
> 
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>  Labels: pull-request-available
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
> important to note that tez app logs files are not specific to a particular 
> test, so we can collect those for the whole module in case of an error



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27759) Include docker daemon logs in case of docker issues

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771400#comment-17771400
 ] 

Zoltan Haindrich commented on HIVE-27759:
-

the central repo is most likely hitting back because it is reaching download 
count limitations...

fix would be to use a local cache or separate registry during test executions



> Include docker daemon logs in case of docker issues
> ---
>
> Key: HIVE-27759
> URL: https://issues.apache.org/jira/browse/HIVE-27759
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> there is a test failure:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4753/2/tests/
> {code}
> docker: Error response from daemon: Get https://registry-1.docker.io/v2/: EOF.
> See 'docker run --help'.
> {code}
> the root cause of EOF is unknown, there might be further details somewhere 
> else, here is a github issue for reference (it's for mac but any ideas are 
> welcome): https://github.com/docker/for-mac/issues/6704



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27719) Save heapdump in case of OOM

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771399#comment-17771399
 ] 

Zoltan Haindrich commented on HIVE-27719:
-

building something like this into the main job is kinda like a reciepie for 
disaster:
* it could possibly take up all space on the executors
* by exhausting all space it will make all jobs fail - even innoecent ones

> Save heapdump in case of OOM
> 
>
> Key: HIVE-27719
> URL: https://issues.apache.org/jira/browse/HIVE-27719
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: Kokila N
>Priority: Major
>
> This applies to 2 places:
> 1. mini llap tests: 1 single JVM has everything (HS2, AM, tasks)
> 2. mini tez test: tez app containers



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (HIVE-27758) Precommit: splits are messed up in the folders

2023-10-03 Thread Zoltan Haindrich (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich resolved HIVE-27758.
-
Resolution: Not A Problem

> Precommit: splits are messed up in the folders
> --
>
> Key: HIVE-27758
> URL: https://issues.apache.org/jira/browse/HIVE-27758
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
> Attachments: Screenshot 2023-09-29 at 9.15.22.png
>
>
> e.g. in the screenshot below, split-07 folder contains logs for another 
> splits, maybe I'm getting something wrong
> !Screenshot 2023-09-29 at 9.15.22.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27719) Save heapdump in case of OOM

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771398#comment-17771398
 ] 

Zoltan Haindrich commented on HIVE-27719:
-

don't do this - only in the debug job which runs only 1 test

> Save heapdump in case of OOM
> 
>
> Key: HIVE-27719
> URL: https://issues.apache.org/jira/browse/HIVE-27719
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: Kokila N
>Priority: Major
>
> This applies to 2 places:
> 1. mini llap tests: 1 single JVM has everything (HS2, AM, tasks)
> 2. mini tez test: tez app containers



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27717) Improve precommit logging to address flaky tests easier

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771396#comment-17771396
 ] 

Zoltan Haindrich commented on HIVE-27717:
-

why would you want to do this? use the job which reruns the same test multiple 
times...

> Improve precommit logging to address flaky tests easier
> ---
>
> Key: HIVE-27717
> URL: https://issues.apache.org/jira/browse/HIVE-27717
> Project: Hive
>  Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (HIVE-27758) Precommit: splits are messed up in the folders

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771395#comment-17771395
 ] 

Zoltan Haindrich edited comment on HIVE-27758 at 10/3/23 10:07 AM:
---

that shouldn't matter much; other then that it might be confusing - because the 
same word is reused:
* the test case was split up into N parts
* meanwhile the executor used M splits

edit:
I think I've just rephrased what you were already saying :D 
> the folders "split-X" belong to the kubernetes pods and "hive.cli.splitY" 
> packages belong to the qsplit profile logic



was (Author: kgyrtkirk):
that shouldn't matter much; other then that it might be confusing - because the 
same word is reused:
* the test case was split up into N parts
* meanwhile the executor used M splits


> Precommit: splits are messed up in the folders
> --
>
> Key: HIVE-27758
> URL: https://issues.apache.org/jira/browse/HIVE-27758
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
> Attachments: Screenshot 2023-09-29 at 9.15.22.png
>
>
> e.g. in the screenshot below, split-07 folder contains logs for another 
> splits, maybe I'm getting something wrong
> !Screenshot 2023-09-29 at 9.15.22.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27758) Precommit: splits are messed up in the folders

2023-10-03 Thread Zoltan Haindrich (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771395#comment-17771395
 ] 

Zoltan Haindrich commented on HIVE-27758:
-

that shouldn't matter much; other then that it might be confusing - because the 
same word is reused:
* the test case was split up into N parts
* meanwhile the executor used M splits


> Precommit: splits are messed up in the folders
> --
>
> Key: HIVE-27758
> URL: https://issues.apache.org/jira/browse/HIVE-27758
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
> Attachments: Screenshot 2023-09-29 at 9.15.22.png
>
>
> e.g. in the screenshot below, split-07 folder contains logs for another 
> splits, maybe I'm getting something wrong
> !Screenshot 2023-09-29 at 9.15.22.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27760) Filter on date type partitioning column producing 0 results

2023-10-03 Thread Dayakar M (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-27760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771391#comment-17771391
 ] 

Dayakar M commented on HIVE-27760:
--

[GitHub Pull Request #4760|https://github.com/apache/hive/pull/4760] is for 1st 
solution.

[GitHub Pull Request #4766|https://github.com/apache/hive/pull/4766] is for 2nd 
solution.

> Filter on date type partitioning column producing 0 results
> ---
>
> Key: HIVE-27760
> URL: https://issues.apache.org/jira/browse/HIVE-27760
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
>  Labels: pull-request-available
>
> Filter on date type partitioning columns producing 0 results.
> {*}Reproduction steps{*}:
> 1. test.q
> {noformat}
> CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate 
> DATE) STORED AS ORC;
> INSERT into test(PartitionDate, a,b) 
> VALUES('2023-01-01','2023-01-01','2023-01-01');
> INSERT into test(PartitionDate, a,b) 
> VALUES('2023-01-02','2023-01-02','2023-01-02');
> select count(*) from test where PartitionDate = '2023-01-01';{noformat}
> 2. Command to execute (pass different timezone than server)
> {noformat}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q 
> -Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
>  
> *RootCause:* As a part of HIVE-27373 issue fix to parse the string to 
> java.sql.Date object, java.text.SimpleDateFormat is replaced with 
> java.time.format.DateTimeFormatter using java.time.LocalDate which represents 
> a Date without TimeZone.  Here this input is passed [here 
> |https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
>  which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and 
> java.sql.Date. Here user timezone is passed different so actual value is 
> getting changed to a different value (for example 2023-01-01 is changed to 
> 2022-12-31) which is not matching with any partition so nothing gets returned.
> *Solution:*
> 1. In MetaStoreDirectSql.java, we should use 
> java.time.format.DateTimeFormatter with java.time.LocalDate so that it will 
> return proper date string.
> 2. Revert the code changes(only formatter and LocalDate) done for HIVE-27373 
> and use SimpleDateFormat and java.sql.Date.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27760) Filter on date type partitioning column producing 0 results

2023-10-03 Thread Dayakar M (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dayakar M updated HIVE-27760:
-
Description: 
Filter on date type partitioning columns producing 0 results.

{*}Reproduction steps{*}:

1. test.q
{noformat}
CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate 
DATE) STORED AS ORC;
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-01','2023-01-01','2023-01-01');
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-02','2023-01-02','2023-01-02');

select count(*) from test where PartitionDate = '2023-01-01';{noformat}
2. Command to execute (pass different timezone than server)
{noformat}
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q 
-Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
 

*RootCause:* As a part of HIVE-27373 issue fix to parse the string to 
java.sql.Date object, java.text.SimpleDateFormat is replaced with 
java.time.format.DateTimeFormatter using java.time.LocalDate which represents a 
Date without TimeZone.  Here this input is passed [here 
|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
 which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and 
java.sql.Date. Here user timezone is passed different so actual value is 
getting changed to a different value (for example 2023-01-01 is changed to 
2022-12-31) which is not matching with any partition so nothing gets returned.

*Solution:*

1. In MetaStoreDirectSql.java, we should use java.time.format.DateTimeFormatter 
with java.time.LocalDate so that it will return proper date string.

2. Revert the code changes(only formatter and LocalDate) done for HIVE-27373 
and use SimpleDateFormat and java.sql.Date.

  was:
Filter on date type partitioning columns producing 0 results.

{*}Reproduction steps{*}:

1. test.q
{noformat}
CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate 
DATE) STORED AS ORC;
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-01','2023-01-01','2023-01-01');
INSERT into test(PartitionDate, a,b) 
VALUES('2023-01-02','2023-01-02','2023-01-02');

select count(*) from test where PartitionDate = '2023-01-01';{noformat}
2. Command to execute (pass different timezone than server)
{noformat}
mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q 
-Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
 

*RootCause:* As a part of HIVE-27373 issue fix to parse the string to 
java.sql.Date object, java.text.SimpleDateFormat is replaced with 
java.time.format.DateTimeFormatter using java.time.LocalDate which represents a 
Date without TimeZone.  Here this input is passed [here 
|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1370]
 which uses SimpleDateFormat(parsing dates in a locale-sensitive manner) and 
java.sql.Date. Here user timezone is passed different so actual value is 
getting changed to a different value (for example 2023-01-01 is changed to 
2022-12-31) which is not matching with any partition so nothing gets returned.

*Solution:* In MetaStoreDirectSql.java, we should use 
java.time.format.DateTimeFormatter with java.time.LocalDate so that it will 
return proper date string.

 


> Filter on date type partitioning column producing 0 results
> ---
>
> Key: HIVE-27760
> URL: https://issues.apache.org/jira/browse/HIVE-27760
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
>  Labels: pull-request-available
>
> Filter on date type partitioning columns producing 0 results.
> {*}Reproduction steps{*}:
> 1. test.q
> {noformat}
> CREATE EXTERNAL TABLE test(a string,b String) PARTITIONED BY(PartitionDate 
> DATE) STORED AS ORC;
> INSERT into test(PartitionDate, a,b) 
> VALUES('2023-01-01','2023-01-01','2023-01-01');
> INSERT into test(PartitionDate, a,b) 
> VALUES('2023-01-02','2023-01-02','2023-01-02');
> select count(*) from test where PartitionDate = '2023-01-01';{noformat}
> 2. Command to execute (pass different timezone than server)
> {noformat}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=test.q 
> -Dtest.output.overwrite=true -Duser.timezone=Asia/Hong_Kong{noformat}
>  
> *RootCause:* As a part of HIVE-27373 issue fix to parse the string to 
> java.sql.Date object, java.text.SimpleDateFormat is replaced with 
> java.time.format.DateTimeFormatter using java.time.LocalDate which represents 
> a Date without TimeZone.  Here this input is passed [here 
> 

[jira] [Assigned] (HIVE-27719) Save heapdump in case of OOM

2023-10-03 Thread Kokila N (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kokila N reassigned HIVE-27719:
---

Assignee: Kokila N

> Save heapdump in case of OOM
> 
>
> Key: HIVE-27719
> URL: https://issues.apache.org/jira/browse/HIVE-27719
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: Kokila N
>Priority: Major
>
> This applies to 2 places:
> 1. mini llap tests: 1 single JVM has everything (HS2, AM, tasks)
> 2. mini tez test: tez app containers



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (HIVE-27719) Save heapdump in case of OOM

2023-10-03 Thread Jira


 [ 
https://issues.apache.org/jira/browse/HIVE-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

László Bodor reassigned HIVE-27719:
---

Assignee: (was: László Bodor)

> Save heapdump in case of OOM
> 
>
> Key: HIVE-27719
> URL: https://issues.apache.org/jira/browse/HIVE-27719
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> This applies to 2 places:
> 1. mini llap tests: 1 single JVM has everything (HS2, AM, tasks)
> 2. mini tez test: tez app containers



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-27758) Precommit: splits are messed up in the folders

2023-10-03 Thread Jira


[ 
https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771373#comment-17771373
 ] 

László Bodor commented on HIVE-27758:
-

[~kgyrtkirk]: can you please clarify please if I'm getting something wrong? the 
folders "split-X" belong to the kubernetes pods and "hive.cli.splitY" packages 
belong to the qsplit profile logic, so maybe I misunderstood something...is 
this file listing expected?

> Precommit: splits are messed up in the folders
> --
>
> Key: HIVE-27758
> URL: https://issues.apache.org/jira/browse/HIVE-27758
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
> Attachments: Screenshot 2023-09-29 at 9.15.22.png
>
>
> e.g. in the screenshot below, split-07 folder contains logs for another 
> splits, maybe I'm getting something wrong
> !Screenshot 2023-09-29 at 9.15.22.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27758) Precommit: splits are messed up in the folders

2023-10-03 Thread Jira


 [ 
https://issues.apache.org/jira/browse/HIVE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

László Bodor updated HIVE-27758:

Description: 
e.g. in the screenshot below, split-07 folder contains logs for another splits, 
maybe I'm getting something wrong
!Screenshot 2023-09-29 at 9.15.22.png! 



  was:
e.g. in the screenshot below, split-07 folder contains logs for another splits
!Screenshot 2023-09-29 at 9.15.22.png! 




> Precommit: splits are messed up in the folders
> --
>
> Key: HIVE-27758
> URL: https://issues.apache.org/jira/browse/HIVE-27758
> Project: Hive
>  Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
> Attachments: Screenshot 2023-09-29 at 9.15.22.png
>
>
> e.g. in the screenshot below, split-07 folder contains logs for another 
> splits, maybe I'm getting something wrong
> !Screenshot 2023-09-29 at 9.15.22.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)