[jira] [Created] (HIVE-25358) Remove reviewer pattern

[Jesus Camacho Rodriguez (Jira)]

Jesus Camacho Rodriguez created HIVE-25358:
--

 Summary: Remove reviewer pattern
 Key: HIVE-25358
 URL: https://issues.apache.org/jira/browse/HIVE-25358
 Project: Hive
  Issue Type: Sub-task
Reporter: Panagiotis Garefalakis
Assignee: Panagiotis Garefalakis
 Fix For: 4.0.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

[Owen O'Malley]

I think we should go ahead and release storage-api 2.8.0 and catch it on
the next cycle. HIVE-25190 is a long standing bug that rarely affects
users. (We have had a user at LinkedIn hit it, which is why I fixed it.)
I'll sign up to make the 2.8.1 (and 2.7.3) bug fix releases afterwards.

.. Owen

On Tue, Jul 20, 2021 at 8:53 PM Chao Sun  wrote:

> Going to check the release and vote here too. Since HIVE-25190 is already
> merged, instead of waiting for another release, should we start another RC1
> with that included?
>
> Chao
>
> On Tue, Jul 20, 2021 at 1:30 PM Dongjoon Hyun  wrote:
>
> > +1
> >
> > * Build and tested locally.
> >
> > Thanks,
> > Dongjoon.
> >
> > On 2021/07/19 23:15:46, "Owen O'Malley"  wrote:
> > > +1 (binding):
> > > * Built and tested
> > > * Built hive main branch using it
> > > * Verified signatures and checksums
> > >
> > > It is too bad that we didn't get HIVE-25190 into it, but that can wait
> > for
> > > 2.8.1.
> > >
> > > .. Owen
> > >
> > > On Mon, Jun 28, 2021 at 9:44 PM Pavan Lanka 
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > I have done the following:
> > > > * Built and Tested storage-release-2.8.0-rc0 using OpenJDK8
> > > > * Built and Tested ORC with updated storage api version
> > > >   - Had to fix a test class that implements PredicateLeaf which has a
> > new
> > > > method. This is a breaking change but I think this should be ok
> > > > * Verified the performance gains of HIVE-24458
> > > >
> > > > Regards,
> > > > Pavan
> > > >
> > > >
> > > > > On Jun 21, 2021, at 8:07 AM, Panos Garefalakis  >
> > > > wrote:
> > > > >
> > > > > Hello all,
> > > > >
> > > > > Following on previous discussions, I would like to propose a new
> > > > > storage-api release including HIVE-24458
> > > > > .
> > > > >
> > > > > Shall we release the following artifacts as Hive Storage API 2.8.0?
> > > > >
> > > > > tar: http://home.apache.org/~pgaref/hive-storage-2.8.0/
> > > > > tag:
> > > >
> https://github.com/apache/hive/releases/tag/storage-release-2.8.0-rc0
> > > > > jiras:
> > https://issues.apache.org/jira/projects/HIVE/versions/12350287
> > > > >
> > > > > Cheers,
> > > > > Panagiotis
> > > >
> > > >
> > >
> >
>

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

[Chao Sun]

Going to check the release and vote here too. Since HIVE-25190 is already
merged, instead of waiting for another release, should we start another RC1
with that included?

Chao

On Tue, Jul 20, 2021 at 1:30 PM Dongjoon Hyun  wrote:

> +1
>
> * Build and tested locally.
>
> Thanks,
> Dongjoon.
>
> On 2021/07/19 23:15:46, "Owen O'Malley"  wrote:
> > +1 (binding):
> > * Built and tested
> > * Built hive main branch using it
> > * Verified signatures and checksums
> >
> > It is too bad that we didn't get HIVE-25190 into it, but that can wait
> for
> > 2.8.1.
> >
> > .. Owen
> >
> > On Mon, Jun 28, 2021 at 9:44 PM Pavan Lanka 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > I have done the following:
> > > * Built and Tested storage-release-2.8.0-rc0 using OpenJDK8
> > > * Built and Tested ORC with updated storage api version
> > >   - Had to fix a test class that implements PredicateLeaf which has a
> new
> > > method. This is a breaking change but I think this should be ok
> > > * Verified the performance gains of HIVE-24458
> > >
> > > Regards,
> > > Pavan
> > >
> > >
> > > > On Jun 21, 2021, at 8:07 AM, Panos Garefalakis 
> > > wrote:
> > > >
> > > > Hello all,
> > > >
> > > > Following on previous discussions, I would like to propose a new
> > > > storage-api release including HIVE-24458
> > > > .
> > > >
> > > > Shall we release the following artifacts as Hive Storage API 2.8.0?
> > > >
> > > > tar: http://home.apache.org/~pgaref/hive-storage-2.8.0/
> > > > tag:
> > > https://github.com/apache/hive/releases/tag/storage-release-2.8.0-rc0
> > > > jiras:
> https://issues.apache.org/jira/projects/HIVE/versions/12350287
> > > >
> > > > Cheers,
> > > > Panagiotis
> > >
> > >
> >
>

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

[Dongjoon Hyun]

+1

* Build and tested locally.

Thanks,
Dongjoon.

On 2021/07/19 23:15:46, "Owen O'Malley"  wrote: 
> +1 (binding):
> * Built and tested
> * Built hive main branch using it
> * Verified signatures and checksums
> 
> It is too bad that we didn't get HIVE-25190 into it, but that can wait for
> 2.8.1.
> 
> .. Owen
> 
> On Mon, Jun 28, 2021 at 9:44 PM Pavan Lanka 
> wrote:
> 
> > +1 (non-binding)
> >
> > I have done the following:
> > * Built and Tested storage-release-2.8.0-rc0 using OpenJDK8
> > * Built and Tested ORC with updated storage api version
> >   - Had to fix a test class that implements PredicateLeaf which has a new
> > method. This is a breaking change but I think this should be ok
> > * Verified the performance gains of HIVE-24458
> >
> > Regards,
> > Pavan
> >
> >
> > > On Jun 21, 2021, at 8:07 AM, Panos Garefalakis 
> > wrote:
> > >
> > > Hello all,
> > >
> > > Following on previous discussions, I would like to propose a new
> > > storage-api release including HIVE-24458
> > > .
> > >
> > > Shall we release the following artifacts as Hive Storage API 2.8.0?
> > >
> > > tar: http://home.apache.org/~pgaref/hive-storage-2.8.0/
> > > tag:
> > https://github.com/apache/hive/releases/tag/storage-release-2.8.0-rc0
> > > jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
> > >
> > > Cheers,
> > > Panagiotis
> >
> >
> 

[jira] [Created] (HIVE-25357) Fix the checkstyle issue in HiveIcebergMetaHook which breaks the build

[Marta Kuczora (Jira)]

Marta Kuczora created HIVE-25357:


 Summary: Fix the checkstyle issue in HiveIcebergMetaHook which 
breaks the build
 Key: HIVE-25357
 URL: https://issues.apache.org/jira/browse/HIVE-25357
 Project: Hive
  Issue Type: Bug
Affects Versions: 4.0.0
Reporter: Marta Kuczora
Assignee: Marta Kuczora
 Fix For: 4.0.0


[ERROR] 
/home/jenkins/agent/workspace/hive-precommit_master/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java:221:3:
 Cyclomatic Complexity is 13 (max allowed is 12). [CyclomaticComplexity]

This issue probably came in with 
[this|https://github.com/apache/hive/commit/76c49b9df957c8c05b81a4016282c03648b728b9]
 commit 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-25356) JDBCSplitFilterAboveJoinRule's onMatch method throws exception because wrong rel is assigned to HiveJdbcConverter conv

[Soumyakanti Das (Jira)]

Soumyakanti Das created HIVE-25356:
--

 Summary: JDBCSplitFilterAboveJoinRule's onMatch method throws 
exception because wrong rel is assigned to HiveJdbcConverter conv
 Key: HIVE-25356
 URL: https://issues.apache.org/jira/browse/HIVE-25356
 Project: Hive
  Issue Type: Bug
Reporter: Soumyakanti Das
Assignee: Soumyakanti Das


{{In the line below, call.rel(0) doesn't return a HiveJdbcConverter.}}

final HiveJdbcConverter conv = call.rel(0);



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

RE: [EXTERNAL] Re: Move Date and Timestamp parsing from ResolverStyle.LENIENT to ResolverStyle.STRICT

[Sankar Hariappan]

+1
Thanks Ashish for the comparison!
I talked to few Hive users (HDInsight) and they supported returning NULL for 
invalid date/timestamp inputs instead of returning incorrect results or 
exception.
Can others pls share your thoughts?

Thanks,
Sankar

From: Ashish Sharma 
Sent: 20 July 2021 14:02
To: dev@hive.apache.org
Cc: sankar.hariap...@microsoft.com.invalid; sank...@apache.org; 
u...@hive.apache.org; David 
Subject: Re: [EXTERNAL] Re: Move Date and Timestamp parsing from 
ResolverStyle.LENIENT to ResolverStyle.STRICT

Hi all,

I also feel that adding more config doesn't make sense in this as we are 
tightening the date and timestamp format. We should decide upon a single 
solution even if it break the compatibility. Below the comparison of HIVE 1.2, 
HIVE 3.2, MYSQL, PostgreSQL, Oracle



Query

Hive 1.2

Hive 3.2

Mysql

PostgreSQL

ORACLE

select cast('2020-20-20' as date);

NULL

2021-08-20

NULL

date/time field value out of range: "2020-20-20"

not a valid month

select cast(null as date);

NULL

NULL

NULL

NULL

NULL

select cast('2020-02-31' as date);

2020-03-02

2020-03-02

NULL

date/time field value out of range: "2020-02-31"

date format picture ends before converting entire input string

select cast('2020/02/20' as date);

NULL

NULL

2020-02-20

2020-02-20

literal does not match format string

select cast('-00-00' as date);

NULL

0002-11-30

NULL

date/time field value out of range: "-00-00"

literal does not match format string


>From the comparison it is quite clear that date and timestamp formatting was 
>much tighter in older versions of HIVE. For most of the wrong date input NULL 
>was the standard response instead of Exception.

Also when I went through the code I found that. While doing the Vector 
implementation of some of the date related UDF like datediff etc. MySql was 
taken as the gold 
standard.
 So it make more sense that  we should comply with MySql as we already refer 
MySql as gold standard and returning NULL as result for wrong dates in cast is 
also 
documented

So I propose to make NULL as the standard response for all parsing errors.

Thanks
Ashish Sharma

On Tue, Jul 13, 2021 at 9:52 PM Stamatis Zampetakis 
mailto:zabe...@gmail.com>> wrote:
Hi all,

Thanks for pushing this forward Ashish!

Actually I am not in favor of creating a flag for this. Either we decide
consciously to break backward compatibility in the hope that we are
improving the expected results or we keep the current behavior.
Adding another flag means that we maintain and support two variants that
makes the problem of test coverage brought by David even worse.

I second David's idea to run some tests over some well adopted DBMS (MySQL,
Oracle, MSSQL, Postgres) to see what they return.
I think Ashish already did some tests over MySQL and MSSQL but personally I
would like to see some more (dates + engines) in order to express
a preference.
We shouldn't forget that since Hive is implemented in Java, having
functions that are inline with the Java APIs is not such a bad idea.
The last comment is slightly supportive of the current behavior.

I am including user@ list in the discussion since we should definitely
consider the feedback of people that are using Hive for real.

Best,
Stamatis

On Tue, Jul 13, 2021 at 4:31 PM David 
mailto:dam6...@gmail.com>> wrote:

> Hello,
>
> Is anyone able to try out a few different vendor RDBMS to see how they
> handle invalid dates, or provide links to documentation, both for invalid
> formatting and things like mm-dd-yyy 12-40-2021?
>
> Thanks.
>
> On Tue, Jul 13, 2021 at 5:14 AM Sankar Hariappan
> mailto:sankar.hariap...@microsoft.com.invalid>>
>  wrote:
>
>> I'm supporting this change to return "NULL" for invalid date/timestamp.
>> In the interest of backward compatibility, can we make all these changes
>> under a flag which can be enabled by default?
>>
>>
>> Thanks,
>> Sankar
>> -Original Message-
>> From: David mailto:dam6...@gmail.com>>
>> Sent: 10 July 2021 07:35

Re: Need write access to update Hive wiki

[Stamatis Zampetakis]

I think Ashutosh (in CC) is the right person to give you permissions to
edit the wiki.

Best,
Stamatis

On Mon, Jul 19, 2021 at 8:17 PM Nikhil Gupta  wrote:

> Hello,
>
> I need write access to LanguageManual+UDF
>  page
> to update the description of date_format​ function.
>
> Confluence Username: *gupta.nikhil0007*
>
> Associated Hive Jira:
> https://issues.apache.org/jira/browse/HIVE-25268
>
> Regards,
> Nikhil Gupta
>

[jira] [Created] (HIVE-25355) EXPLAIN statement for write transactions with hive.txn.readonly.enabled fails

[Pravin Sinha (Jira)]

Pravin Sinha created HIVE-25355:
---

 Summary: EXPLAIN statement for write transactions with 
hive.txn.readonly.enabled fails
 Key: HIVE-25355
 URL: https://issues.apache.org/jira/browse/HIVE-25355
 Project: Hive
  Issue Type: Bug
Reporter: Pravin Sinha
Assignee: Pravin Sinha






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [EXTERNAL] Re: Move Date and Timestamp parsing from ResolverStyle.LENIENT to ResolverStyle.STRICT

[Ashish Sharma]

Hi all,

I also feel that adding more config doesn't make sense in this as we are
tightening the date and timestamp format. We should decide upon a single
solution even if it break the compatibility. Below the comparison of HIVE
1.2, HIVE 3.2, MYSQL, PostgreSQL, Oracle


Query

Hive 1.2

Hive 3.2

Mysql

PostgreSQL

ORACLE

select cast('2020-20-20' as date);

NULL

2021-08-20

NULL

date/time field value out of range: "2020-20-20"

not a valid month

select cast(null as date);

NULL

NULL

NULL

NULL

NULL

select cast('2020-02-31' as date);

2020-03-02

2020-03-02

NULL

date/time field value out of range: "2020-02-31"

date format picture ends before converting entire input string

select cast('2020/02/20' as date);

NULL

NULL

2020-02-20

2020-02-20

literal does not match format string

select cast('-00-00' as date);

NULL

0002-11-30

NULL

date/time field value out of range: "-00-00"

literal does not match format string


>From the comparison it is quite clear that date and timestamp formatting
was much tighter in older versions of HIVE. For most of the wrong date
input *NULL *was the standard response instead of Exception.

Also when I went through the code I found that. While doing the Vector
implementation of some of the date related UDF like datediff etc. MySql was
taken as the gold standard
.
So it make more sense that  we should comply with MySql as we already refer
MySql as gold standard and returning NULL as result for wrong dates in cast
is also documented



*So I propose to make NULL as the standard response for all parsing errors.*

Thanks
Ashish Sharma

On Tue, Jul 13, 2021 at 9:52 PM Stamatis Zampetakis 
wrote:

> Hi all,
>
> Thanks for pushing this forward Ashish!
>
> Actually I am not in favor of creating a flag for this. Either we decide
> consciously to break backward compatibility in the hope that we are
> improving the expected results or we keep the current behavior.
> Adding another flag means that we maintain and support two variants that
> makes the problem of test coverage brought by David even worse.
>
> I second David's idea to run some tests over some well adopted DBMS (MySQL,
> Oracle, MSSQL, Postgres) to see what they return.
> I think Ashish already did some tests over MySQL and MSSQL but personally I
> would like to see some more (dates + engines) in order to express
> a preference.
> We shouldn't forget that since Hive is implemented in Java, having
> functions that are inline with the Java APIs is not such a bad idea.
> The last comment is slightly supportive of the current behavior.
>
> I am including user@ list in the discussion since we should definitely
> consider the feedback of people that are using Hive for real.
>
> Best,
> Stamatis
>
> On Tue, Jul 13, 2021 at 4:31 PM David  wrote:
>
> > Hello,
> >
> > Is anyone able to try out a few different vendor RDBMS to see how they
> > handle invalid dates, or provide links to documentation, both for invalid
> > formatting and things like mm-dd-yyy 12-40-2021?
> >
> > Thanks.
> >
> > On Tue, Jul 13, 2021 at 5:14 AM Sankar Hariappan
> >  wrote:
> >
> >> I'm supporting this change to return "NULL" for invalid date/timestamp.
> >> In the interest of backward compatibility, can we make all these changes
> >> under a flag which can be enabled by default?
> >>
> >>
> >> Thanks,
> >> Sankar
> >> -Original Message-
> >> From: David 
> >> Sent: 10 July 2021 07:35
> >> To: dev 
> >> Cc: sank...@apache.org; Stamatis Zampetakis 
> >> Subject: [EXTERNAL] Re: Move Date and Timestamp parsing from
> >> ResolverStyle.LENIENT to ResolverStyle.STRICT
> >>
> >> Hello,
> >>
> >> I too would be in favor of this. It drastically cuts down on the test
> >> matrix for Hive if we can clamp down on timestamp formats. With that
> being
> >> said, I've tried this and it's a big effort.  I put it down without
> getting
> >> consensus or buy-in or engagement on the effort. Please check out my
> work
> >> here:
> >>
> >>
> >>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fplugins%2Fservlet%2Fmobile%23issue%2FHIVE-24814data=04%7C01%7CSankar.Hariappan%40microsoft.com%7Cd47432b9d7654d66a46908d943472338%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637614795446338436%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=mvYgaG7liJOwUZmMvgwlo%2B1HvcUsrnzXA3Ltfz5yEYE%3Dreserved=0
> >>
> >>
> >> On Fri, Jul 9, 2021, 9:49 PM Ashish Sharma <
> ashishkumarsharm...@gmail.com
> >> >
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > When casting incorrect date or timestamp literals to DATE or TIMESTAMP
> >> > data type hive returns wrong values
> >> >
> >> > hive> select cast('2020-20-20' as date);
> 

[jira] [Created] (HIVE-25354) Handle unsupported queries for Iceberg tables

[Marton Bod (Jira)]

Marton Bod created HIVE-25354:
-

 Summary: Handle unsupported queries for Iceberg tables
 Key: HIVE-25354
 URL: https://issues.apache.org/jira/browse/HIVE-25354
 Project: Hive
  Issue Type: Improvement
Reporter: Marton Bod
Assignee: Marton Bod


In Iceberg, there will be several Hive commands that will be unsupported either 
temporarily or else. For example, right now all commands containing the 
PARTITION keyword would fail the Semantic Analysis given that the HMS table is 
always unpartitioned for Iceberg. The resulting error message for the user 
would be confusing. We should provide a common, unified error handling for 
these queries.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-25353) Incremental rebuild of partitioned insert only MV in presence of delete operations

[Krisztian Kasa (Jira)]

Krisztian Kasa created HIVE-25353:
-

 Summary: Incremental rebuild of partitioned insert only MV in 
presence of delete operations
 Key: HIVE-25353
 URL: https://issues.apache.org/jira/browse/HIVE-25353
 Project: Hive
  Issue Type: Improvement
  Components: CBO, Materialized views
Reporter: Krisztian Kasa
Assignee: Krisztian Kasa






--
This message was sent by Atlassian Jira
(v8.3.4#803005)