Re: [VOTE] Apache Hive 2.3.10 Release Candidate 1

[Dongjoon Hyun]

+1 (non-binding)

Thank you so much all!

Dongjoon.

On 2024/05/06 13:57:09 Cheng Pan wrote:
> +1 (non-binding)
> 
> Pass integration test with Apache Spark[1] and Apache Kyuubi[2].
> 
> [1] https://github.com/apache/spark/pull/45372
> [2] https://github.com/apache/kyuubi/pull/6328
> 
> Thanks,
> Cheng Pan
> 
> 
> 

Re: [VOTE] Apache Hive 2.3.10 Release Candidate 0

[Dongjoon Hyun]

Hi, Chao.

The vote email seems to have a wrong repository link, 1106. IIUC, 1128 is the 
valid one, isn't it?

Dongjoon.

On 2024/04/20 20:01:53 Chao Sun wrote:
> Apache Hive 2.3.10 Release Candidate 0 is available here:
> https://people.apache.org/~sunchao/apache-hive-2.3.10-rc-0/
> Maven artifacts are available here:
> https://repository.apache.org/content/repositories/orgapachehive-1128
> The
> tag release-2.3.10-rc0 has been applied to the source for this
> release in github, you can see it at
> https://github.com/apache/hive/tree/release-2.3.10-rc0
> Voting will conclude in 72 hours (or whenever I scrounge together enough
> votes).
> 
> Hive PMC Members: Please test and vote.
> 
> Thanks.
> 

[jira] [Created] (HIVE-26918) Upgrade jamon-runtime to 2.4.1

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-26918:


 Summary: Upgrade jamon-runtime to 2.4.1
 Key: HIVE-26918
 URL: https://issues.apache.org/jira/browse/HIVE-26918
 Project: Hive
  Issue Type: Bug
  Components: Web UI
Reporter: Dongjoon Hyun






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-26912) Publish SBOM artifacts

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-26912:


 Summary: Publish SBOM artifacts
 Key: HIVE-26912
 URL: https://issues.apache.org/jira/browse/HIVE-26912
 Project: Hive
  Issue Type: Improvement
  Components: Build Infrastructure
Affects Versions: 4.0.0
Reporter: Dongjoon Hyun






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-26260) Use `Reader.getSchema` instead of deprecated `Reader.getTypes`

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-26260:


 Summary: Use `Reader.getSchema` instead of deprecated 
`Reader.getTypes`
 Key: HIVE-26260
 URL: https://issues.apache.org/jira/browse/HIVE-26260
 Project: Hive
  Issue Type: Improvement
  Components: ORC
Affects Versions: 3.2.0
Reporter: Dongjoon Hyun






--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Re: [VOTE] Should we release Hive Storage API 2.8.1 rc1?

[Dongjoon Hyun]

Thank you for sharing. What is the JIRA ID for redact issue?

Dongjoon

On 2021/07/28 16:22:03, Owen O'Malley  wrote: 
> Ok, pavan found that hive 25190 broke the redact mask code in orc. Let me fix 
> that. 
> 
> .. Owen
> 
> > On Jul 26, 2021, at 14:18, Owen O'Malley  wrote:
> > 
> > 
> > Hello all,
> > 
> > I would like to propose a new storage-api release including HIVE-25386 and 
> > HIVE-25190.
> > 
> > Shall we release the following artifacts as Hive Storage API 2.8.1?
> > 
> > tar: http://home.apache.org/~omalley/hive-storage-2.8.1/
> > tag: https://github.com/apache/hive/releases/tag/storage-release-2.8.1-rc1
> > jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350456
> > 
> > .. Owen
> 

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

[Dongjoon Hyun]

Great!

Thank you, Owen and Panos!

Bests,
Dongjoon.

On 2021/07/26 16:45:38, "Owen O'Malley"  wrote: 
> Thanks, Dongjoon.
> 
> I've committed the fix to master and storage-branch-2.8. Given that I
> wanted to get HIVE-25190 released,
> I'll roll RCs for 2.8.1 and 2.7.3 today.
> 
> .. Owen
> 
> On Mon, Jul 26, 2021 at 8:36 AM Panos Garefalakis 
> wrote:
> 
> > Thanks Dongjoon for the details!
> > Looks like we are going to need a follow-up release sooner than later!
> >
> > On the positive side, HIVE-25386 <https://github.com/apache/hive/pull/2531
> > >
> > passed tests and we can now move forward.
> >
> > Cheers,
> > Panagiotis
> >
> > On Mon, Jul 26, 2021 at 9:56 AM Dongjoon Hyun  wrote:
> >
> > > Hi, All.
> > >
> > > It turns out that 2.8.0 has a regression in terms of the dependency
> > > although it passed all tests.
> > > At Apache Hive 2.2.0, HIVE-13906 removed the Guava compile dependency.
> > > However, recently, HIVE-24542 converts the test dependency to the compile
> > > dependency.
> > >
> > >
> > >
> > https://github.com/apache/hive/pull/1784/files#diff-e5e45b8481f3d87f4fa561d45b22fa20bb5084e5240d825717e6bde511b733baL122
> > >
> > > I noticed this breaking dependency change during Apache ORC integration
> > > stage.
> > >
> > > https://github.com/apache/orc/pull/773
> > >
> > > For now, I filed a JIRA issue and made a PR. Please comment your opinion
> > > on the Hive and ORC PRs.
> > > - HIVE-25386: hive-storage-api should not have guava compile dependency
> > > - https://github.com/apache/hive/pull/2531
> > >
> > > Bests,
> > > Dongjoon.
> > >
> > > On 2021/07/23 16:40:39, "Owen O'Malley"  wrote:
> > > > I can do the publishing. Thank you all!
> > > >
> > > > .. Owen
> > > >
> > > > On Fri, Jul 23, 2021 at 10:16 AM Panos Garefalakis  > >
> > > > wrote:
> > > >
> > > > > Thank you all for your help! With 3 binding votes this release
> > passed!
> > > > > I am going to send a vote result follow up but will need some PMC
> > help
> > > > > creating the new Storage API version (on Jira) and releasing (upload
> > > > > artifacts)!
> > > > >
> > > > > Cheers,
> > > > > Panagiotis
> > > > >
> > > > > On Fri, Jul 23, 2021 at 2:21 AM Chao Sun  wrote:
> > > > >
> > > > > > Thanks Owen! I just verified the checksum and gpg signature and
> > they
> > > both
> > > > > > look good, so +1 too.
> > > > > >
> > > > > > Panos: please fix the "Fix version" of the JIRA when you get a
> > > chance.
> > > > > > Thanks.
> > > > > >
> > > > > > Best,
> > > > > > Chao
> > > > > >
> > > > > >
> > > > > > On Thu, Jul 22, 2021 at 2:29 PM Owen O'Malley <
> > > owen.omal...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Chao,
> > > > > >>Panos key doesn't seem to have propagated to the Apache
> > servers.
> > > It
> > > > > >> referenced here:
> > > > > >>
> > > > > >> https://people.apache.org/keys/committer/ as "pgaref
> > > > > >> 7DFAB216AB7D96B3B2072184DC11DE4D00F8FA1D"
> > > > > >>
> > > > > >> The key itself can be found here:
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > >
> > https://keyserver.ubuntu.com/pks/lookup?search=pgaref=on=index
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Jul 21, 2021 at 8:47 PM Chao Sun 
> > > wrote:
> > > > > >>
> > > > > >> > I built the source from the branch and ran the tests, which all
> > > > > passed.
> > > > > >> > However I was not able to find the public GPG key. Panos: could
> > > you
> > > > > >> point
> > > > > >> > me to the location?
> > > > > >> >
> > > > > >> > Also seems we should create a new version 2.8.0 in the JIRA
> > page

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

[Dongjoon Hyun]

Hi, All.

It turns out that 2.8.0 has a regression in terms of the dependency although it 
passed all tests.
At Apache Hive 2.2.0, HIVE-13906 removed the Guava compile dependency.
However, recently, HIVE-24542 converts the test dependency to the compile 
dependency.

https://github.com/apache/hive/pull/1784/files#diff-e5e45b8481f3d87f4fa561d45b22fa20bb5084e5240d825717e6bde511b733baL122

I noticed this breaking dependency change during Apache ORC integration stage.

https://github.com/apache/orc/pull/773

For now, I filed a JIRA issue and made a PR. Please comment your opinion on the 
Hive and ORC PRs.
- HIVE-25386: hive-storage-api should not have guava compile dependency
- https://github.com/apache/hive/pull/2531

Bests,
Dongjoon.

On 2021/07/23 16:40:39, "Owen O'Malley"  wrote: 
> I can do the publishing. Thank you all!
> 
> .. Owen
> 
> On Fri, Jul 23, 2021 at 10:16 AM Panos Garefalakis 
> wrote:
> 
> > Thank you all for your help! With 3 binding votes this release passed!
> > I am going to send a vote result follow up but will need some PMC help
> > creating the new Storage API version (on Jira) and releasing (upload
> > artifacts)!
> >
> > Cheers,
> > Panagiotis
> >
> > On Fri, Jul 23, 2021 at 2:21 AM Chao Sun  wrote:
> >
> > > Thanks Owen! I just verified the checksum and gpg signature and they both
> > > look good, so +1 too.
> > >
> > > Panos: please fix the "Fix version" of the JIRA when you get a chance.
> > > Thanks.
> > >
> > > Best,
> > > Chao
> > >
> > >
> > > On Thu, Jul 22, 2021 at 2:29 PM Owen O'Malley 
> > > wrote:
> > >
> > >> Chao,
> > >>Panos key doesn't seem to have propagated to the Apache servers. It
> > >> referenced here:
> > >>
> > >> https://people.apache.org/keys/committer/ as "pgaref
> > >> 7DFAB216AB7D96B3B2072184DC11DE4D00F8FA1D"
> > >>
> > >> The key itself can be found here:
> > >>
> > >>
> > >>
> > https://keyserver.ubuntu.com/pks/lookup?search=pgaref=on=index
> > >>
> > >>
> > >> On Wed, Jul 21, 2021 at 8:47 PM Chao Sun  wrote:
> > >>
> > >> > I built the source from the branch and ran the tests, which all
> > passed.
> > >> > However I was not able to find the public GPG key. Panos: could you
> > >> point
> > >> > me to the location?
> > >> >
> > >> > Also seems we should create a new version 2.8.0 in the JIRA page:
> > >> >
> > >> >
> > >>
> > https://issues.apache.org/jira/projects/HIVE?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page
> > >> > and update "Fix version" of
> > >> > https://issues.apache.org/jira/browse/HIVE-24458
> > >> > .
> > >> >
> > >> > Chao
> > >> >
> > >> > On Wed, Jul 21, 2021 at 9:10 AM Szehon Ho 
> > >> wrote:
> > >> >
> > >> > > +1 (binding)
> > >> > >
> > >> > > * Built module
> > >> > > * Ran tests
> > >> > > * Checked artifact checksum and signature
> > >> > >
> > >> > > Thanks
> > >> > > Szehon
> > >> > >
> > >> > > On Tue, Jul 20, 2021 at 2:11 PM Owen O'Malley <
> > owen.omal...@gmail.com
> > >> >
> > >> > > wrote:
> > >> > >
> > >> > > > I think we should go ahead and release storage-api 2.8.0 and catch
> > >> it
> > >> > on
> > >> > > > the next cycle. HIVE-25190 is a long standing bug that rarely
> > >> affects
> > >> > > > users. (We have had a user at LinkedIn hit it, which is why I
> > fixed
> > >> > it.)
> > >> > > > I'll sign up to make the 2.8.1 (and 2.7.3) bug fix releases
> > >> afterwards.
> > >> > > >
> > >> > > > .. Owen
> > >> > > >
> > >> > > > On Tue, Jul 20, 2021 at 8:53 PM Chao Sun 
> > >> wrote:
> > >> > > >
> > >> > > > > Going to check the release and vote here too. Since HIVE-25190
> > is
> > >> > > already
> > >> > > > > merged, instead of waiting for another release, should we start
> > >> > another
> > >> 

[jira] [Created] (HIVE-25386) hive-storage-api should not have guava compile dependency

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-25386:


 Summary: hive-storage-api should not have guava compile dependency
 Key: HIVE-25386
 URL: https://issues.apache.org/jira/browse/HIVE-25386
 Project: Hive
  Issue Type: Bug
  Components: storage-api
Affects Versions: 4.0.0
Reporter: Dongjoon Hyun


https://mvnrepository.com/artifact/org.apache.hive/hive-storage-api/2.8.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-25384) Bump ORC to 1.6.9

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-25384:


 Summary: Bump ORC to 1.6.9
 Key: HIVE-25384
 URL: https://issues.apache.org/jira/browse/HIVE-25384
 Project: Hive
  Issue Type: Bug
Reporter: Dongjoon Hyun


ORC-804 affects ORC 1.6.0 ~ 1.6.8.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

[Dongjoon Hyun]

+1

* Build and tested locally.

Thanks,
Dongjoon.

On 2021/07/19 23:15:46, "Owen O'Malley"  wrote: 
> +1 (binding):
> * Built and tested
> * Built hive main branch using it
> * Verified signatures and checksums
> 
> It is too bad that we didn't get HIVE-25190 into it, but that can wait for
> 2.8.1.
> 
> .. Owen
> 
> On Mon, Jun 28, 2021 at 9:44 PM Pavan Lanka 
> wrote:
> 
> > +1 (non-binding)
> >
> > I have done the following:
> > * Built and Tested storage-release-2.8.0-rc0 using OpenJDK8
> > * Built and Tested ORC with updated storage api version
> >   - Had to fix a test class that implements PredicateLeaf which has a new
> > method. This is a breaking change but I think this should be ok
> > * Verified the performance gains of HIVE-24458
> >
> > Regards,
> > Pavan
> >
> >
> > > On Jun 21, 2021, at 8:07 AM, Panos Garefalakis 
> > wrote:
> > >
> > > Hello all,
> > >
> > > Following on previous discussions, I would like to propose a new
> > > storage-api release including HIVE-24458
> > > .
> > >
> > > Shall we release the following artifacts as Hive Storage API 2.8.0?
> > >
> > > tar: http://home.apache.org/~pgaref/hive-storage-2.8.0/
> > > tag:
> > https://github.com/apache/hive/releases/tag/storage-release-2.8.0-rc0
> > > jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
> > >
> > > Cheers,
> > > Panagiotis
> >
> >
> 

Re: [VOTE] Apache Hive 2.3.9 Release Candidate 0

[Dongjoon Hyun]

Thank you, Chao.

+1 according to the test result in Apache Spark.

https://github.com/apache/spark/pull/32750

Bests,
Dongjoon.

On 2021/06/02 01:02:16, Chao Sun  wrote: 
> Apache Hive 2.3.9 Release Candidate 0 is available here:
> https://people.apache.org/~sunchao/apache-hive-2.3.9-rc-0/
> Maven artifacts are available here:
> https://repository.apache.org/content/repositories/orgapachehive-1106/
> The tag release-2.3.9-rc0 has been applied to the source for this
> release in github, you can see it at
> https://github.com/apache/hive/tree/release-2.3.9-rc0
> Voting will conclude in 72 hours (or whenever I scrounge together enough
> votes).
> 
> Hive PMC Members: Please test and vote.
> 
> Thanks.
> 

Re: Upgrade ORC to 1.6

[Dongjoon Hyun]

Thank you, Panagiotis and Laszio.

This is a big move forward in both Hive and Hive project.

ORC community will start to release 1.6.7 next week.

Bests,
Dongjoon.

On 2021/01/15 14:03:13, Panos Garefalakis  wrote: 
> Hey Laszlo,
> 
> Thanks for taking a look!
> Yes, releasing ORC-1.6.7 is definitely the way to go (snapshot was only
> used for testing) and I am planning to start a discussion on the ORC list
> soon.
> 
> However, I wanted to get feedback from the Hive community first and address
> any potential concerns for this (significant) change -- so please share
> your thoughts on the PR :)
> 
> Cheers,
> Panagiotis
> 
> On Fri, Jan 15, 2021 at 2:52 PM László Bodor 
> wrote:
> 
> > Hi!
> >
> > Thanks, Panos, that's very cool!
> > I guess we won't depend on a snapshot, is there a chance that ORC
> > community can fix issues in the near future and release 1.6.7 officially to
> > depend upon?
> >
> > Regards,
> > Laszlo Bodor
> >
> >
> > Panos Garefalakis  ezt írta (időpont: 2021. jan. 14.,
> > Cs, 18:33):
> >
> >> Hello Hive team,
> >>
> >> I am happy to announce that as of today Apache HIve precommit tests with
> >> ORC-1.6.7 (snapshot) are passing on master branch!
> >> There were more than a few compatibility and bug fixes required to make
> >> the
> >> jump from 1.5 but the version upgrade actually enables (among other
> >> things):
> >>
> >>- Column-level encryption (ORC-14)
> >>- Timestamps with local timezone (ORC-189)
> >>- Zstd support (ORC-636)
> >>- String stats min/max value trimming (ORC-203)
> >>- Timestamp predicates with nanosecond precision (ORC-611)
> >>
> >> Please feel free to share your feedback as part of HIVE-23553
> >> 
> >> Of course, we still maintain a lot of duplicate logic as part of the LLAP
> >> engine that can be optimized down the line -- planning to add a few more
> >> subtasks there.
> >>
> >> Cheers,
> >> Panagiotis
> >>
> >
> 

Re: [VOTE] Apache Hive 2.3.8 Release Candidate 3

[Dongjoon Hyun]

+1

Thank you, Chao and all.

Bests,
Dongjoon.

On 2021/01/12 16:01:06, Szehon Ho  wrote: 
> +1
> 
> Verified signatures
> Created a table, insert data, ran simple queries.
> 
> Thanks Chao,
> Szehon
> 
> On Mon, Jan 11, 2021 at 5:43 AM Xuefu Zhang  wrote:
> 
> > +1. I performed the following:
> >
> > 1. Downloaded bin and src tarball and verified signature and checksum
> > 2. Freshly initiated a mysql based metastore and ran Hive metastore service
> > on top of it.
> > 3. Created a simple table and queried it with Hive CLI.
> > 4. Didn't test beeline, however.
> >
> > Thanks,
> > Xuefu
> >
> > On Thu, Jan 7, 2021 at 11:25 PM Chao Sun  wrote:
> >
> > > Apache Hive 2.3.8 Release Candidate 3 is available here:
> > > https://people.apache.org/~sunchao/apache-hive-2.3.8-rc-3
> > > Maven artifacts are available here:
> > > https://repository.apache.org/content/repositories/orgapachehive-1105
> > > The tag release-2.3.8-rc3 has been applied to the source for this
> > > release in github, you can see it at
> > > https://github.com/apache/hive/tree/release-2.3.8-rc3
> > > Voting will conclude in 72 hours (or whenever I scrounge together enough
> > > votes).
> > >
> > > Hive PMC Members: Please test and vote.
> > >
> > > Thanks.
> > >
> >
> 

Re: [VOTE] Apache Hive 2.3.8 Release Candidate 2

[Dongjoon Hyun]

+1

Thank you so much, Chao.

Bests,
Dongjoon.

On 2020/12/14 19:02:10, Chao Sun  wrote: 
> Apache Hive 2.3.8 Release Candidate 2 is available here:
> https://people.apache.org/~sunchao/apache-hive-2.3.8-rc-2/
> Maven artifacts are available here:
> https://repository.apache.org/content/repositories/orgapachehive-1104
> The tag release-2.3.8-rc2 has been applied to the source for this
> release in github, you can see it at
> https://github.com/apache/hive/tree/release-2.3.8-rc2
> Voting will conclude in 72 hours (or whenever I scrounge together enough
> votes).
> 
> Hive PMC Members: Please test and vote.
> 
> Thanks.
> 

[jira] [Created] (HIVE-24536) Upgrade ORC to 1.6.6

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-24536:


 Summary: Upgrade ORC to 1.6.6
 Key: HIVE-24536
 URL: https://issues.apache.org/jira/browse/HIVE-24536
 Project: Hive
  Issue Type: New Feature
  Components: ORC
Affects Versions: 4.0.0
Reporter: Dongjoon Hyun






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-24316) Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-24316:


 Summary: Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1
 Key: HIVE-24316
 URL: https://issues.apache.org/jira/browse/HIVE-24316
 Project: Hive
  Issue Type: Bug
  Components: ORC
Affects Versions: 3.1.3
Reporter: Dongjoon Hyun






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-24222) Upgrade ORC to 1.5.12

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-24222:


 Summary: Upgrade ORC to 1.5.12
 Key: HIVE-24222
 URL: https://issues.apache.org/jira/browse/HIVE-24222
 Project: Hive
  Issue Type: Bug
  Components: ORC
Affects Versions: 4.0.0
Reporter: Dongjoon Hyun






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-24030) Upgrade ORC to 1.5.10

[Dongjoon Hyun (Jira)]

Dongjoon Hyun created HIVE-24030:


 Summary: Upgrade ORC to 1.5.10
 Key: HIVE-24030
 URL: https://issues.apache.org/jira/browse/HIVE-24030
 Project: Hive
  Issue Type: Bug
  Components: ORC
Affects Versions: 4.0.0
Reporter: Dongjoon Hyun






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-19658) Update Hive to use ORC 1.4.4

[Dongjoon Hyun (JIRA)]

Dongjoon Hyun created HIVE-19658:


 Summary: Update Hive to use ORC 1.4.4
 Key: HIVE-19658
 URL: https://issues.apache.org/jira/browse/HIVE-19658
 Project: Hive
  Issue Type: Bug
  Components: ORC
Affects Versions: 3.0.0
Reporter: Dongjoon Hyun


ORC 1.4.4 (released on May 14th) includes nine fixes. This issue aims to update 
Hive to use it.

https://issues.apache.org/jira/issues/?filter=12342568=project%20%3D%20ORC%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%201.4.4



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-19497) SessionHiveMetaStoreClient.getTable should respect default catalog

[Dongjoon Hyun (JIRA)]

Dongjoon Hyun created HIVE-19497:


 Summary: SessionHiveMetaStoreClient.getTable should respect 
default catalog
 Key: HIVE-19497
 URL: https://issues.apache.org/jira/browse/HIVE-19497
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 3.0.0
Reporter: Dongjoon Hyun


Currently, SessionHiveMetaStoreClient always use `DEFAULT_CATALOG_NAME` during 
looking up the underlying tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-17186) `double` type constant operation loses precision

[Dongjoon Hyun (JIRA)]

Dongjoon Hyun created HIVE-17186:


 Summary: `double` type constant operation loses precision
 Key: HIVE-17186
 URL: https://issues.apache.org/jira/browse/HIVE-17186
 Project: Hive
  Issue Type: Bug
Reporter: Dongjoon Hyun


This might be an issue where Hive loses a precision and generates a wrong 
result when handling *double* constant operations. This was reported in the 
following environment.

*ENVIRONMENT*
https://github.com/hortonworks/hive-testbench/blob/hive14/tpch-gen/ddl/orc.sql

*SQL*
{code}
hive> explain select l_discount from lineitem where l_discount between 0.06 - 
0.01 and 0.06 + 0.01 limit 10;
OK
Plan not optimized by CBO.

Stage-0
   Fetch Operator
  limit:10
  Stage-1
 Map 1 vectorized
 File Output Operator [FS_9]
compressed:false
Statistics:Num rows: 10 Data size: 80 Basic stats: COMPLETE Column 
stats: COMPLETE
table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
Limit [LIM_8]
   Number of rows:10
   Statistics:Num rows: 10 Data size: 80 Basic stats: COMPLETE 
Column stats: COMPLETE
   Select Operator [OP_7]
  outputColumnNames:["_col0"]
  Statistics:Num rows: 294854 Data size: 2358832 Basic 
stats: COMPLETE Column stats: COMPLETE
  Filter Operator [FIL_6]
 predicate:l_discount BETWEEN 0.049996 AND 
0.06999 (type: boolean)
 Statistics:Num rows: 294854 Data size: 2358832 
Basic stats: COMPLETE Column stats: COMPLETE
 TableScan [TS_0]
alias:lineitem
Statistics:Num rows: 589709 Data size: 
4832986297043 Basic stats: COMPLETE Column stats: COMPLETE

hive> select max(l_discount) from lineitem where l_discount between 0.06 - 0.01 
and 0.06 + 0.01 limit 10;
OK
0.06
Time taken: 314.923 seconds, Fetched: 1 row(s)
{code}

Hive excludes 0.07 differently from the users' intuitiion. Also, this 
difference makes some users confused because they believe that Hive's result is 
the correct one. Is there any way for Hive to fix this?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)