Re: [ANNOUNCE] New committer: Miklos Gergely

[Lefty Leverenz]

Congrats Miklos!

-- Lefty


On Tue, Jul 16, 2019 at 5:25 AM Zoltan Haindrich  wrote:

> Congratulations!!
>
> On 7/16/19 9:38 AM, Peter Vary wrote:
> > Congratulations Miklos! :)
> >
> >> On Jul 15, 2019, at 16:33, Ashutosh Chauhan 
> wrote:
> >>
> >> Apache Hive's Project Management Committee (PMC) has invited Miklos
> Gergely
> >> to become a committer, and we are pleased to announce that he has
> accepted.
> >>
> >> Miklos welcome, thank you for your contributions, and we look forward
> your
> >> further interactions with the community!
> >>
> >> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> >
>

Re: [ANNOUNCE] New committer: Rajkumar Singh

[Lefty Leverenz]

Congratulations!

-- Lefty


On Fri, Jul 26, 2019 at 12:51 AM Naresh P R 
wrote:

> Congrats Raj!!!
>
> --
> Regards,
> Naresh P R
>
> On Fri, Jul 26, 2019 at 9:23 AM Ashutosh Chauhan 
> wrote:
>
> > Apache Hive's Project Management Committee (PMC) has invited Rajkumar
> Singh
> > to become a committer, and we are pleased to announce that he has
> accepted.
> >
> > Raj welcome, thank you for your contributions, and we look forward your
> > further interactions with the community!
> >
> > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> >
>

Re: [ANNOUNCE] New committer: Laszlo Bodor

[Lefty Leverenz]

Congratulations Laszlo!

-- Lefty


On Mon, Apr 15, 2019 at 1:28 PM Zoltán Haindrich  wrote:

> Congratulations Laszlo!
>
> On April 15, 2019 8:15:44 AM GMT+02:00, Peter Vary
>  wrote:
> >Congrats! Well done! :)
> >
> >> On Apr 15, 2019, at 06:50, Ashutosh Bapat
> > wrote:
> >>
> >> Congratulations Laszlo.
> >>
> >> On Mon, Apr 15, 2019 at 9:08 AM Ashutosh Chauhan
> >
> >> wrote:
> >>
> >>> Apache Hive's Project Management Committee (PMC) has invited Laszlo
> >>> Bodor to become a committer, and we are pleased to announce that he
> >has
> >>> accepted.
> >>>
> >>> Laszlo welcome, thank you for your contributions, and we look
> >forward your
> >>> further interactions with the community!
> >>>
> >>> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> >>>
> >>
> >>
> >> --
> >> --
> >> Best Wishes,
> >> Ashutosh Bapat
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Declare myself emeritus

[Lefty Leverenz]

Thanks for all your contributions, Edward.

-- Lefty


On Tue, Mar 19, 2019 at 1:44 PM Edward Capriolo 
wrote:

> Hello,
>
> I would like to declare myself emeritus.
>
> Thank you,
> Edward Capriolo
>
>
> --
> Sorry this was sent from mobile. Will do less grammar and spell check than
> usual.
>

Re: LZO Compression

[Lefty Leverenz]

Does anyone have a reply for this question?

-- Lefty


On Tue, Jan 29, 2019 at 2:27 PM dam6923  wrote:

> Hello,
>
> Quick question about LZO compression.
>
> After reading the docs, it seems to me that I must
> use DeprecatedLzoTextInputFormat in order to work with LZO files.
>
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LZO
>
> However, I am not sure that is correct.  As I understand it, as long as
> there is no LZO index files, and the LZO codec is installed, I should be
> able to use the standard Hive table definitions.  The normal mapper
> facilities will see the file's '.lzo' file extension and map it to the
> compression codec for unpacking.  The specialized LZO TextInputFormat is
> only required if there are a mix of '.lzo' and 'lzo..index' files in the
> table.  The normal facilities would try to process both files because it
> does not know that the 'index' file is not part of the normal data set; it
> does not know that the index file is simply metadata.
>
> Is this correct?
>

Re: [DISCUSS] Move gitbox notification emails to another list?

[Lefty Leverenz]

Remember to update the mailing lists page
 on the Hive website.  And while
you're at it, please document the issues mailing list (HIVE-9838
).

-- Lefty


On Thu, Feb 28, 2019 at 11:50 PM Peter Vary 
wrote:

> Tried to push forward https://issues.apache.org/jira/browse/INFRA-17927 <
> https://issues.apache.org/jira/browse/INFRA-17927>, but we need Ashutosh
> (PMC Chair, or someone from ASF) to create a new mailing list.
>
> So waiting for Ashutosh now.
>
> Thanks,
> Peter
>
> > On Feb 28, 2019, at 19:59, Jesus Camacho Rodriguez <
> jcamachorodrig...@hortonworks.com> wrote:
> >
> > Thanks Vineet.
> >
> > I created https://issues.apache.org/jira/browse/INFRA-17927 for the
> gitbox one.
> >
> > -Jesús
> >
> >
> > On 2/28/19, 10:55 AM, "Vineet Garg"  wrote:
> >
> >FYI I have opened an INFRA jira for RB issue:
> >https://issues.apache.org/jira/browse/INFRA-17926. I personally
> prefer it
> >over github pull request for reviews.
> >
> >
> >On Thu, Feb 28, 2019 at 10:50 AM Jesus Camacho Rodriguez <
> >jcamachorodrig...@hortonworks.com> wrote:
> >
> >> I think it is a good idea, I can ask that the 'opened a new pull
> request'
> >> messages continue being sent to @dev, while the rest are sent to gitbox.
> >>
> >> -Jesús
> >>
> >>
> >> On 2/28/19, 9:55 AM, "Alan Gates"  wrote:
> >>
> >>+1 to sending the github mail to a separate list.
> >>
> >>I agree with Peter that seeing PR reviews is good.  Wouldn't it be
> >> possible
> >>to craft a filter that only allowed through these mails?
> >>
> >>Alan.
> >>
> >>
> >>
> >>On Thu, Feb 28, 2019 at 12:11 AM Peter Vary
>  >>>
> >>wrote:
> >>
> >>> The github mails in this form are just white noise, so I have added
> >>> filters to my mailbox to drop every github mail to another folder,
> >> just as
> >>> proposed by Jesús. So if we can do it by the infra it would be
> >> better. +1
> >>> from me.
> >>>
> >>> On the other hand, I miss the "review request created" messages of
> >> the
> >>> review board. These type of messages are lost in the shower of the
> >> github
> >>> letters. If anyone has an idea how to detect these that would be
> >> awesome.
> >>>
> >>> Thanks,
> >>> Peter
> >>>
>  On Feb 28, 2019, at 08:59, Mani M  wrote:
> 
>  Good.
> 
>  With Regards
>  M.Mani
>  +61 432 461 087
> 
>  On Thu, 28 Feb 2019, 17:39 Jesus Camacho Rodriguez, <
>  jcamachorodrig...@hortonworks.com> wrote:
> 
> > We have had a similar discussion in the Calcite project too.
> >
> > Gitbox emails are being set to the dev@ list. These emails are
> >> produced
> > every time there is activity in the Hive Github repository and are
> >>> creating
> > quite a lot of noise. The result is that it is really difficult to
> >>> follow
> > any activity in the list.
> >
> > A possible alternative would be to send them to another list, e.g.
> > gitbox@h.a.o. The idea is that having them in a list may still be
> >>> useful
> > as they may serve as a searchable archive of activity in the repo.
> >
> > What do you think? Should we open an INFRA ticket to request this?
> >
> > Thanks,
> > Jesús
> >
> >
> >>>
> >>>
> >>
> >>
> >>
> >
> >
>
>

Re: Please unsubscribe me

[Lefty Leverenz]

Daniel, to unsubscribe please send a message to
dev-unsubscr...@hive.apache.org as described here:  Mailing Lists
.

Make sure you're sending the message from the email address that should be
unsubscribed, since it's an automated system.

Thanks.

-- Lefty


On Wed, Jan 30, 2019 at 6:41 AM Daniel Takacs 
wrote:

>
> I tried to unsubscribe several times but still it is not working.  Please
> help me unsubscribe
> Sent from my iPhone

Re: Hive Confluence access

[Lefty Leverenz]

You have edit permissions for the wiki now.  Welcome to the Hive wiki team,
M. Mani!

Hive developers:  Will those of you who can grant wiki permissions please
keep an eye out for such requests?  I'm not monitoring the email lists as
closely as I used to.  Thanks.

-- Lefty


On Fri, Feb 1, 2019 at 6:51 PM Mani M  wrote:

> Hi Team,
>
> Kindly provide me the Hive confluence access to update the below document.
>
>
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-Built-inFunctions
>
> My userid : rmsm...@gmail.com
>
>
> With Regards
> M.Mani
> +61 432 461 087
>
> On Fri, 25 Jan 2019, 22:06 Mani M 
> > HI Lefty,
> >
> > Kindly provide me the Hive confluence access to update the below
> document.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-Built-inFunctions
> >
> > My userid : rmsm...@gmail.com
> > With Regards
> > M.Mani
> >
>

Re: [ANNOUNCE] New committer: Bharathkrishna Guruvayoor Murali

[Lefty Leverenz]

Congratulations!

-- Lefty


On Mon, Dec 3, 2018 at 12:29 AM Marta Kuczora 
wrote:

> Congratulations Bharath!
>
> On Mon, Dec 3, 2018 at 8:45 AM Peter Vary 
> wrote:
>
> > Congratulations!
> >
> > > On Dec 3, 2018, at 05:32, Sankar Hariappan  >
> > wrote:
> > >
> > > Congrats Bharath!
> > >
> > > Best regards
> > > Sankar
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On 03/12/18, 7:38 AM, "Vihang Karajgaonkar"
> 
> > wrote:
> > >
> > >> Congratulations Bharath!
> > >>
> > >> On Sun, Dec 2, 2018 at 9:33 AM Sahil Takiar 
> > wrote:
> > >>
> > >>> Congrats Bharath!
> > >>>
> > >>> On Sun, Dec 2, 2018 at 11:14 AM Andrew Sherman
> > >>>  wrote:
> > >>>
> >  Congratulations Bharath!
> > 
> >  On Sat, Dec 1, 2018 at 10:26 AM Ashutosh Chauhan <
> > hashut...@apache.org>
> >  wrote:
> > 
> > > Apache Hive's Project Management Committee (PMC) has invited
> > > Bharathkrishna
> > > Guruvayoor Murali to become a committer, and we are pleased to
> > announce
> > > that
> > > he has accepted.
> > >
> > > Bharath, welcome, thank you for your contributions, and we look
> > forward
> > > your
> > > further interactions with the community!
> > >
> > > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> > >
> > 
> > >>>
> > >>>
> > >>> --
> > >>> Sahil Takiar
> > >>> Software Engineer
> > >>> takiar.sa...@gmail.com | (510) 673-0309
> > >>>
> >
> >
>

Re: [ANNOUNCE] New committer: Mahesh Behera

[Lefty Leverenz]

Congrats!

-- Lefty


On Mon, Nov 19, 2018 at 12:16 PM Zoltan Haindrich  wrote:

> Congratulations!!
>
> On 19 November 2018 19:48:10 CET, Thejas Nair 
> wrote:
> >Congrats Mahesh!
> >Very well deserved!
> >
> >On Fri, Nov 16, 2018 at 6:24 PM Ashutosh Chauhan 
> >wrote:
> >>
> >>  Apache Hive's Project Management Committee (PMC) has invited Mahesh
> >> Behera to become a committer, and we are pleased to announce that he
> >has
> >> accepted.
> >> Mahesh, welcome, thank you for your contributions, and we look
> >forward to
> >> your further interactions with the community!
> >>
> >> Thanks,
> >> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [ANNOUNCE] New PMC Member : Zoltan

[Lefty Leverenz]

Congratulations!

-- Lefty


On Thu, Nov 1, 2018 at 12:45 AM Sankar Hariappan 
wrote:

> Congrats Zoltan!
>
> Best regards
> Sankar
>
>
>
>
>
>
>
> On 31/10/18, 11:12 PM, "Vihang Karajgaonkar" 
> wrote:
>
> >Congrats Zoltan!
> >
> >On Wed, Oct 31, 2018 at 1:10 AM, Peter Vary 
> >wrote:
> >
> >> Congratulations Zoltan!
> >> Well deserved!
> >>
> >> > On Oct 31, 2018, at 06:16, Prasanth Jayachandran <
> >> pjayachand...@hortonworks.com> wrote:
> >> >
> >> > Congratulations!
> >> >
> >> >> On Oct 30, 2018, at 10:15 PM, Deepak Jaiswal <
> djais...@hortonworks.com>
> >> wrote:
> >> >>
> >> >> Congratulations Zoltan!
> >> >>
> >> >> On 10/30/18, 10:08 PM, "Ashutosh Chauhan" 
> >> wrote:
> >> >>
> >> >>Hello Hive community,
> >> >>
> >> >>   I'm pleased to announce that Zoltan Haindrich has accepted the
> Apache
> >> >>   Hive PMC's invitation, and is our newest PMC member. Many thanks to
> >> >>   Zoltan for all of his hard work.
> >> >>
> >> >>   Please join me in congratulating Zoltan!
> >> >>
> >> >>   Thanks,
> >> >>   Ashutosh
> >> >>
> >> >>
> >> >
> >>
> >>
>

Re: [ANNOUNCE] New committer: Nishant Bangarwa

[Lefty Leverenz]

Congratulations Nishant!

-- Lefty


On Mon, Oct 15, 2018 at 3:15 AM Ashutosh Chauhan 
wrote:

> Apache Hive's Project Management Committee (PMC) has invited Nishant
> Bangarwa
> to become a committer, and we are pleased to announce that he has accepted.
>
> Nishant, welcome, thank you for your contributions, and we look forward
> your
> further interactions with the community!
>
> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
>

Re: [ANNOUNCE] New committer: Janaki Lahorani

[Lefty Leverenz]

Congratulations!

-- Lefty


On Wed, Oct 10, 2018 at 4:35 AM Peter Vary 
wrote:

> Congratulations Janaki!
>
> > On Oct 9, 2018, at 20:20, Deepak Jaiswal 
> wrote:
> >
> > Congratulations Janaki.
> >
> > On 10/9/18, 8:52 AM, "Vihang Karajgaonkar" 
> wrote:
> >
> >Congratulations Janaki!
> >
> >On Tue, Oct 9, 2018 at 8:27 AM Andrew Sherman
> 
> >wrote:
> >
> >> Congratulations Janaki!
> >>
> >> On Mon, Oct 8, 2018 at 10:05 PM Ashutosh Chauhan 
> >> wrote:
> >>
> >>> Apache Hive's Project Management Committee (PMC) has invited Janaki
> >>> Lahorani to become a committer, and we are pleased to announce that she
> >> has
> >>> accepted.
> >>> Janaki, welcome, thank you for your contributions, and we look forward
> to
> >>> your further interactions with the community!
> >>>
> >>> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> >>>
> >>
> >
> >
>
>

Re: Confluence edit permission

[Lefty Leverenz]

Hi Ashutosh, we might need a nickname for you since your the second
Ashutosh on the Hive project.  ;)

All you need is a Confluence username, as described here:
https://cwiki.apache.org/confluence/display/Hive/AboutThisWiki#AboutThisWiki-Howtogetpermissiontoedit

Once you tell me (or various others) your username, we'll sign you up with
edit permissions.  (You don't really have to go through the user@hive
mailing list.)

-- Lefty


On Tue, Oct 2, 2018 at 11:06 PM Ashutosh Bapat 
wrote:

> Hi Lefty,
> I would like to get permissions to edit pages in confluence. I am working
> with Hive team in Hortonworks. I couldn't find a way to request edit
> permissions through confluence, hence this mail.
>
>
> --
> Best Wishes,
> Ashutosh Bapat
>

Re: Confluence edit permission

[Lefty Leverenz]

Daniel, you have edit permission now.  Welcome to the Hive wiki team!

-- Lefty


On Mon, Sep 3, 2018 at 8:45 AM Dániel Vörös  wrote:

> Hi All,
>
> I'd like to get edit permission to be able to document changes
> of HIVE-20191 on this page:
> https://cwiki.apache.org/confluence/display/Hive/HowToContribute
>
> Could someone please help me with this?
>
> My username is dvoros.
>
> Best Regards,
> Daniel
>

Re: [ANNOUNCE] New committer: Andrew Sherman

[Lefty Leverenz]

Congratulations Andrew!

-- Lefty


On Tue, Aug 28, 2018 at 11:36 AM Ashutosh Chauhan 
wrote:

> Apache Hive's Project Management Committee (PMC) has invited Andrew Sherman
> to become a committer, and we are pleased to announce that he has accepted.
>
> Andrew, welcome, thank you for your contributions, and we look forward to
> your
> further interactions with the community!
>
> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Lefty Leverenz]

Congratulations Slim!

-- Lefty


On Tue, Jul 31, 2018 at 7:40 AM Rajesh Balamohan 
wrote:

> Congratulations Slim!
>
> ~Rajesh.B
>
> On Tue, Jul 31, 2018 at 3:34 PM Marta Kuczora
> 
> wrote:
>
> > Congratulations Slim!
> >
> > On Mon, Jul 30, 2018 at 2:01 AM Ashutosh Chauhan 
> > wrote:
> >
> > > Apache Hive's Project Management Committee (PMC) has invited Slim
> > Bouguerra
> > > to become a committer, and we are pleased to announce that he has
> > accepted.
> > >
> > > Slim, welcome, thank you for your contributions, and we look forward
> your
> > > further interactions with the community!
> > >
> > > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> > >
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Lefty Leverenz]

Congratulations Vineet!

-- Lefty


On Wed, Aug 1, 2018 at 3:05 PM Vaibhav Gumashta 
wrote:

> Congrats Vineet!
>
> On 8/1/18, 11:33 AM, "Chaoyu Tang"  wrote:
>
> Congratulations Vineet!
>
> On Tue, Jul 31, 2018 at 7:39 AM, Rajesh Balamohan <
> rajesh.balamo...@gmail.com> wrote:
>
> > Congratulations Vineet!
> >
> > ~Rajesh.B
> >
> >
> > On Tue, Jul 31, 2018 at 3:34 PM Marta Kuczora
> > 
> > wrote:
> >
> > > Congratulations Vineet!
> > >
> > > On Mon, Jul 30, 2018 at 9:45 AM Peter Vary
> 
> > > wrote:
> > >
> > > > Congratulations Vineet!
> > > >
> > > > > On Jul 30, 2018, at 01:59, Ashutosh Chauhan <
> hashut...@apache.org>
> > > > wrote:
> > > > >
> > > > > On behalf of the Hive PMC I am delighted to announce Vineet
> Garg is
> > > > joining
> > > > > Hive PMC.
> > > > > Thanks Vineet for all your contributions till now. Looking
> forward to
> > > > many
> > > > > more.
> > > > >
> > > > > Welcome, Vineet!
> > > > >
> > > > > Thanks,
> > > > > Ashutosh
> > > >
> > > >
> > >
> >
> >
> > --
> > ~Rajesh.B
> >
>
>
>

Re: [ANNOUNCE] New PMC Member : Peter Vary

[Lefty Leverenz]

Congratulations Peter!

-- Lefty


On Wed, Aug 1, 2018 at 3:05 PM Vaibhav Gumashta 
wrote:

> Congrats Peter!
>
> On 8/1/18, 11:31 AM, "Chaoyu Tang"  wrote:
>
> Congratulations, Peter.
>
> On Wed, Aug 1, 2018 at 2:08 PM, Peter Vary  >
> wrote:
>
> > Thanks everyone!
> >
> > Rajesh Balamohan  ezt írta (időpont: 2018.
> júl.
> > 31.,
> > Ke 13:41):
> >
> > > Congratulations Peter!
> > >
> > > ~Rajesh.B
> > >
> > >
> > > On Tue, Jul 31, 2018 at 3:58 PM Marta Kuczora
> > > 
> > > wrote:
> > >
> > > > Congratulations Peter!
> > > >
> > > > On Mon, Jul 30, 2018 at 7:53 PM Andrew Sherman
> > > >  wrote:
> > > >
> > > > > Congratulations Peter!
> > > > >
> > > > > On Sun, Jul 29, 2018 at 1:32 PM Vineet Garg <
> vg...@hortonworks.com>
> > > > wrote:
> > > > >
> > > > > > Congratulations Peter!
> > > > > >
> > > > > > > On Jul 26, 2018, at 11:25 AM, Ashutosh Chauhan <
> > > hashut...@apache.org
> > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > On behalf of the Hive PMC I am delighted to announce Peter
> Vary
> > is
> > > > > > joining
> > > > > > > Hive PMC.
> > > > > > > Thanks Peter for all your contributions till now. Looking
> forward
> > > to
> > > > > many
> > > > > > > more.
> > > > > > >
> > > > > > > Welcome, Peter!
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ashutosh
> > > > > >
> > > > > >
> > > > >
> > >
> >
>
>
>

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

[Lefty Leverenz]

Congratulations Sahil!

-- Lefty


On Wed, Aug 1, 2018 at 3:05 PM Vaibhav Gumashta 
wrote:

> Congrats Sahil!
>
> On 8/1/18, 11:32 AM, "Chaoyu Tang"  wrote:
>
> Congratulations Sahil!
>
> On Tue, Jul 31, 2018 at 7:40 AM, Rajesh Balamohan <
> rbalamo...@apache.org>
> wrote:
>
> > Congratulations Sahil!
> >
> > ~Rajesh.B
> >
> >
> > On Tue, Jul 31, 2018 at 3:57 PM Marta Kuczora
> > 
> > wrote:
> >
> > > Congratulations Sahil!
> > >
> > > On Mon, Jul 30, 2018 at 9:44 AM Peter Vary
> 
> > > wrote:
> > >
> > > > Congratulations Sahil!
> > > >
> > > > > On Jul 29, 2018, at 22:32, Vineet Garg 
> > wrote:
> > > > >
> > > > > Congratulations Sahil!
> > > > >
> > > > >> On Jul 26, 2018, at 11:28 AM, Ashutosh Chauhan <
> > hashut...@apache.org>
> > > > wrote:
> > > > >>
> > > > >> On behalf of the Hive PMC I am delighted to announce Sahil
> Takiar is
> > > > >> joining Hive PMC.
> > > > >> Thanks Sahil for all your contributions till now. Looking
> forward to
> > > > many
> > > > >> more.
> > > > >>
> > > > >> Welcome, Sahil!
> > > > >>
> > > > >> Thanks,
> > > > >> Ashutosh
> > > > >
> > > >
> > > >
> >
>
>
>

Re: [ANNOUNCE] New PMC Member : Vihang Karajgaonkar

[Lefty Leverenz]

Congratulations Vihang!

-- Lefty


On Wed, Aug 1, 2018 at 3:05 PM Vaibhav Gumashta 
wrote:

> Congrats Vihang!
>
> On 8/1/18, 11:33 AM, "Chaoyu Tang"  wrote:
>
> Congratulations Vihang.
>
> On Tue, Jul 31, 2018 at 7:39 AM, Rajesh Balamohan <
> rbalamo...@apache.org>
> wrote:
>
> > Congratulations Vihang!
> >
> > ~Rajesh.B
> >
> >
> > On Tue, Jul 31, 2018 at 3:35 PM Marta Kuczora
> > 
> > wrote:
> >
> > > Congratulations Vihang!
> > >
> > > On Mon, Jul 30, 2018 at 9:44 AM Peter Vary
> 
> > > wrote:
> > >
> > > > Congratulations Vihang!
> > > >
> > > > > On Jul 29, 2018, at 22:32, Vineet Garg 
> > wrote:
> > > > >
> > > > > Congratulations Vihang!
> > > > >
> > > > >> On Jul 26, 2018, at 11:27 AM, Ashutosh Chauhan <
> > hashut...@apache.org>
> > > > wrote:
> > > > >>
> > > > >> On behalf of the Hive PMC I am delighted to announce Vihang
> > > > Karajgaonkar
> > > > >> is joining Hive PMC.
> > > > >> Thanks Vihang for all your contributions till now. Looking
> forward
> > to
> > > > many
> > > > >> more.
> > > > >>
> > > > >> Welcome, Vihang!
> > > > >>
> > > > >> Thanks,
> > > > >> Ashutosh
> > > > >
> > > >
> > > >
> >
>
>
>

Re: External vs. Managed Tables - Current state

[Lefty Leverenz]

Agreed, the Hive wiki is woefully incomplete.  But it does mention DROP for
external tables in two sections of the DDL doc:

   - External Tables
   
:
"When dropping an EXTERNAL table, data in the table is NOT deleted from
   the file system."
   - Drop Table
   
:
"When dropping an EXTERNAL table, data in the table will NOT be deleted
   from the file system."

A wiki page comparing managed and external tables would be very helpful,
and the list is a good start.  I suggest you open a JIRA issue.

Who can help Lars with this task?

-- Lefty


On Thu, Jul 19, 2018 at 3:04 PM Lars Francke  wrote:

> Hi,
>
> I've been wondering if anyone can tell me what the differences and
> limitations of managed vs. external tables are these days.
>
> I've seen the docs from Hortonworks[1] that list these features as not
> supported by External tables:
> * Query cache
> * Materialized views, except in a limited way
> * Default statistics gathering
> * Compute queries using statistics
> * Automatic runtime filtering
> * File merging after insert
>
> But there are no details. For some of them I'm not even sure what they are
> supposed to mean because I couldn't find any documentation in our Wiki :(
>
> So I'd love to create a Wiki page that describes the differences between
> managed & external tables and list relevant JIRAs but I need your help for
> that.
>
> I scanned the code for all references to TableType and this is the list I
> found but I'm almost certainly missing something:
>
> * ARCHIVE/UNARCHIVE - DDLTask - Only works for managed tables
> * TRUNCATE - DDLSemanticAnalyzer - Only works for managed tables
> * MERGE/CONCATENATE - HiveRelOpMaterializationValidator - Only works for
> managed tables
> * Constraints - DDLSemanticAnalyzer -  (NOT NULL, DEFAULT, CHECK, only RELY
> ist allowed)
> * IMPORT - ImportSemanticAnalyzer - This has some wild restrictions I
> didn't follow for external tables
> * Query Results Caching - https://issues.apache.org/jira/browse/HIVE-18513
> SemanticAnalyzer - Documentation missing for Results Cache
>
> So there's a bunch of those from the Hortonworks list missing. If anyone
> could point me to documentation or code for those that'd be great.
>
> I also didn't find the most obvious/oldest one: Data is not deleted for
> EXTERNAL tables on DROP, I'm sure I just missed it.
>
> And last but not least: I have not looked at the grammar to see what's
> already forbidden at that level.
>
> As I said: Any hints would be greatly appreciated.
>
> Thank you!
>
> Lars
>
> A side note: There are so many fantastic features in Hive but because lots
> of them are not or under-documented most people I know just stick to the
> basics. That's a pity :(
>
> [1] <
>
> https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/using-hiveql/content/hive_hive_3_tables.html
> >
>

Re: [ANNOUNCE] New committer: Adam Szita

[Lefty Leverenz]

Congratulations Adam!

-- Lefty


On Thu, Jun 21, 2018 at 1:46 AM Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Congratulations!
>
> Thanks
> Prasanth
>
>
>
> On Wed, Jun 20, 2018 at 10:44 PM -0700, "Vihang Karajgaonkar"
> mailto:vih...@cloudera.com.INVALID>> wrote:
>
>
> Congrats Adam!
>
> On Wed, Jun 20, 2018 at 8:45 PM, Zoltan Haindrich  wrote:
>
> > Congratulations Ádám!
> >
> > On 20 June 2018 22:20:16 CEST, Deepak Jaiswal
> > wrote:
> > >Congratulations Adam.
> > >
> > >On 6/20/18, 12:02 PM, "Ashutosh Chauhan"  wrote:
> > >
> > >Apache Hive's Project Management Committee (PMC) has invited Adam Szita
> > >to become a committer, and we are pleased to announce that he has
> > >accepted.
> > >
> > >Adam, welcome, thank you for your contributions, and we look forward
> > >your
> > >further interactions with the community!
> > >
> > >Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> > >
> >
>
>

Re: New committer announcement : Marta Kuczora

[Lefty Leverenz]

Congratulations Marta!

-- Lefty


On Thu, Jun 21, 2018 at 1:46 AM Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Congratulations!
>
> Thanks
> Prasanth
>
>
>
> On Wed, Jun 20, 2018 at 10:44 PM -0700, "Vihang Karajgaonkar"
> mailto:vih...@cloudera.com.INVALID>> wrote:
>
>
> Congrats Marta!
>
> On Wed, Jun 20, 2018 at 8:46 PM, Zoltan Haindrich  wrote:
>
> > Congratulations Márta!
> >
> > On 20 June 2018 22:20:30 CEST, Deepak Jaiswal
> > wrote:
> > >Congratulations Marta.
> > >
> > >On 6/20/18, 12:06 PM, "Ashutosh Chauhan"  wrote:
> > >
> > >Apache Hive's Project Management Committee (PMC) has invited Marta
> > >Kuczora
> > >to become a committer, and we are pleased to announce that he has
> > >accepted.
> > >
> > >Marta, welcome, thank you for your contributions, and we look forward
> > >your
> > >further interactions with the community!
> > >
> > >Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> > >
> >
>
>

Re: Need edit permission to Hive confluence

[Lefty Leverenz]

Done.  Welcome to the Hive wiki team, Janaki!

-- Lefty


On Mon, Jun 18, 2018 at 1:46 PM Janaki Latha Lahorani
 wrote:

> Hi,
>
> I would like to update documentation related to some minor enhancements.  I
> would be very grateful if I have the permission to edit.
>
> Username: janulatha
>
> Thanks,
> Janaki.
>

Re: Publishing Hive RC artifacts

[Lefty Leverenz]

Oh, I see you've already updated the wiki.  Presumably that takes care of
it, thanks.

-- Lefty


On Sat, May 19, 2018 at 3:35 PM Lefty Leverenz <leftylever...@gmail.com>
wrote:

> Vineet, did you get an answer?  Does the wiki need to be revised?
>
> -- Lefty
>
>
> On Tue, May 15, 2018 at 6:06 PM Vineet Garg <vg...@hortonworks.com> wrote:
>
>> Hello,
>>
>> I am unable to publish Hive 3.0 RC artifacts to maven. Following wiki
>> page (
>> https://cwiki.apache.org/confluence/display/Hive/HowToRelease#HowToRelease-Voting)
>> I am running following command:
>>
>>
>> mvn deploy -DskipTests -Papache-release -Dmaven.javadoc.skip=true
>>
>> But doing so result in error:
>>
>> ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy (default-deploy)
>> on project hive: Failed to deploy artifacts/metadata: Cannot access
>> ${repourl} with type default using the available connector factories:
>> BasicRepositoryConnectorFactory: Cannot access ${repourl} using the
>> registered transporter factories: WagonTransporterFactory: Unsupported
>> transport protocol -> [Help 1]
>> [ERROR]
>>
>> Anybody know how do I fix this and publish RC artifacts to maven?
>>
>> Vineet
>>
>

Re: Publishing Hive RC artifacts

[Lefty Leverenz]

Vineet, did you get an answer?  Does the wiki need to be revised?

-- Lefty


On Tue, May 15, 2018 at 6:06 PM Vineet Garg  wrote:

> Hello,
>
> I am unable to publish Hive 3.0 RC artifacts to maven. Following wiki page
> (
> https://cwiki.apache.org/confluence/display/Hive/HowToRelease#HowToRelease-Voting)
> I am running following command:
>
>
> mvn deploy -DskipTests -Papache-release -Dmaven.javadoc.skip=true
>
> But doing so result in error:
>
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy (default-deploy)
> on project hive: Failed to deploy artifacts/metadata: Cannot access
> ${repourl} with type default using the available connector factories:
> BasicRepositoryConnectorFactory: Cannot access ${repourl} using the
> registered transporter factories: WagonTransporterFactory: Unsupported
> transport protocol -> [Help 1]
> [ERROR]
>
> Anybody know how do I fix this and publish RC artifacts to maven?
>
> Vineet
>

Re: Apache Hive 3.0.0 Release branch cutoff

[Lefty Leverenz]

This would be a good time for contributors to document their 3.0.0 patches.

108 jiras have TODOC3.0 labels, but undoubtedly a few dozen more should
have gotten labeled since January 1st (when I stopped scanning email for
doc issues).

36 developers are the assignees for those 108 issues.  If you're one of
them, please don't hide your contributions in the code -- use the Hive wiki
to let the world know about your brilliant patches.

Thanks to those who already took care of their docs!

-- Lefty


On Mon, Apr 9, 2018 at 5:15 PM Vineet Garg  wrote:

> Hello,
>
> To keep the naming conventions for release branch I have created branch-3
> for Apache hive 3.0.0 release. Please use this branch instead of
> branch-3.0.0.
>
> Vineet G
> .
> On Apr 9, 2018, at 1:45 PM, Vineet Garg > wrote:
>
> Hello,
>
> The branch for 3.0.0 release has been cut off (branch-3.0.0).
>
> I am going to update all unresolved JIRA’s which aren’t marked blocker
> with fix version 3.0.0 to 3.1.0. Please update them if you would like to
> get your patch in 3.0.0.
>
> Thanks,
> Vineet G
>
>

Re: Need edit permission to Hive Confluence page

[Lefty Leverenz]

Bharath, first you need to get a Confluence username as described here:

About This Wiki -- How to get permission to edit



(It's okay to use the dev@hive list instead of user@hive, just tell us your
username in this thread.)

-- Lefty


On Thu, Apr 5, 2018 at 2:57 PM Bharathkrishna Guruvayoor Murali <
bhar...@cloudera.com> wrote:

> Hi,
>
> I need edit permission to Confluence page.
> Currently I need to make a doc update related to HIVE-16944.
>
> I would like to update SchemaTool wiki page
>  to
> include the parameters driver and URL which are currently missing in the
> documentation.
>
> Someone who has the rights please add me.
>
> Thanks,
> Bharath
>

Re: Documenting standalone metastore

[Lefty Leverenz]

Agreed.

In the last 3 years there were only 7 changes to the metastore admin doc,
so maintaining parallel docs might not be much of a problem.

-- Lefty


On Mon, Apr 2, 2018 at 2:06 PM Thejas Nair  wrote:

> Sounds good to me.
> We have done something similar in past for HiveServer1 vs HiveServer2,
> ReplicationV1 vs V2 etc .
>
>
> On Mon, Apr 2, 2018 at 10:42 AM, Alan Gates  wrote:
> > I've started looking at what will be required to document the new
> > standalone metastore.  I started by reviewing the existing admin guide
> for
> > the metastore at
> >
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin
> >
> > Much of this will need to change for the standalone metastore, as we'll
> > need to add in information for the standalone case, many configuration
> > values and tool names have changed (though the old ones still work for
> > backwards compatibility).  I am concerned that if I put all of the
> changes
> > in this document it will make it hard to use for people administering
> Hive
> > 1 and 2, which is everyone right now and will continue to be a majority
> for
> > at least the next year.
> >
> > So I propose to add a new page for Hive 3.0, and clearly edit the current
> > page to say it only applies to Hive 1 and 2 and put in a pointer to the
> > Hive 3 page.  Much of the content of that page will mirror the current
> one,
> > which is painful from a maintenance viewpoint; however, it will produce a
> > more usable set of documentation for our users.
> >
> > Seem reasonable?  I wanted to check before proceeding as this is not how
> we
> > have usually done our documentation.
> >
> > Alan.
>

Re: Documentation for partition expressions

[Lefty Leverenz]

Do you mean these sections in the DDL doc or something more elaborate?

   - Add Partitions
   

   - Alter Column -- Partial Partition Specification
   



-- Lefty


On Mon, Mar 26, 2018 at 12:37 AM Alexander Kolbasov 
wrote:

> Can someone point me to the documentation for partition expressions?
>
> Thanks,
>
> - Alex
>

Re: Confluence - Need edit permission

[Lefty Leverenz]

Done.  Welcome to the Hive wiki team, Karthik!

-- Lefty


On Mon, Mar 26, 2018 at 8:39 AM Karthik P  wrote:

> Confluence:
> User ID : kpalanisamy
> Name : Karthik Palanisamy
>
>
> On Mon, Mar 26, 2018 at 2:51 PM, Karthik P 
> wrote:
>
> > Team,
> >
> > I need edit permission to Confluence HBaseBulkLoad
> > . Some
> > distribution like HDP has enabled 'Tez' as execution engine which by
> > default. So HFile generation may not work properly and will throw the
> > following exception.
> >
> > {code}
> >
> > Caused by: java.io.IOException: wrong key class: org.apache.hadoop.io
> > .LongWritable is not class org.apache.hadoop.hive.ql.io.HiveKey
> >
> > at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2332)
> >
> > at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2384)
> >
> > at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitio
> > ner.readPartitions(TotalOrderPartitioner.java:306)
> >
> > at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitio
> > ner.setConf(TotalOrderPartitioner.java:88)
> >
> > ... 27 more
> > {code}
> >
> >
> > Some ref,  TEZ-2741 
> >
> >
> > *Document fix:* Set map-reduce engine when generating HFiles.
> > *set hive.execution.engine=mr;*
> >
> > --
> > Thank you,
> > *Karthik Palanisamy*
> > Bangalore, *India*
> > Mobile : +91 9940089181
> > Skype : karthik.p01
> >
>
>
>
> --
> Thank you,
> *Karthik Palanisamy*
> Bangalore, *India*
> Mobile : +91 9940089181
> Skype : karthik.p01
>

Re: Hive 3.0 release

[Lefty Leverenz]

+1.  Could some attention be given to documentation?  This would need to
come from developers, since I've cut back my participation.

-- Lefty



On Wed, Mar 21, 2018 at 6:00 PM Alan Gates  wrote:

> +1.  And +1 to Vihang's comment that this will be helpful for the
> standalone-metastore.  We should work on producing a standalone-metastore
> release in the same time frame so that the schema's, etc. match.  I can RM
> that unless someone else wants to.
>
> Alan.
>
> On Wed, Mar 21, 2018 at 12:05 PM, Andrew Sherman 
> wrote:
>
> > +1
> >
> > -Andrew
> >
> > On Wed, Mar 21, 2018 at 11:33 AM, Jesus Camacho Rodriguez <
> > jcama...@apache.org> wrote:
> >
> > > +1
> > >
> > > -Jesús
> > >
> > > On 3/21/18, 10:44 AM, "Prasanth Jayachandran" <
> > > pjayachand...@hortonworks.com> wrote:
> > >
> > > +1
> > >
> > > Thanks
> > > Prasanth
> > >
> > >
> > >
> > > On Wed, Mar 21, 2018 at 9:59 AM -0700, "Vihang Karajgaonkar" <
> > > vih...@cloudera.com> wrote:
> > >
> > >
> > > +1 on releasing Hive 3.0.0. This would be useful for
> > > standalone-metastore
> > > work as well.
> > >
> > > On Wed, Mar 21, 2018 at 8:49 AM, Gunther Hagleitner <
> > > ghagleit...@hortonworks.com> wrote:
> > >
> > > > +1
> > > >
> > > > Thank you,
> > > > Gunther
> > > >
> > > >
> > > >
> > > > On Tue, Mar 20, 2018 at 11:07 PM -0700, "Thejas Nair" <
> > > > thejas.n...@gmail.com> wrote:
> > > >
> > > >
> > > > +1
> > > >
> > > >
> > > > On Tue, Mar 20, 2018 at 4:46 PM, Ashutosh Chauhan  wrote:
> > > > > Hi all,
> > > > >
> > > > > It's almost a year since we branched off branch-2 Since then
> > there
> > > are
> > > > more
> > > > > than 1700 commits which master has received. There are lots of
> > new
> > > > features
> > > > > hidden in these commits. I think its about time to make a
> release
> > > so that
> > > > > users can get their hands on them.
> > > > >
> > > > > I propose that we cut a branch for 3.0 by this month end and
> then
> > > target
> > > > > release from it in following weeks.
> > > > > Of course, we want to continue supporting branch-2 as well and
> > > make dot
> > > > > releases from it.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Thanks,
> > > > > Ashutosh
> > > > >
> > > > > PS: Any volunteers for RM ?
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> >
>

Re: Review Request 65686: HIVE-18728 Secure webHCat with SSL

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65686/#review197767
---




hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/AppConfig.java
Lines 113-117 (patched)
<https://reviews.apache.org/r/65686/#comment278047>

These configs look fine to me.  After the patch is committed they will need 
to be documented in  
https://cwiki.apache.org/confluence/display/Hive/WebHCat+Configure#WebHCatConfigure-ConfigurationVariables.
 
HIVE-18728's description has the documentation.


- Lefty Leverenz


On Feb. 16, 2018, 2:19 p.m., Oleksiy Sayankin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65686/
> ---
> 
> (Updated Feb. 16, 2018, 2:19 p.m.)
> 
> 
> Review request for hive, Lefty Leverenz, Lefty Leverenz, Rui Li, Lenni Kuff, 
> Szehon Ho, and Szehon Ho.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Initiall commit
> 
> 
> Diffs
> -
> 
>   
> hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/AppConfig.java
>  1fd9e47 
>   
> hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Main.java
>  d183b2e 
> 
> 
> Diff: https://reviews.apache.org/r/65686/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Oleksiy Sayankin
> 
>

Re: [Announce] New committer: Deepak Jaiswal

[Lefty Leverenz]

Congratulations!

-- Lefty


On Mon, Jan 8, 2018 at 10:41 AM, Vihang Karajgaonkar 
wrote:

> Congrats Deepak!
>
> On Mon, Jan 8, 2018 at 10:25 AM, Mithun RK  wrote:
>
> > Congratulations, Deepak!
> >
> > On Mon, Jan 8, 2018 at 10:14 AM D K  wrote:
> >
> > > Congratulations Deepak!
> > >
> > > On Fri, Jan 5, 2018 at 2:18 PM, Ashutosh Chauhan  >
> > > wrote:
> > >
> > > > The Project Management Committee (PMC) for Apache Hive has invited
> > Deepak
> > > > Jaiswal to become a committer and we are pleased to announce that he
> > has
> > > >  accepted.
> > > >
> > > > Welcome, Deepak!
> > > >
> > > > Thanks,
> > > >  Ashutosh
> > > >
> > >
> >
>

New year, new role doing Hive docs

[Lefty Leverenz]

As of today, I'm not monitoring the Hive mailing lists for doc issues.
Instead, I'll work on the backlog of doc tasks.

This means that I won't add TODOC labels to new commits, and won't write
doc notes with links to wikidocs that need to be updated.

Some developers label their own jiras with TODOC labels and take care of
the documentation, but many do not.  If you have any ideas about how to
optimize the doc process, please participate in the dev@hive thread [DISCUSS]
Do-it-yourself docs

.

If you need to get my attention and I miss the usual Hive email message,
please contact me directly at lef...@apache.org.

-- Lefty

Re: [DISCUSS] Do-it-yourself docs

[Lefty Leverenz]

We have three goals with just a few tools of persuasion.

Goals:

   1. Identify JIRA issues that need documentation.
   2. Add doc notes or release notes to JIRA issues.
   3. Document Hive code and procedures in the wiki.  *(main goal)*

I've been helping with #1 and #2, adding ~17 TODOC labels & doc notes a
month, but soon I'll stop that and focus on #3.

Tools:

   - TODOC labels
   - Yetus checks by Hive QA (?)
   - JIRA subtasks (?)
   - Education:  mailing list, wiki (How to Contribute
   <https://cwiki.apache.org/confluence/display/Hive/HowToContribute>, How
   to Commit
   
<https://cwiki.apache.org/confluence/display/Hive/HowToCommit#HowToCommit-Commit>
   )
   - Nudging:  JIRA comments

-- Lefty


On Mon, Dec 4, 2017 at 10:59 AM, Eugene Koifman <ekoif...@hortonworks.com>
wrote:

> Perhaps this should be a 2 stage process.  One to approve the code and one
> to approve the doc.
> It seems odd to update the Wiki (which isn’t tracked using the same Git
> repo as the code) before
> the code changes have been agreed to.  Both approvals would be required to
> commit.
>
> Eugene
>
>
> On 12/3/17, 2:49 PM, "Prasanth Jayachandran" <j.prasant...@gmail.com>
> wrote:
>
> +1 for Yetus integration to -1 patches without docs.
>
>
> Thanks and Regards,
> Prasanth Jayachandran
>
>
> On Sat, Dec 2, 2017 at 3:04 AM, Klára Barna Zsombor <
> zsomb...@gmail.com>
> wrote:
>
> > Could this be somehow integrated into the Yetus checks? I'm thinking
> that
> > if the Jira being tested does not have one of the "Doc-Performed",
> > "To-Doc", "Doc-Not-Needed" labels then it would get a -1 from Yetus.
> > Peter what do you think? Is Yetus extendable in this way?
> >
> > On Thu, Nov 30, 2017 at 2:58 AM, Lefty Leverenz <
> leftylever...@gmail.com>
> > wrote:
> >
> > > Hive contributors are responsible for documenting their own
> commits,
> > > although many seem to be unaware of this or too busy with other
> tasks.
> > How
> > > can we boost the number of jiras that get documented?
> > >
> > >
> > > Our current process is to put a TODOC** label on each
> committed
> > > issue that needs wiki documentation, then remove it when the doc
> is done.
> > > But nobody tallies the TODOC labels at release time or pressures
> > > contributors to do their documentation, so we have a large backlog
> of
> > > unfinished doc tasks.
> > >
> > >
> > > For several years I've monitored the dev@hive mailing list for
> issues
> > that
> > > should be documented in the wiki.  Whenever a committed patch
> needs doc
> > and
> > > the contributor hasn't taken care of it, I add a TODOC label and
> write a
> > > doc note naming new configuration parameters, reserved words, or
> HiveQL
> > > syntax.  (This is convenient for searches.)  I also give links to
> places
> > in
> > > the wiki where the docs belong.
> > >
> > >
> > > Soon, I'll stop monitoring the Hive mailing lists and writing doc
> notes.
> > > My time can be better spent doing documentation, instead of just
> pointing
> > > out that it needs to be done.  But I can't tackle the whole
> backlog, and
> > > many future commits won't even get a TODOC label.
> > >
> > >
> > > What can we do to improve the Hive doc process?
> > >
> > > -- Lefty
> > >
> >
>
>
>

[DISCUSS] Do-it-yourself docs

[Lefty Leverenz]

Hive contributors are responsible for documenting their own commits,
although many seem to be unaware of this or too busy with other tasks.  How
can we boost the number of jiras that get documented?


Our current process is to put a TODOC** label on each committed
issue that needs wiki documentation, then remove it when the doc is done.
But nobody tallies the TODOC labels at release time or pressures
contributors to do their documentation, so we have a large backlog of
unfinished doc tasks.


For several years I've monitored the dev@hive mailing list for issues that
should be documented in the wiki.  Whenever a committed patch needs doc and
the contributor hasn't taken care of it, I add a TODOC label and write a
doc note naming new configuration parameters, reserved words, or HiveQL
syntax.  (This is convenient for searches.)  I also give links to places in
the wiki where the docs belong.


Soon, I'll stop monitoring the Hive mailing lists and writing doc notes.
My time can be better spent doing documentation, instead of just pointing
out that it needs to be done.  But I can't tackle the whole backlog, and
many future commits won't even get a TODOC label.


What can we do to improve the Hive doc process?

-- Lefty

Re: [Announce] New committer: Tao Li

[Lefty Leverenz]

Congratulations Tao!

-- Lefty


On Fri, Nov 3, 2017 at 1:18 PM, Vihang Karajgaonkar 
wrote:

> Congratulations Tao!
>
> On Fri, Nov 3, 2017 at 10:12 AM, Jesus Camacho Rodriguez <
> jcama...@apache.org> wrote:
>
> > Congrats Tao!
> >
> >
> >
> >
> > On 11/3/17, 8:32 AM, "Sergio Pena"  wrote:
> >
> > >Congratulations Tao !!!
> > >
> > >On Wed, Nov 1, 2017 at 2:13 PM, Peter Vary  wrote:
> > >
> > >> Congratulations Tao! :)
> > >>
> > >> > On Nov 1, 2017, at 6:49 PM, Ashutosh Chauhan 
> > >> wrote:
> > >> >
> > >> > The Project Management Committee (PMC) for Apache Hive has invited
> Tao
> > >> Li to
> > >> > become a committer and we are pleased to announce that he has
> > accepted.
> > >> >
> > >> > Welcome, Tao!
> > >> >
> > >> > Thanks,
> > >> > Ashutosh
> > >>
> > >>
> >
> >
>

Re: Review Request 63427: HIVE-17936

[Lefty Leverenz]

> On Nov. 1, 2017, 3:48 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 3017-3018 (patched)
> > <https://reviews.apache.org/r/63427/diff/2/?file=1873282#file1873282line3017>
> >
> > Please spell out the acronyms TS, DPP, and nDVs in the parameter 
> > description.
> > 
> > Also, the description isn't clear to me -- "to control if ..." what?  
> > Something about reduction, or feeding?  (Forgive my ignorance, I'm just 
> > looking at the grammatical structure of the sentence along with the 
> > parameter name.)
> 
> Deepak Jaiswal wrote:
> Thanks for the feedback. How about this below?
> 
> The factor to decide if semijoin branch feeding into a TableScan which 
> has an outgoing Dynamic Partition pruning(DPP) branch based on number of 
> distinct values.
> 
> Lefty Leverenz wrote:
> Is "feeding" what gets decided?  (In other words, should it be "feeds"?)  
> If not, what's the verb that "decide if" refers to?
> 
> Thanks for spelling out the acronyms.  Keeping DPP in parentheses is 
> good, although you need a space before the opening parenthesis.  And maybe 
> "pruning" should be "Pruning" just for consistency.
> 
> Deepak Jaiswal wrote:
> Thanks. Let me update the patch.

Looks good, thanks again.

+1 (just for the parameter description)


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63427/#review189778
---


On Nov. 2, 2017, 8:45 p.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63427/
> ---
> 
> (Updated Nov. 2, 2017, 8:45 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Jason Dere.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Dynamic Semijoin Reduction : markSemiJoinForDPP marks unwanted semijoin 
> branches
> 
> In method markSemiJoinForDPP (HIVE-17399), the nDVs comparison should not 
> have equality as there is a chance that the values are same on both sides and 
> the branch is still marked as good when it shouldn't be.
> Add a configurable factor to see how useful this is if nDVs on smaller side 
> are only slightly less than that on TS side.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6631a6e45d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java da30c3b642 
>   ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 6cc0a7f7a9 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 
> 1a1a4d9b2d 
> 
> 
> Diff: https://reviews.apache.org/r/63427/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Re: Review Request 63427: HIVE-17396

[Lefty Leverenz]

> On Nov. 1, 2017, 3:48 a.m., Lefty Leverenz wrote:
> >

Nit:  This RB summary has the wrong JIRA number -- it should be 17936, not 
17396.

But the first line of the description has the correct JIRA text, so it's easy 
enough to find with a search.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63427/#review189778
---


On Oct. 31, 2017, 1:19 a.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63427/
> ---
> 
> (Updated Oct. 31, 2017, 1:19 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Jason Dere.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Dynamic Semijoin Reduction : markSemiJoinForDPP marks unwanted semijoin 
> branches
> 
> In method markSemiJoinForDPP (HIVE-17399), the nDVs comparison should not 
> have equality as there is a chance that the values are same on both sides and 
> the branch is still marked as good when it shouldn't be.
> Add a configurable factor to see how useful this is if nDVs on smaller side 
> are only slightly less than that on TS side.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6631a6e45d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java da30c3b642 
>   ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 6cc0a7f7a9 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 
> 1a1a4d9b2d 
> 
> 
> Diff: https://reviews.apache.org/r/63427/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Re: Review Request 63427: HIVE-17396

[Lefty Leverenz]

> On Nov. 1, 2017, 3:48 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 3017-3018 (patched)
> > <https://reviews.apache.org/r/63427/diff/2/?file=1873282#file1873282line3017>
> >
> > Please spell out the acronyms TS, DPP, and nDVs in the parameter 
> > description.
> > 
> > Also, the description isn't clear to me -- "to control if ..." what?  
> > Something about reduction, or feeding?  (Forgive my ignorance, I'm just 
> > looking at the grammatical structure of the sentence along with the 
> > parameter name.)
> 
> Deepak Jaiswal wrote:
> Thanks for the feedback. How about this below?
> 
> The factor to decide if semijoin branch feeding into a TableScan which 
> has an outgoing Dynamic Partition pruning(DPP) branch based on number of 
> distinct values.

Is "feeding" what gets decided?  (In other words, should it be "feeds"?)  If 
not, what's the verb that "decide if" refers to?

Thanks for spelling out the acronyms.  Keeping DPP in parentheses is good, 
although you need a space before the opening parenthesis.  And maybe "pruning" 
should be "Pruning" just for consistency.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63427/#review189778
---


On Oct. 31, 2017, 1:19 a.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63427/
> ---
> 
> (Updated Oct. 31, 2017, 1:19 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Jason Dere.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Dynamic Semijoin Reduction : markSemiJoinForDPP marks unwanted semijoin 
> branches
> 
> In method markSemiJoinForDPP (HIVE-17399), the nDVs comparison should not 
> have equality as there is a chance that the values are same on both sides and 
> the branch is still marked as good when it shouldn't be.
> Add a configurable factor to see how useful this is if nDVs on smaller side 
> are only slightly less than that on TS side.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6631a6e45d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java da30c3b642 
>   ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 6cc0a7f7a9 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 
> 1a1a4d9b2d 
> 
> 
> Diff: https://reviews.apache.org/r/63427/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Re: Review Request 63427: HIVE-17396

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63427/#review189778
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3017-3018 (patched)
<https://reviews.apache.org/r/63427/#comment267005>

Please spell out the acronyms TS, DPP, and nDVs in the parameter 
description.

Also, the description isn't clear to me -- "to control if ..." what?  
Something about reduction, or feeding?  (Forgive my ignorance, I'm just looking 
at the grammatical structure of the sentence along with the parameter name.)


- Lefty Leverenz


On Oct. 31, 2017, 1:19 a.m., Deepak Jaiswal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63427/
> ---
> 
> (Updated Oct. 31, 2017, 1:19 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Jason Dere.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Dynamic Semijoin Reduction : markSemiJoinForDPP marks unwanted semijoin 
> branches
> 
> In method markSemiJoinForDPP (HIVE-17399), the nDVs comparison should not 
> have equality as there is a chance that the values are same on both sides and 
> the branch is still marked as good when it shouldn't be.
> Add a configurable factor to see how useful this is if nDVs on smaller side 
> are only slightly less than that on TS side.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6631a6e45d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java da30c3b642 
>   ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 6cc0a7f7a9 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 
> 1a1a4d9b2d 
> 
> 
> Diff: https://reviews.apache.org/r/63427/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Deepak Jaiswal
> 
>

Re: [Announce] New committer: Sankar Hariappan

[Lefty Leverenz]

Congrats Sankar!

-- Lefty


On Sat, Sep 30, 2017 at 9:57 PM, Vihang Karajgaonkar 
wrote:

> Congrats Sankar!
>
> On Sat, Sep 30, 2017 at 7:51 AM, Sushanth Sowmyan 
> wrote:
>
> > Welcome aboard! :)
> >
> > On Sep 30, 2017 3:28 AM, "Barna Zsombor Klara" <
> zsombor.kl...@cloudera.com
> > >
> > wrote:
> >
> > Congrats Sankar!
> >
> > Rajesh Balamohan  (időpont: 2017. szept. 30.,
> Szo,
> > 2:24) ezt írta:
> >
> > > Congrats Sankar!!
> > >
> > > ~Rajesh.B
> > >
> > > On Sat, Sep 30, 2017 at 4:30 AM, Vaibhav Gumashta <
> > > vgumas...@hortonworks.com
> > > > wrote:
> > >
> > > > Congratulations Sankar!
> > > >
> > > > On 9/29/17, 3:58 PM, "Thejas Nair"  wrote:
> > > >
> > > > >Congrats Sankar!
> > > > >
> > > > >On Fri, Sep 29, 2017 at 11:36 AM, Peter Vary 
> > > wrote:
> > > > >
> > > > >> Congratulations Sankar!
> > > > >>
> > > > >> > On Sep 29, 2017, at 7:56 PM, Ashutosh Chauhan <
> > hashut...@apache.org
> > > >
> > > > >> wrote:
> > > > >> >
> > > > >> > The Project Management Committee (PMC) for Apache Hive has
> invited
> > > > >>Sankar
> > > > >> > Harriapan to become a committer and we are pleased to announce
> > that
> > > he
> > > > >> has
> > > > >> > accepted.
> > > > >> >
> > > > >> > Welcome, Sankar!
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Ashutosh
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: [Announce] New committer: Anishek Agarwal

[Lefty Leverenz]

Congrats Anishek!

-- Lefty


On Sat, Sep 30, 2017 at 9:57 PM, Vihang Karajgaonkar 
wrote:

> Congrats Anishek!
>
> On Sat, Sep 30, 2017 at 7:51 AM, Sushanth Sowmyan 
> wrote:
>
> > Welcome aboard! :)
> >
> > On Sep 30, 2017 3:27 AM, "Barna Zsombor Klara" <
> zsombor.kl...@cloudera.com
> > >
> > wrote:
> >
> > > Congratulations Anishek!
> > >
> > > Rajesh Balamohan  (időpont: 2017. szept. 30.,
> > Szo,
> > > 2:25) ezt írta:
> > >
> > > > Congrats Anishek!!
> > > >
> > > > ~Rajesh.B
> > > >
> > > > On Sat, Sep 30, 2017 at 4:30 AM, Vaibhav Gumashta <
> > > > vgumas...@hortonworks.com
> > > > > wrote:
> > > >
> > > > > Congratulations Anishek!
> > > > >
> > > > >
> > > > > On 9/29/17, 3:57 PM, "Thejas Nair"  wrote:
> > > > >
> > > > > >Congrats Anishek!
> > > > > >
> > > > > >On Fri, Sep 29, 2017 at 11:36 AM, Peter Vary 
> > > > wrote:
> > > > > >
> > > > > >> Congratulations Anishek!
> > > > > >>
> > > > > >> > On Sep 29, 2017, at 7:55 PM, Ashutosh Chauhan <
> > > hashut...@apache.org
> > > > >
> > > > > >> wrote:
> > > > > >> >
> > > > > >> > The Project Management Committee (PMC) for Apache Hive has
> > invited
> > > > > >> Anishek
> > > > > >> > Agarwal to become a committer and we are pleased to announce
> > that
> > > he
> > > > > >>has
> > > > > >> > accepted.
> > > > > >> >
> > > > > >> > Welcome, Anishek!
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Ashutosh
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: New committer : Barna Zsombor Klara

[Lefty Leverenz]

My congratulations also!

-- Lefty


On Tue, Sep 19, 2017 at 4:03 AM, Barna Zsombor Klara <
zsombor.kl...@cloudera.com> wrote:

> Thank you everyone! :)
>
> On Tue, Sep 19, 2017 at 7:41 AM, Vaibhav Gumashta <
> vgumas...@hortonworks.com
> > wrote:
>
> > Congratulations Zsombor!
> > 
> > From: Prasanth Jayachandran 
> > Sent: Monday, September 18, 2017 5:52 PM
> > To: dev@hive.apache.org
> > Subject: Re: New committer : Barna Zsombor Klara
> >
> > Congratulations Zsombor!!
> >
> > Thanks
> > Prasanth
> > > On Sep 18, 2017, at 5:50 PM, Chetna C  wrote:
> > >
> > > Congratulations Zsombor!
> > >
> > > Thanks,
> > > Chetna Chaudhari
> > >
> > > On 19 September 2017 at 10:37, Sergio Pena 
> > wrote:
> > >
> > >> Great, congratulations Zsombor !!!
> > >>
> > >> On Mon, Sep 18, 2017 at 12:44 PM, Sahil Takiar <
> takiar.sa...@gmail.com>
> > >> wrote:
> > >>
> > >>> Congrats Zsombor!
> > >>>
> > >>> On Mon, Sep 18, 2017 at 9:50 AM, Vihang Karajgaonkar <
> > >> vih...@cloudera.com>
> > >>> wrote:
> > >>>
> >  Congratulations Zsombor!
> > 
> >  On Mon, Sep 18, 2017 at 8:15 AM, Zoltan Haindrich <
> >  zhaindr...@hortonworks.com> wrote:
> > 
> > > Congratulations!!
> > >
> > > On 16 Sep 2017 00:52, Ashutosh Chauhan 
> wrote:
> > > The Project Management Committee (PMC) for Apache Hive has invited
> > >>> Barna
> > > Zsombor Klara to become a committer and we are pleased to announce
> > >> that
> >  he
> > > has accepted.
> > >
> > > Barna, thank you for your contributions, and we look forward your
> > > further interactions
> > > with the community!
> > >
> > > Welcome, Barna!
> > >
> > > Thanks,
> > > Ashutosh (on behalf of Apache Hive PMC)
> > >
> > >
> > 
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Sahil Takiar
> > >>> Software Engineer at Cloudera
> > >>> takiar.sa...@gmail.com | (510) 673-0309
> > >>>
> > >>
> >
> >
> >
> >
>

Re: [DISCUSS] Separating out the metastore as its own TLP

[Lefty Leverenz]

Johndee (and everyone else), wiki edit privileges are easy to get:  About
This Wiki -- How to get permission to edit
<https://cwiki.apache.org/confluence/display/Hive/AboutThisWiki#AboutThisWiki-Howtogetpermissiontoedit>
.

-- Lefty


On Thu, Jul 27, 2017 at 2:32 PM, Johndee Cloudera <john...@cloudera.com>
wrote:

> Well if I cannot get Metastore, how about Hadoop Metastore it is simple and
> self explanatory to a degree.
>
> @Alan,
>
> Sorry to make the name suggestion here but I could not comment or edit the
> page you created.
>
> On Mon, Jul 24, 2017 at 7:17 PM, Gopal Vijayaraghavan <gop...@apache.org>
> wrote:
>
> > Hi,
> >
> >
> > Changing the name isn't really optional or "being google-able" [2].
> >
> > The naming is a crucial part of trademark protection [1], which is the
> > only protection ASF has against hostile Embrace & Extends.
> >
> > Fragmented forks with the same name is particularly bad, especially if
> the
> > feature in question can be only used by a proprietary tool (like Dain's
> > suggestion about Presto view metadata, except it only works with a
> per-cpu
> > license).
> >
> > The safe path isn't pretty, it still ends up with IcedTea and IceWeasel …
> > but at least, those are clearly weird.
> >
> > Cheers,
> > Gopal
> >
> > [1] - https://en.wikipedia.org/wiki/A_moron_in_a_hurry#United_States
> > [2] - https://packages.debian.org/jessie/misc/metastore
> >
> > On 7/24/17, 3:04 PM, "Carl Steinbach" <cwsteinb...@gmail.com> wrote:
> >
> > +1 to Vihang's suggestion. Changing the name will only cause
> confusion.
> >
> > On Mon, Jul 24, 2017 at 2:28 PM, Johndee Cloudera <
> > john...@cloudera.com>
> > wrote:
> >
> > > +1 Vihang, I do not really like Catalog as it could create
> confusion
> > with
> > > the Catalog daemon from impala.
> > >
> > > On Mon, Jul 24, 2017 at 5:20 PM, Vihang Karajgaonkar <
> > vih...@cloudera.com>
> > > wrote:
> > >
> > > > Before we see a flood of name suggestions :) Why not just keep it
> > > > Metastore? Its already well-known in the community and easy to
> > relate to.
> > > >
> > > > On Mon, Jul 24, 2017 at 2:13 PM, Alan Gates <
> alanfga...@gmail.com>
> > > wrote:
> > > >
> >     > > > In the same vein Carter and Gunther suggested Omegastore.  Pick
> > your
> > > > > alphabet and whether it’s a catalog or a store I guess.
> > > > >
> > > > > Alan.
> > > > >
> > > > > On Mon, Jul 24, 2017 at 1:35 PM, Sergey Shelukhin <
> > > > ser...@hortonworks.com>
> > > > > wrote:
> > > > >
> > > > > > I’d like to suggest ZCatalog.
> > > > > >
> > > > > > On 17/7/11, 15:41, "Lefty Leverenz" <leftylever...@gmail.com
> >
> > wrote:
> > > > > >
> > > > > > >>> I'd like to suggest Riven.  (Owen O'Malley)
> > > > > > >
> > > > > > >> How about "Flora"?  (Andrew Sherman)
> > > > > > >
> > > > > > >Nice idea and thanks for introducing me to that book,
> Andrew.
> > > > > > >
> > > > > > >Along the same lines, how about "Honeycomb"?
> > > > > > >
> > > > > > >But since the idea is to make the metastore useful for many
> > > projects,
> > > > a
> > > > > > >generic name that starts with "Meta" would be less confusing
> > ...
> > > even
> > > > > > >though it breaks the tradition of Apache projects having
> > quirky
> > > names.
> > > > > > >Unfortunately "Metalog" is already in use.  "Metamorph" has
> > other
> > > > > > >connotations, but it's cool.
> > > > > > >
> > > > > > >Naming enthusiasm notwithstanding, I'm +/-0 on the idea of
> > splitting
> > > > off
> > > > > > >the metastore into a new project:  -0.5 for the sake of Hive
> > and
> > > +0.5
> > > > > for
> > > > > > >the greater good.  Wishy-washy, that's me.
> > > > > > >
> > > > > > >-- Lefty
> > > > > > >
> > > > > > >
> > > > > > >On Tue, Jul 11, 2017 at 1:04 PM, Andrew Sherman <
> > > > asher...@cloudera.com>
> > > > > > >wrote:
> > > > > > >
> > > > > > >> On Fri, Jun 30, 2017 at 5:05 PM, Owen O'Malley <
> > > > > owen.omal...@gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > On Fri, Jun 30, 2017 at 3:26 PM, Chao Sun <
> > sunc...@apache.org>
> > > > > wrote:
> > > > > > >> >
> > > > > > >> > > and maybe a different project name?
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> > Yes, it certainly needs a new name. I'd like to suggest
> > Riven.
> > > > > > >> >
> > > > > > >> > .. Owen
> > > > > > >> >
> > > > > > >>
> > > > > > >> How about "Flora"?
> > > > > > >>
> > > > > > >> (Flora is the protagonist of The Bees by Laline Paull)
> > > > > > >>
> > > > > > >> -Andrew
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > - JRB
> > >
> >
> >
> >
> >
>
>
> --
> - JRB
>

Re: [ANNOUNCE] Apache Hive 2.3.0 Released

[Lefty Leverenz]

Congratulations to all who contributed to release 2.3.0!

A few jiras still need to be documented in the wiki:


   1. HIVE-14233   Improve
   vectorization for ACID by eliminating row-by-row stitching  (Saket Saurabh)
   2. HIVE-15207   Implement
   a capability to detect incorrect sequence numbers  (Aihua Xu)
   3. HIVE-15434   Add
   UDF to allow interrogation of uniontype values  (David Maughan)
   4. HIVE-15805   Some
   minor improvement on the validation tool  (Aihua Xu)
   5. HIVE-15880   Allow
   insert overwrite and truncate table query to use auto.purge table property
(Vihang Karajgaonkar)
   6. HIVE-16072   LLAP:
   Add some additional jvm metrics for hadoop-metrics2  (Prasanth Jayachandran)
   7. HIVE-16260   Remove
   parallel edges of semijoin with map joins.  (Deepak Jaiswal)
   8. HIVE-16274   Support
   tuning of NDV of columns using lower/upper bounds  (Pengcheng Xiong)
   9. HIVE-16321   Possible
   deadlock in metastore with Acid enabled  (Eugene Koifman)

In honor of the Tour de France, everyone who documents one of these jiras
within one week qualifies as a member of the peloton
.  Don't get
left behind!  Here's how to get permission to edit the wiki

.

-- Lefty


On Tue, Jul 18, 2017 at 8:49 PM, Pengcheng Xiong  wrote:

> The Apache Hive team is proud to announce the release of Apache Hive
> version 2.3.0.
>
> The Apache Hive (TM) data warehouse software facilitates querying and
> managing large datasets residing in distributed storage. Built on top
> of Apache Hadoop (TM), it provides, among others:
>
> * Tools to enable easy data extract/transform/load (ETL)
>
> * A mechanism to impose structure on a variety of data formats
>
> * Access to files stored either directly in Apache HDFS (TM) or in other
>   data storage systems such as Apache HBase (TM)
>
> * Query execution via Apache Hadoop MapReduce and Apache Tez frameworks.
>
> For Hive release details and downloads, please
> visit:https://hive.apache.org/downloads.html
>
> Hive 2.3.0 Release Notes are available here:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> version=12340269=Text=12310843
>
> We would like to thank the many contributors who made this release
> possible.
>
> Regards,
>
> The Apache Hive Team
>

Re: [VOTE] Apache Hive 2.3.0 Release Candidate 1

[Lefty Leverenz]

Is the list of jiras compiled from github or from the "Fix Version/s" field
in JIRA?

If it's a Fix Version/s list, it could be incomplete because many jiras
were originally set to fix version 2.2.0 and ended up in branch-2.3 but not
branch-2.2.  Some of them have been updated, but probably not all.

-- Lefty


On Fri, Jul 14, 2017 at 5:47 PM, Gunther Hagleitner <
ghagleit...@hortonworks.com> wrote:

> I'm +1. Checked signature and checksum, verified binaries, compiled, ran
> some tests and queries - didn't find any issues.
>
> The concern seems to be that we don't release often enough - rebasing the
> branch and re-stabilizing would just delay having a release further. We can
> always put a new one together, rather than scrapping this one.
>
> Thanks,
> Gunther.
> 
> From: Eugene Koifman 
> Sent: Friday, July 14, 2017 10:18 AM
> To: dev@hive.apache.org
> Subject: Re: [VOTE] Apache Hive 2.3.0 Release Candidate 1
>
> Another issue is that the branch for 2.3 was cut 2 or 3 month ago.  Given
> how infrequently releases are made, would it not make more sense to recut
> it at this point?
>
> On 7/14/17, 8:20 AM, "Sergio Pena"  wrote:
>
> What happened with the 2.2.0 release? is it confusing to jump from 2.1
> ->
> 2.3 without having a 2.2 release previously?
>
> On Fri, Jul 14, 2017 at 2:02 AM, Pengcheng Xiong 
> wrote:
>
> > Apache Hive 2.3.0 Release Candidate 1 is available here:
> >
> > Artifacts:
> > tag: *https://github.com/apache/hive/releases/tag/release-2.3.0-rc1
> > *
> > tar ball: http://home.apache.org/~pxiong/apache-hive-2.3.0
> > 
> >
> > Voting will conclude in 72 hours.
> >
> > Hive PMC Members: Please test and vote.
> >
> > Here is my +1 after running rat check, md5 check and simple queries.
> >
> > Thanks.
> >
> > Best
> > Pengcheng
> >
>
>
>
>
>

Re: Review Request 60753: Add HLL as an alternative to FM sketch to compute stats

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60753/#review180255
---



Typo:  The RB description says "HIVE-16966" but it should be HIVE-16996.

- Lefty Leverenz


On July 10, 2017, 9:29 p.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60753/
> ---
> 
> (Updated July 10, 2017, 9:29 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16966
> 
> 
> Diffs
> -
> 
>   common/pom.xml e6722babd8 
>   common/src/java/org/apache/hadoop/hive/common/HiveStatsUtils.java 
> 7c9d72fbd2 
>   
> common/src/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimator.java
>  PRE-CREATION 
>   
> common/src/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimatorFactory.java
>  PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLConstants.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLDenseRegister.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLRegister.java 
> PRE-CREATION 
>   
> common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLSparseRegister.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLogUtils.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5700fb9325 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java.orig da48a7ccbd 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/NumDistinctValueEstimator.java
>  92f9a845e3 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java
>  36b2c9c56b 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java
>  a88ef84e5c 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java
>  8ac6561aec 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java
>  2aa4046a46 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMerger.java
>  33c7e3e52c 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
>  fe890e4e27 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
>  3179b23438 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DecimalColumnStatsMerger.java
>  c13add9d9c 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DoubleColumnStatsMerger.java
>  fbdba24b0a 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/LongColumnStatsMerger.java
>  ac65590505 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/StringColumnStatsMerger.java
>  41587477d3 
>   pom.xml f9fae59a5d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/DecimalNumDistinctValueEstimator.java
>  a05906edfa 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/DoubleNumDistinctValueEstimator.java
>  e76fc74dbc 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java
>  2ebfcb2360 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/LongNumDistinctValueEstimator.java
>  1c197a028a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/NumDistinctValueEstimator.java
>  fa70f49857 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/StringNumDistinctValueEstimator.java
>  601901c163 
>   ql/src/test/queries/clientpositive/hll.q PRE-CREATION 
>   ql/src/test/results/clientpositive/hll.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60753/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>

Re: [DISCUSS] Separating out the metastore as its own TLP

[Lefty Leverenz]

>> I'd like to suggest Riven.  (Owen O'Malley)

> How about "Flora"?  (Andrew Sherman)

Nice idea and thanks for introducing me to that book, Andrew.

Along the same lines, how about "Honeycomb"?

But since the idea is to make the metastore useful for many projects, a
generic name that starts with "Meta" would be less confusing ... even
though it breaks the tradition of Apache projects having quirky names.
Unfortunately "Metalog" is already in use.  "Metamorph" has other
connotations, but it's cool.

Naming enthusiasm notwithstanding, I'm +/-0 on the idea of splitting off
the metastore into a new project:  -0.5 for the sake of Hive and +0.5 for
the greater good.  Wishy-washy, that's me.

-- Lefty


On Tue, Jul 11, 2017 at 1:04 PM, Andrew Sherman 
wrote:

> On Fri, Jun 30, 2017 at 5:05 PM, Owen O'Malley 
> wrote:
>
> > On Fri, Jun 30, 2017 at 3:26 PM, Chao Sun  wrote:
> >
> > > and maybe a different project name?
> > >
> >
> > Yes, it certainly needs a new name. I'd like to suggest Riven.
> >
> > .. Owen
> >
>
> How about "Flora"?
>
> (Flora is the protagonist of The Bees by Laline Paull)
>
> -Andrew
>

Re: [Announce] New committer: Peter Vary

[Lefty Leverenz]

Congrats!

-- Lefty

On Fri, Jul 7, 2017 at 5:04 PM, Chaoyu Tang  wrote:

> Congratulations Peter!
>
> On Fri, Jul 7, 2017 at 3:29 PM, Jimmy Xiang  wrote:
>
> > Congrats!!
> >
> > On Fri, Jul 7, 2017 at 11:38 AM, Sergio Pena 
> > wrote:
> > > Congrats Peter !!
> > >
> > > On Fri, Jul 7, 2017 at 12:51 PM, Vaibhav Gumashta <
> > vgumas...@hortonworks.com
> > >> wrote:
> > >
> > >> Congratulations Peter!
> > >>
> > >> On 7/7/17, 10:46 AM, "Vineet Garg"  wrote:
> > >>
> > >> >Congrats Peter!
> > >> >
> > >> >> On Jul 7, 2017, at 9:27 AM, Xuefu Zhang  wrote:
> > >> >>
> > >> >> Congratulations!
> > >> >>
> > >> >> On Fri, Jul 7, 2017 at 4:17 AM, Adam Szita 
> > wrote:
> > >> >>
> > >> >>> Congrats all!
> > >> >>>
> > >> >>> On 7 July 2017 at 10:03, Zoltan Haindrich <
> > zhaindr...@hortonworks.com>
> > >> >>> wrote:
> > >> >>>
> > >>  Congratulations Peter, Teddy, Deepesh, Vihang and Sahil!
> > >>  It's great to see that the Hive community is growing!
> > >> 
> > >>  On 6 Jul 2017 02:52, Ashutosh Chauhan 
> > wrote:
> > >>  The Project Management Committee (PMC) for Apache Hive has
> invited
> > >> Peter
> > >>  Vary to become a committer and we are pleased to announce that he
> > has
> > >>  accepted.
> > >> 
> > >>  Welcome, Peter!
> > >> 
> > >>  Thanks,
> > >>  Ashutosh
> > >> 
> > >> 
> > >> >>>
> > >> >
> > >>
> > >>
> >
>

Re: [Announce] New committer: Deepesh Khandelwal

[Lefty Leverenz]

Congratulations!

-- Lefty

On Fri, Jul 7, 2017 at 5:04 PM, Chaoyu Tang  wrote:

> Congratulations Deepesh!
>
> On Fri, Jul 7, 2017 at 4:07 PM, Peter Vary  wrote:
>
> > Congratulations Deepesh!
> >
> > > On Jul 7, 2017, at 8:38 PM, Sergio Pena 
> > wrote:
> > >
> > > Congratulations Deepesh !!!
> > >
> > > On Wed, Jul 5, 2017 at 7:48 PM, Ashutosh Chauhan  >
> > > wrote:
> > >
> > >> The Project Management Committee (PMC) for Apache Hive has invited
> > Deepesh
> > >> Khandelwal to become a committer and we are pleased to announce that
> he
> > has
> > >> accepted.
> > >>
> > >> Welcome, Deepesh!
> > >>
> > >> Thanks,
> > >> Ashutosh
> > >>
> >
> >
>

Re: [Announce] New committer: Teddy Choi

[Lefty Leverenz]

Congrats!

-- Lefty

On Fri, Jul 7, 2017 at 5:05 PM, Chaoyu Tang  wrote:

> Congratulations Teddy!
>
> On Fri, Jul 7, 2017 at 4:07 PM, Peter Vary  wrote:
>
> > Congratulatins Teddy!
> >
> > > On Jul 7, 2017, at 9:28 PM, Jimmy Xiang  wrote:
> > >
> > > Congrats!!
> > >
> > > On Fri, Jul 7, 2017 at 11:37 AM, Sergio Pena  >
> > wrote:
> > >> Congratulations Teddy!
> > >>
> > >> On Fri, Jul 7, 2017 at 12:51 PM, Vaibhav Gumashta <
> > vgumas...@hortonworks.com
> > >>> wrote:
> > >>
> > >>> Congratulations Teddy!
> > >>>
> > >>> On 7/7/17, 10:46 AM, "Vineet Garg"  wrote:
> > >>>
> >  Congratulations!
> > 
> > > On Jul 7, 2017, at 10:25 AM, Gunther Hagleitner
> > >  wrote:
> > >
> > > Congrats Teddy!
> > > 
> > > From: Xuefu Zhang 
> > > Sent: Friday, July 07, 2017 9:26 AM
> > > To: dev@hive.apache.org
> > > Subject: Re: [Announce] New committer: Teddy Choi
> > >
> > > Congratulations!
> > >
> > > On Fri, Jul 7, 2017 at 12:09 AM, Matthew McCline
> > > 
> > > wrote:
> > >
> > >> Congratulations Teddy!
> > >>
> > >> Get Outlook for iOS
> > >>
> > >>
> > >>
> > >> On Wed, Jul 5, 2017 at 5:53 PM -0700, "Ashutosh Chauhan" <
> > >> hashut...@apache.org> wrote:
> > >>
> > >>
> > >> The Project Management Committee (PMC) for Apache Hive has invited
> > >> Teddy
> > >> Choi to become a committer and we are pleased to announce that he
> > has
> > >> accepted.
> > >>
> > >> Welcome, Teddy!
> > >>
> > >> Thanks,
> > >> Ashutosh
> > >>
> > >>
> > >
> > >
> > 
> > >>>
> > >>>
> >
> >
>

Re: [Announce] New committer: Sahil Takiar

[Lefty Leverenz]

Congratulations!

-- Lefty

On Fri, Jul 7, 2017 at 5:05 PM, Chaoyu Tang  wrote:

> Congratulations Sahil !!!
>
> On Fri, Jul 7, 2017 at 2:38 PM, Sergio Pena 
> wrote:
>
> > Congratulations Sahil !!!
> >
> > On Wed, Jul 5, 2017 at 7:49 PM, Ashutosh Chauhan 
> > wrote:
> >
> > > The Project Management Committee (PMC) for Apache Hive has invited
> Sahil
> > > Takiar to become a committer and we are pleased to announce that he has
> > > accepted.
> > >
> > > Welcome, Sahil!
> > >
> > > Thanks,
> > > Ashutosh
> > >
> >
>

Re: [Announce] New committer: Vihang Karajgaonkar

[Lefty Leverenz]

Congratulations!

-- Lefty

On Fri, Jul 7, 2017 at 5:06 PM, Chaoyu Tang  wrote:

> Congratulations Vihang !!!.
>
> On Fri, Jul 7, 2017 at 2:38 PM, Sergio Pena 
> wrote:
>
> > Congratulations Vihang !!!.
> >
> > - Sergio
> >
> > On Fri, Jul 7, 2017 at 12:50 PM, Vaibhav Gumashta <
> > vgumas...@hortonworks.com
> > > wrote:
> >
> > > Congratulations Vihang!
> > >
> > > On 7/7/17, 10:47 AM, "Vineet Garg"  wrote:
> > >
> > > >Congratulations Vihang!
> > > >
> > > >> On Jul 5, 2017, at 5:51 PM, Ashutosh Chauhan 
> > > >>wrote:
> > > >>
> > > >> The Project Management Committee (PMC) for Apache Hive has invited
> > > >>Vihang
> > > >> Karajgaonkar to become a committer and we are pleased to announce
> that
> > > >>he
> > > >> has accepted.
> > > >>
> > > >> Welcome, Vihang!
> > > >>
> > > >> Thanks,
> > > >> Ashutosh
> > > >
> > >
> > >
> >
>

Re: [ANNOUNCE] New PMC Member : Matt McCline

[Lefty Leverenz]

Congratulations Matt!  Well deserved.

-- Lefty

On Thu, Jul 6, 2017 at 11:31 AM, Ashutosh Chauhan 
wrote:

> On behalf of the Hive PMC I am delighted to announce Matt McCline is
> joining Hive PMC.
> Matt is a long time contributor in Hive and is focusing on vectorization
> these days.
>
> Welcome, Matt!
>
> Thanks,
> Ashutosh
>

[jira] [Created] (HIVE-16822) Document how to update the errata.txt file

[Lefty Leverenz (JIRA)]

Lefty Leverenz created HIVE-16822:
-

 Summary: Document how to update the errata.txt file
 Key: HIVE-16822
 URL: https://issues.apache.org/jira/browse/HIVE-16822
 Project: Hive
  Issue Type: Bug
  Components: Documentation
Reporter: Lefty Leverenz


The wiki should explain when and how to update the errata.txt file.

Details belong in How to Commit, and there should be a cross reference in How 
to Contribute so that non-committers will learn about errata.txt.

* [How To Commit -- Commit | 
https://cwiki.apache.org/confluence/display/Hive/HowToCommit#HowToCommit-Commit]
* [How To Contribute -- Contributing Your Work | 
https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-ContributingYourWork]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [Announce] New PMC members

[Lefty Leverenz]

Congratulations to the new PMC members, and also to Hive for having such a
strong and talented community!

-- Lefty


On Fri, May 26, 2017 at 1:12 PM, Eugene Koifman 
wrote:

> Congratualtions!
>
> On 5/26/17, 10:10 AM, "Prasanth Jayachandran" <
> pjayachand...@hortonworks.com> wrote:
>
> Congratulations to all of you!
>
> Thanks
> Prasanth
> > On May 26, 2017, at 10:09 AM, Pengcheng Xiong 
> wrote:
> >
> > Yongzhi, Daniel, Vaibhav, Sergio, Aihua, and Chaoyu!
> >
> > Congratulations to all of you!
> >
> > Best
> > Pengcheng
> >
> > On Thu, May 25, 2017 at 10:26 PM, Peter Vary 
> wrote:
> >
> >> Wow!
> >> That's a spring shower of PMCs. :)
> >> Well deserved Yongzhi, Daniel, Vaibhav, Sergio, Aihua, Chaoyu!
> >>
> >> Congratulations to all of you!
> >>
> >> Peter
> >>
> >> 2017. máj. 26. 6:42 ezt írta ("Ashutosh Chauhan" <
> hashut...@apache.org>):
> >>
> >> The Project Management Committee (PMC) for Apache Hive has invited
> Yongzhi
> >> Chen to become a PMC member and we are pleased to announce that he
> has
> >> accepted.
> >>
> >> Please join me in congratulating Yongzhi!
> >>
> >> Thanks,
> >> Ashutosh on behalf of Hive PMC
> >>
>
>
>
>

Re: Welcome Rui Li to Hive PMC

[Lefty Leverenz]

Congratulations!

-- Lefty

On Thu, May 25, 2017 at 12:40 AM, Chao Sun  wrote:

> Congratulations Rui!!
>
> On Wed, May 24, 2017 at 9:19 PM, Xuefu Zhang  wrote:
>
> > Hi all,
> >
> > It's an honer to announce that Apache Hive PMC has recently voted to
> invite
> > Rui Li as a new Hive PMC member. Rui is a long time Hive contributor and
> > committer, and has made significant contribution in Hive especially in
> Hive
> > on Spark. Please join me in congratulating him and looking forward to a
> > bigger role that he will play in Apache Hive project.
> >
> > Thanks,
> > Xuefu
> >
>

Re: Jimmy Xiang now a Hive PMC member

[Lefty Leverenz]

Congratulations!

-- Lefty

On Thu, May 25, 2017 at 12:41 AM, Chao Sun  wrote:

> Congratulations Jimmy!!
>
> On Wed, May 24, 2017 at 9:16 PM, Xuefu Zhang  wrote:
>
>> Hi all,
>>
>> It's an honer to announce that Apache Hive PMC has recently voted to
>> invite
>> Jimmy Xiang as a new Hive PMC member. Please join me in congratulating him
>> and looking forward to a bigger role that he will play in Apache Hive
>> project.
>>
>> Thanks,
>> Xuefu
>>
>
>

Re: Review Request 58936: HIVE-16143 : Improve msck repair batching

[Lefty Leverenz]

> On May 12, 2017, 3:01 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 3357-3361 (patched)
> > <https://reviews.apache.org/r/58936/diff/3/?file=1714847#file1714847line3357>
> >
> > Looks good, just some minor suggestions:
> > 
> > - please use newlines (\n) so the generated template file won't put the 
> > entire description on one line
> > - "if" seems unnecessary and confusing in "or if batch size is reduced 
> > to 0"
> > - "which ever" should be "whichever"
> > - a comma before "whichever" might be good
> > - please end the description with a period
> 
> Vihang Karajgaonkar wrote:
> Thanks for the review. Added the suggested changes.

+1  Thanks for the fixes.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58936/#review174756
---


On May 12, 2017, 9:35 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58936/
> ---
> 
> (Updated May 12, 2017, 9:35 p.m.)
> 
> 
> Review request for hive, Aihua Xu, Sergio Pena, and Sahil Takiar.
> 
> 
> Bugs: HIVE-16143
> https://issues.apache.org/jira/browse/HIVE-16143
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16143 : Improve msck repair batching
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> d3ea824c21f2fbf98177cb12a18019416f36a3f9 
>   common/src/java/org/apache/hive/common/util/RetryUtilities.java 
> PRE-CREATION 
>   common/src/test/org/apache/hive/common/util/TestRetryUtilities.java 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/create_like.q 
> 38f384e4c547d3c93d510b89fccfbc2b8e2cba09 
>   itests/hive-blobstore/src/test/results/clientpositive/create_like.q.out 
> 0d362a716291637404a3859fe81068594d82c9e0 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 
> 2ae1eacb68cef6990ae3f2050af0bed7c8e9843f 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 917e565f28b2c9aaea18033ea3b6b20fa41fcd0a 
>   
> ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/msck_repair_0.q 
> 22542331621ca4ce5277c2f46a4264b7540a4d1e 
>   ql/src/test/queries/clientpositive/msck_repair_1.q 
> ea596cbbd2d4c230f2b5afbe379fc1e8836b6fbd 
>   ql/src/test/queries/clientpositive/msck_repair_2.q 
> d8338211e970ebac68a7471ee0960ccf2d51cba3 
>   ql/src/test/queries/clientpositive/msck_repair_3.q 
> fdefca121a2de361dbd19e7ef34fb220e1733ed2 
>   ql/src/test/queries/clientpositive/msck_repair_batchsize.q 
> e56e97ac36a6544f3e20478fdb0e8fa783a857ef 
>   ql/src/test/results/clientpositive/msck_repair_0.q.out 
> 2e0d9dc423071ebbd9a55606f196cf7752e27b1a 
>   ql/src/test/results/clientpositive/msck_repair_1.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_2.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_3.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_batchsize.q.out 
> ba99024163a1f2c59d59e9ed7ea276c154c99d24 
>   ql/src/test/results/clientpositive/repair.q.out 
> c1834640a35500c521a904a115a718c94546df10 
> 
> 
> Diff: https://reviews.apache.org/r/58936/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

Re: [Announce] New committer: Vineet Garg

[Lefty Leverenz]

Congratulations Vineet!

-- Lefty


On Wed, May 10, 2017 at 2:08 PM, Vineet Garg  wrote:

> Thanks all!
>
> > On May 10, 2017, at 12:47 AM, Rui Li  wrote:
> >
> > Congrats :)
> >
> > On Wed, May 10, 2017 at 2:06 PM, Zoltan Haindrich <
> > zhaindr...@hortonworks.com> wrote:
> >
> >> Congratulations!
> >>
> >>
> >> On 10 May 2017 7:57 a.m., Prasanth Jayachandran <
> >> pjayachand...@hortonworks.com> wrote:
> >> Congratulations Vineeth!!
> >>
> >> Thanks
> >> Prasanth
> >>
> >>
> >>
> >> On Tue, May 9, 2017 at 10:52 PM -0700, "Jesus Camacho Rodriguez" <
> >> jcama...@apache.org> wrote:
> >>
> >>
> >> Congrats Vineet! Well deserved!
> >>
> >> --
> >> Jesús
> >>
> >>
> >>
> >>
> >>
> >> On 5/10/17, 6:45 AM, "Peter Vary"  wrote:
> >>
> >>> Congratulations Vineet! :)
> >>>
> >>> 2017. máj. 9. 22:25 ezt írta ("Ashutosh Chauhan" ):
> >>>
>  The Project Management Committee (PMC) for Apache Hive has invited
> >> Vineet
>  Garg to become a committer and we are pleased to announce that he has
>  accepted.
> 
>  Welcome, Vineet!
> 
>  Thanks,
>  Ashutosh
> 
> >>
> >>
> >>
> >>
> >>
> >
> >
> > --
> > Best regards!
> > Rui Li
> > Cell: (+86) 13564950210
>
>

Re: Review Request 58936: HIVE-16143 : Improve msck repair batching

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58936/#review174756
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3351-3355 (original), 3351-3355 (patched)
<https://reviews.apache.org/r/58936/#comment247989>

Please use newlines (\n) so the generated template file won't put the 
entire description on one line.  For an example, see SPARK_JOB_MAX_TASKS 
several lines above this.

(The template file is discussed in 
https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration#AdminManualConfiguration-hive-site.xmlandhive-default.xml.template.)

Also, "(Not batch wise)" shouldn't have a capital N and needs a period 
after the closing parenthesis.



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3357-3361 (patched)
<https://reviews.apache.org/r/58936/#comment247988>

Looks good, just some minor suggestions:

- please use newlines (\n) so the generated template file won't put the 
entire description on one line
- "if" seems unnecessary and confusing in "or if batch size is reduced to 0"
- "which ever" should be "whichever"
- a comma before "whichever" might be good
- please end the description with a period


- Lefty Leverenz


On May 11, 2017, 12:04 a.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58936/
> ---
> 
> (Updated May 11, 2017, 12:04 a.m.)
> 
> 
> Review request for hive, Aihua Xu, Sergio Pena, and Sahil Takiar.
> 
> 
> Bugs: HIVE-16143
> https://issues.apache.org/jira/browse/HIVE-16143
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16143 : Improve msck repair batching
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> d3ea824c21f2fbf98177cb12a18019416f36a3f9 
>   common/src/java/org/apache/hive/common/util/RetryUtilities.java 
> PRE-CREATION 
>   common/src/test/org/apache/hive/common/util/TestRetryUtilities.java 
> PRE-CREATION 
>   itests/hive-blobstore/src/test/queries/clientpositive/create_like.q 
> 38f384e4c547d3c93d510b89fccfbc2b8e2cba09 
>   itests/hive-blobstore/src/test/results/clientpositive/create_like.q.out 
> 0d362a716291637404a3859fe81068594d82c9e0 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 
> 2ae1eacb68cef6990ae3f2050af0bed7c8e9843f 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 917e565f28b2c9aaea18033ea3b6b20fa41fcd0a 
>   
> ql/src/test/org/apache/hadoop/hive/ql/exec/TestMsckCreatePartitionsInBatches.java
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/msck_repair_0.q 
> 22542331621ca4ce5277c2f46a4264b7540a4d1e 
>   ql/src/test/queries/clientpositive/msck_repair_1.q 
> ea596cbbd2d4c230f2b5afbe379fc1e8836b6fbd 
>   ql/src/test/queries/clientpositive/msck_repair_2.q 
> d8338211e970ebac68a7471ee0960ccf2d51cba3 
>   ql/src/test/queries/clientpositive/msck_repair_3.q 
> fdefca121a2de361dbd19e7ef34fb220e1733ed2 
>   ql/src/test/queries/clientpositive/msck_repair_batchsize.q 
> e56e97ac36a6544f3e20478fdb0e8fa783a857ef 
>   ql/src/test/results/clientpositive/msck_repair_0.q.out 
> 2e0d9dc423071ebbd9a55606f196cf7752e27b1a 
>   ql/src/test/results/clientpositive/msck_repair_1.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_2.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_3.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
>   ql/src/test/results/clientpositive/msck_repair_batchsize.q.out 
> ba99024163a1f2c59d59e9ed7ea276c154c99d24 
>   ql/src/test/results/clientpositive/repair.q.out 
> c1834640a35500c521a904a115a718c94546df10 
> 
> 
> Diff: https://reviews.apache.org/r/58936/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

Re: Welcome new Hive committer, Zhihai Xu

[Lefty Leverenz]

Congratulations Zhihai!

-- Lefty


On Sat, May 6, 2017 at 2:06 AM, Zoltan Haindrich  wrote:

> Congratulations Zhihai!
>
> On 6 May 2017 9:52 a.m., Mohammad Islam 
> wrote:
> Congrats Zhihai!!
>
> On Friday, May 5, 2017 9:52 AM, Xuefu Zhang  wrote:
>
>
>  Hi all,
>
> I'm very please to announce that Hive PMC has recently voted to offer
> Zhihai a committership which he accepted. Please join me in congratulating
> on this recognition and thanking him for his contributions to Hive.
>
> Regards,
> Xuefu
>
>
>
>
>

Re: Review Request 58777: HIVE-16546: LLAP: Fail map join tasks if hash table memory exceeds threshold

[Lefty Leverenz]

> On April 30, 2017, 8:12 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 3155-3158 (patched)
> > <https://reviews.apache.org/r/58777/diff/4/?file=1702345#file1702345line3155>
> >
> > "after every these many rows" -> "after every interval of this many 
> > rows"
> > 
> > "exceed" -> "exceeds"
> > 
> > Also, please add another \n to the description after "when running" 
> > (line 3157 in diff revision 4).
> 
> Prasanth_J wrote:
> Thanks Lefty! Fixed in the new patch.

+1  Thanks for the fixes, Prasanth.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58777/#review173439
---


On May 2, 2017, 8:22 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58777/
> ---
> 
> (Updated May 2, 2017, 8:22 a.m.)
> 
> 
> Review request for hive, Gunther Hagleitner, Sergey Shelukhin, and Siddharth 
> Seth.
> 
> 
> Bugs: HIVE-16546
> https://issues.apache.org/jira/browse/HIVE-16546
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16546: LLAP: Fail map join tasks if hash table memory exceeds threshold
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/MemoryEstimate.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3400560 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/IncrementalObjectSizeEstimator.java
>  ff6e7ce 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFAvg.txt 4393c3b 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMax.txt 7468c2f 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMaxDecimal.txt 57b7ea5 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMaxIntervalDayTime.txt 
> 749e97e 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMaxString.txt 9dfc147 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMaxTimestamp.txt 
> 32ecb34 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFSum.txt bd0f14d 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFVar.txt dc9d4b1 
>   ql/src/gen/vectorization/UDAFTemplates/VectorUDAFVarDecimal.txt 01062a9 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/mapjoin/MapJoinMemoryExhaustionError.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/mapjoin/MapJoinMemoryExhaustionException.java
>  dbe00b6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/mapjoin/MapJoinMemoryExhaustionHandler.java
>  7fc3226 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 595d1bd 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java
>  04e24bd 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 
> a3bccc6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HybridHashTableContainer.java
>  04e89e8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java
>  c86e5f5 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java
>  6d71fef 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java 7b13e90 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezProcessor.java 486d43a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorAggregationBufferBatch.java
>  630046d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java 
> 5b4c7c3 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java
>  0866f63 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFAvgDecimal.java
>  74e25ae 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFAvgTimestamp.java
>  483d9dc 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFBloomFilter.java
>  2139eae 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFBloomFilterMerge.java
>  d2446d5 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFCount.java
>  494febc 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFCountMerge.java
>  dec88cb 
>  

Re: Review Request 58865: HIVE-16552: Limit the number of tasks a Spark job may contain

[Lefty Leverenz]

> On April 30, 2017, 4:15 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Lines 3340-3341 (patched)
> > <https://reviews.apache.org/r/58865/diff/1/?file=1703679#file1703679line3340>
> >
> > "... more tasks than the maximu, ..." -> maximum
> > 
> > Also, please add one or more newlines (\n) so the generated file 
> > hive-default.xml.template won't put the entire parameter description on a 
> > single line.

+1  Looks good now, thanks.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58865/#review173436
---


On May 3, 2017, 6:14 p.m., Xuefu Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58865/
> ---
> 
> (Updated May 3, 2017, 6:14 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-16552
> https://issues.apache.org/jira/browse/HIVE-16552
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> See JIRA description
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 84398c6 
>   itests/src/test/resources/testconfiguration.properties 753f3a9 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 32a7730 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RemoteSparkJobMonitor.java
>  dd73f3e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobMonitor.java 
> 0b224f2 
>   ql/src/test/queries/clientnegative/spark_job_max_tasks.q PRE-CREATION 
>   ql/src/test/results/clientnegative/spark/spark_job_max_tasks.q.out 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58865/diff/4/
> 
> 
> Testing
> ---
> 
> Test locally
> 
> 
> Thanks,
> 
> Xuefu Zhang
> 
>

Re: Review Request 58777: HIVE-16546: LLAP: Fail map join tasks if hash table memory exceeds threshold

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58777/#review173439
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3155-3158 (patched)
<https://reviews.apache.org/r/58777/#comment246428>

"after every these many rows" -> "after every interval of this many rows"

"exceed" -> "exceeds"

Also, please add another \n to the description after "when running" (line 
3157 in diff revision 4).


- Lefty Leverenz


On April 28, 2017, 4:53 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58777/
> ---
> 
> (Updated April 28, 2017, 4:53 a.m.)
> 
> 
> Review request for hive, Gunther Hagleitner, Sergey Shelukhin, and Siddharth 
> Seth.
> 
> 
> Bugs: HIVE-16546
> https://issues.apache.org/jira/browse/HIVE-16546
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16546: LLAP: Fail map join tasks if hash table memory exceeds threshold
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/MemoryEstimate.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d3ea824 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/mapjoin/MapJoinMemoryExhaustionError.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/mapjoin/MapJoinMemoryExhaustionException.java
>  dbe00b6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/mapjoin/MapJoinMemoryExhaustionHandler.java
>  7fc3226 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 595d1bd 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java
>  04e24bd 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 
> a3bccc6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HybridHashTableContainer.java
>  04e89e8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java
>  c86e5f5 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java
>  6d71fef 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java 7b13e90 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/MapRecordProcessor.java 
> 24d3526 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezProcessor.java 486d43a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashMap.java
>  6242daf 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashMultiSet.java
>  1a41961 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashSet.java
>  331867c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashTable.java
>  b93e977 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTable.java
>  b6db3bc 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTableLoader.java
>  49ecdd1 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastKeyStore.java
>  be51693 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashMap.java
>  6fe98f9 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashMultiSet.java
>  9140aee 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashSet.java
>  d3efb11 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashTable.java
>  8bfa07c 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastMultiKeyHashMap.java
>  add4788 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastMultiKeyHashMultiSet.java
>  faefdbb 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastMultiKeyHashSet.java
>  5328910 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastStringHashMap.java
>  f13034f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastStringHashMultiSet.java
>  53ad7b4 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastStringHashSet.java
>  723c729 
>

Re: Review Request 58865: HIVE-16552: Limit the number of tasks a Spark job may contain

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58865/#review173436
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3340-3341 (patched)
<https://reviews.apache.org/r/58865/#comment246427>

"... more tasks than the maximu, ..." -> maximum

Also, please add one or more newlines (\n) so the generated file 
hive-default.xml.template won't put the entire parameter description on a 
single line.


- Lefty Leverenz


On April 28, 2017, 11:34 p.m., Xuefu Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58865/
> ---
> 
> (Updated April 28, 2017, 11:34 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-16552
> https://issues.apache.org/jira/browse/HIVE-16552
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> See JIRA description
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d3ea824 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 32a7730 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RemoteSparkJobMonitor.java
>  dd73f3e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobMonitor.java 
> 0b224f2 
> 
> 
> Diff: https://reviews.apache.org/r/58865/diff/1/
> 
> 
> Testing
> ---
> 
> Test locally
> 
> 
> Thanks,
> 
> Xuefu Zhang
> 
>

Re: Review Request 58654: HIVE-16503: LLAP: Oversubscribe memory for noconditional task size

[Lefty Leverenz]

To unsubscribe please send a message to dev-unsubscr...@hive.apache.org as
described here:  Mailing Lists .
Thanks.

-- Lefty


On Sun, Apr 23, 2017 at 6:21 PM,  wrote:

> Please remove me from this list serve
>
> Sent from my iPhone
>
> > On Apr 23, 2017, at 8:23 PM, j.prasant...@gmail.com wrote:
> >
> >
> > ---
> > This is an automatically generated e-mail. To reply, visit:
> > https://reviews.apache.org/r/58654/
> > ---
> >
> > Review request for hive, Gunther Hagleitner and Siddharth Seth.
> >
> >
> > Bugs: HIVE-16503
> >https://issues.apache.org/jira/browse/HIVE-16503
> >
> >
> > Repository: hive-git
> >
> >
> > Description
> > ---
> >
> > HIVE-16503: LLAP: Oversubscribe memory for noconditional task size
> >
> >
> > Diffs
> > -
> >
> >  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> 8e5a9aa53d27a06c3fff58e0d47068efd7ad898f
> >  ql/pom.xml e5d063f94ca10410530146dbac56733f823756b9
> >  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java
> 637bc54fbd4803e31647d33dd9fdca2b32b3ed63
> >  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/
> LlapClusterStateForCompile.java b2e8614cea6d25b57a400ad29237b298e4ab7164
> >  ql/src/test/org/apache/hadoop/hive/ql/exec/TestOperators.java
> 57e573ab7a2b438f3884ec1eb3b285349578a785
> >  ql/src/test/queries/clientpositive/explainuser_4.q
> f58afa88e3ac50e0a97f1f83e7ffc9af2afff092
> >  ql/src/test/queries/clientpositive/tez_smb_main.q
> ee24691dd6ab60693fd5e8b22558587e8728976f
> >  ql/src/test/queries/clientpositive/tez_vector_dynpart_hashjoin_1.q
> 7dd30039f192818ced6d2fefd439fc0ee0fcdfc1
> >  ql/src/test/results/clientpositive/llap/tez_smb_main.q.out
> b583bfffcc7070a4ed868b86c3b06e4bbc876640
> >
> >
> > Diff: https://reviews.apache.org/r/58654/diff/1/
> >
> >
> > Testing
> > ---
> >
> >
> > Thanks,
> >
> > Prasanth_J
> >
>

Re: TODOC labels for releases 2.2, 2.3, and 3.0

[Lefty Leverenz]

Well, I got that wrong -- apparently Hive's branch-2.2 is now the same as
Owen's.  That simplifies things considerably.

So any TODOC2.2 label in Hive's branch-2.2 can be changed to TODOC2.2.0,
and then all remaining TODOC2.2 labels can be changed to TODOC2.3.

That's a lot easier than examining every JIRA issue.  There might be a few
recent ones that ought to be TODOC3.0, but we'll catch them as best we can.

Corrections, anyone?  Since I already muddled part of this, I won't make
any global label changes until people have a chance to review my plan.

-- Lefty


On Wed, Mar 29, 2017 at 1:20 AM, Lefty Leverenz <leftylever...@gmail.com>
wrote:

> For the upcoming releases, we'll use these TODOC labels in JIRA:
>
>- *TODOC3.0* for patches committed to master (only) after 23 March
>11:06 pm PST.
>- *TODOC2.3* for patches committed to branch-2.
>   - Patches committed to master *before* 23 March 11:06 pm PST (when
>   branch-2 was split off) are already labeled *TODOC2.2* but those
>   labels need to be changed.
>- *TODOC2.2.0* for patches committed to Owen's branch-2.2
><https://github.com/omalley/hive/tree/branch-2.2>, which presumably
>will be moved to Hive's github tree eventually *but is not the same as
>Hive's branch-2.2 <https://github.com/apache/hive/tree/branch-2.2>*.
>
> So the old label *TODOC2.2* will become obsolete because it's ambiguous
> now that release 2.2.0 is cherry-picking patches made since 2.1.1.
>
> References:  About next 2.3 release
> <http://mail-archives.apache.org/mod_mbox/hive-dev/201703.mbox/%3ccams4yue4wn3zabeet-acn6smhup0k8ffgeu3w6m4y9wgjfs...@mail.gmail.com%3e>
> , Re: Backward incompatible changes
> <http://mail-archives.apache.org/mod_mbox/hive-dev/201703.mbox/%3cCAHfHakEL-+C=q5rnrunxfcmqabnpm5m_h6qrokcrns24cv2...@mail.gmail.com%3e>
> .
>
> Can anyone explain what Hive's branch-2.2 is, and how we're going to avoid
> confusing it with Owen's branch-2.2?
>
> -- Lefty
>

TODOC labels for releases 2.2, 2.3, and 3.0

[Lefty Leverenz]

For the upcoming releases, we'll use these TODOC labels in JIRA:

   - *TODOC3.0* for patches committed to master (only) after 23 March 11:06
   pm PST.
   - *TODOC2.3* for patches committed to branch-2.
  - Patches committed to master *before* 23 March 11:06 pm PST (when
  branch-2 was split off) are already labeled *TODOC2.2* but those
  labels need to be changed.
   - *TODOC2.2.0* for patches committed to Owen's branch-2.2
   , which presumably will
   be moved to Hive's github tree eventually *but is not the same as Hive's
   branch-2.2 *.

So the old label *TODOC2.2* will become obsolete because it's ambiguous now
that release 2.2.0 is cherry-picking patches made since 2.1.1.

References:  About next 2.3 release

, Re: Backward incompatible changes

.

Can anyone explain what Hive's branch-2.2 is, and how we're going to avoid
confusing it with Owen's branch-2.2?

-- Lefty

Re: Backward incompatible changes

[Lefty Leverenz]

We'll have to revise some TODOC2.2 labels in JIRA, as well as fix versions.

-- Lefty


On Fri, Mar 24, 2017 at 2:19 PM, Ashutosh Chauhan 
wrote:

> Sounds good to me. I have created Fix versions 3.0 and 2.3 in jira. We can
> use 2.3 for next release from branch-2.
>
> On Fri, Mar 24, 2017 at 10:46 AM, Owen O'Malley 
> wrote:
>
> > All,
> >
> > I'm glad to see a plan to release master before the incompatible changes.
> > It will be great to have a release that uses the separated out ORC code
> > base.
> >
> > As I had previously discussed on list, I've been working on building a
> > proposed 2.2 branch, which builds on 2.1.1 and cherry picks a bunch of
> > changes. The list of patches was picked to synergize with the efforts of
> > the QA team at Hortonworks. You can look at the patches on
> > https://github.com/omalley/hive/tree/branch-2.2 .
> >
> > The branch is still pretty rough, but I don't expect many large code
> > changes. I'd like to propose that we take my branch as branch-2.2 and set
> > up Pengcheng's branch as branch-2.3. We should also consider the
> packaging
> > changes (shrouding protobuf, guava, and kyro) that would make integration
> > with Spark easier in branch-2.3.
> >
> > Thanks,
> >Owen
> >
> >
> > On Thu, Mar 23, 2017 at 11:15 PM, Ashutosh Chauhan  >
> > wrote:
> >
> > > So, I just pushed branch-2 from current tip.
> > > Now, lets get 2.2 release out as soon as possible.
> > >
> > > Thanks,
> > > Ashutosh
> > >
> > > On Thu, Mar 23, 2017 at 9:46 PM, Ashutosh Chauhan <
> hashut...@apache.org>
> > > wrote:
> > >
> > > > Cutting a branch should not slow down a 2.2 release. If any thing,
> this
> > > > should help in achieving stabilization faster for release since
> branch
> > > > won't get any new potentially destabilizing changes but only patches
> to
> > > fix
> > > > existing known issues.
> > > >
> > > > On Thu, Mar 23, 2017 at 8:22 AM, Eugene Koifman <
> > > ekoif...@hortonworks.com>
> > > > wrote:
> > > >
> > > >> +1 to make a release first
> > > >>
> > > >> On 3/22/17, 2:06 PM, "Sergey Shelukhin" 
> > wrote:
> > > >>
> > > >> Hmm.. should we release these first, and then cut branch-2?
> > > >> Otherwise during the releases, the patches for 2.2/2.3 will need
> > to
> > > >> go to
> > > >> 3 (4?) places (master, branch-2, branch-2.2, branch-2.3?).
> > > >> There’s no rush to cut the branch if everything in 2.2/2.3 has
> to
> > go
> > > >> to
> > > >> 3.0 anyway.
> > > >>
> > > >> On 17/3/22, 13:53, "Pengcheng Xiong"  wrote:
> > > >>
> > > >> >I would like to work as the Release Manager if possible. As
> Owen
> > > >> points
> > > >> >out, he is working on 2.2 and I will work on 2.3. Thanks.
> > > >> >
> > > >> >On Wed, Mar 22, 2017 at 1:32 PM, Ashutosh Chauhan <
> > > >> hashut...@apache.org>
> > > >> >wrote:
> > > >> >
> > > >> >> Unless there is more feedback, I plan to cut branch-2 in a
> day
> > or
> > > >> two
> > > >> >>from
> > > >> >> current master. As multiple people have suggested on this
> > thread,
> > > >> we
> > > >> >>should
> > > >> >> do a 2.2 release soon. Currently there are 177 issues
> > > >> >>  > > >> >> 3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20cf%
> > > >> >> 5B12310320%5D%20%3D%202.2.0%20ORDER%20BY%20priority%20DESC>
> > > >> >> targeted for 2.2 release. We can use branch-2 to land these
> > > >> patches and
> > > >> >>for
> > > >> >> additional stabilization efforts. Any volunteer for Release
> > > Manager
> > > >> >>driving
> > > >> >> 2.2 release?
> > > >> >>
> > > >> >> Thanks,
> > > >> >> Ashutosh
> > > >> >>
> > > >> >> On Fri, Mar 10, 2017 at 4:23 PM, Ashutosh Chauhan <
> > > >> hashut...@apache.org>
> > > >> >> wrote:
> > > >> >>
> > > >> >> > I hear what you are saying. Lets begin with 3 concerns:
> > > >> >> >
> > > >> >> > - How will we keep the community motivated on fixing both
> > > master
> > > >> and
> > > >> >> > branch-2?
> > > >> >> > Until we do a stable release from master, stable releases
> can
> > > >> come
> > > >> >>only
> > > >> >> > from branch-2. If a contributor wants to see their fix
> reach
> > to
> > > >> users
> > > >> >>on
> > > >> >> a
> > > >> >> > stable line quickly they would have to have a fix on
> > branch-2.
> > > >> Also, a
> > > >> >> > release manager can pick whatever fixes she wants, so even
> if
> > > >> >>contributor
> > > >> >> > doesn't commit it on branch-2, a release manger who wants
> to
> > > do a
> > > >> >>release
> > > >> >> > containing a set of fixes thats always possible.
> > > >> >> >
> > > >> >> > - *Harder cherry-picks between master and branch-2*.
> > > >> >> > That is certainly possible. But hope is we want to 

Re: [ANNOUNCE] New PMC Member : Eugene Koifman

[Lefty Leverenz]

More congratulations!

-- Lefty

On Wed, Mar 15, 2017 at 1:27 PM, Eugene Koifman 
wrote:

> Thank you everyone!
>
> On 3/15/17, 12:21 PM, "Gunther Hagleitner" 
> wrote:
>
> Congratulations!
> 
> From: Sergey Shelukhin 
> Sent: Wednesday, March 15, 2017 11:18 AM
> To: dev@hive.apache.org
> Subject: Re: [ANNOUNCE] New PMC Member : Eugene Koifman
>
> Congrats!
>
> On 17/3/15, 01:02, "Zoltan Haindrich" 
> wrote:
>
> >Congrats Eugene!!
> >
> >On 15 Mar 2017 07:50, Peter Vary  wrote:
> >Congratulations! :)
> >
> >2017. márc. 15. 7:05 ezt írta ("Vaibhav Gumashta"
> > >>):
> >
> >> Congrats Eugene!
> >>
> >>
> >> On 3/14/17, 11:03 PM, "Rajesh Balamohan" 
> wrote:
> >>
> >> >Congrats Eugene!! :)
> >> >
> >> >~Rajesh.B
> >> >
> >> >On Wed, Mar 15, 2017 at 11:21 AM, Pengcheng Xiong <
> pxi...@apache.org>
> >> >wrote:
> >> >
> >> >> Congrats! Well deserved!
> >> >>
> >> >> Thanks.
> >> >> Pengcheng
> >> >>
> >> >> On Tue, Mar 14, 2017 at 10:39 PM, Ashutosh Chauhan
> >> >>
> >> >> wrote:
> >> >>
> >> >> > On behalf of the Hive PMC I am delighted to announce Eugene
> >>Koifman is
> >> >> > joining Hive PMC.
> >> >> > Eugene is a long time contributor in Hive and is focusing on
> ACID
> >> >>support
> >> >> > areas these days.
> >> >> >
> >> >> > Welcome, Eugene!
> >> >> >
> >> >> > Thanks,
> >> >> > Ashutosh
> >> >> >
> >> >>
> >>
> >>
> >
>
>
>
>
>

Re: [ANNOUNCE] New committer: Zoltan Haindrich

[Lefty Leverenz]

Congratulations Zoltan, and thanks for all your wiki contributions!

-- Lefty


On Tue, Feb 21, 2017 at 10:00 PM, Zoltan Haindrich  wrote:

> Thank you everyone!
>
> It's really an honor to be associated with this project!
>
> About me: I've a pretty broad spectrum of interest in general, I always
> want to clearly understand the big picture. I tend to focus on creating
> flexible solutions.
> I usually collect all kind of (sometimes wierd) ideas while I work on
> things - but I only employ any of them when it aids the introduction of
> some new feature...
>
> I'm looking forward to continue working with you on Hive!
>
> cheers,
> Zoltan
>
>
> On 02/22/2017 01:47 AM, Sergey Shelukhin wrote:
>
>> Congratulations!
>>
>> On 17/2/21, 16:43, "Prasanth Jayachandran" > >
>> wrote:
>>
>> Congratulations Zoltan!!
>>>
>>> Thanks
>>> Prasanth
>>>
>>>
>>>
>>>
>>> On Tue, Feb 21, 2017 at 4:35 PM -0800, "Eugene Koifman"
>>> > wrote:
>>>
>>>
>>> Congratulations!
>>>
>>> On 2/21/17, 4:17 PM, "Vihang Karajgaonkar"  wrote:
>>>
>>>Congrats Zoltan!
>>>
>>>On Tue, Feb 21, 2017 at 4:16 PM, Vaibhav Gumashta  wrote:
>>>
>>>> Congrats Zoltan!
>>>>
>>>> On 2/21/17, 4:16 PM, "Jimmy Xiang"  wrote:
>>>>
>>>> >Congrats, Zoltan!!
>>>> >
>>>> >On Tue, Feb 21, 2017 at 4:15 PM, Sushanth Sowmyan
>>>> >wrote:
>>>> >> Congrats, Zoltan!
>>>> >>
>>>> >> Welcome aboard. :)
>>>> >>
>>>> >> On Feb 21, 2017 15:42, "Rajesh Balamohan"
>>>> wrote:
>>>> >>
>>>> >>> Congrats Zoltan. :)
>>>> >>>
>>>> >>> ~Rajesh.B
>>>> >>>
>>>> >>> On Wed, Feb 22, 2017 at 4:43 AM, Wei Zheng
>>>> >>>wrote:
>>>> >>>
>>>> >>> > Congrats Zoltan!
>>>> >>> >
>>>> >>> > Thanks,
>>>> >>> > Wei
>>>> >>> >
>>>> >>> > On 2/21/17, 13:09, "Alan Gates"  wrote:
>>>> >>> >
>>>> >>> > On behalf of the Hive PMC I am happy to announce Zoltan
>>>> >>>Haindrich is
>>>> >>> > our newest committer.  He has been contributing to Hive for
>>> several
>>>> >>> months
>>>> >>> > across a number of areas, including the parser, HiveServer2,
>>> and
>>>> >>>cleaning
>>>> >>> > up unit tests and documentation.  Please join me in welcoming
>>> Zoltan
>>>> >>>to
>>>> >>> > Hive.
>>>> >>> >
>>>> >>> > Zoltan, feel free to say a few words introducing yourself
>>> if you
>>>> >>> would
>>>> >>> > like to.
>>>> >>> >
>>>> >>> > Alan.
>>>> >>> >
>>>> >>> >
>>>> >>> >
>>>> >>>
>>>> >
>>>>
>>>>
>>>
>>>
>>>
>>>
>>

Re: a gap in JIRA-created emails

[Lefty Leverenz]

Are you talking about missing JIRA emails for commit comments and changes
of status?  I saw that for six issues on Monday, Jan. 23 (or so) but not
since then.  The issues were HIVE-15439
, HIVE-15579
, HIVE-15589
, HIVE-15591
, HIVE-15646
, and HIVE-15662
.

If the problem had persisted, I would have updated INFRA-9221
 "New issues@hive mailing
list not getting JIRA commit messages".

But you're reporting a problem with missing messages since Friday, so maybe
it's a different issue.  In any case, you can file an INFRA bug.

-- Lefty


On Tue, Jan 31, 2017 at 3:06 PM, Sergey Shelukhin 
wrote:

> I wonder what happened to those.
> I see that the JIRA notification config is still correct and has "Single
> Email Address (dev@hive.apache.org)”.
> I also see 3 messages for the 3 most recent jiras, but not the ones since
> Friday until today.
> What would our “complain to jira people” channel be?
>

Re: why does HiveConf.java have explicit line breaks?

[Lefty Leverenz]

Those line breaks are for the generated template file
(conf/hive-default.xml.template), which formats configuration parameters
and their descriptions appropriately for hive-site.xml files.

For example, *hive.input.format* doesn't have explicit line breaks so its
long description is all on one line:
https://github.com/apache/hive/blob/master/data/conf/hive-site.xml#L200-204.

Rather than removing "\n" in parameter descriptions, we should add them
where needed.

-- Lefty


On Fri, Jan 13, 2017 at 1:01 PM, Sergey Shelukhin 
wrote:

> Subj. All the setting descriptions are in the “Blah blah blah\n” + “Blah
> blah” form.
> Do we really need the explicit line break there? To maintain the short
> lines in the source we could just sum strings w/o \n.
>
>

State of the docs

[Lefty Leverenz]

Here's the tally of Hive JIRA issues that need documentation, as determined
by TODOC labels:


   - 76 for 2.2 (master)
   - 12 for 2.1.1
   - 60 for 2.1
   - 64 for 2.0
   - 32 for 1.3
   - 49 for 1.2
   - 48 for 15 (1.1.0, also known as 0.15.0)
   -   5 for 1.0 (also known as 0.14.1)
   - 63 for 14 (0.14)
   - 21 for 13 (0.13)
   - 13 for 12 (0.12)
   -   9 for 11 (0.11)
   -   8 for 10 (0.10)

Grand total January 2017:460 issues need wikidoc
   March 2016:   353
   February 2015:  166

-- Lefty

[jira] [Created] (HIVE-15523) Fix broken "Builds" link on Hive home page

[Lefty Leverenz (JIRA)]

Lefty Leverenz created HIVE-15523:
-

 Summary: Fix broken "Builds" link on Hive home page
 Key: HIVE-15523
 URL: https://issues.apache.org/jira/browse/HIVE-15523
 Project: Hive
  Issue Type: Bug
  Components: Website
Reporter: Lefty Leverenz


The Builds link in the side bar on hive.apache.org points to 
http://bigtop01.cloudera.org:8080/view/Hive/, which times out with the message 
"the server where this page is located isn't responding."

This was reported by Laurel Hale (thanks, Laurel).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Can anyone help to assign the privilege to me to update the wiki of hive on spark

[Lefty Leverenz]

Kelly, first you need to get a Confluence account as described in About
This Wiki

.

-- Lefty


On Wed, Dec 28, 2016 at 10:37 PM, Zhang, Liyun 
wrote:

> Hi：
>
>   I want to update wiki confluence/display/Hive/Hive+on+Spark%3A+Getting+Started> of hive on
> spark because HIVE-8373, my account is kellyzly, can anyone help to assign
> the privilege to me to update wiki?
>
>
>
>
>
>
>
> Best Regards
>
> Kelly Zhang/Zhang,Liyun
>
>
>
>
>
>
>
> -Original Message-
> From: Rui Li (JIRA) [mailto:j...@apache.org]
> Sent: Wednesday, December 28, 2016 5:36 PM
> To: Zhang, Liyun 
> Subject: [jira] [Commented] (HIVE-8373) OOM for a simple query with
> spark.master=local [Spark Branch]
>
>
>
>
>
> [ https://issues.apache.org/jira/browse/HIVE-8373?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel=15782500#comment-15782500 ]
>
>
>
> Rui Li commented on HIVE-8373:
>
> --
>
>
>
> Thanks [~asears] for the explanations. [~kellyzly], yes please go ahead
> update the wiki. I think we can put it in the [known issue|
> https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+
> Started#HiveonSpark:GettingStarted-CommonIssues(Greenareresolved,
> willberemovedfromthislist)] part.
>
>
>
> > OOM for a simple query with spark.master=local [Spark Branch]
>
> > -
>
> >
>
> > Key: HIVE-8373
>
> > URL: https://issues.apache.org/jira/browse/HIVE-8373
>
> > Project: Hive
>
> >  Issue Type: Bug
>
> >  Components: Spark
>
> >Reporter: Xuefu Zhang
>
> >Assignee: liyunzhang_intel
>
> >
>
> > I have a straigh forward query to run in Spark local mode, but get an
> OOM even though the data volumn is tiny:
>
> > {code}
>
> > Exception in thread "Spark Context Cleaner"
>
> > Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Spark Context Cleaner"
>
> > Exception in thread "Executor task launch worker-1"
>
> > Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Executor task launch worker-1"
>
> > Exception in thread "Keep-Alive-Timer"
>
> > Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Keep-Alive-Timer"
>
> > Exception in thread "Driver Heartbeater"
>
> > Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Driver Heartbeater"
>
> > {code}
>
> > The query is:
>
> > {code}
>
> > select product_name, avg(item_price) as avg_price from product join
>
> > item on item.product_pk=product.product_pk group by product_name order
>
> > by avg_price; {code}
>
>
>
>
>
>
>
> --
>
> This message was sent by Atlassian JIRA
>
> (v6.3.4#6332)
>
>
> Best Regards
> Kelly Zhang/Zhang,Liyun
>
>

[jira] [Created] (HIVE-15515) Remove the docs directory

[Lefty Leverenz (JIRA)]

Lefty Leverenz created HIVE-15515:
-

 Summary: Remove the docs directory
 Key: HIVE-15515
 URL: https://issues.apache.org/jira/browse/HIVE-15515
 Project: Hive
  Issue Type: Bug
  Components: Documentation
Reporter: Lefty Leverenz


Hive xdocs have not been used since 2012.  The docs directory only holds six 
xml documents, and their contents are in the wiki.

It's past time to remove the docs directory from the Hive code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Naveen Gangam has become a Hive Committer

[Lefty Leverenz]

Congratulations Naveen!

-- Lefty


On Fri, Dec 16, 2016 at 10:10 AM, Xuefu Zhang  wrote:

> Bcc: dev/user
>
> Hi all,
>
> It's my honor to announce that Apache Hive PMC has voted on and approved
> Naveen's committership. Please join me in congratulate him on his
> contributions and achievements.
>
> Regards,
> Xuefu
>

Re: Error in Jira wiki page for ACID on Hive

[Lefty Leverenz]

It's fixed now (see Worker
).
Thanks, Michael and Josh.

Michael, you need a Confluence account to get edit privileges for the Hive
wiki, as described here:

https://cwiki.apache.org/confluence/display/Hive/AboutThisWiki#AboutThisWiki-Howtogetpermissiontoedit

-- Lefty



On Sat, Dec 17, 2016 at 1:33 PM, Josh Elser  wrote:

> Hi Michael,
>
> Thanks for reporting the issue. I'm forwarding your request to the Hive
> developers mailing list. They manage this wiki and should be capable of
> helping resolve the issue, as well as answering any other questions you
> might have. Please interface with them directly in the future.
>
> Deguzis, Michael T. wrote:
>
>> It appears it was meant "Workers" here, but I don't see any place (Yes I
>> am logged in) to report or suggest an editThe total number of Workrs in
>> the Hive
>>
>> /Warehouse determines the maximum number of concurrent compactions./
>>
>> Page:
>>
>> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions
>> 
>>
>> Thank you,
>>
>> 
>> 
>>
>> Michael DeGuzis | Technical Analyst II | IT Dept. | Geisinger Health
>> System
>>
>> mtdegu...@geisinger.edu  | 570.214.7067
>> office | 570.973.0452 pager | 24-12 mail code
>>
>> 100 N. Academy Ave Danville, PA 17822-2290 | www.geisinger.org
>> 
>>
>>
>> 
>>
>> IMPORTANT WARNING: The information in this message (and the documents
>> attached to it, if any) is confidential and may be legally privileged.
>> It is intended solely for the addressee. Access to this message by
>> anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken, or omitted to be
>> taken, in reliance on it is prohibited and may be unlawful. If you have
>> received this message in error, please delete all electronic copies of
>> this message (and the documents attached to it, if any), destroy any
>> hard copies you may have created and notify me immediately by replying
>> to this email. Thank you. Geisinger Health System utilizes an encryption
>> process to safeguard Protected Health Information and other confidential
>> data contained in external e-mail messages. If email is encrypted, the
>> recipient will receive an e-mail instructing them to sign on to the
>> Geisinger Health System Secure E-mail Message Center to retrieve the
>> encrypted e-mail.
>>
>>

Re: [ANNOUNCE] New Hive Committer - Rajesh Balamohan

[Lefty Leverenz]

Congratulations Rajesh!

-- Lefty


On Tue, Dec 13, 2016 at 11:58 PM, Rajesh Balamohan 
wrote:

> Thanks a lot for providing this opportunity and to all for their messages.
> :)
>
> ~Rajesh.B
>
> On Wed, Dec 14, 2016 at 11:33 AM, Dharmesh Kakadia 
> wrote:
>
> > Congrats Rajesh !
> >
> > Thanks,
> > Dharmesh
> >
> > On Tue, Dec 13, 2016 at 7:37 PM, Vikram Dixit K 
> > wrote:
> >
> >> Congrats Rajesh! :)
> >>
> >> On Tue, Dec 13, 2016 at 9:36 PM, Pengcheng Xiong 
> >> wrote:
> >>
> >>> Congrats Rajesh! :)
> >>>
> >>> On Tue, Dec 13, 2016 at 6:51 PM, Prasanth Jayachandran <
> >>> prasan...@apache.org
> >>> > wrote:
> >>>
> >>> > The Apache Hive PMC has voted to make Rajesh Balamohan a committer on
> >>> the
> >>> > Apache Hive Project. Please join me in congratulating Rajesh.
> >>> >
> >>> > Congratulations Rajesh!
> >>> >
> >>> > Thanks
> >>> > Prasanth
> >>>
> >>
> >>
> >>
> >> --
> >> Nothing better than when appreciated for hard work.
> >> -Mark
> >>
> >
> >
>

Re: Request for addition as contributor

[Lefty Leverenz]

You got it now, Alan.

-- Lefty


On Fri, Dec 9, 2016 at 11:34 AM, Alan Gates <alanfga...@gmail.com> wrote:

>
> > On Dec 9, 2016, at 00:38, Lefty Leverenz <leftylever...@gmail.com>
> wrote:
> >
> > For the wiki you need a Confluence username.  (See About This Wiki
> > <https://cwiki.apache.org/confluence/display/Hive/
> AboutThisWiki#AboutThisWiki-Howtogetpermissiontoedit>.)
> > The wiki permissions page didn't recognize szita or Adam Szita.
> >
> > Alan (and everyone), Space Tools permissions
> > <https://cwiki.apache.org/confluence/spaces/spacepermissions.action?key=
> Hive>
> > is where you can check which permissions you have for the Hive wiki.  You
> > don't have admin -- do you want it?
>
> Sure.
>
> Alan.
>
> >
> > -- Lefty
> >
> >
> > On Thu, Dec 8, 2016 at 7:27 AM, Alan Gates <alanfga...@gmail.com> wrote:
> >
> >> On JIRA or the wiki?  I tried to add you to the JIRA but it didn't find
> a
> >> szita or Adam Szita user.  I'm not sure I have permission to add you to
> the
> >> wiki.
> >>
> >> Alan.
> >>
> >>> On Dec 8, 2016, at 07:21, Adam Szita <sz...@cloudera.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Can you add my userid (szita) as contributor to Hive please.
> >>>
> >>> Thanks,
> >>> Adam
> >>
> >>
>
>

Re: Request for addition as contributor

[Lefty Leverenz]

For the wiki you need a Confluence username.  (See About This Wiki
.)
 The wiki permissions page didn't recognize szita or Adam Szita.

Alan (and everyone), Space Tools permissions

is where you can check which permissions you have for the Hive wiki.  You
don't have admin -- do you want it?

-- Lefty


On Thu, Dec 8, 2016 at 7:27 AM, Alan Gates  wrote:

> On JIRA or the wiki?  I tried to add you to the JIRA but it didn't find a
> szita or Adam Szita user.  I'm not sure I have permission to add you to the
> wiki.
>
> Alan.
>
> > On Dec 8, 2016, at 07:21, Adam Szita  wrote:
> >
> > Hi,
> >
> > Can you add my userid (szita) as contributor to Hive please.
> >
> > Thanks,
> > Adam
>
>

Re: Permissions to edit pages

[Lefty Leverenz]

Anishek, you need a Confluence username as described here:  How to get
permission to edit

.

-- Lefty


On Mon, Nov 21, 2016 at 8:16 PM, Anishek Agarwal 
wrote:

> Hello,
>
> Can I please have the permission to edit the pages for Hive @
> https://cwiki.apache.org/confluence/display/Hive/Home
>
> Regards,
> Anishek
>

Re: Review Request 53659: HIVE-15148 disallow loading data into bucketed tables (by default)

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53659/#review155771
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (lines 1013 - 1014)
<https://reviews.apache.org/r/53659/#comment225857>

carterian -> Cartesian


- Lefty Leverenz


On Nov. 10, 2016, 1:27 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53659/
> ---
> 
> (Updated Nov. 10, 2016, 1:27 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 56289c9 
>   data/scripts/q_test_init.sql 0c8668c 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java 
> a7005f1 
>   ql/src/test/queries/clientnegative/alter_concatenate_indexed_table.q 
> 4193315 
>   ql/src/test/queries/clientnegative/bucket_mapjoin_mismatch1.q 7dce81f 
>   ql/src/test/queries/clientnegative/dynamic_partitions_with_whitelist.q 
> 8623db8 
>   ql/src/test/queries/clientnegative/insertover_dynapart_ifnotexists.q 
> a8f77c2 
>   ql/src/test/queries/clientpositive/alter_concatenate_indexed_table.q 
> d2bc440 
>   ql/src/test/queries/clientpositive/alter_merge.q 0ff0294 
>   ql/src/test/queries/clientpositive/alter_merge_2.q e09703d 
>   ql/src/test/queries/clientpositive/alter_merge_stats.q 0af87e2 
>   ql/src/test/queries/clientpositive/autoColumnStats_1.q bb7252a 
>   ql/src/test/queries/clientpositive/autoColumnStats_2.q c1abcb1 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_1.q 6c2fbba 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_11.q aa345b5 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_12.q b344bd0 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_16.q 83b67f8 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_2.q 2cf809b 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_3.q b14b32b 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_4.q b08649e 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_5.q 489f556 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_7.q 161bc8a 
>   ql/src/test/queries/clientpositive/auto_sortmerge_join_8.q cc2ba28 
>   ql/src/test/queries/clientpositive/bucket_map_join_spark1.q 3f9ccc4 
>   ql/src/test/queries/clientpositive/bucket_map_join_spark2.q 4a70671 
>   ql/src/test/queries/clientpositive/bucket_map_join_spark3.q 9ae7986 
>   ql/src/test/queries/clientpositive/bucket_map_join_tez1.q ade04dd 
>   ql/src/test/queries/clientpositive/bucket_map_join_tez2.q fd997df 
>   ql/src/test/queries/clientpositive/bucketcontext_1.q ad0f473 
>   ql/src/test/queries/clientpositive/bucketcontext_2.q 931f55a 
>   ql/src/test/queries/clientpositive/bucketcontext_3.q e03b285 
>   ql/src/test/queries/clientpositive/bucketcontext_4.q af08f07 
>   ql/src/test/queries/clientpositive/bucketcontext_5.q 5078072 
>   ql/src/test/queries/clientpositive/bucketcontext_6.q 5549f3b 
>   ql/src/test/queries/clientpositive/bucketcontext_7.q 44cdb5e 
>   ql/src/test/queries/clientpositive/bucketcontext_8.q bb655e2 
>   ql/src/test/queries/clientpositive/bucketizedhiveinputformat_auto.q a2b0bbe 
>   ql/src/test/queries/clientpositive/bucketmapjoin1.q 2fd3804 
>   ql/src/test/queries/clientpositive/bucketmapjoin10.q 09c0ae2 
>   ql/src/test/queries/clientpositive/bucketmapjoin11.q d330b77 
>   ql/src/test/queries/clientpositive/bucketmapjoin12.q 43a9de4 
>   ql/src/test/queries/clientpositive/bucketmapjoin2.q d14261f 
>   ql/src/test/queries/clientpositive/bucketmapjoin3.q 78c23d5 
>   ql/src/test/queries/clientpositive/bucketmapjoin4.q 54626e7 
>   ql/src/test/queries/clientpositive/bucketmapjoin5.q 6c03bbb 
>   ql/src/test/queries/clientpositive/bucketmapjoin7.q cd4efe7 
>   ql/src/test/queries/clientpositive/bucketmapjoin8.q f467ea6 
>   ql/src/test/queries/clientpositive/bucketmapjoin9.q f1d5f58 
>   ql/src/test/queries/clientpositive/bucketmapjoin_negative.q ea140dd 
>   ql/src/test/queries/clientpositive/bucketmapjoin_negative2.q 36d34dc 
>   ql/src/test/queries/clientpositive/bucketmapjoin_negative3.q 6398fff 
>   ql/src/test/queries/clientpositive/cbo_rp_udaf_percentile_approx_23.q 
> 0f2d0dd 
>   ql/src/test/queries/clientpositive/char_1.q e617a49 
>   ql/src/test/queries/clientpositive/char_serde.q 4340b4d 
>   ql/src/test/queries/clientpositive/create_merge_compressed

Re: Review Request 52923: HIVE-14979 Removing stale Zookeeper locks at HiveServer2 initialization

[Lefty Leverenz]

> On Oct. 20, 2016, 7:19 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, lines 1766-1772
> > <https://reviews.apache.org/r/52923/diff/4/?file=1541386#file1541386line1766>
> >
> > Looks good, thanks.  Just one more nit (on 2 lines):  Zookeeper -> 
> > ZooKeeper.
> > 
> > See https://zookeeper.apache.org if you don't believe me.  ;)
> 
> Peter Vary wrote:
> Thanks, this escaped my attention. Sorry!

+1  Looks good now, thanks Peter.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52923/#review153357
---


On Oct. 26, 2016, 3:24 p.m., Peter Vary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52923/
> ---
> 
> (Updated Oct. 26, 2016, 3:24 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Marta Kuczora, Miklos Csanady, 
> namit jain, Sergio Pena, and Barna Zsombor Klara.
> 
> 
> Bugs: HIVE-14979
> https://issues.apache.org/jira/browse/HIVE-14979
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Adding a new configuration option to HiveConf to signal whether stale lock 
> removal is requested on startup.
> Adding a new method to ZooKeeperHiveLockManager to remove stale locks
> Modifying the HiveServer2 to instantiate a lock manager and call the new 
> method if defined by the configuration.
> 
> Please take extra care when reviewing these:
> - Modifying the lock fetching method to use the clientIp from the lock, and 
> not update with the current ip - Not sure why it was done before
> - Instantiation of the lock manager - I might not chose the best method for it
> 
> Open for any suggestions :)
> 
> Thanks,
> Peter
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8ffae3b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
>  14d0ef4 
>   
> ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java
>  3f9926e 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 590b1f3 
> 
> Diff: https://reviews.apache.org/r/52923/diff/
> 
> 
> Testing
> ---
> 
> Created 2 unit test cases:
> - Removing own locks
> - Not removing other server's locks
> 
> Manually tested the Lock manager instantiation method on HiveServer2
> 
> 
> Thanks,
> 
> Peter Vary
> 
>

Re: [DISCUSS] Pre-commit tests before commits

[Lefty Leverenz]

>
> @Prasanth - separate page on the wiki for 'Adding / modifying tests'. That can
> be discussed on the list, or modified directly.


Here's what the wiki has so far:

   - Tips for Adding New Tests
   
(written
   in 2011)
   - Developer Guide:  Adding new unit tests
   

(outdated
   instructions)

-- Lefty


On Mon, Oct 24, 2016 at 4:49 PM, Siddharth Seth  wrote:

> There seems to be general consensus to follow this approach for commits,
> and waiting for pre-commit tests. I'm going to add this to the Hive Wiki.
>
> @Alan, On using Yetus - I don't really know much about it, so cannot
> comment. However, Hive does use a parallel test framework which generates
> reports, compiles hive via custom scripts, etc - I suspect that will
> complicate integration with a third party tool. Any one want to volunteer
> for the findbugs, javac error integration?
>
> @Prasanth - separate page on the wiki for 'Adding / modifying tests'. That
> can be discussed on the list, or modified directly.
>
> Thanks,
> Sid
>
> On Mon, Oct 17, 2016 at 12:07 PM, Sergey Shelukhin  >
> wrote:
>
> > +1
> >
> > On 16/10/13, 23:10, "Siddharth Seth"  wrote:
> >
> > >There's been a lot of work to make the test runs faster, as well as more
> > >reliable via HIVE-14547, HIVE-13503, and several other jiras. Test
> > >runtimes
> > >are around the 1 hour mark, and going down. There were a few green
> > >pre-commit runs (after years?). At the same time, there's still some
> flaky
> > >tests.
> > >
> > >We really should try to keep the test runtimes down, as well as the
> number
> > >of failures - so that the pre-commit runs can provide useful
> information.
> > >
> > >I'm not sure what the current approach w.r.t precommit runs before a
> > >commit. What I've seen in other projects is that the pre-commit needs to
> > >run, and come back clean (mostly) before a commit goes in. Between what
> > >used to be 5 day wait times, and inconsistent runs - I don't think this
> is
> > >always followed in Hive.
> > >
> > >It'll be useful to start relying on pre-commit test results again. Given
> > >the flaky tests, I'd suggest the following
> > >1. Pre-commit must be run on a patch before committing (with very few
> > >exceptions)
> > >2. A green test run is ideal
> > >3. In case there are failures - keep track of these as sub-jiras under a
> > >flaky test umbrella jira (Some under HIVE-14547 already) - to be
> > >eventually
> > >fixed.
> > >4. Before committing - cite relevant jiras for a flaky test (create and
> > >cite if it doesn't already exist).
> > >
> > >This should help us build up a list of flaky tests over various runs,
> > >which
> > >will hopefully get fixed at some point.
> > >
> > >Thoughts?
> > >
> > >Thanks,
> > >Sid
> >
> >
>

Re: Review Request 52923: HIVE-14979 Removing stale Zookeeper locks at HiveServer2 initialization

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52923/#review153357
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (lines 1766 - 1772)
<https://reviews.apache.org/r/52923/#comment222656>

Looks good, thanks.  Just one more nit (on 2 lines):  Zookeeper -> 
ZooKeeper.

See https://zookeeper.apache.org if you don't believe me.  ;)


- Lefty Leverenz


On Oct. 19, 2016, 11:50 a.m., Peter Vary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52923/
> ---
> 
> (Updated Oct. 19, 2016, 11:50 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Marta Kuczora, Miklos Csanady, 
> namit jain, Sergio Pena, and Barna Zsombor Klara.
> 
> 
> Bugs: HIVE-14979
> https://issues.apache.org/jira/browse/HIVE-14979
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Adding a new configuration option to HiveConf to signal whether stale lock 
> removal is requested on startup.
> Adding a new method to ZooKeeperHiveLockManager to remove stale locks
> Modifying the HiveServer2 to instantiate a lock manager and call the new 
> method if defined by the configuration.
> 
> Please take extra care when reviewing these:
> - Modifying the lock fetching method to use the clientIp from the lock, and 
> not update with the current ip - Not sure why it was done before
> - Instantiation of the lock manager - I might not chose the best method for it
> 
> Open for any suggestions :)
> 
> Thanks,
> Peter
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8ffae3b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
>  14d0ef4 
>   
> ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java
>  3f9926e 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 590b1f3 
> 
> Diff: https://reviews.apache.org/r/52923/diff/
> 
> 
> Testing
> ---
> 
> Created 2 unit test cases:
> - Removing own locks
> - Not removing other server's locks
> 
> Manually tested the Lock manager instantiation method on HiveServer2
> 
> 
> Thanks,
> 
> Peter Vary
> 
>

Re: Review Request 52923: HIVE-14979 Removing stale Zookeeper locks at HiveServer2 initialization

[Lefty Leverenz]

> On Oct. 18, 2016, 7:28 p.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, lines 1766-1772
> > <https://reviews.apache.org/r/52923/diff/2/?file=1540463#file1540463line1766>
> >
> > 1. camel caps "zookeeper" -> "ZooKeeper"
> > 2. all-caps "sql" -> "SQL" and final period
> > 3. (to separate "SQL" from the commands) rephrase as "using the SQL 
> > commands UNLOCK TABLE, UNLOCK DATABASE."
> > 4. are the UNLOCK commands part of ZooKeeper? then maybe you should say 
> > "the ZooKeeper commands" instead of "the SQL commands"
> > 
> > Congrats on fixing some other typos before I saw them!

Thanks Peter for nudging me to publish my comments.  (Oops.)


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52923/#review153057
---


On Oct. 18, 2016, 9:23 a.m., Peter Vary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52923/
> ---
> 
> (Updated Oct. 18, 2016, 9:23 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Marta Kuczora, Miklos Csanady, 
> namit jain, Sergio Pena, and Barna Zsombor Klara.
> 
> 
> Bugs: HIVE-14979
> https://issues.apache.org/jira/browse/HIVE-14979
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Adding a new configuration option to HiveConf to signal whether stale lock 
> removal is requested on startup.
> Adding a new method to ZooKeeperHiveLockManager to remove stale locks
> Modifying the HiveServer2 to instantiate a lock manager and call the new 
> method if defined by the configuration.
> 
> Please take extra care when reviewing these:
> - Modifying the lock fetching method to use the clientIp from the lock, and 
> not update with the current ip - Not sure why it was done before
> - Instantiation of the lock manager - I might not chose the best method for it
> 
> Open for any suggestions :)
> 
> Thanks,
> Peter
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8ffae3b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
>  14d0ef4 
>   
> ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java
>  3f9926e 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 590b1f3 
> 
> Diff: https://reviews.apache.org/r/52923/diff/
> 
> 
> Testing
> ---
> 
> Created 2 unit test cases:
> - Removing own locks
> - Not removing other server's locks
> 
> Manually tested the Lock manager instantiation method on HiveServer2
> 
> 
> Thanks,
> 
> Peter Vary
> 
>

Re: Review Request 52923: HIVE-14979 Removing stale Zookeeper locks at HiveServer2 initialization

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52923/#review153057
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (lines 1766 - 1772)
<https://reviews.apache.org/r/52923/#comment222332>

1. camel caps "zookeeper" -> "ZooKeeper"
2. all-caps "sql" -> "SQL" and final period
3. (to separate "SQL" from the commands) rephrase as "using the SQL 
commands UNLOCK TABLE, UNLOCK DATABASE."
4. are the UNLOCK commands part of ZooKeeper? then maybe you should say 
"the ZooKeeper commands" instead of "the SQL commands"

Congrats on fixing some other typos before I saw them!


- Lefty Leverenz


On Oct. 18, 2016, 9:23 a.m., Peter Vary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52923/
> ---
> 
> (Updated Oct. 18, 2016, 9:23 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Marta Kuczora, Miklos Csanady, 
> namit jain, Sergio Pena, and Barna Zsombor Klara.
> 
> 
> Bugs: HIVE-14979
> https://issues.apache.org/jira/browse/HIVE-14979
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Adding a new configuration option to HiveConf to signal whether stale lock 
> removal is requested on startup.
> Adding a new method to ZooKeeperHiveLockManager to remove stale locks
> Modifying the HiveServer2 to instantiate a lock manager and call the new 
> method if defined by the configuration.
> 
> Please take extra care when reviewing these:
> - Modifying the lock fetching method to use the clientIp from the lock, and 
> not update with the current ip - Not sure why it was done before
> - Instantiation of the lock manager - I might not chose the best method for it
> 
> Open for any suggestions :)
> 
> Thanks,
> Peter
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8ffae3b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
>  14d0ef4 
>   
> ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java
>  3f9926e 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 590b1f3 
> 
> Diff: https://reviews.apache.org/r/52923/diff/
> 
> 
> Testing
> ---
> 
> Created 2 unit test cases:
> - Removing own locks
> - Not removing other server's locks
> 
> Manually tested the Lock manager instantiation method on HiveServer2
> 
> 
> Thanks,
> 
> Peter Vary
> 
>

Re: [jira] [Created] (HIVE-14845) There is no example about how to use Webhcat Rest api when enable kerberos

[Lefty Leverenz]

The Hive Web Interface is a GUI which is unrelated to the WebHCat API.  The
WebHCat documentation is here:  WebHCat
.

The Configuration page

describes a few parameters to set for Kerberos
(see templeton.hive.properties and templeton.kerberos.*).

Do you need more information than that?

-- Lefty


On Tue, Sep 27, 2016 at 4:09 AM, sangshenghong (JIRA) 
wrote:

> sangshenghong created HIVE-14845:
> 
>
>  Summary: There is no example about how to use Webhcat Rest
> api  when enable kerberos
>  Key: HIVE-14845
>  URL: https://issues.apache.org/jira/browse/HIVE-14845
>  Project: Hive
>   Issue Type: Wish
>   Components: WebHCat
> Reporter: sangshenghong
> Priority: Minor
>
>
> I am trying to use WebHcat rest api to create table and query table when
> enabling Kerberos, but there is no code example about how to use Kerberos
> auth to connect  Webhcat. please check the following link:
>
> https://cwiki.apache.org/confluence/display/Hive/HiveWebInterface
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

Re: [DISCUSS] Making storage-api a separately released artifact

[Lefty Leverenz]

Sergey's idea is creative, although it leads to confusion about JIRA fix
versions.  Issues would be given fix versions based on assumptions about
whether SA or Hive will be released first.  (That's hard to predict when
it's months away.)

Keeping the version numbers tied together is very appealing.  Would it be
possible to have incompatible changes in SA force a bump in the Hive
release number?  Hm, I guess that means Hive would need a release at the
same time as SA, but only for incompatible changes.

What's the likelihood of another subproject getting spun off eventually?
If that happened, the 4th minor version wouldn't make sense.  A 5th minor
version wouldn't work either.

-- Lefty


On Fri, Aug 19, 2016 at 9:46 PM, Sergey Shelukhin 
wrote:

> I am suggesting we always skip the number. So only one component gets the
> next one :) In your example Hive trunk would be 2.3, and if SA is released
> again it would become 2.4. Otherwise we’d need a compat table cause
> versions will be totally out of sync.
>
> On 16/8/19, 16:31, "Owen O'Malley"  wrote:
>
> >That won't necessarily work, especially in the beginning. If we release SA
> >2.2.0 and use it for Hive trunk with the assumption that the next Hive
> >release will be 2.2. What do we do when we need to make an incompatible
> >change in SA? I guess we could release SA as 2.3.0 and when hive makes its
> >next release skip over Hive 2.2 and go straight to Hive 2.3.0. In general
> >I
> >think that we'd be better off with the release numbers not tied together.
> >
> >.. Owen
> >
> >On Fri, Aug 19, 2016 at 4:14 PM, Sergey Shelukhin  >
> >wrote:
> >
> >> Can we just run the versions thru? I.e. increment it every time but
> >> release only one component (or both if they happen to align I guess).
> >> E.g. storage-api will be released at 2.2, and say 2.3 if it moves fast,
> >> then Hive 2.4, then storage-api 2.5, etc.
> >> That might make it easier to reason about compatibility because the
> >>order
> >> is obvious.
> >>
> >> On 16/8/19, 09:04, "Sergio Pena"  wrote:
> >>
> >> >I see Parquet is currently using the SearchArgument class for
> >>predicates
> >> >push down.
> >> >Will this class be part of the new sub-module or project?
> >> >
> >> >Following Sushanth idea, can we have other API interfaces in the new
> >> >project that other components can use?
> >> >Perhaps having this may be a good reason to create a project.
> >> >
> >> >I'm -1 with the 4th minor version. As Owen mentioned, changing the 4th
> >> >version number for incompatible changes is ugly and confusing.
> >> >I like the new project idea more, +1, but  the storage-api may be too
> >> >small
> >> >for a new project.
> >> >
> >> >- Sergio
> >> >
> >> >On Wed, Aug 17, 2016 at 2:05 PM, Owen O'Malley 
> >> wrote:
> >> >
> >> >> On Wed, Aug 17, 2016 at 10:46 AM, Alan Gates 
> >> >>wrote:
> >> >>
> >> >> > +1 for making the API clean and easy for other projects to work
> >>with.
> >> >> A
> >> >> > few questions:
> >> >> >
> >> >> > 1) Would this also make it easier for Parquet and others to
> >>implement
> >> >> > Hive’s ACID interfaces?
> >> >> >
> >> >>
> >> >> Currently the ACID interfaces haven't been moved over to storage-api,
> >> >> although it would make sense to do so at some point.
> >> >>
> >> >>
> >> >> >
> >> >> > 2) Would we make any attempt to coordinate version numbers between
> >> >>Hive
> >> >> > and the storage module, or would a given version of Hive just
> >>depend
> >> >>on a
> >> >> > given version of the storage module?
> >> >> >
> >> >>
> >> >> The two options that I see are:
> >> >>
> >> >> * Let the numbers run separately starting from 2.2.0.
> >> >> * Tie the numbers together with an additional level of versioning
> >>(eg.
> >> >> 2.2.0.0).
> >> >>
> >> >> I think that letting the two version numbers diverge is better in the
> >> >>long
> >> >> term. For example, if you need to make an incompatible change, it is
> >> >>pretty
> >> >> ugly to do it as a fourth level version number (eg. an incompatible
> >> >>change
> >> >> from 2.2.0.0 to 2.2.0.1). At the beginning, I expect that storage-api
> >> >>would
> >> >> move faster than Hive, but as it stabilizes I expect it might start
> >> >>moving
> >> >> slower than Hive.
> >> >>
> >> >> I'd propose that we have Hive's build use a released version of
> >> >>storage-api
> >> >> rather than a snapshot.
> >> >>
> >> >> Thoughts?
> >> >>
> >> >>Owen
> >> >>
> >> >>
> >> >> > Alan.
> >> >> >
> >> >> > > On Aug 15, 2016, at 17:01, Owen O'Malley 
> >> wrote:
> >> >> > >
> >> >> > > All,
> >> >> > >
> >> >> > > As part of moving ORC out of Hive, we pulled all of the
> >> >>vectorization
> >> >> > > storage and sarg classes into a separate module, which is named
> >> >> > > storage-api.  Although it is currently only used by ORC, it
> >>could be
> >> >> used
> >> >> > > by 

Re: Review Request 50359: HIVE-14270: Write temporary data to HDFS when doing inserts on tables located on S3

[Lefty Leverenz]

> On Aug. 10, 2016, 5:31 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, lines 3091-3092
> > <https://reviews.apache.org/r/50359/diff/7/?file=1469104#file1469104line3091>
> >
> > Tiny nit:  Either make "It" lowercase or move the parenthetical 
> > sentence after the first sentence, with a final period like this:
> > 
> > "Enable the use of scratch directories directly on blob storage 
> > systems. (It may cause performance penalties.)"

Looks good now.  +1 for the parameter descriptions.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50359/#review145307
---


On Aug. 10, 2016, 9:08 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50359/
> ---
> 
> (Updated Aug. 10, 2016, 9:08 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14270
> https://issues.apache.org/jira/browse/HIVE-14270
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch will create a temporary directory for Hive intermediate data on 
> HDFS when S3 tables are used.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/BlobStorageUtils.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 9f5f619359701b948f57d599a5bdc2ecbdff280a 
>   common/src/test/org/apache/hadoop/hive/common/TestBlobStorageUtils.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/Context.java 
> 89893eba9fd2316b9a393f06edefa837bb815faf 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 5bd78862e1064d7f64a5d764571015a8df1101e8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> a01a7bdbfec962b6617e98091cdb1325c5b0e84f 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestContext.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50359/diff/
> 
> 
> Testing
> ---
> 
> NO PATCH
> ** NON-PARTITIONED TABLE
> 
> - create table dummy (id int);
>3.651s
> - insert into table s3dummy values (1);   
>   39.231s
> - insert overwrite table s3dummy values (1);  
>   42.569s
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 30.136s
> 
> EXTERNAL TABLE
> 
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   9.297s
> - insert into table s3dummy_ext values (1);   
>   45.855s
> 
> WITH PATCH
> 
> ** NON-PARTITIONED TABLE
> - create table s3dummy (id int) location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   3.945s
> - insert into table s3dummy values (1);   
>   15.025s
> - insert overwrite table s3dummy values (1);  
>   25.149s 
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 19.158s  
> - from dummy insert overwrite table s3dummy select *; 
>   25.469s  
> - from dummy insert into table s3dummy select *;  
>   14.501s
> 
> ** EXTERNAL TABLE
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   4.827s
> - insert into table s3dummy_ext values (1);   
>   16.070s
> 
> ** PARTITIONED TABLE
> - create table s3dummypart (id int) partitioned by (part int)
>   location 's3a://spena-bucket/user/hive/warehouse/s3dummypart';  
>3.176s
> - alter table s3dummypart add partition (part=1); 
>3.229s
> - alter table s3dummypart add partition (part=2); 
>3.124s
> - insert into table s3dummypart partition (part=1) values (1);
>   14.876s
> - insert overwrite table s3dummypart partition (part=1) values 

Re: Permission to edit comments and close my Jira

[Lefty Leverenz]

Marta, I believe you need committer status to resolve a Jira issue, so I
did it for you (see HIVE-14387
).

For editing your own comments, perhaps someone else can tell you what
permission is needed.

-- Lefty


On Mon, Aug 8, 2016 at 10:32 AM, Marta Kuczora 
wrote:

> Hello,
>
> I noticed that I cannot edit my comments on a Hive Jira.
> I am also not able to resolve my Jiras. I have the HIVE-14387 issue which
> could be closed, but I am not able to set its status.
>
> Could one of the Hive Jira admins please give me permission to edit my
> comments and close my issues? My username is kuczoram.
>
> Thanks and regards,
> Marta
>

Re: Review Request 50359: HIVE-14270: Write temporary data to HDFS when doing inserts on tables located on S3

[Lefty Leverenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50359/#review145307
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (lines 3091 - 3092)
<https://reviews.apache.org/r/50359/#comment211495>

Tiny nit:  Either make "It" lowercase or move the parenthetical sentence 
after the first sentence, with a final period like this:

"Enable the use of scratch directories directly on blob storage systems. 
(It may cause performance penalties.)"


- Lefty Leverenz


On Aug. 9, 2016, 7:53 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50359/
> ---
> 
> (Updated Aug. 9, 2016, 7:53 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14270
> https://issues.apache.org/jira/browse/HIVE-14270
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch will create a temporary directory for Hive intermediate data on 
> HDFS when S3 tables are used.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/BlobStorageUtils.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 9f5f619359701b948f57d599a5bdc2ecbdff280a 
>   common/src/test/org/apache/hadoop/hive/common/TestBlobStorageUtils.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/Context.java 
> 89893eba9fd2316b9a393f06edefa837bb815faf 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 5bd78862e1064d7f64a5d764571015a8df1101e8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> a01a7bdbfec962b6617e98091cdb1325c5b0e84f 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestContext.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50359/diff/
> 
> 
> Testing
> ---
> 
> NO PATCH
> ** NON-PARTITIONED TABLE
> 
> - create table dummy (id int);
>3.651s
> - insert into table s3dummy values (1);   
>   39.231s
> - insert overwrite table s3dummy values (1);  
>   42.569s
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 30.136s
> 
> EXTERNAL TABLE
> 
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   9.297s
> - insert into table s3dummy_ext values (1);   
>   45.855s
> 
> WITH PATCH
> 
> ** NON-PARTITIONED TABLE
> - create table s3dummy (id int) location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   3.945s
> - insert into table s3dummy values (1);   
>   15.025s
> - insert overwrite table s3dummy values (1);  
>   25.149s 
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 19.158s  
> - from dummy insert overwrite table s3dummy select *; 
>   25.469s  
> - from dummy insert into table s3dummy select *;  
>   14.501s
> 
> ** EXTERNAL TABLE
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   4.827s
> - insert into table s3dummy_ext values (1);   
>   16.070s
> 
> ** PARTITIONED TABLE
> - create table s3dummypart (id int) partitioned by (part int)
>   location 's3a://spena-bucket/user/hive/warehouse/s3dummypart';  
>3.176s
> - alter table s3dummypart add partition (part=1); 
>3.229s
> - alter table s3dummypart add partition (part=2); 
>3.124s
> - insert into table s3dummypart partition (part=1) values (1);
>   14.876s
> - insert overwrite table s3dummypart partition (part=1) values (1);   
>   27.594s 
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummypart' select * 
> from dummypart; 22.298s  
> - 

Re: Review Request 50359: HIVE-14270: Write temporary data to HDFS when doing inserts on tables located on S3

[Lefty Leverenz]

> On July 30, 2016, 8:44 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, lines 3066-3067
> > <https://reviews.apache.org/r/50359/diff/5/?file=1456811#file1456811line3066>
> >
> > Typo:  Commad-separated --> Comma-separated
> > 
> > Redundancy:  "... supported blobstore schemes that Hive officially 
> > supports" (omit "supported")
> > 
> > Nit:  A period could be added at the end.
> 
> Lefty Leverenz wrote:
> Looks good now, thanks Sergio.

Aarrgh, forgot to publish that.

Adding a trivial comment for the new config.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50359/#review144255
---


On Aug. 9, 2016, 7:53 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50359/
> ---
> 
> (Updated Aug. 9, 2016, 7:53 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14270
> https://issues.apache.org/jira/browse/HIVE-14270
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch will create a temporary directory for Hive intermediate data on 
> HDFS when S3 tables are used.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/BlobStorageUtils.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 9f5f619359701b948f57d599a5bdc2ecbdff280a 
>   common/src/test/org/apache/hadoop/hive/common/TestBlobStorageUtils.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/Context.java 
> 89893eba9fd2316b9a393f06edefa837bb815faf 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 5bd78862e1064d7f64a5d764571015a8df1101e8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> a01a7bdbfec962b6617e98091cdb1325c5b0e84f 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestContext.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50359/diff/
> 
> 
> Testing
> ---
> 
> NO PATCH
> ** NON-PARTITIONED TABLE
> 
> - create table dummy (id int);
>3.651s
> - insert into table s3dummy values (1);   
>   39.231s
> - insert overwrite table s3dummy values (1);  
>   42.569s
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 30.136s
> 
> EXTERNAL TABLE
> 
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   9.297s
> - insert into table s3dummy_ext values (1);   
>   45.855s
> 
> WITH PATCH
> 
> ** NON-PARTITIONED TABLE
> - create table s3dummy (id int) location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   3.945s
> - insert into table s3dummy values (1);   
>   15.025s
> - insert overwrite table s3dummy values (1);  
>   25.149s 
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 19.158s  
> - from dummy insert overwrite table s3dummy select *; 
>   25.469s  
> - from dummy insert into table s3dummy select *;  
>   14.501s
> 
> ** EXTERNAL TABLE
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   4.827s
> - insert into table s3dummy_ext values (1);   
>   16.070s
> 
> ** PARTITIONED TABLE
> - create table s3dummypart (id int) partitioned by (part int)
>   location 's3a://spena-bucket/user/hive/warehouse/s3dummypart';  
>3.176s
> - alter table s3dummypart add partition (part=1); 
>3.229s
> - alter table s3dummypart add partition (part=2); 
>3.124s
> - insert into table s3dummypart partition (part=1) values (1);
>   14.876s
> - in

Re: Review Request 50359: HIVE-14270: Write temporary data to HDFS when doing inserts on tables located on S3

[Lefty Leverenz]

> On July 30, 2016, 8:44 a.m., Lefty Leverenz wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, lines 3066-3067
> > <https://reviews.apache.org/r/50359/diff/5/?file=1456811#file1456811line3066>
> >
> > Typo:  Commad-separated --> Comma-separated
> > 
> > Redundancy:  "... supported blobstore schemes that Hive officially 
> > supports" (omit "supported")
> > 
> > Nit:  A period could be added at the end.

Looks good now, thanks Sergio.


- Lefty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50359/#review144255
---


On Aug. 9, 2016, 7:53 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50359/
> ---
> 
> (Updated Aug. 9, 2016, 7:53 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-14270
> https://issues.apache.org/jira/browse/HIVE-14270
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> This patch will create a temporary directory for Hive intermediate data on 
> HDFS when S3 tables are used.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/BlobStorageUtils.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 9f5f619359701b948f57d599a5bdc2ecbdff280a 
>   common/src/test/org/apache/hadoop/hive/common/TestBlobStorageUtils.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/Context.java 
> 89893eba9fd2316b9a393f06edefa837bb815faf 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 5bd78862e1064d7f64a5d764571015a8df1101e8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> a01a7bdbfec962b6617e98091cdb1325c5b0e84f 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestContext.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50359/diff/
> 
> 
> Testing
> ---
> 
> NO PATCH
> ** NON-PARTITIONED TABLE
> 
> - create table dummy (id int);
>3.651s
> - insert into table s3dummy values (1);   
>   39.231s
> - insert overwrite table s3dummy values (1);  
>   42.569s
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 30.136s
> 
> EXTERNAL TABLE
> 
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   9.297s
> - insert into table s3dummy_ext values (1);   
>   45.855s
> 
> WITH PATCH
> 
> ** NON-PARTITIONED TABLE
> - create table s3dummy (id int) location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   3.945s
> - insert into table s3dummy values (1);   
>   15.025s
> - insert overwrite table s3dummy values (1);  
>   25.149s 
> - insert overwrite directory 's3a://spena-bucket/dirs/s3dummy' select * from 
> dummy; 19.158s  
> - from dummy insert overwrite table s3dummy select *; 
>   25.469s  
> - from dummy insert into table s3dummy select *;  
>   14.501s
> 
> ** EXTERNAL TABLE
> - create table s3dummy_ext like s3dummy location 
> 's3a://spena-bucket/user/hive/warehouse/s3dummy';   4.827s
> - insert into table s3dummy_ext values (1);   
>   16.070s
> 
> ** PARTITIONED TABLE
> - create table s3dummypart (id int) partitioned by (part int)
>   location 's3a://spena-bucket/user/hive/warehouse/s3dummypart';  
>3.176s
> - alter table s3dummypart add partition (part=1); 
>3.229s
> - alter table s3dummypart add partition (part=2); 
>3.124s
> - insert into table s3dummypart partition (part=1) values (1);
>   14.876s
> - insert overwrite table s3dummypart partition (part=1) values (1);   
>   27.594s 
> - i