Review Request 68124: HIVE-20252

[Deepak Jaiswal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68124/
---

Review request for hive, Jesús Camacho Rodríguez and Jason Dere.


Bugs: HIVE-20252
https://issues.apache.org/jira/browse/HIVE-20252


Repository: hive-git


Description
---

See Jira.

removeSemiJoinCyclesDueToMapsideJoins is deprecated, although it has changes. I 
will eventually remove it and can be ignored.


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 
011dadf495 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java c3eb886fd2 


Diff: https://reviews.apache.org/r/68124/diff/1/


Testing
---


Thanks,

Deepak Jaiswal

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Anshuman Dwivedi]

Congrats to Vineet, Vihang, Peter, Slim.



Rgds,
Anshuman Dwivedi




-Anishek Agarwal  wrote: -

To: dev@hive.apache.org
From: Anishek Agarwal 
Date: 07/31/2018 10:51AM
Subject: Re: [ANNOUNCE] New PMC Member : Vineet Garg


Congrats Vineet!

On Tue, Jul 31, 2018 at 10:30 AM Vineet Garg  wrote:

> Thanks all!
>
> > On Jul 30, 2018, at 6:07 PM, Vihang Karajgaonkar
>  wrote:
> >
> > Congrats Vineet!
> >
> > On Mon, Jul 30, 2018 at 4:04 PM, Prasanth Jayachandran <
> > pjayachand...@hortonworks.com> wrote:
> >
> >> Congratulations Vineet!
> >>
> >> Thanks
> >> Prasanth
> >>
> >>
> >>
> >> On Mon, Jul 30, 2018 at 12:11 PM -0700, "Xuefu Zhang"
> >> mailto:xu...@uber.com.INVALID>> wrote:
> >>
> >>
> >> Congratulations!
> >>
> >> On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
> >> jcamachorodrig...@hortonworks.com> wrote:
> >>
> >>> Congrats Vineet!
> >>>
> >>> On 7/30/18, 10:53 AM, "Andrew Sherman"
> >>> wrote:
> >>>
> >>>Congratulations Vineet!
> >>>
> >>>On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal <
> >>> djais...@hortonworks.com>
> >>>wrote:
> >>>
>  Congratulations Vineet!
> 
>  On 7/30/18, 12:45 AM, "Peter Vary"
> >>> wrote:
> 
> Congratulations Vineet!
> 
> > On Jul 30, 2018, at 01:59, Ashutosh Chauhan <
> >>> hashut...@apache.org>
>  wrote:
> >
> > On behalf of the Hive PMC I am delighted to announce Vineet
> >>> Garg is
>  joining
> > Hive PMC.
> > Thanks Vineet for all your contributions till now. Looking
> >>> forward
>  to many
> > more.
> >
> > Welcome, Vineet!
> >
> > Thanks,
> > Ashutosh
> 
> 
> 
> 
> >>>
> >>>
> >>>
> >>
> >>
>
>
=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

Re: [ANNOUNCE] New PMC Member : Vihang Karajgaonkar

[Anishek Agarwal]

Congratulations Vihang!

On Tue, Jul 31, 2018 at 6:39 AM Vihang Karajgaonkar
 wrote:

> Thanks Everyone!
>
> On Mon, Jul 30, 2018 at 4:05 PM, Prasanth Jayachandran <
> pjayachand...@hortonworks.com> wrote:
>
> > Congratulations Vihang!
> >
> > Thanks
> > Prasanth
> >
> >
> >
> > On Mon, Jul 30, 2018 at 12:34 PM -0700, "Xuefu Zhang"
> > mailto:xu...@uber.com.INVALID>> wrote:
> >
> >
> > Congratulations, Vihang!
> >
> > On Mon, Jul 30, 2018 at 10:53 AM, Andrew Sherman <
> > asher...@cloudera.com.invalid> wrote:
> >
> > > Congratulations Vihang!
> > >
> > >
> > >
> > > On Mon, Jul 30, 2018 at 12:44 AM Peter Vary
> > > wrote:
> > >
> > > > Congratulations Vihang!
> > > >
> > > > > On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> > > > >
> > > > > Congratulations Vihang!
> > > > >
> > > > >> On Jul 26, 2018, at 11:27 AM, Ashutosh Chauhan
> > > > wrote:
> > > > >>
> > > > >> On behalf of the Hive PMC I am delighted to announce Vihang
> > > > Karajgaonkar
> > > > >> is joining Hive PMC.
> > > > >> Thanks Vihang for all your contributions till now. Looking forward
> > to
> > > > many
> > > > >> more.
> > > > >>
> > > > >> Welcome, Vihang!
> > > > >>
> > > > >> Thanks,
> > > > >> Ashutosh
> > > > >
> > > >
> > > >
> > >
> >
> >
>

Re: [ANNOUNCE] New PMC Member : Peter Vary

[Anishek Agarwal]

Congratulations Peter!

On Tue, Jul 31, 2018 at 6:38 AM Vihang Karajgaonkar
 wrote:

> Congrats Peter!
>
> On Mon, Jul 30, 2018 at 4:04 PM, Prasanth Jayachandran <
> pjayachand...@hortonworks.com> wrote:
>
> > Congratulations Peter!
> >
> > Thanks
> > Prasanth
> >
> >
> >
> > On Mon, Jul 30, 2018 at 12:21 PM -0700, "Xuefu Zhang"
> > mailto:xu...@uber.com.INVALID>> wrote:
> >
> >
> > Congratulations, Peter!
> >
> > On Mon, Jul 30, 2018 at 12:11 PM, Jesus Camacho Rodriguez <
> > jcamachorodrig...@hortonworks.com> wrote:
> >
> > > Congrats Peter!
> > >
> > > On 7/30/18, 10:53 AM, "Andrew Sherman"
> > > wrote:
> > >
> > > Congratulations Peter!
> > >
> > > On Sun, Jul 29, 2018 at 1:32 PM Vineet Garg
> > > wrote:
> > >
> > > > Congratulations Peter!
> > > >
> > > > > On Jul 26, 2018, at 11:25 AM, Ashutosh Chauhan <
> > > hashut...@apache.org>
> > > > wrote:
> > > > >
> > > > > On behalf of the Hive PMC I am delighted to announce Peter Vary
> > is
> > > > joining
> > > > > Hive PMC.
> > > > > Thanks Peter for all your contributions till now. Looking
> forward
> > > to many
> > > > > more.
> > > > >
> > > > > Welcome, Peter!
> > > > >
> > > > > Thanks,
> > > > > Ashutosh
> > > >
> > > >
> > >
> > >
> > >
> >
> >
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Anishek Agarwal]

Congratulations Slim!

On Tue, Jul 31, 2018 at 10:29 AM Vineet Garg  wrote:

> Congrats Slim!
>
> > On Jul 30, 2018, at 6:08 PM, Vihang Karajgaonkar
>  wrote:
> >
> > Congrats Slim!
> >
> > On Mon, Jul 30, 2018 at 4:35 PM, Deepak Jaiswal <
> djais...@hortonworks.com>
> > wrote:
> >
> >> Congrats Slim!
> >>
> >> On 7/30/18, 4:03 PM, "Prasanth Jayachandran" <
> >> pjayachand...@hortonworks.com> wrote:
> >>
> >>Congratulations Slim!
> >>
> >>> On Jul 30, 2018, at 4:00 PM, Sergey Shelukhin <
> >> ser...@hortonworks.com> wrote:
> >>>
> >>> Congrats!
> >>>
> >>> On 18/7/30, 12:53, "Gunther Hagleitner"  >>>
> >>> wrote:
> >>>
>  Congratulations!
> 
>  Thanks,
>  Gunther.
>  
>  From: Xuefu Zhang 
>  Sent: Monday, July 30, 2018 12:11 PM
>  To: dev@hive.apache.org
>  Subject: Re: [ANNOUNCE] New committer: Slim Bouguerra
> 
>  congratulations!!!
> 
>  On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
>  jcamachorodrig...@hortonworks.com> wrote:
> 
> > Congrats Slim!
> >
> > On 7/30/18, 10:53 AM, "Andrew Sherman"
> >> 
> > wrote:
> >
> >   Congratulations Slim!
> >
> >   On Mon, Jul 30, 2018 at 12:46 AM Peter Vary
> >  >>
> >   wrote:
> >
> >> Congratulations Slim!
> >>
> >>> On Jul 30, 2018, at 02:00, Ashutosh Chauhan
> > 
> >> wrote:
> >>>
> >>> Apache Hive's Project Management Committee (PMC) has invited
> > Slim
> >> Bouguerra
> >>> to become a committer, and we are pleased to announce that he
> > has
> >> accepted.
> >>>
> >>> Slim, welcome, thank you for your contributions, and we look
> > forward your
> >>> further interactions with the community!
> >>>
> >>> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> >>
> >>
> >
> >
> >
> >>>
> >>
> >>
> >>
> >>
>
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Anishek Agarwal]

Congrats Vineet!

On Tue, Jul 31, 2018 at 10:30 AM Vineet Garg  wrote:

> Thanks all!
>
> > On Jul 30, 2018, at 6:07 PM, Vihang Karajgaonkar
>  wrote:
> >
> > Congrats Vineet!
> >
> > On Mon, Jul 30, 2018 at 4:04 PM, Prasanth Jayachandran <
> > pjayachand...@hortonworks.com> wrote:
> >
> >> Congratulations Vineet!
> >>
> >> Thanks
> >> Prasanth
> >>
> >>
> >>
> >> On Mon, Jul 30, 2018 at 12:11 PM -0700, "Xuefu Zhang"
> >> mailto:xu...@uber.com.INVALID>> wrote:
> >>
> >>
> >> Congratulations!
> >>
> >> On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
> >> jcamachorodrig...@hortonworks.com> wrote:
> >>
> >>> Congrats Vineet!
> >>>
> >>> On 7/30/18, 10:53 AM, "Andrew Sherman"
> >>> wrote:
> >>>
> >>>Congratulations Vineet!
> >>>
> >>>On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal <
> >>> djais...@hortonworks.com>
> >>>wrote:
> >>>
>  Congratulations Vineet!
> 
>  On 7/30/18, 12:45 AM, "Peter Vary"
> >>> wrote:
> 
> Congratulations Vineet!
> 
> > On Jul 30, 2018, at 01:59, Ashutosh Chauhan <
> >>> hashut...@apache.org>
>  wrote:
> >
> > On behalf of the Hive PMC I am delighted to announce Vineet
> >>> Garg is
>  joining
> > Hive PMC.
> > Thanks Vineet for all your contributions till now. Looking
> >>> forward
>  to many
> > more.
> >
> > Welcome, Vineet!
> >
> > Thanks,
> > Ashutosh
> 
> 
> 
> 
> >>>
> >>>
> >>>
> >>
> >>
>
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Vineet Garg]

Thanks all!

> On Jul 30, 2018, at 6:07 PM, Vihang Karajgaonkar 
>  wrote:
> 
> Congrats Vineet!
> 
> On Mon, Jul 30, 2018 at 4:04 PM, Prasanth Jayachandran <
> pjayachand...@hortonworks.com> wrote:
> 
>> Congratulations Vineet!
>> 
>> Thanks
>> Prasanth
>> 
>> 
>> 
>> On Mon, Jul 30, 2018 at 12:11 PM -0700, "Xuefu Zhang"
>> mailto:xu...@uber.com.INVALID>> wrote:
>> 
>> 
>> Congratulations!
>> 
>> On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
>> jcamachorodrig...@hortonworks.com> wrote:
>> 
>>> Congrats Vineet!
>>> 
>>> On 7/30/18, 10:53 AM, "Andrew Sherman"
>>> wrote:
>>> 
>>>Congratulations Vineet!
>>> 
>>>On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal <
>>> djais...@hortonworks.com>
>>>wrote:
>>> 
 Congratulations Vineet!
 
 On 7/30/18, 12:45 AM, "Peter Vary"
>>> wrote:
 
Congratulations Vineet!
 
> On Jul 30, 2018, at 01:59, Ashutosh Chauhan <
>>> hashut...@apache.org>
 wrote:
> 
> On behalf of the Hive PMC I am delighted to announce Vineet
>>> Garg is
 joining
> Hive PMC.
> Thanks Vineet for all your contributions till now. Looking
>>> forward
 to many
> more.
> 
> Welcome, Vineet!
> 
> Thanks,
> Ashutosh
 
 
 
 
>>> 
>>> 
>>> 
>> 
>> 

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Vineet Garg]

Congrats Slim!

> On Jul 30, 2018, at 6:08 PM, Vihang Karajgaonkar 
>  wrote:
> 
> Congrats Slim!
> 
> On Mon, Jul 30, 2018 at 4:35 PM, Deepak Jaiswal 
> wrote:
> 
>> Congrats Slim!
>> 
>> On 7/30/18, 4:03 PM, "Prasanth Jayachandran" <
>> pjayachand...@hortonworks.com> wrote:
>> 
>>Congratulations Slim!
>> 
>>> On Jul 30, 2018, at 4:00 PM, Sergey Shelukhin <
>> ser...@hortonworks.com> wrote:
>>> 
>>> Congrats!
>>> 
>>> On 18/7/30, 12:53, "Gunther Hagleitner" >> 
>>> wrote:
>>> 
 Congratulations!
 
 Thanks,
 Gunther.
 
 From: Xuefu Zhang 
 Sent: Monday, July 30, 2018 12:11 PM
 To: dev@hive.apache.org
 Subject: Re: [ANNOUNCE] New committer: Slim Bouguerra
 
 congratulations!!!
 
 On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
 jcamachorodrig...@hortonworks.com> wrote:
 
> Congrats Slim!
> 
> On 7/30/18, 10:53 AM, "Andrew Sherman"
>> 
> wrote:
> 
>   Congratulations Slim!
> 
>   On Mon, Jul 30, 2018 at 12:46 AM Peter Vary
> > 
>   wrote:
> 
>> Congratulations Slim!
>> 
>>> On Jul 30, 2018, at 02:00, Ashutosh Chauhan
> 
>> wrote:
>>> 
>>> Apache Hive's Project Management Committee (PMC) has invited
> Slim
>> Bouguerra
>>> to become a committer, and we are pleased to announce that he
> has
>> accepted.
>>> 
>>> Slim, welcome, thank you for your contributions, and we look
> forward your
>>> further interactions with the community!
>>> 
>>> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
>> 
>> 
> 
> 
> 
>>> 
>> 
>> 
>> 
>> 

Review Request 68121: HIVE-20220 : Incorrect result when hive.groupby.skewindata is enabled

[Ganesha Shreedhara]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68121/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

hive.groupby.skewindata makes use of rand UDF to randomly distribute grouped by 
keys to the reducers and hence avoids overloading a single reducer when there 
is a skew in data. 

This random distribution of keys is buggy when the reducer fails to fetch the 
mapper output due to a faulty datanode or any other reason. When reducer finds 
that it can't fetch mapper output, it sends a signal to Application Master to 
reattempt the corresponding map task. The reattempted map task will now get the 
different random value from rand function and hence the keys that gets 
distributed now to the reducer will not be same as the previous run. 

 

Steps to reproduce:

create table test(id int);

insert into test values 
(1),(2),(2),(3),(3),(3),(4),(4),(4),(4),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7),(8),(8),(8),(8),(8),(8),(8),(8),(9),(9),(9),(9),(9),(9),(9),(9),(9);

SET hive.groupby.skewindata=true;

SET mapreduce.reduce.reduces=2;

//Add a debug port for reducer

select count(1) from test group by id;

//Remove mapper's intermediate output file when map stage is completed and one 
out of 2 reduce tasks is completed and then continue the run. This causes 2nd 
reducer to send event to Application Master to rerun the map task. 

The following is the expected result. 

1
2
3
4
5
6
8
8
9 

 

But you may get different result due to a different value returned by the rand 
function in the second run causing different distribution of keys.

This needs to be fixed such that the mapper distributes the same keys even if 
it is reattempted multiple times.


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 39c77b3fe5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 250a085084 
  ql/src/test/queries/clientpositive/groupby_skew_rand_seed.q PRE-CREATION 
  ql/src/test/queries/clientpositive/groupby_skew_rand_seed1.q PRE-CREATION 
  ql/src/test/results/clientpositive/groupby_skew_rand_seed.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/groupby_skew_rand_seed1.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/68121/diff/1/


Testing
---

Qtests added


Thanks,

Ganesha Shreedhara

[jira] [Created] (HIVE-20275) hive produces incorrect result when using MIN()/MAX() on varchar with hive.vectorized.reuse.scratch.columns enabled

[Rajkumar Singh (JIRA)]

Rajkumar Singh created HIVE-20275:
-

 Summary: hive produces incorrect result when using MIN()/MAX() on 
varchar with hive.vectorized.reuse.scratch.columns enabled
 Key: HIVE-20275
 URL: https://issues.apache.org/jira/browse/HIVE-20275
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 3.1.0
 Environment: Hive3.1,java8
Reporter: Rajkumar Singh


Steps to reproduce:
{code}
create table testhive3 (name varchar(8), `time` double);

 insert into table testhive3 values
 ('ABC', 1),
 ('ABC', 2),
 ('DEF', 1),
 ('DEF', 2),
 ('DEF', 1),
 ('DEF', 2),
 ('ABC', 1),
 ('ABC', 2),
 ('DEF', 1),
 ('DEF', 2),
 ('ABC', 1),
 ('ABC', 2),
 ('ABC', 1),
 ('ABC', 2),
 ('DEF', 1),
 ('DEF', 2),
 ('ABC', 1),
 ('ABC', 2),
 ('ABC', 1),
 ('ABC', 2),
 ('DEF', 1),
 ('DEF', 2),
 ('ABC', 1),
 ('ABC', 2),
 ('DEF', 1),
 ('DEF', 2),
 ( 'ABC', 1),
 ( NULL, NULL),
 ( 'ABC', 1),
 ( 'ABC', 2),
 ( 'DEF', 1),
 ('DEF', 2),
 ('ABC', 1),
 ( 'ABC', 2),
 ('ABC', 1),
 ( 'ABC', 2),
 ( 'DEF', 1),
 ('DEF', 2);

 select name, `time` from testhive3 where name = 'ABC' group by name, `time`;

 +---+---+
| name  | time  |
+---+---+
| ABC   | 1.0   |
| ABC   | 2.0   |
+---+---+


 select min(name), `time` from testhive3 where name = 'ABC' group by name, 
`time`;

 +---+---+
|  _c0  | time  |
+---+---+
| NULL  | 1.0   |
| NULL  | 2.0   |
+---+---+

set hive.vectorized.reuse.scratch.columns=false;
select min(name), `time` from testhive3 where name = 'ABC' group by name, 
`time`;
+--+---+
| _c0  | time  |
+--+---+
| ABC  | 1.0   |
| ABC  | 2.0   |
+--+---+
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68086: HIVE-20241

[Jesús Camacho Rodríguez]

> On July 31, 2018, 1:27 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
> > Lines 219 (patched)
> > 
> >
> > This token is unused. Can be deleted.

Thanks!


> On July 31, 2018, 1:27 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
> > Line 1990 (original), 1992 (patched)
> > 
> >
> > Looks this rule is unused and can be deleted.

Thanks!


- Jesús


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68086/#review206632
---


On July 31, 2018, 12:57 a.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68086/
> ---
> 
> (Updated July 31, 2018, 12:57 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-20241
> https://issues.apache.org/jira/browse/HIVE-20241
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-20241
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 
> 125ad19278b30a8b3a018e20805524c76f864e9a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 397cee2a5f5db8874b402377f36d9a33551327c3 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 
> a1f5133c8626cb5e2fb64ea29236bea48aacacea 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> fa923853785b078dd30f000bd803e1bf4b86bcde 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
> 6be48ca23beeadd03d822f211130176d41ca5164 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> b5adf1bd046bece1affce6bb0f962cdf5c4af03b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 
> 7a2a2c7a2841640adc6414087efa8ef5e9ab360d 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 
> 871844b30d17ac351da90c8a7ba03bcc6341b085 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java 
> 6af783302220de51c9c3933039a4122b28400d12 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 
> af2ece44dc9853607a8737f31284f9442f6c1d69 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 
> 71127c20b7e7d99394c5251ecf7a04a89e0abfc4 
>   ql/src/test/queries/clientpositive/partition_ctas.q PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/partition_ctas.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68086/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>

Re: Review Request 68086: HIVE-20241

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68086/
---

(Updated July 31, 2018, 2:17 a.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-20241
https://issues.apache.org/jira/browse/HIVE-20241


Repository: hive-git


Description
---

HIVE-20241


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties 
125ad19278b30a8b3a018e20805524c76f864e9a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
397cee2a5f5db8874b402377f36d9a33551327c3 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 
a1f5133c8626cb5e2fb64ea29236bea48aacacea 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
fa923853785b078dd30f000bd803e1bf4b86bcde 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
6be48ca23beeadd03d822f211130176d41ca5164 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
b5adf1bd046bece1affce6bb0f962cdf5c4af03b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 
7a2a2c7a2841640adc6414087efa8ef5e9ab360d 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 
871844b30d17ac351da90c8a7ba03bcc6341b085 
  ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java 
6af783302220de51c9c3933039a4122b28400d12 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 
af2ece44dc9853607a8737f31284f9442f6c1d69 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 
71127c20b7e7d99394c5251ecf7a04a89e0abfc4 
  ql/src/test/queries/clientpositive/partition_ctas.q PRE-CREATION 
  ql/src/test/results/clientpositive/llap/partition_ctas.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/68086/diff/3/

Changes: https://reviews.apache.org/r/68086/diff/2-3/


Testing
---


Thanks,

Jesús Camacho Rodríguez

[ANNOUNCE] Apache Hive 3.1.0 Released

[Vineet Garg]

The Apache Hive team is proud to announce the release of Apache Hive
version 3.1.0.

The Apache Hive (TM) data warehouse software facilitates querying and
managing large datasets residing in distributed storage. Built on top
of Apache Hadoop (TM), it provides, among others:

* Tools to enable easy data extract/transform/load (ETL)

* A mechanism to impose structure on a variety of data formats

* Access to files stored either directly in Apache HDFS (TM) or in other
  data storage systems such as Apache HBase (TM)

* Query execution via Apache Hadoop MapReduce, Apache Tez and Apache
Spark frameworks.

For Hive release details and downloads, please
visit:https://hive.apache.org/downloads.html

Hive 3.1.0 Release Notes are available here:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12343014=Text=12310843


We would like to thank the many contributors who made this release
possible.

Regards,

The Apache Hive Team

Re: Review Request 68086: HIVE-20241

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68086/#review206632
---




ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
Lines 219 (patched)


This token is unused. Can be deleted.



ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
Line 1990 (original), 1992 (patched)


Looks this rule is unused and can be deleted.


- Ashutosh Chauhan


On July 31, 2018, 12:57 a.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68086/
> ---
> 
> (Updated July 31, 2018, 12:57 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-20241
> https://issues.apache.org/jira/browse/HIVE-20241
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-20241
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 
> 125ad19278b30a8b3a018e20805524c76f864e9a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 397cee2a5f5db8874b402377f36d9a33551327c3 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 
> a1f5133c8626cb5e2fb64ea29236bea48aacacea 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> fa923853785b078dd30f000bd803e1bf4b86bcde 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
> 6be48ca23beeadd03d822f211130176d41ca5164 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> b5adf1bd046bece1affce6bb0f962cdf5c4af03b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 
> 7a2a2c7a2841640adc6414087efa8ef5e9ab360d 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 
> 871844b30d17ac351da90c8a7ba03bcc6341b085 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java 
> 6af783302220de51c9c3933039a4122b28400d12 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 
> af2ece44dc9853607a8737f31284f9442f6c1d69 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 
> 71127c20b7e7d99394c5251ecf7a04a89e0abfc4 
>   ql/src/test/queries/clientpositive/partition_ctas.q PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/partition_ctas.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68086/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>

Re: [ANNOUNCE] New PMC Member : Vihang Karajgaonkar

[Vihang Karajgaonkar]

Thanks Everyone!

On Mon, Jul 30, 2018 at 4:05 PM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Congratulations Vihang!
>
> Thanks
> Prasanth
>
>
>
> On Mon, Jul 30, 2018 at 12:34 PM -0700, "Xuefu Zhang"
> mailto:xu...@uber.com.INVALID>> wrote:
>
>
> Congratulations, Vihang!
>
> On Mon, Jul 30, 2018 at 10:53 AM, Andrew Sherman <
> asher...@cloudera.com.invalid> wrote:
>
> > Congratulations Vihang!
> >
> >
> >
> > On Mon, Jul 30, 2018 at 12:44 AM Peter Vary
> > wrote:
> >
> > > Congratulations Vihang!
> > >
> > > > On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> > > >
> > > > Congratulations Vihang!
> > > >
> > > >> On Jul 26, 2018, at 11:27 AM, Ashutosh Chauhan
> > > wrote:
> > > >>
> > > >> On behalf of the Hive PMC I am delighted to announce Vihang
> > > Karajgaonkar
> > > >> is joining Hive PMC.
> > > >> Thanks Vihang for all your contributions till now. Looking forward
> to
> > > many
> > > >> more.
> > > >>
> > > >> Welcome, Vihang!
> > > >>
> > > >> Thanks,
> > > >> Ashutosh
> > > >
> > >
> > >
> >
>
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Vihang Karajgaonkar]

Congrats Slim!

On Mon, Jul 30, 2018 at 4:35 PM, Deepak Jaiswal 
wrote:

> Congrats Slim!
>
> On 7/30/18, 4:03 PM, "Prasanth Jayachandran" <
> pjayachand...@hortonworks.com> wrote:
>
> Congratulations Slim!
>
> > On Jul 30, 2018, at 4:00 PM, Sergey Shelukhin <
> ser...@hortonworks.com> wrote:
> >
> > Congrats!
> >
> > On 18/7/30, 12:53, "Gunther Hagleitner"  >
> > wrote:
> >
> >> Congratulations!
> >>
> >> Thanks,
> >> Gunther.
> >> 
> >> From: Xuefu Zhang 
> >> Sent: Monday, July 30, 2018 12:11 PM
> >> To: dev@hive.apache.org
> >> Subject: Re: [ANNOUNCE] New committer: Slim Bouguerra
> >>
> >> congratulations!!!
> >>
> >> On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
> >> jcamachorodrig...@hortonworks.com> wrote:
> >>
> >>> Congrats Slim!
> >>>
> >>> On 7/30/18, 10:53 AM, "Andrew Sherman"
> 
> >>> wrote:
> >>>
> >>>Congratulations Slim!
> >>>
> >>>On Mon, Jul 30, 2018 at 12:46 AM Peter Vary
> >>>  
> >>>wrote:
> >>>
>  Congratulations Slim!
> 
> > On Jul 30, 2018, at 02:00, Ashutosh Chauhan
> >>> 
>  wrote:
> >
> > Apache Hive's Project Management Committee (PMC) has invited
> >>> Slim
>  Bouguerra
> > to become a committer, and we are pleased to announce that he
> >>> has
>  accepted.
> >
> > Slim, welcome, thank you for your contributions, and we look
> >>> forward your
> > further interactions with the community!
> >
> > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> 
> 
> >>>
> >>>
> >>>
> >
>
>
>
>

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

[Vihang Karajgaonkar]

Congratulations Sahil!

On Mon, Jul 30, 2018 at 4:05 PM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Congratulations Sahil!
>
> Thanks
> Prasanth
>
>
>
> On Mon, Jul 30, 2018 at 12:33 PM -0700, "Xuefu Zhang"
> mailto:xu...@uber.com.INVALID>> wrote:
>
>
> Congrats, Sahil!
>
> On Mon, Jul 30, 2018 at 10:53 AM, Andrew Sherman <
> asher...@cloudera.com.invalid> wrote:
>
> > Congratulations Sahil!
> >
> > On Mon, Jul 30, 2018 at 7:29 AM Sahil Takiar
> > wrote:
> >
> > > Thanks!
> > >
> > > On Mon, Jul 30, 2018 at 2:44 AM, Peter Vary
> > > wrote:
> > >
> > > > Congratulations Sahil!
> > > >
> > > > > On Jul 29, 2018, at 22:32, Vineet Garg
> > wrote:
> > > > >
> > > > > Congratulations Sahil!
> > > > >
> > > > >> On Jul 26, 2018, at 11:28 AM, Ashutosh Chauhan <
> > hashut...@apache.org>
> > > > wrote:
> > > > >>
> > > > >> On behalf of the Hive PMC I am delighted to announce Sahil Takiar
> is
> > > > >> joining Hive PMC.
> > > > >> Thanks Sahil for all your contributions till now. Looking forward
> to
> > > > many
> > > > >> more.
> > > > >>
> > > > >> Welcome, Sahil!
> > > > >>
> > > > >> Thanks,
> > > > >> Ashutosh
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Sahil Takiar
> > > Software Engineer
> > > takiar.sa...@gmail.com | (510) 673-0309
> > >
> >
>
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Vihang Karajgaonkar]

Congrats Vineet!

On Mon, Jul 30, 2018 at 4:04 PM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Congratulations Vineet!
>
> Thanks
> Prasanth
>
>
>
> On Mon, Jul 30, 2018 at 12:11 PM -0700, "Xuefu Zhang"
> mailto:xu...@uber.com.INVALID>> wrote:
>
>
> Congratulations!
>
> On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
> jcamachorodrig...@hortonworks.com> wrote:
>
> > Congrats Vineet!
> >
> > On 7/30/18, 10:53 AM, "Andrew Sherman"
> > wrote:
> >
> > Congratulations Vineet!
> >
> > On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal <
> > djais...@hortonworks.com>
> > wrote:
> >
> > > Congratulations Vineet!
> > >
> > > On 7/30/18, 12:45 AM, "Peter Vary"
> > wrote:
> > >
> > > Congratulations Vineet!
> > >
> > > > On Jul 30, 2018, at 01:59, Ashutosh Chauhan <
> > hashut...@apache.org>
> > > wrote:
> > > >
> > > > On behalf of the Hive PMC I am delighted to announce Vineet
> > Garg is
> > > joining
> > > > Hive PMC.
> > > > Thanks Vineet for all your contributions till now. Looking
> > forward
> > > to many
> > > > more.
> > > >
> > > > Welcome, Vineet!
> > > >
> > > > Thanks,
> > > > Ashutosh
> > >
> > >
> > >
> > >
> >
> >
> >
>
>

Re: [ANNOUNCE] New PMC Member : Peter Vary

[Vihang Karajgaonkar]

Congrats Peter!

On Mon, Jul 30, 2018 at 4:04 PM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:

> Congratulations Peter!
>
> Thanks
> Prasanth
>
>
>
> On Mon, Jul 30, 2018 at 12:21 PM -0700, "Xuefu Zhang"
> mailto:xu...@uber.com.INVALID>> wrote:
>
>
> Congratulations, Peter!
>
> On Mon, Jul 30, 2018 at 12:11 PM, Jesus Camacho Rodriguez <
> jcamachorodrig...@hortonworks.com> wrote:
>
> > Congrats Peter!
> >
> > On 7/30/18, 10:53 AM, "Andrew Sherman"
> > wrote:
> >
> > Congratulations Peter!
> >
> > On Sun, Jul 29, 2018 at 1:32 PM Vineet Garg
> > wrote:
> >
> > > Congratulations Peter!
> > >
> > > > On Jul 26, 2018, at 11:25 AM, Ashutosh Chauhan <
> > hashut...@apache.org>
> > > wrote:
> > > >
> > > > On behalf of the Hive PMC I am delighted to announce Peter Vary
> is
> > > joining
> > > > Hive PMC.
> > > > Thanks Peter for all your contributions till now. Looking forward
> > to many
> > > > more.
> > > >
> > > > Welcome, Peter!
> > > >
> > > > Thanks,
> > > > Ashutosh
> > >
> > >
> >
> >
> >
>
>

Re: Review Request 68086: HIVE-20241

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68086/
---

(Updated July 31, 2018, 12:57 a.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-20241
https://issues.apache.org/jira/browse/HIVE-20241


Repository: hive-git


Description
---

HIVE-20241


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties 
125ad19278b30a8b3a018e20805524c76f864e9a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
397cee2a5f5db8874b402377f36d9a33551327c3 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 
a1f5133c8626cb5e2fb64ea29236bea48aacacea 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
fa923853785b078dd30f000bd803e1bf4b86bcde 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
6be48ca23beeadd03d822f211130176d41ca5164 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
b5adf1bd046bece1affce6bb0f962cdf5c4af03b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 
7a2a2c7a2841640adc6414087efa8ef5e9ab360d 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 
871844b30d17ac351da90c8a7ba03bcc6341b085 
  ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java 
6af783302220de51c9c3933039a4122b28400d12 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 
af2ece44dc9853607a8737f31284f9442f6c1d69 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 
71127c20b7e7d99394c5251ecf7a04a89e0abfc4 
  ql/src/test/queries/clientpositive/partition_ctas.q PRE-CREATION 
  ql/src/test/results/clientpositive/llap/partition_ctas.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/68086/diff/2/

Changes: https://reviews.apache.org/r/68086/diff/1-2/


Testing
---


Thanks,

Jesús Camacho Rodríguez

Re: Review Request 68115: HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List object inspectors

[j . prasanth . j]

> On July 31, 2018, 12:17 a.m., Gopal V wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
> > Lines 76 (patched)
> > 
> >
> > The type -> ObjectInspector is the only one which I think looks like it 
> > is there for "intern-ing" of object inpsectors (So that object == can be 
> > used instead of a deep equals check).
> > 
> > Of course, I'm not sure that's a "real" problem unless a query takes > 
> > 5 minutes to compile.
> > 
> > If that doesn't show up in the leaks, it might better to leave that one 
> > alone.

but, unbounded cache adversely affect the query compile if HS2 goes in GC 
pause? It is still softvalues but that doesn't get cleaned up aggressively 
(until full GC). Being bounded also gives predictable perf (doesn't reach the 
point of GC). right?


- Prasanth_J


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68115/#review206627
---


On July 31, 2018, 12:08 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68115/
> ---
> 
> (Updated July 31, 2018, 12:08 a.m.)
> 
> 
> Review request for hive, Gopal V and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-20274
> https://issues.apache.org/jira/browse/HIVE-20274
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List 
> object inspectors
> 
> 
> Diffs
> -
> 
>   
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
>  b45a9dee72f5fb3a139ee205a05e23c068b1fa0c 
>   
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java
>  3a88a400250afa0e29001ec118e006f937678089 
>   
> serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java
>  cb1440e98b8fad399c4c02a7440d2a30483455f4 
> 
> 
> Diff: https://reviews.apache.org/r/68115/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 68115: HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List object inspectors

[Gopal V]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68115/#review206627
---




serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
Lines 76 (patched)


The type -> ObjectInspector is the only one which I think looks like it is 
there for "intern-ing" of object inpsectors (So that object == can be used 
instead of a deep equals check).

Of course, I'm not sure that's a "real" problem unless a query takes > 5 
minutes to compile.

If that doesn't show up in the leaks, it might better to leave that one 
alone.


- Gopal V


On July 31, 2018, 12:08 a.m., Prasanth_J wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68115/
> ---
> 
> (Updated July 31, 2018, 12:08 a.m.)
> 
> 
> Review request for hive, Gopal V and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-20274
> https://issues.apache.org/jira/browse/HIVE-20274
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List 
> object inspectors
> 
> 
> Diffs
> -
> 
>   
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
>  b45a9dee72f5fb3a139ee205a05e23c068b1fa0c 
>   
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java
>  3a88a400250afa0e29001ec118e006f937678089 
>   
> serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java
>  cb1440e98b8fad399c4c02a7440d2a30483455f4 
> 
> 
> Diff: https://reviews.apache.org/r/68115/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Prasanth_J
> 
>

Re: Review Request 68115: HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List object inspectors

[j . prasanth . j]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68115/
---

(Updated July 31, 2018, 12:08 a.m.)


Review request for hive, Gopal V and Sergey Shelukhin.


Bugs: HIVE-20274
https://issues.apache.org/jira/browse/HIVE-20274


Repository: hive-git


Description
---

HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List object 
inspectors


Diffs
-

  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
 b45a9dee72f5fb3a139ee205a05e23c068b1fa0c 
  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java
 3a88a400250afa0e29001ec118e006f937678089 
  
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java
 cb1440e98b8fad399c4c02a7440d2a30483455f4 


Diff: https://reviews.apache.org/r/68115/diff/1/


Testing
---


Thanks,

Prasanth_J

Review Request 68115: HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List object inspectors

[j . prasanth . j]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68115/
---

Review request for hive, Gopal V and Sergey Shelukhin.


Repository: hive-git


Description
---

HIVE-20274: HiveServer2 ObjectInspectorFactory leaks for Struct and List object 
inspectors


Diffs
-

  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
 b45a9dee72f5fb3a139ee205a05e23c068b1fa0c 
  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ReflectionStructObjectInspector.java
 3a88a400250afa0e29001ec118e006f937678089 
  
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestReflectionObjectInspectors.java
 cb1440e98b8fad399c4c02a7440d2a30483455f4 


Diff: https://reviews.apache.org/r/68115/diff/1/


Testing
---


Thanks,

Prasanth_J

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Deepak Jaiswal]

Congrats Slim!

On 7/30/18, 4:03 PM, "Prasanth Jayachandran"  
wrote:

Congratulations Slim!

> On Jul 30, 2018, at 4:00 PM, Sergey Shelukhin  
wrote:
> 
> Congrats!
> 
> On 18/7/30, 12:53, "Gunther Hagleitner" 
> wrote:
> 
>> Congratulations!
>> 
>> Thanks,
>> Gunther.
>> 
>> From: Xuefu Zhang 
>> Sent: Monday, July 30, 2018 12:11 PM
>> To: dev@hive.apache.org
>> Subject: Re: [ANNOUNCE] New committer: Slim Bouguerra
>> 
>> congratulations!!!
>> 
>> On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
>> jcamachorodrig...@hortonworks.com> wrote:
>> 
>>> Congrats Slim!
>>> 
>>> On 7/30/18, 10:53 AM, "Andrew Sherman" 
>>> wrote:
>>> 
>>>Congratulations Slim!
>>> 
>>>On Mon, Jul 30, 2018 at 12:46 AM Peter Vary
>>> >>> 
>>>wrote:
>>> 
 Congratulations Slim!
 
> On Jul 30, 2018, at 02:00, Ashutosh Chauhan
>>> 
 wrote:
> 
> Apache Hive's Project Management Committee (PMC) has invited
>>> Slim
 Bouguerra
> to become a committer, and we are pleased to announce that he
>>> has
 accepted.
> 
> Slim, welcome, thank you for your contributions, and we look
>>> forward your
> further interactions with the community!
> 
> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
 
 
>>> 
>>> 
>>> 
> 

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

[Prasanth Jayachandran]

Congratulations Sahil!

Thanks
Prasanth



On Mon, Jul 30, 2018 at 12:33 PM -0700, "Xuefu Zhang" 
mailto:xu...@uber.com.INVALID>> wrote:


Congrats, Sahil!

On Mon, Jul 30, 2018 at 10:53 AM, Andrew Sherman <
asher...@cloudera.com.invalid> wrote:

> Congratulations Sahil!
>
> On Mon, Jul 30, 2018 at 7:29 AM Sahil Takiar
> wrote:
>
> > Thanks!
> >
> > On Mon, Jul 30, 2018 at 2:44 AM, Peter Vary
> > wrote:
> >
> > > Congratulations Sahil!
> > >
> > > > On Jul 29, 2018, at 22:32, Vineet Garg
> wrote:
> > > >
> > > > Congratulations Sahil!
> > > >
> > > >> On Jul 26, 2018, at 11:28 AM, Ashutosh Chauhan <
> hashut...@apache.org>
> > > wrote:
> > > >>
> > > >> On behalf of the Hive PMC I am delighted to announce Sahil Takiar is
> > > >> joining Hive PMC.
> > > >> Thanks Sahil for all your contributions till now. Looking forward to
> > > many
> > > >> more.
> > > >>
> > > >> Welcome, Sahil!
> > > >>
> > > >> Thanks,
> > > >> Ashutosh
> > > >
> > >
> > >
> >
> >
> > --
> > Sahil Takiar
> > Software Engineer
> > takiar.sa...@gmail.com | (510) 673-0309
> >
>

Re: [ANNOUNCE] New PMC Member : Peter Vary

[Prasanth Jayachandran]

Congratulations Peter!

Thanks
Prasanth



On Mon, Jul 30, 2018 at 12:21 PM -0700, "Xuefu Zhang" 
mailto:xu...@uber.com.INVALID>> wrote:


Congratulations, Peter!

On Mon, Jul 30, 2018 at 12:11 PM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> Congrats Peter!
>
> On 7/30/18, 10:53 AM, "Andrew Sherman"
> wrote:
>
> Congratulations Peter!
>
> On Sun, Jul 29, 2018 at 1:32 PM Vineet Garg
> wrote:
>
> > Congratulations Peter!
> >
> > > On Jul 26, 2018, at 11:25 AM, Ashutosh Chauhan <
> hashut...@apache.org>
> > wrote:
> > >
> > > On behalf of the Hive PMC I am delighted to announce Peter Vary is
> > joining
> > > Hive PMC.
> > > Thanks Peter for all your contributions till now. Looking forward
> to many
> > > more.
> > >
> > > Welcome, Peter!
> > >
> > > Thanks,
> > > Ashutosh
> >
> >
>
>
>

Re: [ANNOUNCE] New PMC Member : Vihang Karajgaonkar

[Prasanth Jayachandran]

Congratulations Vihang!

Thanks
Prasanth



On Mon, Jul 30, 2018 at 12:34 PM -0700, "Xuefu Zhang" 
mailto:xu...@uber.com.INVALID>> wrote:


Congratulations, Vihang!

On Mon, Jul 30, 2018 at 10:53 AM, Andrew Sherman <
asher...@cloudera.com.invalid> wrote:

> Congratulations Vihang!
>
>
>
> On Mon, Jul 30, 2018 at 12:44 AM Peter Vary
> wrote:
>
> > Congratulations Vihang!
> >
> > > On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> > >
> > > Congratulations Vihang!
> > >
> > >> On Jul 26, 2018, at 11:27 AM, Ashutosh Chauhan
> > wrote:
> > >>
> > >> On behalf of the Hive PMC I am delighted to announce Vihang
> > Karajgaonkar
> > >> is joining Hive PMC.
> > >> Thanks Vihang for all your contributions till now. Looking forward to
> > many
> > >> more.
> > >>
> > >> Welcome, Vihang!
> > >>
> > >> Thanks,
> > >> Ashutosh
> > >
> >
> >
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Prasanth Jayachandran]

Congratulations Vineet!

Thanks
Prasanth



On Mon, Jul 30, 2018 at 12:11 PM -0700, "Xuefu Zhang" 
mailto:xu...@uber.com.INVALID>> wrote:


Congratulations!

On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> Congrats Vineet!
>
> On 7/30/18, 10:53 AM, "Andrew Sherman"
> wrote:
>
> Congratulations Vineet!
>
> On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal <
> djais...@hortonworks.com>
> wrote:
>
> > Congratulations Vineet!
> >
> > On 7/30/18, 12:45 AM, "Peter Vary"
> wrote:
> >
> > Congratulations Vineet!
> >
> > > On Jul 30, 2018, at 01:59, Ashutosh Chauhan <
> hashut...@apache.org>
> > wrote:
> > >
> > > On behalf of the Hive PMC I am delighted to announce Vineet
> Garg is
> > joining
> > > Hive PMC.
> > > Thanks Vineet for all your contributions till now. Looking
> forward
> > to many
> > > more.
> > >
> > > Welcome, Vineet!
> > >
> > > Thanks,
> > > Ashutosh
> >
> >
> >
> >
>
>
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Prasanth Jayachandran]

Congratulations Slim!

> On Jul 30, 2018, at 4:00 PM, Sergey Shelukhin  wrote:
> 
> Congrats!
> 
> On 18/7/30, 12:53, "Gunther Hagleitner" 
> wrote:
> 
>> Congratulations!
>> 
>> Thanks,
>> Gunther.
>> 
>> From: Xuefu Zhang 
>> Sent: Monday, July 30, 2018 12:11 PM
>> To: dev@hive.apache.org
>> Subject: Re: [ANNOUNCE] New committer: Slim Bouguerra
>> 
>> congratulations!!!
>> 
>> On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
>> jcamachorodrig...@hortonworks.com> wrote:
>> 
>>> Congrats Slim!
>>> 
>>> On 7/30/18, 10:53 AM, "Andrew Sherman" 
>>> wrote:
>>> 
>>>Congratulations Slim!
>>> 
>>>On Mon, Jul 30, 2018 at 12:46 AM Peter Vary
>>> >>> 
>>>wrote:
>>> 
 Congratulations Slim!
 
> On Jul 30, 2018, at 02:00, Ashutosh Chauhan
>>> 
 wrote:
> 
> Apache Hive's Project Management Committee (PMC) has invited
>>> Slim
 Bouguerra
> to become a committer, and we are pleased to announce that he
>>> has
 accepted.
> 
> Slim, welcome, thank you for your contributions, and we look
>>> forward your
> further interactions with the community!
> 
> Ashutosh Chauhan (on behalf of the Apache Hive PMC)
 
 
>>> 
>>> 
>>> 
> 

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Sergey Shelukhin]

Congrats!

On 18/7/30, 12:53, "Gunther Hagleitner" 
wrote:

>Congratulations!
>
>Thanks,
>Gunther.
>
>From: Xuefu Zhang 
>Sent: Monday, July 30, 2018 12:11 PM
>To: dev@hive.apache.org
>Subject: Re: [ANNOUNCE] New committer: Slim Bouguerra
>
>congratulations!!!
>
>On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
>jcamachorodrig...@hortonworks.com> wrote:
>
>> Congrats Slim!
>>
>> On 7/30/18, 10:53 AM, "Andrew Sherman" 
>> wrote:
>>
>> Congratulations Slim!
>>
>> On Mon, Jul 30, 2018 at 12:46 AM Peter Vary
>>> >
>> wrote:
>>
>> > Congratulations Slim!
>> >
>> > > On Jul 30, 2018, at 02:00, Ashutosh Chauhan
>>
>> > wrote:
>> > >
>> > > Apache Hive's Project Management Committee (PMC) has invited
>>Slim
>> > Bouguerra
>> > > to become a committer, and we are pleased to announce that he
>>has
>> > accepted.
>> > >
>> > > Slim, welcome, thank you for your contributions, and we look
>> forward your
>> > > further interactions with the community!
>> > >
>> > > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
>> >
>> >
>>
>>
>>

[jira] [Created] (HIVE-20274) HiveServer2 ObjectInspectorFactory leaks for Struct and List object inspectors

[Gopal V (JIRA)]

Gopal V created HIVE-20274:
--

 Summary: HiveServer2 ObjectInspectorFactory leaks for Struct and 
List object inspectors
 Key: HIVE-20274
 URL: https://issues.apache.org/jira/browse/HIVE-20274
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 3.0.0, 4.0.0
Reporter: Gopal V


Fix in HIVE-19860 needs to be applied to 

{code}
  static ConcurrentHashMap
  cachedStandardListObjectInspector = new 
ConcurrentHashMap();
...
  static ConcurrentHashMap>, StandardStructObjectInspector> 
cachedStandardStructObjectInspector =
  new ConcurrentHashMap>, 
StandardStructObjectInspector>();
...
  static ConcurrentHashMap, ColumnarStructObjectInspector> 
cachedColumnarStructObjectInspector =
  new ConcurrentHashMap, ColumnarStructObjectInspector>();
{code}

And possibly for 

{code}
  static ConcurrentHashMap objectInspectorCache = new 
ConcurrentHashMap();
...
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68086: HIVE-20241

[Jesús Camacho Rodríguez]

> On July 30, 2018, 10:06 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
> > Lines 2002 (patched)
> > 
> >
> > I don't think we want to allow type in partitioned by clause. Only 
> > _createTablePartitionColumnSpec_ should be allowed.

The problem is that this rule matches normal CREATE TABLE statement as well as 
CTAS. In SemanticAnalyzer, I check whether the right type of partitionSpec has 
been specified and fail otherwise.


> On July 30, 2018, 10:06 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
> > Lines 7620 (patched)
> > 
> >
> > Will this guanatee atomicity for Acid and MM tables? Specifically case 
> > where table gets created but some or all of partitions didnt get added in 
> > metastore. Add TODO comment if this will be handled later.

The commitTxn is triggered by the Driver when tasks have been completed, thus I 
think current model guarantees atomiticy for data for any kind of transactional 
table.

However, if tasks fail, we may end up with an empty table and dangling 
partitions in the metastore. I believe this problem probably exists nowadays 
even for INSERT statements?


> On July 30, 2018, 10:06 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/queries/clientnegative/partition_ctas_wrong_order.q
> > Lines 3 (patched)
> > 
> >
> > Is there a reason to disallow partition column other than last one, 
> > since columns are referred explictly by name, I imagine user would expect 
> > this to work. Especially since we have recently allowed this to work for 
> > insert statements as well.

Yes, it would be good to support this, I will work on it. Difficulty was that 
for CTAS we infer the table schema from the query schema, so I will need to 
change that logic.


- Jesús


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68086/#review206615
---


On July 27, 2018, 6:12 p.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68086/
> ---
> 
> (Updated July 27, 2018, 6:12 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-20241
> https://issues.apache.org/jira/browse/HIVE-20241
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-20241
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 
> 50bfe6a06a77814decb6d48a19c56e8a41e34432 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 397cee2a5f5db8874b402377f36d9a33551327c3 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 
> a1f5133c8626cb5e2fb64ea29236bea48aacacea 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> fa923853785b078dd30f000bd803e1bf4b86bcde 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
> 6be48ca23beeadd03d822f211130176d41ca5164 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> b5adf1bd046bece1affce6bb0f962cdf5c4af03b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 
> 7a2a2c7a2841640adc6414087efa8ef5e9ab360d 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 
> 871844b30d17ac351da90c8a7ba03bcc6341b085 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java 
> 6af783302220de51c9c3933039a4122b28400d12 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 
> af2ece44dc9853607a8737f31284f9442f6c1d69 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 
> 71127c20b7e7d99394c5251ecf7a04a89e0abfc4 
>   ql/src/test/queries/clientnegative/partition_ctas_wrong_order.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/partition_ctas.q PRE-CREATION 
>   ql/src/test/results/clientnegative/partition_ctas_wrong_order.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/partition_ctas.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68086/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>

Re: Review Request 68099: SerDe to support Teradata Binary Format

[Lu Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68099/
---

(Updated July 30, 2018, 10:22 p.m.)


Review request for hive and Carl Steinbach.


Changes
---

fix the issues found by Hive QA


Bugs: HIVE-20225
https://issues.apache.org/jira/browse/HIVE-20225


Repository: hive-git


Description
---

When using TPT/BTEQ to export Data from Teradata, Teradata will export binary 
files based on the schema.

A Customized SerDe is needed in order to directly read these files from Hive.

CREATE EXTERNAL TABLE `TABLE1`(
...)
PARTITIONED BY (
...)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.contrib.serde2.TeradataBinarySerde'
STORED AS INPUTFORMAT
 
'org.apache.hadoop.hive.contrib.fileformat.teradata.TeradataBinaryFileInputFormat'
OUTPUTFORMAT
 
'org.apache.hadoop.hive.contrib.fileformat.teradata.TeradataBinaryFileOutputFormat'
LOCATION ...;

SELECT * FROM `TABLE1`;
Problem Statement:

Right now the fast way to export data from Teradata is using TPT. However, the 
Hive could not directly utilize these exported binary format because it doesn't 
have a SerDe for these files.

Result:

Provided with the SerDe, Hive can operate upon the exported Teradata Binary 
Format file transparently.


Diffs (updated)
-

  
contrib/src/java/org/apache/hadoop/hive/contrib/fileformat/teradata/TeradataBinaryFileInputFormat.java
 PRE-CREATION 
  
contrib/src/java/org/apache/hadoop/hive/contrib/fileformat/teradata/TeradataBinaryFileOutputFormat.java
 PRE-CREATION 
  
contrib/src/java/org/apache/hadoop/hive/contrib/fileformat/teradata/TeradataBinaryRecordReader.java
 PRE-CREATION 
  
contrib/src/java/org/apache/hadoop/hive/contrib/serde2/teradata/TeradataBinaryDataInputStream.java
 PRE-CREATION 
  
contrib/src/java/org/apache/hadoop/hive/contrib/serde2/teradata/TeradataBinaryDataOutputStream.java
 PRE-CREATION 
  
contrib/src/java/org/apache/hadoop/hive/contrib/serde2/teradata/TeradataBinarySerde.java
 PRE-CREATION 
  
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestGeneralFunctions.java
 PRE-CREATION 
  
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeForDate.java
 PRE-CREATION 
  
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeForDecimal.java
 PRE-CREATION 
  
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeForTimeStamp.java
 PRE-CREATION 
  
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeGeneral.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/68099/diff/2/

Changes: https://reviews.apache.org/r/68099/diff/1-2/


Testing
---

Junit tests have been added for Serialization and Deserialization functions


Thanks,

Lu Li

Re: Review Request 68086: HIVE-20241

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68086/#review206615
---




ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
Lines 2002 (patched)


I don't think we want to allow type in partitioned by clause. Only 
_createTablePartitionColumnSpec_ should be allowed.



ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 7620 (patched)


Will this guanatee atomicity for Acid and MM tables? Specifically case 
where table gets created but some or all of partitions didnt get added in 
metastore. Add TODO comment if this will be handled later.



ql/src/test/queries/clientnegative/partition_ctas_wrong_order.q
Lines 3 (patched)


Is there a reason to disallow partition column other than last one, since 
columns are referred explictly by name, I imagine user would expect this to 
work. Especially since we have recently allowed this to work for insert 
statements as well.


- Ashutosh Chauhan


On July 27, 2018, 6:12 p.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68086/
> ---
> 
> (Updated July 27, 2018, 6:12 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-20241
> https://issues.apache.org/jira/browse/HIVE-20241
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-20241
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 
> 50bfe6a06a77814decb6d48a19c56e8a41e34432 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> 397cee2a5f5db8874b402377f36d9a33551327c3 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 
> a1f5133c8626cb5e2fb64ea29236bea48aacacea 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> fa923853785b078dd30f000bd803e1bf4b86bcde 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
> 6be48ca23beeadd03d822f211130176d41ca5164 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> b5adf1bd046bece1affce6bb0f962cdf5c4af03b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 
> 7a2a2c7a2841640adc6414087efa8ef5e9ab360d 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 
> 871844b30d17ac351da90c8a7ba03bcc6341b085 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java 
> 6af783302220de51c9c3933039a4122b28400d12 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 
> af2ece44dc9853607a8737f31284f9442f6c1d69 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 
> 71127c20b7e7d99394c5251ecf7a04a89e0abfc4 
>   ql/src/test/queries/clientnegative/partition_ctas_wrong_order.q 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/partition_ctas.q PRE-CREATION 
>   ql/src/test/results/clientnegative/partition_ctas_wrong_order.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/partition_ctas.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68086/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Gunther Hagleitner]

Congratulations!

Thanks,
Gunther.

From: Xuefu Zhang 
Sent: Monday, July 30, 2018 12:11 PM
To: dev@hive.apache.org
Subject: Re: [ANNOUNCE] New committer: Slim Bouguerra

congratulations!!!

On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> Congrats Slim!
>
> On 7/30/18, 10:53 AM, "Andrew Sherman" 
> wrote:
>
> Congratulations Slim!
>
> On Mon, Jul 30, 2018 at 12:46 AM Peter Vary  >
> wrote:
>
> > Congratulations Slim!
> >
> > > On Jul 30, 2018, at 02:00, Ashutosh Chauhan 
> > wrote:
> > >
> > > Apache Hive's Project Management Committee (PMC) has invited Slim
> > Bouguerra
> > > to become a committer, and we are pleased to announce that he has
> > accepted.
> > >
> > > Slim, welcome, thank you for your contributions, and we look
> forward your
> > > further interactions with the community!
> > >
> > > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> >
> >
>
>
>

Re: [ANNOUNCE] New PMC Member : Vihang Karajgaonkar

[Xuefu Zhang]

Congratulations, Vihang!

On Mon, Jul 30, 2018 at 10:53 AM, Andrew Sherman <
asher...@cloudera.com.invalid> wrote:

> Congratulations Vihang!
>
>
>
> On Mon, Jul 30, 2018 at 12:44 AM Peter Vary 
> wrote:
>
> > Congratulations Vihang!
> >
> > > On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> > >
> > > Congratulations Vihang!
> > >
> > >> On Jul 26, 2018, at 11:27 AM, Ashutosh Chauhan 
> > wrote:
> > >>
> > >> On behalf of the Hive PMC I am delighted to announce Vihang
> > Karajgaonkar
> > >> is joining Hive PMC.
> > >> Thanks Vihang for all your contributions till now. Looking forward to
> > many
> > >> more.
> > >>
> > >> Welcome, Vihang!
> > >>
> > >> Thanks,
> > >> Ashutosh
> > >
> >
> >
>

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

[Xuefu Zhang]

Congrats, Sahil!

On Mon, Jul 30, 2018 at 10:53 AM, Andrew Sherman <
asher...@cloudera.com.invalid> wrote:

> Congratulations Sahil!
>
> On Mon, Jul 30, 2018 at 7:29 AM Sahil Takiar 
> wrote:
>
> > Thanks!
> >
> > On Mon, Jul 30, 2018 at 2:44 AM, Peter Vary 
> > wrote:
> >
> > > Congratulations Sahil!
> > >
> > > > On Jul 29, 2018, at 22:32, Vineet Garg 
> wrote:
> > > >
> > > > Congratulations Sahil!
> > > >
> > > >> On Jul 26, 2018, at 11:28 AM, Ashutosh Chauhan <
> hashut...@apache.org>
> > > wrote:
> > > >>
> > > >> On behalf of the Hive PMC I am delighted to announce Sahil Takiar is
> > > >> joining Hive PMC.
> > > >> Thanks Sahil for all your contributions till now. Looking forward to
> > > many
> > > >> more.
> > > >>
> > > >> Welcome, Sahil!
> > > >>
> > > >> Thanks,
> > > >> Ashutosh
> > > >
> > >
> > >
> >
> >
> > --
> > Sahil Takiar
> > Software Engineer
> > takiar.sa...@gmail.com | (510) 673-0309
> >
>

Re: [ANNOUNCE] New PMC Member : Peter Vary

[Xuefu Zhang]

Congratulations, Peter!

On Mon, Jul 30, 2018 at 12:11 PM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> Congrats Peter!
>
> On 7/30/18, 10:53 AM, "Andrew Sherman" 
> wrote:
>
> Congratulations Peter!
>
> On Sun, Jul 29, 2018 at 1:32 PM Vineet Garg 
> wrote:
>
> > Congratulations Peter!
> >
> > > On Jul 26, 2018, at 11:25 AM, Ashutosh Chauhan <
> hashut...@apache.org>
> > wrote:
> > >
> > > On behalf of the Hive PMC I am delighted to announce Peter Vary is
> > joining
> > > Hive PMC.
> > > Thanks Peter for all your contributions till now. Looking forward
> to many
> > > more.
> > >
> > > Welcome, Peter!
> > >
> > > Thanks,
> > > Ashutosh
> >
> >
>
>
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Xuefu Zhang]

congratulations!!!

On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> Congrats Slim!
>
> On 7/30/18, 10:53 AM, "Andrew Sherman" 
> wrote:
>
> Congratulations Slim!
>
> On Mon, Jul 30, 2018 at 12:46 AM Peter Vary  >
> wrote:
>
> > Congratulations Slim!
> >
> > > On Jul 30, 2018, at 02:00, Ashutosh Chauhan 
> > wrote:
> > >
> > > Apache Hive's Project Management Committee (PMC) has invited Slim
> > Bouguerra
> > > to become a committer, and we are pleased to announce that he has
> > accepted.
> > >
> > > Slim, welcome, thank you for your contributions, and we look
> forward your
> > > further interactions with the community!
> > >
> > > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
> >
> >
>
>
>

Re: [ANNOUNCE] New PMC Member : Peter Vary

[Jesus Camacho Rodriguez]

Congrats Peter!

On 7/30/18, 10:53 AM, "Andrew Sherman"  wrote:

Congratulations Peter!

On Sun, Jul 29, 2018 at 1:32 PM Vineet Garg  wrote:

> Congratulations Peter!
>
> > On Jul 26, 2018, at 11:25 AM, Ashutosh Chauhan 
> wrote:
> >
> > On behalf of the Hive PMC I am delighted to announce Peter Vary is
> joining
> > Hive PMC.
> > Thanks Peter for all your contributions till now. Looking forward to 
many
> > more.
> >
> > Welcome, Peter!
> >
> > Thanks,
> > Ashutosh
>
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Xuefu Zhang]

Congratulations!

On Mon, Jul 30, 2018 at 12:10 PM, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> Congrats Vineet!
>
> On 7/30/18, 10:53 AM, "Andrew Sherman" 
> wrote:
>
> Congratulations Vineet!
>
> On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal <
> djais...@hortonworks.com>
> wrote:
>
> > Congratulations Vineet!
> >
> > On 7/30/18, 12:45 AM, "Peter Vary" 
> wrote:
> >
> > Congratulations Vineet!
> >
> > > On Jul 30, 2018, at 01:59, Ashutosh Chauhan <
> hashut...@apache.org>
> > wrote:
> > >
> > > On behalf of the Hive PMC I am delighted to announce Vineet
> Garg is
> > joining
> > > Hive PMC.
> > > Thanks Vineet for all your contributions till now. Looking
> forward
> > to many
> > > more.
> > >
> > > Welcome, Vineet!
> > >
> > > Thanks,
> > > Ashutosh
> >
> >
> >
> >
>
>
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Jesus Camacho Rodriguez]

Congrats Vineet!

On 7/30/18, 10:53 AM, "Andrew Sherman"  wrote:

Congratulations Vineet!

On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal 
wrote:

> Congratulations Vineet!
>
> On 7/30/18, 12:45 AM, "Peter Vary"  wrote:
>
> Congratulations Vineet!
>
> > On Jul 30, 2018, at 01:59, Ashutosh Chauhan 
> wrote:
> >
> > On behalf of the Hive PMC I am delighted to announce Vineet Garg is
> joining
> > Hive PMC.
> > Thanks Vineet for all your contributions till now. Looking forward
> to many
> > more.
> >
> > Welcome, Vineet!
> >
> > Thanks,
> > Ashutosh
>
>
>
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Jesus Camacho Rodriguez]

Congrats Slim!

On 7/30/18, 10:53 AM, "Andrew Sherman"  wrote:

Congratulations Slim!

On Mon, Jul 30, 2018 at 12:46 AM Peter Vary 
wrote:

> Congratulations Slim!
>
> > On Jul 30, 2018, at 02:00, Ashutosh Chauhan 
> wrote:
> >
> > Apache Hive's Project Management Committee (PMC) has invited Slim
> Bouguerra
> > to become a committer, and we are pleased to announce that he has
> accepted.
> >
> > Slim, welcome, thank you for your contributions, and we look forward 
your
> > further interactions with the community!
> >
> > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
>
>

Re: Review Request 68109: HIVE-20260 NDV of a column shouldn't be scaled when row count is changed by filter on another column

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68109/#review206607
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
Line 354 (original), 355 (patched)


add a comment here:
We assume columns are uncorrelated. That is filters on different columns 
will result in filtering out different rows. So, we scale down the ndv of a 
column only when row count is decreased by its own filter. Under correlated 
assumption, we would have scaled down ndv for every column for every filter 
condition. We dont do that. 
This makes our estimate more conservative than need to be which is good 
since this will result in overestimates when we are wrong but avoids OOM had we 
chosen the other assumption. In future, we need to capture correlatedness of 
columns in metadata so that we can account for that.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
Line 2633 (original), 2647 (patched)


Add assert newNDV <= newNumRows.


- Ashutosh Chauhan


On July 30, 2018, 4:17 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68109/
> ---
> 
> (Updated July 30, 2018, 4:17 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-20260
> https://issues.apache.org/jira/browse/HIVE-20260
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> * keep track of used column; and only rescale affected columns
> * much more conservative than old logic - possible too much...
> * wip patch
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/AnnotateStatsProcCtx.java
>  47ee949fbcfa9391c640719a57fab39279c009db 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
>  3c2b0854269d5426153958096a8b5b5ad3612c0f 
>   ql/src/test/queries/clientpositive/stat_estimate_drill.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/stat_estimate_related_col.q 
> 52da2f759a009daa372a53446e2f0fd4a88152be 
>   ql/src/test/results/clientpositive/stat_estimate_drill.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/stat_estimate_related_col.q.out 
> 669adafda3a45f7846face3d99817cd1b9cb3664 
> 
> 
> Diff: https://reviews.apache.org/r/68109/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zoltan Haindrich
> 
>

[jira] [Created] (HIVE-20273) Spark jobs aren't cancelled if getSparkJobInfo or getSparkStagesInfo

[Sahil Takiar (JIRA)]

Sahil Takiar created HIVE-20273:
---

 Summary: Spark jobs aren't cancelled if getSparkJobInfo or 
getSparkStagesInfo
 Key: HIVE-20273
 URL: https://issues.apache.org/jira/browse/HIVE-20273
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Sahil Takiar
Assignee: Sahil Takiar


HIVE-19053 and HIVE-19733 add handling of {{InterruptedException}} to 
{{#getSparkJobInfo}} and {{#getSparkStagesInfo}} in {{RemoteSparkJobStatus}}, 
but that means the {{InterruptedException}} is wrapped in a {{HiveException}} 
and then thrown. The {{HiveException}} is then cause in 
{{RemoteSparkJobMonitor}} and then wrapped in another Hive exception. The 
double nesting of hive exception causes the logic in 
{{SparkTask#setSparkException}} to break, and it doesn't kill the job if an 
interrupted exception is thrown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68108: HIVE-19097 related equals and in operators may cause inaccurate stats estimations

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68108/#review206603
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Line 2141 (original), 2141 (patched)


Yes..its ok to lower this since IN is more performant in runtime, we want 
to execute IN at runtime.



ql/src/test/results/clientpositive/cbo_simple_select.q.out
Line 866 (original), 866 (patched)


This didnt get rewritten into IN. Is that expected?



ql/src/test/results/clientpositive/druid_basic3.q.out
Line 280 (original), 280 (patched)


No folding of OR into IN ? for druid also, IN is more performant.



ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out
Line 197 (original)


yeah.. likely AST gen of IN token isn't present.



ql/src/test/results/clientpositive/llap/vectorized_case.q.out
Line 54 (original), 54 (patched)


yeah.. i think thats because now constants are of type integer. Note in OR 
clause they had S suffix which made them smallint.

This used to happen because of 
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java#L1157

This is during parsing of expresions. We need to enhance this logic now for 
INs as well.



ql/src/test/results/clientpositive/perf/tez/query15.q.out
Line 74 (original), 74 (patched)


No folding back to IN ?



ql/src/test/results/clientpositive/perf/tez/query45.q.out
Line 81 (original), 81 (patched)


No folding back to IN?



ql/src/test/results/clientpositive/perf/tez/query63.q.out
Line 135 (original), 135 (patched)


No folding back to IN ?



ql/src/test/results/clientpositive/perf/tez/query8.q.out
Line 337 (original), 337 (patched)


No folding back to IN ?



ql/src/test/results/clientpositive/pointlookup.q.out
Line 43 (original), 43 (patched)


These tests were written with default of 31 in mind. So, lets update the 
test with that value of config.



ql/src/test/results/clientpositive/pointlookup2.q.out
Line 115 (original), 111 (patched)


Lets set the config to 31 to retain original intention of tests.



ql/src/test/results/clientpositive/vector_non_constant_in_expr.q.out
Line 22 (original), 22 (patched)


these ORs didnt get folded in IN, expected?


- Ashutosh Chauhan


On July 30, 2018, 4:13 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68108/
> ---
> 
> (Updated July 30, 2018, 4:13 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Gopal V.
> 
> 
> Bugs: HIVE-19097
> https://issues.apache.org/jira/browse/HIVE-19097
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> * open in to or
> * close ors into in at 2
> * wip patch
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 39c77b3fe52cb7d7d255138bb71d77a170347b52 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
>  f544f586321c2496e9f3bc3b428ae5689bf046a9 
>   ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestCounterMapping.java 
> b57b5ddc2cba688897bac4cf29eaf9679b6de375 
>   ql/src/test/results/clientpositive/alter_partition_coltype.q.out 
> 5d033a3c0181d697f08ac1aeb62ef57071602050 
>   ql/src/test/results/clientpositive/annotate_stats_filter.q.out 
> 54395886d2a58d83b0fd853c0008fc99f7063f0c 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out 
> 29ef214ff0e8678fe25bd1793a9522ea0ae61d32 
>   ql/src/test/results/clientpositive/auto_join19.q.out 
> 3e07ec06de767099aeb14beea586bca4cca0784e 
>   ql/src/test/results/clientpositive/cbo_rp_simple_select.q.out 
> d12b5f64cc70e0039ce9179c5c6fb90d8beba0d1 
>   ql/src/test/results/clientpositive/cbo_simple_select.q.out 
> 588d924e6af9b704577b9acf9a01d139c1b13af7 
>   ql/src/test/results/clientpositive/druid_basic3.q.out 
> 54719f751769722c6b099682b9742b1e21c6ebec 
>   ql/src/test/results/clientpositive/druid_intervals.q.out 
> a5203c31822ef6cb8c70a0caa9b682b6f73e88f5 
>   ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out 
> 

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

[Andrew Sherman]

Congratulations Sahil!

On Mon, Jul 30, 2018 at 7:29 AM Sahil Takiar  wrote:

> Thanks!
>
> On Mon, Jul 30, 2018 at 2:44 AM, Peter Vary 
> wrote:
>
> > Congratulations Sahil!
> >
> > > On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> > >
> > > Congratulations Sahil!
> > >
> > >> On Jul 26, 2018, at 11:28 AM, Ashutosh Chauhan 
> > wrote:
> > >>
> > >> On behalf of the Hive PMC I am delighted to announce Sahil Takiar is
> > >> joining Hive PMC.
> > >> Thanks Sahil for all your contributions till now. Looking forward to
> > many
> > >> more.
> > >>
> > >> Welcome, Sahil!
> > >>
> > >> Thanks,
> > >> Ashutosh
> > >
> >
> >
>
>
> --
> Sahil Takiar
> Software Engineer
> takiar.sa...@gmail.com | (510) 673-0309
>

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Andrew Sherman]

Congratulations Vineet!

On Mon, Jul 30, 2018 at 12:52 AM Deepak Jaiswal 
wrote:

> Congratulations Vineet!
>
> On 7/30/18, 12:45 AM, "Peter Vary"  wrote:
>
> Congratulations Vineet!
>
> > On Jul 30, 2018, at 01:59, Ashutosh Chauhan 
> wrote:
> >
> > On behalf of the Hive PMC I am delighted to announce Vineet Garg is
> joining
> > Hive PMC.
> > Thanks Vineet for all your contributions till now. Looking forward
> to many
> > more.
> >
> > Welcome, Vineet!
> >
> > Thanks,
> > Ashutosh
>
>
>
>

Re: [ANNOUNCE] New PMC Member : Peter Vary

[Andrew Sherman]

Congratulations Peter!

On Sun, Jul 29, 2018 at 1:32 PM Vineet Garg  wrote:

> Congratulations Peter!
>
> > On Jul 26, 2018, at 11:25 AM, Ashutosh Chauhan 
> wrote:
> >
> > On behalf of the Hive PMC I am delighted to announce Peter Vary is
> joining
> > Hive PMC.
> > Thanks Peter for all your contributions till now. Looking forward to many
> > more.
> >
> > Welcome, Peter!
> >
> > Thanks,
> > Ashutosh
>
>

Re: [ANNOUNCE] New PMC Member : Vihang Karajgaonkar

[Andrew Sherman]

Congratulations Vihang!



On Mon, Jul 30, 2018 at 12:44 AM Peter Vary 
wrote:

> Congratulations Vihang!
>
> > On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> >
> > Congratulations Vihang!
> >
> >> On Jul 26, 2018, at 11:27 AM, Ashutosh Chauhan 
> wrote:
> >>
> >> On behalf of the Hive PMC I am delighted to announce Vihang
> Karajgaonkar
> >> is joining Hive PMC.
> >> Thanks Vihang for all your contributions till now. Looking forward to
> many
> >> more.
> >>
> >> Welcome, Vihang!
> >>
> >> Thanks,
> >> Ashutosh
> >
>
>

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Andrew Sherman]

Congratulations Slim!

On Mon, Jul 30, 2018 at 12:46 AM Peter Vary 
wrote:

> Congratulations Slim!
>
> > On Jul 30, 2018, at 02:00, Ashutosh Chauhan 
> wrote:
> >
> > Apache Hive's Project Management Committee (PMC) has invited Slim
> Bouguerra
> > to become a committer, and we are pleased to announce that he has
> accepted.
> >
> > Slim, welcome, thank you for your contributions, and we look forward your
> > further interactions with the community!
> >
> > Ashutosh Chauhan (on behalf of the Apache Hive PMC)
>
>

[jira] [Created] (HIVE-20272) Parsing error on fields with dashes inside nested structures

[Eugeniu (JIRA)]

Eugeniu created HIVE-20272:
--

 Summary: Parsing error on fields with dashes inside nested 
structures
 Key: HIVE-20272
 URL: https://issues.apache.org/jira/browse/HIVE-20272
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.3.3
Reporter: Eugeniu


Following table declaration throws an error

{code}
CREATE EXTERNAL TABLE t (
parent STRUCT<
`child-field`: STRUCT<
field1: BOOLEAN,
field2: TIMESTAMP
>
>
)
ROW FORMAT
SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE
LOCATION 's3://bucket/path/';
{code}

Exception:
{code}
ERROR : FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.IllegalArgumentException: 
Error: : expected at the position 12 of 
'struct>' but '-' is found.
{code}

Expected behavior: table is created without raising exceptions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

hive debug with tez

[Jim Hopper]

Hi,

I am trying to debug hive-exec (3.0.0) code running in local mode with Tez
as an execution engine.
So far I managed to successfully create tez session and complete compile
phase.

2018-07-30T17:50:05,773  INFO [main] common.TezYARNUtils: Ignoring
'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2018-07-30T17:50:05,987  INFO [main] client.LocalClient: DAGAppMaster
thread has been created
2018-07-30T17:50:05,987  INFO [main] client.LocalClient: DAGAppMaster is
not created wait for 100ms...
2018-07-30T17:50:05,988  INFO [DAGAppMaster Thread] client.LocalClient:
Using working directory:
/tmp/hive/_tez_session_dir/84a05842-702d-4a8d-b5c6-bdedfbd8d203/.tez/application_1532969405620_0001_wd
2018-07-30T17:50:06,087  INFO [main] client.LocalClient: DAGAppMaster is
not created wait for 100ms...
2018-07-30T17:50:06,145  INFO [DAGAppMaster Thread] app.DAGAppMaster:
Created DAGAppMaster for application appattempt_1532969405620_0001_00,
versionInfo=[ component=tez-dag, version=0.9.1,
revision=23b58b2b996eee255aab1a045412de00677ca2f1, SCM-URL=scm:git:
https://git-wip-us.apache.org/repos/asf/tez.git,
buildTime=2017-12-13T00:06:01Z ]
2018-07-30T17:50:06,157  INFO [DAGAppMaster Thread] shim.HadoopShimsLoader:
Trying to locate HadoopShimProvider for hadoopVersion=3.1.0,
majorVersion=3, minorVersion=1
2018-07-30T17:50:06,158  INFO [DAGAppMaster Thread] shim.HadoopShimsLoader:
Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim,
providerName=null, overrideProviderViaConfig=null, hadoopVersion=3.1.0,
majorVersion=3, minorVersion=1
2018-07-30T17:50:06,172  INFO [DAGAppMaster Thread] app.DAGAppMaster: AM
Level configured TaskSchedulers: [0:TezUber:null]
2018-07-30T17:50:06,172  INFO [DAGAppMaster Thread] app.DAGAppMaster: AM
Level configured ContainerLaunchers: [0:TezUber:null]
2018-07-30T17:50:06,172  INFO [DAGAppMaster Thread] app.DAGAppMaster: AM
Level configured TaskCommunicators: [0:TezUber:null]
2018-07-30T17:50:06,172  INFO [DAGAppMaster Thread] app.DAGAppMaster:
Comparing client version with AM version, clientVersion=0.9.1,
AMVersion=0.9.1
2018-07-30T17:50:06,187  INFO [main] client.LocalClient: DAGAppMaster
state: NEW
2018-07-30T17:50:06,187  INFO [main] client.LocalClient: DAGAppMaster is
not started wait for 100ms...
2018-07-30T17:50:06,187  INFO [main] client.LocalClient: DAGAppMaster is
not created wait for 100ms...
2018-07-30T17:50:06,205  INFO [DAGAppMaster Thread]
app.TaskCommunicatorManager: Creating Default Local Task Communicator
2018-07-30T17:50:06,228  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType
for class org.apache.tez.dag.app.rm.container.AMContainerMap
2018-07-30T17:50:06,231  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class
org.apache.tez.dag.app.rm.node.AMNodeTracker
2018-07-30T17:50:06,233  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType
for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2018-07-30T17:50:06,233  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class
org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2018-07-30T17:50:06,233  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.dag.event.VertexEventType for
class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2018-07-30T17:50:06,233  INFO [DAGAppMaster Thread] app.DAGAppMaster: Using
concurrent dispatcher: false
2018-07-30T17:50:06,234  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class
org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2018-07-30T17:50:06,234  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for
class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2018-07-30T17:50:06,235  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for
independent dispatch using: class
org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2018-07-30T17:50:06,235  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for
class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2018-07-30T17:50:06,243  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class
org.apache.tez.dag.app.rm.TaskSchedulerManager
2018-07-30T17:50:06,251  INFO [DAGAppMaster Thread]
launcher.TezContainerLauncherImpl: Creating LocalContainerLauncher
2018-07-30T17:50:06,282  INFO [DAGAppMaster Thread] common.AsyncDispatcher:
Registering class 

Re: Review Request 68108: HIVE-19097 related equals and in operators may cause inaccurate stats estimations

[Zoltan Haindrich]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68108/#review206599
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Line 2141 (original), 2141 (patched)


Is it ok to lower this?
introduced in https://issues.apache.org/jira/browse/HIVE-11573
I think this feature was good as is; because it was originally designed to 
reduce computation for big OR s



ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out
Line 197 (original)


I suspect that somehow the calcite optree to sql converter have run into 
problems during interpreting IN...



ql/src/test/results/clientpositive/llap/vectorized_case.q.out
Line 54 (original), 54 (patched)


this filter is rewritten into an IN close; which resulted in the drop of 
vectorization - because of casting issues


- Zoltan Haindrich


On July 30, 2018, 4:13 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68108/
> ---
> 
> (Updated July 30, 2018, 4:13 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Gopal V.
> 
> 
> Bugs: HIVE-19097
> https://issues.apache.org/jira/browse/HIVE-19097
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> * open in to or
> * close ors into in at 2
> * wip patch
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 39c77b3fe52cb7d7d255138bb71d77a170347b52 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
>  f544f586321c2496e9f3bc3b428ae5689bf046a9 
>   ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestCounterMapping.java 
> b57b5ddc2cba688897bac4cf29eaf9679b6de375 
>   ql/src/test/results/clientpositive/alter_partition_coltype.q.out 
> 5d033a3c0181d697f08ac1aeb62ef57071602050 
>   ql/src/test/results/clientpositive/annotate_stats_filter.q.out 
> 54395886d2a58d83b0fd853c0008fc99f7063f0c 
>   ql/src/test/results/clientpositive/annotate_stats_part.q.out 
> 29ef214ff0e8678fe25bd1793a9522ea0ae61d32 
>   ql/src/test/results/clientpositive/auto_join19.q.out 
> 3e07ec06de767099aeb14beea586bca4cca0784e 
>   ql/src/test/results/clientpositive/cbo_rp_simple_select.q.out 
> d12b5f64cc70e0039ce9179c5c6fb90d8beba0d1 
>   ql/src/test/results/clientpositive/cbo_simple_select.q.out 
> 588d924e6af9b704577b9acf9a01d139c1b13af7 
>   ql/src/test/results/clientpositive/druid_basic3.q.out 
> 54719f751769722c6b099682b9742b1e21c6ebec 
>   ql/src/test/results/clientpositive/druid_intervals.q.out 
> a5203c31822ef6cb8c70a0caa9b682b6f73e88f5 
>   ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out 
> 97922c2636f5b7bc101505bd9fd69b8f24719003 
>   ql/src/test/results/clientpositive/filter_cond_pushdown.q.out 
> b84a2d4b796acc0cc80eef48ddecb97aeb96f7c5 
>   ql/src/test/results/clientpositive/fold_eq_with_case_when.q.out 
> d06fb603458941dd7f79485c6cd8b3a105995c54 
>   ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 
> 6217adb5d4a1c04506d8020a9c9d4b7932c637a3 
>   ql/src/test/results/clientpositive/llap/bucketpruning1.q.out 
> cc637db050d5ec915220865c654e7596a8fe 
>   ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_7.q.out 
> c7f5b887b6396cf60b02b70e87371a21a0052b57 
>   ql/src/test/results/clientpositive/llap/cbo_simple_select.q.out 
> a35edb42a851d5c6ca6ff7fb21a8172788c34d55 
>   ql/src/test/results/clientpositive/llap/check_constraint.q.out 
> 123a3e46fccefba6ee12b98424b785c0eaef4eed 
>   ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
> 8f06ee58ceed021a254491f62069e1b8e18c1541 
>   ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 
> e03cd3437e34179d0557ff1de63ca31cd7f1e3fe 
>   ql/src/test/results/clientpositive/llap/explainuser_1.q.out 
> 708fa176170bb615d1032c810a8332fb64c9a23d 
>   ql/src/test/results/clientpositive/llap/kryo.q.out 
> 234bae89c7a25da3b0db2efa397444dd6c5872c4 
>   ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out 
> 88ddd9c0767b6c23f423ab1ad7e350eba7f85d1a 
>   ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out 
> 1841f1f4d3222314374d155fe5f7484a7e4628f2 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out 
> d7c92d8c59e26a899129f1f816fc79d9a67b5c6b 
>   ql/src/test/results/clientpositive/llap/vector_between_in.q.out 
> 12ae1032eabb51a6c9c9f1a7204483128eb2848d 
>   ql/src/test/results/clientpositive/llap/vector_string_decimal.q.out 
> 54d9914caa6f1cfc569ee8237e83bfc345382e66 
>   
> 

[jira] [Created] (HIVE-20271) Improve HoS query cancellation handling

[Sahil Takiar (JIRA)]

Sahil Takiar created HIVE-20271:
---

 Summary: Improve HoS query cancellation handling
 Key: HIVE-20271
 URL: https://issues.apache.org/jira/browse/HIVE-20271
 Project: Hive
  Issue Type: Improvement
  Components: Spark
Reporter: Sahil Takiar
Assignee: Sahil Takiar


Uber-JIRA for improving HoS query cancellation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 68109: HIVE-20260 NDV of a column shouldn't be scaled when row count is changed by filter on another column

[Zoltan Haindrich]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68109/
---

Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-20260
https://issues.apache.org/jira/browse/HIVE-20260


Repository: hive-git


Description
---

* keep track of used column; and only rescale affected columns
* much more conservative than old logic - possible too much...
* wip patch


Diffs
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/AnnotateStatsProcCtx.java
 47ee949fbcfa9391c640719a57fab39279c009db 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 3c2b0854269d5426153958096a8b5b5ad3612c0f 
  ql/src/test/queries/clientpositive/stat_estimate_drill.q PRE-CREATION 
  ql/src/test/queries/clientpositive/stat_estimate_related_col.q 
52da2f759a009daa372a53446e2f0fd4a88152be 
  ql/src/test/results/clientpositive/stat_estimate_drill.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/stat_estimate_related_col.q.out 
669adafda3a45f7846face3d99817cd1b9cb3664 


Diff: https://reviews.apache.org/r/68109/diff/1/


Testing
---


Thanks,

Zoltan Haindrich

Review Request 68108: HIVE-19097 related equals and in operators may cause inaccurate stats estimations

[Zoltan Haindrich]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68108/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

* open in to or
* close ors into in at 2
* wip patch


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
39c77b3fe52cb7d7d255138bb71d77a170347b52 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
 f544f586321c2496e9f3bc3b428ae5689bf046a9 
  ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestCounterMapping.java 
b57b5ddc2cba688897bac4cf29eaf9679b6de375 
  ql/src/test/results/clientpositive/alter_partition_coltype.q.out 
5d033a3c0181d697f08ac1aeb62ef57071602050 
  ql/src/test/results/clientpositive/annotate_stats_filter.q.out 
54395886d2a58d83b0fd853c0008fc99f7063f0c 
  ql/src/test/results/clientpositive/annotate_stats_part.q.out 
29ef214ff0e8678fe25bd1793a9522ea0ae61d32 
  ql/src/test/results/clientpositive/auto_join19.q.out 
3e07ec06de767099aeb14beea586bca4cca0784e 
  ql/src/test/results/clientpositive/cbo_rp_simple_select.q.out 
d12b5f64cc70e0039ce9179c5c6fb90d8beba0d1 
  ql/src/test/results/clientpositive/cbo_simple_select.q.out 
588d924e6af9b704577b9acf9a01d139c1b13af7 
  ql/src/test/results/clientpositive/druid_basic3.q.out 
54719f751769722c6b099682b9742b1e21c6ebec 
  ql/src/test/results/clientpositive/druid_intervals.q.out 
a5203c31822ef6cb8c70a0caa9b682b6f73e88f5 
  ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out 
97922c2636f5b7bc101505bd9fd69b8f24719003 
  ql/src/test/results/clientpositive/filter_cond_pushdown.q.out 
b84a2d4b796acc0cc80eef48ddecb97aeb96f7c5 
  ql/src/test/results/clientpositive/fold_eq_with_case_when.q.out 
d06fb603458941dd7f79485c6cd8b3a105995c54 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 
6217adb5d4a1c04506d8020a9c9d4b7932c637a3 
  ql/src/test/results/clientpositive/llap/bucketpruning1.q.out 
cc637db050d5ec915220865c654e7596a8fe 
  ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_7.q.out 
c7f5b887b6396cf60b02b70e87371a21a0052b57 
  ql/src/test/results/clientpositive/llap/cbo_simple_select.q.out 
a35edb42a851d5c6ca6ff7fb21a8172788c34d55 
  ql/src/test/results/clientpositive/llap/check_constraint.q.out 
123a3e46fccefba6ee12b98424b785c0eaef4eed 
  ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
8f06ee58ceed021a254491f62069e1b8e18c1541 
  ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 
e03cd3437e34179d0557ff1de63ca31cd7f1e3fe 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 
708fa176170bb615d1032c810a8332fb64c9a23d 
  ql/src/test/results/clientpositive/llap/kryo.q.out 
234bae89c7a25da3b0db2efa397444dd6c5872c4 
  ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out 
88ddd9c0767b6c23f423ab1ad7e350eba7f85d1a 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out 
1841f1f4d3222314374d155fe5f7484a7e4628f2 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out 
d7c92d8c59e26a899129f1f816fc79d9a67b5c6b 
  ql/src/test/results/clientpositive/llap/vector_between_in.q.out 
12ae1032eabb51a6c9c9f1a7204483128eb2848d 
  ql/src/test/results/clientpositive/llap/vector_string_decimal.q.out 
54d9914caa6f1cfc569ee8237e83bfc345382e66 
  
ql/src/test/results/clientpositive/llap/vector_windowing_multipartitioning.q.out
 725ed34acb6b9e1e5e074a31571a202720480aaa 
  ql/src/test/results/clientpositive/llap/vector_windowing_navfn.q.out 
74ac56d1c6989e33f48f82a25742c1c020a9494b 
  ql/src/test/results/clientpositive/llap/vectorized_case.q.out 
d444ae86a10ec96a999895b118aab3fa7ac9c653 
  
ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out
 ba004e97168807da0718c75d371dcdfdc741dca1 
  ql/src/test/results/clientpositive/pcr.q.out 
919b71234d56a63789c33470022bbf637b05 
  ql/src/test/results/clientpositive/perf/spark/query13.q.out 
fb2a061c635f22594bfeb1baa5e8af61bb6b80fe 
  ql/src/test/results/clientpositive/perf/spark/query15.q.out 
3d6fbdac777e68b65ddd1d21f888f8f53ab4b704 
  ql/src/test/results/clientpositive/perf/spark/query34.q.out 
b40081e4f07b0a0cfb10157ae917c233e1295de9 
  ql/src/test/results/clientpositive/perf/spark/query45.q.out 
d61f8b80521748965ba8b19992013ef028599c50 
  ql/src/test/results/clientpositive/perf/spark/query48.q.out 
60a4767a14575b2c3adf998a561c5a459eb2d2dc 
  ql/src/test/results/clientpositive/perf/spark/query53.q.out 
2b1cdfea988efd767f18b8ff92792c0e97fde280 
  ql/src/test/results/clientpositive/perf/spark/query63.q.out 
b506455dbfd317a8cc9d64fd211a4e366993aa79 
  ql/src/test/results/clientpositive/perf/spark/query71.q.out 
bf9c06debf98463c8554e4ea2f00a94bb5f4def1 
  ql/src/test/results/clientpositive/perf/spark/query73.q.out 
20ec874e88dd07785fee487892c1338ce5c85596 
  

Re: Question on Class Naming

[Alan Gates]

Class naming isn't very consistent across much of Hive.  I'd name it
GenericUDFTimestampAnsi or something that captures the fact that it's a UDF
and not a cast.

Alan.

On Sat, Jul 28, 2018 at 8:23 PM Shawn Weeks 
wrote:

> Hi, I’m working on a proper ansi sql to_timestamp function for a project
> and I’d like it to be done so that I can contribute it. I’m trying to
> figure out what to call the class though. Normally I’d assume
> GenericUDFTimestamp would make sense to match the existing to_date function
> but currently the timestamp casting function is called that while all the
> other casting functions are called GenericUDFToTypeName. Any ideas?
>
> Thanks
> Shawn
>

[jira] [Created] (HIVE-20270) Don't serialize hashCode for groupByKey

[Sahil Takiar (JIRA)]

Sahil Takiar created HIVE-20270:
---

 Summary: Don't serialize hashCode for groupByKey
 Key: HIVE-20270
 URL: https://issues.apache.org/jira/browse/HIVE-20270
 Project: Hive
  Issue Type: Bug
  Components: Spark
Reporter: Sahil Takiar
Assignee: Sahil Takiar


Similar to HIVE-20032, but for {{groupByKey}}. The tricky part with 
{{groupByKey}} is we need to preserve the {{hashCode}} until the key gets 
partitioned (via the {{HashPartitioner}}) but after that we don't really need 
to preserve the {{hashCode}}. The {{groupByKey}} operator in Spark does require 
a {{hashCode}} since it puts everything in a map, but it can use a different 
hash-code than the one specified in {{HiveKey}}. The hashcode in {{HiveKey}} is 
only important for determining the partition the key should be assigned to.

The drawback is that computing the hashcode for each {{HiveKey}} might require 
more CPU resources, but we should profile it just in case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68011: HIVE-19770 Support for CBO for queries with multiple same columns in select

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68011/#review206598
---




ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out
Line 115 (original), 119 (patched)


Constant propagation missed opportunity?



ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out
Line 157 (original), 161 (patched)


Constant propagation missed opportunity?



ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out
Lines 264 (patched)


This looks like a perf regression, shouldn't we be pushing this predicate 
through the Select to the TS?



ql/src/test/results/clientpositive/smb_mapjoin_20.q.out
Line 130 (original), 130 (patched)


Constant propagation missed opportunity?



ql/src/test/results/clientpositive/smb_mapjoin_20.q.out
Line 299 (original), 299 (patched)


Constant propagation missed opportunity?


- Jesús Camacho Rodríguez


On July 27, 2018, 11:14 p.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68011/
> ---
> 
> (Updated July 27, 2018, 11:14 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Jesús Camacho Rodríguez.
> 
> 
> Bugs: HIVE-19770
> https://issues.apache.org/jira/browse/HIVE-19770
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> See Jira
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java f008c4dfae 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 37c841fbd1 
>   ql/src/test/queries/clientpositive/masking_8.q 94e4106101 
>   ql/src/test/results/clientnegative/ambiguous_col.q.out a2915a4a5d 
>   ql/src/test/results/clientnegative/create_view_failure5.q.out d79dc64a30 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out 
> 6c45fcb7ac 
>   ql/src/test/results/clientpositive/char_udf1.q.out 69d76d7269 
>   ql/src/test/results/clientpositive/keyword_2.q.out f1d63b6e5f 
>   ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 
> f707ab47be 
>   ql/src/test/results/clientpositive/llap/explainanalyze_2.q.out ab86821f07 
>   ql/src/test/results/clientpositive/llap/explainuser_2.q.out 5f5f5f6015 
>   ql/src/test/results/clientpositive/llap/limit_join_transpose.q.out 
> ebaac18127 
>   ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_part.q.out 
> 97752f3c25 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_part_llap_io.q.out
>  23c33a3141 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_part_update.q.out
>  eeabb8cc61 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_part_update_llap_io.q.out
>  f15a144a96 
>   ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_table.q.out 
> a043b679ae 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_table_llap_io.q.out
>  35c1fae6d0 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_table_update.q.out
>  730d3d2312 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acid_table_update_llap_io.q.out
>  95bfa2507d 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acidvec_part_llap_io.q.out
>  7e1cce3f4f 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acidvec_part_update.q.out
>  242b95e603 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acidvec_part_update_llap_io.q.out
>  53cb8fc8c4 
>   ql/src/test/results/clientpositive/llap/schema_evol_orc_acidvec_table.q.out 
> 219ad7a82e 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acidvec_table_llap_io.q.out
>  ce9fe84d1e 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acidvec_table_update.q.out
>  f8df92faf7 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_acidvec_table_update_llap_io.q.out
>  2292dcb24b 
>   ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_part.q.out 
> e02a1e247d 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_part_all_primitive.q.out
>  f70efd6f94 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_part_all_primitive_llap_io.q.out
>  351d201700 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_part_llap_io.q.out
>  603c669f51 
>   ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_table.q.out 
> 454bf264d3 
>   
> ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_table_llap_io.q.out
>  7ccda3dc91 
>   

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

[Sahil Takiar]

Thanks!

On Mon, Jul 30, 2018 at 2:44 AM, Peter Vary 
wrote:

> Congratulations Sahil!
>
> > On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> >
> > Congratulations Sahil!
> >
> >> On Jul 26, 2018, at 11:28 AM, Ashutosh Chauhan 
> wrote:
> >>
> >> On behalf of the Hive PMC I am delighted to announce Sahil Takiar is
> >> joining Hive PMC.
> >> Thanks Sahil for all your contributions till now. Looking forward to
> many
> >> more.
> >>
> >> Welcome, Sahil!
> >>
> >> Thanks,
> >> Ashutosh
> >
>
>


-- 
Sahil Takiar
Software Engineer
takiar.sa...@gmail.com | (510) 673-0309

[GitHub] hive pull request #404: HIVE-20209: Metastore connection fails for first att...

[sankarh]

Github user sankarh closed the pull request at:

https://github.com/apache/hive/pull/404


---

[jira] [Created] (HIVE-20269) Unwind between

[Zoltan Haindrich (JIRA)]

Zoltan Haindrich created HIVE-20269:
---

 Summary: Unwind between
 Key: HIVE-20269
 URL: https://issues.apache.org/jira/browse/HIVE-20269
 Project: Hive
  Issue Type: Bug
  Components: Logical Optimizer
Reporter: Zoltan Haindrich


rewrite "x between a and b" to "a<=x && x<=b" could help in using those 
restrictions more during optimizations



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Deepak Jaiswal]

Congratulations Vineet!

On 7/30/18, 12:45 AM, "Peter Vary"  wrote:

Congratulations Vineet!

> On Jul 30, 2018, at 01:59, Ashutosh Chauhan  wrote:
> 
> On behalf of the Hive PMC I am delighted to announce Vineet Garg is 
joining
> Hive PMC.
> Thanks Vineet for all your contributions till now. Looking forward to many
> more.
> 
> Welcome, Vineet!
> 
> Thanks,
> Ashutosh

Re: [ANNOUNCE] New committer: Slim Bouguerra

[Peter Vary]

Congratulations Slim!

> On Jul 30, 2018, at 02:00, Ashutosh Chauhan  wrote:
> 
> Apache Hive's Project Management Committee (PMC) has invited Slim Bouguerra
> to become a committer, and we are pleased to announce that he has accepted.
> 
> Slim, welcome, thank you for your contributions, and we look forward your
> further interactions with the community!
> 
> Ashutosh Chauhan (on behalf of the Apache Hive PMC)

Re: [ANNOUNCE] New PMC Member : Vineet Garg

[Peter Vary]

Congratulations Vineet!

> On Jul 30, 2018, at 01:59, Ashutosh Chauhan  wrote:
> 
> On behalf of the Hive PMC I am delighted to announce Vineet Garg is joining
> Hive PMC.
> Thanks Vineet for all your contributions till now. Looking forward to many
> more.
> 
> Welcome, Vineet!
> 
> Thanks,
> Ashutosh

Re: [ANNOUNCE] New PMC Member : Vihang Karajgaonkar

[Peter Vary]

Congratulations Vihang!

> On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> 
> Congratulations Vihang!
> 
>> On Jul 26, 2018, at 11:27 AM, Ashutosh Chauhan  wrote:
>> 
>> On behalf of the Hive PMC I am delighted to announce Vihang  Karajgaonkar
>> is joining Hive PMC.
>> Thanks Vihang for all your contributions till now. Looking forward to many
>> more.
>> 
>> Welcome, Vihang!
>> 
>> Thanks,
>> Ashutosh
> 

Re: [ANNOUNCE] New PMC Member : Sahil Takiar

[Peter Vary]

Congratulations Sahil!

> On Jul 29, 2018, at 22:32, Vineet Garg  wrote:
> 
> Congratulations Sahil!
> 
>> On Jul 26, 2018, at 11:28 AM, Ashutosh Chauhan  wrote:
>> 
>> On behalf of the Hive PMC I am delighted to announce Sahil Takiar is
>> joining Hive PMC.
>> Thanks Sahil for all your contributions till now. Looking forward to many
>> more.
>> 
>> Welcome, Sahil!
>> 
>> Thanks,
>> Ashutosh
>