date:20170323

Re: Review Request 57772: HIVE-16242 Run BeeLine tests parallel

2017-03-23 Thread Peter Vary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57772/
---

(Updated March 23, 2017, 9:11 a.m.)


Review request for hive, Carl Steinbach, Zoltan Haindrich, Marta Kuczora, 
Miklos Csanady, Naveen Gangam, Vihang Karajgaonkar, Yongzhi Chen, and Barna 
Zsombor Klara.


Changes
---

As Yongzhi pointed out, the "Acquired the compile lock." messages should be 
removed as well.


Bugs: HIVE-16242
https://issues.apache.org/jira/browse/HIVE-16242


Repository: hive-git


Description
---

- Added a parallelized + parametrized junit runner
- Removed the lock related logs from the output


Diffs (updated)
-

  itests/qtest/src/test/java/org/apache/hadoop/hive/cli/TestBeeLineDriver.java 
24eeb9d 
  itests/util/src/main/java/org/apache/hive/beeline/Parallelized.java 
PRE-CREATION 
  itests/util/src/main/java/org/apache/hive/beeline/qfile/QFile.java 49d6d24 
  ql/src/test/results/clientpositive/beeline/drop_with_concurrency.q.out 
d22c9ec 
  ql/src/test/results/clientpositive/beeline/escape_comments.q.out 5f9df93 


Diff: https://reviews.apache.org/r/57772/diff/4/

Changes: https://reviews.apache.org/r/57772/diff/3-4/


Testing
---

Manually run the tests several times.
Validated that the tests are running in parallel.


Thanks,

Peter Vary

Re: Review Request 57728: HIVE-16231: Parquet timestamp may be stored differently since HIVE-12767

2017-03-23 Thread Barna Zsombor Klara



> On March 22, 2017, 6:27 p.m., Sergio Pena wrote:
> > common/src/java/org/apache/hive/common/util/DateUtils.java
> > Lines 84 (patched)
> > 
> >
> > Is there another class where to put this method? I don't think 
> > DateUtils is the place where we should keep this.

I couldn't find a much better fit. I looked at HiveUtils and ParquetTableUtils 
but DateUtils seemed better. I can create a TimeZoneUtils class, but I don't 
know if we will ever have a second function in it. Do you have a utility class 
in mind that would be better?


- Barna Zsombor


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57728/#review169758
---


On March 21, 2017, 5:28 p.m., Barna Zsombor Klara wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57728/
> ---
> 
> (Updated March 21, 2017, 5:28 p.m.)
> 
> 
> Review request for hive and Sergio Pena.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16231: Parquet timestamp may be stored differently since HIVE-12767
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hive/common/util/DateUtils.java 
> a1068ecce94e9ff1ae78008a0d8c6d67ca4f2690 
>   common/src/test/org/apache/hive/common/util/TestDateUtils.java PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetOutputFormat.java
>  26f1e75c7d659a634cd4eef3a0cb8e886b22722f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ParquetRecordReaderBase.java 
> 8e33b7d437894b33b35f32913a3bc02f2a849ce3 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/timestamp/NanoTimeUtils.java 
> 5dc808800290f3274afbdff12134ac34387a746b 
>   ql/src/test/queries/clientpositive/parquet_int96_timestamp.q 
> 5de2c3f1244b8340b97eb0547fe66e52d80fb065 
> 
> 
> Diff: https://reviews.apache.org/r/57728/diff/2/
> 
> 
> Testing
> ---
> 
> Tested loading timestamps from a parquet file written by spark.
> 
> 
> Thanks,
> 
> Barna Zsombor Klara
> 
>

Re: Review Request 57772: HIVE-16242 Run BeeLine tests parallel

2017-03-23 Thread Peter Vary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57772/
---

(Updated March 23, 2017, 2:50 p.m.)


Review request for hive, Carl Steinbach, Zoltan Haindrich, Marta Kuczora, 
Miklos Csanady, Naveen Gangam, Vihang Karajgaonkar, Yongzhi Chen, and Barna 
Zsombor Klara.


Changes
---

Changed the patch based on Yongchi's comment.


Bugs: HIVE-16242
https://issues.apache.org/jira/browse/HIVE-16242


Repository: hive-git


Description
---

- Added a parallelized + parametrized junit runner
- Removed the lock related logs from the output


Diffs (updated)
-

  itests/qtest/src/test/java/org/apache/hadoop/hive/cli/TestBeeLineDriver.java 
24eeb9d 
  itests/util/src/main/java/org/apache/hive/beeline/Parallelized.java 
PRE-CREATION 
  itests/util/src/main/java/org/apache/hive/beeline/qfile/QFile.java 49d6d24 


Diff: https://reviews.apache.org/r/57772/diff/5/

Changes: https://reviews.apache.org/r/57772/diff/4-5/


Testing
---

Manually run the tests several times.
Validated that the tests are running in parallel.


Thanks,

Peter Vary

Re: Backward incompatible changes

2017-03-23 Thread Eugene Koifman

+1 to make a release first

On 3/22/17, 2:06 PM, "Sergey Shelukhin"  wrote:

Hmm.. should we release these first, and then cut branch-2?
Otherwise during the releases, the patches for 2.2/2.3 will need to go to
3 (4?) places (master, branch-2, branch-2.2, branch-2.3?).
There’s no rush to cut the branch if everything in 2.2/2.3 has to go to
3.0 anyway.

On 17/3/22, 13:53, "Pengcheng Xiong"  wrote:

>I would like to work as the Release Manager if possible. As Owen points
>out, he is working on 2.2 and I will work on 2.3. Thanks.
>
>On Wed, Mar 22, 2017 at 1:32 PM, Ashutosh Chauhan 
>wrote:
>
>> Unless there is more feedback, I plan to cut branch-2 in a day or two
>>from
>> current master. As multiple people have suggested on this thread, we
>>should
>> do a 2.2 release soon. Currently there are 177 issues
>> > 3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20cf%
>> 5B12310320%5D%20%3D%202.2.0%20ORDER%20BY%20priority%20DESC>
>> targeted for 2.2 release. We can use branch-2 to land these patches and
>>for
>> additional stabilization efforts. Any volunteer for Release Manager
>>driving
>> 2.2 release?
>>
>> Thanks,
>> Ashutosh
>>
>> On Fri, Mar 10, 2017 at 4:23 PM, Ashutosh Chauhan 
>> wrote:
>>
>> > I hear what you are saying. Lets begin with 3 concerns:
>> >
>> > - How will we keep the community motivated on fixing both master and
>> > branch-2?
>> > Until we do a stable release from master, stable releases can come
>>only
>> > from branch-2. If a contributor wants to see their fix reach to users
>>on
>> a
>> > stable line quickly they would have to have a fix on branch-2. Also, a
>> > release manager can pick whatever fixes she wants, so even if
>>contributor
>> > doesn't commit it on branch-2, a release manger who wants to do a
>>release
>> > containing a set of fixes thats always possible.
>> >
>> > - *Harder cherry-picks between master and branch-2*.
>> > That is certainly possible. But hope is we want to keep branch-2
>>stable,
>> > so we don't backport large features which may run into this issue.
>> Smaller
>> > focussed bug fix backport should be possible.
>> >
>> >
>> >- *Removal of MR2 on the master branch*.
>> > This is something I personally would like to see. But exact timing of
>>it
>> > will be decided by community. I am certainly not saying that as soon
>>as
>> > branch-2 is created, lets remove MR2 on master.
>> >
>> > I would also say that in the end ASF is volunteer organization, we
>>cant
>> > force people to adopt one branch or another. Its upto the contributors
>> what
>> > jiras they work on and when and where they commit it.
>> > By not creating a branch-2 only thing we can guarantee is that rate of
>> > development on master to remain slow because we don't want to start
>>doing
>> > backward incompatible changes without explicitly acknowledging that.
>> >
>> > Thanks,
>> > Ashutosh
>> >
>> > On Thu, Mar 9, 2017 at 12:01 PM, Sergio Pena
>>
>> > wrote:
>> >
>> >> Hey Ashutosh, thanks for soliciting feedback on this.
>> >>
>> >> I like the idea you're proposing; maintaining compatibility and at
>>the
>> >> same time adding newer features to
>> >> Hive consumes a lot of development time and effort.
>> >>
>> >> However, I think some users and companies have just started to use
>>Hive
>> >> 2.x
>> >> branch as their main major upgrade on Hive
>> >> (possible due to waiting for stabilization and testing upgrades), but
>> >> cutting this major branch that just has 1 year of life
>> >> might make us look like we will forget about the quality of Hive 2.x
>>as
>> we
>> >> did with branch-1.
>> >>
>> >> Hive 1.x latest version was 1.2, and its development stopped because
>>new
>> >> features on Hive 2.x
>> >> Hive 2.x latest version is 2.1, and we want to create Hive 3.x
>>because
>> of
>> >> newer features and incompatibilities.
>> >> Will Hive 3.x have the same future after 3.1 is released?
>> >>
>> >> What I'm also concerned is about these three things:
>> >>
>> >>- *Branch-2 quality commitment*.
>> >>How will we keep the community motivated on fixing both master and
>> >>branch-2?
>> >>- *Harder cherry-picks between master and branch-2*.
>> >>Because master will be incompatible by nature, then cherry-picks
>>to
>> >>branch-2 will be harder.
>> >>- *Removal of MR2 on the master branch*.
>> >>This was marked as deprecated just last year, but MR2 is still an
>> >> engine
>> >>that is used by several users.
>> >

Re: Review Request 57632: HIVE-16206: Provide wrapper classes for current metrics reporters to allow uniform instantiation through reflection

2017-03-23 Thread Sunitha Beeram via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57632/
---

(Updated March 23, 2017, 4:27 p.m.)


Review request for hive, Carl Steinbach and Ratandeep Ratti.


Bugs: Hive-16206
https://issues.apache.org/jira/browse/Hive-16206


Repository: hive-git


Description (updated)
---

HIVE-16206: Minor fixes


Diffs (updated)
-

  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/CodahaleMetrics.java
 e8abf6cf06afc9fa590af3a447eacc67735a69e6 
  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/CodahaleReporter.java
 PRE-CREATION 
  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/ConsoleMetricsReporter.java
 PRE-CREATION 
  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/JmxMetricsReporter.java
 PRE-CREATION 
  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/JsonFileMetricsReporter.java
 PRE-CREATION 
  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/Metrics2Reporter.java
 PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
1fb32533d58af4ec622feb320bf9315da5db6e76 
  
common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestCodahaleMetrics.java
 aa4e75f9f8160d1b54b14c1a23ea42e156bd45ca 
  
common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestCodahaleReportersConf.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/57632/diff/6/

Changes: https://reviews.apache.org/r/57632/diff/5-6/


Testing
---

Updated unit tests and all unit tests passed locally.


Thanks,

Sunitha Beeram

[jira] [Created] (HIVE-16286) Log canceled query id

2017-03-23 Thread Jimmy Xiang (JIRA)

Jimmy Xiang created HIVE-16286:
--

 Summary: Log canceled query id
 Key: HIVE-16286
 URL: https://issues.apache.org/jira/browse/HIVE-16286
 Project: Hive
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Trivial


Currently, just a generic message is logged when a query is canceled. It is 
better to log the query id as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16287) Alter table partition rename with location - moves partition back to hive warehouse

2017-03-23 Thread Ying Chen (JIRA)

Ying Chen created HIVE-16287:


 Summary: Alter table partition rename with location - moves 
partition back to hive warehouse
 Key: HIVE-16287
 URL: https://issues.apache.org/jira/browse/HIVE-16287
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 2.1.0
 Environment: RHEL 6.8 
Reporter: Ying Chen
Priority: Minor


I was renaming my partition in a table that I've created using the location 
clause, and noticed that when after rename is completed, my partition is moved 
to the hive warehouse (hive.metastore.warehouse.dir).

{quote}
create table test_local_part (col1 int) partitioned by (col2 int) location 
'/tmp/testtable/test_local_part';
insert into test_local_part  partition (col2=1) values (1),(3);
insert into test_local_part  partition (col2=2) values (3);
alter table test_local_part partition (col2='1') rename to partition (col2='4');
{quote}

Running: 
  {{describe formatted test_local_part partition (col2='2')}} 

# Detailed Partition Information 
Partition Value:[2]  
Database:   default  
Table:  test_local_part  
CreateTime: Mon Mar 20 13:25:28 PDT 2017 
LastAccessTime: UNKNOWN  
Protect Mode:   None 
Location:   
*hdfs://my.server.com:8020/tmp/testtable/test_local_part/col2=2*

Running: 
   {{describe formatted test_local_part partition (col2='4')}} 

# Detailed Partition Information 
Partition Value:[4]  
Database:   default  
Table:  test_local_part  
CreateTime: Mon Mar 20 13:24:53 PDT 2017 
LastAccessTime: UNKNOWN  
Protect Mode:   None 
Location:   
*hdfs://my.server.com:8020/apps/hive/warehouse/test_local_part/col2=4*





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Alter table partition rename with location - moves partition back to hive warehouse

2017-03-23 Thread Ying Chen

Thanks!  I've created a JIRA:
https://issues.apache.org/jira/browse/HIVE-16287

So, the right behavior would be to create the new partition name in the
same location of the table.

Thanks!

On Wed, Mar 22, 2017 at 11:31 AM, Sergio Pena 
wrote:

> I don't think that behavior is correct. The rename should create the new
> partition name in the same location of the table. Could you create a JIRA
> for that?
>
> On Wed, Mar 22, 2017 at 10:24 AM, Ying Chen  wrote:
>
> > cross posting from u...@hive.apache.org.  (sorry for additional posting)
> > I would like some opinions from the dev community relating to - if change
> > of behavior is desired, how should it be changed ?
> > Also, would it be better that I create a JIRA and have comments there?
> >
> > Thanks much..
> >
> > 
> > Hello all -
> >
> > I was renaming my partition in a table that I've created using the
> location
> > clause, and noticed that when after rename is completed, my partition is
> > moved to the hive warehouse (hive.metastore.warehouse.dir). I was
> > wondering, what should be the correct behavior?  Should the partition be
> > renamed and maintain on the same file system, or no name change and not
> > moved (so treating it like if someone declared external table) ?  I don't
> > think it should be moved to hive.metastore.warehouse.dir
> >
> > A similar JIRA was open for renaming table:  https://issues.apache.org/
> > jira/browse/HIVE-14909
> > In which, if the table is determined not belonging to
> /apps/hive/warehouse
> > (ie created by location clause), then table is not moved.
> >
> > Thanks much!
> > Ying
> >
> > ==
> > This is a problem for Hive 2.1 ...
> >
> > create table test_local_part (col1 int) partitioned by (col2 int)
> location
> > '/tmp/testtable/test_local_part';
> > insert into test_local_part  partition (col2=1) values (1),(3);
> > insert into test_local_part  partition (col2=2) values (3);
> > alter table test_local_part partition (col2='1') rename to partition
> > (col2='4');
> >
> > Running:
> >   describe formatted test_local_part partition (col2='2')
> >
> > # Detailed Partition Information
> > Partition Value: [2]
> > Database:   default
> > Table:   test_local_part
> > CreateTime: Mon Mar 20 13:25:28 PDT 2017
> > LastAccessTime: UNKNOWN
> > Protect Mode:   None
> > Location:
> > *hdfs://my.server.com:8020/tmp/testtable/test_local_part/col2=2
> > *
> >
> > Running:
> >describe formatted test_local_part partition (col2='4')
> >
> > # Detailed Partition Information
> > Partition Value: [4]
> > Database:   default
> > Table:   test_local_part
> > CreateTime: Mon Mar 20 13:24:53 PDT 2017
> > LastAccessTime: UNKNOWN
> > Protect Mode:   None
> > Location:
> > *hdfs://my.server.com:8020/apps/hive/warehouse/test_local_part/col2=4
> > 
> *
> > Partition Parameters:
> >
>

Re: Review Request 57728: HIVE-16231: Parquet timestamp may be stored differently since HIVE-12767

2017-03-23 Thread Sergio Pena



> On March 22, 2017, 6:27 p.m., Sergio Pena wrote:
> > common/src/java/org/apache/hive/common/util/DateUtils.java
> > Lines 84 (patched)
> > 
> >
> > Is there another class where to put this method? I don't think 
> > DateUtils is the place where we should keep this.
> 
> Barna Zsombor Klara wrote:
> I couldn't find a much better fit. I looked at HiveUtils and 
> ParquetTableUtils but DateUtils seemed better. I can create a TimeZoneUtils 
> class, but I don't know if we will ever have a second function in it. Do you 
> have a utility class in mind that would be better?

Well, NanoTimeUtils is not a good name either, but it has the getCalendar() 
public method that returns a calendar based on UTC or local timezone. 
Eventually, we should rename this method to TimeUtils.java or something, but I 
don't know if external apps are using it.


- Sergio


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57728/#review169758
---


On March 21, 2017, 5:28 p.m., Barna Zsombor Klara wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57728/
> ---
> 
> (Updated March 21, 2017, 5:28 p.m.)
> 
> 
> Review request for hive and Sergio Pena.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16231: Parquet timestamp may be stored differently since HIVE-12767
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hive/common/util/DateUtils.java 
> a1068ecce94e9ff1ae78008a0d8c6d67ca4f2690 
>   common/src/test/org/apache/hive/common/util/TestDateUtils.java PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetOutputFormat.java
>  26f1e75c7d659a634cd4eef3a0cb8e886b22722f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ParquetRecordReaderBase.java 
> 8e33b7d437894b33b35f32913a3bc02f2a849ce3 
>   
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/timestamp/NanoTimeUtils.java 
> 5dc808800290f3274afbdff12134ac34387a746b 
>   ql/src/test/queries/clientpositive/parquet_int96_timestamp.q 
> 5de2c3f1244b8340b97eb0547fe66e52d80fb065 
> 
> 
> Diff: https://reviews.apache.org/r/57728/diff/2/
> 
> 
> Testing
> ---
> 
> Tested loading timestamps from a parquet file written by spark.
> 
> 
> Thanks,
> 
> Barna Zsombor Klara
> 
>

Re: Review Request 57626: HIVE-16164: Provide mechanism for passing HMS notification ID between transactional and non-transactional listeners.

2017-03-23 Thread Sergio Pena

> On March 23, 2017, 12:27 a.m., Alexander Kolbasov wrote:
> > This is good, but I am wondering what is the purpose in using different 
> > events between transactional and non-transactional listeners in most cases? 
> > It would be much simpler and cleaner to reuse existing event except for one 
> > or two cases where this isn't possible.

I don't think it would be simpler. Hive could have both or either one of 
transactional or non-transactoinal listeners enabled. If transactional 
listeners are not enabled but listeners, then we have to create the event 
later. If none of listeners are enabled, then the event should not be created. 
This will another IF condition.

I.e.

CreateDatabaseEvent event = null;

try {
  if (!transactionalListeners.isEmpty()) {
event = new CreateDatabaseEvent(...);
event.setEnvironmentContext(...);

call transactionalListeners
  }
} catch (...) {
  if (listeners.isEmpty()) {
if (event == null) {
   event = new CreateDatabaseEvent(...);
   event.setEnvironmentContext(...);
}

call listeners
  }
}

The above code can reuse the CreateDatabaseEvent, but we still need to create 
it if transactionalListeners was disabled. Also, we would need to add code to 
all events to accept the setStatus() method in case we're reusing the event. 
That's why I just remove that and keep it simple by leaving the old way to 
recreate the event.

try {
  if (!transactionalListeners.isEmpty()) {
MetaStoreListenerNotifier
   .newNotification(EventType.CREATE_DATABASE, new 
CreateDatabaseEvent(db, true, this))

.setEnvironmentContext(environmentContext).notify(transactionalListeners);
  }
} catch (...) {
  if (listeners.isEmpty()) {
MetaStoreListenerNotifier
   .newNotification(EventType.CREATE_DATABASE, new 
CreateDatabaseEvent(db, success, this))
   .setEnvironmentContext(environmentContext).notify(listeners);
  }
}

> On March 23, 2017, 12:27 a.m., Alexander Kolbasov wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Line 2491 (original), 2526 (patched)
> > 
> >
> > Why is this the case? The above condition is never null.

Right, but it can be empty.

> On March 23, 2017, 12:27 a.m., Alexander Kolbasov wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Lines 2530 (patched)
> > 
> >
> > So why do we notify them twice - once with success and another time 
> > with failure? Something is wrong here.

I don't know the idea behind this notification. Maybe someone else is 
processing failed events for logging or something. I don't want to remove it 
because of legacy problems.

> On March 23, 2017, 12:27 a.m., Alexander Kolbasov wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Line 2820 (original)
> > 
> >
> > Just to make sure - this is kind of independent cleanup - right?

Yes. I am trying to coming up with a better approach for this cleanup.

> On March 23, 2017, 12:27 a.m., Alexander Kolbasov wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreListenerNotifier.java
> > Lines 195 (patched)
> > 
> >
> > In all places where you call it, you know that listeners != null && 
> > !listeners.empty()
> > 
> > So may be this should be replaced with preconditions check?

That would be a good thing. I'm still figuring out a better way to improve the 
notifier to avoid checking IF() twice (one outside the class, and another in 
this class). But PreCondition is a good one as this class is just an internal 
API.

> On March 23, 2017, 12:27 a.m., Alexander Kolbasov wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/events/ListenerEvent.java
> > Lines 120 (patched)
> > 
> >
> > Loking at the way it is used, I would consider adding constructor with 
> > parameters instead. They are always used like this:
> > 
> > 
> > if (!listeners.isEmpty()) {
> >   MetaStoreListenerNotifier
> >   .newNotification(EventType.CREATE_INDEX, new 
> > AddIndexEvent(index, success, this))
> >   .addParameters(transactionalListenerResponses)
> >   .notify(listeners);
> > }
> > 
> > This is essentially equivalent to constructing a new event with the 
> > given set of parameters.

They're always used only on the non-transactional listeners but 
transactionalListeners. I will think about another static

Review Request 57893: HIVE-16285: Servlet for dynamically configuring log levels

2017-03-23 Thread j . prasanth . j


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57893/
---

Review request for hive.


Bugs: HIVE-16285
https://issues.apache.org/jira/browse/HIVE-16285


Repository: hive-git


Description
---

Many long running services like HS2, LLAP etc. will benefit from having an 
endpoint to dynamically change log levels for various loggers. This will help 
greatly with debuggability without requiring a restart of the service.


Diffs
-

  common/src/java/org/apache/hive/http/HttpServer.java 
db5650d0d2695011cfb07b23950bd611186a25f6 
  common/src/java/org/apache/hive/http/Log4j2ConfiguratorServlet.java 
PRE-CREATION 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorService.java
 9eaa7d731b07352757237a5d26d1229ca34f35a0 
  
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java
 18ce03c484801c4fe1f5aa4f194520cdb22a07f2 
  ql/src/java/org/apache/hadoop/hive/ql/exec/AppMasterEventOperator.java 
bf30ef1c4730fd3e95fb17dfb08d0f420097680a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java 
df1898e9f51a53f38da9caea5366a32e3e4ed7c3 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DemuxOperator.java 
c1847425fd1592e3645f39b355ff76fca435c749 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 
a9d03d060adeaa5cad6bef48a63c048f23819d01 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 
6d6c608fafe8935b3d1ba02f6a304db2034eef47 
  ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java 
3a366f67394704f08fd019d0739cb4f76942e10d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java 
02827635873ffc319fca40fefbb3cea0fdc4c418 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 
07aa2ea6a3006ad4cf4629d4c44cb4d3781d4db6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java 
2a46b301e96a9317df9fa251955d62db943a56a9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MuxOperator.java 
984924368a854d4c6dd55c8f7b30b756461fe70d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 
8b04cd44fa780b545d102b3b533cb495835b4b71 
  ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java 
e3cb765e0a822c96b79e28b658dd8cbb5a401842 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 
e03f4b701d0f29dc1a26e9b7b8662103c888252c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 
7c1e344ce5c07dd5480aee930447034e4285a407 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ScriptOperator.java 
4767af19a3b44bb588ea3d449ba2030612946a5e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java 
94af09780b89482b4a8b8a52682af4d6eeb6d915 
  ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 
68477ca5789ebead7694c2610b7dc0617f798fe6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/UnionOperator.java 
3df5533d382e8c35fb04fa377684850656450590 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecReducer.java 
1d26a133ec37773eb7128807136eeccbb36e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ObjectCache.java 
cfe17506ed0b600bbaf5a2cc2b07f6af6a302628 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ColumnarSplitSizeEstimator.java 
ecd4ddcfdb61717eb0a2424650673d07884d9293 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HostAffinitySplitLocationProvider.java
 dcb985fbf91cb2a582bd0ca4245eabfbcb7a2bba 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/LlapObjectCache.java 
1ce8ee931163ed1c3645aefde4f4583a96fe 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinCommonOperator.java
 f8541321747c7c005e772e9d99597e1880020b82 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinGenerateResultOperator.java
 cb3041382918d5005a261f3ae767cb475e318e56 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerBigOnlyLongOperator.java
 43f3951d7aab3c163b6ca6d4a74520c92876f72b 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerBigOnlyMultiKeyOperator.java
 95fb0c27e5564f1c778a246a9306d469736fc6eb 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerBigOnlyStringOperator.java
 044e3e624b7f414d5d6b84b836de0fce5645f740 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerLongOperator.java
 c85e1d88c35067c70e0c6273afe4b40ffc25aeb5 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerMultiKeyOperator.java
 a108cd0f67347f2847d31df6a2c851c6c43422c4 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerStringOperator.java
 3211d7d1d61acb434dfde260207cd298046fb1bf 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinLeftSemiLongOperator.java
 b02e6fdedc6a709e940065a572cd05c0193880e9 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinLeftSemiMultiKeyOperator.ja

[jira] [Created] (HIVE-16288) Add blobstore tests for ORC and RCFILE file formats

2017-03-23 Thread Thomas Poepping (JIRA)

Thomas Poepping created HIVE-16288:
--

 Summary: Add blobstore tests for ORC and RCFILE file formats
 Key: HIVE-16288
 URL: https://issues.apache.org/jira/browse/HIVE-16288
 Project: Hive
  Issue Type: Test
  Components: Tests
Affects Versions: 2.1.1
Reporter: Thomas Poepping
Assignee: Thomas Poepping


This patch adds four tests each for ORC and RCFILE when running against 
blobstore filesystems:
  * Test for bucketed tables
  * Test for nonpartitioned tables
  * Test for partitioned tables
  * Test for partitioned tables with nonstandard partition locations



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16289) add hints for semijoin reduction

2017-03-23 Thread Sergey Shelukhin (JIRA)

Sergey Shelukhin created HIVE-16289:
---

 Summary: add hints for semijoin reduction
 Key: HIVE-16289
 URL: https://issues.apache.org/jira/browse/HIVE-16289
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin


For now hints will only impact bloom filter size if semijoin is enabled.
In a follow-up, after some cost-based semi-join decision logic is added, they 
may also influence it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 57893: HIVE-16285: Servlet for dynamically configuring log levels

2017-03-23 Thread j . prasanth . j


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57893/
---

(Updated March 24, 2017, 12:11 a.m.)


Review request for hive.


Changes
---

Updated javadoc


Bugs: HIVE-16285
https://issues.apache.org/jira/browse/HIVE-16285


Repository: hive-git


Description
---

Many long running services like HS2, LLAP etc. will benefit from having an 
endpoint to dynamically change log levels for various loggers. This will help 
greatly with debuggability without requiring a restart of the service.


Diffs (updated)
-

  common/src/java/org/apache/hive/http/HttpServer.java db5650d 
  common/src/java/org/apache/hive/http/Log4j2ConfiguratorServlet.java 
PRE-CREATION 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorService.java
 9eaa7d7 
  
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java
 18ce03c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/AppMasterEventOperator.java 
bf30ef1 
  ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java df1898e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DemuxOperator.java c184742 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java a9d03d0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 6d6c608 
  ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java 3a366f6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java 0282763 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 07aa2ea 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java 2a46b30 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MuxOperator.java 9849243 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 8b04cd4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/OrcFileMergeOperator.java e3cb765 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java e03f4b7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 7c1e344 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ScriptOperator.java 4767af1 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java 94af097 
  ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 68477ca 
  ql/src/java/org/apache/hadoop/hive/ql/exec/UnionOperator.java 3df5533 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecReducer.java 1d2 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ObjectCache.java cfe1750 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ColumnarSplitSizeEstimator.java 
ecd4ddc 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HostAffinitySplitLocationProvider.java
 dcb985f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/LlapObjectCache.java 1ce8ee9 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinCommonOperator.java
 f854132 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinGenerateResultOperator.java
 cb30413 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerBigOnlyLongOperator.java
 43f3951 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerBigOnlyMultiKeyOperator.java
 95fb0c2 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerBigOnlyStringOperator.java
 044e3e6 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerLongOperator.java
 c85e1d8 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerMultiKeyOperator.java
 a108cd0 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerStringOperator.java
 3211d7d 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinLeftSemiLongOperator.java
 b02e6fd 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinLeftSemiMultiKeyOperator.java
 36b8f3f 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinLeftSemiStringOperator.java
 0b3de0a 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinOuterGenerateResultOperator.java
 0e2d65a 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinOuterLongOperator.java
 72309e8 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinOuterMultiKeyOperator.java
 a4fc7d3 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinOuterStringOperator.java
 6e7e5cb 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashTable.java
 b93e977 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashTable.java
 8bfa07c 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java
 fc5aea5 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java 9299306 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 8318a62 
  
ql/src/java/org/apache/hadoop/hive

Review Request 57903: HIVE-16282 Add faststart for semijoin reducer

2017-03-23 Thread Deepak Jaiswal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57903/
---

Review request for hive, Gopal V, Jason Dere, and Siddharth Seth.


Bugs: HIVE-16282
https://issues.apache.org/jira/browse/HIVE-16282


Repository: hive-git


Description
---

Currently all reducers have slow start meaning, the mappers get preference in 
scheduling. However, incase of semijoin reductions, the reducer of semijoin 
branch should get priority before the mapper to which it feeds in.


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java aa2dfc7 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
 727f7bc 
  ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java 905431f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezWork.java 97f3300 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 38461d5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceWork.java ee784dc 
  ql/src/java/org/apache/hadoop/hive/ql/plan/TezEdgeProperty.java a3aa12f 


Diff: https://reviews.apache.org/r/57903/diff/1/


Testing
---


Thanks,

Deepak Jaiswal

Re: Backward incompatible changes

2017-03-23 Thread Ashutosh Chauhan

Cutting a branch should not slow down a 2.2 release. If any thing, this
should help in achieving stabilization faster for release since branch
won't get any new potentially destabilizing changes but only patches to fix
existing known issues.

On Thu, Mar 23, 2017 at 8:22 AM, Eugene Koifman 
wrote:

> +1 to make a release first
>
> On 3/22/17, 2:06 PM, "Sergey Shelukhin"  wrote:
>
> Hmm.. should we release these first, and then cut branch-2?
> Otherwise during the releases, the patches for 2.2/2.3 will need to go
> to
> 3 (4?) places (master, branch-2, branch-2.2, branch-2.3?).
> There’s no rush to cut the branch if everything in 2.2/2.3 has to go to
> 3.0 anyway.
>
> On 17/3/22, 13:53, "Pengcheng Xiong"  wrote:
>
> >I would like to work as the Release Manager if possible. As Owen
> points
> >out, he is working on 2.2 and I will work on 2.3. Thanks.
> >
> >On Wed, Mar 22, 2017 at 1:32 PM, Ashutosh Chauhan <
> hashut...@apache.org>
> >wrote:
> >
> >> Unless there is more feedback, I plan to cut branch-2 in a day or
> two
> >>from
> >> current master. As multiple people have suggested on this thread, we
> >>should
> >> do a 2.2 release soon. Currently there are 177 issues
> >>  >> 3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20cf%
> >> 5B12310320%5D%20%3D%202.2.0%20ORDER%20BY%20priority%20DESC>
> >> targeted for 2.2 release. We can use branch-2 to land these patches
> and
> >>for
> >> additional stabilization efforts. Any volunteer for Release Manager
> >>driving
> >> 2.2 release?
> >>
> >> Thanks,
> >> Ashutosh
> >>
> >> On Fri, Mar 10, 2017 at 4:23 PM, Ashutosh Chauhan <
> hashut...@apache.org>
> >> wrote:
> >>
> >> > I hear what you are saying. Lets begin with 3 concerns:
> >> >
> >> > - How will we keep the community motivated on fixing both master
> and
> >> > branch-2?
> >> > Until we do a stable release from master, stable releases can come
> >>only
> >> > from branch-2. If a contributor wants to see their fix reach to
> users
> >>on
> >> a
> >> > stable line quickly they would have to have a fix on branch-2.
> Also, a
> >> > release manager can pick whatever fixes she wants, so even if
> >>contributor
> >> > doesn't commit it on branch-2, a release manger who wants to do a
> >>release
> >> > containing a set of fixes thats always possible.
> >> >
> >> > - *Harder cherry-picks between master and branch-2*.
> >> > That is certainly possible. But hope is we want to keep branch-2
> >>stable,
> >> > so we don't backport large features which may run into this issue.
> >> Smaller
> >> > focussed bug fix backport should be possible.
> >> >
> >> >
> >> >- *Removal of MR2 on the master branch*.
> >> > This is something I personally would like to see. But exact
> timing of
> >>it
> >> > will be decided by community. I am certainly not saying that as
> soon
> >>as
> >> > branch-2 is created, lets remove MR2 on master.
> >> >
> >> > I would also say that in the end ASF is volunteer organization, we
> >>cant
> >> > force people to adopt one branch or another. Its upto the
> contributors
> >> what
> >> > jiras they work on and when and where they commit it.
> >> > By not creating a branch-2 only thing we can guarantee is that
> rate of
> >> > development on master to remain slow because we don't want to
> start
> >>doing
> >> > backward incompatible changes without explicitly acknowledging
> that.
> >> >
> >> > Thanks,
> >> > Ashutosh
> >> >
> >> > On Thu, Mar 9, 2017 at 12:01 PM, Sergio Pena
> >>
> >> > wrote:
> >> >
> >> >> Hey Ashutosh, thanks for soliciting feedback on this.
> >> >>
> >> >> I like the idea you're proposing; maintaining compatibility and
> at
> >>the
> >> >> same time adding newer features to
> >> >> Hive consumes a lot of development time and effort.
> >> >>
> >> >> However, I think some users and companies have just started to
> use
> >>Hive
> >> >> 2.x
> >> >> branch as their main major upgrade on Hive
> >> >> (possible due to waiting for stabilization and testing
> upgrades), but
> >> >> cutting this major branch that just has 1 year of life
> >> >> might make us look like we will forget about the quality of Hive
> 2.x
> >>as
> >> we
> >> >> did with branch-1.
> >> >>
> >> >> Hive 1.x latest version was 1.2, and its development stopped
> because
> >>new
> >> >> features on Hive 2.x
> >> >> Hive 2.x latest version is 2.1, and we want to create Hive 3.x
> >>because
> >> of
> >> >> newer features and incompatibilities.
> >> >> Will Hive 3.x have the same future after 3.1 is released?
> >> >>
> >> >> Wha

Re: Backward incompatible changes

2017-03-23 Thread Ashutosh Chauhan

So, I just pushed branch-2 from current tip.
Now, lets get 2.2 release out as soon as possible.

Thanks,
Ashutosh

On Thu, Mar 23, 2017 at 9:46 PM, Ashutosh Chauhan 
wrote:

> Cutting a branch should not slow down a 2.2 release. If any thing, this
> should help in achieving stabilization faster for release since branch
> won't get any new potentially destabilizing changes but only patches to fix
> existing known issues.
>
> On Thu, Mar 23, 2017 at 8:22 AM, Eugene Koifman 
> wrote:
>
>> +1 to make a release first
>>
>> On 3/22/17, 2:06 PM, "Sergey Shelukhin"  wrote:
>>
>> Hmm.. should we release these first, and then cut branch-2?
>> Otherwise during the releases, the patches for 2.2/2.3 will need to
>> go to
>> 3 (4?) places (master, branch-2, branch-2.2, branch-2.3?).
>> There’s no rush to cut the branch if everything in 2.2/2.3 has to go
>> to
>> 3.0 anyway.
>>
>> On 17/3/22, 13:53, "Pengcheng Xiong"  wrote:
>>
>> >I would like to work as the Release Manager if possible. As Owen
>> points
>> >out, he is working on 2.2 and I will work on 2.3. Thanks.
>> >
>> >On Wed, Mar 22, 2017 at 1:32 PM, Ashutosh Chauhan <
>> hashut...@apache.org>
>> >wrote:
>> >
>> >> Unless there is more feedback, I plan to cut branch-2 in a day or
>> two
>> >>from
>> >> current master. As multiple people have suggested on this thread,
>> we
>> >>should
>> >> do a 2.2 release soon. Currently there are 177 issues
>> >> > >> 3D%20HIVE%20AND%20resolution%20%3D%20Unresolved%20AND%20cf%
>> >> 5B12310320%5D%20%3D%202.2.0%20ORDER%20BY%20priority%20DESC>
>> >> targeted for 2.2 release. We can use branch-2 to land these
>> patches and
>> >>for
>> >> additional stabilization efforts. Any volunteer for Release Manager
>> >>driving
>> >> 2.2 release?
>> >>
>> >> Thanks,
>> >> Ashutosh
>> >>
>> >> On Fri, Mar 10, 2017 at 4:23 PM, Ashutosh Chauhan <
>> hashut...@apache.org>
>> >> wrote:
>> >>
>> >> > I hear what you are saying. Lets begin with 3 concerns:
>> >> >
>> >> > - How will we keep the community motivated on fixing both master
>> and
>> >> > branch-2?
>> >> > Until we do a stable release from master, stable releases can
>> come
>> >>only
>> >> > from branch-2. If a contributor wants to see their fix reach to
>> users
>> >>on
>> >> a
>> >> > stable line quickly they would have to have a fix on branch-2.
>> Also, a
>> >> > release manager can pick whatever fixes she wants, so even if
>> >>contributor
>> >> > doesn't commit it on branch-2, a release manger who wants to do a
>> >>release
>> >> > containing a set of fixes thats always possible.
>> >> >
>> >> > - *Harder cherry-picks between master and branch-2*.
>> >> > That is certainly possible. But hope is we want to keep branch-2
>> >>stable,
>> >> > so we don't backport large features which may run into this
>> issue.
>> >> Smaller
>> >> > focussed bug fix backport should be possible.
>> >> >
>> >> >
>> >> >- *Removal of MR2 on the master branch*.
>> >> > This is something I personally would like to see. But exact
>> timing of
>> >>it
>> >> > will be decided by community. I am certainly not saying that as
>> soon
>> >>as
>> >> > branch-2 is created, lets remove MR2 on master.
>> >> >
>> >> > I would also say that in the end ASF is volunteer organization,
>> we
>> >>cant
>> >> > force people to adopt one branch or another. Its upto the
>> contributors
>> >> what
>> >> > jiras they work on and when and where they commit it.
>> >> > By not creating a branch-2 only thing we can guarantee is that
>> rate of
>> >> > development on master to remain slow because we don't want to
>> start
>> >>doing
>> >> > backward incompatible changes without explicitly acknowledging
>> that.
>> >> >
>> >> > Thanks,
>> >> > Ashutosh
>> >> >
>> >> > On Thu, Mar 9, 2017 at 12:01 PM, Sergio Pena
>> >>
>> >> > wrote:
>> >> >
>> >> >> Hey Ashutosh, thanks for soliciting feedback on this.
>> >> >>
>> >> >> I like the idea you're proposing; maintaining compatibility and
>> at
>> >>the
>> >> >> same time adding newer features to
>> >> >> Hive consumes a lot of development time and effort.
>> >> >>
>> >> >> However, I think some users and companies have just started to
>> use
>> >>Hive
>> >> >> 2.x
>> >> >> branch as their main major upgrade on Hive
>> >> >> (possible due to waiting for stabilization and testing
>> upgrades), but
>> >> >> cutting this major branch that just has 1 year of life
>> >> >> might make us look like we will forget about the quality of
>> Hive 2.x
>> >>as
>> >> we
>> >> >> did with branch-1.
>> >> >>
>> >> >> Hive 1.x latest version was 1.2, and it

Re: Review Request 57772: HIVE-16242 Run BeeLine tests parallel

Re: Review Request 57728: HIVE-16231: Parquet timestamp may be stored differently since HIVE-12767

Re: Review Request 57772: HIVE-16242 Run BeeLine tests parallel

Re: Backward incompatible changes

Re: Review Request 57632: HIVE-16206: Provide wrapper classes for current metrics reporters to allow uniform instantiation through reflection

[jira] [Created] (HIVE-16286) Log canceled query id

[jira] [Created] (HIVE-16287) Alter table partition rename with location - moves partition back to hive warehouse

Re: Alter table partition rename with location - moves partition back to hive warehouse

Re: Review Request 57728: HIVE-16231: Parquet timestamp may be stored differently since HIVE-12767

Re: Review Request 57626: HIVE-16164: Provide mechanism for passing HMS notification ID between transactional and non-transactional listeners.

Review Request 57893: HIVE-16285: Servlet for dynamically configuring log levels

[jira] [Created] (HIVE-16288) Add blobstore tests for ORC and RCFILE file formats

[jira] [Created] (HIVE-16289) add hints for semijoin reduction

Re: Review Request 57893: HIVE-16285: Servlet for dynamically configuring log levels

Review Request 57903: HIVE-16282 Add faststart for semijoin reducer

Re: Backward incompatible changes

Re: Backward incompatible changes

17 matches

Site Navigation

Mail list logo

Footer information