Build failed in Jenkins: HIVE-TRUNK-JAVA8 #64

2015-04-28 Thread hiveqa
See 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/64/changes

Changes:

[Ashutosh Chauhan] HIVE-10451 :  PTF deserializer fails if values are not used 
in reducer (Ashutosh Chauhan via Gopal V)

[j.prasanth.j] HIVE-10370: Hive does not compile with -Phadoop-1 option 
(Prasanth Jayachandran reviewed by Vaibhav Gumashta)

[j.prasanth.j] HIVE-10431: HIVE-9555 broke hadoop-1 build (Sergey Shelukhin 
reviewed by Prasanth Jayachandran)

[j.prasanth.j] HIVE-10443: HIVE-9870 broke hadoop-1 build (Vaibhav Gumashta 
reviewed by Prasanth Jayachandran)

[Xuefu Zhang] HIVE-10372: Update parquet version to 1.6.0 (Ferdinand via Xuefu)

[thejas] HIVE-10447 : Beeline JDBC Driver to support 2 way SSL (Hari Sankar 
Sivarama Subramaniyan  via Thejas Nair)

[Ashutosh Chauhan] HIVE-10462 : CBO (Calcite Return Path): MapJoin and SMBJoin 
conversion not triggered (Jesus Camacho Rodriguez via Ashutosh Chauhan)

[Ashutosh Chauhan] HIVE-10490 : HBase Snapshot IF fails at run time with 
missing dependency of MetricsRegistry (Ashutosh Chauhan via Gunther Hagleitner)

[jdere] HIVE-10421: DROP TABLE with qualified table name ignores database name 
when checking partitions (Jason Dere via Thejas Nair)

[jdere] HIVE-10441: Fix confusing log statement in SessionState about 
hive.execution.engine setting (Jason Dere reviewed by Gunther Hagnleitner)

[thejas] HIVE-4625 : HS2 should not attempt to get delegation token from 
metastore if using embedded metastore (Hari Sankar Sivarama Subramaniyan via 
Thejas Nair)

[thejas] HIVE-10465 : whitelist restrictions don't get initialized in new copy 
of HiveConf (Thejas Nair, reviewed by Daniel Dai)

[j.prasanth.j] HIVE-10442: HIVE-10098 broke hadoop-1 build (Yongzhi Chen 
reviewed by Prasanth Jayachandran)

[khorgath] HIVE-10477 : Provide option to disable Spark tests (Hari 
Subramaniyan via Sushanth Sowmyan)

--
[...truncated 321 lines...]
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testBasic.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepNone.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.testFailingQFile.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/MockLocalCommandFactory.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testAlternativeTestJVM.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/context
A
ptest2/src/test/java/org/apache/hive/ptest/execution/context/TestCloudComputeService.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/context/TestCloudExecutionContextProvider.java
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestJIRAService.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testIsolatedFailsOnRsyncOne.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.testFailingUnitTest.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testShutdownBeforeExec.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestPhase.testExecInstancesWithFailure.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/ssh
A
ptest2/src/test/java/org/apache/hive/ptest/execution/ssh/TestSSHCommandExecutor.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/ssh/TestRSyncCommandExecutor.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepGit.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestPhase.testRsyncFromLocalToRemoteInstancesWithFailureUnknown.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/MockSSHCommandExecutor.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/MockRSyncCommandExecutor.java
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestPrepPhase.java
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.java
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestReportParser.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestReportingPhase.java
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testIsolatedFailsOnExec.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testParallelFailsOnRsync.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/TestPhase.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestLogDirectoryCleaner.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testPrepSvn.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.testPassingUnitTest.approved.txt
A

Re: Do we still support hadoop-1.2.x API (-Phadoop-1)?

2015-04-28 Thread Szehon Ho
Hadoop 2 has been GA for a little over a year, there is still a fairly
significant user base that uses hadoop-1 and would not be happy with this
change.  It should be removed at some point, but I'm not in favor of
removing in next release which would be too soon.

Perhaps we can declare it deprecated in some future release (perhaps 1.3),
then another release to formally remove it, as was done in HBase.  HBase
did the formal removal in a major release (1.0) which is a lot cleaner, not
sure if we have that luxury now that Hive 1.0 is forked.

Thanks,
Szehon

On Tue, Apr 28, 2015 at 5:03 PM, Prasanth Jayachandran 
pjayachand...@hortonworks.com wrote:

 I recently filed 5 issues (HIVE-10430, 10431, 10442, 10443, 10444) related
 to build breakage of hadoop-1. There could potentially be more breakage.
 Some patches were added to reduce the number of file systems calls to
 improve performance but with supporting hadoop-1 we cannot directly use
 such APIs. Also we are not enforcing hadoop-1 build checks in hive QA to
 make sure every commit comes out clean on hadoop-1 and hadoop-2. I think it
 will be good if we can focus only hadoop-2. Not only it will simplify
 development but also will reduce the shims layer.

 Thanks
 Prasanth


  On Apr 28, 2015, at 4:40 PM, Owen O'Malley omal...@apache.org wrote:
 
  It has been three years since Hadoop 2.0.0 was first released and I
 believe
  that the vast majority of users that want to run Hive 1.x have moved over
  to Hadoop 2.x already. It will dramatically simplify Hive development if
 we
  drop backwards compatibility with the old Hadoop 1.x line.
 
  Thanks,
Owen




Re: Do we still support hadoop-1.2.x API (-Phadoop-1)?

2015-04-28 Thread Prasanth Jayachandran
I recently filed 5 issues (HIVE-10430, 10431, 10442, 10443, 10444) related to 
build breakage of hadoop-1. There could potentially be more breakage. Some 
patches were added to reduce the number of file systems calls to improve 
performance but with supporting hadoop-1 we cannot directly use such APIs. Also 
we are not enforcing hadoop-1 build checks in hive QA to make sure every commit 
comes out clean on hadoop-1 and hadoop-2. I think it will be good if we can 
focus only hadoop-2. Not only it will simplify development but also will reduce 
the shims layer.

Thanks
Prasanth


 On Apr 28, 2015, at 4:40 PM, Owen O'Malley omal...@apache.org wrote:
 
 It has been three years since Hadoop 2.0.0 was first released and I believe
 that the vast majority of users that want to run Hive 1.x have moved over
 to Hadoop 2.x already. It will dramatically simplify Hive development if we
 drop backwards compatibility with the old Hadoop 1.x line.
 
 Thanks,
   Owen



Review Request 33650: HIVE-10516:Measure Hive CLI's performance difference before and after implementation is switched

2015-04-28 Thread cheng xu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33650/
---

Review request for hive and Xuefu Zhang.


Repository: hive-git


Description
---

Add microbenchmark for the current cli tool


Diffs
-

  itests/hive-jmh/pom.xml f3764c7 
  itests/hive-jmh/src/main/java/org/apache/hive/benchmark/cli/CliBench.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/33650/diff/


Testing
---

Test locally


Thanks,

cheng xu



Re: Do we still support hadoop-1.2.x API (-Phadoop-1)?

2015-04-28 Thread Owen O'Malley
It has been three years since Hadoop 2.0.0 was first released and I believe
that the vast majority of users that want to run Hive 1.x have moved over
to Hadoop 2.x already. It will dramatically simplify Hive development if we
drop backwards compatibility with the old Hadoop 1.x line.

Thanks,
   Owen


[jira] [Created] (HIVE-10526) CBO (Calcite Return Path): HiveCost epsilon comparison should take row count in to account

2015-04-28 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-10526:
-

 Summary: CBO (Calcite Return Path): HiveCost epsilon comparison 
should take row count in to account
 Key: HIVE-10526
 URL: https://issues.apache.org/jira/browse/HIVE-10526
 Project: Hive
  Issue Type: Sub-task
Reporter: Laljo John Pullokkaran
Assignee: Laljo John Pullokkaran






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Do we still support hadoop-1.2.x API (-Phadoop-1)?

2015-04-28 Thread Lefty Leverenz
This thread needs [DISCUSS] in the subject.

-- Lefty

On Tue, Apr 28, 2015 at 10:58 PM, Owen O'Malley omal...@apache.org wrote:

 On Tue, Apr 28, 2015 at 5:25 PM, Szehon Ho sze...@cloudera.com wrote:

  Hadoop 2 has been GA for a little over a year, there is still a fairly
  significant user base that uses hadoop-1 and would not be happy with this
  change.


 The question isn't whether there are people running Hadoop 1.x, it is
 whether those people are likely to install a new version of Hive running on
 their old Hadoop cluster. As a point of reference, CDH 4 shipped Hadoop 2.0
 and Hive 0.10 and HDP 2.0 shipped Hadoop 2.0 and Hive 0.12.


  Perhaps we can declare it deprecated in some future release (perhaps
 1.3),
  then another release to formally remove it, as was done in HBase.


 Are you interested in managing a Hadoop 1.x compatible version of Hive?
 Maybe we should call the new release Hive 2.0 and enable you to maintain
 the Hive 1.x branch with backwards compatibility with Hadoop 1.x. That
 would enable the rest of the Hive community to move forward and take
 advantage of the powerful new features in Hadoop 2.x.

 .. Owen



[jira] [Created] (HIVE-10528) Hiveserver2 in HTTP mode is not applying auth_to_local rules

2015-04-28 Thread Abdelrahman Shettia (JIRA)
Abdelrahman Shettia created HIVE-10528:
--

 Summary: Hiveserver2 in HTTP mode is not applying auth_to_local 
rules
 Key: HIVE-10528
 URL: https://issues.apache.org/jira/browse/HIVE-10528
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 0.14.0
 Environment: Centos 6
Reporter: Abdelrahman Shettia


PROBLEM: Authenticating to HS2 in HTTP mode with Kerberos, auth_to_local 
mappings do not get applied.  Because of this various permissions checks which 
rely on the local cluster name for a user are going to fail.


STEPS TO REPRODUCE:

1.  Create  kerberos cluster  and HS2 in HTTP mode
2.  Create a new user, test, along with a kerberos principal for this user
3.  Create a separate principal, mapped-test
4.  Create an auth_to_local rule to make sure that mapped-test is mapped to test
5.  As the test user, connect to HS2 with beeline and create a simple table:

{code}
CREATE TABLE permtest (field1 int);
{code}

There is no need to load anything into this table.

6.  Establish that it works as the test user:

{code}
show create table permtest;
{code}

7.  Drop the test identity and become mapped-test
8.  Re-connect to HS2 with beeline, re-run the above command:
{code}
show create table permtest;
{code}

You will find that when this is done in HTTP mode, you will get an HDFS error 
(because of StorageBasedAuthorization doing a HDFS permissions check) and the 
user will be mapped-test and NOT test as it should be.

ANALYSIS:  This appears to be HTTP specific and the problem seems to come in 
{{ThriftHttpServlet$HttpKerberosServerAction.getPrincipalWithoutRealmAndHost()}}:

{code}
  try {
fullKerberosName = 
ShimLoader.getHadoopShims().getKerberosNameShim(fullPrincipal);
  } catch (IOException e) {
throw new HttpAuthenticationException(e);
  }
  return fullKerberosName.getServiceName();
{code}

getServiceName applies no auth_to_local rules.  Seems like maybe this should be 
getShortName()?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Do we still support hadoop-1.2.x API (-Phadoop-1)?

2015-04-28 Thread Owen O'Malley
On Tue, Apr 28, 2015 at 5:25 PM, Szehon Ho sze...@cloudera.com wrote:

 Hadoop 2 has been GA for a little over a year, there is still a fairly
 significant user base that uses hadoop-1 and would not be happy with this
 change.


The question isn't whether there are people running Hadoop 1.x, it is
whether those people are likely to install a new version of Hive running on
their old Hadoop cluster. As a point of reference, CDH 4 shipped Hadoop 2.0
and Hive 0.10 and HDP 2.0 shipped Hadoop 2.0 and Hive 0.12.


 Perhaps we can declare it deprecated in some future release (perhaps 1.3),
 then another release to formally remove it, as was done in HBase.


Are you interested in managing a Hadoop 1.x compatible version of Hive?
Maybe we should call the new release Hive 2.0 and enable you to maintain
the Hive 1.x branch with backwards compatibility with Hadoop 1.x. That
would enable the rest of the Hive community to move forward and take
advantage of the powerful new features in Hadoop 2.x.

.. Owen


[jira] [Created] (HIVE-10527) NPE in SparkUtilities::isDedicatedCluster

2015-04-28 Thread Rui Li (JIRA)
Rui Li created HIVE-10527:
-

 Summary: NPE in SparkUtilities::isDedicatedCluster
 Key: HIVE-10527
 URL: https://issues.apache.org/jira/browse/HIVE-10527
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Rui Li
Assignee: Rui Li


We should add {{spark.master}} to HiveConf when it doesn't exist.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HIVE-10529) Cleanup tezcontext reference in org.apache.hadoop.hive.ql.exec.tez.HashTableLoader

2015-04-28 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created HIVE-10529:
---

 Summary: Cleanup tezcontext reference in 
org.apache.hadoop.hive.ql.exec.tez.HashTableLoader
 Key: HIVE-10529
 URL: https://issues.apache.org/jira/browse/HIVE-10529
 Project: Hive
  Issue Type: Bug
Reporter: Rajesh Balamohan






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Review Request 33633: HIVE-10499 Ensure Session/ZooKeeperClient instances are closed

2015-04-28 Thread Szehon Ho

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33633/#review81878
---

Ship it!


Ship It!

- Szehon Ho


On April 28, 2015, 5:56 p.m., Jimmy Xiang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/33633/
 ---
 
 (Updated April 28, 2015, 5:56 p.m.)
 
 
 Review request for hive and Szehon Ho.
 
 
 Bugs: HIVE-10499
 https://issues.apache.org/jira/browse/HIVE-10499
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 Make sure Session/SessionState/ZooKeeperHiveClient instances are closed if 
 some exception is thrown in a wrong place, so as to avoid some possible 
 resource leakage.
 
 
 Diffs
 -
 
   jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientHelper.java 496c820 
   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
 f14b974 
   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
 e02997a 
   service/src/java/org/apache/hive/service/server/HiveServer2.java 222cb45 
 
 Diff: https://reviews.apache.org/r/33633/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Jimmy Xiang
 




Re: Review Request 33633: HIVE-10499 Ensure Session/ZooKeeperClient instances are closed

2015-04-28 Thread Jimmy Xiang


 On April 28, 2015, 6:03 p.m., Szehon Ho wrote:
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java, 
  line 568
  https://reviews.apache.org/r/33633/diff/1/?file=944227#file944227line568
 
  Is it necessary to call close twice?  Can we consolidate the two 
  finally blocks?

The first one is in a block and may not get a chance to be executed. After the 
first one closes it, it is set to null. The second one is to make sure it is 
really closed. The first one throws an Exception. The second one just logs a 
warning. I thought about consolidating them but preferred not to change the 
existing behavior.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33633/#review81845
---


On April 28, 2015, 5:56 p.m., Jimmy Xiang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/33633/
 ---
 
 (Updated April 28, 2015, 5:56 p.m.)
 
 
 Review request for hive and Szehon Ho.
 
 
 Bugs: HIVE-10499
 https://issues.apache.org/jira/browse/HIVE-10499
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 Make sure Session/SessionState/ZooKeeperHiveClient instances are closed if 
 some exception is thrown in a wrong place, so as to avoid some possible 
 resource leakage.
 
 
 Diffs
 -
 
   jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientHelper.java 496c820 
   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
 f14b974 
   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
 e02997a 
   service/src/java/org/apache/hive/service/server/HiveServer2.java 222cb45 
 
 Diff: https://reviews.apache.org/r/33633/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Jimmy Xiang
 




[jira] [Created] (HIVE-10525) loading data into list bucketing table when null in skew column

2015-04-28 Thread Gabriel C Balan (JIRA)
Gabriel C Balan created HIVE-10525:
--

 Summary: loading data into list bucketing table when null in skew 
column
 Key: HIVE-10525
 URL: https://issues.apache.org/jira/browse/HIVE-10525
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 1.1.0
 Environment: linux
Reporter: Gabriel C Balan
Priority: Minor


I'm trying to load data into a list bucketing table.
The insert statement fails when there are nulls going into the skew column.
If this is the expected behavior, there is no mention of this restriction in 
the doc.

 more *null.csv
::
has-null.csv
::
1
2
\N
3
::
no-null.csv
::
1
2
3


set hive.mapred.supports.subdirectories=true;
set hive.optimize.listbucketing=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;

create table src_with_null (x int);
load data local inpath 'has-null.csv' overwrite into table src_with_null;

create table src_no_null (x int);
load data local inpath 'no-null.csv' overwrite into table src_no_null;

create table lb (x int) partitioned by (p string) 
skewed by ( x ) on (1) STORED AS DIRECTORIES
stored as rcfile;

insert overwrite table lb partition (p = 'foo') select * from src_with_null;
--fails

insert overwrite table lb partition (p = 'foo') select * from src_no_null;
--succeeds

I see this in ${hive.log.dir}/hive.log

2015-04-28 13:43:47,646 WARN  [Thread-82]: mapred.LocalJobRunner 
(LocalJobRunner.java:run(560)) - job_local402607316_0001
java.lang.Exception: java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row {x:null}
at 
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at 
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row {x:null}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at 
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error 
while processing row {x:null}
at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:507)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)
... 10 more
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.FileSinkOperator.generateListBucketingDirName(FileSinkOperator.java:833)
at 
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:615)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at 
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at 
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:497)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HIVE-10524) Add utility method ExprNodeDescUtils.forwardTrack()

2015-04-28 Thread Jason Dere (JIRA)
Jason Dere created HIVE-10524:
-

 Summary: Add utility method ExprNodeDescUtils.forwardTrack()
 Key: HIVE-10524
 URL: https://issues.apache.org/jira/browse/HIVE-10524
 Project: Hive
  Issue Type: Bug
Reporter: Jason Dere
Assignee: Jason Dere


ExprNodeDescUtils has a method backtrack(), which is able to take an 
ExprNodeDesc from an operator and convert it to an equivalent expression based 
on the columns of a parent operator. Adding a forwardTrack() method to do 
something similar, but for a child operator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Review Request 28372: Address review comments

2015-04-28 Thread Ryan Blue

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28372/#review81884
---


Looking good to me. I found a few things to fix, and I'm also wondering where 
the qtest outputs are.


ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ParquetToHiveSchemaConverter.java
https://reviews.apache.org/r/28372/#comment132397

Sergio discovered that MAP_KEY_VALUE was incorrectly used in place of MAP. 
So instead of throwing UnsupportedOperationException, MAP_KEY_VALUE should be 
used as a synonym for MAP. That way we can read old data.



ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java
https://reviews.apache.org/r/28372/#comment132398

Shouldn't you also pass the exception so that the stack trace is printed?



ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetToHiveSchemaConverter.java
https://reviews.apache.org/r/28372/#comment132399

Minor: Would be good to use the Types API instead of constructors.



ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetToHiveSchemaConverter.java
https://reviews.apache.org/r/28372/#comment132401

I think the maps and lists should be constructed using the current best 
practice, ConversionPatterns, instead of by hand. That way we're testing what 
is actually going to be in data files.


- Ryan Blue


On April 16, 2015, 8:01 p.m., Ashish Singh wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28372/
 ---
 
 (Updated April 16, 2015, 8:01 p.m.)
 
 
 Review request for hive.
 
 
 Bugs: HIVE-8950
 https://issues.apache.org/jira/browse/HIVE-8950
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 HIVE-8950: Add support in ParquetHiveSerde to create table schema from a 
 parquet file
 
 
 Diffs
 -
 
   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
 e138800e6dadd6fe76345f21eb76c906165c438d 
   
 hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 
 53d913435cd43c96f044cf8668461fc686817ef4 
   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
 3f267ff0eb20560c36a19b74353f9d6749c8b333 
   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
 907ad2b8d7bae9b13eb4d9605ff2a3fe60c03ee8 
   
 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ParquetSchemaReader.java
  PRE-CREATION 
   
 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ParquetToHiveSchemaConverter.java
  PRE-CREATION 
   
 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java 
 7fd5e9612d4e3c9bf3b816bc48dbdbe59fb8a5a8 
   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
 a029f10c116fd46070b2d41790043f0a7001390f 
   
 ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetToHiveSchemaConverter.java
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_array_of_multi_field_struct_gen_schema.q
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_array_of_optional_elements_gen_schema.q
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_array_of_required_elements_gen_schema.q
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_array_of_single_field_struct_gen_schema.q
  PRE-CREATION 
   ql/src/test/queries/clientpositive/parquet_array_of_structs_gen_schema.q 
 PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_array_of_structs_gen_schema_ext.q 
 PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_array_of_unannotated_groups_gen_schema.q
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_array_of_unannotated_primitives_gen_schema.q
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_avro_array_of_primitives_gen_schema.q
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_avro_array_of_single_field_struct_gen_schema.q
  PRE-CREATION 
   ql/src/test/queries/clientpositive/parquet_decimal_gen_schema.q 
 PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_thrift_array_of_primitives_gen_schema.q
  PRE-CREATION 
   
 ql/src/test/queries/clientpositive/parquet_thrift_array_of_single_field_struct_gen_schema.q
  PRE-CREATION 
   ql/src/test/results/clientpositive/create_view_partitioned.q.out 
 ebf9a6bc4f2321d7f539b7a445b3f279e3285b8a 
   
 ql/src/test/results/clientpositive/parquet_array_of_multi_field_struct_gen_schema.q.out
  PRE-CREATION 
   
 ql/src/test/results/clientpositive/parquet_array_of_optional_elements_gen_schema.q.out
  PRE-CREATION 
   
 ql/src/test/results/clientpositive/parquet_array_of_required_elements_gen_schema.q.out
  PRE-CREATION 
   
 ql/src/test/results/clientpositive/parquet_array_of_single_field_struct_gen_schema.q.out
  PRE-CREATION 
   
 

Re: Failing minimr tests

2015-04-28 Thread Hari Subramaniyan
I have  a patch for this, will be nice if someone can review and verify it.

Thanks
Hari

From: Szehon Ho sze...@cloudera.com
Sent: Monday, April 27, 2015 6:25 PM
To: dev@hive.apache.org
Subject: Re: Failing minimr tests

I got around to create a JIRA for this:  HIVE-10514
https://issues.apache.org/jira/browse/HIVE-10514, be great if somebody
with some cycles could take a look.

Thanks
Szehon

On Tue, Apr 14, 2015 at 8:43 AM, Thejas Nair thejas.n...@gmail.com wrote:

 code too large seems to refer to a method that is larger than 64kb -

 http://stackoverflow.com/questions/2407912/code-too-large-compilation-error-in-java
 I wonder if some auto generated method is getting too large.


 On Mon, Apr 13, 2015 at 11:20 PM, Szehon Ho sze...@cloudera.com wrote:
  FYI, was just taking a brief look at the recent failing minimr tests and
  found this exception when the build framework generates the tests:
 
  [ERROR] Failed to execute goal
  org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
  (default-testCompile) on project hive-it-qfile: Compilation failure
  [ERROR]
 /home/hiveptest/54.147.211.107-hiveptest-0/apache-svn-trunk-source/itests/qtest/target/generated-test-sources/java/org/apache/hadoop/hive/cli/TestCliDriver.java:[100,22]
  code too large
 
 
  I believe this is the command that it's running :
 
  timeout 2h mvn -B test
  -Dmaven.repo.local=/home/hiveptest//54.166.80.165-hiveptest-0/maven
  -Phadoop-2 -Phadoop-2 -Dtest=TestMinimrCliDriver
 
 -Dminimr.query.files=infer_bucket_sort_map_operators.q,join1.q,bucketmapjoin7.q,udf_using.q
 
 
  Thanks,
 
  Szehon



[jira] [Created] (HIVE-10520) LLAP: Must reset small table result columns for Native Vectorization of Map Join

2015-04-28 Thread Matt McCline (JIRA)
Matt McCline created HIVE-10520:
---

 Summary: LLAP: Must reset small table result columns for Native 
Vectorization of Map Join
 Key: HIVE-10520
 URL: https://issues.apache.org/jira/browse/HIVE-10520
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Matt McCline
Assignee: Matt McCline
Priority: Blocker






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Do we still support hadoop-1.2.x API (-Phadoop-1)?

2015-04-28 Thread Alexander Pivovarov
Hi Everyone

I tried to compile the latest hive with hadoop-1 profile.
It failed because TestLazySimpleFast (164) uses Text.copyBytes() which is
hadoop-2.x API

So, which hadoop API should we use in hive? old hadoop-1.x or new
hadoop-2.x?

Alex


Re: Review Request 33633: HIVE-10499 Ensure Session/ZooKeeperClient instances are closed

2015-04-28 Thread Szehon Ho

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33633/#review81845
---



service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
https://reviews.apache.org/r/33633/#comment132336

Is it necessary to call close twice?  Can we consolidate the two finally 
blocks?


- Szehon Ho


On April 28, 2015, 5:56 p.m., Jimmy Xiang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/33633/
 ---
 
 (Updated April 28, 2015, 5:56 p.m.)
 
 
 Review request for hive and Szehon Ho.
 
 
 Bugs: HIVE-10499
 https://issues.apache.org/jira/browse/HIVE-10499
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 Make sure Session/SessionState/ZooKeeperHiveClient instances are closed if 
 some exception is thrown in a wrong place, so as to avoid some possible 
 resource leakage.
 
 
 Diffs
 -
 
   jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientHelper.java 496c820 
   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
 f14b974 
   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
 e02997a 
   service/src/java/org/apache/hive/service/server/HiveServer2.java 222cb45 
 
 Diff: https://reviews.apache.org/r/33633/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Jimmy Xiang
 




Review Request 33633: HIVE-10499 Ensure Session/ZooKeeperClient instances are closed

2015-04-28 Thread Jimmy Xiang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33633/
---

Review request for hive and Szehon Ho.


Bugs: HIVE-10499
https://issues.apache.org/jira/browse/HIVE-10499


Repository: hive-git


Description
---

Make sure Session/SessionState/ZooKeeperHiveClient instances are closed if some 
exception is thrown in a wrong place, so as to avoid some possible resource 
leakage.


Diffs
-

  jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientHelper.java 496c820 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
f14b974 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
e02997a 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 222cb45 

Diff: https://reviews.apache.org/r/33633/diff/


Testing
---


Thanks,

Jimmy Xiang



Re: Do we still support hadoop-1.2.x API (-Phadoop-1)?

2015-04-28 Thread Ashutosh Chauhan
I think its time to discuss about dropping support for Hadoop-1 line. What
do folks think about Hive-1.2 being last release supporting Hadoop-1 line?

Thanks,
Ashutosh

On Tue, Apr 28, 2015 at 11:02 AM, Alexander Pivovarov apivova...@gmail.com
wrote:

 Hi Everyone

 I tried to compile the latest hive with hadoop-1 profile.
 It failed because TestLazySimpleFast (164) uses Text.copyBytes() which is
 hadoop-2.x API

 So, which hadoop API should we use in hive? old hadoop-1.x or new
 hadoop-2.x?

 Alex



[jira] [Created] (HIVE-10521) TxnHandler.timeOutTxns only times out some of the expired transactions

2015-04-28 Thread Alan Gates (JIRA)
Alan Gates created HIVE-10521:
-

 Summary: TxnHandler.timeOutTxns only times out some of the expired 
transactions
 Key: HIVE-10521
 URL: https://issues.apache.org/jira/browse/HIVE-10521
 Project: Hive
  Issue Type: Bug
  Components: Transactions
Affects Versions: 1.0.0, 0.14.0, 1.1.0
Reporter: Alan Gates
Assignee: Alan Gates


{code}
  for (int i = 0; i  20  rs.next(); i++) deadTxns.add(rs.getLong(1));
  // We don't care whether all of the transactions get deleted or not,
  // if some didn't it most likely means someone else deleted them in the 
interum
  if (deadTxns.size()  0) abortTxns(dbConn, deadTxns);
{code}
While it makes sense to limit the number of transactions aborted in one pass 
(since this get's translated to an IN clause) we should still make sure all are 
timed out.  Also, 20 seems pretty small as a batch size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Review Request 28109: HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe

2015-04-28 Thread Vaibhav Gumashta

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28109/
---

(Updated April 28, 2015, 6:40 p.m.)


Review request for hive and Thejas Nair.


Bugs: HIVE-8890
https://issues.apache.org/jira/browse/HIVE-8890


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-8890


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java c9ee423 
  pom.xml 9a1dae9 
  service/pom.xml 421bb9b 
  
service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 
ca1eae6 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 222cb45 

Diff: https://reviews.apache.org/r/28109/diff/


Testing
---


Thanks,

Vaibhav Gumashta



Review Request 33637: HIVE-10519 Move TestGenericUDF classes to udf.generic package

2015-04-28 Thread Alexander Pivovarov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33637/
---

Review request for hive and Jason Dere.


Bugs: HIVE-10519
https://issues.apache.org/jira/browse/HIVE-10519


Repository: hive-git


Description
---

HIVE-10519 Move TestGenericUDF classes to udf.generic package


Diffs
-

  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDate.java 
0d40ff7cb3c47ed4c2a8d2e88d9d8c32eb87d09d 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDateAdd.java 
0f3ecd18ae04ca9e44f0d55400fbacd3a879f9fc 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDateDiff.java 
849e70c17a1e2c839bf857ca22b5b61cda6de9e1 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDateSub.java 
ae1b3db5568f86c26e281c3cedb42821ebc5b450 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFUtils.java 
7399a7cd964913a1ae7f18b85cb050bba56acfe6 

Diff: https://reviews.apache.org/r/33637/diff/


Testing
---


Thanks,

Alexander Pivovarov



Re: Review Request 33637: HIVE-10519 Move TestGenericUDF classes to udf.generic package

2015-04-28 Thread Alexander Pivovarov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33637/
---

(Updated April 28, 2015, 6:59 p.m.)


Review request for hive and Jason Dere.


Changes
---

Organize Imports


Bugs: HIVE-10519
https://issues.apache.org/jira/browse/HIVE-10519


Repository: hive-git


Description
---

HIVE-10519 Move TestGenericUDF classes to udf.generic package


Diffs (updated)
-

  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDate.java 
0d40ff7cb3c47ed4c2a8d2e88d9d8c32eb87d09d 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDateAdd.java 
0f3ecd18ae04ca9e44f0d55400fbacd3a879f9fc 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDateDiff.java 
849e70c17a1e2c839bf857ca22b5b61cda6de9e1 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFDateSub.java 
ae1b3db5568f86c26e281c3cedb42821ebc5b450 
  ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFUtils.java 
7399a7cd964913a1ae7f18b85cb050bba56acfe6 

Diff: https://reviews.apache.org/r/33637/diff/


Testing
---


Thanks,

Alexander Pivovarov



[jira] [Created] (HIVE-10522) CBO (Calcite Return Path): fix the wrong needed column names when TS is created

2015-04-28 Thread Pengcheng Xiong (JIRA)
Pengcheng Xiong created HIVE-10522:
--

 Summary: CBO (Calcite Return Path): fix the wrong needed column 
names when TS is created
 Key: HIVE-10522
 URL: https://issues.apache.org/jira/browse/HIVE-10522
 Project: Hive
  Issue Type: Sub-task
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong
Priority: Critical






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Review Request 28109: HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe

2015-04-28 Thread Vaibhav Gumashta


 On Jan. 30, 2015, 8:26 p.m., Thejas Nair wrote:
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1337
  https://reviews.apache.org/r/28109/diff/3/?file=836237#file836237line1337
 
  how about using seconds instead of milliseconds, most people would not 
  care about the millisecond part.

HIVE_ZOOKEEPER_CONNECTION_BASESLEEPTIME uses ms and it is reasonable to expect 
it to be configured  1s. I'm thinking it's better to keep consistent units. 
What do you think?


- Vaibhav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28109/#review70303
---


On April 28, 2015, 6:40 p.m., Vaibhav Gumashta wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28109/
 ---
 
 (Updated April 28, 2015, 6:40 p.m.)
 
 
 Review request for hive and Thejas Nair.
 
 
 Bugs: HIVE-8890
 https://issues.apache.org/jira/browse/HIVE-8890
 
 
 Repository: hive-git
 
 
 Description
 ---
 
 https://issues.apache.org/jira/browse/HIVE-8890
 
 
 Diffs
 -
 
   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java c9ee423 
   pom.xml 9a1dae9 
   service/pom.xml 421bb9b 
   
 service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java
  ca1eae6 
   service/src/java/org/apache/hive/service/server/HiveServer2.java 222cb45 
 
 Diff: https://reviews.apache.org/r/28109/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Vaibhav Gumashta
 




Review Request 33618: HIVE-10518 Replace US_ASCII charset with US-ASCII in encode and decode description

2015-04-28 Thread Alexander Pivovarov

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33618/
---

Review request for hive and Jason Dere.


Bugs: HIVE-10518
https://issues.apache.org/jira/browse/HIVE-10518


Repository: hive-git


Description
---

HIVE-10518 Replace US_ASCII charset with US-ASCII in encode and decode 
description


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDecode.java 
f4366c7975c09d915f98d8e04c1039802cabdefa 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEncode.java 
64f8a14f894635f49e32b095dc0974f9d5500e38 
  ql/src/test/results/clientpositive/udf_decode.q.out 
97f0acf96dfe6e41b6841a65bdd33c7efe955e42 

Diff: https://reviews.apache.org/r/33618/diff/


Testing
---


Thanks,

Alexander Pivovarov



[jira] [Created] (HIVE-10519) Move TestGenericUDF classes to udf.generic package

2015-04-28 Thread Alexander Pivovarov (JIRA)
Alexander Pivovarov created HIVE-10519:
--

 Summary: Move TestGenericUDF classes to udf.generic package
 Key: HIVE-10519
 URL: https://issues.apache.org/jira/browse/HIVE-10519
 Project: Hive
  Issue Type: Bug
  Components: UDF
Reporter: Alexander Pivovarov
Assignee: Alexander Pivovarov
Priority: Trivial


5 TestGenericUDF classes are located in udf package instead of udf.generic
{code}
TestGenericUDFDate.java
TestGenericUDFDateAdd.java
TestGenericUDFDateDiff.java
TestGenericUDFDateSub.java
TestGenericUDFUtils.java
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HIVE-10523) Hive HCatalog Core 1.2.0 can not be built with hadoop-1 profile

2015-04-28 Thread Alexander Pivovarov (JIRA)
Alexander Pivovarov created HIVE-10523:
--

 Summary: Hive HCatalog Core 1.2.0 can not be built with hadoop-1 
profile
 Key: HIVE-10523
 URL: https://issues.apache.org/jira/browse/HIVE-10523
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Alexander Pivovarov


I tried to built HCatalog with hadoop-1 profile and got the following error
{code}
$ git status
On branch branch-1.2
Your branch is up-to-date with 'origin/branch-1.2'

$ mvn clean install -DskipTests -Phadoop-1

[ERROR] COMPILATION ERROR : 
[INFO] -
[ERROR] 
/workhive/hive/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java:[515,19]
 cannot find symbol
  symbol:   method isFile()
  location: variable fileStatus of type org.apache.hadoop.fs.FileStatus
[ERROR] 
/workhive/hive/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java:[545,26]
 cannot find symbol
  symbol:   method isDirectory()
  location: variable fileStatus of type org.apache.hadoop.fs.FileStatus
[INFO] 2 errors 
[INFO] -
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Hive ... SUCCESS [  3.181 s]
[INFO] Hive Shims Common .. SUCCESS [  4.292 s]
[INFO] Hive Shims 0.20S ... SUCCESS [  1.035 s]
[INFO] Hive Shims 0.23  SUCCESS [  5.692 s]
[INFO] Hive Shims Scheduler ... SUCCESS [  1.681 s]
[INFO] Hive Shims . SUCCESS [  1.302 s]
[INFO] Hive Common  SUCCESS [  4.787 s]
[INFO] Hive Serde . SUCCESS [  5.501 s]
[INFO] Hive Metastore . SUCCESS [ 15.634 s]
[INFO] Hive Ant Utilities . SUCCESS [  0.695 s]
[INFO] Spark Remote Client  SUCCESS [  9.376 s]
[INFO] Hive Query Language  SUCCESS [01:19 min]
[INFO] Hive Service ... SUCCESS [  5.310 s]
[INFO] Hive Accumulo Handler .. SUCCESS [  2.462 s]
[INFO] Hive JDBC .. SUCCESS [  8.817 s]
[INFO] Hive Beeline ... SUCCESS [  1.636 s]
[INFO] Hive CLI ... SUCCESS [  4.843 s]
[INFO] Hive Contrib ... SUCCESS [  1.501 s]
[INFO] Hive HBase Handler . SUCCESS [ 11.925 s]
[INFO] Hive HCatalog .. SUCCESS [  0.265 s]
[INFO] Hive HCatalog Core . FAILURE [  1.003 s]
[INFO] Hive HCatalog Pig Adapter .. SKIPPED
[INFO] Hive HCatalog Server Extensions  SKIPPED
[INFO] Hive HCatalog Webhcat Java Client .. SKIPPED
[INFO] Hive HCatalog Webhcat .. SKIPPED
[INFO] Hive HCatalog Streaming  SKIPPED
[INFO] Hive HWI ... SKIPPED
[INFO] Hive ODBC .. SKIPPED
[INFO] Hive Shims Aggregator .. SKIPPED
[INFO] Hive TestUtils . SKIPPED
[INFO] Hive Packaging . SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 02:51 min
[INFO] Finished at: 2015-04-28T12:20:09-07:00
[INFO] Final Memory: 179M/649M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on 
project hive-hcatalog-core: Compilation failure: Compilation failure:
[ERROR] 
/workhive/hive/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java:[515,19]
 cannot find symbol
[ERROR] symbol:   method isFile()
[ERROR] location: variable fileStatus of type org.apache.hadoop.fs.FileStatus
[ERROR] 
/workhive/hive/hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java:[545,26]
 cannot find symbol
[ERROR] symbol:   method isDirectory()
[ERROR] location: variable fileStatus of type org.apache.hadoop.fs.FileStatus
[ERROR] - [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.

Re: Preparation for Hive-1.2 release

2015-04-28 Thread Sushanth Sowmyan
Hi Folks,

As per the timeline set on
https://cwiki.apache.org/confluence/display/Hive/Hive+1.2+Release+Status
, the time for requests for inclusion of any new features to
branch-1.2 is now closed. Any JIRAS already marked in the lists
previously will be honoured and considered release blockers, but no
new feature JIRAs will be included.

The list is still open for any new bugfix jira to be added to the
list. I will start getting RC0 ready now. The purpose of RC0 is not
for it to be a true release-candidate, but simply to go through the
process of creating a release, verifying instructions set to create a
release, so that we can discover any hiccups along the way. Once that
is done, the next deadline of note for the community is 15:01,
Thursday, 30th Apr 2015 PDT, which is the deadline for requesting any
bugfix jiras for inclusion.

After this deadline, the list will go into further lockdown, and allow
inclusion of only jiras already in the currently accepted lists, and
either critical bugfixes which are product breakages without
workarounds, or bugs discovered during testing of the RCs.

Thanks,
-Sushanth



On Mon, Apr 27, 2015 at 4:51 PM, Sushanth Sowmyan khorg...@gmail.com wrote:
 HIVE-10509 and HIVE-10510 have been created with the individual
 patches that upgraded master and branch-1.2 respectively, and marked
 as resolved.

 I'll also add it in to the HowToRelease doc to ensure that from the
 next release onward, we create a jira for this.

 Also, friendly reminder for folks :  If you have any other jira you
 want included for 1.2, please go and add it to
 https://cwiki.apache.org/confluence/display/Hive/Hive+1.2+Release+Status

 Tonight midnight will be the deadline for feature include requests,
 and this list will go on lockdown for any further requests for feature
 inclusions. Bugfixes will still be okay to request for inclusion.



 On Mon, Apr 27, 2015 at 4:41 PM, Sushanth Sowmyan khorg...@gmail.com wrote:
 Also, Agreed with Sergio/Szehon, we should have a jira that's marked
 as committed with a patch attached for the version bump patch.
 Creating one.

 On Mon, Apr 27, 2015 at 4:39 PM, Sushanth Sowmyan khorg...@gmail.com wrote:
 Vinod,

 If 2.7.0 is not-yet-ready, then we should not be bumping 1.2 to
 2.7.0. But, we should be okay to bump master(1.3) up to 2.7.0 so that
 we start general-purpose testing against that.

 Please go ahead and create a jira for that, and we should be able to
 bump up trunk to go against that.

 Thanks,
 -Sushanth