[jira] [Created] (HIVE-10957) QueryPlan's start time is incorrect in certain cases

[Gunther Hagleitner (JIRA)]

Gunther Hagleitner created HIVE-10957:
-

 Summary: QueryPlan's start time is incorrect in certain cases
 Key: HIVE-10957
 URL: https://issues.apache.org/jira/browse/HIVE-10957
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Wei Zheng






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10956) HS2 leaks HMS connections

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HIVE-10956:
--

 Summary: HS2 leaks HMS connections
 Key: HIVE-10956
 URL: https://issues.apache.org/jira/browse/HIVE-10956
 Project: Hive
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


HS2 uses threadlocal to cache HMS client in class Hive. When the thread is 
dead, the HMS client is not closed. So the connection to the HMS is leaked.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10955) CliDriver leaves tables behind at end of test run

[Ashutosh Chauhan (JIRA)]

Ashutosh Chauhan created HIVE-10955:
---

 Summary: CliDriver leaves tables behind at end of test run 
 Key: HIVE-10955
 URL: https://issues.apache.org/jira/browse/HIVE-10955
 Project: Hive
  Issue Type: Test
  Components: Testing Infrastructure
Affects Versions: 1.3.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan


When run serially with other drives, this causes problems



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10954) AggregateStatsCache duplicated in HBaseMetastore

[Alan Gates (JIRA)]

Alan Gates created HIVE-10954:
-

 Summary: AggregateStatsCache duplicated in HBaseMetastore
 Key: HIVE-10954
 URL: https://issues.apache.org/jira/browse/HIVE-10954
 Project: Hive
  Issue Type: Task
  Components: Metastore
Affects Versions: hbase-metastore-branch
Reporter: Alan Gates
Assignee: Alan Gates
 Fix For: hbase-metastore-branch


With the latest merge of trunk in hbase-metastore the hbase branch now includes 
two copies of AggregateStatsCache.  This is because it was moved it from hbase 
to the metastore in general.  We need to remove the hbase specific copy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 35160: Fix the indeterministic stats for some hive queries

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35160/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

fix stats for accumulo queries


Diffs
-

  accumulo-handler/src/test/results/positive/accumulo_predicate_pushdown.q.out 
47e92c2 
  accumulo-handler/src/test/results/positive/accumulo_queries.q.out c99c423 

Diff: https://reviews.apache.org/r/35160/diff/


Testing
---


Thanks,

pengcheng xiong

[jira] [Created] (HIVE-10953) Get partial stats instead of complete stats in some queries

[Daniel Dai (JIRA)]

Daniel Dai created HIVE-10953:
-

 Summary: Get partial stats instead of complete stats in some 
queries
 Key: HIVE-10953
 URL: https://issues.apache.org/jira/browse/HIVE-10953
 Project: Hive
  Issue Type: Sub-task
Reporter: Daniel Dai
Assignee: Vaibhav Gumashta
 Fix For: hbase-metastore-branch


In ppd_constant_where.q, the result is different than benchmark:

Result:
Statistics: Num rows: 0 Data size: 11624 Basic stats: PARTIAL Column stats: NONE

Benchmark:
Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: 
NONE

This might cause quite a few failures so we need to investigate it first.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10952) Describe a non-partitioned table fail

[Daniel Dai (JIRA)]

Daniel Dai created HIVE-10952:
-

 Summary: Describe a non-partitioned table fail
 Key: HIVE-10952
 URL: https://issues.apache.org/jira/browse/HIVE-10952
 Project: Hive
  Issue Type: Sub-task
  Components: Metastore
Reporter: Daniel Dai
 Fix For: hbase-metastore-branch


This section of alter1.q fail:
create table alter1(a int, b int);
describe extended alter1;

Exception:
{code}
Trying to fetch a non-existent storage descriptor from hash 
iNVRGkfwwQDGK9oX0fo9XA==^M

at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$QualifiedNameUtil.getAttemptTableName(DDLSemanticAnalyzer.java:1765)
at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$QualifiedNameUtil.getTableName(DDLSemanticAnalyzer.java:1807)
at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDescribeTable(DDLSemanticAnalyzer.java:1985)
at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:318)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1128)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1176)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1065)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1055)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
at 
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1069)
at 
org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1043)
at 
org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:139)
at 
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter1(TestCliDriver.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:176)
at junit.framework.TestCase.runBare(TestCase.java:141)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:129)
at junit.framework.TestSuite.runTest(TestSuite.java:255)
at junit.framework.TestSuite.run(TestSuite.java:250)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch 
table alter1. java.lang.RuntimeException: Woh, bad!  Trying to fetch a 
non-existent storage descriptor from hash iNVRGkfwwQDGK9oX0fo9XA==^M

at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1121)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1068)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1055)
at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$QualifiedNameUtil.getAttemptTableName(DDLSemanticAnalyzer.java:1747)
{code}
The partitioned counterpart alter2.q pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10951) Describe a non-partitioned table table

[Daniel Dai (JIRA)]

Daniel Dai created HIVE-10951:
-

 Summary: Describe a non-partitioned table table 
 Key: HIVE-10951
 URL: https://issues.apache.org/jira/browse/HIVE-10951
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Daniel Dai
 Fix For: hbase-metastore-branch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10950) Unit test against HBase Metastore

[Daniel Dai (JIRA)]

Daniel Dai created HIVE-10950:
-

 Summary: Unit test against HBase Metastore
 Key: HIVE-10950
 URL: https://issues.apache.org/jira/browse/HIVE-10950
 Project: Hive
  Issue Type: Sub-task
  Components: Metastore
Reporter: Daniel Dai
Assignee: Daniel Dai
 Fix For: hbase-metastore-branch


We need to run the entire Hive UT against HBase Metastore and make sure they 
pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 34059: HIVE-10673 Dynamically partitioned hash join for Tez

[Jason Dere]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34059/
---

(Updated June 5, 2015, 9:47 p.m.)


Review request for hive, Matt McCline and Vikram Dixit Kumaraswamy.


Changes
---

Patch v4 - fixed rebase


Bugs: HIVE-10673
https://issues.apache.org/jira/browse/HIVE-10673


Repository: hive-git


Description
---

Reduce-side hash join (using MapJoinOperator), where the Tez inputs to the 
reducer are unsorted.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a724fd1 
  itests/src/test/resources/testconfiguration.properties 47a1107 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 15cafdd 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java d7f1b42 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesAdapter.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesFromKeyValue.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesFromKeyValues.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ReduceRecordProcessor.java 
545d7c6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ReduceRecordSource.java 
cdabe3a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java 
e9bd44a 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinCommonOperator.java
 4c8c4b1 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 
bcffdbc 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java 4d84f0f 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java 
bca91dd 
  ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezProcContext.java adc31ae 
  ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java 0edfc5d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezWork.java 6db8220 
  ql/src/java/org/apache/hadoop/hive/ql/plan/BaseWork.java a342738 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java fb3c4a3 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java cee9100 
  ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/tez_vector_dynpart_hashjoin_1.q 
PRE-CREATION 
  ql/src/test/results/clientpositive/tez/tez_dynpart_hashjoin_1.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/tez/tez_dynpart_hashjoin_2.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/tez/tez_vector_dynpart_hashjoin_1.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/34059/diff/


Testing
---

q-file tests added


Thanks,

Jason Dere

[jira] [Created] (HIVE-10949) Disable hive-minikdc tests in Windows

[Hari Sankar Sivarama Subramaniyan (JIRA)]

Hari Sankar Sivarama Subramaniyan created HIVE-10949:


 Summary: Disable hive-minikdc tests in Windows
 Key: HIVE-10949
 URL: https://issues.apache.org/jira/browse/HIVE-10949
 Project: Hive
  Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan


hive-minikdc needs to be disabled for Windows OS since we dont have kerberos 
support yet for Hadoop Cluster running under Windows OS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 35136: HIVE-10911 Add support for date datatype in the value based windowing function

[Aihua Xu]

> On June 5, 2015, 6:22 p.m., Ashutosh Chauhan wrote:
> > data/files/emp2.txt, line 1
> > 
> >
> > Can you add few rows with NULL value for date column? Want to make sure 
> >  DateValueBoundaryScanner.isGreater() and isEqual() methods are null 
> > resilient.

Added the support to deal with NULL date. Compared to Oracle DB, since HIVE 
puts NULL date rows first while Oracle puts in the last, there are some 
differences for some windowing. But seems both are OK.


- Aihua


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35136/#review86824
---


On June 5, 2015, 8:42 p.m., Aihua Xu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35136/
> ---
> 
> (Updated June 5, 2015, 8:42 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-10911 Add support for date datatype in the value based windowing function
> 
> 
> Diffs
> -
> 
>   data/files/emp2.txt PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java c1b8a1d 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/WindowingTableFunction.java 
> 40fd6a4 
>   ql/src/test/queries/clientpositive/windowing_windowspec3.q PRE-CREATION 
>   ql/src/test/results/clientpositive/windowing_windowspec3.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/35136/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Aihua Xu
> 
>

Re: Review Request 35136: HIVE-10911 Add support for date datatype in the value based windowing function

[Aihua Xu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35136/
---

(Updated June 5, 2015, 8:42 p.m.)


Review request for hive.


Repository: hive-git


Description
---

HIVE-10911 Add support for date datatype in the value based windowing function


Diffs (updated)
-

  data/files/emp2.txt PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java c1b8a1d 
  ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/WindowingTableFunction.java 
40fd6a4 
  ql/src/test/queries/clientpositive/windowing_windowspec3.q PRE-CREATION 
  ql/src/test/results/clientpositive/windowing_windowspec3.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/35136/diff/


Testing
---


Thanks,

Aihua Xu

[jira] [Created] (HIVE-10948) Slf4j warning in HiveCLI due to spark

[Bing Li (JIRA)]

Bing Li created HIVE-10948:
--

 Summary: Slf4j warning in HiveCLI due to spark
 Key: HIVE-10948
 URL: https://issues.apache.org/jira/browse/HIVE-10948
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 1.2.0
Reporter: Bing Li
Assignee: Bing Li
Priority: Minor


The spark-assembly-1.3.1.jar is added to the Hive classpath 
./hive.distro:  export SPARK_HOME=$sparkHome
./hive.distro:  sparkAssemblyPath=`ls ${SPARK_HOME}/lib/spark-assembly-*.jar`
./hive.distro:  CLASSPATH="${CLASSPATH}:${sparkAssemblyPath}"

When launch HiveCLI, we could see the following message:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/.../hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/.../spark/lib/spark-assembly-1.3.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindingsfor an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/.../hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/.../spark/lib/spark-assembly-1.3.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindingsfor an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10947) LLAP: preemption appears to count against failure count for the task

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-10947:
---

 Summary: LLAP: preemption appears to count against failure count 
for the task
 Key: HIVE-10947
 URL: https://issues.apache.org/jira/browse/HIVE-10947
 Project: Hive
  Issue Type: Sub-task
Reporter: Sergey Shelukhin
Assignee: Siddharth Seth


Looks like the following stack in very parallel workload counts as task error 
and DAG fails:
{noformat}
: Error while processing statement: FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, 
vertexId=vertex_1433459966952_0482_4_03, diagnostics=[Task failed, 
taskId=task_1433459966952_0482_4_03_22, diagnostics=[TaskAttempt 0 killed, 
TaskAttempt 1 killed, TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 
killed, TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, 
TaskAttempt 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 
11 killed, TaskAttempt 12 killed, TaskAttempt 13 killed, TaskAttempt 14 killed, 
TaskAttempt 15 killed, TaskAttempt 16 killed, TaskAttempt 17 killed, 
TaskAttempt 18 killed, TaskAttempt 19 failed, info=[Error: Failure while 
running task: 
attempt_1433459966952_0482_4_03_22_19:java.lang.RuntimeException: 
java.lang.RuntimeException: Map operator initialization failed
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:181)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:146)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:349)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:60)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1654)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:60)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:35)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:256)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:157)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async 
initialization failed
at 
org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:416)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:388)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:511)
at 
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:464)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:378)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:241)
... 15 more
Caused by: java.util.concurrent.CancellationException
at java.util.concurrent.FutureTask.report(FutureTask.java:121)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at 
org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:408)
... 20 more
], TaskAttempt 20 failed, info=[Error: Failure while running task: 
attempt_1433459966952_0482_4_03_22_20:java.lang.RuntimeException: 
java.lang.RuntimeException: Map operator initialization failed
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:181)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:146)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:349)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:60)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1654)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Ca

Re: Review Request 35136: HIVE-10911 Add support for date datatype in the value based windowing function

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35136/#review86824
---



data/files/emp2.txt


Can you add few rows with NULL value for date column? Want to make sure  
DateValueBoundaryScanner.isGreater() and isEqual() methods are null resilient.


- Ashutosh Chauhan


On June 5, 2015, 3:46 p.m., Aihua Xu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35136/
> ---
> 
> (Updated June 5, 2015, 3:46 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-10911 Add support for date datatype in the value based windowing function
> 
> 
> Diffs
> -
> 
>   data/files/emp2.txt PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java 
> c1b8a1dc42fc8a70fa1ece609f85f9cffe6795d4 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/WindowingTableFunction.java 
> 40fd6a459a733bc465afe6108a184f2e8abf1a7d 
>   ql/src/test/queries/clientpositive/windowing_windowspec3.q PRE-CREATION 
>   ql/src/test/results/clientpositive/windowing_windowspec3.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/35136/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Aihua Xu
> 
>

[jira] [Created] (HIVE-10946) LLAP: recent optimization introduced wrong assert to elevator causing test to fail

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-10946:
---

 Summary: LLAP: recent optimization introduced wrong assert to 
elevator causing test to fail
 Key: HIVE-10946
 URL: https://issues.apache.org/jira/browse/HIVE-10946
 Project: Hive
  Issue Type: Sub-task
Reporter: Sergey Shelukhin


Doesn't happen in real queries that I run but orc_llap fails for 0-column case. 
Need to fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: connection pooling for hive JDBC client

[Thejas Nair]

It is better to use re-use the hive jdbc connection if possible, as
there are costs associated with creating new connection. However,
there are known issues in running multiple queries simultaneously
using a single connection. For now, it woudl be safer to run queries
sequentially within a connection (ie, don't share it across threads
and run concurrent queries). (You can ofcourse run queries in parallel
using different connections).

Regarding connection pooling libraries, I am not sure if that has been
tested. There might be issues like that unsupported api call you
mentioned, that need to be fixed.



On Fri, Jun 5, 2015 at 6:50 AM, McWhorter, David
 wrote:
> Hello, I sent this email to the users list a few days ago but no one there 
> seems to be able to help, so I am trying the dev list.  I am working on an 
> application that queries and interacts with hive using the JDBC API.  In many 
> other cases, using a JDBC connection pool such as commons-dbcp or BoneCP or 
> HikariCP is a recommended practice and results in much better performance.  
> All of the examples I’ve found of using accessing Hive through the JDBC API 
> (such as 
> https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode)
>  use a raw JDBC connection to Hive directly.  A few questions:
>
>   1.  Should hive work with connection pools such as commons-dbcp, BoneCP, or 
> HikariCP?
>  *   Note: I have tried all BoneCP and HikariCP, and cannot seem to get 
> either to work because HiveConnection.setReadOnly throws a 
> java.sql.SQLException saying “Method not supported”.  I am using hive 0.14.0.
>   2.  Do you recommended using a JDBC connection pool for interacting with 
> hive from an application that will execute many repeated and concurrent 
> queries/statements?
>
> Also, just to be clear, I am not asking about configuring the Hive metastore 
> to use a connection pool to connect to its underlying database (MySQL, 
> PostgreSQL, etc), but about using a connection pool to interact with and 
> query Hive through the JDBC api from an application.
>
> Thank you,
> David McWhorter
>
> —
>
> David McWhorter
> Senior Developer, Foundations
> Informatics and Technology Services
> Office: 434.260.5232 | Mobile: 434.227.2551
> david_mcwhor...@premierinc.com  |  
> Premier, Inc. (NASDAQ: PINC)

Re: hive metastore in Apache Ambari

[Thejas Nair]

ambari should update the hive-site.xml .
Does restarting the hive service help ?
Note that the hive-site.xml under the /etc/hive/conf.server/ dir is
one that is used by metastore server and hiveserver2, you might want
to check if that is updated.

On Fri, Jun 5, 2015 at 1:02 AM, Divya Gehlot  wrote:
> Hi,
> I have set up HDP 2.1 on cloud through Apache Ambari.
>
> When we set up configuration of hive in hive metastore in Apache Ambari  
> ,should the same reflect in hive-site.xml
> For instance I have created new MYSQL database through Apache Ambari UI .
> But when I check the hive -site.xml I see this
> 
>   javax.jdo.option.ConnectionURL
>   
> jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true
>   JDBC connect string for a JDBC metastore
> 
>
> 
>   javax.jdo.option.ConnectionDriverName
>   org.apache.derby.jdbc.EmbeddedDriver
>   Driver class name for a JDBC metastore
> 
>
> 
>   hive.hwi.war.file
>   /usr/lib/hive/lib/hive-hwi-0.13.0.2.1.7.0-784.war
>   This is the WAR file with the jsp content for Hive Web 
> Interface
> 
>
> 
>
> My question is do we need to update the hive-site.xml to match up with Hive 
> Metastore configuration
> or it should automatically update the hive-site.xml (which did not happen in 
> my case)
>
> Please help in resolving this doubt.
>
> Thanks,
> Regards,
> Divya
>
>
>

Review Request 35136: HIVE-10911 Add support for date datatype in the value based windowing function

[Aihua Xu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35136/
---

Review request for hive.


Repository: hive-git


Description
---

HIVE-10911 Add support for date datatype in the value based windowing function


Diffs
-

  data/files/emp2.txt PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java 
c1b8a1dc42fc8a70fa1ece609f85f9cffe6795d4 
  ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/WindowingTableFunction.java 
40fd6a459a733bc465afe6108a184f2e8abf1a7d 
  ql/src/test/queries/clientpositive/windowing_windowspec3.q PRE-CREATION 
  ql/src/test/results/clientpositive/windowing_windowspec3.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/35136/diff/


Testing
---


Thanks,

Aihua Xu

hive metastore in Apache Ambari

[Divya Gehlot]

Hi,
I have set up HDP 2.1 on cloud through Apache Ambari.

When we set up configuration of hive in hive metastore in Apache Ambari  
,should the same reflect in hive-site.xml
For instance I have created new MYSQL database through Apache Ambari UI .
But when I check the hive -site.xml I see this

  javax.jdo.option.ConnectionURL
  
jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true
  JDBC connect string for a JDBC metastore



  javax.jdo.option.ConnectionDriverName
  org.apache.derby.jdbc.EmbeddedDriver
  Driver class name for a JDBC metastore



  hive.hwi.war.file
  /usr/lib/hive/lib/hive-hwi-0.13.0.2.1.7.0-784.war
  This is the WAR file with the jsp content for Hive Web 
Interface




My question is do we need to update the hive-site.xml to match up with Hive 
Metastore configuration 
or it should automatically update the hive-site.xml (which did not happen in my 
case)

Please help in resolving this doubt.

Thanks,
Regards,
Divya 


  

connection pooling for hive JDBC client

[McWhorter, David]

Hello, I sent this email to the users list a few days ago but no one there 
seems to be able to help, so I am trying the dev list.  I am working on an 
application that queries and interacts with hive using the JDBC API.  In many 
other cases, using a JDBC connection pool such as commons-dbcp or BoneCP or 
HikariCP is a recommended practice and results in much better performance.  All 
of the examples I’ve found of using accessing Hive through the JDBC API (such 
as 
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode)
 use a raw JDBC connection to Hive directly.  A few questions:

  1.  Should hive work with connection pools such as commons-dbcp, BoneCP, or 
HikariCP?
 *   Note: I have tried all BoneCP and HikariCP, and cannot seem to get 
either to work because HiveConnection.setReadOnly throws a 
java.sql.SQLException saying “Method not supported”.  I am using hive 0.14.0.
  2.  Do you recommended using a JDBC connection pool for interacting with hive 
from an application that will execute many repeated and concurrent 
queries/statements?

Also, just to be clear, I am not asking about configuring the Hive metastore to 
use a connection pool to connect to its underlying database (MySQL, PostgreSQL, 
etc), but about using a connection pool to interact with and query Hive through 
the JDBC api from an application.

Thank you,
David McWhorter

—

David McWhorter
Senior Developer, Foundations
Informatics and Technology Services
Office: 434.260.5232 | Mobile: 434.227.2551
david_mcwhor...@premierinc.com  |  
Premier, Inc. (NASDAQ: PINC)

Review Request 35132: HIVE-10906 Value based UDAF function without orderby expression throws NPE

[Aihua Xu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35132/
---

Review request for hive.


Repository: hive-git


Description
---

HIVE-10906 Value based UDAF function without orderby expression throws NPE


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingSpec.java 
50f86dfd4166992ebbebf1fd34d5bf8549cbab1c 
  ql/src/test/queries/clientpositive/windowing_windowspec3.q PRE-CREATION 
  ql/src/test/results/clientpositive/windowing_windowspec3.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/35132/diff/


Testing
---


Thanks,

Aihua Xu

[jira] [Created] (HIVE-10945) NullPointerException in JvmPauseMonitor

[Rajesh Balamohan (JIRA)]

Rajesh Balamohan created HIVE-10945:
---

 Summary: NullPointerException in JvmPauseMonitor 
 Key: HIVE-10945
 URL: https://issues.apache.org/jira/browse/HIVE-10945
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Reporter: Rajesh Balamohan


{noformat}
[org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@8b91134]: WARN 
common.JvmPauseMonitor: Error Reporting JvmPauseMonitor to Metrics system
java.lang.NullPointerException
at 
org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor.incrementMetricsCounter(JvmPauseMonitor.java:203)
at 
org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:195)
at java.lang.Thread.run(Thread.java:745)
{noformat}

This error is continuously thrown in HS2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-10944) Fix HS2 for Metrics

[Szehon Ho (JIRA)]

Szehon Ho created HIVE-10944:


 Summary: Fix HS2 for Metrics
 Key: HIVE-10944
 URL: https://issues.apache.org/jira/browse/HIVE-10944
 Project: Hive
  Issue Type: Bug
Reporter: Szehon Ho


Some issues about initializing the new HS2 metrics
1.  Metrics is not working properly in HS2 due to wrong init checks
2.  If not enabled, JVMPauseMonitor logs trash to HS2 logs as it wasnt checking 
if metrics was enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)