[jira] [Created] (HIVE-15925) org.apache.hadoop.hive.serde2.io.DoubleWritable cannot be cast to org.apache.hadoop.io.IntWritable

[readme_kylin (JIRA)]

readme_kylin created HIVE-15925:
---

 Summary: org.apache.hadoop.hive.serde2.io.DoubleWritable cannot be 
cast to org.apache.hadoop.io.IntWritable
 Key: HIVE-15925
 URL: https://issues.apache.org/jira/browse/HIVE-15925
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.1.0
 Environment: hadoop2.6.4
spark2.0.1
Reporter: readme_kylin


i try to use spark to read the hive table.when i use the show function to see 
the table contents.error comes.

here is the code:
import org.apache.spark.sql.SparkSession
  val spark = 
SparkSession.builder().appName("test_view").enableHiveSupport().getOrCreate()
import spark.sql
val pass_rate=spark.sql("select * from pass_rate_score_kylin where 
datekey='20170131'")

pass_rate.printSchema
root
 |-- uid: long (nullable = true)
 |-- pass_times: integer (nullable = true)
 |-- fail_times: integer (nullable = true)
 |-- pass_rate: double (nullable = true)
 |-- prediction: double (nullable = true)
 |-- prediction_bucket: decimal(20,1) (nullable = true)
 |-- test: long (nullable = true)
 |-- datekey: string (nullable = true)

pass_rate.show()


error info:
17/02/15 15:36:55 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 2.0 
(TID 8, QZ146): java.lang.ClassCastException: 
org.apache.hadoop.hive.serde2.io.DoubleWritable cannot be cast to 
org.apache.hadoop.io.IntWritable
at 
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.get(WritableIntObjectInspector.java:36)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$14$$anonfun$apply$5.apply(TableReader.scala:393)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$14$$anonfun$apply$5.apply(TableReader.scala:393)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$fillObject$2.apply(TableReader.scala:435)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$fillObject$2.apply(TableReader.scala:426)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at 
org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:247)
at 
org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:240)
at 
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
at 
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
at 
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:86)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

17/02/15 15:36:57 ERROR scheduler.TaskSetManager: Task 0 in stage 2.0 failed 4 
times; aborting job
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in 
stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 
11, QZ146): java.lang.ClassCastException: 
org.apache.hadoop.hive.serde2.io.DoubleWritable cannot be cast to 
org.apache.hadoop.io.IntWritable
at 
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.get(WritableIntObjectInspector.java:36)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$14$$anonfun$apply$5.apply(TableReader.scala:393)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$14$$anonfun$apply$5.apply(TableReader.scala:393)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$fillObject$2.apply(TableReader.scala:435)
at 
org.apache.spark.sql.hive.HadoopTableReader$$anonfun$fillObject$2.apply(TableReader.scala:426)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at 
org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:247)
at 
org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:240)
at 
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
at 
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
at 
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
at org.apache

Re: Review Request 53845: 'like any' and 'like all' operators in hive

[Simanchal Das]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---

(Updated Feb. 15, 2017, 6:47 a.m.)


Review request for hive, Carl Steinbach and Vineet Garg.


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-15229


In Teradata 'like any' and 'like all' operators are mostly used when we are 
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator 
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%', 
'%retail%', '%bank%', '%insurance%');

--Can be written using multiple like condition 
select col1 from table1 where col2 like '%accountant%' or col2 like 
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like 
'%insurance%' ;

--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%', 
'%retail%', '%bank%', '%insurance%');

--Can be written using multiple like operator 
select col1 from table1 where col2 like '%accountant%' and col2 like 
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like 
'%insurance%' ;

Problem statement:

Now a days so many data warehouse projects are being migrated from Teradata to 
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be 
migrated smoothly instead of converting these operators to multiple like 
operators.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0f05160 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f80642b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g eb81393 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 81efadc 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java 
PRE-CREATION 
  ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION 
  ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION 
  ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION 
  ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION 
  ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/show_functions.q.out 3c9bb4a 
  ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/53845/diff/


Testing
---

Junit test cases and query.q files are attached


Thanks,

Simanchal Das

Re: Review Request 56688: HIVE-15881: Use new thread count variable name instead of mapred.dfsclient.parallelism.max

[Thomas Poepping]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56688/#review165645
---




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (lines 3302 - 3303)


Just to be more clear, maybe "(recommended > 1 for blobstore)"



ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java (lines 2123 - 2124)


to avoid duplicating code, could we wrap this functionality in a helper 
method? then this method could also detect if 
DEPRECATED_MAPRED_DFSCLIENT_PARALLELISM_MAX is being used, and if so, log a WARN



ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java (lines 2989 - 2991)


what if my maxThreads is 1000, but I only have one input path? weren't we 
going to add a check to not create a too-large threadPool?


- Thomas Poepping


On Feb. 14, 2017, 10:58 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56688/
> ---
> 
> (Updated Feb. 14, 2017, 10:58 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-15881
> https://issues.apache.org/jira/browse/HIVE-15881
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Deprecates the use of mapred.dfsclient.parallelism.max, and instead use 
> hive.exec.input.listing.max.threads
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> 0bff243f08188c98ffe16c6602f628412881fc66 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 
> e81cbce3e333d44a4088c10491f399e92a505293 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 
> 5a9d83ca4890a68a63f8f6c145d30906a58c7e73 
> 
> Diff: https://reviews.apache.org/r/56688/diff/
> 
> 
> Testing
> ---
> 
> Added tests to TestUtilities.
> Waiting for HiveQA
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Review Request 56695: Fix for HIVE-15904

[Deepak Jaiswal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56695/
---

Review request for hive and Jason Dere.


Bugs: HIVE-15904
https://issues.apache.org/jira/browse/HIVE-15904


Repository: hive-git


Description
---

Check the output of getChildren) method before accessing the values to avoid 
NPE.


Diffs
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
 c8691e8 
  ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q e686af6 
  ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 
cacde93 

Diff: https://reviews.apache.org/r/56695/diff/


Testing
---


Thanks,

Deepak Jaiswal

[GitHub] hive pull request #148: HIVE-15900

[thejasmn]

GitHub user thejasmn opened a pull request:

https://github.com/apache/hive/pull/148

HIVE-15900



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/thejasmn/hive HIVE-15900

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/148.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #148


commit 73a2b4be8e30d3dfa8c5f974540f6c8a43da4327
Author: Thejas M Nair 
Date:   2017-02-14T06:43:50Z

refactor tests to specifically test stdout/stderr

commit 45f7b9347fedf9b5f0c213c493f0a4d2e729e5c9
Author: Thejas M Nair 
Date:   2017-02-14T19:45:45Z

improve minihs2 tez type usage

commit 06751edf48c2c81f04a479ad0f000c5d6b370d32
Author: Thejas M Nair 
Date:   2017-02-15T03:38:12Z

the beeline fix and test updates




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-15924) move ORC PPD failure message caused by a dynamic value to DEBUG level

[Prasanth Jayachandran (JIRA)]

Prasanth Jayachandran created HIVE-15924:


 Summary: move ORC PPD failure message caused by a dynamic value to 
DEBUG level
 Key: HIVE-15924
 URL: https://issues.apache.org/jira/browse/HIVE-15924
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Sergey Shelukhin
Assignee: Prasanth Jayachandran


Several WARN msgs are observed like below when running LLAP with default 
configurations
{code}
2017-02-14T17:42:06,665  WARN [IO-Elevator-Thread-8 
(1484282558103_6753_2_05_30_2)] impl.RecordReaderImpl: 
IllegalStateException when evaluating predicate. Skipping ORC PPD. Exception: 
Failed to retrieve dynamic value for RS_19_store_ss_store_sk_min StatsType: 
Long PredicateType: null
2017-02-14T17:42:06,665  WARN [IO-Elevator-Thread-3 
(1484282558103_6753_2_05_57_0)] impl.RecordReaderImpl: 
IllegalStateException when evaluating predicate. Skipping ORC PPD. Exception: 
Failed to retrieve dynamic value for RS_19_store_ss_store_sk_min StatsType: 
Long PredicateType: null
2017-02-14T17:42:06,665  WARN [IO-Elevator-Thread-8 
(1484282558103_6753_2_05_30_2)] impl.RecordReaderImpl: 
IllegalStateException when evaluating predicate. Skipping ORC PPD. Exception: 
Failed to retrieve dynamic value for RS_13_item_ss_item_sk_min StatsType: Long 
PredicateType: null
2017-02-14T17:42:06,665  WARN [IO-Elevator-Thread-3 
(1484282558103_6753_2_05_57_0)] impl.RecordReaderImpl: 
IllegalStateException when evaluating predicate. Skipping ORC PPD. Exception: 
Failed to retrieve dynamic value for RS_13_item_ss_item_sk_min StatsType: Long 
PredicateType: null
2017-02-14T17:42:06,665  WARN [IO-Elevator-Thread-8 
(1484282558103_6753_2_05_30_2)] impl.RecordReaderImpl: 
IllegalStateException when evaluating predicate. Skipping ORC PPD. Exception: 
Failed to retrieve dynamic value for RS_19_store_ss_store_sk_min StatsType: 
Long PredicateType: null
{code}





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Review Request 56688: HIVE-15881: Use new thread count variable name instead of mapred.dfsclient.parallelism.max

[Sergio Pena]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56688/
---

Review request for hive.


Bugs: HIVE-15881
https://issues.apache.org/jira/browse/HIVE-15881


Repository: hive-git


Description
---

Deprecates the use of mapred.dfsclient.parallelism.max, and instead use 
hive.exec.input.listing.max.threads


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
0bff243f08188c98ffe16c6602f628412881fc66 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 
e81cbce3e333d44a4088c10491f399e92a505293 
  ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java 
5a9d83ca4890a68a63f8f6c145d30906a58c7e73 

Diff: https://reviews.apache.org/r/56688/diff/


Testing
---

Added tests to TestUtilities.
Waiting for HiveQA


Thanks,

Sergio Pena

[jira] [Created] (HIVE-15923) Hive default partition causes errors in get partitions

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-15923:
---

 Summary: Hive default partition causes errors in get partitions
 Key: HIVE-15923
 URL: https://issues.apache.org/jira/browse/HIVE-15923
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin


This is the ORM error, direct SQL fails too before that, with a similar error.

{noformat}

2017-02-14T17:45:11,158 ERROR [09fdd887-0164-4f55-97e9-4ba147d962be main] 
metastore.ObjectStore:java.lang.ClassCastException: 
org.apache.hadoop.hive.ql.plan.ExprNodeConstantDefaultDesc cannot be cast to 
java.lang.Long
at 
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaLongObjectInspector.get(JavaLongObjectInspector.java:40)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getDouble(PrimitiveObjectInspectorUtils.java:801)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]at 
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$DoubleConverter.convert(P
rimitiveObjectInspectorConverter.java:240) 
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPEqualOrGreaterThan.evaluate(GenericUDFOPEqualOrGreaterThan.java:145)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBetween.evaluate(GenericUDFBetween.java:57)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:187)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:88)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd.evaluate(GenericUDFOPAnd.java:63)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:187)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:68)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.evaluateExprOnPart(PartExprEvalUtils.java:126)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Review Request 56687: Intern strings in various critical places to reduce memory consumption.

[Misha Dmitriev]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56687/
---

Review request for hive, Chaoyu Tang and Mohit Sabharwal.


Bugs: https://issues.apache.org/jira/browse/HIVE-15882

https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/HIVE-15882


Repository: hive-git


Description
---

See the description of the problem in 
https://issues.apache.org/jira/browse/HIVE-15882 Interning strings per this 
review removes most of the overhead due to duplicate strings.

Also, where maps in several places are created from other maps, use the 
original map's size for the new map. This is to avoid the situation when a map 
with default capacity (typically 16) is created to hold just 2-3 entries, and 
the rest of the internal 16-entry array is wasted.


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/StringInternUtils.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 
e81cbce3e333d44a4088c10491f399e92a505293 
  ql/src/java/org/apache/hadoop/hive/ql/hooks/Entity.java 
08420664d59f28f75872c25c9f8ee42577b23451 
  ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java 
e91064b9c75e8adb2b36f21ff19ec0c1539b03b9 
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 
51530ac16c92cc75d501bfcb573557754ba0c964 
  ql/src/java/org/apache/hadoop/hive/ql/io/SymbolicInputFormat.java 
55b3b551a1dac92583b6e03b10beb8172ca93d45 
  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java 
82dc89803be9cf9e0018720eeceb90ff450bfdc8 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java 
c0edde9e92314d86482b5c46178987e79fae57fe 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 
c6ae6f290857cfd10f1023058ede99bf4a10f057 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
24d16812515bdfa90b4be7a295c0388fcdfe95ef 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java
 ede4fcbe342052ad86dadebcc49da2c0f515ea98 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/NullScanTaskDispatcher.java
 0882ae2c6205b1636cbc92e76ef66bb70faadc76 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java 
68b0ad9ea63f051f16fec3652d8525f7ab07eb3f 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java 
d4bdd96eaf8d179bed43b8a8c3be0d338940154a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MsckDesc.java 
b7a7e4b7a5f8941b080c7805d224d3885885f444 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java 
73981e826870139a42ad881103fdb0a2ef8433a2 

Diff: https://reviews.apache.org/r/56687/diff/


Testing
---


Thanks,

Misha Dmitriev

[jira] [Created] (HIVE-15922) SchemaEvolution must guarantee that getFileIncluded is not null

[Owen O'Malley (JIRA)]

Owen O'Malley created HIVE-15922:


 Summary: SchemaEvolution must guarantee that getFileIncluded is 
not null
 Key: HIVE-15922
 URL: https://issues.apache.org/jira/browse/HIVE-15922
 Project: Hive
  Issue Type: Bug
  Components: ORC
Affects Versions: 2.1.1
Reporter: Owen O'Malley
 Fix For: 2.1.2


This only impacts branch-2.1, because it is already fixed in master by 
HIVE-14007.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Review Request 56686: HIVE-15905: Inefficient plan for correlated subqueries

[Vineet Garg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56686/
---

Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-15905
https://issues.apache.org/jira/browse/HIVE-15905


Repository: hive-git


Description
---

This in-corporates changes from CALCITE-1494 and fixes issues to improve 
correlated subquery plan


Diffs
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java
 9c26801 
  ql/src/test/queries/clientpositive/subquery_multi.q aff7f20 
  ql/src/test/results/clientpositive/constprog_partitioner.q.out 2a44269 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 21fd10c 
  ql/src/test/results/clientpositive/llap/subquery_exists.q.out 23d68ab 
  ql/src/test/results/clientpositive/llap/subquery_in.q.out 155a901 
  ql/src/test/results/clientpositive/llap/subquery_multi.q.out 6e33d07 
  ql/src/test/results/clientpositive/llap/subquery_notin.q.out 8530a35 
  ql/src/test/results/clientpositive/llap/subquery_scalar.q.out ae87ab2 
  ql/src/test/results/clientpositive/llap/subquery_views.q.out d3fdec0 
  ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out 7cbd4f6 
  ql/src/test/results/clientpositive/perf/query1.q.out a47548f 
  ql/src/test/results/clientpositive/perf/query16.q.out 1be5fba 
  ql/src/test/results/clientpositive/perf/query30.q.out 8d38fe2 
  ql/src/test/results/clientpositive/perf/query6.q.out 094459e 
  ql/src/test/results/clientpositive/perf/query69.q.out 71ddd71 
  ql/src/test/results/clientpositive/perf/query81.q.out 8ca0068 
  ql/src/test/results/clientpositive/semijoin5.q.out 07b7470 
  ql/src/test/results/clientpositive/spark/constprog_partitioner.q.out 5313e9b 
  ql/src/test/results/clientpositive/spark/subquery_exists.q.out bc25efe 
  ql/src/test/results/clientpositive/spark/subquery_in.q.out 0fde046 
  ql/src/test/results/clientpositive/spark/vector_mapjoin_reduce.q.out f071824 
  ql/src/test/results/clientpositive/subquery_exists.q.out 412c579 
  ql/src/test/results/clientpositive/subquery_exists_having.q.out 51c98ed 
  ql/src/test/results/clientpositive/subquery_in_having.q.out 2d44a15 
  ql/src/test/results/clientpositive/subquery_notexists.q.out b916d2f 
  ql/src/test/results/clientpositive/subquery_notexists_having.q.out 0ca743d 
  ql/src/test/results/clientpositive/subquery_notin_having.q.out 644e70e 
  ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out bdfdc34 
  ql/src/test/results/clientpositive/vector_mapjoin_reduce.q.out 5ea9a65 

Diff: https://reviews.apache.org/r/56686/diff/


Testing
---

* Updated existing tests
* Pre-commit testing


Thanks,

Vineet Garg

[jira] [Created] (HIVE-15921) Re-order the slider stop command to avoid a force if possible

[Siddharth Seth (JIRA)]

Siddharth Seth created HIVE-15921:
-

 Summary: Re-order the slider stop command to avoid a force if 
possible
 Key: HIVE-15921
 URL: https://issues.apache.org/jira/browse/HIVE-15921
 Project: Hive
  Issue Type: Bug
Reporter: Siddharth Seth
Assignee: Siddharth Seth


A graceful stop is required for slider --service llapstatus to work properly



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15920) Implement a blocking version of a command to compact

[Eugene Koifman (JIRA)]

Eugene Koifman created HIVE-15920:
-

 Summary: Implement a blocking version of a command to compact
 Key: HIVE-15920
 URL: https://issues.apache.org/jira/browse/HIVE-15920
 Project: Hive
  Issue Type: Improvement
  Components: Transactions
Reporter: Eugene Koifman
Assignee: Eugene Koifman


currently 
{noformat}
alter table AcidTable compact 'major'
{noformat} 
is supported which enqueues a msg to compact.

Would be nice for testing and script building to support 
{noformat} 
alter table AcidTable compact 'major' blocking
{noformat} 
perhaps another variation is to block until either compaction is done or until 
cleaning is finished.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15919) Row count mismatch for count * query

[Aswathy Chellammal Sreekumar (JIRA)]

Aswathy Chellammal Sreekumar created HIVE-15919:
---

 Summary: Row count mismatch for count * query
 Key: HIVE-15919
 URL: https://issues.apache.org/jira/browse/HIVE-15919
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Reporter: Aswathy Chellammal Sreekumar
 Attachments: table_14.q, table_6.q

The following query is returning different output when run against hive and 
postgres.

Query:

SELECT COUNT (*)
FROM
(SELECT LAG(COALESCE(t2.int_col_14, t1.int_col_80),22) OVER (ORDER BY 
t1.tinyint_col_52 DESC) AS int_col
FROM table_6 t1
INNER JOIN table_14 t2 ON ((t2.decimal0101_col_55) = (t1.decimal0101_col_9))) 
AS FOO;

>From hive: 0
>From postgres: 66903279

Attaching ddl and data files for the tables.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15918) Add some debug messages to identify an issue getting runtimeInfo from tez

[Siddharth Seth (JIRA)]

Siddharth Seth created HIVE-15918:
-

 Summary: Add some debug messages to identify an issue getting 
runtimeInfo from tez
 Key: HIVE-15918
 URL: https://issues.apache.org/jira/browse/HIVE-15918
 Project: Hive
  Issue Type: Task
  Components: llap
Reporter: Siddharth Seth
Assignee: Siddharth Seth






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15917) incorrect error handling from BackgroundWork

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-15917:
---

 Summary: incorrect error handling from BackgroundWork
 Key: HIVE-15917
 URL: https://issues.apache.org/jira/browse/HIVE-15917
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15916) Add blobstore tests for CTAS

[JIRA]

Juan Rodríguez Hortalá created HIVE-15916:
-

 Summary: Add blobstore tests for CTAS
 Key: HIVE-15916
 URL: https://issues.apache.org/jira/browse/HIVE-15916
 Project: Hive
  Issue Type: Improvement
  Components: Test
Reporter: Juan Rodríguez Hortalá
Assignee: Juan Rodríguez Hortalá


This patch covers 3 tests checking CTAS operations against blobstore 
filesystems. The tests check we can create a table with a CTAS statement from 
another table, for the source-target combinations blobtore-blobstore, 
blobstore-hdfs, hdfs-blobstore, and for two target tables, one in the same 
default database as  the source, and another in a new database. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15915) Emit progress percentage in getting operation status

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HIVE-15915:
--

 Summary: Emit progress percentage in getting operation status
 Key: HIVE-15915
 URL: https://issues.apache.org/jira/browse/HIVE-15915
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


When running a query asynchronously, client may want to check the progress 
periodically. HIVE-15473 is to support progressing bar on beeline for Tez. For 
this issue, we just want the progress percentage. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15914) Fix issues with druid-handler pom file

[Jesus Camacho Rodriguez (JIRA)]

Jesus Camacho Rodriguez created HIVE-15914:
--

 Summary: Fix issues with druid-handler pom file
 Key: HIVE-15914
 URL: https://issues.apache.org/jira/browse/HIVE-15914
 Project: Hive
  Issue Type: Bug
  Components: Druid integration
Affects Versions: 2.2.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez


Patch fixes multiple issues, including warnings when Hive is compiled due to 
multiple definitions of the same dependency (joda-time).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15913) JsonSerDe should support JSON Schema

[Elliot West (JIRA)]

Elliot West created HIVE-15913:
--

 Summary: JsonSerDe should support JSON Schema
 Key: HIVE-15913
 URL: https://issues.apache.org/jira/browse/HIVE-15913
 Project: Hive
  Issue Type: Improvement
  Components: Serializers/Deserializers
Reporter: Elliot West


Hive's 
[{{AvroSerDe}}|https://cwiki.apache.org/confluence/display/Hive/AvroSerDe] has 
some very nice features around the use of Avro Schema definitions. For example, 
the SerDe can derive the table definition from an Avro schema declared as a 
[literal in the table 
properties|https://cwiki.apache.org/confluence/display/Hive/AvroSerDe#AvroSerDe-Useschema.literalandembedtheschemainthecreatestatement],
 or [from a URI 
resource|https://cwiki.apache.org/confluence/display/Hive/AvroSerDe#AvroSerDe-Useavro.schema.url].
 This makes it really simple to integrate Hive into an Avro based data pipeline.

The JSON data format also has some schema support in the form of the [JSON 
Schema standard|http://json-schema.org]. This standard is gaining in popularity 
and is embedded into some high profile use cases.

It would be very useful to users of JSON data if the {{JsonSerDe}} could have a 
comparable set of schema related features to that of Avro's, based upon the 
JSON Schema standard.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 53845: 'like any' and 'like all' operators in hive

[Simanchal Das]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53845/
---

(Updated Feb. 14, 2017, 12:39 p.m.)


Review request for hive, Carl Steinbach and Vineet Garg.


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-15229


In Teradata 'like any' and 'like all' operators are mostly used when we are 
matching a text field with numbers of patterns.
'like any' and 'like all' operator are equivalents of multiple like operator 
like example below.
--like any
select col1 from table1 where col2 like any ('%accountant%', '%accounting%', 
'%retail%', '%bank%', '%insurance%');

--Can be written using multiple like condition 
select col1 from table1 where col2 like '%accountant%' or col2 like 
'%accounting%' or col2 like '%retail%' or col2 like '%bank%' or col2 like 
'%insurance%' ;

--like all
select col1 from table1 where col2 like all ('%accountant%', '%accounting%', 
'%retail%', '%bank%', '%insurance%');

--Can be written using multiple like operator 
select col1 from table1 where col2 like '%accountant%' and col2 like 
'%accounting%' and col2 like '%retail%' and col2 like '%bank%' and col2 like 
'%insurance%' ;

Problem statement:

Now a days so many data warehouse projects are being migrated from Teradata to 
Hive.
Always Data engineer and Business analyst are searching for these two operator.
If we introduce these two operator in hive then so many scripts will be 
migrated smoothly instead of converting these operators to multiple like 
operators.


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 0f05160 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f80642b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g eb81393 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 81efadc 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAll.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLikeAny.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAll.java 
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLikeAny.java 
PRE-CREATION 
  ql/src/test/queries/clientnegative/udf_likeall_wrong1.q PRE-CREATION 
  ql/src/test/queries/clientnegative/udf_likeany_wrong1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/udf_likeall.q PRE-CREATION 
  ql/src/test/queries/clientpositive/udf_likeany.q PRE-CREATION 
  ql/src/test/results/clientnegative/udf_likeall_wrong1.q.out PRE-CREATION 
  ql/src/test/results/clientnegative/udf_likeany_wrong1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/show_functions.q.out 3c9bb4a 
  ql/src/test/results/clientpositive/udf_likeall.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/udf_likeany.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/53845/diff/


Testing
---

Junit test cases and query.q files are attached


Thanks,

Simanchal Das

[jira] [Created] (HIVE-15912) Failed to get spark memory/core info

[KaiXu (JIRA)]

KaiXu created HIVE-15912:


 Summary: Failed to get spark memory/core info
 Key: HIVE-15912
 URL: https://issues.apache.org/jira/browse/HIVE-15912
 Project: Hive
  Issue Type: Bug
  Components: Hive, Spark
Affects Versions: 2.2.0
 Environment: hadoop2.7.1
spark2.0.2
Hive2.2
Reporter: KaiXu


Hive on Spark, failed with error:
Starting Spark Job = 12a8cb8c-ed0d-4049-ae06-8d32d13fe285
Failed to monitor Job[ 6] with exception 'java.lang.IllegalStateException(RPC 
channel is closed.)'
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.spark.SparkTask

Hive's log:
2017-02-14T19:03:09,147  INFO [stderr-redir-1] client.SparkClientImpl: 17/02/14 
19:03:09 INFO yarn.Client: Application report for 
application_1486905599813_0403 (state: ACCEPTED)
2017-02-14T19:03:10,817  WARN [5bcf13e5-cb54-4cfe-a0d4-9a6556ab48b1 main] 
spark.SetSparkReducerParallelism: Failed to get spark memory/core info
java.util.concurrent.TimeoutException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:49) 
~[netty-all-4.0.29.Final.jar:4.0.29.Final]
at 
org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.getExecutorCount(RemoteHiveSparkClient.java:155)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.getExecutorCount(RemoteHiveSparkClient.java:165)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.getMemoryAndCores(SparkSessionImpl.java:77)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.optimizer.spark.SetSparkReducerParallelism.process(SetSparkReducerParallelism.java:119)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:158)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.runJoinOptimizations(SparkCompiler.java:291)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.optimizeOperatorPlan(SparkCompiler.java:120)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:140) 
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11085)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:279)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:510) 
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1302) 
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1442) 
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1222) 
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1212) 
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) 
~[hive-cli-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) 
~[hive-cli-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400) 
~[hive-cli-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) 
~[hive-cli-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:430) 
~[hive-cli-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:446) 
~[hive-cli-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749) 
~[hive-cli-2.2.0-SNAPSH

[GitHub] hive pull request #147: HIVE-15910: Improvements in Hive Unit Test by using ...

[sankarh]

GitHub user sankarh opened a pull request:

https://github.com/apache/hive/pull/147

HIVE-15910: Improvements in Hive Unit Test by using In-memory Derby DB



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sankarh/hive HIVE-15910

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/147.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #147


commit 4c6596e94524eb8ac0066e1371ed6ceb2c650036
Author: Sankar Hariappan 
Date:   2017-02-14T11:39:02Z

HIVE-15910: Improvements in Hive Unit Test by using In-memory Derby DB




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-15911) Creating a view based on another view throws an exception “FAILED: NullPointerException null”

[yangfang (JIRA)]

yangfang created HIVE-15911:
---

 Summary: Creating a view based on another view throws an exception 
“FAILED: NullPointerException null”
 Key: HIVE-15911
 URL: https://issues.apache.org/jira/browse/HIVE-15911
 Project: Hive
  Issue Type: Bug
  Components: Views
Affects Versions: 2.1.1, 2.1.0
 Environment: hive2.1.0
Reporter: yangfang


when I create a new view based on another view I an exception “FAILED: 
NullPointerException null”:

hive> create view view2(a,b) as select a, b from view1;
FAILED: NullPointerException null
hive> 

The hive log show error stack：
2017-02-13T20:54:29,288 ERROR ql.Driver (:()) - FAILED: NullPointerException 
null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:710)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:474)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:331)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1170)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1265)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1092)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1080)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
at 
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Sankar Hariappan (JIRA)]

Sankar Hariappan created HIVE-15910:
---

 Summary: Improvements in Hive Unit Test by using In-memory Derby DB
 Key: HIVE-15910
 URL: https://issues.apache.org/jira/browse/HIVE-15910
 Project: Hive
  Issue Type: Test
  Components: Tests
Reporter: Sankar Hariappan
Assignee: Sankar Hariappan


Hive UT currently uses Derby DB with storage on disk which have some practical 
problems.
1. The run-time of Hive unit tests are high as need to operate on the disk 
quite often.
2. It can cause conflict if multiple test cases operates on the same table name 
(such as table being created already exist).

To solve these problems, we shall use an in-memory storage option of Derby DB 
which can be even persisted if the test case demands that.
https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] hive pull request #144: HIVE-15858: Beeline ^C doesn't close the session

[sankarh]

Github user sankarh closed the pull request at:

https://github.com/apache/hive/pull/144


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-15909) HiveMetaStoreChecker::checkTable can be expensive in ObjectStores

[Rajesh Balamohan (JIRA)]

Rajesh Balamohan created HIVE-15909:
---

 Summary: HiveMetaStoreChecker::checkTable can be expensive in 
ObjectStores
 Key: HIVE-15909
 URL: https://issues.apache.org/jira/browse/HIVE-15909
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Rajesh Balamohan
Priority: Minor


With objectstores like s3, HiveMetaStoreChecker::checkTable can be expensive 
with partitioned dataset.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)