[jira] [Created] (HIVE-12701) select on table with boolean as partition column shows wrong result

[Sudipto Nandan (JIRA)]

Sudipto Nandan created HIVE-12701:
-

 Summary: select on table with boolean as partition column shows 
wrong result
 Key: HIVE-12701
 URL: https://issues.apache.org/jira/browse/HIVE-12701
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Sudipto Nandan


create table hive_aprm02ht7(a int, b int, c int) partitioned by (p boolean) row 
format delimited fields terminated by ',' stored as textfile;

load data local inpath 'hive_data8.txt' into table hive_aprm02ht7 partition 
(p=true);
load data local inpath 'hive_data8.txt' into table hive_aprm02ht7 partition 
(p=false);

describe hive_aprm02ht7;
col_namedata_type   comment
a   int
b   int
c   int
p   boolean

# Partition Information
# col_name  data_type   comment

p   boolean

show partitions hive_aprm02ht7;
OK
p=false
p=true
Time taken: 0.057 seconds, Fetched: 2 row(s)

-- everything is shown as true. But first three should be true and the last 
three rows should be false
hive>  select * from hive_aprm02ht7 where p in (true,false);
OK
1   2   3   true
4   5   6   true
7   8   9   true
1   2   3   true
4   5   6   true
7   8   9   true
Time taken: 0.068 seconds, Fetched: 6 row(s)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] hive pull request: HIVE-12698 : introduce HiveAuthorizationTransla...

[thejasmn]

GitHub user thejasmn opened a pull request:

https://github.com/apache/hive/pull/58

HIVE-12698 : introduce HiveAuthorizationTranslator interface for isolating 
authori…

…zation impls from hive internal classes

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/thejasmn/hive HIVE-12698

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/58.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #58


commit 27e73f2a45aa3e5a158c14fb0693567158cef0d7
Author: Thejas Nair 
Date:   2015-12-17T02:36:52Z

introduce HiveAuthorizationTranslator interface for isolating authorization 
impls from hive internal classes




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-12700) complex join keys cannot be recognized in Hive 0.13

[Xiaoyong Zhu (JIRA)]

Xiaoyong Zhu created HIVE-12700:
---

 Summary: complex join keys cannot be recognized in Hive 0.13
 Key: HIVE-12700
 URL: https://issues.apache.org/jira/browse/HIVE-12700
 Project: Hive
  Issue Type: Bug
  Components: Logical Optimizer
Affects Versions: 0.13.1
Reporter: Xiaoyong Zhu
Priority: Critical


Hi Experts

I am using Hive 0.13 and find a potential bug. Attached “implicit join.hql” has 
several join keys (for example store_sales.ss_addr_sk = 
customer_address.ca_address_sk) and cannot be regonized by Hive. In such cases 
hive won’t be able to optimize and can only do a cross join first which makes 
the job runs really long. If I change the log to explicit join keys, then it 
works well.

For the below simple query hive can regcogonize the join keys, and I think Hive 
should be able to handle the complex situations such as my example, right?
 
SELECT * 
FROM table1 t1, table2 t2, table3 t3 
WHERE t1.id = t2.id AND t2.id = t3.id AND t1.zipcode = '02535';



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12699) LLAP: hive.llap.daemon.work.dirs setting backward compat name doesn't work

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-12699:
---

 Summary: LLAP: hive.llap.daemon.work.dirs setting backward compat 
name doesn't work 
 Key: HIVE-12699
 URL: https://issues.apache.org/jira/browse/HIVE-12699
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41457: HIVE-11865

[John Pullokkaran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41457/#review110839
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java (line 104)


What about const propagation?


- John Pullokkaran


On Dec. 16, 2015, 7:14 p.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41457/
> ---
> 
> (Updated Dec. 16, 2015, 7:14 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and John Pullokkaran.
> 
> 
> Bugs: HIVE-11865
> https://issues.apache.org/jira/browse/HIVE-11865
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Disable Hive PPD optimizer when CBO has optimized the plan
> 
> 
> Diffs
> -
> 
>   hbase-handler/src/test/results/positive/hbase_pushdown.q.out 
> be96eecc8642f85a493ae6c72964bc4e30c4db57 
>   hbase-handler/src/test/results/positive/hbase_queries.q.out 
> 3907bc946679c0ce2492692dcb7e2cf252c6e188 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 
> 8f48e7df4b9c081528b82545d445f75f67f93bfc 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java
>  db625c1c3d47b5ee2744950db2fb71804945cbec 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTSTransposeRule.java
>  8321504774c5ec5c7c9de3aecdf78dcd8408c4cf 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTransposeRule.java
>  7e484b90deed2dfb120faa2699eb9b9b7a4e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterSortTransposeRule.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinAddNotNullRule.java
>  2a415d5abd9c054956f46cd26f9df2257d67ecf4 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortMergeRule.java
>  efde6652ca270d8e8d8f682fc0ff260702f46d5a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortProjectTransposeRule.java
>  feec3c25929cac0bb3b6f50d074557a3c2745889 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
> 36a12bf6b2c3ef7233eda7c949725e02c880d7f7 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 27549dceac8b5913b1fb1604bde213ae26b243bf 
>   ql/src/java/org/apache/hadoop/hive/ql/ppd/SimplePredicatePushDown.java 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/auto_join12.q.out 
> 6f08aa8f043113f0a99acd10c848f60a122a3f0b 
>   ql/src/test/results/clientpositive/auto_join13.q.out 
> d29818cf438449a6a1413edd7d474c4ab35f2148 
>   ql/src/test/results/clientpositive/auto_join16.q.out 
> 38738ede2b4cd49aa24490eafb81dec1de78bea5 
>   ql/src/test/results/clientpositive/auto_join2.q.out 
> ce40ff30cb9cc2ab734ee37c0f3dc0ae85bbd202 
>   ql/src/test/results/clientpositive/auto_join33.q.out 
> 8b13bd0cb7fd869c58c9d092b2e8710391c96503 
>   ql/src/test/results/clientpositive/auto_join8.q.out 
> 80dd575c58add873ae34967b1d255ee5b63a9b07 
>   ql/src/test/results/clientpositive/auto_join_filters.q.out 
> e0ed3737f9b25d45bdb365b13aeed3a40c50135f 
>   ql/src/test/results/clientpositive/auto_join_nulls.q.out 
> 954bf069760c6f0d7e5420c7e9a3b9756d5aa4df 
>   ql/src/test/results/clientpositive/auto_join_stats.q.out 
> feb818679df7d9f63f2d2d7a7c9afb2d0f5874d8 
>   ql/src/test/results/clientpositive/auto_join_stats2.q.out 
> e0d0146dd6b86e9f69a467dcde0c020c31af499d 
>   ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 
> c63016d0a3610eaa6cd82a4adc5d1bf6f0925cce 
>   ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out 
> 096f5d445bc7e61381775270bef04e911eac5bb6 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 
> 30853c42a52e1def5a913352580e0b5391ac2050 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out 
> 3e8dbcfd5bcc66ccd83b01c08c7745b02f05328f 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out 
> e6c2832003a1bb804f99125866bff6117e4dab73 
>   ql/src/test/results/clientpositive/cbo_rp_auto_join1.q.out 
> 5c09fe0ad2d30c6c597abdad27b309fd8d00f678 
>   ql/src/test/results/clientpositive/cbo_rp_join0.q.out 
> 3c6bb7319fd7605a62f082b989f723d5aab6c3fe 
>   ql/src/test/results/clientpositive/cbo_rp_join1.q.out 
> e77002866174f85168859c4145e7016c9d829bb6 
>   ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out 
> b14caa80b3346036f9ec835fe4e2294f47eb7ee5 
>   ql/src/test/results/clientpositive/cbo_rp_outer_join_ppr.q.java1.7.out 
> 263053705347793513a44d49968dedd4fdffecfe 
>   ql/src/test/results/clientpositive/constprog_partitioner.q.out 
> 08c0aebafe1644a5adbec6a56879b39617bfbf69 
>   ql/src/test/results/clientpositive/cross_join.q.out 
> f3649

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

[John Pullokkaran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40865/#review110836
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexExecutorImpl.java
 (line 43)


Nit Pick: trailing space



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexExecutorImpl.java
 (line 63)


Why are we printing stack trace?
Log it instead



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveReduceExpressionsRule.java
 (line 235)


Nit Pick: space


- John Pullokkaran


On Dec. 17, 2015, 1:26 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40865/
> ---
> 
> (Updated Dec. 17, 2015, 1:26 a.m.)
> 
> 
> Review request for hive and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-11927
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteSemanticException.java
>  336745b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexExecutorImpl.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveReduceExpressionsRule.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
>  425514d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
>  3f2267d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
>  631a4ca 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  2825f77 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 36a12bf 
>   ql/src/test/queries/clientpositive/cbo_const.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/constantfolding.q PRE-CREATION 
>   ql/src/test/results/clientpositive/annotate_stats_select.q.out c4d59c8 
>   ql/src/test/results/clientpositive/bucketizedhiveinputformat.q.out 277b0f7 
>   ql/src/test/results/clientpositive/cast1.q.out 0bdecba 
>   ql/src/test/results/clientpositive/cbo_const.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/cbo_rp_cross_product_check_2.q.out 
> 62b611b 
>   ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out b14caa8 
>   ql/src/test/results/clientpositive/constantfolding.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/cross_product_check_1.q.out e7d6900 
>   ql/src/test/results/clientpositive/cross_product_check_2.q.out df438c9 
>   ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out b85d387 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 24ac550 
>   ql/src/test/results/clientpositive/groupby_ppd.q.out 6164a26 
>   ql/src/test/results/clientpositive/groupby_sort_1_23.q.out 7333677 
>   ql/src/test/results/clientpositive/groupby_sort_skew_1_23.q.out e19d1de 
>   ql/src/test/results/clientpositive/input_part1.q.out d6f4d3e 
>   ql/src/test/results/clientpositive/input_part5.q.out f2d7335 
>   ql/src/test/results/clientpositive/input_part6.q.out fa51cdf 
>   ql/src/test/results/clientpositive/lineage2.q.out ec8b76b 
>   ql/src/test/results/clientpositive/lineage3.q.out 747dc9a 
>   ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out 
> be77ba8 
>   ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 79348f3 
>   ql/src/test/results/clientpositive/orc_predicate_pushdown.q.out f6d8388 
>   ql/src/test/results/clientpositive/parquet_predicate_pushdown.q.out b322ef1 
>   ql/src/test/results/clientpositive/partition_multilevels.q.out c1c8778 
>   ql/src/test/results/clientpositive/perf/query31.q.out c479d91 
>   ql/src/test/results/clientpositive/perf/query39.q.out 0ad62cc 
>   ql/src/test/results/clientpositive/perf/query42.q.out a3264c7 
>   ql/src/test/results/clientpositive/perf/query52.q.out ac0c1e6 
>   ql/src/test/results/clientpositive/perf/query64.q.out 9735b45 
>   ql/src/test/results/clientpositive/perf/query66.q.out 42bcb83 
>   ql/src/test/results/clientpositive/perf/query75.q.out 70a1649 
>   ql/src/test/results/clientpositive/pointlookup2.q.out a442425 
>   ql/src/test/results/clientpositive/quotedid_basic.q.out 519f647 
>   ql/src/test/results/clientpositive/quotedid_partition.q.out d34a005 
>   ql/src/test/results/clientpositive/spark/bucket_map_join_tez1.q.out b5e7846 
>   ql/src/test/results/clientpositive/spark/cross_product_check_1.q.out 
> 5b03dcf 
>   ql/src/test/results/clientpositive/spark/cr

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

[John Pullokkaran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40865/#review110835
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
(line 41)


Remove trailing space.


- John Pullokkaran


On Dec. 17, 2015, 1:26 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40865/
> ---
> 
> (Updated Dec. 17, 2015, 1:26 a.m.)
> 
> 
> Review request for hive and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-11927
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteSemanticException.java
>  336745b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexExecutorImpl.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveReduceExpressionsRule.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
>  425514d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
>  3f2267d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
>  631a4ca 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  2825f77 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 36a12bf 
>   ql/src/test/queries/clientpositive/cbo_const.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/constantfolding.q PRE-CREATION 
>   ql/src/test/results/clientpositive/annotate_stats_select.q.out c4d59c8 
>   ql/src/test/results/clientpositive/bucketizedhiveinputformat.q.out 277b0f7 
>   ql/src/test/results/clientpositive/cast1.q.out 0bdecba 
>   ql/src/test/results/clientpositive/cbo_const.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/cbo_rp_cross_product_check_2.q.out 
> 62b611b 
>   ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out b14caa8 
>   ql/src/test/results/clientpositive/constantfolding.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/cross_product_check_1.q.out e7d6900 
>   ql/src/test/results/clientpositive/cross_product_check_2.q.out df438c9 
>   ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out b85d387 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 24ac550 
>   ql/src/test/results/clientpositive/groupby_ppd.q.out 6164a26 
>   ql/src/test/results/clientpositive/groupby_sort_1_23.q.out 7333677 
>   ql/src/test/results/clientpositive/groupby_sort_skew_1_23.q.out e19d1de 
>   ql/src/test/results/clientpositive/input_part1.q.out d6f4d3e 
>   ql/src/test/results/clientpositive/input_part5.q.out f2d7335 
>   ql/src/test/results/clientpositive/input_part6.q.out fa51cdf 
>   ql/src/test/results/clientpositive/lineage2.q.out ec8b76b 
>   ql/src/test/results/clientpositive/lineage3.q.out 747dc9a 
>   ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out 
> be77ba8 
>   ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 79348f3 
>   ql/src/test/results/clientpositive/orc_predicate_pushdown.q.out f6d8388 
>   ql/src/test/results/clientpositive/parquet_predicate_pushdown.q.out b322ef1 
>   ql/src/test/results/clientpositive/partition_multilevels.q.out c1c8778 
>   ql/src/test/results/clientpositive/perf/query31.q.out c479d91 
>   ql/src/test/results/clientpositive/perf/query39.q.out 0ad62cc 
>   ql/src/test/results/clientpositive/perf/query42.q.out a3264c7 
>   ql/src/test/results/clientpositive/perf/query52.q.out ac0c1e6 
>   ql/src/test/results/clientpositive/perf/query64.q.out 9735b45 
>   ql/src/test/results/clientpositive/perf/query66.q.out 42bcb83 
>   ql/src/test/results/clientpositive/perf/query75.q.out 70a1649 
>   ql/src/test/results/clientpositive/pointlookup2.q.out a442425 
>   ql/src/test/results/clientpositive/quotedid_basic.q.out 519f647 
>   ql/src/test/results/clientpositive/quotedid_partition.q.out d34a005 
>   ql/src/test/results/clientpositive/spark/bucket_map_join_tez1.q.out b5e7846 
>   ql/src/test/results/clientpositive/spark/cross_product_check_1.q.out 
> 5b03dcf 
>   ql/src/test/results/clientpositive/spark/cross_product_check_2.q.out 
> 93c502d 
>   ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out b597ebd 
>   ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out 239e803 
>   ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out 
> 8370bbe 
>   ql/src/test/results/clientpositive/spark/union_remove_25.q.out b771fe9 
>   ql/src/test/results/clientposi

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

[John Pullokkaran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40865/#review110834
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
(line 49)


What about rest of the factories?


- John Pullokkaran


On Dec. 17, 2015, 1:26 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40865/
> ---
> 
> (Updated Dec. 17, 2015, 1:26 a.m.)
> 
> 
> Review request for hive and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-11927
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteSemanticException.java
>  336745b 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexExecutorImpl.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveReduceExpressionsRule.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
>  425514d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
>  3f2267d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
>  631a4ca 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
>  2825f77 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 36a12bf 
>   ql/src/test/queries/clientpositive/cbo_const.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/constantfolding.q PRE-CREATION 
>   ql/src/test/results/clientpositive/annotate_stats_select.q.out c4d59c8 
>   ql/src/test/results/clientpositive/bucketizedhiveinputformat.q.out 277b0f7 
>   ql/src/test/results/clientpositive/cast1.q.out 0bdecba 
>   ql/src/test/results/clientpositive/cbo_const.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/cbo_rp_cross_product_check_2.q.out 
> 62b611b 
>   ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out b14caa8 
>   ql/src/test/results/clientpositive/constantfolding.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/cross_product_check_1.q.out e7d6900 
>   ql/src/test/results/clientpositive/cross_product_check_2.q.out df438c9 
>   ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out b85d387 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 24ac550 
>   ql/src/test/results/clientpositive/groupby_ppd.q.out 6164a26 
>   ql/src/test/results/clientpositive/groupby_sort_1_23.q.out 7333677 
>   ql/src/test/results/clientpositive/groupby_sort_skew_1_23.q.out e19d1de 
>   ql/src/test/results/clientpositive/input_part1.q.out d6f4d3e 
>   ql/src/test/results/clientpositive/input_part5.q.out f2d7335 
>   ql/src/test/results/clientpositive/input_part6.q.out fa51cdf 
>   ql/src/test/results/clientpositive/lineage2.q.out ec8b76b 
>   ql/src/test/results/clientpositive/lineage3.q.out 747dc9a 
>   ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out 
> be77ba8 
>   ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 79348f3 
>   ql/src/test/results/clientpositive/orc_predicate_pushdown.q.out f6d8388 
>   ql/src/test/results/clientpositive/parquet_predicate_pushdown.q.out b322ef1 
>   ql/src/test/results/clientpositive/partition_multilevels.q.out c1c8778 
>   ql/src/test/results/clientpositive/perf/query31.q.out c479d91 
>   ql/src/test/results/clientpositive/perf/query39.q.out 0ad62cc 
>   ql/src/test/results/clientpositive/perf/query42.q.out a3264c7 
>   ql/src/test/results/clientpositive/perf/query52.q.out ac0c1e6 
>   ql/src/test/results/clientpositive/perf/query64.q.out 9735b45 
>   ql/src/test/results/clientpositive/perf/query66.q.out 42bcb83 
>   ql/src/test/results/clientpositive/perf/query75.q.out 70a1649 
>   ql/src/test/results/clientpositive/pointlookup2.q.out a442425 
>   ql/src/test/results/clientpositive/quotedid_basic.q.out 519f647 
>   ql/src/test/results/clientpositive/quotedid_partition.q.out d34a005 
>   ql/src/test/results/clientpositive/spark/bucket_map_join_tez1.q.out b5e7846 
>   ql/src/test/results/clientpositive/spark/cross_product_check_1.q.out 
> 5b03dcf 
>   ql/src/test/results/clientpositive/spark/cross_product_check_2.q.out 
> 93c502d 
>   ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out b597ebd 
>   ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out 239e803 
>   ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out 
> 8370bbe 
>   ql/src/test/results/clientpositive/spark/union_remove_25.q.out b771fe9 
>   ql/src/test/results

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40865/
---

(Updated Dec. 17, 2015, 1:26 a.m.)


Review request for hive and John Pullokkaran.


Repository: hive-git


Description
---

HIVE-11927


Diffs (updated)
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteSemanticException.java
 336745b 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexExecutorImpl.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveReduceExpressionsRule.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
 425514d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
 3f2267d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
 631a4ca 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 2825f77 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 36a12bf 
  ql/src/test/queries/clientpositive/cbo_const.q PRE-CREATION 
  ql/src/test/queries/clientpositive/constantfolding.q PRE-CREATION 
  ql/src/test/results/clientpositive/annotate_stats_select.q.out c4d59c8 
  ql/src/test/results/clientpositive/bucketizedhiveinputformat.q.out 277b0f7 
  ql/src/test/results/clientpositive/cast1.q.out 0bdecba 
  ql/src/test/results/clientpositive/cbo_const.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cbo_rp_cross_product_check_2.q.out 62b611b 
  ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out b14caa8 
  ql/src/test/results/clientpositive/constantfolding.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/cross_product_check_1.q.out e7d6900 
  ql/src/test/results/clientpositive/cross_product_check_2.q.out df438c9 
  ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out b85d387 
  ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 24ac550 
  ql/src/test/results/clientpositive/groupby_ppd.q.out 6164a26 
  ql/src/test/results/clientpositive/groupby_sort_1_23.q.out 7333677 
  ql/src/test/results/clientpositive/groupby_sort_skew_1_23.q.out e19d1de 
  ql/src/test/results/clientpositive/input_part1.q.out d6f4d3e 
  ql/src/test/results/clientpositive/input_part5.q.out f2d7335 
  ql/src/test/results/clientpositive/input_part6.q.out fa51cdf 
  ql/src/test/results/clientpositive/lineage2.q.out ec8b76b 
  ql/src/test/results/clientpositive/lineage3.q.out 747dc9a 
  ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out be77ba8 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out 79348f3 
  ql/src/test/results/clientpositive/orc_predicate_pushdown.q.out f6d8388 
  ql/src/test/results/clientpositive/parquet_predicate_pushdown.q.out b322ef1 
  ql/src/test/results/clientpositive/partition_multilevels.q.out c1c8778 
  ql/src/test/results/clientpositive/perf/query31.q.out c479d91 
  ql/src/test/results/clientpositive/perf/query39.q.out 0ad62cc 
  ql/src/test/results/clientpositive/perf/query42.q.out a3264c7 
  ql/src/test/results/clientpositive/perf/query52.q.out ac0c1e6 
  ql/src/test/results/clientpositive/perf/query64.q.out 9735b45 
  ql/src/test/results/clientpositive/perf/query66.q.out 42bcb83 
  ql/src/test/results/clientpositive/perf/query75.q.out 70a1649 
  ql/src/test/results/clientpositive/pointlookup2.q.out a442425 
  ql/src/test/results/clientpositive/quotedid_basic.q.out 519f647 
  ql/src/test/results/clientpositive/quotedid_partition.q.out d34a005 
  ql/src/test/results/clientpositive/spark/bucket_map_join_tez1.q.out b5e7846 
  ql/src/test/results/clientpositive/spark/cross_product_check_1.q.out 5b03dcf 
  ql/src/test/results/clientpositive/spark/cross_product_check_2.q.out 93c502d 
  ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out b597ebd 
  ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out 239e803 
  ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out 8370bbe 
  ql/src/test/results/clientpositive/spark/union_remove_25.q.out b771fe9 
  ql/src/test/results/clientpositive/spark/union_view.q.out cce7710 
  ql/src/test/results/clientpositive/subquery_notin.q.out ed86079 
  ql/src/test/results/clientpositive/subquery_notin_having.q.java1.7.out 
7b7ccda 
  ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out 3241787 
  ql/src/test/results/clientpositive/tez/bucket_map_join_tez1.q.out 9582334 
  ql/src/test/results/clientpositive/tez/cross_product_check_1.q.out 0e7c681 
  ql/src/test/results/clientpositive/tez/cross_product_check_2.q.out efd8b5d 
  ql/src/test/results/clientpositive/tez/dynpart_sort_optimization2.q.out 
346e52f 
  ql/src/test/results/clientpositive/tez/explainuser_1.q.out c081309 
  ql/src/test/res

Re: January Hive User Group Meeting

[Xuefu Zhang]

Yeah. I can try to set up a webex for this. However, I'd encourage folks to
attend in person to get more live experience, especially for those from
local Bay Area.

Thanks,
Xuefu

On Wed, Dec 16, 2015 at 3:42 PM, Mich Talebzadeh 
wrote:

> Thanks for heads up.
>
>
>
> Will it be possible to remote to this meetings for live sessions?
>
>
>
> Regards,
>
>
>
>
>
> Mich Talebzadeh
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Xuefu Zhang [mailto:xzh...@cloudera.com]
> *Sent:* 16 December 2015 23:39
> *To:* u...@hive.apache.org; dev@hive.apache.org; Thejas M Nair <
> thejas.n...@yahoo.com>
> *Subject:* January Hive User Group Meeting
>
>
>
> Dear Hive users and developers,
>
> Hive community is considering a user group meeting[1] January 21, 2016 at
> Cloudera facility in Palo Alto, CA. This will be a great opportunity for
> vast users and developers to find out what's happening in the community and
> share each other's experience with Hive. Therefore, I'd urge you to attend
> the meetup. Please RSVP and the list will be closed a few days ahead of the
> event.
>
> At the same time, I'd like to solicit light talks (15 minutes max) from
> users and developers. If you have a proposal, please let me or Thejas know.
> Your participation is greatly appreciated.
>
> Sincerely,
>
> Xuefu
>
> [1] http://www.meetup.com/Hive-User-Group-Meeting/events/227463783/
>

[jira] [Created] (HIVE-12698) Remove exposure to internal privilege and principal classes in HiveAuthorizer

[Thejas M Nair (JIRA)]

Thejas M Nair created HIVE-12698:


 Summary: Remove exposure to internal privilege and principal 
classes in HiveAuthorizer
 Key: HIVE-12698
 URL: https://issues.apache.org/jira/browse/HIVE-12698
 Project: Hive
  Issue Type: Bug
  Components: Authorization
Affects Versions: 1.3.0, 2.0.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Fix For: 1.3.0, 2.0.0


The changes in HIVE-11179 expose several internal classes to HiveAuthorization 
implementations. These include PrivilegeObjectDesc, PrivilegeDesc, 
PrincipalDesc and AuthorizationUtils.

We should avoid exposing that to all Authorization implementations, but also 
make the ability to customize the mapping of internal classes to the public api 
classes possible for Apache Sentry (incubating).





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 41482: StatsSetupConst.COLUMN_STATS_ACCURATE is not used correctly

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41482/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

HIVE-12661


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 2ff76ee 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
23068f8 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java f6fbe74 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 4fb6c00 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java c682df2 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java d2a5948 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java 03c1c3f 
  ql/src/test/queries/clientpositive/columnStatsUpdateForStatsOptimizer.q 
PRE-CREATION 
  ql/src/test/results/clientnegative/stats_partialscan_autogether.q.out d03c249 
  ql/src/test/results/clientnegative/unset_table_property.q.out 158ed38 
  ql/src/test/results/clientpositive/alter_file_format.q.out c9e88f8 
  ql/src/test/results/clientpositive/alter_merge_stats_orc.q.out cefe069 
  
ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2_h23.q.out
 29a4c4b 
  
ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table_h23.q.out 
4d0f841 
  ql/src/test/results/clientpositive/alter_partition_clusterby_sortby.q.out 
f5e8d1f 
  ql/src/test/results/clientpositive/alter_partition_coltype.q.out 08bd2b8 
  ql/src/test/results/clientpositive/alter_skewed_table.q.out 03904e6 
  ql/src/test/results/clientpositive/alter_table_not_sorted.q.out 40974e4 
  ql/src/test/results/clientpositive/alter_table_serde2.q.out 3b63e7d 
  ql/src/test/results/clientpositive/analyze_table_null_partition.q.out c7254d1 
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 91b89d7 
  ql/src/test/results/clientpositive/auto_sortmerge_join_1.q.out a48c148 
  ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out 206a619 
  ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 30853c4 
  ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out fbc500b 
  ql/src/test/results/clientpositive/auto_sortmerge_join_3.q.out 9ae6861 
  ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out 357bc4c 
  ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out b763e52 
  ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out cef3bd3 
  ql/src/test/results/clientpositive/auto_sortmerge_join_8.q.out 567e3cf 
  ql/src/test/results/clientpositive/binary_output_format.q.out 30d0037 
  ql/src/test/results/clientpositive/bucket1.q.out 8009514 
  ql/src/test/results/clientpositive/bucket2.q.out 75de8a9 
  ql/src/test/results/clientpositive/bucket3.q.out c459870 
  ql/src/test/results/clientpositive/bucket_map_join_1.q.out c7a8a20 
  ql/src/test/results/clientpositive/bucket_map_join_2.q.out 3c3793f 
  ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out 8411a9b 
  ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out 008d57a 
  ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out d5ca1ad 
  ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out 42abaa3 
  ql/src/test/results/clientpositive/bucketcontext_1.q.out d28e582 
  ql/src/test/results/clientpositive/bucketcontext_2.q.out 9f92030 
  ql/src/test/results/clientpositive/bucketcontext_3.q.out 5a847f0 
  ql/src/test/results/clientpositive/bucketcontext_4.q.out 19ed7f9 
  ql/src/test/results/clientpositive/bucketcontext_5.q.out e1a911b 
  ql/src/test/results/clientpositive/bucketcontext_6.q.out 0d3bf6e 
  ql/src/test/results/clientpositive/bucketcontext_7.q.out 15f1335 
  ql/src/test/results/clientpositive/bucketcontext_8.q.out 25d0c3c 
  ql/src/test/results/clientpositive/bucketmapjoin1.q.out 8999fe6 
  ql/src/test/results/clientpositive/bucketmapjoin10.q.out ec6069f 
  ql/src/test/results/clientpositive/bucketmapjoin11.q.out 15be0af 
  ql/src/test/results/clientpositive/bucketmapjoin12.q.out b87685c 
  ql/src/test/results/clientpositive/bucketmapjoin13.q.out cdb7637 
  ql/src/test/results/clientpositive/bucketmapjoin2.q.out fb0e601 
  ql/src/test/results/clientpositive/bucketmapjoin3.q.out e71c5e1 
  ql/src/test/results/clientpositive/bucketmapjoin4.q.out 7dd2202 
  ql/src/test/results/clientpositive/bucketmapjoin5.q.out 15969db 
  ql/src/test/results/clientpositive/bucketmapjoin8.q.out 0ef69ea 
  ql/src/test/results/clientpositive/bucketmapjoin9.q.out 02c5153 
  ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 124ccb3 
  ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 03d95a9 
  ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out b2b6d01 
  ql/src/test/results/clientpositive/cbo_rp_outer_join_ppr.q.java1.7.out 
2630537 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer.q

January Hive User Group Meeting

[Xuefu Zhang]

Dear Hive users and developers,

Hive community is considering a user group meeting[1] January 21, 2016 at
Cloudera facility in Palo Alto, CA. This will be a great opportunity for
vast users and developers to find out what's happening in the community and
share each other's experience with Hive. Therefore, I'd urge you to attend
the meetup. Please RSVP and the list will be closed a few days ahead of the
event.

At the same time, I'd like to solicit light talks (15 minutes max) from
users and developers. If you have a proposal, please let me or Thejas know.
Your participation is greatly appreciated.

Sincerely,
Xuefu

[1] http://www.meetup.com/Hive-User-Group-Meeting/events/227463783/

[jira] [Created] (HIVE-12697) Remove deprecated post option from webhcat test files

[Aswathy Chellammal Sreekumar (JIRA)]

Aswathy Chellammal Sreekumar created HIVE-12697:
---

 Summary: Remove deprecated post option from webhcat test files
 Key: HIVE-12697
 URL: https://issues.apache.org/jira/browse/HIVE-12697
 Project: Hive
  Issue Type: Test
  Components: WebHCat
Affects Versions: 2.0.0
Reporter: Aswathy Chellammal Sreekumar
Assignee: Aswathy Chellammal Sreekumar


Tests are still having the deprecated post option user.name. Need to remove 
them and add the same to query string



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12696) LlapServiceDriver can fail if only the packaged logger config is present

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-12696:
---

 Summary: LlapServiceDriver can fail if only the packaged logger 
config is present
 Key: HIVE-12696
 URL: https://issues.apache.org/jira/browse/HIVE-12696
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Priority: Minor


I was incrementally updating my setup on some VM and didn't have the logger 
config file, so the packaged one was picked up apparently, which caused this:
{noformat}
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path 
in absolute URI: 
jar:file:/home/vagrant/llap/apache-hive-2.0.0-SNAPSHOT-bin/lib/hive-llap-server-2.0.0-SNAPSHOT.jar!/llap-daemon-log4j2.properties
at org.apache.hadoop.fs.Path.initialize(Path.java:205)
at org.apache.hadoop.fs.Path.(Path.java:171)
at 
org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:234)
at 
org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:58)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
jar:file:/home/vagrant/llap/apache-hive-2.0.0-SNAPSHOT-bin/lib/hive-llap-server-2.0.0-SNAPSHOT.jar!/llap-daemon-log4j2.properties
at java.net.URI.checkPath(URI.java:1823)
at java.net.URI.(URI.java:745)
at org.apache.hadoop.fs.Path.initialize(Path.java:202)
... 3 more
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41387: HIVE-12675: PerfLogger should log performance metrics at debug level

[John Pullokkaran]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41387/#review110748
---

Ship it!


Ship It!

- John Pullokkaran


On Dec. 15, 2015, 2:26 a.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41387/
> ---
> 
> (Updated Dec. 15, 2015, 2:26 a.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> PerfLogger should log performance metrics at debug level. Please see the jira.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java f6eb64b 
> 
> Diff: https://reviews.apache.org/r/41387/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

[jira] [Created] (HIVE-12695) LLAP: use somebody else's cluster

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-12695:
---

 Summary: LLAP: use somebody else's cluster
 Key: HIVE-12695
 URL: https://issues.apache.org/jira/browse/HIVE-12695
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin


For non-HS2 case cluster sharing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12694) LLAP: Slider destroy semantics require force

[Gopal V (JIRA)]

Gopal V created HIVE-12694:
--

 Summary: LLAP: Slider destroy semantics require force
 Key: HIVE-12694
 URL: https://issues.apache.org/jira/browse/HIVE-12694
 Project: Hive
  Issue Type: Bug
  Components: llap
Affects Versions: 2.0.0, 2.1.0
Reporter: Gopal V
Assignee: Gopal V


{code}
2015-12-16 20:10:55,118 [main] ERROR main.ServiceLauncher - Destroy will 
permanently delete directories and registries. Reissue this command with the 
--force option if you want to proceed.
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Some unit tests fail locally

[Wei Zheng]

Mohit and I ran into some unit test failures when running them locally, e.g.

TestSessionHooks
TestPlainSaslHelper
TestSessionGlobalInitFile
TestTxnCommands
TestDbTxnManager

I did a git bisect and looks like it's due to HIVE-12628. This JIRA introduced 
hive-common dependency for ql, and since there's a hive-site.xml under 
hive-common, the failing tests mentioned above picked up this wrong 
hive-site.xml.

But I don't understand why those tests didn't fail on ptest.

Tickets opened for this issue:
HIVE-12685
HIVE-12670

Thanks,
Wei

Review Request 41457: HIVE-11865

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41457/
---

Review request for hive, Ashutosh Chauhan and John Pullokkaran.


Bugs: HIVE-11865
https://issues.apache.org/jira/browse/HIVE-11865


Repository: hive-git


Description
---

Disable Hive PPD optimizer when CBO has optimized the plan


Diffs
-

  hbase-handler/src/test/results/positive/hbase_pushdown.q.out 
be96eecc8642f85a493ae6c72964bc4e30c4db57 
  hbase-handler/src/test/results/positive/hbase_queries.q.out 
3907bc946679c0ce2492692dcb7e2cf252c6e188 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 
8f48e7df4b9c081528b82545d445f75f67f93bfc 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java
 db625c1c3d47b5ee2744950db2fb71804945cbec 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTSTransposeRule.java
 8321504774c5ec5c7c9de3aecdf78dcd8408c4cf 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTransposeRule.java
 7e484b90deed2dfb120faa2699eb9b9b7a4e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterSortTransposeRule.java
 PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinAddNotNullRule.java
 2a415d5abd9c054956f46cd26f9df2257d67ecf4 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortMergeRule.java
 efde6652ca270d8e8d8f682fc0ff260702f46d5a 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortProjectTransposeRule.java
 feec3c25929cac0bb3b6f50d074557a3c2745889 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
36a12bf6b2c3ef7233eda7c949725e02c880d7f7 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
27549dceac8b5913b1fb1604bde213ae26b243bf 
  ql/src/java/org/apache/hadoop/hive/ql/ppd/SimplePredicatePushDown.java 
PRE-CREATION 
  ql/src/test/results/clientpositive/auto_join12.q.out 
6f08aa8f043113f0a99acd10c848f60a122a3f0b 
  ql/src/test/results/clientpositive/auto_join13.q.out 
d29818cf438449a6a1413edd7d474c4ab35f2148 
  ql/src/test/results/clientpositive/auto_join16.q.out 
38738ede2b4cd49aa24490eafb81dec1de78bea5 
  ql/src/test/results/clientpositive/auto_join2.q.out 
ce40ff30cb9cc2ab734ee37c0f3dc0ae85bbd202 
  ql/src/test/results/clientpositive/auto_join33.q.out 
8b13bd0cb7fd869c58c9d092b2e8710391c96503 
  ql/src/test/results/clientpositive/auto_join8.q.out 
80dd575c58add873ae34967b1d255ee5b63a9b07 
  ql/src/test/results/clientpositive/auto_join_filters.q.out 
e0ed3737f9b25d45bdb365b13aeed3a40c50135f 
  ql/src/test/results/clientpositive/auto_join_nulls.q.out 
954bf069760c6f0d7e5420c7e9a3b9756d5aa4df 
  ql/src/test/results/clientpositive/auto_join_stats.q.out 
feb818679df7d9f63f2d2d7a7c9afb2d0f5874d8 
  ql/src/test/results/clientpositive/auto_join_stats2.q.out 
e0d0146dd6b86e9f69a467dcde0c020c31af499d 
  ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 
c63016d0a3610eaa6cd82a4adc5d1bf6f0925cce 
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out 
096f5d445bc7e61381775270bef04e911eac5bb6 
  ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 
30853c42a52e1def5a913352580e0b5391ac2050 
  ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out 
3e8dbcfd5bcc66ccd83b01c08c7745b02f05328f 
  ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out 
e6c2832003a1bb804f99125866bff6117e4dab73 
  ql/src/test/results/clientpositive/cbo_rp_auto_join1.q.out 
5c09fe0ad2d30c6c597abdad27b309fd8d00f678 
  ql/src/test/results/clientpositive/cbo_rp_join0.q.out 
3c6bb7319fd7605a62f082b989f723d5aab6c3fe 
  ql/src/test/results/clientpositive/cbo_rp_join1.q.out 
e77002866174f85168859c4145e7016c9d829bb6 
  ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out 
b14caa80b3346036f9ec835fe4e2294f47eb7ee5 
  ql/src/test/results/clientpositive/cbo_rp_outer_join_ppr.q.java1.7.out 
263053705347793513a44d49968dedd4fdffecfe 
  ql/src/test/results/clientpositive/constprog_partitioner.q.out 
08c0aebafe1644a5adbec6a56879b39617bfbf69 
  ql/src/test/results/clientpositive/cross_join.q.out 
f36496e8503fedc67d6c9555b6e0dd5b0c9bd6ed 
  ql/src/test/results/clientpositive/cross_product_check_1.q.out 
e7d6900d64851fa21f976ca20b5d7d753aa5d28f 
  ql/src/test/results/clientpositive/cross_product_check_2.q.out 
df438c9ae7c95033383eae28bf8bada55c08b5c4 
  ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 
b85d387f3910598a3300df9adc34dce952715a2e 
  ql/src/test/results/clientpositive/explain_logical.q.out 
d302918ceddd0fe10ea16d1fb32379bc823a158d 
  ql/src/test/results/clientpositive/filter_cond_pushdown.q.out 
b1cfb25a8fba4e5d3941d1615418ae68e8eaa6bf 
  ql/src/test/results/clientpositive/fold_eq_with_case_when.q.out 
231d98f0faaa1ff5a8368648ea62e7632301c391 
  ql/src/test/results/clientpositive/fouter_join_

[jira] [Created] (HIVE-12693) LLAP: Use Slider Anti-Affinity scheduling mode for daemon distribution

[Gopal V (JIRA)]

Gopal V created HIVE-12693:
--

 Summary: LLAP: Use Slider Anti-Affinity scheduling mode for daemon 
distribution
 Key: HIVE-12693
 URL: https://issues.apache.org/jira/browse/HIVE-12693
 Project: Hive
  Issue Type: Improvement
  Components: llap
Affects Versions: 2.0.0, 2.1.0
Reporter: Gopal V
Assignee: Gopal V


Slider has SLIDER-82 which adds anti-affinity placement policies for 
containers, to avoid colliding on to the same machine when deploying LLAP 
instances.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12692) Make use of the Tez HadoopShim in TaskRunner usage

[Siddharth Seth (JIRA)]

Siddharth Seth created HIVE-12692:
-

 Summary: Make use of the Tez HadoopShim in TaskRunner usage
 Key: HIVE-12692
 URL: https://issues.apache.org/jira/browse/HIVE-12692
 Project: Hive
  Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: Siddharth Seth
Assignee: Siddharth Seth


TEZ-2910 adds shims for Hadoop to make use of caller context and other changing 
hadoop APIs. Hive usage of TezTaskRunner needs to work with this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40589: Implement limit push down through union all in CBO

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40589/
---

(Updated Dec. 16, 2015, 6:46 p.m.)


Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

HIVE-11775


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 243f281 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortUnionReduceRule.java
 PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 36a12bf 
  ql/src/test/queries/clientpositive/cbo_SortUnionTransposeRule.q PRE-CREATION 
  ql/src/test/queries/clientpositive/limit_join_transpose.q 80430c6 
  ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_3.q cecbbd7 
  ql/src/test/results/clientpositive/cbo_SortUnionTransposeRule.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/40589/diff/


Testing
---


Thanks,

pengcheng xiong

[jira] [Created] (HIVE-12691) Compute stats on hbase tables causes Zookeeper connection leaks.

[Naveen Gangam (JIRA)]

Naveen Gangam created HIVE-12691:


 Summary: Compute stats on hbase tables causes Zookeeper connection 
leaks.
 Key: HIVE-12691
 URL: https://issues.apache.org/jira/browse/HIVE-12691
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 1.1.0
Reporter: Naveen Gangam
Assignee: Naveen Gangam


hive.stats.autogather defaults to true in newer hive releases which causes 
stats to be collected on hbase-backed hive tables.

Using HTable APIs causes a new zookeeper connections to be created. So if 
HTable.close() is not called, the underlying ZK connection remains open as in 
HIVE-12250.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: HIVE-TRUNK-JAVA8 #144

[hiveqa]

See 


Changes:

[jcamacho] HIVE-12662: StackOverflowError in HiveSortJoinReduceRule when 
limit=0 (Jesus Camacho Rodriguez, reviewed by Pengcheng Xiong)

[Szehon Ho] HIVE-12628 : Eliminate flakiness in TestMetrics (Szehon, reviewed 
by Jimmy Xiang)

[Alan Gates] HIVE-10982 Customizable the value of 
java.sql.statement.setFetchSize in Hive JDBC Driver (Bing Li via Alan Gates)

[ekoifman] HIVE-12620 Misc improvement to Acid module(Eugene Koifman, reviewed 
by Wei Zheng, Jason Dere)

[omalley] HIVE-12055. Move WriterImpl over to orc module.

[mmccline] HIVE-12435 SELECT COUNT(CASE WHEN...) GROUPBY returns 1 for 'NULL' 
in a case of ORC and vectorization is enabled. (Matt McCline, reviewed by 
Prasanth J)

[sershe] HIVE-12668 : package script for LLAP was broken by recent config 
changes (Sergey Shelukhin, reviewed by Prasanth Jayachandran)

[harisankar] HIVE-12570 : Incorrect error message Expression not in GROUP BY 
key thrown instead of Invalid function (Hari Subramaniyan, reviewed by Laljo 
John Pullokkaran)

[harisankar] HIVE-12526 : PerfLogger for hive compiler and optimizer (Hari 
Subramaniyan, reviewed by Jesus Camacho Rodriguez)

[pxiong] HIVE-12663: Support quoted table names/columns when ACID is on 
(Pengcheng Xiong, reviewed by Eugene Koifman)

[harisankar] HIVE-12666: PCRExprProcFactory.GenericFuncExprProcessor.process() 
aggressively removes dynamic partition pruner generated synthetic join 
predicates (Hari Subramaniyan, reviewed by Laljo John Pullokkaran)

[Szehon Ho] HIVE-12431 : Support timeout for compile lock (Mohit Sabharwal via 
Szehon)

[harisankar] HIVE-11107 : Support for Performance regression test suite with 
TPCDS (Hari Subramaniyan, reviewed by Ashutosh Chauhan)

--
[...truncated 310 lines...]
A
ptest2/src/test/java/org/apache/hive/ptest/execution/conf/TestTestParser.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestPhase.testRsyncFromLocalToRemoteInstancesWithFailureOne.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestHostExecutor.testIsolatedFailsOnRsyncOne.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.testFailingUnitTest.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestPhase.testExecInstancesWithFailure.approved.txt
Aptest2/src/test/java/org/apache/hive/ptest/execution/ssh
A
ptest2/src/test/java/org/apache/hive/ptest/execution/ssh/TestRSyncCommandExecutor.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/ssh/TestSSHCommandExecutor.java
Aptest2/src/test/java/org/apache/hive/ptest/execution/AbstractTestPhase.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestScripts.testBatch.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.testPassingQFileTest.approved.txt
A
ptest2/src/test/java/org/apache/hive/ptest/execution/MockSSHCommandExecutor.java
A
ptest2/src/test/java/org/apache/hive/ptest/execution/MockRSyncCommandExecutor.java
Aptest2/src/test/java/org/apache/hive/ptest/api
Aptest2/src/test/java/org/apache/hive/ptest/api/server
Aptest2/src/test/java/org/apache/hive/ptest/api/server/TestTestExecutor.java
Aptest2/src/test/java/org/apache/hive/ptest/api/server/TestTestLogger.java
Aptest2/src/test/resources
Aptest2/src/test/resources/test-configuration.properties
Aptest2/src/test/resources/test-outputs
A
ptest2/src/test/resources/test-outputs/index_auth.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
A
ptest2/src/test/resources/test-outputs/skewjoin.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
A
ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TestCliDriver.txt
A
ptest2/src/test/resources/test-outputs/union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-hive.log
A
ptest2/src/test/resources/test-outputs/union_remove_9.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
Aptest2/src/test/resources/test-outputs/SomeTest-truncated.xml
A
ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
A
ptest2/src/test/resources/test-outputs/skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-hive.log
A
ptest2/src/test/resources/test-outputs/skewjoin_union_remove_1.q-TestCliDriver.txt
A
ptest2/src/test/resources/test-outputs/union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TestCliDriver.txt
A
ptest2/src/test/resources/test-outputs/index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-hive.log
Aptest2/src/test/resources/test-outputs/index_auth.q-hive.log
A
ptest2/src/test/resources/test-outputs/skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TestCliDriver.txt
A
ptest2/s

Re: Review Request 37925: HIVE-11487: Add getNumPartitionsByFilter api in metastore api

[Akshay Goyal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37925/
---

(Updated Dec. 16, 2015, 12:18 p.m.)


Review request for hive, Amareshwari Sriramadasu and Sergey Shelukhin.


Changes
---

Added another test case with filter and updated with trunk


Bugs: HIVE-11487
https://issues.apache.org/jira/browse/HIVE-11487


Repository: hive-git


Description
---

Adding api for getting number of partitions for a filter. In order to be more 
optimal when only interested in the number. getAllPartitions will construct all 
the partition object which can be time consuming and not required.


Diffs (updated)
-

  metastore/if/hive_metastore.thrift bb754f1 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
0940fd7 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
c5e7a5f 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
aa96f77 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
36b316a 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java abfe2b8 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java e118a3b 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
b9509ab 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 c1156b3 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 bf20e99 
  metastore/src/test/org/apache/hadoop/hive/metastore/TestObjectStore.java 
9089d1c 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java c682df2 

Diff: https://reviews.apache.org/r/37925/diff/


Testing
---


Thanks,

Akshay Goyal

[jira] [Created] (HIVE-12690) 'skip-header' only works on the first file split

[JIRA]

靳峥 created HIVE-12690:
-

 Summary: 'skip-header' only works on the first file split
 Key: HIVE-12690
 URL: https://issues.apache.org/jira/browse/HIVE-12690
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 0.14.0
Reporter: 靳峥
 Fix For: 0.15.0


the problem of HIVE-5795



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)