Re: Review Request 41807: further improve user level explain

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41807/
---

(Updated Jan. 21, 2016, 7:46 a.m.)


Review request for hive and John Pullokkaran.


Repository: hive-git


Description
---

HIVE-12709


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Attr.java 
5d355d2 
  common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Op.java d0c1037 
  common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Stage.java 
455d59f 
  
common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParser.java
 b193fef 
  
common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParserUtils.java
 PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Vertex.java 
be01b8b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java f48db6a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java bc67e5a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/BaseWork.java ad34b98 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 3f38f74 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableLikeDesc.java f051712 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java 40a8477 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java d04cb78 
  ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 0f2855e 
  ql/src/java/org/apache/hadoop/hive/ql/plan/HashTableSinkDesc.java 7c8eee2 
  ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java eb83fd6 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewJoinDesc.java 12f01e5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 427aac1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java 4b93e7c 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 5bea6fb 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 2f69b7f 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceWork.java 1c31962 
  ql/src/java/org/apache/hadoop/hive/ql/plan/SelectDesc.java e7bbab4 
  ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 4e52bac 
  ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java 2fdb0a1 
  ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 098aa89 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFQueryInputDef.java ae55741 
  
ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PartitionedTableFunctionDef.java 
46f106c 
  ql/src/test/queries/clientpositive/explainuser_4.q PRE-CREATION 
  ql/src/test/results/clientpositive/perf/query12.q.out 648d64d 
  ql/src/test/results/clientpositive/perf/query13.q.out 6113d18 
  ql/src/test/results/clientpositive/perf/query15.q.out c237828 
  ql/src/test/results/clientpositive/perf/query17.q.out 0e42d05 
  ql/src/test/results/clientpositive/perf/query18.q.out 83f21dc 
  ql/src/test/results/clientpositive/perf/query19.q.out d565839 
  ql/src/test/results/clientpositive/perf/query20.q.out 40c9a17 
  ql/src/test/results/clientpositive/perf/query21.q.out 2634d3d 
  ql/src/test/results/clientpositive/perf/query22.q.out 72935ed 
  ql/src/test/results/clientpositive/perf/query25.q.out dabc1c5 
  ql/src/test/results/clientpositive/perf/query26.q.out 84cc238 
  ql/src/test/results/clientpositive/perf/query27.q.out ac38683 
  ql/src/test/results/clientpositive/perf/query28.q.out cac9ac3 
  ql/src/test/results/clientpositive/perf/query29.q.out 8991f35 
  ql/src/test/results/clientpositive/perf/query3.q.out 871ad74 
  ql/src/test/results/clientpositive/perf/query31.q.out 909d64c 
  ql/src/test/results/clientpositive/perf/query32.q.out da2ac8f 
  ql/src/test/results/clientpositive/perf/query34.q.out 1cb7c28 
  ql/src/test/results/clientpositive/perf/query39.q.out 9f3e650 
  ql/src/test/results/clientpositive/perf/query40.q.out b2d6262 
  ql/src/test/results/clientpositive/perf/query42.q.out 9ede45d 
  ql/src/test/results/clientpositive/perf/query43.q.out 7fa4f53 
  ql/src/test/results/clientpositive/perf/query45.q.out f55f20c 
  ql/src/test/results/clientpositive/perf/query46.q.out 0ded912 
  ql/src/test/results/clientpositive/perf/query48.q.out 40b76f0 
  ql/src/test/results/clientpositive/perf/query50.q.out 8b41d05 
  ql/src/test/results/clientpositive/perf/query51.q.out f57af2c 
  ql/src/test/results/clientpositive/perf/query52.q.out 7e7224b 
  ql/src/test/results/clientpositive/perf/query54.q.out 56c70b7 
  ql/src/test/results/clientpositive/perf/query55.q.out d36f378 
  ql/src/test/results/clientpositive/perf/query58.q.out c2f7189 
  ql/src/test/results/clientpositive/perf/query64.q.out 9331673 
  ql/src/test/results/clientpositive/perf/query65.q.out 12bdd33 
  ql/src/test/results/clientpositive/perf/query66.q.out a25664e 
  ql/src/test/results/clientpositive/perf/query67.q.out dda5347 
  ql/src/test/results/clientpositive/perf/query68.q.out 8ecde89 
  ql/src/test/results/clientpositive/perf/query7.q.out

[jira] [Created] (HIVE-12897) Improve dynamic partition loading

[Ashutosh Chauhan (JIRA)]

Ashutosh Chauhan created HIVE-12897:
---

 Summary: Improve dynamic partition loading
 Key: HIVE-12897
 URL: https://issues.apache.org/jira/browse/HIVE-12897
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
 Attachments: HIVE-12897.patch

There are many redundant calls to metastore which is not needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41377: HIVE-12528 don't start HS2 Tez sessions in a single thread

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41377/
---

(Updated Jan. 21, 2016, 12:27 a.m.)


Review request for hive, Gunther Hagleitner and Siddharth Seth.


Repository: hive-git


Description
---

see JIRA


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 32049eb 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java e8864ae 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
3bfe35a 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezSessionPool.java 
a2791a1 

Diff: https://reviews.apache.org/r/41377/diff/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 42359: HIVE-12805: CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver skewjoin.q failure

[Hari Sankar Sivarama Subramaniyan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42359/
---

(Updated Jan. 21, 2016, 12:23 a.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Changes
---

Addressed Ashutosh's comment


Repository: hive-git


Description
---

HIVE-12805 CBO: Calcite Operator To Hive Operator (Calcite Return Path): 
MiniTezCliDriver skewjoin.q failure


Diffs (updated)
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveMultiJoin.java
 660f01d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveInsertExchange4JoinRule.java
 d6e3915 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinToMultiJoinRule.java
 5d169a1 

Diff: https://reviews.apache.org/r/42359/diff/


Testing
---

Clean precommit


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 42359: HIVE-12805: CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver skewjoin.q failure

[Hari Sankar Sivarama Subramaniyan]

> On Jan. 20, 2016, 10:18 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinToMultiJoinRule.java,
> >  line 263
> > 
> >
> > Do we really need to find uniqueness of expression?
> > Can we do numKeys = joinLeafPredInfo.getJoinExprs(i).size() ?

Makes sense, apparently joinKeyExprs themselves are implemented using 
ImmutableSet which takes care of duplicates. Have made change in the next patch 
upload.


- Hari Sankar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42359/#review115499
---


On Jan. 19, 2016, 9 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42359/
> ---
> 
> (Updated Jan. 19, 2016, 9 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-12805 CBO: Calcite Operator To Hive Operator (Calcite Return Path): 
> MiniTezCliDriver skewjoin.q failure
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveMultiJoin.java
>  660f01d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveInsertExchange4JoinRule.java
>  d6e3915 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinToMultiJoinRule.java
>  5d169a1 
> 
> Diff: https://reviews.apache.org/r/42359/diff/
> 
> 
> Testing
> ---
> 
> Clean precommit
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 42322: HIVE-12864

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42322/#review115506
---



ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (line 11185)


The patch looks good to me +1. Just one small thing and it is up to you to 
modify it or not. Although Stack works here, with the same time/space 
complexity, I would prefer a simple Queue. First dequeue the head, then enqueue 
the children from left to right. Right now a Stack is used and the children are 
pushed right to left...



ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryUtils.java (line 201)


The same with above comments.


- pengcheng xiong


On Jan. 14, 2016, 9:23 p.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42322/
> ---
> 
> (Updated Jan. 14, 2016, 9:23 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and John Pullokkaran.
> 
> 
> Bugs: HIVE-12864
> https://issues.apache.org/jira/browse/HIVE-12864
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> StackOverflowError parsing queries with very large predicates
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ASTNode.java 
> 8400ee8cb21e9d8896c9b4f3b50058ed2cb6d983 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> c38699dc80a3e9bee1f5298ef28870aa373d90d3 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryUtils.java 
> 362a285a8499d82064aa1ae84d811a8700cd3199 
> 
> Diff: https://reviews.apache.org/r/42322/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jesús Camacho Rodríguez
> 
>

[jira] [Created] (HIVE-12896) IndexArrayOutOfBoundsException during vectorized map join

[Jason Dere (JIRA)]

Jason Dere created HIVE-12896:
-

 Summary: IndexArrayOutOfBoundsException during vectorized map join
 Key: HIVE-12896
 URL: https://issues.apache.org/jira/browse/HIVE-12896
 Project: Hive
  Issue Type: Bug
  Components: Vectorization
Affects Versions: 2.0.0
Reporter: Jason Dere
Assignee: Matt McCline


Trying a simple join on a couple of the TPCDS tables. Query works with 
vectorization disabled.
{noformat}
select * from tpcds_bin_partitioned_orc_10.customer, 
tpcds_bin_partitioned_orc_10.customer_demographics where c_current_cdemo_sk = 
cd_demo_sk limit 20;
{noformat}

{noformat}
], TaskAttempt 3 failed, info=[Error: Failure while running task: 
attempt_1448429572030_8225_4_01_03_3:java.lang.RuntimeException: 
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
Hive Runtime Error while processing row 
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:195)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:160)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row 
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:354)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:172)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error 
while processing row 
at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
java.lang.ArrayIndexOutOfBoundsException
at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:385)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:115)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:114)
at 
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:168)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45)
... 18 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
at 
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:152)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow$StringReaderByValue.apply(VectorDeserializeRow.java:345)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeByValue(VectorDeserializeRow.java:684)
at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultSingleValue(VectorMapJoinGenerateResultOperator.java:183)
at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerGenerateResultOperator.finishInner(VectorMapJoinInnerGenerateResultOperator.java:180)
at 
org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:373)
... 24 more
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 42359: HIVE-12805: CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver skewjoin.q failure

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42359/#review115499
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinToMultiJoinRule.java
 (line 263)


Do we really need to find uniqueness of expression?
Can we do numKeys = joinLeafPredInfo.getJoinExprs(i).size() ?


- Ashutosh Chauhan


On Jan. 19, 2016, 9 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42359/
> ---
> 
> (Updated Jan. 19, 2016, 9 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-12805 CBO: Calcite Operator To Hive Operator (Calcite Return Path): 
> MiniTezCliDriver skewjoin.q failure
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveMultiJoin.java
>  660f01d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveInsertExchange4JoinRule.java
>  d6e3915 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinToMultiJoinRule.java
>  5d169a1 
> 
> Diff: https://reviews.apache.org/r/42359/diff/
> 
> 
> Testing
> ---
> 
> Clean precommit
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

[jira] [Created] (HIVE-12895) Bucket files not renamed with multiple insert overwrite table statements

[Charles Pritchard (JIRA)]

Charles Pritchard created HIVE-12895:


 Summary: Bucket files not renamed with multiple insert overwrite 
table statements
 Key: HIVE-12895
 URL: https://issues.apache.org/jira/browse/HIVE-12895
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Charles Pritchard


With two tables that have different cluster by columns, using multiple INSERT 
OVERWRITE TABLE syntax results in the output files of one of the tables being 
named "_bucket_number_0", which is not picked up by analyzer/select later on.


This is with:
Tbl1: CLUSTERED BY (col1) SORTED BY(col1) INTO 1 BUCKETS;
Tbl2: CLUSTERED BY (col2) SORTED BY(col2) INTO 1 BUCKETS;

FROM statement
INSERT OVERWRITE TABLE tbl1 select...
INSERT OVERWRITE TABLE tbl2 select...;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41377: HIVE-12528 don't start HS2 Tez sessions in a single thread

[Siddharth Seth]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41377/#review115481
---

Ship it!


Other than the minor changes around the comment, and Exception propagation - 
looks good to me.

- Siddharth Seth


On Jan. 4, 2016, 11:12 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41377/
> ---
> 
> (Updated Jan. 4, 2016, 11:12 p.m.)
> 
> 
> Review request for hive, Gunther Hagleitner and Siddharth Seth.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see JIRA
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ffe0d9a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java e8864ae 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
> 3bfe35a 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezSessionPool.java 
> a2791a1 
> 
> Diff: https://reviews.apache.org/r/41377/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 41377: HIVE-12528 don't start HS2 Tez sessions in a single thread

[Siddharth Seth]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41377/#review115473
---



ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java (line 
145)


This exception is not propagated anywhere ? In the single thread case - an 
Exception from start is propagated upwards (where I'm assuming it is handled to 
shutdown HiveServer or some such).
Thread uncaughtExceptionHandler / Callable with a ThreadPoolExecutor ?



ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java (line 
206)


Is this statement still valid ? If not, please delete or modify accordingly.


- Siddharth Seth


On Jan. 4, 2016, 11:12 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41377/
> ---
> 
> (Updated Jan. 4, 2016, 11:12 p.m.)
> 
> 
> Review request for hive, Gunther Hagleitner and Siddharth Seth.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see JIRA
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ffe0d9a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java e8864ae 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
> 3bfe35a 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezSessionPool.java 
> a2791a1 
> 
> Diff: https://reviews.apache.org/r/41377/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: January Hive User Group Meeting

[Dmitry Tolpeko]

If possible please share the slides after the meeting.

Thank you!

Dmitry

On Wed, Jan 20, 2016 at 8:45 PM, Xuefu Zhang  wrote:

> Hi all,
>
> As a reminder, the meeting will be held tomorrow as scheduled. Please
> refer to the meetup page[1] for details. Looking forward to meeting you all!
>
> Thanks,
> Xuefu
>
> [1] http://www.meetup.com/Hive-User-Group-Meeting/events/227463783/
>
> On Wed, Dec 16, 2015 at 3:38 PM, Xuefu Zhang  wrote:
>
>> Dear Hive users and developers,
>>
>> Hive community is considering a user group meeting[1] January 21, 2016
>> at Cloudera facility in Palo Alto, CA. This will be a great opportunity
>> for vast users and developers to find out what's happening in the
>> community and share each other's experience with Hive. Therefore, I'd urge
>> you to attend the meetup. Please RSVP and the list will be closed a few
>> days ahead of the event.
>>
>> At the same time, I'd like to solicit light talks (15 minutes max) from
>> users and developers. If you have a proposal, please let me or Thejas know.
>> Your participation is greatly appreciated.
>>
>> Sincerely,
>> Xuefu
>>
>> [1] http://www.meetup.com/Hive-User-Group-Meeting/events/227463783/
>>
>
>

Re: January Hive User Group Meeting

[Xuefu Zhang]

Hi all,

As a reminder, the meeting will be held tomorrow as scheduled. Please refer
to the meetup page[1] for details. Looking forward to meeting you all!

Thanks,
Xuefu

[1] http://www.meetup.com/Hive-User-Group-Meeting/events/227463783/

On Wed, Dec 16, 2015 at 3:38 PM, Xuefu Zhang  wrote:

> Dear Hive users and developers,
>
> Hive community is considering a user group meeting[1] January 21, 2016 at
> Cloudera facility in Palo Alto, CA. This will be a great opportunity for
> vast users and developers to find out what's happening in the community
> and share each other's experience with Hive. Therefore, I'd urge you to
> attend the meetup. Please RSVP and the list will be closed a few days ahead
> of the event.
>
> At the same time, I'd like to solicit light talks (15 minutes max) from
> users and developers. If you have a proposal, please let me or Thejas know.
> Your participation is greatly appreciated.
>
> Sincerely,
> Xuefu
>
> [1] http://www.meetup.com/Hive-User-Group-Meeting/events/227463783/
>

Re: Review Request 42508: HIVE-12889: Support COUNT(DISTINCT) for partitioning query.

[Aihua Xu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42508/
---

(Updated Jan. 20, 2016, 5:05 p.m.)


Review request for hive, Chaoyu Tang, Szehon Ho, and Xuefu Zhang.


Repository: hive-git


Description
---

HIVE-12889: Support COUNT(DISTINCT) for partitioning query.


Diffs (updated)
-

  data/files/windowing_distinct.txt PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/functions/HiveSqlCountAggFunction.java
 7937040 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/functions/HiveSqlSumAggFunction.java
 8f62970 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java
 e2fbb4f 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java
 37249f9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 3fefbd7 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 15ca754 
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFInvocationSpec.java 29b8510 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 15773e5 
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingSpec.java a181f7c 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCount.java 
eaf112e 
  ql/src/test/queries/clientpositive/windowing_distinct.q PRE-CREATION 
  ql/src/test/results/clientpositive/windowing_distinct.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/42508/diff/


Testing
---

Support count(distinct) over partitioning window. 

1. Enabling the parser to properly parse such query "count(distinct) over 
(partition by c1)";
2. ORDER BY and windowing frame won't work with the functions of distinct due 
to performance concern and implementation requirement.
3. We insert the distinct fields into the order by list, so during counting, we 
only need to compare the current row against the previous remembered row.


Thanks,

Aihua Xu

Re: Review Request 42190: HIVE-12478

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42190/
---

(Updated Jan. 20, 2016, 3:39 p.m.)


Review request for hive and John Pullokkaran.


Bugs: HIVE-12478
https://issues.apache.org/jira/browse/HIVE-12478


Repository: hive-git


Description
---

Improve Hive/Calcite Trasitive Predicate inference


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java 
4825a617876374085b6fac1192ba1531ec916bce 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveHepPlannerContext.java
 ad79aeec2fbc0454ab1ccc608944752d01324dca 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveVolcanoPlannerContext.java
 8859fc268666cef1be283a9179aa0beb7ef1bdeb 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java 
d15d885d2348d666df069228a93d6c5f914c79df 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveVolcanoPlanner.java
 8610edc5ddc00d523610fb29f5e504c3e876a542 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java
 27b1e76a104dc961cb4bce554602d90b3aa867e0 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveSemiJoin.java
 35586768c2c2b81e4213495632e4457dd3d70443 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveUnion.java
 8b57b3504c407b8a1e73d48ea240c4ec7558b327 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinAddNotNullRule.java
 de880ce26f1e172288f700c8566fbe71f42af115 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinPushTransitivePredicatesRule.java
 703c8c6dbdfa281443cbcf7b08de2266697da8a9 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePreFilteringRule.java
 d37fc0e08d5e41b29539a990e6638385c1135eec 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRulesRegistry.java
 18a065e87e1ec266bf28b4ccfe10a1f863f847c2 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
3fefbd710c4bb81d5f746cd91889b532b0a6029f 
  
ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java
 f1d8d1de00e9de7fa9ffea7d3aa2400e5073ac9c 
  ql/src/test/results/clientpositive/annotate_stats_select.q.out 
b158d8567f6cc02d990d175e93996239aba0c5ed 
  ql/src/test/results/clientpositive/auto_join12.q.out 
8ef3664764d04f53f3685d8f66dc4a353776a488 
  ql/src/test/results/clientpositive/auto_join16.q.out 
c1da6d2968697d304311044d358f1af267dc6e60 
  ql/src/test/results/clientpositive/bucketizedhiveinputformat.q.out 
cfb95be72b32a354faeddc79dc52bc29c7593a2b 
  ql/src/test/results/clientpositive/cast1.q.out 
48a0c14031ef38dd5b4df7efa718a4d6ce04bc94 
  ql/src/test/results/clientpositive/cbo_const.q.out 
adc5232a67b2243dd5f09acaf1f7c49baea5daad 
  ql/src/test/results/clientpositive/cbo_rp_cross_product_check_2.q.out 
f1707eb4146c55338500c1fcaf4ff7199750250f 
  ql/src/test/results/clientpositive/cbo_rp_lineage2.q.out 
1b2a2ab1af5992753c37d053942ecb2ebf775921 
  ql/src/test/results/clientpositive/constprog_partitioner.q.out 
08c0aebafe1644a5adbec6a56879b39617bfbf69 
  ql/src/test/results/clientpositive/correlationoptimizer13.q.out 
61b7bcb675a80dc3267365c65545e107ecbe12d2 
  ql/src/test/results/clientpositive/correlationoptimizer8.q.out 
368a1147d404ae9aab18a64618abf8ed7b3f7189 
  ql/src/test/results/clientpositive/correlationoptimizer9.q.out 
104a97afbb6657255a027ff6d51101f6963c2358 
  ql/src/test/results/clientpositive/cross_product_check_1.q.out 
05eb270a292ecd9fe03db77511a7e2a08a28204c 
  ql/src/test/results/clientpositive/cross_product_check_2.q.out 
a36560f23c1801f0bf988fa4e08dcf49376d8577 
  ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 
743865e073e1768f7f6234fde35ad2e56d8babe9 
  ql/src/test/results/clientpositive/filter_join_breaktask.q.out 
53e9031d4219911f314bb31b74fe4931a32c65b0 
  ql/src/test/results/clientpositive/groupby_position.q.out 
c2566f2ad5d426db8814b7ca85207807f4a5afc7 
  ql/src/test/results/clientpositive/groupby_ppd.q.out 
d17c4b6d9e88d49d64b9725be1e51ca258ad1167 
  ql/src/test/results/clientpositive/index_auto_mult_tables.q.out 
8c71925fbc5352f42abb280fc45a1d5a5713ebad 
  ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out 
b3e698927689dad5229165189779300cb8beca9b 
  ql/src/test/results/clientpositive/input_part1.q.out 
501f7a9f87c66a1dd1d0fec0546597d810c0dbe9 
  ql/src/test/results/clientpositive/input_part5.q.out 
c6ae2fd58d7d41e4b4c0f764de3e28484917cd97 
  ql/src/test/results/clientpositive/input_part6.q.out 
c01d8afb6230440103ed8c1ecb38aea9eaabc832 
  ql/src/test/results/clientpositive/join12.q.out 
8217c86d0e41c78b8bf4ed120b30a85204fa9915 
  ql/src/test/results/clientpositive/join16.q.out 
244eb46dc0ae3b61ed932f043a98bb64295776a1 
  ql/src/test/results/clientpositive/join34.q.out 
e2c2b1a5ab742e26d953f11919e150f847a0515