[jira] [Created] (HIVE-16198) Vectorized array index

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-16198:
-

 Summary: Vectorized array index
 Key: HIVE-16198
 URL: https://issues.apache.org/jira/browse/HIVE-16198
 Project: Hive
  Issue Type: Sub-task
  Components: UDF, Vectorization
Reporter: Teddy Choi
Assignee: Teddy Choi






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16199) Vectorized map index

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-16199:
-

 Summary: Vectorized map index
 Key: HIVE-16199
 URL: https://issues.apache.org/jira/browse/HIVE-16199
 Project: Hive
  Issue Type: Sub-task
Reporter: Teddy Choi






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16200) Vectorized struct field

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-16200:
-

 Summary: Vectorized struct field
 Key: HIVE-16200
 URL: https://issues.apache.org/jira/browse/HIVE-16200
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Teddy Choi


Vectorize ExprNodeFieldDesc for struct data type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16201) Vectorize GenericUDFArray

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-16201:
-

 Summary: Vectorize GenericUDFArray
 Key: HIVE-16201
 URL: https://issues.apache.org/jira/browse/HIVE-16201
 Project: Hive
  Issue Type: Sub-task
  Components: UDF, Vectorization
Reporter: Teddy Choi






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16203) Vectorize GenericUDFStruct

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-16203:
-

 Summary: Vectorize GenericUDFStruct
 Key: HIVE-16203
 URL: https://issues.apache.org/jira/browse/HIVE-16203
 Project: Hive
  Issue Type: Sub-task
  Components: UDF, Vectorization
Reporter: Teddy Choi






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16202) Vectorize GenericUDFMap

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-16202:
-

 Summary: Vectorize GenericUDFMap
 Key: HIVE-16202
 URL: https://issues.apache.org/jira/browse/HIVE-16202
 Project: Hive
  Issue Type: Sub-task
  Components: Vectorization
Reporter: Teddy Choi






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16204) Vectorize GenericUDFNamedStruct

[Teddy Choi (JIRA)]

Teddy Choi created HIVE-16204:
-

 Summary: Vectorize GenericUDFNamedStruct
 Key: HIVE-16204
 URL: https://issues.apache.org/jira/browse/HIVE-16204
 Project: Hive
  Issue Type: Sub-task
  Components: UDF, Vectorization
Reporter: Teddy Choi






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Review Request 57599: HIVE-16152 TestBeeLineDriver logging improvements

[Peter Vary]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57599/
---

Review request for hive, Zoltan Haindrich, Naveen Gangam, Vihang Karajgaonkar, 
and Barna Zsombor Klara.


Bugs: HIVE-16152
https://issues.apache.org/jira/browse/HIVE-16152


Repository: hive-git


Description
---

Enhances the error message when a test fails.
For the actual output see the jira.


Diffs
-

  
itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreBeeLineDriver.java
 acc02eb 
  itests/util/src/main/java/org/apache/hive/beeline/qfile/QFile.java 49d6d24 
  
itests/util/src/main/java/org/apache/hive/beeline/qfile/QFileBeeLineClient.java 
b6eac89 


Diff: https://reviews.apache.org/r/57599/diff/1/


Testing
---

Manually tested various fail cases.
Collected the output in the jira


Thanks,

Peter Vary

Need help to contribute in Hive

[Prince Arora]

Hi All,

I am working in Big data domain over the last few years. After going
through the internals of Hive thrift layer while working in some of the
project, I got interested to contribute to the project. Could you guys
please help me in getting started with process and suggest me some bug
fixes/issues?

Thanks all in advance!
Prince

Hive on Tez - Can we configure resources on local setup

[Kedarnath Dixit]

Hi,


We have single node hive on Tez configured. We dont have HDFS and Any Resource 
Manager like YARN or MESOS.


machine has 8 cores and 32 GB of RAM .


We have complex queries taking significant time.


We see them only using 1 core and few(1-2) GB RAM only, Are there possible 
configurations to allocate more cores and RAM  to see better performance?


Can some one help please ?



Thanks!


With Regards,



~Kedar Dixit

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.

[jira] [Created] (HIVE-16205) Improving type safety in Objectstore

[Vihang Karajgaonkar (JIRA)]

Vihang Karajgaonkar created HIVE-16205:
--

 Summary: Improving type safety in Objectstore
 Key: HIVE-16205
 URL: https://issues.apache.org/jira/browse/HIVE-16205
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar


Modify the queries in ObjectStore for better type safety



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16206) Make Codahale metrics reporters pluggable

[Sunitha Beeram (JIRA)]

Sunitha Beeram created HIVE-16206:
-

 Summary: Make Codahale metrics reporters pluggable
 Key: HIVE-16206
 URL: https://issues.apache.org/jira/browse/HIVE-16206
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Sunitha Beeram
Assignee: Sunitha Beeram


Hive metrics code currently allows pluggable metrics handlers - ie, handlers 
that take care of providing interfaces for metrics collection as well as a 
reporting; one of the 'handlers' is CodahaleMetrics. Codahale can work with 
different reporters - currently supported ones are Console, JMX, JSON file and 
hadoop2 sink. However, adding a new reporter involves changing that class. We 
would like to make this conf driven just the way MetricsFactory handles 
configurable Metrics classes.

Scope of work:
- Provide a new configuration option, HIVE_CODAHALE_REPORTER_CLASSES that 
enumerates classes (like HIVE_METRICS_CLASS and unlike HIVE_METRICS_REPORTER).
- Move JsonFileReporter into its own class.
- Update CodahaleMetrics.java to read new config option and if the new option 
is not present, look for the old option and instantiate accordingly) - ie, make 
the code backward compatible.
- Update and add new tests.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16207) Add support for Complex Types in Fast SerDe

[Matt McCline (JIRA)]

Matt McCline created HIVE-16207:
---

 Summary: Add support for Complex Types in Fast SerDe
 Key: HIVE-16207
 URL: https://issues.apache.org/jira/browse/HIVE-16207
 Project: Hive
  Issue Type: Bug
  Components: Hive
Reporter: Matt McCline
Assignee: Teddy Choi
Priority: Critical


Add complex type support to Fast SerDe classes.  This is needed for fully 
supporting complex types in Vectorization



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Review Request 57614: Auto-gather column stats - phase 2

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57614/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

HIVE-13567


Diffs
-

  data/conf/hive-site.xml 62364fe 
  
ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2_h23.q.out
 c197152 
  
ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table_h23.q.out 
3890aef 
  ql/src/test/results/clientpositive/alter_partition_coltype.q.out daa6255 
  ql/src/test/results/clientpositive/alter_table_add_partition.q.out 7b1075e 
  ql/src/test/results/clientpositive/alter_table_serde2.q.out 9208ec3 
  ql/src/test/results/clientpositive/analyze_table_null_partition.q.out 2bfc04a 
  ql/src/test/results/clientpositive/annotate_stats_filter.q.out e22c3ef 
  ql/src/test/results/clientpositive/annotate_stats_groupby.q.out a8e4854 
  ql/src/test/results/clientpositive/auto_join1.q.out 5f4bb74 
  ql/src/test/results/clientpositive/auto_join14.q.out 1dd677c 
  ql/src/test/results/clientpositive/auto_join17.q.out d39c36e 
  ql/src/test/results/clientpositive/auto_join19.q.out 3f70055 
  ql/src/test/results/clientpositive/auto_join19_inclause.q.out 3f70055 
  ql/src/test/results/clientpositive/auto_join2.q.out b17d344 
  ql/src/test/results/clientpositive/auto_join25.q.out 534bdb6 
  ql/src/test/results/clientpositive/auto_join26.q.out b05145d 
  ql/src/test/results/clientpositive/auto_join3.q.out 35e8273 
  ql/src/test/results/clientpositive/auto_join4.q.out eacf056 
  ql/src/test/results/clientpositive/auto_join5.q.out d526595 
  ql/src/test/results/clientpositive/auto_join6.q.out 53caf7d 
  ql/src/test/results/clientpositive/auto_join7.q.out aec4e82 
  ql/src/test/results/clientpositive/auto_join8.q.out ccbafba 
  ql/src/test/results/clientpositive/auto_join9.q.out d7d7d18 
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 156be41 
  ql/src/test/results/clientpositive/auto_sortmerge_join_13.q.out 8c7658c 
  ql/src/test/results/clientpositive/binary_output_format.q.out 96afeb6 
  ql/src/test/results/clientpositive/bucket1.q.out 78571aa 
  ql/src/test/results/clientpositive/bucket2.q.out a8e9f90 
  ql/src/test/results/clientpositive/bucket3.q.out b1173e7 
  ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out ba79294 
  ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out 8974d4c 
  ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out 0453f99 
  ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out 5743944 
  ql/src/test/results/clientpositive/bucketmapjoin13.q.out 71b2924 
  ql/src/test/results/clientpositive/bucketmapjoin5.q.out b108607 
  ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 4aa7f82 
  ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 09b5af9 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_1.q.out 165f0dc 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_3.q.out 11c7c39 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out 1d794c3 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_5.q.out 1e70105 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out f3d3006 
  ql/src/test/results/clientpositive/case_sensitivity.q.out b3969cc 
  ql/src/test/results/clientpositive/cast1.q.out 1a246c0 
  ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out 
f260f03 
  ql/src/test/results/clientpositive/cbo_rp_auto_join17.q.out b296280 
  ql/src/test/results/clientpositive/cbo_rp_gby2_map_multi_distinct.q.out 
d4d70bc 
  
ql/src/test/results/clientpositive/cbo_rp_groupby3_noskew_multi_distinct.q.out 
c09764c 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 
af21343 
  ql/src/test/results/clientpositive/constprog_dp.q.out 8cf301d 
  ql/src/test/results/clientpositive/correlationoptimizer5.q.out 00bdb4c 
  ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd 
  ql/src/test/results/clientpositive/deleteAnalyze.q.out 4382522 
  ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out 
d199574 
  ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 87166a7 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid2.q.out 
76d0b7b 
  ql/src/test/results/clientpositive/explain_ddl.q.out e108e22 
  ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out b212da9 
  ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out 
b5f4fee 
  ql/src/test/results/clientpositive/filter_join_breaktask.q.out 8f9b636 
  ql/src/test/results/clientpositive/groupby1.q.out 46e09dd 
  ql/src/test/results/clientpositive/groupby10.q.out 66832b0 
  ql/src/test/results/clientpositive/groupby11.q.out 1d0e86a 
  ql/src/test/results/clientpositive/groupby12.q.out b17da54 
  ql/src/test/results/clientposit

Re: Review Request 57586: HIVE-16183: Fix potential thread safety issues with static variables

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57586/#review168921
---




cli/src/java/org/apache/hadoop/hive/cli/RCFileCat.java
Line 265 (original), 265 (patched)


not sure if removing static from methods is needed... I usually prefer to 
ADD static to methods if they don't depend on an instance :)
Non-binding


- Sergey Shelukhin


On March 14, 2017, 4:32 a.m., Xuefu Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57586/
> ---
> 
> (Updated March 14, 2017, 4:32 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-16183
> https://issues.apache.org/jira/browse/HIVE-16183
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Please see JIRA description
> 
> 
> Diffs
> -
> 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 7e6846d 
>   beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java 181f0d2 
>   cli/src/java/org/apache/hadoop/hive/cli/RCFileCat.java f1806a0 
>   cli/src/test/org/apache/hadoop/hive/cli/TestRCFileCat.java 11ceb31 
>   common/src/java/org/apache/hadoop/hive/common/LogUtils.java c2a0d9a 
>   common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 926b4a6 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java 
> 9c30ee7 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java 6381a21 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 4ac25c2 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 6693134 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 
> 5b0c2bf 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CuckooSetBytes.java
>  6383e8a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTable.java
>  9030e5f 
>   ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryImpl.java 6582cdd 
>   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndex.java a1408e9 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 7727114 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 4995bdf 
>   ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java d391164 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 369584b 
>   ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/stats/PartialScanTask.java 
> 90b1dff 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 044d64c 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 0e67ea6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPrunerUtils.java
>  4d3e74e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java
>  93202c3 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
> 50eda15 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/VectorizerReason.java
>  e0a6198 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
> 36009bf 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> f175663 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingSpec.java 01b5559 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractVectorDesc.java e85a418 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 0b49294 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java ca69697 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 9ae30ab 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorAppMasterEventDesc.java 
> 2e11321 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorFileSinkDesc.java 325ac91 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorFilterDesc.java 6feed84 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorGroupByDesc.java f8554e2 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorLimitDesc.java c9bc45a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorMapJoinDesc.java 3aa65d3 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorMapJoinInfo.java 9429785 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorPartitionDesc.java 4078c7d 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkDesc.java 
> 2eb44b8 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkInfo.java 
> 8c35415 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorSMBJoinDesc.java 031f11e 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorSelectDesc.java c2c9450 
>   
> ql/src/java/org/apache/hadoop/hive/ql/plan/VectorSparkHashTableSinkDesc.java 
> 7fb59db 
>   
> ql/src/java/org/apache/hadoop/hive/ql/plan/VectorSparkPartitionPruningSinkDesc.java
>  c0bc7e4 
>   

Re: Review Request 57586: HIVE-16183: Fix potential thread safety issues with static variables

[Xuefu Zhang]

> On March 14, 2017, 6:18 p.m., Sergey Shelukhin wrote:
> > cli/src/java/org/apache/hadoop/hive/cli/RCFileCat.java
> > Line 265 (original), 265 (patched)
> > 
> >
> > not sure if removing static from methods is needed... I usually prefer 
> > to ADD static to methods if they don't depend on an instance :)
> > Non-binding

Given that "test" is changed to an instance variable, this method needs to be 
instance method as well.


- Xuefu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57586/#review168921
---


On March 14, 2017, 4:32 a.m., Xuefu Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57586/
> ---
> 
> (Updated March 14, 2017, 4:32 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-16183
> https://issues.apache.org/jira/browse/HIVE-16183
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Please see JIRA description
> 
> 
> Diffs
> -
> 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 7e6846d 
>   beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java 181f0d2 
>   cli/src/java/org/apache/hadoop/hive/cli/RCFileCat.java f1806a0 
>   cli/src/test/org/apache/hadoop/hive/cli/TestRCFileCat.java 11ceb31 
>   common/src/java/org/apache/hadoop/hive/common/LogUtils.java c2a0d9a 
>   common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 926b4a6 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java 
> 9c30ee7 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java 6381a21 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 4ac25c2 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 6693134 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 
> 5b0c2bf 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CuckooSetBytes.java
>  6383e8a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTable.java
>  9030e5f 
>   ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryImpl.java 6582cdd 
>   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndex.java a1408e9 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 7727114 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 4995bdf 
>   ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java d391164 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 369584b 
>   ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/stats/PartialScanTask.java 
> 90b1dff 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 044d64c 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 0e67ea6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPrunerUtils.java
>  4d3e74e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java
>  93202c3 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
> 50eda15 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/VectorizerReason.java
>  e0a6198 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
> 36009bf 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> f175663 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingSpec.java 01b5559 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractVectorDesc.java e85a418 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 0b49294 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java ca69697 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 9ae30ab 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorAppMasterEventDesc.java 
> 2e11321 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorFileSinkDesc.java 325ac91 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorFilterDesc.java 6feed84 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorGroupByDesc.java f8554e2 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorLimitDesc.java c9bc45a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorMapJoinDesc.java 3aa65d3 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorMapJoinInfo.java 9429785 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorPartitionDesc.java 4078c7d 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkDesc.java 
> 2eb44b8 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkInfo.java 
> 8c35415 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorSMBJoinDesc.java 031f11e 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorSelectDesc.java c2c9450 
> 

Re: Need help to contribute in Hive

[Alan Gates]

Prince, welcome to Hive.  Probably the best way is to find a JIRA no one else 
is working on, assign it to yourself, and then start working on it.  I did a 
quick search on open JIRAs relating to the thrift interface and found 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20HIVE%20AND%20status%20%3D%20Open%20AND%20component%20%3D%20%22Thrift%20API%22
 but feel free to take on one from another area of Hive.  You may also find the 
wiki information on contributing useful: 
https://cwiki.apache.org/confluence/display/Hive/Home#Home-ResourcesforContributors

Alan.

> On Mar 14, 2017, at 7:48 AM, Prince Arora  wrote:
> 
> Hi All,
> 
> I am working in Big data domain over the last few years. After going
> through the internals of Hive thrift layer while working in some of the
> project, I got interested to contribute to the project. Could you guys
> please help me in getting started with process and suggest me some bug
> fixes/issues?
> 
> Thanks all in advance!
> Prince

[jira] [Created] (HIVE-16208) Vectorization: ProcessingModeHashAggregate::sumBatchSize is never reset

[Gopal V (JIRA)]

Gopal V created HIVE-16208:
--

 Summary: Vectorization: ProcessingModeHashAggregate::sumBatchSize 
is never reset
 Key: HIVE-16208
 URL: https://issues.apache.org/jira/browse/HIVE-16208
 Project: Hive
  Issue Type: Bug
Reporter: Gopal V


When processing >2x the hash-table size in the vectorized group-by, the check 
for fall-back to streaming is wrong because {{sumBatchSize*minReduction}} is 
not modified by a partial flush or a full flush.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16209) Vectorization: Add support for complex types to VectorExtractRow and VectorAssignRow

[Matt McCline (JIRA)]

Matt McCline created HIVE-16209:
---

 Summary: Vectorization: Add support for complex types to 
VectorExtractRow and VectorAssignRow
 Key: HIVE-16209
 URL: https://issues.apache.org/jira/browse/HIVE-16209
 Project: Hive
  Issue Type: Bug
  Components: Hive
Reporter: Matt McCline
Assignee: Teddy Choi
Priority: Critical


Supports complex types in non-native VectorReduceSink, row mode Text 
Vectorization, and some cases of Vectorized Schema Evolution.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Green unit test results

[Vihang Karajgaonkar]

One thing which could be done by each author of a patch is to do a search
on the HiveQA test failures. If there are flaky tests which fail on the
HiveQA run may be should create a JIRA for each of them and link it to the
JIRA. The JIRA could have useful information like stacktraces or attaching
the logs pertaining to the test run. If there is already an existing JIRA
for that flaky test we should still link it. This way we can prioritize the
flaky tests which has been linked a lot of times.

On Mon, Mar 13, 2017 at 11:40 AM, Siddharth Seth  wrote:

> There's far too many flaky tests to enforce a clean test run prior to
> commit.
> However, there also needs to be a very conscious effort to fix flaky tests
> - otherwise, we're not going to reach the state where this can be enforced.
> I'm also not sure whether this is just flaky tests, or whether there are
> real problems. Maybe a few volunteers could drive the test fixes, and as an
> artifact of this, come up with guidelines on writing tests (avoid stats
> comparisons in qfiles, skip lines which may be environment specific,
> introduce a new compare mechanism for lines which may be in a different
> order based on internal structures, etc).
>
> On Thu, Mar 9, 2017 at 2:39 PM, Ashutosh Chauhan 
> wrote:
>
> > I went through all the builds of last 24 hours and though there are
> couple
> > of green runs, we still have quite a bit of flakiness in our tests. Most
> of
> > that is captured via jiras on :
> > https://issues.apache.org/jira/browse/HIVE-15058 Till we have that
> > flakiness I am not sure if we can enforce no commit on test failures
> > policy. Once flakiness is straightened out then this discussion will
> become
> > moot anyways.
> > Towards that I think this flaky detector job is a step in right
> direction.
> >
> >
> > On Thu, Mar 9, 2017 at 11:38 AM, Sergio Pena 
> > wrote:
> >
> > > - Probably avoiding committing a patch if a flaky test is shown on the
> > test
> > > results?
> > > - Should we add a jenkins job that checks for flaky tests like the
> hbase
> > > project did?
> > > https://builds.apache.org/view/H-L/view/HBase/job/HBase-
> > Find-Flaky-Tests/
> > >
> > > On Thu, Mar 9, 2017 at 10:21 AM, Ashutosh Chauhan <
> hashut...@apache.org>
> > > wrote:
> > >
> > > > Great news! Thanks to everyone who contributed in getting our tests
> and
> > > > test infra sorted out.
> > > > We would definitely want to keep the status either green or blue
> > > definitely
> > > > not red :) All our previous efforts in keeping builds green didn't
> bear
> > > > fruit.
> > > > So, I think we need to make some changes here.
> > > >
> > > > Any ideas what we can do to ensure green builds going forward?
> > > >
> > > > Thanks,
> > > > Ashutosh
> > > >
> > > > On Thu, Mar 9, 2017 at 8:07 AM, Sergio Pena <
> sergio.p...@cloudera.com>
> > > > wrote:
> > > >
> > > > > It's actually blue Peter :).
> > > > >
> > > > > But good job, I see that the console output is:
> > > > >
> > > > > {color:red}ERROR:{color} -1 due to no test(s) being added or
> > modified.
> > > > >
> > > > > {color:green}SUCCESS:{color} +1 due to 10336 tests passed
> > > > >
> > > > >
> > > > > On Thu, Mar 9, 2017 at 8:12 AM, Peter Vary 
> > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Congratulations for everyone who have helped taking care of the
> > unit
> > > > test
> > > > > > failures!
> > > > > > I have got my first green run! :)
> > > > > >
> > > > > > If any of you interested in:
> > > > > > https://builds.apache.org/job/PreCommit-HIVE-Build/4049/
> > testReport/
> > > <
> > > > > > https://builds.apache.org/job/PreCommit-HIVE-Build/4049/
> > testReport/>
> > > > :)
> > > > > >
> > > > > > Great day, and again thanks everyone!
> > > > > >
> > > > > > Peter
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (HIVE-16210) Use jvm temporary tmp dir by default

[slim bouguerra (JIRA)]

slim bouguerra created HIVE-16210:
-

 Summary: Use jvm temporary tmp dir by default
 Key: HIVE-16210
 URL: https://issues.apache.org/jira/browse/HIVE-16210
 Project: Hive
  Issue Type: Improvement
  Components: Druid integration
Reporter: slim bouguerra
Assignee: slim bouguerra


instead of using "/tmp" by default, it makes more sense to use the jvm default 
tmp dir. This can have dramatic consequences if the indexed files are huge. For 
instance application run by run containers can be provisioned with a dedicated 
tmp dir. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16211) MERGE statement failing with ClassCastException

[Deepak Jaiswal (JIRA)]

Deepak Jaiswal created HIVE-16211:
-

 Summary: MERGE statement failing with ClassCastException
 Key: HIVE-16211
 URL: https://issues.apache.org/jira/browse/HIVE-16211
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Reporter: Deepak Jaiswal
Assignee: Deepak Jaiswal


Issuing a merge statement gives this error,

hive> 2017-03-14T18:34:02,945 ERROR [17d1c728-8865-47f5-a6fd-2b156d183d0f main] 
ql.Driver: FAILED: ClassCastException 
org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc cannot be cast to 
org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc
java.lang.ClassCastException: 
org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc cannot be cast to 
org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc
at 
org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.generateSemiJoinOperatorPlan(DynamicPartitionPruningOptimization.java:410)
at 
org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(DynamicPartitionPruningOptimization.java:226)
at 
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
at 
org.apache.hadoop.hive.ql.lib.ForwardWalker.walk(ForwardWalker.java:74)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
at 
org.apache.hadoop.hive.ql.parse.TezCompiler.runDynamicPartitionPruning(TezCompiler.java:359)
at 
org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:91)
at 
org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:138)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11159)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10708)
at 
org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:70)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:257)
at 
org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeMerge(UpdateDeleteSemanticAnalyzer.java:729)
at 
org.apache.hadoop.hive.ql.parse.UpdateDeleteSemanticAnalyzer.analyzeInternal(UpdateDeleteSemanticAnalyzer.java:84)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:257)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:455)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1197)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1290)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at 
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16212) MM tables: suspicious ORC HDFS counter changes

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-16212:
---

 Summary: MM tables: suspicious ORC HDFS counter changes
 Key: HIVE-16212
 URL: https://issues.apache.org/jira/browse/HIVE-16212
 Project: Hive
  Issue Type: Sub-task
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin


org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters1]
 (batchId=136)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
 (batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] 
(batchId=136)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
 (batchId=137)

HDFS counters for operation counts go up (which I can repro locally).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Review Request 57620: Support the named columns join

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57620/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

HIVE-15983


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java bf0a11b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/FromClauseParser.g 558ca7f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java e14f1cf 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0732207 
  ql/src/test/queries/clientpositive/named_column_join.q PRE-CREATION 
  ql/src/test/results/clientpositive/named_column_join.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/57620/diff/1/


Testing
---


Thanks,

pengcheng xiong

Re: Review Request 57503: HIVE-16024: MSCK Repair Requires nonstrict hive.mapred.mode

[Sergio Pena]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57503/#review168942
---




common/src/test/org/apache/hadoop/hive/common/TestFixedSizeCollection.java
Lines 32 (patched)


Should we use the java naming convention for method names?



ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMetaStoreChecker.java
Line 219 (original), 223-228 (patched)


Does this mean that if a user attempts to repair a table with more than 
500_000 partitions, the MSCK will fail?

If so, I think we're having the same problem as before. Users won't be able 
to discover all partitions from a table with more than 500_000.

Did we have problems before the patch which added this regression issue? If 
not, should we use PartitionIterable with unlimited number of partitions 
instead? The number of HMS transactions due to PartitionIterable shouldn't be a 
problem if the user increases the batch size. Also, as Vihang mentioned, we're 
just storing two values (partition name + table name), so that consumes less 
memory than using the hive.getPartition() method call.


- Sergio Pena


On March 10, 2017, 10:36 a.m., Barna Zsombor Klara wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57503/
> ---
> 
> (Updated March 10, 2017, 10:36 a.m.)
> 
> 
> Review request for hive, Peter Vary, Sergio Pena, Sahil Takiar, and Vihang 
> Karajgaonkar.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16024: MSCK Repair Requires nonstrict hive.mapred.mode
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FixedSizeCollection.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a479deb7c0c6b779277f1029009b7dfab6dcb9e3 
>   common/src/test/org/apache/hadoop/hive/common/TestFixedSizeCollection.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMetaStoreChecker.java 
> 6805c17a116f5ef0febd36c59d454fa631ae0024 
>   ql/src/test/queries/clientnegative/msck_repair_4.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/msck_repair_0.q 
> ce8ef426a2a58845afc8333259d66725db416584 
>   ql/src/test/results/clientnegative/msck_repair_4.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/msck_repair_0.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
> 
> 
> Diff: https://reviews.apache.org/r/57503/diff/1/
> 
> 
> Testing
> ---
> 
> Tested locally and added qtests/unit tests.
> 
> 
> Thanks,
> 
> Barna Zsombor Klara
> 
>

[jira] [Created] (HIVE-16213) ObjectStore can leak Queries when rolbackTransaction

[Alexander Kolbasov (JIRA)]

Alexander Kolbasov created HIVE-16213:
-

 Summary: ObjectStore can leak Queries when rolbackTransaction 
 Key: HIVE-16213
 URL: https://issues.apache.org/jira/browse/HIVE-16213
 Project: Hive
  Issue Type: Bug
  Components: Hive
Reporter: Alexander Kolbasov


In ObjectStore.java there are a few places with the code similar to:

{code}
Query query = null;
try {
  openTransaction();
  query = pm.newQuery(Something.class);
  ...
  commited = commitTransaction();
} finally {
  if (!commited) {
rollbackTransaction();
  }
  if (query != null) {
query.closeAll();
  }
}
{code}

The problem is that rollbackTransaction() may throw an exception in which case 
query.closeAll() wouldn't be executed. 

The fix would be to wrap rollbackTransaction in its own try-catch block.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16214) Explore the possibillity of introducing a service-client module

[Zoltan Haindrich (JIRA)]

Zoltan Haindrich created HIVE-16214:
---

 Summary: Explore the possibillity of introducing a service-client 
module
 Key: HIVE-16214
 URL: https://issues.apache.org/jira/browse/HIVE-16214
 Project: Hive
  Issue Type: Improvement
Reporter: Zoltan Haindrich
Assignee: Zoltan Haindrich


The jdbc driver pulls in a lot of things from hive...and that may affect the 
jdbc driver user.

In this ticket I experiment with the extraction of the relevant parts of 
service(wrt to the jdbc driver) into a service-client module.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 57518: HIVE-16091 Support subqueries in select

[Vineet Garg]

> On March 10, 2017, 7:46 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
> > Lines 2815 (patched)
> > 
> >
> > Is this needed? Having this check in genSelectLogicalPlan() should be 
> > sufficient?

This is needed since genGBLogicalPlan is called first and I remember for 
subqueries with distinct there was some issue and I had to add this check. Let 
me check and confirm this.


> On March 10, 2017, 7:46 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/subquery_select.q.out
> > Lines 19-24 (patched)
> > 
> >
> > This plan is doing null counting for inner query for IN clause. I think 
> > this is unnecessary.

Since we care if IN returns true, false or NULL we need to know presence of 
NULL so doing count() is necessary.


- Vineet


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57518/#review168654
---


On March 14, 2017, 5:05 a.m., Vineet Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57518/
> ---
> 
> (Updated March 14, 2017, 5:05 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-16091
> https://issues.apache.org/jira/browse/HIVE-16091
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support for subqueries in select statement (with limitations)
> Wiki: https://cwiki.apache.org/confluence/display/Hive/Subqueries+in+SELECT
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 2a7627a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java
>  b7b16b8 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
>  c1768f4 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java bf0a11b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryUtils.java f0165dd 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java 6c10704 
>   ql/src/test/queries/clientnegative/subquery_in_select.q 1365389 
>   ql/src/test/queries/clientnegative/subquery_select_aggregate.q PRE-CREATION 
>   ql/src/test/queries/clientnegative/subquery_select_complex_expr.q 
> PRE-CREATION 
>   ql/src/test/queries/clientnegative/subquery_select_distinct.q PRE-CREATION 
>   ql/src/test/queries/clientnegative/subquery_select_distinct2.q PRE-CREATION 
>   ql/src/test/queries/clientnegative/subquery_select_udf.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/subquery_select.q PRE-CREATION 
>   ql/src/test/results/clientnegative/subquery_in_select.q.out 5c69690 
>   ql/src/test/results/clientnegative/subquery_select_aggregate.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientnegative/subquery_select_distinct.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientnegative/subquery_select_distinct2.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientnegative/subquery_select_udf.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/subquery_select.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/57518/diff/3/
> 
> 
> Testing
> ---
> 
> Added new tests
> Done pre-commit testing
> Pending: More tests
> 
> 
> Thanks,
> 
> Vineet Garg
> 
>

Re: Review Request 57518: HIVE-16091 Support subqueries in select

[Vineet Garg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57518/
---

(Updated March 14, 2017, 10:17 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Fixes fail ptest


Bugs: HIVE-16091
https://issues.apache.org/jira/browse/HIVE-16091


Repository: hive-git


Description
---

Support for subqueries in select statement (with limitations)
Wiki: https://cwiki.apache.org/confluence/display/Hive/Subqueries+in+SELECT


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties e445d3b 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java
 b7b16b8 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java
 c1768f4 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java bf0a11b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryUtils.java f0165dd 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java 6c10704 
  ql/src/test/queries/clientnegative/subquery_in_select.q 1365389 
  ql/src/test/queries/clientnegative/subquery_select_aggregate.q PRE-CREATION 
  ql/src/test/queries/clientnegative/subquery_select_complex_expr.q 
PRE-CREATION 
  ql/src/test/queries/clientnegative/subquery_select_distinct.q PRE-CREATION 
  ql/src/test/queries/clientnegative/subquery_select_distinct2.q PRE-CREATION 
  ql/src/test/queries/clientnegative/subquery_select_udf.q PRE-CREATION 
  ql/src/test/queries/clientpositive/subquery_select.q PRE-CREATION 
  ql/src/test/results/clientnegative/subquery_in_select.q.out 5c69690 
  ql/src/test/results/clientnegative/subquery_select_aggregate.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/subquery_select_distinct.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/subquery_select_distinct2.q.out 
PRE-CREATION 
  ql/src/test/results/clientnegative/subquery_select_udf.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/llap/subquery_select.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/57518/diff/4/

Changes: https://reviews.apache.org/r/57518/diff/3-4/


Testing
---

Added new tests
Done pre-commit testing
Pending: More tests


Thanks,

Vineet Garg

[GitHub] hive pull request #158: HIVE-16214 Service client experiment

[kgyrtkirk]

GitHub user kgyrtkirk opened a pull request:

https://github.com/apache/hive/pull/158

HIVE-16214 Service client experiment



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kgyrtkirk/hive service-client

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/158.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #158


commit 90dc6386d033041737c8c4b14ca59a983ed806f2
Author: Zoltan Haindrich 
Date:   2017-03-14T21:24:16Z

cripple embedded support

commit 2e36af2e1c602970ded201598eb2072cc502c620
Author: Zoltan Haindrich 
Date:   2017-03-14T21:25:10Z

move around files

commit e3f443cee12102e579216a603f94deb8f902e87e
Author: Zoltan Haindrich 
Date:   2017-03-14T21:30:51Z

add reflection to use Hive.class

commit a0ec66108acf167d35c36225fa277b94b814a020
Author: Zoltan Haindrich 
Date:   2017-03-14T21:37:18Z

use TCLIService.Iface instead of implementation

commit 2b1040dbf3093242686bd622bdd85414fb40aac0
Author: Zoltan Haindrich 
Date:   2017-03-14T21:59:54Z

possibly re-enable embedded mode

commit d44603edc987ccc9839d49d0b3dfcb6c6e8c4ebc
Author: Zoltan Haindrich 
Date:   2017-03-14T22:01:12Z

remove unneeded deps from service-client

commit ede65c5cf7d1b83ebea04707d7c4853ad73b5dd6
Author: Zoltan Haindrich 
Date:   2017-03-14T22:05:29Z

make service depend on client instead rpc




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Review Request 57626: HIVE-16164: Provide mechanism for passing HMS notification ID between transactional and non-transactional listeners.

[Sergio Pena]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57626/
---

Review request for hive.


Bugs: HIVE-16164
https://issues.apache.org/jira/browse/HIVE-16164


Repository: hive-git


Description
---

This fix updates the EnvironmentContext with a DB_NOTIFICATION_EVENT_ID 
property from withing the DbNotificationListener class. It then passes the 
EnvironmentContext from transactional listeners to non-transactional listeners 
so that the eventId is shared between them.

The patch provides the following changes:
- DbNotificationListener   Changes to pass the EnvironmentContext from 
transactional to non-transactional listeners.
- HiveAlterHandler Changes to pass the EnvironmentContext from 
transactional to non-transactional listeners.
- MetaStoreListenerNotifierNew helper class that wraps the notification 
call to the listeners.
- TestObjectStore  Verifies that the addNotificationEvent() method 
saves the eventId on the NotificationEvent object.
- TestDbNotificationListener   Verifies that any HMS call is passing the 
DB_NOTIFICATION_EVENT_ID to non-transactional listeners.


Diffs
-

  
hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java
 f7e3e3a0a71094992fdf4bd3ceea2da0bf7d1ff0 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
 1cf47c36cb490ce0b17ffe312cd2e9fc4bb7cd9a 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
bae39acafeb86d04ac8ec66098be125cd3cef3e0 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
07eca38190c1b05bb4a3977e9154423449828957 
  
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreListenerNotifier.java
 PRE-CREATION 
  metastore/src/test/org/apache/hadoop/hive/metastore/TestObjectStore.java 
1f87eeb18f6edf7351b3c8da6a6826c08656e48c 


Diff: https://reviews.apache.org/r/57626/diff/1/


Testing
---

HiveQA showed only one test failure. it is fixed, and waiting for HiveQA to 
complete 100% tests.


Thanks,

Sergio Pena

Re: Review Request 57568: HIVE-15978 Support regr_* functions

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57568/#review168950
---




ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
Lines 211 (patched)


We can instantiate GenericUDAFAverageEvaluatorDecimal by looking at 
arguments.



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
Lines 215 (patched)


See above. Any reason for not supporting decimal?



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
Lines 255 (patched)


This implies we can't use corr, correct. This function is not symmetric. We 
need to be standard compliant.



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
Lines 289 (patched)


Cant use correvaluator then.



ql/src/test/results/clientpositive/udaf_binarysetfunctions.q.out
Lines 399 (patched)


Testing on postgres, got NULL for covar_samp() instead of 0.0


- Ashutosh Chauhan


On March 13, 2017, 7:56 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57568/
> ---
> 
> (Updated March 13, 2017, 7:56 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-15978
> https://issues.apache.org/jira/browse/HIVE-15978
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> all aggregators have been implemented by using some existing aggregator
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 
> 4ac25c24eb1becf1640d35f5b79965e4aefc5421 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDAFBinarySetFunctions.java
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/udaf_binarysetfunctions.q PRE-CREATION 
>   ql/src/test/results/clientpositive/show_functions.q.out 
> 1361b9322199bbee5a97b402253e204004bcbfb9 
>   ql/src/test/results/clientpositive/udaf_binarysetfunctions.q.out 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/57568/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zoltan Haindrich
> 
>

Re: Review Request 57568: HIVE-15978 Support regr_* functions

[Zoltan Haindrich]

> On March 14, 2017, 11:08 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
> > Lines 211 (patched)
> > 
> >
> > We can instantiate GenericUDAFAverageEvaluatorDecimal by looking at 
> > arguments.

i've fixed this


> On March 14, 2017, 11:08 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
> > Lines 255 (patched)
> > 
> >
> > This implies we can't use corr, correct. This function is not 
> > symmetric. We need to be standard compliant.

not entirely...corr works as it should - and its great...the only problem with 
it is that it lists the arguments in twisted order: i've opened a separate 
ticket(HIVE-16178) to address this - that change will eliminate these comments 
too.

i've put the note here because the field names in the correlation evaluator are 
also twisted; that's why I used x when the function name have implied y ; 
HIVE-16178 will fix this problem


> On March 14, 2017, 11:08 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/udaf_binarysetfunctions.q.out
> > Lines 399 (patched)
> > 
> >
> > Testing on postgres, got NULL for covar_samp() instead of 0.0

hive will start giving the correct answer for this case after HIVE-16178 - 
because that is different change; I did't wanted to mix it into this one; but 
it seemed right to run all the binary set functions in this test for 
completeness.


- Zoltan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57568/#review168950
---


On March 13, 2017, 7:56 p.m., Zoltan Haindrich wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57568/
> ---
> 
> (Updated March 13, 2017, 7:56 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-15978
> https://issues.apache.org/jira/browse/HIVE-15978
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> all aggregators have been implemented by using some existing aggregator
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 
> 4ac25c24eb1becf1640d35f5b79965e4aefc5421 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
>  PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDAFBinarySetFunctions.java
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/udaf_binarysetfunctions.q PRE-CREATION 
>   ql/src/test/results/clientpositive/show_functions.q.out 
> 1361b9322199bbee5a97b402253e204004bcbfb9 
>   ql/src/test/results/clientpositive/udaf_binarysetfunctions.q.out 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/57568/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zoltan Haindrich
> 
>

[jira] [Created] (HIVE-16215) counter recording for text cache may not fully work

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-16215:
---

 Summary: counter recording for text cache may not fully work
 Key: HIVE-16215
 URL: https://issues.apache.org/jira/browse/HIVE-16215
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin


StatsRecordingThreadPool is too specific



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 57568: HIVE-15978 Support regr_* functions

[Zoltan Haindrich]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57568/
---

(Updated March 15, 2017, 12:52 a.m.)


Review request for hive.


Changes
---

use decimal averaging aggregator in avgx/avgy


Bugs: HIVE-15978
https://issues.apache.org/jira/browse/HIVE-15978


Repository: hive-git


Description
---

all aggregators have been implemented by using some existing aggregator


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 
4ac25c24eb1becf1640d35f5b79965e4aefc5421 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBinarySetFunctions.java
 PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDAFBinarySetFunctions.java
 PRE-CREATION 
  ql/src/test/queries/clientpositive/udaf_binarysetfunctions.q PRE-CREATION 
  ql/src/test/results/clientpositive/show_functions.q.out 
1361b9322199bbee5a97b402253e204004bcbfb9 
  ql/src/test/results/clientpositive/udaf_binarysetfunctions.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/57568/diff/2/

Changes: https://reviews.apache.org/r/57568/diff/1-2/


Testing
---


Thanks,

Zoltan Haindrich

Re: Review Request 57620: Support the named columns join

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57620/#review168961
---




ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java
Lines 19 (patched)


Should be in o.a.h.h.ql.parse



ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java
Lines 24 (patched)


Add comments.

NamedColsInJoin encapsulates information about using clause of join. e.g., 
select * from a join b using(c); This class describes column c.



ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java
Lines 25 (patched)


Better name: NamedColInJoin.



ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java
Lines 26 (patched)


Better name: tableAliases



ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 1929 (patched)


Comment: We will transform using clause and make it look like an on-clause. 
So, lets generate a valid on-clause AST from using.



ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 1949 (patched)


Error message: Atleast one column is expected in using() Found: 0.



ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 3432 (patched)


Comment: We got using() clause in previous join. Need to generate select 
list as per standard. For * we will have joining columns first non-repeated 
followed by other columns.



ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 3490 (patched)


Remove this comment. Its misplaced since named columns will not be always 
there at this point.



ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 3501 (patched)


Comment: we already added this column in select list.



ql/src/test/queries/clientpositive/named_column_join.q
Lines 29 (patched)


Test for 3-way join:
select * from (select t1.b b from t t1 inner join t t2 using (b)) t3 join t 
t4 using(b);
select * from (select t2.a a from t t1 inner join t t2 using (b)) t3 join t 
t4 using(a);


- Ashutosh Chauhan


On March 14, 2017, 8:43 p.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57620/
> ---
> 
> (Updated March 14, 2017, 8:43 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-15983
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java bf0a11b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/FromClauseParser.g 558ca7f 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java e14f1cf 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0732207 
>   ql/src/test/queries/clientpositive/named_column_join.q PRE-CREATION 
>   ql/src/test/results/clientpositive/named_column_join.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/57620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>

Re: Review Request 57566: HIVE-16189: Table column stats might be invalidated in a failed table rename

[Chaoyu Tang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57566/
---

(Updated March 15, 2017, 1:58 a.m.)


Review request for hive.


Changes
---

Fixed the failed tests and add a new unit test.


Bugs: HIVE-16189
https://issues.apache.org/jira/browse/HIVE-16189


Repository: hive-git


Description
---

If the table rename does not succeed due to its failure in moving the data to 
the new renamed table folder, the changes in TAB_COL_STATS are not rolled back 
which leads to invalid column stats.

This patch changes the order of metadata update and data move in alter table 
rename operation, which makes it easier to roll back metadata changes when 
moving data fails in rename a table.


Diffs (updated)
-

  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
bae39ac 
  ql/src/test/queries/clientpositive/encryption_move_tbl.q 7a5de7b 
  ql/src/test/results/clientpositive/encrypted/encryption_move_tbl.q.out 
cc363ac 


Diff: https://reviews.apache.org/r/57566/diff/4/

Changes: https://reviews.apache.org/r/57566/diff/3-4/


Testing
---


Thanks,

Chaoyu Tang

Re: Review Request 57586: HIVE-16183: Fix potential thread safety issues with static variables

[Rui Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57586/#review168965
---




metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java
Line 58 (original), 56 (patched)


shall we remove the hiveConf parameter as it's not needed?



ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
Line 502 (original), 480 (patched)


do we still need this method?



ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
Line 545 (original), 520 (patched)


same as above


- Rui Li


On March 14, 2017, 4:32 a.m., Xuefu Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57586/
> ---
> 
> (Updated March 14, 2017, 4:32 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-16183
> https://issues.apache.org/jira/browse/HIVE-16183
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Please see JIRA description
> 
> 
> Diffs
> -
> 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 7e6846d 
>   beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java 181f0d2 
>   cli/src/java/org/apache/hadoop/hive/cli/RCFileCat.java f1806a0 
>   cli/src/test/org/apache/hadoop/hive/cli/TestRCFileCat.java 11ceb31 
>   common/src/java/org/apache/hadoop/hive/common/LogUtils.java c2a0d9a 
>   common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 926b4a6 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java 
> 9c30ee7 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java 6381a21 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 4ac25c2 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 6693134 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 
> 5b0c2bf 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CuckooSetBytes.java
>  6383e8a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTable.java
>  9030e5f 
>   ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistoryImpl.java 6582cdd 
>   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndex.java a1408e9 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 7727114 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 4995bdf 
>   ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java d391164 
>   ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 369584b 
>   ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/stats/PartialScanTask.java 
> 90b1dff 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 044d64c 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 0e67ea6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPrunerUtils.java
>  4d3e74e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java
>  93202c3 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 
> 50eda15 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/VectorizerReason.java
>  e0a6198 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
> 36009bf 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
> f175663 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingSpec.java 01b5559 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractVectorDesc.java e85a418 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 0b49294 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java ca69697 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 9ae30ab 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorAppMasterEventDesc.java 
> 2e11321 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorFileSinkDesc.java 325ac91 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorFilterDesc.java 6feed84 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorGroupByDesc.java f8554e2 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorLimitDesc.java c9bc45a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorMapJoinDesc.java 3aa65d3 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorMapJoinInfo.java 9429785 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorPartitionDesc.java 4078c7d 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkDesc.java 
> 2eb44b8 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkInfo.java 
> 8c35415 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/VectorSMBJoinDes

[jira] [Created] (HIVE-16216) update trunk/content/people.mdtext

[Eugene Koifman (JIRA)]

Eugene Koifman created HIVE-16216:
-

 Summary: update trunk/content/people.mdtext
 Key: HIVE-16216
 URL: https://issues.apache.org/jira/browse/HIVE-16216
 Project: Hive
  Issue Type: New Feature
  Components: Documentation
Reporter: Eugene Koifman
Assignee: Eugene Koifman






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 57503: HIVE-16024: MSCK Repair Requires nonstrict hive.mapred.mode

[Naveen Gangam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57503/#review168969
---




common/src/java/org/apache/hadoop/hive/common/FixedSizeCollection.java
Lines 32 (patched)


Since we already use common-collections library in hive, would using 
something like FixedSizeList or something else address the same purpose instead 
of having to add a new class?



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Line 781 (original), 781 (patched)


We already a few config properties around the max batch sizes for 
tables/partitions. Would it make sense to re-use one of them? They are for the 
same purpose, whether it be for read or write.

METASTORE_BATCH_RETRIEVE_TABLE_PARTITION_MAX
METASTORE_DIRECT_SQL_PARTITION_BATCH_SIZE
HIVE_MSCK_REPAIR_BATCH_SIZE
METASTORE_BATCH_RETRIEVE_MAX


- Naveen Gangam


On March 10, 2017, 10:36 a.m., Barna Zsombor Klara wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57503/
> ---
> 
> (Updated March 10, 2017, 10:36 a.m.)
> 
> 
> Review request for hive, Peter Vary, Sergio Pena, Sahil Takiar, and Vihang 
> Karajgaonkar.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-16024: MSCK Repair Requires nonstrict hive.mapred.mode
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/FixedSizeCollection.java 
> PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
> a479deb7c0c6b779277f1029009b7dfab6dcb9e3 
>   common/src/test/org/apache/hadoop/hive/common/TestFixedSizeCollection.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMetaStoreChecker.java 
> 6805c17a116f5ef0febd36c59d454fa631ae0024 
>   ql/src/test/queries/clientnegative/msck_repair_4.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/msck_repair_0.q 
> ce8ef426a2a58845afc8333259d66725db416584 
>   ql/src/test/results/clientnegative/msck_repair_4.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/msck_repair_0.q.out 
> 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba 
> 
> 
> Diff: https://reviews.apache.org/r/57503/diff/1/
> 
> 
> Testing
> ---
> 
> Tested locally and added qtests/unit tests.
> 
> 
> Thanks,
> 
> Barna Zsombor Klara
> 
>

[ANNOUNCE] New PMC Member : Eugene Koifman

[Ashutosh Chauhan]

On behalf of the Hive PMC I am delighted to announce Eugene Koifman is
joining Hive PMC.
Eugene is a long time contributor in Hive and is focusing on ACID support
areas these days.

Welcome, Eugene!

Thanks,
Ashutosh

Re: [ANNOUNCE] New PMC Member : Eugene Koifman

[Pengcheng Xiong]

Congrats! Well deserved!

Thanks.
Pengcheng

On Tue, Mar 14, 2017 at 10:39 PM, Ashutosh Chauhan 
wrote:

> On behalf of the Hive PMC I am delighted to announce Eugene Koifman is
> joining Hive PMC.
> Eugene is a long time contributor in Hive and is focusing on ACID support
> areas these days.
>
> Welcome, Eugene!
>
> Thanks,
> Ashutosh
>

Re: [ANNOUNCE] New PMC Member : Eugene Koifman

[Rajesh Balamohan]

Congrats Eugene!! :)

~Rajesh.B

On Wed, Mar 15, 2017 at 11:21 AM, Pengcheng Xiong  wrote:

> Congrats! Well deserved!
>
> Thanks.
> Pengcheng
>
> On Tue, Mar 14, 2017 at 10:39 PM, Ashutosh Chauhan 
> wrote:
>
> > On behalf of the Hive PMC I am delighted to announce Eugene Koifman is
> > joining Hive PMC.
> > Eugene is a long time contributor in Hive and is focusing on ACID support
> > areas these days.
> >
> > Welcome, Eugene!
> >
> > Thanks,
> > Ashutosh
> >
>

Re: [ANNOUNCE] New PMC Member : Eugene Koifman

[Vaibhav Gumashta]

Congrats Eugene!


On 3/14/17, 11:03 PM, "Rajesh Balamohan"  wrote:

>Congrats Eugene!! :)
>
>~Rajesh.B
>
>On Wed, Mar 15, 2017 at 11:21 AM, Pengcheng Xiong 
>wrote:
>
>> Congrats! Well deserved!
>>
>> Thanks.
>> Pengcheng
>>
>> On Tue, Mar 14, 2017 at 10:39 PM, Ashutosh Chauhan
>>
>> wrote:
>>
>> > On behalf of the Hive PMC I am delighted to announce Eugene Koifman is
>> > joining Hive PMC.
>> > Eugene is a long time contributor in Hive and is focusing on ACID
>>support
>> > areas these days.
>> >
>> > Welcome, Eugene!
>> >
>> > Thanks,
>> > Ashutosh
>> >
>>

Re: Review Request 57620: Support the named columns join

[pengcheng xiong]

> On March 15, 2017, 12:56 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java
> > Lines 25 (patched)
> > 
> >
> > Better name: NamedColInJoin.

I think NamedJoinInfo is more appropriate because this class contains not only 
named column but also other information, e.g., table aliases and join type.


> On March 15, 2017, 12:56 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
> > Lines 1949 (patched)
> > 
> >
> > Error message: Atleast one column is expected in using() Found: 0.

I have a second thought. It is impossible to have 0 columns as the parser will 
prevent user from doing so.


- pengcheng


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57620/#review168961
---


On March 14, 2017, 8:43 p.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57620/
> ---
> 
> (Updated March 14, 2017, 8:43 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-15983
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/NamedJoinInfo.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java bf0a11b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/FromClauseParser.g 558ca7f 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java e14f1cf 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0732207 
>   ql/src/test/queries/clientpositive/named_column_join.q PRE-CREATION 
>   ql/src/test/results/clientpositive/named_column_join.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/57620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>

Re: Review Request 57620: Support the named columns join

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57620/
---

(Updated March 15, 2017, 6:17 a.m.)


Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

HIVE-15983


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java bf0a11b 
  ql/src/java/org/apache/hadoop/hive/ql/parse/FromClauseParser.g 558ca7f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/NamedJoinInfo.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java e14f1cf 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0732207 
  ql/src/test/queries/clientpositive/named_column_join.q PRE-CREATION 
  ql/src/test/results/clientpositive/named_column_join.q.out PRE-CREATION 


Diff: https://reviews.apache.org/r/57620/diff/2/

Changes: https://reviews.apache.org/r/57620/diff/1-2/


Testing
---


Thanks,

pengcheng xiong

Re: [ANNOUNCE] New PMC Member : Eugene Koifman

[Peter Vary]

Congratulations! :)

2017. márc. 15. 7:05 ezt írta ("Vaibhav Gumashta" ):

> Congrats Eugene!
>
>
> On 3/14/17, 11:03 PM, "Rajesh Balamohan"  wrote:
>
> >Congrats Eugene!! :)
> >
> >~Rajesh.B
> >
> >On Wed, Mar 15, 2017 at 11:21 AM, Pengcheng Xiong 
> >wrote:
> >
> >> Congrats! Well deserved!
> >>
> >> Thanks.
> >> Pengcheng
> >>
> >> On Tue, Mar 14, 2017 at 10:39 PM, Ashutosh Chauhan
> >>
> >> wrote:
> >>
> >> > On behalf of the Hive PMC I am delighted to announce Eugene Koifman is
> >> > joining Hive PMC.
> >> > Eugene is a long time contributor in Hive and is focusing on ACID
> >>support
> >> > areas these days.
> >> >
> >> > Welcome, Eugene!
> >> >
> >> > Thanks,
> >> > Ashutosh
> >> >
> >>
>
>