Review Request 37778: HIVE-11634

2015-08-25 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 14f362f 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-08-26 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Aug. 26, 2015, 8:22 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Changes
---

Added unit tests + some minor improvements over the previous patch to support 
struct field expression containing only partition columns, virtual columns or 
constants.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 14f362f 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-08-27 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Aug. 27, 2015, 10:30 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-09-03 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Sept. 3, 2015, 6:06 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
  ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-09-04 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review97804
---



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (line 1194)


We need a flag to control this optimization


- John Pullokkaran


On Sept. 3, 2015, 6:06 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 3, 2015, 6:06 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-09-04 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review97806
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java (line 94)


This should be run before predicate push down so that multi cols from 
different tables would be seperated & pushed down


- John Pullokkaran


On Sept. 3, 2015, 6:06 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 3, 2015, 6:06 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-09-04 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review97816
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 59)


Seperating part cols based on lineage should ideally be handled by ppd 
logic. It already does that for CNF we need to extend it to multi col in clause.


- John Pullokkaran


On Sept. 3, 2015, 6:06 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 3, 2015, 6:06 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-09-04 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review97818
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 77)


What about multi col expressions in Join on clauses


- John Pullokkaran


On Sept. 3, 2015, 6:06 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 3, 2015, 6:06 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-09-04 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review97819
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 128)


This needs to walk complex expressions and extract relevant "in clauses"


- John Pullokkaran


On Sept. 3, 2015, 6:06 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 3, 2015, 6:06 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-09-04 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review97820
---



ql/src/test/queries/clientpositive/pcs.q (line 12)


Add test cases for multi col filters above PTF(Windowing), GB, Project, 
Filter, Union, Join


- John Pullokkaran


On Sept. 3, 2015, 6:06 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 3, 2015, 6:06 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8a00079 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-09-10 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Sept. 10, 2015, 5:29 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d2c5885 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
  ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-09-10 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Sept. 10, 2015, 6:40 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d2c5885 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
  ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-09-17 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review99389
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 176)


I think we do not need this recurvise call; that's the work of the walker.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 276)


Could we add an exit condition if all aliases were partition aliases and 
belong to the same table i.e. we will end up with the same predicate?

This might be useful if we still keep the logic in HIVE-11726.



ql/src/test/queries/clientpositive/pointlookup.q (line 21)





- Jesús Camacho Rodríguez


On Sept. 10, 2015, 6:40 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 10, 2015, 6:40 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d2c5885 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-09-17 Thread Jesús Camacho Rodríguez 


> On Sept. 17, 2015, 6:05 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java,
> >  line 276
> > 
> >
> > Could we add an exit condition if all aliases were partition aliases 
> > and belong to the same table i.e. we will end up with the same predicate?
> > 
> > This might be useful if we still keep the logic in HIVE-11726.

Coming back to this; I think it won't be that easy, because you will still have 
the original predicate, and thus, you will derive it from it.

It seems like the logic in HIVE-11726 should be dropped, and we should only 
keep the pushing IN clause to metastore logic there. I will comment on the JIRA 
case and seek John's feedback.


- Jesús


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review99389
---


On Sept. 10, 2015, 6:40 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Sept. 10, 2015, 6:40 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d2c5885 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-01 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 1, 2015, 11:42 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java dffdb5c 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
825938a 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
eca29df 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
  ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-01 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review101292
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
(line 372)


1. The check "children.size() < 2" is either unnecessary or we have a bug 
some where else.
2. We should also remove single col in clause where col is partition col


- John Pullokkaran


On Oct. 1, 2015, 11:42 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 1, 2015, 11:42 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java dffdb5c 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 825938a 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review101804
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
(line 377)


1. We shouldn't remove non deterministic UDFs.
2. Also make sure that partitionPruner isn't considering non deterministic 
UDFs
3. Please add testcase that involves non deterministic udf
4. Add a test case involving conditional expressions (case, If..)
Ex: select key from t1 where (case when key=1 then 10 else 20 end) in 
(10,11);


- John Pullokkaran


On Oct. 1, 2015, 11:42 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 1, 2015, 11:42 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java dffdb5c 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 825938a 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 7, 2015, 7:47 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 54a529e 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
825938a 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
eca29df 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
  ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread Hari Sankar Sivarama Subramaniyan 


> On Oct. 7, 2015, 6:21 p.m., John Pullokkaran wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java,
> >  line 377
> > 
> >
> > 1. We shouldn't remove non deterministic UDFs.
> > 2. Also make sure that partitionPruner isn't considering non 
> > deterministic UDFs
> > 3. Please add testcase that involves non deterministic udf
> > 4. Add a test case involving conditional expressions (case, If..)
> > Ex: select key from t1 where (case when key=1 then 10 else 20 end) in 
> > (10,11);

Thanks for reviewing the patch, John
1, 3, 4. Done in the new patch
2. Verified in the existing code base


- Hari Sankar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review101804
---


On Oct. 7, 2015, 7:47 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 7, 2015, 7:47 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 54a529e 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 825938a 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review101810
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
(line 384)


WALKERSTATE should be UNKNOWN.


- John Pullokkaran


On Oct. 7, 2015, 7:47 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 7, 2015, 7:47 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 54a529e 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 825938a 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review101814
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
(line 385)


Add unit test cases for:
1. DNF(CNF)
expr1 or expr2 (where expr2 is conjuctine tree)
2. In clause involving part/non-part col that are combined by disjunction


- John Pullokkaran


On Oct. 7, 2015, 7:47 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 7, 2015, 7:47 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 54a529e 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 825938a 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 7, 2015, 10:57 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java bf48f69 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
825938a 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
eca29df 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
  ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review101840
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
(line 392)


This too should be unknown right?


- John Pullokkaran


On Oct. 7, 2015, 10:57 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 7, 2015, 10:57 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java bf48f69 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 825938a 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 55edd90 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 4cfb97e 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-08 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 8, 2015, 5:26 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java bf48f69 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
825938a 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
eca29df 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-11 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review102185
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 154)


Checking for ExprNodeConstantDesc does not seem necessary.
If children are not null, we loop.
Otherwise, we just go to the end of the method.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 173)


This check does not seem necessary, as method will return null by default.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 203)


I think the children.size() should not be necessary, as we will always have 
at least two children, right?



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 215)


exprContainsOnlyPartitionColOrVirtualColOrConstants is recursive. Why 
cannot we pass (ExprNodeGenericFuncDesc)children.get(0) directly to get whether 
we can apply the optimization or not?



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 249)


What about expressions such as a+5, where column a is a partition column 
that belongs to table x? Shouldn't tab alias be x?

I do not understand that here (and subsequently) we call to 
en.getExprString() as an alternative to the table alias when the node is not 
ExprNodeColumnDesc, as I think it is not the way to cluster together the 
partition expressions that referred to the same table. Or I am missing 
something?


- Jesús Camacho Rodríguez


On Oct. 8, 2015, 5:26 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 8, 2015, 5:26 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java bf48f69 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> d83636d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 825938a 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-10-12 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 12, 2015, 10:52 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Changes
---

Incorporated Jesus's comments in the latest upload.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java bf48f69 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
825938a 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
eca29df 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/pointlookup2.q.out 700fbde 
  ql/src/test/results/clientpositive/pointlookup3.q.out 60a276b 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-26 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 26, 2015, 1:55 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java dc79415 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 439f616 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
d83636d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
825938a 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
eca29df 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-28 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 28, 2015, 10:48 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Changes
---

Addressed John's comments from the previous upload.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 59b66cd 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 7ee5081 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
4799b4d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
7cdc730 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
4320f01 
  
ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out
 e30465d 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 
7e28c77 
  
ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out
 1103e80 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-29 Thread Hari Sankar Sivarama Subramaniyan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/
---

(Updated Oct. 29, 2015, 11:32 p.m.)


Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
Pullokkaran.


Repository: hive-git


Description
---

Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 59b66cd 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 7ee5081 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
4799b4d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
7cdc730 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
7262164 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
  ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
  ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
  ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
  ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
  ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
eca29df 
  ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
4320f01 
  
ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out
 e30465d 
  ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
  ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 
7e28c77 
  
ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out
 1103e80 

Diff: https://reviews.apache.org/r/37778/diff/


Testing
---

Local testing done. More unit tests coming in the next patch.


Thanks,

Hari Sankar Sivarama Subramaniyan

Re: Review Request 37778: HIVE-11634

2015-10-29 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review104502
---



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 338)


If all expr nodes are partition cols/constants and if they come from same 
table then we should bail out.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java 
(line 470)


The col names in struct should be unique.
GetName would retutn class name and could result in data corruption as 
there could be duplicate names in struct typeinfo.

We should instead use ExprNodeGenericFuncDesc.newInsatnce which will take 
care of generating appropriate struct col names..


- John Pullokkaran


On Oct. 29, 2015, 11:32 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 29, 2015, 11:32 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 59b66cd 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 7ee5081 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> 4799b4d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 7cdc730 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
> 4320f01 
>   
> ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out
>  e30465d 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 
> 7e28c77 
>   
> ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out
>  1103e80 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-11-02 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review104740
---

Ship it!


Ship It!

- John Pullokkaran


On Oct. 29, 2015, 11:32 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 29, 2015, 11:32 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 59b66cd 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 7ee5081 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> 4799b4d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 7cdc730 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
> 4320f01 
>   
> ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out
>  e30465d 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 
> 7e28c77 
>   
> ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out
>  1103e80 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>

Re: Review Request 37778: HIVE-11634

2015-11-02 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37778/#review104741
---

Ship it!


- John Pullokkaran


On Oct. 29, 2015, 11:32 p.m., Hari Sankar Sivarama Subramaniyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37778/
> ---
> 
> (Updated Oct. 29, 2015, 11:32 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan, Jesús Camacho Rodríguez, and John 
> Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 59b66cd 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 7ee5081 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/PartitionColumnsSeparator.java
>  PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/PointLookupOptimizer.java 
> 4799b4d 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/pcr/PcrExprProcFactory.java 
> 7cdc730 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java 
> 7262164 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java 6a31689 
>   ql/src/test/queries/clientpositive/pcs.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/pointlookup.q 1aef2ef 
>   ql/src/test/queries/clientpositive/pointlookup2.q 31bebbb 
>   ql/src/test/queries/clientpositive/pointlookup3.q 3daa94b 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization_acid.q.out 
> eca29df 
>   ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 
> 4320f01 
>   
> ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out
>  e30465d 
>   ql/src/test/results/clientpositive/pcs.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/pointlookup.q.out 7e19be4 
>   ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 
> 7e28c77 
>   
> ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out
>  1103e80 
> 
> Diff: https://reviews.apache.org/r/37778/diff/
> 
> 
> Testing
> ---
> 
> Local testing done. More unit tests coming in the next patch.
> 
> 
> Thanks,
> 
> Hari Sankar Sivarama Subramaniyan
> 
>