---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41984/#review113327
---
Ship it!
Ship It!
- Sergio Pena
On Jan. 6, 2016, 4:52 p.m., Aihua Xu wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41984/
> ---
>
> (Updated Jan. 6, 2016, 4:52 p.m.)
>
>
> Review request for hive, Sergio Pena and Xuefu Zhang.
>
>
> Repository: hive-git
>
>
> Description
> ---
>
> HIVE-12762: Common join on parquet tables returns incorrect result when
> hive.optimize.index.filter set to true
>
>
> Diffs
> -
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
> 9a7d990baaabfde8e564f00bb1fcfe30cd16dc90
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ProjectionPusher.java
> 017676bec2163f04fb95f43224a4f8743fa49f55
> ql/src/test/queries/clientpositive/parquet_join2.q PRE-CREATION
> ql/src/test/results/clientpositive/parquet_join2.q.out PRE-CREATION
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/ExpressionTree.java
> 577d95d1a15a54c2804349b3e5e68d83b72df664
>
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java
> eeff131cbc14d7ef554517109612ae7d891f8003
>
> Diff: https://reviews.apache.org/r/41984/diff/
>
>
> Testing
> ---
>
> We have two issues: 1. We are filtering the parquet columns based on the last
> filter condition in the query. So if the query contains multiple instances of
> the same table, e.g., join on the same table with different filter
> conditions, then we could get incorrect result; 2. rewriteLeaves
> implementation in SearchArgumentImpl is not accurate since the different
> leaves could be sharing the same object. The current implementation could
> change the leave index multiple times to an incorrect value.
>
> The patch will merge all the filter conditions (create OR expression on all
> the filters) so that the columns which will be used during operator won't be
> filtered during earlier splitting stage. rewriteLeaves is reimplemented to
> get all the unique leaves first and replace in place.
>
>
> Thanks,
>
> Aihua Xu
>
>
