-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26189/
-----------------------------------------------------------
Review request for drill and Jinfeng Ni.
Bugs: DRILL-1465
https://issues.apache.org/jira/browse/DRILL-1465
Repository: drill-git
Description
-------
The DrillPushPartitionFilterRule was handling the case where the DrillFilterRel
appeared above a DrillScanRel. This fix enhances push partition filter rule to
handle DrillFilterRel appearing above a DrillProjectRel. Further, there was an
issue with EasyGroupScan where the new DrillScanRel that had a smaller file
selection was not being considered by the Volcano planner even though the stats
for the new scan had a smaller rowcount - the reason being the digest was an
exact match with the previous DrillScanRel. In this fix I modified the group
scans' digests (both ParquetScan and EasyGroupScan) to include number of files
being considered.
Diffs
-----
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DirPathBuilder.java
f911d7c
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushPartitionFilterIntoScan.java
940b6c2
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java
ff1d5e4
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/easy/EasyGroupScan.java
35d1868
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
6d3cf5a
exec/java-exec/src/test/java/org/apache/drill/TestPartitionFilter.java
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1994/Q1/orders_94_q1.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1994/Q2/orders_94_q2.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1994/Q3/orders_94_q3.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1994/Q4/orders_94_q4.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1995/Q1/orders_95_q1.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1995/Q2/orders_95_q2.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1995/Q3/orders_95_q3.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/csv/1995/Q4/orders_95_q4.csv
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1994/Q1/orders_94_q1.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1994/Q2/orders_94_q2.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1994/Q3/orders_94_q3.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1994/Q4/orders_94_q4.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1995/Q1/orders_95_q1.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1995/Q2/orders_95_q2.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1995/Q3/orders_95_q3.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/json/1995/Q4/orders_95_q4.json
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1994/Q1/orders_94_q1.parquet
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1994/Q2/orders_94_q2.parquet
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1994/Q3/orders_94_q3.parquet
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1994/Q4/orders_94_q4.parquet
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1995/Q1/orders_95_q1.parquet
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1995/Q2/orders_95_q2.parquet
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1995/Q3/orders_95_q3.parquet
PRE-CREATION
exec/java-exec/src/test/resources/multilevel/parquet/1995/Q4/orders_95_q4.parquet
PRE-CREATION
Diff: https://reviews.apache.org/r/26189/diff/
Testing
-------
Added several unit tests for parquet, json and csv formats. These use new
resource files that will also be useful for building other tests. Ran the usual
regression tests.
Thanks,
Aman Sinha
