Diveyam-Mishra opened a new pull request, #5048: URL: https://github.com/apache/calcite/pull/5048
## Jira Link [[CALCITE-7618](https://issues.apache.org/jira/browse/CALCITE-7618)] ## Changes Proposed This PR implements filter pushdown support for the file adapter's CSV table using a planner-rule-based approach instead of a `FilterableTable` interface. This allows Calcite to make more intelligent planning decisions, estimate cost reductions, and display pushed-down predicates in `EXPLAIN` plans. ### Implementation Details: 1. **Rule-Based Pushdown**: * Introduced `CsvFilterTableScanRule` which matches `LogicalFilter` on a `CsvTableScan` and pushes simple equality predicates (`col = literal`) into the scan. * Introduced `CsvProjectFilterTableScanRule` which matches `LogicalProject` → `LogicalFilter` → `CsvTableScan` and pushes down the filter first, preventing the planner from prematurely collapsing projects and filters into a generic `EnumerableCalc` and bypassing pushdown. 2. **Scan State & Costing**: * Updated `CsvTableScan` to store and propagate `@Nullable String[] filterValues`. * Updated `CsvTableScan#computeSelfCost` to reduce planning cost proportionally to the number of pushed-down filters. * Extended `CsvTableScan#explainTerms` to format filters as `filters=[[colIndex=value]]` in `EXPLAIN` outputs. 3. **Execution Support**: * Added `CsvTranslatableTable#scan(DataContext, int[], String[])` which is dynamically invoked by the generated code when filters are present. * Made `CsvEnumerator#converter` package-private so it can be reused inside `CsvTranslatableTable` to resolve correct row converters (ensuring single-column projections return raw objects rather than `Object[]` arrays to prevent class cast errors). 5. **Testing**: * Added target unit tests in `FileAdapterTest.java` verifying pushdown, projection combination, result correctness, and non-pushable residual filter persistence. * Updated existing plans in `testPushDownProjectAggregateWithFilter` to reflect the newly optimized scan plans. To verify the change, run: .\sqlline.bat -u "jdbc:calcite:model=file/src/test/resources/smart.json" -n admin -p admin -e "!set maxwidth 10000" -e "explain plan for select name, empno from EMPS where deptno = 20" Before this change, the plan was: PLAN=EnumerableCalc(expr#0..2=[{inputs}], expr#3=[20], expr#4=[=($t2, $t3)], NAME=[$t1], EMPNO=[$t0], $condition=[$t4]) CsvTableScan(table=[[SALES, EMPS]], fields=[[0, 1, 2]]) After this change, the filter and projection are pushed down into CsvTableScan, resulting in: CsvTableScan(table=[[SALES, EMPS]], fields=[[1, 0]], filters=[[2=20]]) This demonstrates that the scan now reads only the required columns (name, empno) and applies the deptno = 20 filter during the table scan itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
