mach-kernel opened a new pull request, #17911: URL: https://github.com/apache/datafusion/pull/17911
## Which issue does this PR close? #15718, though I can file a new bug report. Should be easy to on a round-tripped plan against a Hive-partitioned table. ## Rationale for this change Since table partition columns are serialized in `table_partition_cols` for ListingScan, https://github.com/apache/datafusion/pull/15737 made a change to remove them from the `TableScan` schema so they would not be duplicated. Unfortunately, queries with projections that reference the partition columns would fail to resolve here: https://github.com/apache/datafusion/blob/daeb6597a0c7344735460bb2dce13879fd89d7bd/datafusion/proto/src/logical_plan/mod.rs#L382-L390 The round-trip test used a partition column in its `SELECT` to exercise this, however the unoptimized plan did not have projections decorated (so they would not get serialized, and there would be no lookup to fail). As in, `scan.projection` above would be `None`. ## What changes are included in this PR? - Do the projection name -> index binding using `ListingTable::schema`, at which point the partition columns are decorated - Since we're testing the round-tripping of the plan, make a plan with `LogicalPlanBuilder` that explicitly specifies a projection to ensure we're testing this functionality - This test fails on `main` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
