blaginin commented on code in PR #17395:
URL: https://github.com/apache/datafusion/pull/17395#discussion_r2320388690
##########
datafusion/datasource/src/source.rs:
##########
@@ -319,35 +318,8 @@ impl ExecutionPlan for DataSourceExec {
projection: &ProjectionExec,
) -> Result<Option<Arc<dyn ExecutionPlan>>> {
match self.data_source.try_swapping_with_projection(projection)? {
- Some(new_plan) => {
- if let Some(new_data_source_exec) =
- new_plan.as_any().downcast_ref::<DataSourceExec>()
- {
- let projection_mapping = ProjectionMapping::try_new(
- projection.expr().iter().cloned(),
- &self.schema(),
- )?;
-
- // Project the equivalence properties to the new schema
- let projected_eq_properties = self
- .cache
- .eq_properties
- .project(&projection_mapping,
new_data_source_exec.schema());
-
- let preserved_exec = DataSourceExec {
- data_source:
Arc::clone(&new_data_source_exec.data_source),
- cache: PlanProperties::new(
- projected_eq_properties,
- new_data_source_exec.cache.partitioning.clone(),
- new_data_source_exec.cache.emission_type,
- new_data_source_exec.cache.boundedness,
- )
-
.with_scheduling_type(new_data_source_exec.cache.scheduling_type),
- };
Review Comment:
for anyone wondering why: previously we had to store cache information
because it was lost on projection
(https://github.com/apache/datafusion/issues/17077).
Now `FileScanConfig::eq_properties` works correctly (fixed by Adrian in
https://github.com/apache/datafusion/pull/17323), so when passed to
`DataSourceExec`, the new information is used
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
