wiedld commented on code in PR #14637:
URL: https://github.com/apache/datafusion/pull/14637#discussion_r1957024693
##########
datafusion/physical-optimizer/src/enforce_sorting/mod.rs:
##########
@@ -298,20 +355,11 @@ pub fn parallelize_sorts(
vec![requirements],
),
))
- } else if is_coalesce_partitions(&requirements.plan) {
- // There is an unnecessary `CoalescePartitionsExec` in the plan.
- // This will handle the recursive `CoalescePartitionsExec` plans.
+ } else if unneeded_coalesce {
requirements = remove_bottleneck_in_subplan(requirements)?;
- // For the removal of self node which is also a
`CoalescePartitionsExec`.
- requirements = requirements.children.swap_remove(0);
+ requirements = requirements.update_plan_from_children()?;
- Ok(Transformed::yes(
- PlanWithCorrespondingCoalescePartitions::new(
-
Arc::new(CoalescePartitionsExec::new(Arc::clone(&requirements.plan))),
- false,
- vec![requirements],
- ),
- ))
Review Comment:
With the context refactor, the `remove_bottleneck_in_subplan` is no longer
removing too many nodes.
As a result, here we no longer have to add back the coalesce again (old line
310).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
