viirya commented on code in PR #8794:
URL: https://github.com/apache/arrow-datafusion/pull/8794#discussion_r1445475030
##########
datafusion/core/src/physical_optimizer/enforce_distribution.rs:
##########
@@ -1218,33 +1217,26 @@ fn ensure_distribution(
}
}
- if enable_round_robin
- // Operator benefits from partitioning (e.g. filter):
- && (would_benefit && repartition_beneficial_stats)
- // Unless partitioning doesn't increase the partition count,
it is not beneficial:
- && child.plan.output_partitioning().partition_count() <
target_partitions
- {
- // Increase parallelism by adding round-robin repartitioning
- // on top of the operator. Note that we only do this if the
- // partition count is not already equal to the desired
partition
- // count.
- child = add_roundrobin_on_top(child, target_partitions)?;
- }
-
// Satisfy the distribution requirement if it is unmet.
match requirement {
Distribution::SinglePartition => {
child = add_spm_on_top(child);
Review Comment:
Although due to `would_benefit`, single partition won't satisfy the
condition to add roundrobin partitioning. This change makes it explicitly clear
that single partition cases don't add roundrobin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
