alamb commented on issue #7001:
URL:
https://github.com/apache/arrow-datafusion/issues/7001#issuecomment-1658659418
Also @wolffcm noted elsewhere that
> I see a lot of DF plans that contain something like this:
```
RepartitionExec: partitioning=Hash(...), input_partitions=12
RepartitionExec: partitioning=RoundRobinBatch(12), input_partitions=2
```
> At first I thought this was an unintentional artifact of how physical
planning happens, but now it seems to me that the intention of the RoundRobin
repartitioning is to ensure we paralellize the Hash repartitioning, which can
be expensive because it looks at each row. Do I have that right?
I believe this is correct, and perhaps would be something to improve in any
improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
