alamb commented on PR #7743: URL: https://github.com/apache/arrow-datafusion/pull/7743#issuecomment-1752106345
> > But now that I look at that plan, perhaps the issue is that there is more than one file in each group, so the sort order can't be maintained 🤔 > > Perhaps if each output file were larger we would avoid that issue. It would be nice if you could configure a desired file output size rather than relying on the plan partitioning... I opened #7767 for this. I think the issue is explained here: https://github.com/apache/arrow-datafusion/blob/3d1b23a04bdc04c526e2dcb06e0cf1995707587d/datafusion/core/src/datasource/physical_plan/mod.rs#L408-L467 (which is not all that easy to find) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
