alamb commented on issue #24438: URL: https://github.com/apache/datafusion/issues/24438#issuecomment-5424270946
> We now see the **core idea**: the external system can indicate that data is unsorted but non‑overlapping. This is **similar to other metadata properties** (e.g., sort order) that come from the external system. Yes, I think is likely a common pattern (and the use case you describe of "more partitions than cores" certainly happens for other users -- like we have it at Influx for example Maybe that is the usecase we can highlight -- "supporting analytics over large amounts of pre-partitioned data" 🤔 It sounds pretty neat and not something that I am aware has been written about before -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
