zinking commented on PR #36967: URL: https://github.com/apache/arrow/pull/36967#issuecomment-1691052654
> Dataset returns results unordered, so does it make sense to ask for a specific offset at all? @pitrou , it's actually common in java HADOOP ecosystem. when a parquet file is big, it will be split into multiple pieces , and multiple scanners will read them simultaneously to increase throughput. within the split, whether it is ordered doesn't matter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
