alamb commented on PR #48468: URL: https://github.com/apache/arrow/pull/48468#issuecomment-4983682076
> (the encoded size isn't known until the data is written) While this is true, the writer typically can get a very good estimate of the currently accumulated size (that is what arrow-rs does) IN arrow-rs we found that there were use cases for both row group row limits **AND** row group byte limits . For example people may wish to limit row group row counts to improve pruning effectiveness. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
