etseidl commented on PR #12217: URL: https://github.com/apache/arrow-rs/pull/12217#issuecomment-6859891018
As I feared, this PR removes a useful optimization. I don't think matching overflow behavior justifies that loss. I think we should add a section to the `WriterProperties` documentation explaining how write batch size interacts with page and dictionary size limits, and how none are guarantees, but best effort. We can then link to that section from the various setters. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
