adriangb commented on PR #9972: URL: https://github.com/apache/arrow-rs/pull/9972#issuecomment-4545728646
> @etseidl after much profiling, debugging, etc. I've been able to get this to work with no performance impact (within noise). I recognize this is a non-trivial change but it introduces no public APIs and in theory if it is problematic in any way we can back out of it. The benefit of doing things this way is that we automatically patch buggy / problematic page size blowouts for everyone, without code changes needed on their end or guessing of column sizes necessary. > > One thing we could do to derisk if you want: add a config option to disable this behavior. > > Thanks for reviewing this, I hope we can make it work 😄 @etseidl sorry to bug you again. I've re-stacked the commits to make the diff more reviewable, a good chunk of the diff is regression tests and benchmarks which I've tried to split out into their own commits to make things more reviewable. Let me know if there's anything else I can do to help make this more palatable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
