yonipeleg33 commented on PR #48468:
URL: https://github.com/apache/arrow/pull/48468#issuecomment-4960902486
@mbasmanova I'm not going to speak on behalf of the maintainers here, but at
least IMO:
1. Your current solution doesn't seem that bad, it's unclear to me that
there's much to gain from pushing it as a feature upstream.
2. As for justifications for this feature, and also as an answer to the
question directed to me:
This pr and my [arrow-rs
pr](https://github.com/apache/arrow-rs/pull/9357) both aim to provide a similar
functionality to parquet-hadoop's `parquet.block.size` config (linked in my pr
description), and this is the main justification for those changes.
And since parquet-hadoop doesn't have adjustable limits (either row
count or byte size), I'd say this feature is out of scope for arrow.
3. If anything, this should be discussed in a separate feature request, not
here.
Again, not representing any official opinion here, just my humble opinion.
Cheers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
