IndifferentArea commented on PR #46730:
URL: https://github.com/apache/arrow/pull/46730#issuecomment-3019299582
Besides parquet related issue, there is **NO buffer sharing** mechanism in
current api.
- `AppendBuffer` should only be used for large string
- `AppendViewFromBuffer`
- for large string sharing, sharing through this interface will save a lot
of memory usage
- for inlined string it costs as much as directly `Append`.
These new interfaces won't introduce more cost for current interfaces from
my view.
I understand tradeoff on parquet a discussed above, but I believe this kind
of buffer sharing interface is lost in current implementation, and there is
definitely more need for this api apart from parquet.
Maybe we can open another issue/PR to discuss specifically whether/how
parquet should use this api on appending a huge page and append view from it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
