raulcd commented on PR #49262:
URL: https://github.com/apache/arrow/pull/49262#issuecomment-3898089343
Hi @rustyconover no need to jump in a call (even though happy to have a chat
and a coffee).
The use case makes sense to me. As per the API, personally, a function like
`ipc.serialize_dictionaries(batch, memo)` makes more sense to me. On the C++
API everything already sits on the `ipc/{reader/writer}.h` so feels natural to
have this into `pyarrow.ipc`.
I think dictionary deltas could shine on your use case. The current
implementation does full dictionary replacement when the pointer changes. For
large dictionaries over many batches that could be a significant improvement
maybe you want to consider it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
