Hi,

Motivated by [1], I wonder if it is possible to write to IPC without
writing the data to an intermediary buffer.

The challenge is that the header of an IPC message [header][data] requires:

* the positions of the buffers
* the total length of the body

For uncompressed data, we could compute these before-hand at `O(C)` where C
is the number of columns. However, I am unable to find a way of computing
these ahead of writing for compressed buffers: we need to compress the data
to know its compressed (and thus buffers) size.

Is this understanding correct?

Best,
Jorge

[1] https://github.com/pola-rs/polars/issues/2639

Reply via email to