Within OmniSci (MapD), we have the following code that takes a pointer and length and reads to a NumPy array before calling py_buffer:

https://github.com/omnisci/pymapd/blob/master/pymapd/shm.pyx#L31-L52

Is it possible to eliminate the NumPy step and go directly do an Arrow buffer? There is both a concern that we're doing an unnecessary memory copy, as well as wanting to defer to the Arrow way of doing things as much as possible rather than having our own shims like these.

Reply via email to