Re: Creating Buffer directly from pointer/length

2018-11-08 Thread Randy Zwitch
/pyarrow.foreign_buffer.html#pyarrow.foreign_buffer This takes an address, size and a Python object for having a reference on the object. In your case the last one can be None. Note that this will not do a copy and thus directly reference the shared memory. Uwe On Thu, Nov 8, 2018, at 6:49 PM, Randy Zwitch

Creating Buffer directly from pointer/length

2018-11-08 Thread Randy Zwitch
Within OmniSci (MapD), we have the following code that takes a pointer and length and reads to a NumPy array before calling py_buffer: https://github.com/omnisci/pymapd/blob/master/pymapd/shm.pyx#L31-L52 Is it possible to eliminate the NumPy step and go directly do an Arrow buffer? There is

Re: Help understanding IPC Message/Buffer structure

2018-07-12 Thread Randy Zwitch
Script, you can reference the > implementation here > < https://github.com/apache/arrow/blob/e14705745bb8d625b3c7dda2857e93cdfe848178/js/src/ipc/reader/binary.ts#L145>. > > Hope this clears things up, > > Paul > > > > On 07/12/2018 11:30 AM, Randy Zwitch wrote: >

Help understanding IPC Message/Buffer structure

2018-07-12 Thread Randy Zwitch
y mean/their structure and how to process them further. Thanks, Randy Zwitch