assignUser commented on issue #43929: URL: https://github.com/apache/arrow/issues/43929#issuecomment-2326823737
Writing to an mmap'ed file will copy the data into the mmap'ed region (so not zero copy) and still write to disk. The write will likely just happen somewhat async in the background and happens with less indirection. So it will appear to be faster but the actual difference is likely much less pronounced then what you measured compared to the 'direct' write to disk sans mmap. I am a bit unclear what you actually want to achieve here so I can't really say what the best thing to do would be. Could you explain your actual use case, so we can avoid an x-y problem. If you want to share the data (zero-copy, as fast as possible) with another process using shared memory (without a real file) or the [C API](https://arrow.apache.org/docs/format/CDataInterface.html). How exactly you want to do that depends on the language your consumers are using but [nanoarrow](https://arrow.apache.org/nanoarrow) will likely be helpful in any case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
