westonpace commented on issue #13978: URL: https://github.com/apache/arrow/issues/13978#issuecomment-1227845349
> If arrow::Buffer doesn't own its own memory, how do you know that arrow has finished using the memory contained in the buffer to free it? This is particularly useful in streaming applications of large amount of arrow ipc. Does arrow ever build references/pointers into the buffer that last beyond the lifetime of the std::shared_ptr<arrow::Buffer>? This is a good question. In general, I would say that an IPC reader/writer should be finished with the buffer by the time it is closed AND destructed (maybe just closed but closed AND destructed doesn't seem too onerous). I am fairly certain the current IPC reader/writer conforms to this rule, at least for the synchronous APIs. Either way, it would be good to document this and make sure we do follow it. Currently, anything using the streaming execution engine (e.g. ExecPlan, dataset scans) could potentially violate the above rule if the plan execution is cancelled partway through. I am hoping to fix this as part of ARROW-15732. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
