[jira] [Commented] (ARROW-1470) [C++] Add BufferAllocator abstract interface
[
https://issues.apache.org/jira/browse/ARROW-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588807#comment-16588807
]
Wes McKinney commented on ARROW-1470:
-
I'd like to take a look at this for 0.11
> [C++] Add BufferAllocator abstract interface
>
>
> Key: ARROW-1470
> URL: https://issues.apache.org/jira/browse/ARROW-1470
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
>Reporter: Wes McKinney
>Priority: Major
> Fix For: 0.11.0
>
>
> There are some situations ({{arrow::ipc::SerializeRecordBatch}} where we pass
> a {{MemoryPool*}} solely to call {{AllocateBuffer}} using it. This is not as
> flexible as it could be, since there are situation where we may wish to
> allocate from shared memory instead.
> So instead:
> {code}
> Func(..., BufferAllocator* allocator, ...) {
> ...
> std::shared_ptr buffer;
> RETURN_NOT_OK(allocator->Allocate(nbytes, &buffer));
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1470) [C++] Add BufferAllocator abstract interface
[
https://issues.apache.org/jira/browse/ARROW-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588805#comment-16588805
]
Wes McKinney commented on ARROW-1470:
-
Yes, these APIs assume that we are allocating CPU memory:
https://github.com/apache/arrow/blob/master/cpp/src/arrow/buffer.cc#L164
That detail is sort of "baked in". A BufferAllocator abstract interface would
allow other kinds of buffers to be returned
> [C++] Add BufferAllocator abstract interface
>
>
> Key: ARROW-1470
> URL: https://issues.apache.org/jira/browse/ARROW-1470
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
>Reporter: Wes McKinney
>Priority: Major
> Fix For: 0.11.0
>
>
> There are some situations ({{arrow::ipc::SerializeRecordBatch}} where we pass
> a {{MemoryPool*}} solely to call {{AllocateBuffer}} using it. This is not as
> flexible as it could be, since there are situation where we may wish to
> allocate from shared memory instead.
> So instead:
> {code}
> Func(..., BufferAllocator* allocator, ...) {
> ...
> std::shared_ptr buffer;
> RETURN_NOT_OK(allocator->Allocate(nbytes, &buffer));
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1470) [C++] Add BufferAllocator abstract interface
[
https://issues.apache.org/jira/browse/ARROW-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433737#comment-16433737
]
Antoine Pitrou commented on ARROW-1470:
---
Could you explain why this couldn't be solved by having a shared
memory-allocating MemoryPool subclass?
> [C++] Add BufferAllocator abstract interface
>
>
> Key: ARROW-1470
> URL: https://issues.apache.org/jira/browse/ARROW-1470
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
>Reporter: Wes McKinney
>Priority: Major
> Fix For: 0.10.0
>
>
> There are some situations ({{arrow::ipc::SerializeRecordBatch}} where we pass
> a {{MemoryPool*}} solely to call {{AllocateBuffer}} using it. This is not as
> flexible as it could be, since there are situation where we may wish to
> allocate from shared memory instead.
> So instead:
> {code}
> Func(..., BufferAllocator* allocator, ...) {
> ...
> std::shared_ptr buffer;
> RETURN_NOT_OK(allocator->Allocate(nbytes, &buffer));
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1470) [C++] Add BufferAllocator abstract interface
[
https://issues.apache.org/jira/browse/ARROW-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160660#comment-16160660
]
Wes McKinney commented on ARROW-1470:
-
Moving this to 0.8.0. I would like to give this some more thoughtful
consideration in a patch than adding it at last minute
> [C++] Add BufferAllocator abstract interface
>
>
> Key: ARROW-1470
> URL: https://issues.apache.org/jira/browse/ARROW-1470
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
> Fix For: 0.8.0
>
>
> There are some situations ({{arrow::ipc::SerializeRecordBatch}} where we pass
> a {{MemoryPool*}} solely to call {{AllocateBuffer}} using it. This is not as
> flexible as it could be, since there are situation where we may wish to
> allocate from shared memory instead.
> So instead:
> {code}
> Func(..., BufferAllocator* allocator, ...) {
> ...
> std::shared_ptr buffer;
> RETURN_NOT_OK(allocator->Allocate(nbytes, &buffer));
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
