[
https://issues.apache.org/jira/browse/ARROW-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16266156#comment-16266156
]
ASF GitHub Bot commented on ARROW-1783:
---------------------------------------
mrocklin commented on issue #1362: ARROW-1783: [Python] Provide a "component"
dict representation of a serialized Python object with minimal allocation
URL: https://github.com/apache/arrow/pull/1362#issuecomment-347033589
```python
>>> list(components.keys())
['num_buffers', 'data', 'num_tensors']
```
I'm curious what the metadata here is supposed to mean and how it relates to
`data`. Just curious though.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [Python] Convert SerializedPyObject to/from sequence of component buffers
> with minimal memory allocation / copying
> ------------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-1783
> URL: https://issues.apache.org/jira/browse/ARROW-1783
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Python
> Reporter: Wes McKinney
> Assignee: Wes McKinney
> Labels: pull-request-available
> Fix For: 0.8.0
>
>
> See discussion on Dask org:
> https://github.com/dask/distributed/pull/931
> It would be valuable for downstream users to compute the serialized payload
> as a sequence of memoryview-compatible objects without having to allocate new
> memory on write. This means that the component tensor messages must have
> their metadata and bodies in separate buffers. This will require a bit of
> work internally reassemble the object from a collection of {{pyarrow.Buffer}}
> objects
> see also ARROW-1509
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
