[
https://issues.apache.org/jira/browse/ARROW-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447675#comment-16447675
]
ASF GitHub Bot commented on ARROW-2451:
---------------------------------------
mitar commented on a change in pull request #1887: ARROW-2451: [Python] Handle
non-object arrays more efficiently in custom serializer.
URL: https://github.com/apache/arrow/pull/1887#discussion_r183298796
##########
File path: python/pyarrow/serialization.py
##########
@@ -37,11 +37,22 @@
# python_to_arrow.cc)
def _serialize_numpy_array_list(obj):
- return obj.tolist(), obj.dtype.str
+ if obj.dtype.str != '|O':
Review comment:
I think you could check here for `obj.dtype.hasobject`:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.dtype.hasobject.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Handle more dtypes efficiently in custom numpy array serializer.
> ----------------------------------------------------------------
>
> Key: ARROW-2451
> URL: https://issues.apache.org/jira/browse/ARROW-2451
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Robert Nishihara
> Assignee: Robert Nishihara
> Priority: Major
> Labels: pull-request-available
> Fix For: JS-0.4.0
>
>
> Right now certain dtypes like bool or fixed length strings are serialized as
> lists, which is inefficient. We can handle these more efficiently by casting
> them to uint8 and saving the original dtype as additional data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
