[
https://issues.apache.org/jira/browse/ARROW-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902129#comment-16902129
]
Joris Van den Bossche commented on ARROW-5610:
----------------------------------------------
[~lidavidm] no apologies needed, it was just a question to check the status :)
> > You want to transfer a table containing a column of that type to and from
> > Python. Right now, you can read that data from Python, but you can't create
> > a table with that type
>
> I'm curious, which error do you get when trying to do so?
I tried this out, and so if you have an IPC message that contains an extension
type unknown to Python / C++, and you read that into a (pyarrow) Table, you
don't get an error at the moment, but it falls back to the storage type. But
which also means that you loose all information about the extension type
defined elsewhere (as discussed above).
To me, it seems that we would need some way to have an "unknown extension type"
in C++ that can have arbitrary name and metadata to be able to receive such
data.
> [Python] Define extension type API in Python to "receive" or "send" a foreign
> extension type
> --------------------------------------------------------------------------------------------
>
> Key: ARROW-5610
> URL: https://issues.apache.org/jira/browse/ARROW-5610
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Wes McKinney
> Priority: Major
> Fix For: 1.0.0
>
>
> In work in ARROW-840, a static {{arrow.py_extension_type}} name is used.
> There will be cases where an extension type is coming from another
> programming language (e.g. Java), so it would be useful to be able to "plug
> in" a Python extension type subclass that will be used to deserialize the
> extension type coming over the wire. This has some different API requirements
> since the serialized representation of the type will not have knowledge of
> Python pickling, etc.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
