[
https://issues.apache.org/jira/browse/ARROW-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17529106#comment-17529106
]
Jonathan Keane commented on ARROW-14379:
----------------------------------------
The main motivation was sf columns here. I don't know of others off the top of
my head that have element-level attributes other than sf and like you mention
models. If the list is small enough, we can implement each of those (and have
already made it pretty easy for others to implement their own versions!). I'm
fine closing this as won't do for now unless|until we find a real usecase.
In response to [~westonpace]: that might probably also work — though I think we
would need nested struct arrays: value, attrs with the attrs having different
possible shapes and I'm not 100% sure if structs could handle elements with
arbitrary sizes like that.
> [R] Create a custom extension of list that stores row-level metadata
> --------------------------------------------------------------------
>
> Key: ARROW-14379
> URL: https://issues.apache.org/jira/browse/ARROW-14379
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: R
> Reporter: Jonathan Keane
> Priority: Major
>
> Since lists can be nested, we should be able store each element as something
> like {{list(value = "foo", attributes = list(attr1 = TRUE, attr2 = "baz"))}}
> and then we can reconstitute that in the R conversion to transfer the
> attributes element to attributes.
> This will be more efficient (since we get compression on the column +
> metadata/attributes) and we also will be able to filter these + use them in
> datasets since each row has all of the information about itself that it needs
> to roundtrip.
> This would get us SF columns for free
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
