bkietz commented on issue #20489:
URL: https://github.com/apache/arrow/issues/20489#issuecomment-1708585897
`FieldPath::Get(list_array)` would return the child data for the list array,
IE the undelimited list values. This is disallowed in scalar expressions
because it results in a column whose length is entirely unrelated to the
lengths of sibling columns. Consider projecting `.s, .l[0]` from
`Table.from_pydict({"s": ["a", "b"], "l": [[1,2,3], [4,5,6]]})`; we can't do
that just as we can't do `Table.from_pydict({".s": ["a", "b"], ".l[0]":
[1,2,3,4,5,6]})`. It might be desirable to let pyarrow's field reference
mechanism be more flexible (as in substrait) and implicitly replace a path like
`.l[0:1]` with a call to `list_slice` or similar. However that should be a
python API decision and should not propagate to the C++ library, where it's
important that using FieldPath::Get be consistently performant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
