slachiewicz commented on issue #38809:
URL: https://github.com/apache/arrow/issues/38809#issuecomment-5385451624
The type-promotion half of this report is fixed. The snippet in the
description works from **19.0.0** onwards:
```
18.1.0 -> ArrowTypeError: struct fields don't match or are in the wrong
order:
Input fields: struct<k1: null, k3: null>
output fields: struct<k1: int64, k2: int64, k3: string>
19.0.0 -> b: struct<k1: int64, k2: int64, k3: string>
25.0.1 -> same
```
Schema unification of structs was never the problem: `MergeStructs` in
`cpp/src/arrow/type.cc` has been there since #36846, the PR that introduced
`promote_options`. The failure was on the cast side, and GH-44555 / #44587
fixed it by letting `CastStruct` fill output fields absent from the input with
nulls. #45246 later added field reordering on top.
On the second half of the report, the claim that `field_merge_options` is
inert is not right. `options.unify_schemas` is set from `promote_options`, in
the same commit that added `field_merge_options`:
https://github.com/apache/arrow/blob/515410b2a1/python/pyarrow/table.pxi#L6338
What is real is that the `unify_schemas` keyword is swallowed by `**kwargs`
without a warning, along with any other unrecognised keyword — so
`unify_schemas=False, promote_options="permissive"` unifies anyway. That is not
specific to struct promotion, so I have split it out as #50957.
There was no Python test covering struct promotion, so I have opened #50956
with the case from this issue plus nested-struct and `list<struct>` variants.
With that merged this can be closed as fixed.
*This comment was created with AI assistance.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
