ion-elgreco opened a new issue, #14154:
URL: https://github.com/apache/datafusion/issues/14154
### Describe the bug
A `LargeList(Struct({"foo": LargeUtf8})`) cannot be coerced to
`List(Struct({"foo": Utf8}))`. It however it works fine for
`LargeList(LargeUtf8) -> List(Utf8)` and `Struct({"foo": LargeUtf8}) ->
Struct({"foo": Utf8})`.
### To Reproduce
```python
import polars as pl
from deltalake import DeltaTable
tmp_path = "test_table__"
df = pl.DataFrame({"foo": [1], "bar": [[{"foo": "!"}]]})
df.write_delta(tmp_path, mode="overwrite", overwrite_schema=True)
DeltaTable(tmp_path).merge(
df.to_arrow(compat_level=1),
predicate="s.foo = t.foo",
source_alias="s",
target_alias="t",
large_dtypes=None,
).when_matched_update_all().execute()
```
```
DeltaError: Generic DeltaTable error: type_coercion
caused by
Error during planning: Failed to coerce then ([LargeList(Field { name:
"item", data_type: Struct([Field { name: "foo", data_type: Utf8View, nullable:
true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true,
dict_id: 0, dict_is_ordered: false, metadata: {} }), List(Field { name:
"element", data_type: Struct([Field { name: "foo", data_type: Utf8, nullable:
true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true,
dict_id: 0, dict_is_ordered: false, metadata: {} }), List(Field { name:
"element", data_type: Struct([Field { name: "foo", data_type: Utf8, nullable:
true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true,
dict_id: 0, dict_is_ordered: false, metadata: {} }), List(Field { name:
"element", data_type: Struct([Field { name: "foo", data_type: Utf8, nullable:
true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true,
dict_id: 0, dict_is_ordered: false, metadata: {} })]) and else (None) to common
types in CA
SE WHEN expression
```
### Expected behavior
Be able to coerce Large/view and normal arrow types in deeply nested types.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
