Hi Experts,
Can anyone please highlight if it is possible to cast struct to map type?
I tried the following but it seems to be producing an error as below.
pyarrow.lib.ArrowNotImplementedError: Unsupported cast from
struct<first_name: string, last_name: string> to map using function cast_map
Note: Snippet is just an example to show the problem.
Code Snippet:
table_schema = pa.schema([pa.field("id", pa.int32()), pa.field("names",
pa.map_(pa.string(), pa.string()))])
table_data = [{"id": 1,"names": {"first_name": "Tyler", "last_name":
"Brady"}},
{"id": 2,"names": {"first_name": "Walsh", "last_name": "Weaver"}}]
tbl = pa.Table.from_pylist(table_data)
print(tbl)
print(tbl.cast(table_schema))
print(tbl)
Error :
id: int64
names: struct<first_name: string, last_name: string>
child 0, first_name: string
child 1, last_name: string
----
id: [[1,2]]
names: [
-- is_valid: all not null
-- child 0 type: string
["Tyler","Walsh"]
-- child 1 type: string
["Brady","Weaver"]]
Traceback (most recent call last):
File "/Users/
infant.a...@cognitedata.com/Documents/Github/HubOcean/demo/pyarrow_types.py",
line 220, in <module>
print(tbl.cast(table_schema))
File "pyarrow/table.pxi", line 3489, in pyarrow.lib.Table.cast
File "pyarrow/table.pxi", line 523, in pyarrow.lib.ChunkedArray.cast
File "/Users/
infant.a...@cognitedata.com/Library/Caches/pypoetry/virtualenvs/demo-LzMA3Hsd-py3.10/lib/python3.10/site-packages/pyarrow/compute.py",
line 391, in cast
return call_function("cast", [arr], options)
File "pyarrow/_compute.pyx", line 560, in pyarrow._compute.call_function
File "pyarrow/_compute.pyx", line 355, in pyarrow._compute.Function.call
File "pyarrow/error.pxi", line 144, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 121, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: Unsupported cast from
struct<first_name: string, last_name: string> to map using function cast_map
Regards,
Alex Vincent
