TheR1sing3un opened a new pull request, #8091:
URL: https://github.com/apache/paimon/pull/8091
## Problem
Reading a `map` nested inside a `struct`/row from a primary-key table panics:
```
pyo3_runtime.PanicException: Arrow datatype Map(...) not supported by Polars.
```
Primary-key tables read through a row-based merge path that converts each
arrow batch with `polars.from_arrow` (`read_next_df`). polars `< 1.32` cannot
decode an arrow `Map` that is nested inside a struct and panics; a top-level
`map` column works fine.
| structure | polars 1.14 – 1.31 | polars >= 1.32 |
| --- | --- | --- |
| top-level `map<...>` column | OK | OK |
| `row<..., map<...>>` (nested) | panic | OK |
## Fix
Raise the polars lower bound to `>=1.32` on Python 3.9+. polars 1.32
requires Python `>=3.9`, so the Python 3.8 entry keeps the existing range (3.8
cannot install a fixed polars wheel; this is an upstream limitation). Adds a
regression test that reads a struct-nested map and a top-level map from a
primary-key table.
## Repro
```python
import polars as pl, pyarrow as pa
inner = pa.struct([pa.field("x", pa.int32())])
t = pa.struct([pa.field("m", pa.map_(pa.string(), inner))])
batch = pa.record_batch([pa.array([{"m": [("k", {"x": 1})]}], type=t)],
names=["c"])
pl.from_arrow(batch) # panics on polars < 1.32, OK on >= 1.32
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
