TheR1sing3un commented on PR #8091: URL: https://github.com/apache/paimon/pull/8091#issuecomment-4603382027
Note: this PR raises the polars floor to dodge the broken versions, but the underlying coupling stays. Primary-key merge reads still decode each arrow batch through `polars.from_arrow` in `read_next_df` (`pypaimon/read/reader/iface/record_batch_reader.py`), so the read path's support for nested/complex arrow types remains tied to whatever the installed polars build can decode. To fix this more fundamentally, that conversion could be reworked to not rely on polars for the arrow -> row step (pyarrow natively supports map/struct), which would remove the version sensitivity entirely. Suggest tracking that as a separate follow-up PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
