kevinjqliu commented on PR #1498:
URL: https://github.com/apache/iceberg-python/pull/1498#issuecomment-2580930119
I think generally we want this kind of error message when calling the
`pyarrow_to_schema` function, which is used in the create table path and the
write path.
```
Column '{field_name}' has an unsupported type: {field_type}
```
Let's take a look at `pyarrow_to_schema`.
It takes a `schema: pa.Schema` as input, this is a [pyarrow Schema
class](https://arrow.apache.org/docs/python/generated/pyarrow.Schema.html#pyarrow.Schema).
And it has a list of
[Fields](https://arrow.apache.org/docs/python/generated/pyarrow.Schema.html#pyarrow.Schema.field).
Each [Field has a corresponding name and
type](https://arrow.apache.org/docs/python/generated/pyarrow.field.html).
It uses the `visit_pyarrow` decorator, which dispatches based on the type of
arrow object.
https://github.com/search?q=repo%3Aapache%2Ficeberg-python%20visit_pyarrow.register&type=code
The function will start from `visit_pyarrow.register(pa.Schema)` and
continues down until it hits the primitives
(`visit_pyarrow.register(pa.DataType)`) .
Before the primitives, it should dispatch to `pa.field`. I think this is
where we want to emit the error since the field has name/type/optionality/etc.
Currently theres no `visit_pyarrow` decorator for `pa.field`, so we'd need
to add one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
