heman026 opened a new issue, #1733:
URL: https://github.com/apache/iceberg-python/issues/1733
### Question
I have read a parquet file and loaded into pyarrow table using read_table
method. I want to write this arrow table to iceberg table. But I am getting the
following error
> "pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays,
consider casting input from `string` to `large_string` first.".
Below is the code I am using.
```
arrow_table = pq.read_table(file,coerce_int96_timestamp_unit='us')
iceberg_table = return catalog.create_table_if_not_exists(iceberg_table,
schema, partition_spec=partition_spec,
properties={
'downcast-ns-timestamp-to-us-on-write': True,
PYARROW_USE_LARGE_TYPES_ON_READ: True
})
iceberg_table.overwrite(arrow_table )
```
Kindly help me resolve this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
