b-phi commented on issue #1619:
URL:
https://github.com/apache/iceberg-python/issues/1619#issuecomment-2648608512
Thanks Kevin, I should have been more clear. I am using the downcasting
functionality, which works great for unpartitioned tables. Even an
`IdentityTransform()` partition over a downcasted timestamp column works fine,
but any of the time truncations like `DayTransform` or `YearTransform` result
in the arrow kernel error above.
A table with the partition spec will handle downcasting timestamps correctly
on write
```python
partition_spec = PartitionSpec(
PartitionField(source_id=1, field_id=1001,
transform=IdentityTransform(), name='ts'),
)
```
But this partition spec throws the error
```python
partition_spec = PartitionSpec(
PartitionField(source_id=1, field_id=1001, transform=DayTransform(),
name='date'),
)
```
The write code
```python
os.environ['PYICEBERG_DOWNCAST_NS_TIMESTAMP_TO_US_ON_WRITE'] = '1'
schema = pa.schema([pa.field('timestamp', pa.timestamp('ns'))])
data = pa.Table.from_pydict({'timestamp': [1]}, schema=schema)
table.append(data)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
