syun64 commented on code in PR #910:
URL: https://github.com/apache/iceberg-python/pull/910#discussion_r1674351468
##########
pyiceberg/io/pyarrow.py:
##########
@@ -1320,7 +1321,16 @@ def _cast_if_needed(self, field: NestedField, values:
pa.Array) -> pa.Array:
and pa.types.is_timestamp(values.type)
and values.type.unit == "ns"
):
- return values.cast(target_type, safe=False)
+ if target_type.tz == "UTC" and values.type.tz in
UTC_ALIASES or not target_type.tz and not values.type.tz:
+ return values.cast(target_type, safe=False)
+ if (
+ pa.types.is_timestamp(target_type)
+ and target_type.unit == "us"
+ and pa.types.is_timestamp(values.type)
+ and values.type.unit in {"s", "ms", "us"}
+ ):
+ if target_type.tz == "UTC" and values.type.tz in
UTC_ALIASES or not target_type.tz and not values.type.tz:
+ return values.cast(target_type)
Review Comment:
Does this look correct that we are casting the types in order to follow the
Iceberg Spec for Parquet Physical and Logical Types?
https://iceberg.apache.org/spec/#parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
