carloea2 commented on code in PR #8704:
URL: https://github.com/apache/texera/pull/8704#discussion_r4368647365
##########
amber/src/main/python/core/models/schema/attribute_type.py:
##########
@@ -98,21 +102,57 @@ def _parse_bool(v):
def _parse_timestamp(v):
+ # A TIMESTAMP reaches a worker as a wall clock with no zone, so a value
+ # parsed here holds none either and compares with a row's. An offset is
+ # moved to the machine's zone first, as the engine's DateParserUtils does.
if _is_empty_value(v):
- return datetime.datetime(1970, 1, 1, tzinfo=datetime.timezone.utc)
+ return datetime.datetime(1970, 1, 1)
normalized_value = str(v)
if normalized_value.endswith("Z"):
normalized_value = normalized_value[:-1] + "+00:00"
parsed_value = datetime.datetime.fromisoformat(normalized_value)
- if (
- parsed_value.tzinfo is None
- or parsed_value.tzinfo.utcoffset(parsed_value) is None
- ):
- return parsed_value.replace(tzinfo=datetime.timezone.utc)
+ if parsed_value.utcoffset() is not None:
+ return _local_wall_clock(parsed_value)
return parsed_value
+def _local_wall_clock(moment):
+ # The wall clock the machine's zone shows at an instant that states its
+ # offset. Windows' localtime refuses an instant before 1970, so the zone is
+ # asked by name where it has one, and otherwise such an instant takes the
+ # machine's standard offset.
+ zone = _named_local_zone()
+ if zone is not None:
+ return moment.astimezone(zone).replace(tzinfo=None)
Review Comment:
Named zones still disagree with the engine before 1900. For
1600-01-01T12:00:00Z, TZ=America/Mexico_City gives 05:23:24 here versus
06:00:00 through DateParserUtils and Timestamp on JDK 17. Asia/Kolkata gives
17:53:28 versus 17:30:00. The 1960 and 2024 controls agree. ZoneInfo uses the
old local-mean-time offset, while Java uses the standard offset before 1900.
Please keep this consistent with the engine and the existing standalone
timestamp helper.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
