Lee-W commented on code in PR #64571:
URL: https://github.com/apache/airflow/pull/64571#discussion_r3309872307
##########
airflow-core/src/airflow/partition_mappers/temporal.py:
##########
@@ -62,6 +180,29 @@ def format(self, dt: datetime) -> str:
"""Format the normalized datetime."""
return dt.strftime(self.output_format)
+ def decode_downstream(self, downstream_key: str) -> datetime:
+ """
+ Recover the period-start datetime from a previously formatted
downstream key.
+
+ Inverse of ``format``. The default implementation uses ``strptime``
with
+ ``output_format``, which works for any format made of standard strptime
+ directives. Subclasses with custom format markers (e.g. ``{quarter}``)
or
+ ambiguous directives (e.g. bare ``%V``) override this.
+ """
+ return datetime.strptime(downstream_key, self.output_format)
+
+ def encode_upstream(self, dt: datetime) -> str:
+ """
+ Format *dt* as an upstream partition key string.
+
+ Pair of :meth:`decode_downstream`: takes a (decoded) period-start
+ datetime and produces a key string in the upstream's ``input_format``
+ with ``timezone`` applied. Used by :class:`RollupMapper` to render each
+ upstream member yielded by the window back into the form upstream
+ producers actually emit.
+ """
+ return make_aware(dt, self._timezone).strftime(self.input_format)
Review Comment:
Strips tzinfo before calling `make_aware` so aware decoded datetimes
round-trip. Round 1 used `dt.replace(tzinfo=None)` directly, which silently
shifted non-UTC offsets (IST midnight became UTC midnight); the landed version
uses `dt.astimezone(self._timezone).replace(tzinfo=None)` so the resulting key
reflects the same instant. Two regression tests cover the `+0000` round-trip
and the `+0530` non-UTC offset case, and the docstring on `encode_upstream` is
updated to explain the conversion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
