kz930 commented on code in PR #8506:
URL: https://github.com/apache/texera/pull/8506#discussion_r4364587507
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/intervalJoin/IntervalJoinOpDesc.scala:
##########
@@ -168,3 +224,47 @@ class IntervalJoinOpDesc extends LogicalOp {
}
}
+
+object IntervalJoinOpDesc {
+
+ /** The upper bound of each interval, as the executor computes it:
+ * `Timestamp.valueOf(t.toLocalDateTime.plusX(constant))`. The constant is
+ * added on the machine's wall clock, so a day across a daylight-saving
+ * change is 23 or 25 hours, and a month counts from the local date. A wall
+ * clock that falls in a gap or an overlap is read at the standard offset,
+ * as java.util.Date reads it, and one before 1900 at the standard offset
+ * today. Each distinct key is worked out once, since the column holds every
+ * pair of the cross join.
+ */
+ val IntervalUpper: String =
+ """def _texera_interval_upper(s, offset):
+ | from dateutil.tz import tzlocal
+ |
+ | zone = _texera_local_zone()
+ | standard = _texera_standard_offset(zone)
+ | moved = _texera_local_wall_clocks(s.dt.tz_localize("UTC")) + offset
Review Comment:
Right, the timestamps already arrive as local wall clocks, so the export now
adds the interval to that wall clock and reads the keys and the bound back into
instants as Timestamp.valueOf does (b9620beb7). The spec feeds both sides the
same naive wall clocks in America/Chicago, including your 03-09 and November
cases plus a gap, an overlap, a month and a pre-1900 case, and it passes
against IntervalJoinOpExec with #8597 supplied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
