[
https://issues.apache.org/jira/browse/BEAM-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877000#comment-15877000
]
Kenneth Knowles commented on BEAM-1524:
---------------------------------------
I agree with this. I think milliseconds are the best choice for minimum
resolution, and the min/max times are chosen so a runner can still use
microseconds without overflow.
The most critical function of timestamps as far as runners being able to
understand them is to drive timers (including runner-internal timers like
window GC). It has nothing to do with user data, actually, so it doesn't make
sense to have false precision. Users can already window by arbitrary precision
using a custom window class and they just need to choose a safe millisecond for
the watermark to drive GC.
The encoding found in the Java SDK does have the lexicographic property you
mention, and should be used as the reference. Is there a known issue there?
> Timestamp precision and byte representation should be consistent across SDKs
> ----------------------------------------------------------------------------
>
> Key: BEAM-1524
> URL: https://issues.apache.org/jira/browse/BEAM-1524
> Project: Beam
> Issue Type: Improvement
> Components: runner-core, sdk-java-core, sdk-py
> Reporter: Vikas Kedigehalli
> Assignee: Kenneth Knowles
>
> Runners cannot have a timestamp precision that is lesser than what a SDK
> uses. Also with Fn API a runner should be able to support multiple SDKs that
> maybe written in different languages. It is important that we standardize the
> precision of timestamp to ensure that runners are compatible across all the
> SDKs.
> NOTE: Timestamps are shifted such that lexicographic ordering of the bytes
> corresponds to chronological order. This also needs to be consistent across
> SDKs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
