shunping opened a new pull request, #35569: URL: https://github.com/apache/beam/pull/35569
When we call `PeriodicImpulse`, either using the default `start_timestamp`, or specifying it to be `Timestamp.now()`. The timestamp that gets passed in is the timestamp when the pipeline is constructed (i.e. when the PTransform is expanded). The time when the pipeline is actually run (i.e. when the PTransform is executed) is a later time. In `ImpulseSeqGenDoFn`, the function implements a catch-up logic to align generated timestamp with the runtime `now`. As a result, we see events are clustered at the start of the pipeline execution. This is especially true for runners (e.g. Dataflow Runner) which needs a decent amount of time to start the pipeline. Here, we add a new flag in `PerodicImpulse` to rebase timestamp. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
