shunping commented on issue #39026: URL: https://github.com/apache/beam/issues/39026#issuecomment-5336742858
> Thanks for reporting this issue. Would it be possible to narrow down the version that introduced this regression. 2.66.0 to 2.74.0 still has a few intermediate versions > > ~Edit: checked history that last Python periodic impulse change happened in [#35569](https://github.com/apache/beam/pull/35569), for Beam 2.67.0. If Beam 2.67.0 already shows this watermark change, it's likely the cause~ > > Edit: actually due to [#35412](https://github.com/apache/beam/pull/35412) (in Beam 2.67.0). I'm able to reproduce the watermark pattern in Beam 2.67.0 > > Note that the orange line is the step of PeriodicImpulse output. Plus, the "overall" watermark is the minimum timestamp of all. It's more a visual effect. The downstream (BigQuery write, etc) watermark (can be shown in other lines in your screenshot) isn't affected, which would control when window gets fired. There fore this is more a visual effect and not affecting semantics of downstream main input branch. Thanks for fixing this! I agree with @Abacn that this is primarily a visual effect of how the freshness is calculated and shown in the chart. It seems that setting the watermark estimator to a future timestamp when the defer path is reached results in a flat freshness chart (i.e., freshness is zero because the estimator is ahead of the current timestamp). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
