Hi David and Xiangyu, For more context,
We have a job running on our cluster aggregating StatsD metrics in tumbling window, which causes CPU usage spikes due to concurrent executions. While staggering the windows using StaggerWindow could help this will impact the job's accuracy. Instead, we want to investigate that by offsetting the trigger times without touching window start and end. we could potentially distribute the aligned windows fired executions over time to alleviate resource contention, without compromising the correctness of the computations. > On Aug 3, 2023, at 10:01 PM, David Anderson <dander...@apache.org> wrote: > > There's already a built-in concept of WindowStagger that provides an > interface for accomplishing this. > > It's not as well integrated (or documented) as it might be, but the > basic mechanism exists. To use it, I believe you would do something > like this: > > assigner = new TumblingEventTimeWindows(Time.seconds(5), 0, > WindowStagger.RANDOM); > > foo.keyBy(...) > .window(assigner) > ... > > The different stagger strategies are documented in [1]. > > [1] > https://nightlies.apache.org/flink/flink-docs-stable/api/java/index.html?org/apache/flink/streaming/api/windowing/assigners/WindowStagger.html > > On Wed, Aug 2, 2023 at 7:13 PM xiangyu feng <xiangyu...@gmail.com> wrote: >> >> Hi Tucker, >> >> Can you describe more about your running job and how the trigger timer is >> configured? Also it would be better if you can attach a FlameGraph to show >> the CPU usage when the timer is triggered. >> >> Best, >> Xiangyu >> >> Tucker Harvey via user <user@flink.apache.org> 于2023年8月1日周二 05:51写道： >>> >>> Hello Flink community! My team is trying to optimize CPU usage on a running >>> job, and we're exploring the option of offsetting the trigger time for >>> smoother CPU patterns. Since adjusting the window will compromise job >>> correctness, we plan to pursue a custom trigger implementation. We were >>> curious if the community had any thoughts or insights on this issue. >>> >>>
