Zhenqiu Huang created FLINK-37900:
-------------------------------------
Summary: Support configurable jitter in rocksdb checkpoint file
uploader
Key: FLINK-37900
URL: https://issues.apache.org/jira/browse/FLINK-37900
Project: Flink
Issue Type: Improvement
Components: Runtime / State Backends
Affects Versions: 2.1.0
Reporter: Zhenqiu Huang
For large Flink jobs (with 512 + containers), it will create lots concurrent
file uploader workload to remote storage. We see some cases that write request
are rate limited by remote object store. We may add a jitter configuration to
smooth the requests into a small periods for example 10 seconds.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
