zzsmdfj created FLINK-13792:
-------------------------------
Summary: source and sink support manual rate limit
Key: FLINK-13792
URL: https://issues.apache.org/jira/browse/FLINK-13792
Project: Flink
Issue Type: Improvement
Components: Connectors / Common
Affects Versions: 1.8.1
Reporter: zzsmdfj
in current flink implement automatic flow control by back pressure, it is
efficient for the most scene, but in some special scenario, do we need
fine-grained flow control to avoid impact on other systems? For example: if i
have window with days(a lot of datas), then do call ProcessWindowFunction when
trigger, this will produce a lot of data to sink, if sink to message queue, it
can have a huge impact to message queue. so if there is sink rate limiter, it
is friendly to external system. for source rate limiter, it is appropriate for
having window operator and accumulating a large amount of historical data.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
