Sorry, not looked into it much... but it occurred to me that it would be great
to have it as "Throttling: Operator" that can be applied anywhere after a
source and before a sink. Each parallel instance of it can operate at the
specified rate limit divided by the number of parallel instances of this
operator.
On Tuesday, February 26, 2019, 7:39:06 PM PST, Thomas Weise
<t...@apache.org> wrote:
Since we want to be able to use the rate limiter for multiple connectors
and potentially elsewhere, plus it is user facing, this seems to be the
right area. Perhaps org.apache.flink.api.common.io specifically?
On Mon, Feb 25, 2019 at 5:40 PM Lakshmi Gururaja Rao <l...@lyft.com.invalid>
wrote:
> Hi all,
>
> The PR that I've been iterating on w.r.t. Kafka ratelimiting —
> https://github.com/apache/flink/pull/7679 is now in a state where a lot of
> the rate limiting logic is generic and does not apply specifically to
> Kafka. I'd thus, like to move it to a module that is outside of the*
> flink-connector-kafka-0.9/. *Looking at some of the other common modules,
> it seems like org.apache.flink.api.java
> <
> https://github.com/lyft/flink/tree/master/flink-core/src/main/java/org/apache/flink/api/java
> >
> maybe
> a good option? Thoughts or suggestions?
>
> Thanks,
> Lakshmi
>
