Hi Jamie, Thanks for the explanation. That makes sense to me. I am wondering if there is a more general way to add a rate limiter to all the connecters rather than doing that for each individual one. For example, maybe we can have the rate limiting logic in the Collector / Output, thus all the connectors (even operators?) could be rate limited.
Thanks, Jiangjie (Becket) Qin On Fri, Feb 1, 2019 at 7:23 AM Lakshmi Gururaja Rao <l...@lyft.com.invalid> wrote: > Thanks for adding more context @Jamie Grier <jgr...@lyft.com> . > > JIRA for this feature: https://issues.apache.org/jira/browse/FLINK-11501. > > Thanks > Lakshmi > > On Thu, Jan 31, 2019 at 3:20 PM Thomas Weise <t...@apache.org> wrote: > > > I think it would be reasonable to have a rate limiter option in the > > consumer, given that others have also looked to solve this. > > > > I think for this and other optional features, it would be good to > implement > > in a way that overrides are possible. Someone else may want to do the > > limiting differently, taking into account more/other factors. > > > > Both, adding the limiter and making the consumer code more adoptable > could > > be split into separate work also. > > > > BTW is there a JIRA for this? > > > > Thomas > > > > > -- > *Lakshmi Gururaja Rao* > SWE > 217.778.7218 <+12177787218> > [image: Lyft] <http://www.lyft.com/> >
