Hi Bill, Thanks for the input! TBH, I am think that suppression buffers are not used *in response *to low traffic conditions. Rather, we are trying to fix the situation when low traffic conditions occur in a suppression buffer (for example, previously, the same suppression buffer had a decent volume of records entering it, thus advancing the stream time). In summary, when these conditions do hit, we want to advance the stream time somehow to resolve this issue. Reflecting on this though, I am not completely certain if this really stops us from implementing per key stream time tracking because the problem wouldn't be made *that *much worse.
Cheers, Richard On Fri, Oct 18, 2019 at 11:46 AM Bill Bejeck <bbej...@gmail.com> wrote: > Hi Richard, > > Thanks for the KIP proposal. I understand the situation you are > describing. > But in my mind, if there is a low traffic condition and you need to keep > records going downstream at regular intervals, I'm wondering if using > suppression is the correct approach. > IMHO it seems it would be better to use the PAPI or a Transform on the DSL > with a scheduled punctuation call. > > Just my 2 cents. > > Thanks, > Bill > > On Thu, Oct 17, 2019 at 7:42 PM Richard Yu <yohan.richard...@gmail.com> > wrote: > > > Hi all, > > > > I wish to discuss this KIP which would help us in resolving some issues > we > > have with suppression buffers. > > Below is the link: > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-539%3A+Implement+mechanism+to+flush+out+records+in+low+volume+suppression+buffers > > > > @John Roesler if you have time, would be great if we could get your > input. > > > > Cheers, > > Richard > > >
