Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-09-27 Thread John Roesler
Hey Jonathan, What's the status of this KIP? I was just in a discussion about suppress, and it sparked my memory of this idea. Thanks, -John On Mon, Mar 11, 2019 at 4:39 PM John Roesler wrote: > > Thanks for the response, Matthias, I agree on both of these points. > > I didn't mean to question

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-05-02 Thread jonathangordon
I'd like to move this KIP along. To summarize where we've landed so far: - Should suppress() flush all records (would be _before_ the time elapsed)? - Or should it preserve buffered records and reload on restart? For this case, should the record be flushed on reload (elapsed time is unknown) or

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-04-07 Thread jonathangordon
On 2019/03/11 21:39:18, John Roesler wrote: > Regarding flushing on shutdown, can you elaborate of the motivation for > doing so? I'm not in any way attached to this idea if you think it's not any additional trouble. It wasn't obvious to me how you would store the "wall clock time last

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-03-11 Thread John Roesler
Thanks for the response, Matthias, I agree on both of these points. I didn't mean to question whether we should discuss it; we should, since as you point out both points affect the behavior of the API. Regarding checking system time, After reviewing the motivation of the KIP, it seems like a

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-03-11 Thread Matthias J. Sax
I agree that there are multiple ways how to avoid calling `System.currentTimeMillis()`. However, the KIP needs to define the public contract to explain users what behavior they can expect (the simplest thing might be to say, it's based on `punctuation()` schedule -- not sure if this is desired or

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-03-11 Thread John Roesler
Hey, all, just chiming in to keep the discussion moving... Regarding whether to flush or not on shutdown, I'm curious why we *would* flush... The record cache does this, but that's because it's not durable. The suppression buffer is already backed by a changelog specifically so that it can

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-02-25 Thread jonathangordon
On 2019/02/21 02:19:27, "Matthias J. Sax" wrote: > thanks for the KIP. Corner case question: > > What happens if an application is stopped an restarted? > > - Should suppress() flush all records (would be _before_ the time elapsed)? > - Or should it preserve buffered records and reload on

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-02-20 Thread Matthias J. Sax
Jonathan, thanks for the KIP. Corner case question: What happens if an application is stopped an restarted? - Should suppress() flush all records (would be _before_ the time elapsed)? - Or should it preserve buffered records and reload on restart? For this case, should the record be flushed

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-01-30 Thread John Roesler
Hi Jonathan, Thanks for the KIP! I think all the reviewers are heads-down right now reviewing code for the imminent 2.2 release, so this discussion may not get much traffic over the next couple of weeks. You might want to just keep bumping it once a week or so until people start finding time to

[DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

2019-01-29 Thread jonathangordon
Hi all, I just published KIP-424: Allow suppression of intermediate events based on wall clock time https://cwiki.apache.org/confluence/display/KAFKA/KIP-424%3A+Allow+suppression+of+intermediate+events+based+on+wall+clock+time I am eager to hear your feedback and concerns. Thanks John Roesler