Re: Order of punctuate() and process() in a stream processor

2017-05-18 Thread Matthias J. Sax
Borowiecki > To: users@kafka.apache.org > Date: 2017/05/17 10:15 > Subject: Re: Order of punctuate() and process() in a stream > processor > > > > Hi Sini, > > This is beyond the score of KIP-138 but > https://issues.apache.org/jira/browse/KAFKA-3

Re: Order of punctuate() and process() in a stream processor

2017-05-18 Thread Peter Sinoros Szabo
: users@kafka.apache.org Date: 2017/05/17 10:15 Subject:Re: Order of punctuate() and process() in a stream processor Hi Sini, This is beyond the score of KIP-138 but https://issues.apache.org/jira/browse/KAFKA-3514 exists to track such improvements Thanks, Michal On 17 May 2017 5:10

Re: Order of punctuate() and process() in a stream processor

2017-05-17 Thread Michal Borowiecki
tream times for myself. It's ok for me to trigger it 3 times, but the > timestamp should not be the same in each, but should be increased by the > schedule time in each punctuate. > > - Sini > > > > From: Thomas Becker > To: "users@kafka.ap

Re: Order of punctuate() and process() in a stream processor

2017-05-17 Thread Peter Sinoros Szabo
quot;Matthias J. Sax" To: users@kafka.apache.org Date: 2017/05/12 19:19 Subject: Re: Order of punctuate() and process() in a stream processor Thanks for sharing. As punctuate is called with "streams time" you see the same time value multiple times. It's again due

Re: Order of punctuate() and process() in a stream processor

2017-05-14 Thread Mahendra Kariya
ely expensive it's especially bad. > > > > > > From: Matthias J. Sax [matth...@confluent.io] > > Sent: Friday, May 12, 2017 1:18 PM > > To: users@kafka.apache.org > > Subject: Re: Order of punctuate() and process() in a stream processo

Re: Order of punctuate() and process() in a stream processor

2017-05-12 Thread Matthias J. Sax
; >> >> I'm a bit troubled by the fact that it fires 3 times despite the stream >> time being advanced all at once; is there a scenario when this is >> beneficial? >> >> >> From: Matthias J. Sax [matth...@confluen

RE: Order of punctuate() and process() in a stream processor

2017-05-12 Thread Thomas Becker
y bad. From: Matthias J. Sax [matth...@confluent.io] Sent: Friday, May 12, 2017 1:18 PM To: users@kafka.apache.org Subject: Re: Order of punctuate() and process() in a stream processor Thanks for sharing. As punctuate is called with "streams time" you see the same time value

Re: Order of punctuate() and process() in a stream processor

2017-05-12 Thread Matthias J. Sax
e > timestamp should not be the same in each, but should be increased by the > schedule time in each punctuate. > > - Sini > > > > From: Thomas Becker > To: "users@kafka.apache.org" > Date: 2017/05/12 18:57 > Subject:RE: Order of

RE: Order of punctuate() and process() in a stream processor

2017-05-12 Thread Peter Sinoros Szabo
t the timestamp should not be the same in each, but should be increased by the schedule time in each punctuate. - Sini From: Thomas Becker To: "users@kafka.apache.org" Date: 2017/05/12 18:57 Subject: RE: Order of punctuate() and process() in a stream processor I&#

RE: Order of punctuate() and process() in a stream processor

2017-05-12 Thread Thomas Becker
users@kafka.apache.org Subject: Re: Order of punctuate() and process() in a stream processor Hi Peter, It's by design. Streams internally tracks time progress (so-called "streams time"). "streams time" get advanced *after* processing a record. Thus, in your case, "stream time&q

Re: Order of punctuate() and process() in a stream processor

2017-05-12 Thread Peter Sinoros Szabo
uot;Matthias J. Sax" To: users@kafka.apache.org Date: 2017/05/12 18:39 Subject: Re: Order of punctuate() and process() in a stream processor Hi Peter, It's by design. Streams internally tracks time progress (so-called "streams time"). "streams time" get advanced *

Re: Order of punctuate() and process() in a stream processor

2017-05-12 Thread Matthias J. Sax
Hi Peter, It's by design. Streams internally tracks time progress (so-called "streams time"). "streams time" get advanced *after* processing a record. Thus, in your case, "stream time" is still at its old value before it processed the first message of you send "burst". After that, "streams time"

Order of punctuate() and process() in a stream processor

2017-05-12 Thread Peter Sinoros Szabo
Hi, Let's assume the following case. - a stream processor that uses the Processor API - context.schedule(1000) is called in the init() - the processor reads only one topic that has one partition - using custom timestamp extractor, but that timestamp is just a wall clock time Image the followin