Re: Custom stream processor not triggering #punctuate()

2017-03-30 Thread Matthias J. Sax
We plan to do a KIP for this. Should come up soon. Please follow dev list for details and participate in the discussion! -Matthias On 3/30/17 11:02 AM, Thomas Becker wrote: > Does this fix the problem though? The docs indicate that new data is > required for each *partition*, not topic.

Re: Custom stream processor not triggering #punctuate()

2017-03-30 Thread Thomas Becker
Does this fix the problem though? The docs indicate that new data is required for each *partition*, not topic. Overall I think the "stream time" notion is a good thing for a lot of use-cases, but some others definitely require wall-clock based windowing. Is something planned for this? -Tommy

Re: Custom stream processor not triggering #punctuate()

2017-03-28 Thread Elliot Crosby-McCullough
Hi Michael, My confusion was that the events are being created, transferred, and received several seconds apart (longer than the punctuate schedule) with no stalling because I'm triggering them by hand, so regardless of what mechanism is being used for timing it should still be called. That

Re: Custom stream processor not triggering #punctuate()

2017-03-28 Thread Michael Noll
Elliot, in the current API, `punctuate()` is called based on the current stream-time (which defaults to event-time), not based on the current wall-clock time / processing-time. See http://docs.confluent.io/ current/streams/faq.html#why-is-punctuate-not-called. The stream-time is advanced only

Custom stream processor not triggering #punctuate()

2017-03-28 Thread Elliot Crosby-McCullough
Hi there, I've written a simple processor which expects to have #process called on it for each message and configures regular punctuate calls via `context.schedule`. Regardless of what configuration I try for timestamp extraction I cannot get #punctuate to be called, despite #process being