Re: ProcessingTimeTimer in ProcessFunction after a savepoint

2017-03-21 Thread Aljoscha Krettek
Thanks, please do keep me posted! > On 20 Mar 2017, at 21:50, Florian König wrote: > > @Aljoscha Thank you for the pointer to ProcessFunction. That looks like a > better approach with less code and other overhead. > > After restoring, the job is both reading new elements and emitting some, but

Re: ProcessingTimeTimer in ProcessFunction after a savepoint

2017-03-20 Thread Florian König
@Aljoscha Thank you for the pointer to ProcessFunction. That looks like a better approach with less code and other overhead. After restoring, the job is both reading new elements and emitting some, but nowhere near as many as expected. I’ll investigate further after switching to ProcessFunction

Re: ProcessingTimeTimer in ProcessFunction after a savepoint

2017-03-20 Thread Aljoscha Krettek
As a general remark, I think the ProcessFunction (https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/process_function.html) could be better suited for implementing such a use case. I did run tests on Flink 1.2 and master with a simple processing-time windowing job. After per

Re: ProcessingTimeTimer in ProcessFunction after a savepoint

2017-03-19 Thread Florian König
@Aljoscha: We’re using 1.2. The intention of our code is as follows: The events that flow through Flink represent scheduling decisions, i.e. they contain the ID of a target entity, a description of an action that should be performed on that entity by some other job, and a timestamp of when that

Re: ProcessingTimeTimer in ProcessFunction after a savepoint

2017-03-17 Thread Aljoscha Krettek
When restoring, processing-time timers that would have fired already should immediately fire. @Florian what Flink version are you using? In Flink 1.1 there was a bug that led to processing-time timers not being reset when restoring. > On 17 Mar 2017, at 15:39, Florian König wrote: > > Hi, > >

Re: ProcessingTimeTimer in ProcessFunction after a savepoint

2017-03-17 Thread Florian König
Hi, funny coincidence, I was just about to ask the same thing. I have noticed this with restored checkpoints in one of my jobs. The timers seem to be gone. My window trigger registers a processing timer, but it seems that these don’t get restored - even if the timer is set to fire in the future

ProcessingTimeTimer in ProcessFunction after a savepoint

2017-03-17 Thread Yassine MARZOUGUI
Hi all, How does the processing time timer behave when a job is taken down with a savepoint and then restarted after the timer was supposed to fire? Will the timer fire at restart because it was missed during the savepoint? I'm wondering because I would like to schedule periodic timers in the fut