Hi Aljoscha, as it turns out the "workaround" I was thinking was functionally working, but had a so to say memory leak. I was under the impression that evicted elements will be removed from the window state...
Anyway, I think that this (triggers not being evaluated when the window state is null) turns out to be blocker for us. Why is this check done? Since a user can do basically whatever she likes in onProcessingTimeTimer() the comment // if we have no state, there is nothing to do is, well, just not true in some cases (e.g. state updates in our case). Cheers, Konstantin On 09.11.2016 14:17, Aljoscha Krettek wrote: > Could you go into some detail of why you need to keep the trigger state? > > Just the basics because you probably cannot (should not) talk about your > internal stuff. > > On Wed, 9 Nov 2016 at 13:16 Konstantin Knauf > <konstantin.kn...@tngtech.com <mailto:konstantin.kn...@tngtech.com>> wrote: > > Sounds good Aljoscha. > > sent from my phone. Plz excuse brevity and tpyos. > --- > Konstantin Knauf *konstantin.kn...@tngtech.com > <mailto:konstantin.kn...@tngtech.com> * +49-174-3413182 > <tel:0174%203413182> > > TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke > > ---- Aljoscha Krettek schrieb ---- > > > Hi, > exactly for this case I want to make a change to when > Trigger.clear() is > called: https://issues.apache.org/jira/browse/FLINK-4994 > > Right now, clear is called when the window is being garbage > collected because we passed the allowed lateness (after this, > nothing will ever be added to a window again) and also when the > Trigger returns PURGE or FIRE_AND_PURGE. > > I want to change it to only be called in the former case. We could > possibly add an onPurge() callback to allow cleaning state on purge > or require people to put the code that they want to run on PURGE in > the Trigger method that returns the PURGE. > > What do you think? > > Cheers, > Aljoscha > > On Tue, 8 Nov 2016 at 18:46 Konstantin Knauf > <konstantin.kn...@tngtech.com <mailto:konstantin.kn...@tngtech.com>> > wrote: > > Hi Aljoscha, > > interesting, this explains it. Well, in our case the PURGE in the > onProcessingTimeTimer is only used to clear KeyValueStates*, and > at this > point there are usually no records in the window state. > > Any Ideas? > > I do have a workaround with an evictor, but it seemed to be > unnecessarily complicated. > > *We can not use clear()-callback for that, since this state should > survive the FIRE_AND_PURGEs in the onElement()-calls. > > Cheers, > > Konstantin > > > On 08.11.2016 18:31, Aljoscha Krettek wrote: > > Hi, > > the timers are not actually deleted but the WindowOperator > will check > > whether there is any window state associated with the window > for which > > the timer fires. If there is no window state the timer will > silently be > > ignored. > > > > Is this a problem for you or did you just want to clarify? If > yes, then > > we should work on finding a solution. > > > > Cheers, > > Aljoscha > > > > On Tue, 8 Nov 2016 at 18:18 Konstantin Knauf > > <konstantin.kn...@tngtech.com > <mailto:konstantin.kn...@tngtech.com> > <mailto:konstantin.kn...@tngtech.com > <mailto:konstantin.kn...@tngtech.com>>> wrote: > > > > Hi everyone, > > > > I just migrated a streaming Job from 1.0.2 to 1.1.3 and > stumbled across > > a problem concerning one of our custom triggers. > > > > The trigger basically FIRE_AND_PURGEs multiple times in > onElement() and > > the window is PURGEd onProcessingTimeTimer(), but it seems > that the all > > registered processing time timers are deleted everytime > the window is > > PURGEd. > > > > clear() is the default implementation, i.e. no-op. > > > > Just wanted to, if this is the expected behavior > (processing time timers > > being deleted on PURGE or FIRE_AND_PURGE) from Flink 1.1 on? > > > > Cheers, > > > > Konstantin > > > > -- > > Konstantin Knauf * konstantin.kn...@tngtech.com > <mailto:konstantin.kn...@tngtech.com> > > <mailto:konstantin.kn...@tngtech.com > <mailto:konstantin.kn...@tngtech.com>> * +49-174-3413182 > <tel:0174%203413182> > > <tel:0174%203413182> > > TNG Technology Consulting GmbH, Betastr. 13a, 85774 > Unterföhring > > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. > Robert Dahlke > > Sitz: Unterföhring * Amtsgericht München * HRB 135082 > > > > > > -- > Konstantin Knauf * konstantin.kn...@tngtech.com > <mailto:konstantin.kn...@tngtech.com> * +49-174-3413182 > <tel:0174%203413182> > TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke > Sitz: Unterföhring * Amtsgericht München * HRB 135082 > -- Konstantin Knauf * konstantin.kn...@tngtech.com * +49-174-3413182 TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke Sitz: Unterföhring * Amtsgericht München * HRB 135082
signature.asc
Description: OpenPGP digital signature
