Re: Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

[Reuven Lax]

I can review it.

On Tue, Aug 14, 2018 at 10:26 AM Huygaa Batsaikhan 
wrote:

> Finally, I have a PR  for the
> annotation itself. Anyone up for reviewing it? Ken has been helping me, but
> he is going to be OOO for a while.
>
> On Tue, Mar 20, 2018 at 4:23 PM Huygaa Batsaikhan 
> wrote:
>
>> As echauchot@ mentioned, it will make it easier and error-free.
>>
>>
>> On Mon, Mar 19, 2018 at 11:59 PM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> Hi Huygaa,
>>>
>>> Cant it be predefined timers?
>>>
>>> Romain
>>>
>>> Le 20 mars 2018 00:52, "Huygaa Batsaikhan"  a écrit :
>>>
>>> Hi everyone, I am working on BEAM-1589
>>> . In short, currently,
>>> there is no default way of saving/flushing state before a window is garbage
>>> collected.
>>>
>>> My current plan is to provide a method annotation, @OnWindowExpiration,
>>> which allows user-provided callback function to be executed before garbage
>>> collection. This annotation behaves very similar to @OnTimer, therefore,
>>> implementation will mostly be a copy of OnTimer code. Let me know if you
>>> have any considerations and suggestions.
>>>
>>> Here is an example usage:
>>> ```
>>> @OnWindowExpiration
>>> public void myCleanupFunction(OnWindowExpirationContext c, State state) {
>>>   c.output(state.read());
>>> }
>>> ```
>>>
>>> Thanks, Huygaa
>>>
>>>
>>>

Re: Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

[Huygaa Batsaikhan]

Finally, I have a PR  for the
annotation itself. Anyone up for reviewing it? Ken has been helping me, but
he is going to be OOO for a while.

On Tue, Mar 20, 2018 at 4:23 PM Huygaa Batsaikhan  wrote:

> As echauchot@ mentioned, it will make it easier and error-free.
>
>
> On Mon, Mar 19, 2018 at 11:59 PM Romain Manni-Bucau 
> wrote:
>
>> Hi Huygaa,
>>
>> Cant it be predefined timers?
>>
>> Romain
>>
>> Le 20 mars 2018 00:52, "Huygaa Batsaikhan"  a écrit :
>>
>> Hi everyone, I am working on BEAM-1589
>> . In short, currently,
>> there is no default way of saving/flushing state before a window is garbage
>> collected.
>>
>> My current plan is to provide a method annotation, @OnWindowExpiration,
>> which allows user-provided callback function to be executed before garbage
>> collection. This annotation behaves very similar to @OnTimer, therefore,
>> implementation will mostly be a copy of OnTimer code. Let me know if you
>> have any considerations and suggestions.
>>
>> Here is an example usage:
>> ```
>> @OnWindowExpiration
>> public void myCleanupFunction(OnWindowExpirationContext c, State state) {
>>   c.output(state.read());
>> }
>> ```
>>
>> Thanks, Huygaa
>>
>>
>>

Re: Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

[Huygaa Batsaikhan]

As echauchot@ mentioned, it will make it easier and error-free.


On Mon, Mar 19, 2018 at 11:59 PM Romain Manni-Bucau 
wrote:

> Hi Huygaa,
>
> Cant it be predefined timers?
>
> Romain
>
> Le 20 mars 2018 00:52, "Huygaa Batsaikhan"  a écrit :
>
> Hi everyone, I am working on BEAM-1589
> . In short, currently,
> there is no default way of saving/flushing state before a window is garbage
> collected.
>
> My current plan is to provide a method annotation, @OnWindowExpiration,
> which allows user-provided callback function to be executed before garbage
> collection. This annotation behaves very similar to @OnTimer, therefore,
> implementation will mostly be a copy of OnTimer code. Let me know if you
> have any considerations and suggestions.
>
> Here is an example usage:
> ```
> @OnWindowExpiration
> public void myCleanupFunction(OnWindowExpirationContext c, State state) {
>   c.output(state.read());
> }
> ```
>
> Thanks, Huygaa
>
>
>

Re: Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

[Jean-Baptiste Onofré]

+1

It sounds good to me.

Regards
JB

Le 20 mars 2018 à 00:52, à 00:52, Huygaa Batsaikhan  a écrit:
>Hi everyone, I am working on BEAM-1589
>. In short, currently,
>there is no default way of saving/flushing state before a window is
>garbage
>collected.
>
>My current plan is to provide a method annotation, @OnWindowExpiration,
>which allows user-provided callback function to be executed before
>garbage
>collection. This annotation behaves very similar to @OnTimer,
>therefore,
>implementation will mostly be a copy of OnTimer code. Let me know if
>you
>have any considerations and suggestions.
>
>Here is an example usage:
>```
>@OnWindowExpiration
>public void myCleanupFunction(OnWindowExpirationContext c, State state)
>{
>  c.output(state.read());
>}
>```
>
>Thanks, Huygaa

Re: Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

[Etienne Chauchot]

Hi,
When coding GroupIntoBatches transform, I had similar need. I implemented it 
with an Ontimer callback set that way:
timer.set(window.maxTimestamp().plus(allowedLateness));
At the time Kenn suggested to set up a new @OnWindowExpiration annotation to do 
exactly that but more easily.
So big +1.
Etienne
Le lundi 19 mars 2018 à 23:51 +, Huygaa Batsaikhan a écrit :
> Hi everyone, I am working on BEAM-1589. In short, currently, there is no 
> default way of saving/flushing state before a
> window is garbage collected.
> 
> My current plan is to provide a method annotation, @OnWindowExpiration, which 
> allows user-provided callback function
> to be executed before garbage collection. This annotation behaves very 
> similar to @OnTimer, therefore, implementation
> will mostly be a copy of OnTimer code. Let me know if you have any 
> considerations and suggestions.
> 
> Here is an example usage:
> ```
> @OnWindowExpiration
> public void myCleanupFunction(OnWindowExpirationContext c, State state) {
>   c.output(state.read());
> }
> ```
> 
> Thanks, Huygaa

Re: Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

[Romain Manni-Bucau]

Hi Huygaa,

Cant it be predefined timers?

Romain

Le 20 mars 2018 00:52, "Huygaa Batsaikhan"  a écrit :

Hi everyone, I am working on BEAM-1589
. In short, currently,
there is no default way of saving/flushing state before a window is garbage
collected.

My current plan is to provide a method annotation, @OnWindowExpiration,
which allows user-provided callback function to be executed before garbage
collection. This annotation behaves very similar to @OnTimer, therefore,
implementation will mostly be a copy of OnTimer code. Let me know if you
have any considerations and suggestions.

Here is an example usage:
```
@OnWindowExpiration
public void myCleanupFunction(OnWindowExpirationContext c, State state) {
  c.output(state.read());
}
```

Thanks, Huygaa