Re: [DISCUSS] Remove TimerInternals.deleteTimer(*) and Timer.cancel()

[JingsongLee]

+1 to remove this, I have not encountered such a strong case.
best, JingsongLee

--
From:Kenneth Knowles 
Time:2017 May 9 (Tue) 05:45
To:dev 
Subject:Re: [DISCUSS] Remove TimerInternals.deleteTimer(*) and Timer.cancel()
Interesting!

I believe the only thing we need to change to remove it for FSR is
https://github.com/apache/beam/blob/master/sdks/java/core
/src/main/java/org/apache/beam/sdk/state/Timer.java#L60

Here is how I might summarize the possibilities, at the risk of having
something quite wrong:

 - Cancel as-is: some runners must manage a tombstone and/or timestamp in
state or may choose to do so for performance.

 - Cancel requires a timestamp on the backend: runners must track the
timestamp in state in order to cancel. Some runners may already need to
track this for other reasons. For special cases like the end of the window
or GC time it can be guessed, but those aren't user timers.

 - Cancel requires a timestamp from the user: Really weird. IMO implies a
timer can be set multiple times and each one is independent. This is
actually an increase in capability in perhaps an interesting way, but I
thought it was too confusing. Also might have a wacky spec around the same
timer set multiple times for the same timestamp (probably fine/idempotent)

Technically, timers are marked `@Experimental`. But, given the interest in
state and timers, making changes here will be very hard on users.

Unless someone objects with a strong case, I am comfortable removing this
from userland and potentially adding it later if there is demand.

Kenn


On Mon, May 8, 2017 at 3:26 AM, Aljoscha Krettek 
wrote:

> I wanted to bring this up before the First Stable release and see what
> other people think. The methods I’m talking about are:
>
> void deleteTimer(StateNamespace namespace, String timerId, TimeDomain
> timeDomain);
>
> @Deprecated
> void deleteTimer(StateNamespace namespace, String timerId);
>
> @Deprecated
> void deleteTimer(TimerData timerKey);
>
> The last two are slated for deletion. Notice that the first one doesn’t
> take the timestamp of the timer to delete, which implies that there is only
> one active timer per namespace/timerId/timeDomain.
>
> These are my motivations for removal:
>  - Timer removal is difficult and has varying levels of support on
> different Runners and varying levels of cost.
>  - Removing the interface now and adding it back later is easy. Having it
> in the FSR and later removing it is quite hard.
>
> I can only speak about the internal Flink implementation where Timers are
> on a Heap (Java Priority Queue). Here removal is quite expensive, see, for
> example this ML thread: https://lists.apache.org/thread.html/
> ac4d8e36360779a9b5047cf21303222980015720aac478e8cf632216@%
> 3Cuser.flink.apache.org%3E. Especially this part:
>
> I thought I would drop my opinion here maybe it is relevant.
>
> We have used the Flink internal timer implementation in many of our
> production applications, this supports the Timer deletion but the deletion
> actually turned out to be a huge performance bottleneck because of the bad
> deletion performance of the Priority queue.
>
> In many of our cases deletion could have been avoided by some more clever
> registration/firing logic and we also ended up completely avoiding deletion
> and instead using "tombstone" markers by setting a flag in the state which
> timers not to fire when they actually trigger.
>
> Gyula
>
> Note that in Flink it’s not possible to delete a timer if you don’t know
> the timestamp. Other systems might store timers in more elaborate data
> structures (hierarchical timer wheels come to mind) where it’s also hard to
> delete a timer without knowing the timestamp.
>
> If we decide to keep timer removal in the user API there’s the possibility
> to simulate timer removal by keeping the timestamp of the currently active
> timer for a given timerID and checking a firing timer against that.
>
> Best,
> Aljoscha
>
>
>
>
>
>

Re: [DISCUSS] Remove TimerInternals.deleteTimer(*) and Timer.cancel()

[Kenneth Knowles]

Interesting!

I believe the only thing we need to change to remove it for FSR is
https://github.com/apache/beam/blob/master/sdks/java/core
/src/main/java/org/apache/beam/sdk/state/Timer.java#L60

Here is how I might summarize the possibilities, at the risk of having
something quite wrong:

 - Cancel as-is: some runners must manage a tombstone and/or timestamp in
state or may choose to do so for performance.

 - Cancel requires a timestamp on the backend: runners must track the
timestamp in state in order to cancel. Some runners may already need to
track this for other reasons. For special cases like the end of the window
or GC time it can be guessed, but those aren't user timers.

 - Cancel requires a timestamp from the user: Really weird. IMO implies a
timer can be set multiple times and each one is independent. This is
actually an increase in capability in perhaps an interesting way, but I
thought it was too confusing. Also might have a wacky spec around the same
timer set multiple times for the same timestamp (probably fine/idempotent)

Technically, timers are marked `@Experimental`. But, given the interest in
state and timers, making changes here will be very hard on users.

Unless someone objects with a strong case, I am comfortable removing this
from userland and potentially adding it later if there is demand.

Kenn


On Mon, May 8, 2017 at 3:26 AM, Aljoscha Krettek 
wrote:

> I wanted to bring this up before the First Stable release and see what
> other people think. The methods I’m talking about are:
>
> void deleteTimer(StateNamespace namespace, String timerId, TimeDomain
> timeDomain);
>
> @Deprecated
> void deleteTimer(StateNamespace namespace, String timerId);
>
> @Deprecated
> void deleteTimer(TimerData timerKey);
>
> The last two are slated for deletion. Notice that the first one doesn’t
> take the timestamp of the timer to delete, which implies that there is only
> one active timer per namespace/timerId/timeDomain.
>
> These are my motivations for removal:
>  - Timer removal is difficult and has varying levels of support on
> different Runners and varying levels of cost.
>  - Removing the interface now and adding it back later is easy. Having it
> in the FSR and later removing it is quite hard.
>
> I can only speak about the internal Flink implementation where Timers are
> on a Heap (Java Priority Queue). Here removal is quite expensive, see, for
> example this ML thread: https://lists.apache.org/thread.html/
> ac4d8e36360779a9b5047cf21303222980015720aac478e8cf632216@%
> 3Cuser.flink.apache.org%3E. Especially this part:
>
> I thought I would drop my opinion here maybe it is relevant.
>
> We have used the Flink internal timer implementation in many of our
> production applications, this supports the Timer deletion but the deletion
> actually turned out to be a huge performance bottleneck because of the bad
> deletion performance of the Priority queue.
>
> In many of our cases deletion could have been avoided by some more clever
> registration/firing logic and we also ended up completely avoiding deletion
> and instead using "tombstone" markers by setting a flag in the state which
> timers not to fire when they actually trigger.
>
> Gyula
>
> Note that in Flink it’s not possible to delete a timer if you don’t know
> the timestamp. Other systems might store timers in more elaborate data
> structures (hierarchical timer wheels come to mind) where it’s also hard to
> delete a timer without knowing the timestamp.
>
> If we decide to keep timer removal in the user API there’s the possibility
> to simulate timer removal by keeping the timestamp of the currently active
> timer for a given timerID and checking a firing timer against that.
>
> Best,
> Aljoscha
>
>
>
>
>
>

Re: Congratulations Davor!

[Davor Bonaci]

Thanks everyone!

On Mon, May 8, 2017 at 12:05 PM, Stas Levin  wrote:

> Congrats!
>
> On Mon, May 8, 2017, 20:07 Tyler Akidau 
> wrote:
>
> > Very nice. Congrats!
> >
> > On Sun, May 7, 2017 at 4:38 AM Prabeesh K.  wrote:
> >
> > > Congrats Davor.
> > >
> > > On 5 May 2017 at 15:52, Robert Metzger  wrote:
> > >
> > > > Congrats Davor!
> > > >
> > > > On Thu, May 4, 2017 at 9:59 PM, Mark Liu  >
> > > > wrote:
> > > >
> > > > > Congratulation Davor
> > > > >
> > > > > On Thu, May 4, 2017 at 11:09 AM, María García Herrero <
> > > > > mari...@google.com.invalid> wrote:
> > > > >
> > > > > > Wow! Congratulations, Davor!
> > > > > >
> > > > > > On Thu, May 4, 2017 at 10:54 AM, Ahmet Altay
> > >  > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Congratulations, well deserved!
> > > > > > >
> > > > > > > On Thu, May 4, 2017 at 10:35 AM, Andrew Psaltis <
> > > > > > psaltis.and...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Congrats Davor!
> > > > > > > >
> > > > > > > > On Thu, May 4, 2017 at 1:34 PM, Melissa Pashniak <
> > > > > > > > meliss...@google.com.invalid> wrote:
> > > > > > > >
> > > > > > > > > Congratulations Davor!
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Thu, May 4, 2017 at 10:32 AM, Robert Bradshaw <
> > > > > > > > > rober...@google.com.invalid> wrote:
> > > > > > > > >
> > > > > > > > > > Congratulations, Davor! Well deserved.
> > > > > > > > > >
> > > > > > > > > > On Thu, May 4, 2017 at 9:53 AM, Hadar Hod
> > > > > >  > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > Congrats, Davor!
> > > > > > > > > > >
> > > > > > > > > > > On Thu, May 4, 2017 at 8:56 AM, Chamikara Jayalath <
> > > > > > > > > chamik...@apache.org
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > >> Congrats Davor. Very well deserved.
> > > > > > > > > > >>
> > > > > > > > > > >> - Cham
> > > > > > > > > > >>
> > > > > > > > > > >> On Thu, May 4, 2017, 8:51 AM tarush grover <
> > > > > > > tarushappt...@gmail.com
> > > > > > > > >
> > > > > > > > > > >> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >> > Congrats Davor
> > > > > > > > > > >> >
> > > > > > > > > > >> > Regards,
> > > > > > > > > > >> > Tarush
> > > > > > > > > > >> > On Thu, 4 May 2017 at 8:54 PM, Frances Perry <
> > > > > > > fran...@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > >> >
> > > > > > > > > > >> > > Woohoo! So well deserved.
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > On Thu, May 4, 2017 at 8:18 AM, Etienne Chauchot <
> > > > > > > > > > echauc...@gmail.com>
> > > > > > > > > > >> > > wrote:
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > > Congratulations Davor!
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > Well deserved indeed!
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > Le 04/05/2017 à 17:02, Thomas Groh a écrit :
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > >> Congratulations!
> > > > > > > > > > >> > > >>
> > > > > > > > > > >> > > >> On Thu, May 4, 2017 at 7:56 AM, Thomas Weise <
> > > > > > > t...@apache.org
> > > > > > > > >
> > > > > > > > > > >> wrote:
> > > > > > > > > > >> > > >>
> > > > > > > > > > >> > > >> Congrats!
> > > > > > > > > > >> > > >>>
> > > > > > > > > > >> > > >>>
> > > > > > > > > > >> > > >>> On Thu, May 4, 2017 at 7:53 AM, Sourabh Bajaj
> <
> > > > > > > > > > >> > > >>> sourabhba...@google.com.invalid> wrote:
> > > > > > > > > > >> > > >>>
> > > > > > > > > > >> > > >>> Congrats!!
> > > > > > > > > > >> > >  On Thu, May 4, 2017 at 7:48 AM Mingmin Xu <
> > > > > > > > > mingm...@gmail.com>
> > > > > > > > > > >> > wrote:
> > > > > > > > > > >> > > 
> > > > > > > > > > >> > >  Congratulations @Davor!
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > On May 4, 2017, at 7:08 AM, Amit Sela <
> > > > > > > > amitsel...@gmail.com
> > > > > > > > > >
> > > > > > > > > > >> > wrote:
> > > > > > > > > > >> > > >>
> > > > > > > > > > >> > > >> Congratulations Davor!
> > > > > > > > > > >> > > >>
> > > > > > > > > > >> > > >> On Thu, May 4, 2017, 10:02 JingsongLee <
> > > > > > > > > > lzljs3620...@aliyun.com
> > > > > > > > > > >> >
> > > > > > > > > > >> > > >>>
> > > > > > > > > > >> > > >> wrote:
> > > > > > > > > > >> > > 
> > > > > > > > > > >> > > > Congratulations!
> > > > > > > > > > >> > > >>>
> > > > > > > > > > >> > --
> > > > > --
> > > > > > > > --
> > > > > > > > > > >> > > >>> From:Jesse Anderson <
> > > je...@bigdatainstitute.io>
> > > > > > > > > > >> > > >>> Time:2017 May 4 (Thu) 21:36
> > > > > > > > > > >> > > >>> To:dev 
> > 

First stable release: release candidate #1

[Davor Bonaci]

The release candidate #1 for the version 2.0.0 has been built.

The complete staging area is available for review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint 8F0D334F [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v1.2.3-RC3" [5],
* website pull request listing the release and publishing the API reference
manual [6],
* Python artifacts are deployed along with the source release to the
dist.apache.org [2].

There are known issues in this candidate, so there's no need to vote on it,
but please post on this thread any additional issues you may run into.
Also, see separate thread on acceptance criteria [7].

Thanks!

Davor

[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12339746
[2] https://dist.apache.org/repos/dist/dev/beam/2.0.0-RC1/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1014
[5] https://github.com/apache/beam/tree/v2.0.0-RC1
[6] https://github.com/apache/beam-site/pull/231
[7]
https://lists.apache.org/thread.html/981c2f13c0daf29876059b14dbe97e75bcc9e40d3ac38e33a6ecf3f9@%3Cdev.beam.apache.org%3E

First stable release: Acceptance criteria

[Davor Bonaci]

Based on the process previously discussed [1], I've seeded the acceptance
criteria document [2].

Please consider contributing to this effort by:
* proposing additional acceptance criteria, and/or
* supporting criteria proposed by others, and/or
* validating a criteria.

Please note that acceptance criteria shouldn't been too deep or too broad
-- those are covered by automated tests and hackathon we had earlier. This
should be "sanity-check"-type of criteria: simple, surface-level things.

If you discover issues while validating a criteria, please:
* file a new JIRA issue, tag it as Fix Versions: “2.0.0”, and
* post on the dev@ mailing list on the thread about that specific release
candidate.

Thanks!

Davor

[1]
https://lists.apache.org/thread.html/37caa5a94cec1405638410857f489d7cf7fa12bbe3c36e9925b2d6e2@%3Cdev.beam.apache.org%3E
[2]
https://docs.google.com/document/d/1XwojJ4Mj3wSlnBO1YlBs51P8kuGygYRj2lrNrqmAUvo/

Re: Process for getting the first stable release out

[Robert Bradshaw]

On Mon, May 8, 2017 at 1:12 PM, Dan Halperin
 wrote:
> I like putting in master then CPing into release, because we should have a
> high bar for what goes into release. It should absolutely NOT default to
> everything; we should have to justify everything.

Agreed the bar should be high, and "release" should not be the
default. However, for things we know we want to be in the release
(which is much of the work going on right now) it's different. E.g.
instructions like "no backwards compatible changes in master *unless*
you're planning on cherry picking them" would be better stated "no
backwards compatible changes in master" and then the debate about
whether such a change should be in release would happen on the
release-sitting PR.

Right now we're using the master branch as a mix of release-intended
and post-release PRs, which is cumbersome and error-prone.

> E.g., https://github.com/apache/beam/pull/2958 - where I open the CP but
> suggest this may not be worthy for release as it's just cleaning up logs
> and errors.

It's easy to discuss on the PR the best destination, and change it to
be against one or the other before it's merged. There certainly may be
cases where we decide to merge into master to be safe and optionally
CP after the fact, but for many PRs it's clear where they should end
up.

> On Mon, May 8, 2017 at 1:10 PM, Robert Bradshaw > wrote:
>
>> On Mon, May 8, 2017 at 12:57 PM, Davor Bonaci  wrote:
>> > We cannot do (clean) merges; both branches contain unwanted changes in
>> the
>> > other branch. So, we have to cherry-pick regardless where we merge first.
>>
>> Shouldn't the set of changes wanted in release but not in master be
>> quite small (if any)? In that case, one could do an explicit revert
>> following merge from release to master, when needed. The extra work
>> scales in terms of how much we want to diverge (rather than all the
>> changes we want in release that should also be in master, which is the
>> bulk of them, touched by significantly more people.)
>>
>> > With post commits running automatically on master only, that seems like a
>> > logical starting point. But, it doesn't matter really -- either way
>> works.
>> >
>> > On Mon, May 8, 2017 at 12:30 PM, Robert Bradshaw <
>> > rober...@google.com.invalid> wrote:
>> >
>> >> An alternative strategy, given the number of outstanding changes,
>> >> would be to create release-intended PRs against the release branch
>> >> itself, then periodically merge back to master. This would reduce the
>> >> need for manual (and error-prone) cherry-picking.
>> >>
>> >> On Fri, May 5, 2017 at 5:21 PM, Davor Bonaci  wrote:
>> >> > The release branch is now created [1]. Anything for the first stable
>> >> > release should go into master as usual, and then get cherry-picked
>> into
>> >> the
>> >> > release branch.
>> >> >
>> >> > I'll create the first RC shortly and then start a doc around the
>> >> acceptance
>> >> > criteria.
>> >> >
>> >> > From this point onward, backward-incompatible changes should *not* be
>> >> > merged to master, unless they are also getting cherry-picked into the
>> >> > release branch.
>> >> >
>> >> > Davor
>> >> >
>> >> > [1] https://github.com/apache/beam/tree/release-2.0.0
>> >> >
>> >> > On Fri, May 5, 2017 at 1:57 PM, Thomas Groh > >
>> >> > wrote:
>> >> >
>> >> >> I'm also +1 on the branch. It'll help us make sure that what we're
>> >> getting
>> >> >> in is what we need for the FSR.
>> >> >>
>> >> >> On Fri, May 5, 2017 at 12:41 PM, Dan Halperin 
>> >> wrote:
>> >> >>
>> >> >> > I am +1 on cutting the branch, and the sentiment that we expect the
>> >> first
>> >> >> > pancake
>> >> >> > > first-pancake
>> >> >
>> >> >> > will
>> >> >> > be not ready to serve customers.
>> >> >> >
>> >> >> > On Fri, May 5, 2017 at 11:40 AM, Kenneth Knowles
>> >> > >> >> >
>> >> >> > wrote:
>> >> >> >
>> >> >> > > On Thu, May 4, 2017 at 12:07 PM, Davor Bonaci 
>> >> >> wrote:
>> >> >> > >
>> >> >> > > > I'd like to propose the following (tweaked) process for this
>> >> special
>> >> >> > > > release:
>> >> >> > > >
>> >> >> > > > * Create a release branch, and start building release
>> candidates
>> >> >> *now*
>> >> >> > > > This would accelerate branch creation compared to the normal
>> >> process,
>> >> >> > but
>> >> >> > > > would separate the first stable release from other development
>> on
>> >> the
>> >> >> > > > master branch. This yields to stability and avoids unnecessary
>> >> churn.
>> >> >> > > >
>> >> >> > >
>> >> >> > > +1 to cutting a release branch now.
>> >> >> > >
>> >> >> > > This sounds compatible with the release process [1] to me,
>> actually.
>> >> >> This
>> >> >> > > thread seems like the dev@ thread where we "decide to release"
>> and
>> >> I
>> >> >> > agree
>> 

Re: Process for getting the first stable release out

[Dan Halperin]

I like putting in master then CPing into release, because we should have a
high bar for what goes into release. It should absolutely NOT default to
everything; we should have to justify everything.

E.g., https://github.com/apache/beam/pull/2958 - where I open the CP but
suggest this may not be worthy for release as it's just cleaning up logs
and errors.

On Mon, May 8, 2017 at 1:10 PM, Robert Bradshaw  wrote:

> On Mon, May 8, 2017 at 12:57 PM, Davor Bonaci  wrote:
> > We cannot do (clean) merges; both branches contain unwanted changes in
> the
> > other branch. So, we have to cherry-pick regardless where we merge first.
>
> Shouldn't the set of changes wanted in release but not in master be
> quite small (if any)? In that case, one could do an explicit revert
> following merge from release to master, when needed. The extra work
> scales in terms of how much we want to diverge (rather than all the
> changes we want in release that should also be in master, which is the
> bulk of them, touched by significantly more people.)
>
> > With post commits running automatically on master only, that seems like a
> > logical starting point. But, it doesn't matter really -- either way
> works.
> >
> > On Mon, May 8, 2017 at 12:30 PM, Robert Bradshaw <
> > rober...@google.com.invalid> wrote:
> >
> >> An alternative strategy, given the number of outstanding changes,
> >> would be to create release-intended PRs against the release branch
> >> itself, then periodically merge back to master. This would reduce the
> >> need for manual (and error-prone) cherry-picking.
> >>
> >> On Fri, May 5, 2017 at 5:21 PM, Davor Bonaci  wrote:
> >> > The release branch is now created [1]. Anything for the first stable
> >> > release should go into master as usual, and then get cherry-picked
> into
> >> the
> >> > release branch.
> >> >
> >> > I'll create the first RC shortly and then start a doc around the
> >> acceptance
> >> > criteria.
> >> >
> >> > From this point onward, backward-incompatible changes should *not* be
> >> > merged to master, unless they are also getting cherry-picked into the
> >> > release branch.
> >> >
> >> > Davor
> >> >
> >> > [1] https://github.com/apache/beam/tree/release-2.0.0
> >> >
> >> > On Fri, May 5, 2017 at 1:57 PM, Thomas Groh  >
> >> > wrote:
> >> >
> >> >> I'm also +1 on the branch. It'll help us make sure that what we're
> >> getting
> >> >> in is what we need for the FSR.
> >> >>
> >> >> On Fri, May 5, 2017 at 12:41 PM, Dan Halperin 
> >> wrote:
> >> >>
> >> >> > I am +1 on cutting the branch, and the sentiment that we expect the
> >> first
> >> >> > pancake
> >> >> >  first-pancake
> >> >
> >> >> > will
> >> >> > be not ready to serve customers.
> >> >> >
> >> >> > On Fri, May 5, 2017 at 11:40 AM, Kenneth Knowles
> >>  >> >> >
> >> >> > wrote:
> >> >> >
> >> >> > > On Thu, May 4, 2017 at 12:07 PM, Davor Bonaci 
> >> >> wrote:
> >> >> > >
> >> >> > > > I'd like to propose the following (tweaked) process for this
> >> special
> >> >> > > > release:
> >> >> > > >
> >> >> > > > * Create a release branch, and start building release
> candidates
> >> >> *now*
> >> >> > > > This would accelerate branch creation compared to the normal
> >> process,
> >> >> > but
> >> >> > > > would separate the first stable release from other development
> on
> >> the
> >> >> > > > master branch. This yields to stability and avoids unnecessary
> >> churn.
> >> >> > > >
> >> >> > >
> >> >> > > +1 to cutting a release branch now.
> >> >> > >
> >> >> > > This sounds compatible with the release process [1] to me,
> actually.
> >> >> This
> >> >> > > thread seems like the dev@ thread where we "decide to release"
> and
> >> I
> >> >> > agree
> >> >> > > that we should decide to release. Certainly `master` is not ready
> >> nor
> >> >> is
> >> >> > > the web site - there are ~29 issues as I write this though many
> are
> >> not
> >> >> > > really significant code changes. But we should never wait until
> >> >> `master`
> >> >> > is
> >> >> > > "ready".
> >> >> > >
> >> >> > > We know what we want to get done, and there are no radical
> changes,
> >> so
> >> >> I
> >> >> > > think that makes this the right time to branch. We can easily
> cherry
> >> >> pick
> >> >> > > fixes for our burndown list to ensure we don't introduce
> additional
> >> >> > > blockers.
> >> >> > >
> >> >> > > Some of the burndown list are of the form "investigate if this
> >> >> suspected
> >> >> > > bug still repros" and a release candidate is the perfect thing to
> >> use
> >> >> for
> >> >> > > that.
> >> >> > >
> >> >> > > [1] https://beam.apache.org/contribute/release-guide/#
> >> >> decide-to-release
> >> >> > >
> >> >> >
> >> >>
> >>
>

Re: Process for getting the first stable release out

[Robert Bradshaw]

On Mon, May 8, 2017 at 12:57 PM, Davor Bonaci  wrote:
> We cannot do (clean) merges; both branches contain unwanted changes in the
> other branch. So, we have to cherry-pick regardless where we merge first.

Shouldn't the set of changes wanted in release but not in master be
quite small (if any)? In that case, one could do an explicit revert
following merge from release to master, when needed. The extra work
scales in terms of how much we want to diverge (rather than all the
changes we want in release that should also be in master, which is the
bulk of them, touched by significantly more people.)

> With post commits running automatically on master only, that seems like a
> logical starting point. But, it doesn't matter really -- either way works.
>
> On Mon, May 8, 2017 at 12:30 PM, Robert Bradshaw <
> rober...@google.com.invalid> wrote:
>
>> An alternative strategy, given the number of outstanding changes,
>> would be to create release-intended PRs against the release branch
>> itself, then periodically merge back to master. This would reduce the
>> need for manual (and error-prone) cherry-picking.
>>
>> On Fri, May 5, 2017 at 5:21 PM, Davor Bonaci  wrote:
>> > The release branch is now created [1]. Anything for the first stable
>> > release should go into master as usual, and then get cherry-picked into
>> the
>> > release branch.
>> >
>> > I'll create the first RC shortly and then start a doc around the
>> acceptance
>> > criteria.
>> >
>> > From this point onward, backward-incompatible changes should *not* be
>> > merged to master, unless they are also getting cherry-picked into the
>> > release branch.
>> >
>> > Davor
>> >
>> > [1] https://github.com/apache/beam/tree/release-2.0.0
>> >
>> > On Fri, May 5, 2017 at 1:57 PM, Thomas Groh 
>> > wrote:
>> >
>> >> I'm also +1 on the branch. It'll help us make sure that what we're
>> getting
>> >> in is what we need for the FSR.
>> >>
>> >> On Fri, May 5, 2017 at 12:41 PM, Dan Halperin 
>> wrote:
>> >>
>> >> > I am +1 on cutting the branch, and the sentiment that we expect the
>> first
>> >> > pancake
>> >> > > >
>> >> > will
>> >> > be not ready to serve customers.
>> >> >
>> >> > On Fri, May 5, 2017 at 11:40 AM, Kenneth Knowles
>> > >> >
>> >> > wrote:
>> >> >
>> >> > > On Thu, May 4, 2017 at 12:07 PM, Davor Bonaci 
>> >> wrote:
>> >> > >
>> >> > > > I'd like to propose the following (tweaked) process for this
>> special
>> >> > > > release:
>> >> > > >
>> >> > > > * Create a release branch, and start building release candidates
>> >> *now*
>> >> > > > This would accelerate branch creation compared to the normal
>> process,
>> >> > but
>> >> > > > would separate the first stable release from other development on
>> the
>> >> > > > master branch. This yields to stability and avoids unnecessary
>> churn.
>> >> > > >
>> >> > >
>> >> > > +1 to cutting a release branch now.
>> >> > >
>> >> > > This sounds compatible with the release process [1] to me, actually.
>> >> This
>> >> > > thread seems like the dev@ thread where we "decide to release" and
>> I
>> >> > agree
>> >> > > that we should decide to release. Certainly `master` is not ready
>> nor
>> >> is
>> >> > > the web site - there are ~29 issues as I write this though many are
>> not
>> >> > > really significant code changes. But we should never wait until
>> >> `master`
>> >> > is
>> >> > > "ready".
>> >> > >
>> >> > > We know what we want to get done, and there are no radical changes,
>> so
>> >> I
>> >> > > think that makes this the right time to branch. We can easily cherry
>> >> pick
>> >> > > fixes for our burndown list to ensure we don't introduce additional
>> >> > > blockers.
>> >> > >
>> >> > > Some of the burndown list are of the form "investigate if this
>> >> suspected
>> >> > > bug still repros" and a release candidate is the perfect thing to
>> use
>> >> for
>> >> > > that.
>> >> > >
>> >> > > [1] https://beam.apache.org/contribute/release-guide/#
>> >> decide-to-release
>> >> > >
>> >> >
>> >>
>>

Re: Process for getting the first stable release out

[Davor Bonaci]

We cannot do (clean) merges; both branches contain unwanted changes in the
other branch. So, we have to cherry-pick regardless where we merge first.
With post commits running automatically on master only, that seems like a
logical starting point. But, it doesn't matter really -- either way works.

On Mon, May 8, 2017 at 12:30 PM, Robert Bradshaw <
rober...@google.com.invalid> wrote:

> An alternative strategy, given the number of outstanding changes,
> would be to create release-intended PRs against the release branch
> itself, then periodically merge back to master. This would reduce the
> need for manual (and error-prone) cherry-picking.
>
> On Fri, May 5, 2017 at 5:21 PM, Davor Bonaci  wrote:
> > The release branch is now created [1]. Anything for the first stable
> > release should go into master as usual, and then get cherry-picked into
> the
> > release branch.
> >
> > I'll create the first RC shortly and then start a doc around the
> acceptance
> > criteria.
> >
> > From this point onward, backward-incompatible changes should *not* be
> > merged to master, unless they are also getting cherry-picked into the
> > release branch.
> >
> > Davor
> >
> > [1] https://github.com/apache/beam/tree/release-2.0.0
> >
> > On Fri, May 5, 2017 at 1:57 PM, Thomas Groh 
> > wrote:
> >
> >> I'm also +1 on the branch. It'll help us make sure that what we're
> getting
> >> in is what we need for the FSR.
> >>
> >> On Fri, May 5, 2017 at 12:41 PM, Dan Halperin 
> wrote:
> >>
> >> > I am +1 on cutting the branch, and the sentiment that we expect the
> first
> >> > pancake
> >> >  >
> >> > will
> >> > be not ready to serve customers.
> >> >
> >> > On Fri, May 5, 2017 at 11:40 AM, Kenneth Knowles
>  >> >
> >> > wrote:
> >> >
> >> > > On Thu, May 4, 2017 at 12:07 PM, Davor Bonaci 
> >> wrote:
> >> > >
> >> > > > I'd like to propose the following (tweaked) process for this
> special
> >> > > > release:
> >> > > >
> >> > > > * Create a release branch, and start building release candidates
> >> *now*
> >> > > > This would accelerate branch creation compared to the normal
> process,
> >> > but
> >> > > > would separate the first stable release from other development on
> the
> >> > > > master branch. This yields to stability and avoids unnecessary
> churn.
> >> > > >
> >> > >
> >> > > +1 to cutting a release branch now.
> >> > >
> >> > > This sounds compatible with the release process [1] to me, actually.
> >> This
> >> > > thread seems like the dev@ thread where we "decide to release" and
> I
> >> > agree
> >> > > that we should decide to release. Certainly `master` is not ready
> nor
> >> is
> >> > > the web site - there are ~29 issues as I write this though many are
> not
> >> > > really significant code changes. But we should never wait until
> >> `master`
> >> > is
> >> > > "ready".
> >> > >
> >> > > We know what we want to get done, and there are no radical changes,
> so
> >> I
> >> > > think that makes this the right time to branch. We can easily cherry
> >> pick
> >> > > fixes for our burndown list to ensure we don't introduce additional
> >> > > blockers.
> >> > >
> >> > > Some of the burndown list are of the form "investigate if this
> >> suspected
> >> > > bug still repros" and a release candidate is the perfect thing to
> use
> >> for
> >> > > that.
> >> > >
> >> > > [1] https://beam.apache.org/contribute/release-guide/#
> >> decide-to-release
> >> > >
> >> >
> >>
>

Re: On emitting from finshBundle

[Robert Bradshaw]

I think we agree that emitting from FinalizeBundle without specifying
the Window explicitly should be an error for anything but the global
window (similar to global combine with defaults). The question is
whether it should be allowable in the global window case for
convenience (again, similar to global combine), as long as we can
detect its abuse.

I'd be interested in hearing if others have thoughts on this as well.

- Robert


On Fri, May 5, 2017 at 2:05 PM, Kenneth Knowles  wrote:
> On Fri, May 5, 2017 at 1:53 PM, Robert Bradshaw > wrote:
>
>> On Fri, May 5, 2017 at 1:33 PM, Kenneth Knowles 
>> wrote:
>> > On Fri, May 5, 2017 at 12:43 PM, Robert Bradshaw <
>> > rober...@google.com.invalid> wrote:
>> > To
>> > maintain that, safely, we would add GloballyWindowedOutput or some such
>> > with the output() and outputTimestamp() method. This is routine work
>> quite
>> > analogous to what we already have. We would validate that the output
>> > windowing strategy is global windows if the parameter is present.
>>
>> So the current recommendation is to write
>>
>> c.output(
>> [value],
>> BoundedWindow.TIMESTAMP_MIN_VALUE,
>> GlobalWindow.INSTANCE);
>>
>> but we could recommend the user use a different context to not have to
>> specify BoundedWindow.TIMESTAMP_MIN_VALUE and GlobalWindow.INSTANCE so
>> that we could check this at compile time?
>>
>
> Yup. We are trying to take everything out of every context and get them
> into grokkable parameters.
>
>
>
>> >> We
>> >> > can also enforce that the correct subclass of window is output, though
>> >> this
>> >> > is not implemented as such right now.
>> >>
>> >> Are you thinking a type check based on the reflectively obtained
>> >> parameter of the WindowFn?
>> >>
>> >
>> > Yes. Though we don't do it today, it would be OutputWithWindow or some
>> > such (which is actually the same as FinishBundleContext) where W was
>> > some window class that would be checked against the window coder. We
>> cannot
>> > use the same inner class trick to avoid referencing the type, but that
>> > shouldn't be burdensome in this case.
>>
>> Is adding a type parameter backwards compatible?
>
>
> We'd just make a new type rather than change FinishBundleContext. I just
> wanted to mention that it is a tiny change. It is not important to get in
> for FSR because of this.
>
> Kenn
>
>
> I suppose with raw
>> types it is... Not sure how this would work with a proper BatchignDoFn
>> that may output to any window type, but maybe.
>>
>> Python's a bit different as it doesn't use contexts to emit values.
>>
>> >> But this behavior is all to support people who want to write transforms
>> >> > that work with essentially only one windowing strategy, basically
>> >> > pre-windowing stuff. Are there really any meaningful alternatives to
>> the
>> >> > global window? If so, I don't think that is realistically what this is
>> >> > about. Such transforms should be permitted, but not encouraged or
>> >> supported
>> >> > automatically, as they are antithetical to the unified model. So I
>> would
>> >> > also suggest that they should validate that their input is globally
>> >> > windowed and explicitly output to the global window. Trivial and they
>> >> > already should be doing that. They also have the capability of storing
>> >> the
>> >> > input's WindowFn and producing behavior identical to what you
>> describe.
>> >>
>> >> What it boils down to is that this is a common case that only works
>> >> easily in the Global window. Another example that I can think of like
>> >> this in the SDK, namely global combine. If you're in the global
>> >> window, you get a singleton PCollection, and we don't require users to
>> >> do extra for this behavior. If someone tries to use this transform in
>> >> a non-globally-windowed setting, only then do we give an error (this
>> >> time at pipeline construction) and force the user to do extra work.
>> >> It's a balance between having something that works correctly in the
>> >> unified model but requires extra (potentially tricky and error-prone)
>> >> effort only iff one needs it.
>>
>> The question above is the crux of the issue--are we striking the right
>> balance in requiring the user to specify a Window in all cases when we
>> can infer it in the by far most common case (like with global
>> combine's default value) and can detect and warn the user when they
>> don't fall into that case.
>>

Re: First stable release: version designation?

[Robert Bradshaw]

I also have a definite (I guess that's closer to strong that slight)
preference for 2.0.

With version numbers, a gap is less likely to cause trouble than the
ambiguity of an overlap, and easy to document (vs. with ambiguity, one
wouldn't even think to consult the documentation without knowing the
history).

On Mon, May 8, 2017 at 6:41 AM, Pei HE  wrote:
> I vote for 2.0.
>
> On Sun, May 7, 2017 at 7:50 PM, Prabeesh K.  wrote:
>
>> I also vote for 2.0.
>>
>> On 5 May 2017 at 21:33, Hadar Hod  wrote:
>>
>> > I also vote for 2.0, for the same reasons Dan stated.
>> > As Cham mentioned, we can clarify any confusion in the documentation.
>> >
>> > On Fri, May 5, 2017 at 9:50 AM, Ahmet Altay 
>> > wrote:
>> >
>> > > I would also like to vote for strong 2.0 with the same reasons as Dan
>> > > mentioned. It will be less confusing for the users overall.
>> > >
>> > > Ahmet
>> > >
>> > > On Fri, May 5, 2017 at 9:33 AM, Davor Bonaci  wrote:
>> > >
>> > > > Strongly for 2.0.0:
>> > > > * Aljoscha
>> > > > * Cham
>> > > > * Dan
>> > > > * Luke
>> > > >
>> > > > Slight preference toward 2.0.0, but fine with 1.0.0:
>> > > > * Davor
>> > > > * Ismael
>> > > > * Kenn
>> > > >
>> > > > Strongly for 1.0.0: none.
>> > > >
>> > > > Slight preference toward 1.0.0, but fine with 2.0.0:
>> > > > * Amit
>> > > > * Jesse
>> > > > * JB
>> > > > * Manu
>> > > > * Mingmin
>> > > > * Ted
>> > > > * Thomas W.
>> > > >
>> > > > Unbelievably, the tally is 7 : 7. However, the 2.0 camp tends to feel
>> > > more
>> > > > strongly, and we have nobody who feels strongly for 1.0. Thus, it
>> seems
>> > > > going with 2.0.0 is the path of least resistance.
>> > > >
>> > > > With that, I'll start building the 2.0.0 RCs, and we'll formally
>> > > > ratify/reject this decision in an RC vote.
>> > > >
>> > > > On Thu, May 4, 2017 at 6:30 PM, María García Herrero <
>> > > > mari...@google.com.invalid> wrote:
>> > > >
>> > > > > The bigger letters aimed to indicate "strongly in favor of" as
>> > opposed
>> > > to
>> > > > > "weakly in favor of." I'm OK with not using a doc, just responding
>> to
>> > > > Ted's
>> > > > > question.
>> > > > >
>> > > > > On Thu, May 4, 2017 at 3:39 PM, Ted Yu 
>> wrote:
>> > > > >
>> > > > > > What's the difference between first and second, third and fourth
>> > > > columns
>> > > > > ?
>> > > > > >
>> > > > > > On Thu, May 4, 2017 at 3:36 PM, María García Herrero <
>> > > > > > mari...@google.com.invalid> wrote:
>> > > > > >
>> > > > > > > Thanks for the suggestion, Ted. Get your vote in here
>> > > > > > > > > 1ABx3U8ojcfUkFig3hG53lOYl73tdk
>> > > > > > > Wqz5B6eQ40TEgk/edit?usp=sharing>
>> > > > > > > .
>> > > > > > > I have already added all the votes that Davor compiled 3 hours
>> > ago
>> > > > and
>> > > > > > the
>> > > > > > > responses afterwards.
>> > > > > > >
>> > > > > > > On Thu, May 4, 2017 at 12:49 PM, Ted Yu 
>> > > wrote:
>> > > > > > >
>> > > > > > > > Maybe create a google doc with columns as the camps.
>> > > > > > > >
>> > > > > > > > Each person can put his/her name under the camp in his/her
>> > favor.
>> > > > > > > >
>> > > > > > > > On Thu, May 4, 2017 at 12:32 PM, Thomas Weise <
>> t...@apache.org>
>> > > > > wrote:
>> > > > > > > >
>> > > > > > > > > I'm in the relaxed 1.0.0 camp.
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > sent from mobile
>> > > > > > > > > On May 4, 2017 12:29 PM, "Mingmin Xu" 
>> > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > I slightly prefer1.0.0 for the *first* stable release,
>> but
>> > > fine
>> > > > > > with
>> > > > > > > > > 2.0.0.
>> > > > > > > > > >
>> > > > > > > > > > On Thu, May 4, 2017 at 12:25 PM, Lukasz Cwik
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Put me under Strongly for 2.0.0
>> > > > > > > > > > >
>> > > > > > > > > > > On Thu, May 4, 2017 at 12:24 PM, Kenneth Knowles
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > I'll join Davor's group.
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Thu, May 4, 2017 at 12:07 PM, Davor Bonaci <
>> > > > > > da...@apache.org>
>> > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > I don't think we have reached a consensus here yet.
>> > > Let's
>> > > > > > > > > re-examine
>> > > > > > > > > > > this
>> > > > > > > > > > > > > after some time has passed.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > If I understand everyone's opinion correctly, this
>> is
>> > > the
>> > > > > > > > summary:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Strongly for 2.0.0:
>> > > > > > > > > > > > > * Aljoscha
>> > > > > > > > > > > > > * Dan
>> > > > > > > > > > > > >

Re: First stable release: version designation?

[Pei HE]

I vote for 2.0.

On Sun, May 7, 2017 at 7:50 PM, Prabeesh K.  wrote:

> I also vote for 2.0.
>
> On 5 May 2017 at 21:33, Hadar Hod  wrote:
>
> > I also vote for 2.0, for the same reasons Dan stated.
> > As Cham mentioned, we can clarify any confusion in the documentation.
> >
> > On Fri, May 5, 2017 at 9:50 AM, Ahmet Altay 
> > wrote:
> >
> > > I would also like to vote for strong 2.0 with the same reasons as Dan
> > > mentioned. It will be less confusing for the users overall.
> > >
> > > Ahmet
> > >
> > > On Fri, May 5, 2017 at 9:33 AM, Davor Bonaci  wrote:
> > >
> > > > Strongly for 2.0.0:
> > > > * Aljoscha
> > > > * Cham
> > > > * Dan
> > > > * Luke
> > > >
> > > > Slight preference toward 2.0.0, but fine with 1.0.0:
> > > > * Davor
> > > > * Ismael
> > > > * Kenn
> > > >
> > > > Strongly for 1.0.0: none.
> > > >
> > > > Slight preference toward 1.0.0, but fine with 2.0.0:
> > > > * Amit
> > > > * Jesse
> > > > * JB
> > > > * Manu
> > > > * Mingmin
> > > > * Ted
> > > > * Thomas W.
> > > >
> > > > Unbelievably, the tally is 7 : 7. However, the 2.0 camp tends to feel
> > > more
> > > > strongly, and we have nobody who feels strongly for 1.0. Thus, it
> seems
> > > > going with 2.0.0 is the path of least resistance.
> > > >
> > > > With that, I'll start building the 2.0.0 RCs, and we'll formally
> > > > ratify/reject this decision in an RC vote.
> > > >
> > > > On Thu, May 4, 2017 at 6:30 PM, María García Herrero <
> > > > mari...@google.com.invalid> wrote:
> > > >
> > > > > The bigger letters aimed to indicate "strongly in favor of" as
> > opposed
> > > to
> > > > > "weakly in favor of." I'm OK with not using a doc, just responding
> to
> > > > Ted's
> > > > > question.
> > > > >
> > > > > On Thu, May 4, 2017 at 3:39 PM, Ted Yu 
> wrote:
> > > > >
> > > > > > What's the difference between first and second, third and fourth
> > > > columns
> > > > > ?
> > > > > >
> > > > > > On Thu, May 4, 2017 at 3:36 PM, María García Herrero <
> > > > > > mari...@google.com.invalid> wrote:
> > > > > >
> > > > > > > Thanks for the suggestion, Ted. Get your vote in here
> > > > > > >  > 1ABx3U8ojcfUkFig3hG53lOYl73tdk
> > > > > > > Wqz5B6eQ40TEgk/edit?usp=sharing>
> > > > > > > .
> > > > > > > I have already added all the votes that Davor compiled 3 hours
> > ago
> > > > and
> > > > > > the
> > > > > > > responses afterwards.
> > > > > > >
> > > > > > > On Thu, May 4, 2017 at 12:49 PM, Ted Yu 
> > > wrote:
> > > > > > >
> > > > > > > > Maybe create a google doc with columns as the camps.
> > > > > > > >
> > > > > > > > Each person can put his/her name under the camp in his/her
> > favor.
> > > > > > > >
> > > > > > > > On Thu, May 4, 2017 at 12:32 PM, Thomas Weise <
> t...@apache.org>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > I'm in the relaxed 1.0.0 camp.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > sent from mobile
> > > > > > > > > On May 4, 2017 12:29 PM, "Mingmin Xu" 
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > I slightly prefer1.0.0 for the *first* stable release,
> but
> > > fine
> > > > > > with
> > > > > > > > > 2.0.0.
> > > > > > > > > >
> > > > > > > > > > On Thu, May 4, 2017 at 12:25 PM, Lukasz Cwik
> > > > > > >  > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Put me under Strongly for 2.0.0
> > > > > > > > > > >
> > > > > > > > > > > On Thu, May 4, 2017 at 12:24 PM, Kenneth Knowles
> > > > > > > > >  > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I'll join Davor's group.
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, May 4, 2017 at 12:07 PM, Davor Bonaci <
> > > > > > da...@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > I don't think we have reached a consensus here yet.
> > > Let's
> > > > > > > > > re-examine
> > > > > > > > > > > this
> > > > > > > > > > > > > after some time has passed.
> > > > > > > > > > > > >
> > > > > > > > > > > > > If I understand everyone's opinion correctly, this
> is
> > > the
> > > > > > > > summary:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Strongly for 2.0.0:
> > > > > > > > > > > > > * Aljoscha
> > > > > > > > > > > > > * Dan
> > > > > > > > > > > > >
> > > > > > > > > > > > > Slight preference toward 2.0.0, but fine with
> 1.0.0:
> > > > > > > > > > > > > * Davor
> > > > > > > > > > > > >
> > > > > > > > > > > > > Strongly for 1.0.0: none.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Slight preference toward 1.0.0, but fine with
> 2.0.0:
> > > > > > > > > > > > > * Amit
> > > > > > > > > > > > > * Jesse
> > > > > > > > > > > > > * JB
> > > > > > > > > > > > > * Ted
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any additional opinions?
> > > > > 

[DISCUSS] Remove TimerInternals.deleteTimer(*) and Timer.cancel()

[Aljoscha Krettek]

I wanted to bring this up before the First Stable release and see what other 
people think. The methods I’m talking about are:

void deleteTimer(StateNamespace namespace, String timerId, TimeDomain 
timeDomain);

@Deprecated
void deleteTimer(StateNamespace namespace, String timerId);

@Deprecated
void deleteTimer(TimerData timerKey);

The last two are slated for deletion. Notice that the first one doesn’t take 
the timestamp of the timer to delete, which implies that there is only one 
active timer per namespace/timerId/timeDomain.

These are my motivations for removal:
 - Timer removal is difficult and has varying levels of support on different 
Runners and varying levels of cost. 
 - Removing the interface now and adding it back later is easy. Having it in 
the FSR and later removing it is quite hard.

I can only speak about the internal Flink implementation where Timers are on a 
Heap (Java Priority Queue). Here removal is quite expensive, see, for example 
this ML thread: 
https://lists.apache.org/thread.html/ac4d8e36360779a9b5047cf21303222980015720aac478e8cf632216@%3Cuser.flink.apache.org%3E.
 Especially this part:

I thought I would drop my opinion here maybe it is relevant.

We have used the Flink internal timer implementation in many of our production 
applications, this supports the Timer deletion but the deletion actually turned 
out to be a huge performance bottleneck because of the bad deletion performance 
of the Priority queue.

In many of our cases deletion could have been avoided by some more clever 
registration/firing logic and we also ended up completely avoiding deletion and 
instead using "tombstone" markers by setting a flag in the state which timers 
not to fire when they actually trigger.

Gyula

Note that in Flink it’s not possible to delete a timer if you don’t know the 
timestamp. Other systems might store timers in more elaborate data structures 
(hierarchical timer wheels come to mind) where it’s also hard to delete a timer 
without knowing the timestamp.

If we decide to keep timer removal in the user API there’s the possibility to 
simulate timer removal by keeping the timestamp of the currently active timer 
for a given timerID and checking a firing timer against that.

Best,
Aljoscha