Our approach of having 60+ subpackages is I think fairly unique in ASF
terms, where most projects just have one package that they release, so
the ASF guidance doesn't consider this situation -- we are on our own
to come up with something that works for us.
The only other project I know that has as many sub-packages is Maven,
and they do one email = one package = one vote
<https://lists.apache.org/[email protected]:gte=1d:VOTE>
(they don't batch either)
For example these are started on the same day
<https://lists.apache.org/thread.html/r92026b74d0c32b2d3dc06794ec071375d6a135057b41b13359263003%40%3Cdev.maven.apache.org%3E>
<https://lists.apache.org/thread.html/r6cd5e53070bc033fae07d3e236c7c0cbdf956da358759a46919821ac%40%3Cdev.maven.apache.org%3E>
For me what we are silently (and wrongly) assuming 1 mail = 1 VOTE.
But I do not believe anywhere in all ASF documentation there is such
a requirement.
My counter argument to this point: the ASF guidance talks about "Votes
on whether *a* package is ready to be released" (emphasis mine). But
ultimately I think the guidance is just that -- guidance and we should
just come up with and write down how we want to operate here.
Could you give an snippet/example of what 20 votes in one email would
look like? Are you thinking of the same email we send right now or
something different?
On Wed, 10 Mar, 2021 at 04:02, Jarek Potiuk <[email protected]> wrote:
TL;DR; My very, very strong preference goes for one email which is
effectively N votes for N packages we release. I actually think we
are not able to efficiently follow up and manage 1-vote for email for
~20 or so packages we are going to release every month.
I think we should just refer back to what ASF wrote about it.
<https://www.apache.org/foundation/voting.html>
> VOTES ON PACKAGE RELEASES
> Votes on whether a package is ready to be released use majority
approval -- i.e. at least three PMC members must vote affirmatively
for release, and there must be more positive than negative votes.
Releases may not be vetoed. Generally the community will cancel the
release vote if anyone identifies serious problems, but in most cases
the ultimate decision, lies with the individual serving as release
manager. The specifics of the process may vary from project to
project, but the 'minimum
> quorum of three +1 votes' rule is universal.
"Generally the community will cancel the release vote if anyone
identifies serious problems, but in most cases the ultimate decision,
lies with the individual serving as release manager." - but as I read
it - it's NOT the PMC, but the community decides when to cancel
release and the ultimate decision lies in the hands of the release
manager.
This has happened in the past with Airflow many times, that the
release manager usually Kaxil decided on canceling the release vote.
And yes - there is no mentioning of partial "canceling". But, this
does not mean that there MUST be a single mail per vote either. I
think we can have a single email and a N votes that cover all
packages in a single email. And you can vote in bulk on all packages.
And some users might vote only on some of the packages. For me this
is purely for the efficiency of process and PMCs responding to the
votes.
Imagine we have 20 packages to send regularly (guesstimate),
following up 20 messages to vote will be quite overwhelming -
borderline ridiculous in fact. I am talking about following up and
counting, not sending. Sending is easy.
We often have difficulty chasing PMCs to respond to one email, and
the job of tracking how many responses were sent to each of the 20
emails and following up on each of them will make the release
manager's life miserable.
First two weeks after sending votes, the release manager will have to
chase 20 emails, follow up, and count the votes for every single
release. I would certainly reject such a task as a release manager.
It is not easy to automate at all and is a lot of very stupid and
tim-consuming overhead on top of the work done already by release
manager.
I do not really think the analogy of ice cream and dogs is
appropriate (at all). It's misleading actually. We are not voting on
different things. It's all about cancelling some votes out of many
that we agreed to in bulk at most. But not about changing the votes.
This can already be done if the community raises an issue - with the
ultimate decision of the release manager - so nothing changes here
vs. the current Apache voting process. For me it's only the question
about granularity of the votes in one email and whether we can cast a
"bulk" N votes by single +1. I think we can.
For me what we are silently (and wrongly) assuming 1 mail = 1 VOTE.
But I do not believe anywhere in all ASF documentation there is such
a requirement. We have example "templates" for emails but I do not
know if anywhere this 1-1 relationship (email = Vote) is mentioned.
I'd love to see it if it is. For me as long as it is clear what we
are voting on and what eventually gets cancelled and we summaarise it
in the "RESULT" email - we should be good with having N votes in a
single email.
So for me "bulk" voting looks as follows:
1) we send 20 packages and start 20 votes in one email.
2) Anyone - user, PMC could respond +1 (for all) or +1 (some) or +1
(- this packages I tested which do not work).
3) Community (ultimately release manager) decides which of those
votes to cancel - if any (providing that all of the votes have +3
votes from PMC members).
4) Releasing packages can still be done in waves - if we see after 3
days that we can to release 10 out of the 20 package (but 10 has just
+2 PMC votes) - the release manager can still release the 10 and
continue voting on the remaining 10 (and even eventually cancel them).
Also a few words about testing those 20 packages.
Regarding testing - yes the packages MUST be tested by PMC members.
This is a must. But testing means in this case - bulk installing the
packages on top of airflow and see if the providers have been
installed and shown in `airlow providers list` with the correct
version. I believe this is more than enough to fulfill the "test"
criteria when paired with user testing of new/big functionalities.
This is (or should be - I followed it for sure) already part of the
process (alongside signing/checksum verification). And with svn
checkout where you can pull the whole folder to your machine, this
can be done rather quickly.
As Kaxil proposed in the separate thread - also a judgement of
release manager can be used to see whether some additional testing is
necessary.
And with single email and Github Issue we can very efficiently
organise the users to test their changes (even automate it).
I have shown during the last two weeks that this is not only doable,
but that our users and contributors generally respond very well to
such requests.
Those contributors await their changes to be released and they also
want to make sure it is tested before they get to the hands of the
users.
J.
On Tue, Mar 9, 2021 at 12:40 PM Kaxil Naik <[email protected]
<mailto:[email protected]>> wrote:
+1 -- I think users can pick and choose which providers to test and
vote on. And since each vote would be in a separate email, a single
provider bug in a particular provider won't derail conversation.
Regards,
Kaxil
On Tue, Mar 9, 2021, 11:38 Kaxil Naik <[email protected]
<mailto:[email protected]>> wrote:
+1
On Sat, Mar 6, 2021, 21:33 Ash Berlin-Taylor <[email protected]
<mailto:[email protected]>> wrote:
Just for absolutely clarity here: my -1 has nothing to do with the
vote on the release, merely against adopting the proposed policy
after this release/current set of votes.
-a
On 4 March 2021 11:04:13 GMT, Ash Berlin-Taylor <[email protected]
<mailto:[email protected]>> wrote:
(Split out proposal from voting thread here
<https://lists.apache.org/thread.html/r47daa0e594eb01c8196a96b62c40d5282b25c41ca1520781934e9236%40%3Cdev.airflow.apache.org%3E>.)
I don't like the idea of changing what a vote means mid way
through. To take an extreme case:
"+1 if you like ice cream"
*people vote*
"Vote now changed to +1 if you kick dogs".
Now clearly that is not what you are proposing, but changing what
the vote is on at all opens a slippery slope and I'd rather we
didn't continue/codify this precedent.
*This gets a -1 from me on adopting this part of the proposal.*
Not changing the vote is also why I favour many smaller
releases/votes: a problem with one doesn't block the others, and
additional I feel it is easier for non-core contributors to get
involved and test just the provider they are interested in,
without feeling daunted that they would have to test all the rest
to cast their vote.
Much of the release process is already automated, up-to-and
including sending emails (dev/send_email.py -- may need tweaking
for provider release votes.) so this is can be achieved with
little extra work to the release manager.
To be clear, I do not have a problem with batch releases and
batch votes (even if it is not my first choice, and I would
rather N parallel votes as default.)
-ash
On Wed, 3 Mar, 2021 at 22:44, Jarek Potiuk <[email protected]
<mailto:[email protected]>> wrote:
I just wanted to use the opportunity to describe the current
process for deciding providers, because apparently not everyone
is aware that we already have an established and documented
process for provider's releases that was discussed on the
devlist and it is documented in our release process description.
Possibly we will extract it into a separate policy and maybe
discuss some aspects of it (the discussion was raised today at
the dev call of ours but I wanted to make sure that we are all
referring to the same "starting point" and the "process" I based
my recent actions on.
*1) Batching the providers as the default*
The decision on when to release and particularly preference for
releasing providers in batches is described in the
<https://github.com/apache/airflow/blob/master/dev/README_RELEASE_PROVIDER_PACKAGES.md#decide-when-to-release>
> Decide when to release
> You can release provider packages separately from the main
Airflow on an ad-hoc basis, whenever we find that
a given provider needs to be released - due to new features or
due to bug fixes
You can release each provider package separately, but due to
voting and release overhead we try to group
releases of provider packages together.
*2) Possibility of excluding certain packages from the release.*
The possibility of excluding certain packages which (for
whatever reason) we decide to remove at the discretion of
release manager is described here:
https://github.com/apache/airflow/blob/master/dev/README_RELEASE_PROVIDER_PACKAGES.md#prepare-voting-email-for-providers-release-candidate
<https://github.com/apache/airflow/blob/master/dev/README_RELEASE_PROVIDER_PACKAGESmd#prepare-voting-email-for-providers-release-candidate>
Prepare voting email for Providers release candidate
...
> Due to the nature of packages, not all packages have to be
released as convenience packages in the final release. During
the voting process the voting PMCs might decide to exclude
certain packages from the release if some critical problems have
been found in some packages.
And the process of removing it is part of the described release
process:
> In case you decided to remove some of the packages. remove
them from dist folder now:
> ls dist/*<provider>*
> rm dist/*<provider>*
The issue of excluding certain packages have been discussed in
this thread in the mailing list :
<https://lists.apache.org/thread.html/rc620a8a503cc7b14850c0a2a1fca1f6051d08a7e3e6d2cbdeb691dda%40%3Cdev.airflow.apache.org%3E>
- where we had a -1 veto from a PMC member on the whole batch
of providers where we found a cncf.kubrenetes and google
providers had critical problems.
We discussed it then and the two PMC members proposed a solution
that was not objected to by anyone in the VOTE thread - to
remove the packages from the batch.
I continued this in the continuation of the voting thread
<https://lists.apache.org/thread.html/r752c5d5171de4ff626663d30e1d50c4b0d2994f66bf8918d816dabd8%40%3Cdev.airflow.apache.org%3E>
with the following message which specifically pointed out to my
proposal where I specifically linked to the message above and
asked for comments.
> As discussed before: -1 on a single provider does not
invalidate the whole
vote (from
<https://github.com/apache/airflow/tree/master/dev#vote-and-verify-the-backport-providers-release-candidate>
):
> "Due to the nature of backport packages, not all packages have
to be
released as convenience packages in the final release. During
the voting
process the voting PMCs might decide to exclude certain packages
from the
release if some critical problems have been found in some
packages."
> We will merge the fix and most likely release a new google
package right
after this one. Looking at the super-localized problem here my
current
decision will be to release 2020.10.29 'google package" -
together with
other packages and release 2020.11.01 (or smth) - but only the
google one -
right after we merge the fix.
> Any comments to that?
--
+48 660 796 129
