Based on the experiment I've done I'd update it to make it similar to the follow up email I sent here: https://lists.apache.org/thread.html/reac8871d8ad55d5c0c33819e8a10b0c38cba09297bde84a89adb102c%40%3Cdev.airflow.apache.org%3E My goal is to streamline it - so that neither release manager, nor PMCs, nor users spend too much unnecessary overhead time for the process - automation of information and grouping is key to that as this is something we will continue doing for many months ahead - every month repeating the mantra.
* Providers list with information if this is a bugfix or feature ( I assume we could get away with doc updates with my proposal of updating documentation only, not packages) * List of people and links to #PRs of the changes in each provider * if applicable, assessment of the release manager. Something along the lines of (wording can be probably improved): ------------------------- Please respond with one of: * +1/-1/0 [all] : if you vote on all packages * +1/-1/0 [ package1 package2 ]: if you vote on selected packages * +1/-1/0 [ all but package1 package2 ]: if you want to exclude certain packages from the vote Here, the list of packages we are voting on. You can cast your vote individually for each package, "bulk" vote on all the providers, or specifically exclude providers from your vote: * [ ] amazon 1.2.0 * [ ] google 3.0.0 * [ ] dingding 1.1.3 * [ ] datadog 2.0.4 ..... The assessment of the release manager is that amazon and google needs extensive testing, dingding, datadog contain trivial fixes that are likely to be OK without extensive testing. The current status of the provider tests is kept in https://github.com/apache/airflow/issues/14670 Details of the changes: * amazon 1.2.0 Features: * #1234 - this by @ashb * #1223 - that by @kaxil Bugfixes: * #1234 - this by @potiuk * #1244 that * google .... Breaking changes: * ,,,,, --------------------------- J. On Wed, Mar 10, 2021 at 10:16 AM Ash Berlin-Taylor <a...@apache.org> wrote: > Our approach of having 60+ subpackages is I think fairly unique in ASF > terms, where most projects just have one package that they release, so the > ASF guidance doesn't consider this situation -- we are on our own to come > up with something that works for us. > > The only other project I know that has as many sub-packages is Maven, and > they do one email = one package = one vote > https://lists.apache.org/list.html?d...@maven.apache.org:gte=1d:VOTE (they > don't batch either) > > For example these are started on the same day > > > https://lists.apache.org/thread.html/r92026b74d0c32b2d3dc06794ec071375d6a135057b41b13359263003%40%3Cdev.maven.apache.org%3E > > https://lists.apache.org/thread.html/r6cd5e53070bc033fae07d3e236c7c0cbdf956da358759a46919821ac%40%3Cdev.maven.apache.org%3E > > For me what we are silently (and wrongly) assuming 1 mail = 1 VOTE. But I > do not believe anywhere in all ASF documentation there is such a > requirement. > > > My counter argument to this point: the ASF guidance talks about "Votes on > whether *a* package is ready to be released" (emphasis mine). But > ultimately I think the guidance is just that -- guidance and we should just > come up with and write down how we want to operate here. > > Could you give an snippet/example of what 20 votes in one email would look > like? Are you thinking of the same email we send right now or something > different? > > > On Wed, 10 Mar, 2021 at 04:02, Jarek Potiuk <ja...@potiuk.com> wrote: > > TL;DR; My very, very strong preference goes for one email which is > effectively N votes for N packages we release. I actually think we are not > able to efficiently follow up and manage 1-vote for email for ~20 or so > packages we are going to release every month. > > I think we should just refer back to what ASF wrote about it. > https://www.apache.org/foundation/voting.html > > > VOTES ON PACKAGE RELEASES > > Votes on whether a package is ready to be released use majority approval > -- i.e. at least three PMC members must vote affirmatively for release, and > there must be more positive than negative votes. Releases may not be > vetoed. Generally the community will cancel the release vote if anyone > identifies serious problems, but in most cases the ultimate decision, lies > with the individual serving as release manager. The specifics of the > process may vary from project to project, but the 'minimum > > quorum of three +1 votes' rule is universal. > > "Generally the community will cancel the release vote if anyone identifies > serious problems, but in most cases the ultimate decision, lies with the > individual serving as release manager." - but as I read it - it's NOT the > PMC, but the community decides when to cancel release and the ultimate > decision lies in the hands of the release manager. > This has happened in the past with Airflow many times, that the release > manager usually Kaxil decided on canceling the release vote. > > And yes - there is no mentioning of partial "canceling". But, this does > not mean that there MUST be a single mail per vote either. I think we can > have a single email and a N votes that cover all packages in a single > email. And you can vote in bulk on all packages. And some users might vote > only on some of the packages. For me this is purely for the efficiency of > process and PMCs responding to the votes. > > Imagine we have 20 packages to send regularly (guesstimate), following up > 20 messages to vote will be quite overwhelming - borderline ridiculous in > fact. I am talking about following up and counting, not sending. Sending is > easy. > We often have difficulty chasing PMCs to respond to one email, and the job > of tracking how many responses were sent to each of the 20 emails and > following up on each of them will make the release manager's life > miserable. > First two weeks after sending votes, the release manager will have to > chase 20 emails, follow up, and count the votes for every single release. I > would certainly reject such a task as a release manager. It is not easy to > automate at all and is a lot of very stupid and tim-consuming overhead on > top of the work done already by release manager. > > I do not really think the analogy of ice cream and dogs is appropriate (at > all). It's misleading actually. We are not voting on different things. It's > all about cancelling some votes out of many that we agreed to in bulk at > most. But not about changing the votes. This can already be done if the > community raises an issue - with the ultimate decision of the release > manager - so nothing changes here vs. the current Apache voting process. > For me it's only the question about granularity of the votes in one email > and whether we can cast a "bulk" N votes by single +1. I think we can. > > For me what we are silently (and wrongly) assuming 1 mail = 1 VOTE. But I > do not believe anywhere in all ASF documentation there is such a > requirement. We have example "templates" for emails but I do not know if > anywhere this 1-1 relationship (email = Vote) is mentioned. I'd love to see > it if it is. For me as long as it is clear what we are voting on and what > eventually gets cancelled and we summaarise it in the "RESULT" email - we > should be good with having N votes in a single email. > > So for me "bulk" voting looks as follows: > > 1) we send 20 packages and start 20 votes in one email. > 2) Anyone - user, PMC could respond +1 (for all) or +1 (some) or +1 (- > this packages I tested which do not work). > 3) Community (ultimately release manager) decides which of those votes to > cancel - if any (providing that all of the votes have +3 votes from PMC > members). > 4) Releasing packages can still be done in waves - if we see after 3 days > that we can to release 10 out of the 20 package (but 10 has just +2 PMC > votes) - the release manager can still release the 10 and continue voting > on the remaining 10 (and even eventually cancel them). > > Also a few words about testing those 20 packages. > > Regarding testing - yes the packages MUST be tested by PMC members. This > is a must. But testing means in this case - bulk installing the packages on > top of airflow and see if the providers have been installed and shown in > `airlow providers list` with the correct version. I believe this is more > than enough to fulfill the "test" criteria when paired with user testing of > new/big functionalities. > This is (or should be - I followed it for sure) already part of the > process (alongside signing/checksum verification). And with svn checkout > where you can pull the whole folder to your machine, this can be done > rather quickly. > As Kaxil proposed in the separate thread - also a judgement of release > manager can be used to see whether some additional testing is necessary. > And with single email and Github Issue we can very efficiently organise > the users to test their changes (even automate it). > I have shown during the last two weeks that this is not only doable, but > that our users and contributors generally respond very well to such > requests. > Those contributors await their changes to be released and they also want > to make sure it is tested before they get to the hands of the users. > > J. > > > > > On Tue, Mar 9, 2021 at 12:40 PM Kaxil Naik <kaxiln...@gmail.com> wrote: > >> +1 -- I think users can pick and choose which providers to test and vote >> on. And since each vote would be in a separate email, a single provider bug >> in a particular provider won't derail conversation. >> >> Regards, >> Kaxil >> >> On Tue, Mar 9, 2021, 11:38 Kaxil Naik <kaxiln...@gmail.com> wrote: >> >>> +1 >>> >>> On Sat, Mar 6, 2021, 21:33 Ash Berlin-Taylor <a...@apache.org> wrote: >>> >>>> Just for absolutely clarity here: my -1 has nothing to do with the vote >>>> on the release, merely against adopting the proposed policy after this >>>> release/current set of votes. >>>> >>>> -a >>>> >>>> On 4 March 2021 11:04:13 GMT, Ash Berlin-Taylor <a...@apache.org> wrote: >>>>> >>>>> (Split out proposal from voting thread here >>>>> https://lists.apache.org/thread.html/r47daa0e594eb01c8196a96b62c40d5282b25c41ca1520781934e9236%40%3Cdev.airflow.apache.org%3E >>>>> .) >>>>> >>>>> I don't like the idea of changing what a vote means mid way through. >>>>> To take an extreme case: >>>>> >>>>> "+1 if you like ice cream" >>>>> >>>>> *people vote* >>>>> >>>>> "Vote now changed to +1 if you kick dogs". >>>>> >>>>> Now clearly that is not what you are proposing, but changing what the >>>>> vote is on at all opens a slippery slope and I'd rather we didn't >>>>> continue/codify this precedent. >>>>> >>>>> *This gets a -1 from me on adopting this part of the proposal.* >>>>> >>>>> Not changing the vote is also why I favour many smaller >>>>> releases/votes: a problem with one doesn't block the others, and >>>>> additional >>>>> I feel it is easier for non-core contributors to get involved and test >>>>> just >>>>> the provider they are interested in, without feeling daunted that they >>>>> would have to test all the rest to cast their vote. >>>>> >>>>> Much of the release process is already automated, up-to-and including >>>>> sending emails (dev/send_email.py -- may need tweaking for provider >>>>> release >>>>> votes.) so this is can be achieved with little extra work to the release >>>>> manager. >>>>> >>>>> To be clear, I do not have a problem with batch releases and batch >>>>> votes (even if it is not my first choice, and I would rather N parallel >>>>> votes as default.) >>>>> >>>>> -ash >>>>> >>>>> >>>>> On Wed, 3 Mar, 2021 at 22:44, Jarek Potiuk <ja...@potiuk.com> wrote: >>>>> >>>>> I just wanted to use the opportunity to describe the current process >>>>> for deciding providers, because apparently not everyone is aware that we >>>>> already have an established and documented process for provider's releases >>>>> that was discussed on the devlist and it is documented in our release >>>>> process description. >>>>> >>>>> Possibly we will extract it into a separate policy and maybe discuss >>>>> some aspects of it (the discussion was raised today at the dev call of >>>>> ours >>>>> but I wanted to make sure that we are all referring to the same "starting >>>>> point" and the "process" I based my recent actions on. >>>>> >>>>> *1) Batching the providers as the default* >>>>> >>>>> The decision on when to release and particularly preference for >>>>> releasing providers in batches is described in the >>>>> https://github.com/apache/airflow/blob/master/dev/README_RELEASE_PROVIDER_PACKAGES.md#decide-when-to-release >>>>> >>>>> >>>>> > Decide when to release >>>>> > You can release provider packages separately from the main Airflow >>>>> on an ad-hoc basis, whenever we find that >>>>> a given provider needs to be released - due to new features or due to >>>>> bug fixes >>>>> You can release each provider package separately, but due to voting >>>>> and release overhead we try to group >>>>> releases of provider packages together. >>>>> >>>>> *2) Possibility of excluding certain packages from the release.* >>>>> >>>>> The possibility of excluding certain packages which (for whatever >>>>> reason) we decide to remove at the discretion of release manager is >>>>> described here: >>>>> https://github.com/apache/airflow/blob/master/dev/README_RELEASE_PROVIDER_PACKAGES.md#prepare-voting-email-for-providers-release-candidate >>>>> <https://github.com/apache/airflow/blob/master/dev/README_RELEASE_PROVIDER_PACKAGESmd#prepare-voting-email-for-providers-release-candidate> >>>>> >>>>> Prepare voting email for Providers release candidate >>>>> ... >>>>> >>>>> > Due to the nature of packages, not all packages have to be released >>>>> as convenience packages in the final release. During the voting process >>>>> the >>>>> voting PMCs might decide to exclude certain packages from the release if >>>>> some critical problems have been found in some packages. >>>>> >>>>> And the process of removing it is part of the described release >>>>> process: >>>>> >>>>> > In case you decided to remove some of the packages. remove them from >>>>> dist folder now: >>>>> >>>>> > ls dist/*<provider>* >>>>> > rm dist/*<provider>* >>>>> >>>>> The issue of excluding certain packages have been discussed in this >>>>> thread in the mailing list : >>>>> https://lists.apache.org/thread.html/rc620a8a503cc7b14850c0a2a1fca1f6051d08a7e3e6d2cbdeb691dda%40%3Cdev.airflow.apache.org%3E >>>>> - where we had a -1 veto from a PMC member on the whole batch of providers >>>>> where we found a cncf.kubrenetes and google providers had critical >>>>> problems. >>>>> >>>>> We discussed it then and the two PMC members proposed a solution that >>>>> was not objected to by anyone in the VOTE thread - to remove the packages >>>>> from the batch. >>>>> >>>>> I continued this in the continuation of the voting thread >>>>> https://lists.apache.org/thread.html/r752c5d5171de4ff626663d30e1d50c4b0d2994f66bf8918d816dabd8%40%3Cdev.airflow.apache.org%3E >>>>> with the following message which specifically pointed out to my proposal >>>>> where I specifically linked to the message above and asked for comments. >>>>> >>>>> > As discussed before: -1 on a single provider does not invalidate the >>>>> whole >>>>> vote (from >>>>> >>>>> https://github.com/apache/airflow/tree/master/dev#vote-and-verify-the-backport-providers-release-candidate >>>>> ): >>>>> >>>>> > "Due to the nature of backport packages, not all packages have to be >>>>> released as convenience packages in the final release. During the >>>>> voting >>>>> process the voting PMCs might decide to exclude certain packages from >>>>> the >>>>> release if some critical problems have been found in some packages." >>>>> >>>>> > We will merge the fix and most likely release a new google package >>>>> right >>>>> after this one. Looking at the super-localized problem here my current >>>>> decision will be to release 2020.10.29 'google package" - together with >>>>> other packages and release 2020.11.01 (or smth) - but only the google >>>>> one - >>>>> right after we merge the fix. >>>>> >>>>> > Any comments to that? >>>>> >>>>> >>>>> >>>>> > > -- > +48 660 796 129 > > -- +48 660 796 129
