I added the point about logging to the AIP. I'd love to hear more thoughts on how the SEMVER process might work for those numerous packages - Ash I'd love to understand what scheme you think is the best.
One more comment for my options - I really like the point that Vikram made about the "lack" of the relation between the provider packages and "airflow core". So I really like the idea (no matter which one we choose eventually) of prefixing the version with 2.*. J. On Mon, Sep 14, 2020 at 12:01 PM Jarek Potiuk <[email protected]> wrote: > >> > We have to make sure that we have no dependencies core -> providers >> >> How do we handle writing logs to S3/GCS/Azure Blob storage, which >> depends on the hook from the provider to work? >> > > Good point. I will make sure that I address it in the PR as well. I think > those should only work when > the corresponding package his installed. And raise a warning otherwise. > > >> >> > Versioning proposal is CALVER with MAJOR AIRFLOW prefix (2.YYYY.MM.DD) >> > for all 2.* Airflow releases. In the future 3.YYYY.MM.DD for all 3.* >> > releases. Backwards incompatibility is only allowed between MAJOR >> > Airflow Versions >> >> I'd personally prefer use to use SemVar to allow us to make the freedom >> to make backwards-incompatible changes to a provider (within the same >> major Airflow release). This is what HashiCorp have done with their >> terraform providers modules. >> > > This is detail that we can still decide on independently on voting. But in > my opinion this > one is super difficult (procedurally and regarding involvement of release > management time > and effort needed) if we want to release each provider separately. > > But I am open to it as well if you help to define how SEMVER > implementation should look like and > if we come up that does not require a heavy overhead of release management > that cannot > be automated. > > What is your proposal for the implementation of SEMVER in this case? > > I can see the following options (but if you have other proposal - I am > happy to hear them): > > a) 2.0.N for all providers released in 2.0. But then according to semver > those > would be just bugfixes and no new features added) so technically it's not > really SEMVER - > it's basically the same as 2 + CALVER but with incrementing number > instead of date. > > b) 2.0.X.Y incrementing X always when new features are added and Y's when > there are bugfixes? > > c) 2.X.Y for all 2.* providers with X increasing when there are new > features and Ys when there are bugfixes. > > I think, if we go for the b) c) who, when (and based on what criteria) > will be deciding > whether X or Y should be increased for those 60+operators? Would that be > contributors committing the code or release manager releasing those > packages? > Note that the latter literally requires to review those all packages and > decide on a > case-by-case basis. I would never want to do it personally on regular > basis. > > What do you think Ash? Which option do you propose? > > J. > > > >> -ash >> >> On Sep 13 2020, at 9:28 pm, Jarek Potiuk <[email protected]> >> wrote: >> >> > Hello Everyone, >> > >> > Last week, at the Airflow 2.0 meeting the people involved made a >> decision >> > that we are releasing Airlow 2.0 as a set of separate "core" and >> > "providers" packages - similarly to the 1.10 "backport providers" >> packages. >> > >> > This decision effectively implements the "spirit" of the AIP-8 >> > proposed by >> > Tim Swast in January 2019. It's not an exact implementation - we've >> learned >> > a lot since the original proposal and the final implementation is >> slightly >> > different - rather than "multiple" repositories we stay with the >> mono-repo >> > approach, but we aim to achieve many of the goals targeted by the AIP-8. >> > >> > I updated the original AIP-8 >> > < >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303 >> > >> > to reflect those changes. >> > >> > This is the formal Vote for this proposal. It follows the "vote on code >> > modification" policy of the ASF: >> > >> https://www.apache.org/foundation/voting.html#votes-on-code-modification >> > >> > We need three +1 votes for the proposal to proceed. All committers >> > have a >> > binding vote. >> > >> > The vote will last 72 hours, which means that it ends on Wed 16th Sep >> 2020, >> > 10:30 PM CEST >> > >> > Consider this my binding +1. >> > >> > J >> > >> > -- >> > Jarek Potiuk >> > Polidea | Principal Software Engineer >> > >> > M: +48 660 796 129 >> > >> > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
