Thanks @Jarek Potiuk <ja...@potiuk.com>! I vaguely knew the process but not in such detail, thanks for putting it together in this email. I will visit the document https://github.com/apache/airflow?tab=readme-ov-file#what-goes-into-the-next-release and if I find any clarifications, I will send it across as a PR.
One suggestion: what if we put it together in a document and put it on the airflow website under https://airflow.apache.org/community/ where we mention "how we release"? Might be a bit too much given the context people will have while visiting the airflow website.. Thanks & Regards, Amogh Desai On Wed, Jan 17, 2024 at 6:13 PM Jarek Potiuk <ja...@potiuk.com> wrote: > I separate it out, because it seems that despite the efforts to explain and > document how our releases work It's not clear even for the PMC chair, so > likely it warrants a separate thread - also it will be easier to find it in > the archives this way. > > I think this is an important topic that all maintainers should be aware of, > so let me take a task to explain it in a longer email (and separate > thread). > > I created it in a form of FAQ, because it seems that similar questions > might be asked by others. > > *Do we have a process defined here?* > > Answering Bolke's question - who was apparently confused what our process > is: > > > I see that some improvements to Airflow.io (two weeks ago) were not > included and some provider updates neither. Haven't checked anything else > yet. > > Apparently there is some confusion about the process, but yes we have a > well defined and well tested (pretty much 4 years now) process that we > follow., We follow it since I remember actually - it's been also done the > same in 1.10 - but with some variations, Likely we do it the same way since > the beginning of 2.0, but it has been refined and improved over time - by > those who volunteered their time in the release process (a lot of ad-hoc > discussion have been traditionally happening in #release-management slack > channel) and as of few months ago we even documented it (It was in November > 2023) - with this PR https://github.com/apache/airflow/pull/35245 > > It is currently described in a prominent place in our most important (and > over the last year or so the README, we made the README pretty short and > contains only super-important information on how Airflow is developed) > README.md under *"What goes into the next release?"* chapter: > > > https://github.com/apache/airflow?tab=readme-ov-file#what-goes-into-the-next-release > . > > *How does the selection process for cherry-picking work?* > > In short (and this is the most important thing that every maintainer should > be aware of): *those maintainers who think that issue should be included > should mark it with the next (in this case 2.8.1) milestone*. It's up to > individual maintainers who want to include certain changes to take care > about it and mark the issues they think are bug fixes, to go into the next > release > > This is the only thing that the maintainer has to do to get the PR proposed > to be considered in the next patchlevel release. Sometimes - if > controversial - maintainers discuss the proposals in #release-management > channel, sometimes in #development or in the PR itself (especially if the > release manager decides to not include it and changes the milestone (and > explains why). > > *What's the release manager's role ?* > > Release manager's job is purely mechanical (as also mandated by the Apache > Software Foundation release manager role description) to assess cherry-pick > ability of those changes. Release manager - at the sole discretion and > individual decision (this is the only place in the whole ASF setup where a > single person has such power to make individual decisions) can reject some > of those who other maintainers think should be included. But the release > manager on his own does not make proposals on what should be included. > > *Is this process following the ASF rules?* > > I believe yes, The release manager's role is nicely described here: > https://infra.apache.org/release-publishing.html#releasemanager. And there > is a far more complete description here that describes the whole process > https://www.apache.org/legal/release-policy.html#management - also > mentioning that it's the PMC's responsibility (and particularly PMC > chair's) to adhere to the process. > > *What's the role of individual maintainers?* > > The role of maintainers (collectively) to propose things for the next > release. In our case it happens with setting the milestone on a PR. > > *When proposed PRs are rejected?* > > There are various reasons to reject those - if too complex to cherry-pick > or when the release manager assesses it's a new feature, not a bugfix. > Essentially (according to semver) when it comes to the user-facing changes, > the PATCHLEVEL release should contain only bugfixes. and may contain docs > changes if they are fixing/improving docs (not about new features) and also > environment/build script changes (so non-user-facing changes) as they are > pretty much always needed to keep the things nicely building - those are > usually skipped from the changelog as non-user facing). > > *Why are provider changes not cherry-picked?* > > In our case - basically none of the provider changes are cherry-picked - > unless they are needed to make the builds work well (sometimes happen). > Providers are ALWAYS released from the latest main code, not from the > v2-8-stable branch. In fact all the tests and ci checks for providers are > skipped in the non-main (v2* branches). So yes - not seeing provider > changes cherry-picked is absolutely expected. > > *Do we skip over some changes when releasing a patchlevel release? What's > the purpose of patch-level releases?* > > Answering Bolke's question: > > > Is that intentional? Ie. is that the purpose of this release. Other > big(ger) and more recent changes have been included hence asking. > > The purpose of that release is as described in SemVer - to give the users > bugfix-only release that has no new features. Of course it's sometimes > debatable whether changes are features/bugfixes, but we usually use > #release-management to quickly chat about it, and eventually the release > manager always makes a comment in the PR when the milestone is changed and > explains the reasoning. > > Skipping is not intentional because we never "skip" things when > cherry-picking, It's *reverse* - those maintainer who think that certain > bug fixes (or internal changes or sometimes even feature changes that we > classify really as "bugfix" SHOULD intentionally mark those PRs they want > with 2.8.1 (or basically next patch-level) to be *included. * So there is > no skipping, if maintainer did not deliberately mark PR as upcoming > milestone, it will just not be included > > *Where do some maintainers know about it **from ? * > > Because the maintainers who actively participate - either acting as release > managers (those who raised their hand and acted as release managers > generally speaking) and those who wanted their changes to be part of the > past releases have been doing it for years. For years this has been simply > followed and discussed in #release-management channel and #development (and > in devlist whenever there were new releases) and generally the maintainers > who took part in those discussions and release process are aware of that - > also many maintainers know the process as it "soaked" in when they were > just watching. > > However recently we've made an attempt to document it (the PR above). So > you could learn it by reading it (even if it does not have the whole > context above). > > *Why do some people not know about it?* > > Not sure. Maybe because they were not interested and never asked? Maybe > because there was never a long email about it at our devlist, or the > documentation about it in README was too succinct? > > Or maybe we need a better way of communicating it - I am not sure. But I > hope this email will clarify a lot of it, and will prove valuable when > searching in devlist. > > Maybe even someone who manages to read it all will update the README > description of ours to explain it better :) and maybe create a nice FAQ > that we can put in our docs? > > I really hope this mail - even if long - will make people a bit more aware > of the process, and if someone has an idea how the "collective" awareness > can be improved - I think it's a good idea to send PR or email (or maybe > even record a video :) ??) that will be a better way of communicating it. > > J. > > > On Wed, Jan 17, 2024 at 9:03 AM Bolke de Bruin <bdbr...@gmail.com> wrote: > > > Just checking: > > > > I see that some improvements to Airflow.io (two weeks ago) were not > > included and some provider updates neither. Haven't checked anything > else > > yet. > > > > Is that intentional? Ie. is that the purpose of this release. Other > > big(ger) and more recent changes have been included hence asking. > > > > B. > > > > Sent from my iPhone > > > > > On 16 Jan 2024, at 20:18, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > > +1 (binding): Checked all my changes, I ran airflow in a few > > combinations > > > (MySQL / Postgres Local/Celery executor. It looks and works well - run > a > > > few dags and navigated through a number of screens. Checked licences, > > > signatures, checksums, performed a "reproducible build" check and it > > worked > > > (with a small glitch explained below). > > > > > > BTW. The new "hatchling-built" package looks good and it nicely > installs > > > airflow + extras (and it has a far better and cleaner set of extras - > > > finally you can reproducibly install airflow with `all` extra if you > > > ***REALLY*** want :). > > > > > > Reproducibility (also for other PMC members doing the check): I found a > > > small glitch in the "reproducible" part of verifying the packages. > While > > > wheel and sdist packages are exactly the same binary-wise, the > > > source-tarball was binarry different for me. I decompressed it and > > compared > > > the content - they are identical - but there is one difference which I > > > overlooked - the group permissions for files in Ephraim's tarball are > > > different from mine. I have totally forgotten about the fact that umask > > > might set different group/other permissions when checking out the files > > > from git. Fix will be coming shortly - in the meantime I recommend > anyone > > > who will be doing the comparison to uncompress both tarballs and > compare > > > the contents with `diff -r`. > > > > > > > > > > > >> On Tue, Jan 16, 2024 at 11:30 AM Ephraim Anierobi < > > >> ephraimanier...@apache.org> wrote: > > >> > > >> Hey fellow Airflowers, > > >> > > >> I have cut Airflow 2.8.1rc1. This email is calling a vote on the > > release, > > >> which will last at least 72 hours, from Tuesday, January 16, 2024 at > > 10:30 > > >> am UTC > > >> until Friday, January 19, 2024, at 10:30 am UTC > > >> < > > >> > > > https://www.timeanddate.com/worldclock/fixedtime.html?msg=8&iso=20240119T1030&p1=1440 > > >>> , > > >> and until 3 binding +1 votes have been received. > > >> > > >> Status of testing of the release is kept at > > >> https://github.com/apache/airflow/issues/36808 > > >> > > >> Consider this my (binding) +1. > > >> > > >> Airflow 2.8.1rc1 is available at: > > >> https://dist.apache.org/repos/dist/dev/airflow/2.8.1rc1/ > > >> > > >> *apache-airflow-2.8.1-source.tar.gz* is a source release that comes > with > > >> INSTALL instructions. > > >> *apache-airflow-2.8.1.tar.gz* is the binary Python "sdist" release. > > >> *apache_airflow-2.8.1-py3-none-any.whl* is the binary Python wheel > > "binary" > > >> release. > > >> > > >> Public keys are available at: > > >> https://dist.apache.org/repos/dist/release/airflow/KEYS > > >> > > >> Please vote accordingly: > > >> > > >> [ ] +1 approve > > >> [ ] +0 no opinion > > >> [ ] -1 disapprove with the reason > > >> > > >> Only votes from PMC members are binding, but all members of the > > community > > >> are encouraged to test the release and vote with "(non-binding)". > > >> > > >> The test procedure for PMC members is described in: > > >> > > >> > > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-pmc-members > > >> > > >> The test procedure for and Contributors who would like to test this RC > > is > > >> described in: > > >> > > >> > > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-contributors > > >> > > >> > > >> Please note that the version number excludes the `rcX` string, so it's > > now > > >> simply 2.8.1. This will allow us to rename the artifact without > > modifying > > >> the artifact checksums when we actually release. > > >> > > >> Release Notes: > > >> https://github.com/apache/airflow/blob/2.8.1rc1/RELEASE_NOTES.rst > > >> > > >> Changes since 2.8.0: > > >> > > >> *Significant Changes* > > >> > > >> *Target version for core dependency ``pendulum`` package set to 3 > > >> (#36281).* > > >> Support for pendulum 2.1.2 will be saved for a while, presumably until > > the > > >> next feature version of Airflow. > > >> It is advised to upgrade user code to use pendulum 3 as soon as > > possible. > > >> > > >> *Airflow packaging specification follows modern Python packaging > > standards > > >> (#36537).* > > >> We standardized Airflow dependency configuration to follow latest > > >> development in Python packaging by > > >> using ``pyproject.toml``. Airflow is now compliant with those accepted > > >> PEPs: > > >> > > >> * `PEP-440 Version Identification and Dependency Specification < > > >> https://www.python.org/dev/peps/pep-0440/>`__ > > >> * `PEP-517 A build-system independent format for source trees < > > >> https://www.python.org/dev/peps/pep-0517/>`__ > > >> * `PEP-518 Specifying Minimum Build System Requirements for Python > > Projects > > >> <https://www.python.org/dev/peps/pep-0518/>`__ > > >> * `PEP-561 Distributing and Packaging Type Information < > > >> https://www.python.org/dev/peps/pep-0561/>`__ > > >> * `PEP-621 Storing project metadata in pyproject.toml < > > >> https://www.python.org/dev/peps/pep-0621/>`__ > > >> * `PEP-660 Editable installs for pyproject.toml based builds (wheel > > based) > > >> < > > >> https://www.python.org/dev/peps/pep-0660/>`__ > > >> * `PEP-685 Comparison of extra names for optional distribution > > dependencies > > >> <https://www.python.org/dev/peps/pep-0685/>`__ > > >> > > >> Also we implement multiple license files support coming from Draft, > not > > yet > > >> accepted (but supported by hatchling) PEP: > > >> * `PEP 639 Improving License Clarity with Better Package Metadata < > > >> https://peps.python.org/pep-0639/>`__ > > >> > > >> This has almost no noticeable impact on users if they are using modern > > >> Python packaging and development tools, generally > > >> speaking Airflow should behave as it did before when installing it > from > > >> PyPI and it should be much easier to install > > >> it for development purposes using ``pip install -e ".[devel]"``. > > >> > > >> The differences from the user side are: > > >> > > >> * Airflow extras now get extras normalized to ``-`` (following > PEP-685) > > >> instead of ``_`` and ``.`` > > >> (as it was before in some extras). When you install airflow with such > > >> extras (for example ``dbt.core`` or > > >> ``all_dbs``) you should use ``-`` instead of ``_`` and ``.``. > > >> > > >> In most modern tools this will work in backwards-compatible way, but > in > > >> some old version of those tools you might need to > > >> replace ``_`` and ``.`` with ``-``. You can also get warnings that the > > >> extra you are installing does not exist - but usually > > >> this warning is harmless and the extra is installed anyway. It is, > > however, > > >> recommended to change to use ``-`` in extras in your dependency > > >> specifications for all Airflow extras. > > >> > > >> * Released airflow package does not contain ``devel``, ``devel-*``, > > ``doc`` > > >> and ``doc-gen`` extras. > > >> Those extras are only available when you install Airflow from sources > > in > > >> ``--editable`` mode. This is > > >> because those extras are only used for development and documentation > > >> building purposes and are not needed > > >> when you install Airflow for production use. Those dependencies had > > >> unspecified and varying behaviour for > > >> released packages anyway and you were not supposed to use them in > > >> released packages. > > >> > > >> * The ``all`` and ``all-*`` extras were not always working correctly > > when > > >> installing Airflow using constraints > > >> because they were also considered as development-only dependencies. > > With > > >> this change, those dependencies are > > >> now properly handling constraints and they will install properly with > > >> constraints, pulling the right set > > >> of providers and dependencies when constraints are used. > > >> > > >> *Graphviz dependency is now an optional one, not required one > (#36647).* > > >> The ``graphviz`` dependency has been problematic as Airflow required > > >> dependency - especially for > > >> ARM-based installations. Graphviz packages require binary graphviz > > >> libraries - which is already a > > >> limitation, but they also require to install graphviz Python bindings > > to be > > >> build and installed. > > >> This does not work for older Linux installation but - more > importantly - > > >> when you try to install > > >> Graphviz libraries for Python 3.8, 3.9 for ARM M1 MacBooks, the > packages > > >> fail to install because > > >> Python bindings compilation for M1 can only work for Python 3.10+. > > >> > > >> This is not a breaking change technically - the CLIs to render the > DAGs > > is > > >> still there and IF you > > >> already have graphviz installed, it will continue working as it did > > before. > > >> The only problem when it > > >> does not work is where you do not have graphviz installed it will > raise > > an > > >> error and inform that you need it. > > >> > > >> Graphviz will remain to be installed for most users: > > >> > > >> * the Airflow Image will still contain graphviz library, because > > >> it is added there as extra > > >> * when previous version of Airflow has been installed already, then > > >> graphviz library is already installed there and Airflow will > > >> continue working as it did > > >> > > >> The only change will be a new installation of new version of Airflow > > from > > >> the scratch, where graphviz will > > >> need to be specified as extra or installed separately in order to > enable > > >> DAG rendering option. > > >> > > >> *Bug Fixes* > > >> - Fix airflow-scheduler exiting with code 0 on exceptions (#36800) > > >> - Fix Callback exception when a removed task is the last one in the > > >> ``taskinstance`` list (#36693) > > >> - Allow anonymous user edit/show resource when set > > >> ``AUTH_ROLE_PUBLIC=admin`` (#36750) > > >> - Better error message when sqlite URL uses relative path (#36774) > > >> - Explicit string cast required to force integer-type run_ids to be > > passed > > >> as strings instead of integers (#36756) > > >> - Add log lookup exception for empty ``op`` subtypes (#35536) > > >> - Remove unused index on task instance (#36737) > > >> - Fix check on subclass for ``typing.Union`` in > > ``_infer_multiple_outputs`` > > >> for Python 3.10+ (#36728) > > >> - Make sure ``multiple_outputs`` is inferred correctly even when using > > >> ``TypedDict`` (#36652) > > >> - Add back FAB constant in legacy security manager (#36719) > > >> - Fix AttributeError when using ``Dagrun.update_state`` (#36712) > > >> - Do not let ``EventsTimetable`` schedule past events if > > ``catchup=False`` > > >> (#36134) > > >> - Support encryption for triggers parameters (#36492) > > >> - Fix the type hint for ``tis_query`` in ``_process_executor_events`` > > >> (#36655) > > >> - Redirect to index when user does not have permission to access a > page > > >> (#36623) > > >> - Avoid using dict as default value in ``call_regular_interval`` > > (#36608) > > >> - Remove option to set a task instance to running state in UI (#36518) > > >> - Fix details tab not showing when using dynamic task mapping (#36522) > > >> - Raise error when ``DagRun`` fails while running ``dag test`` > (#36517) > > >> - Refactor ``_manage_executor_state`` by refreshing TIs in batch > > (#36502) > > >> - Add flask config: ``MAX_CONTENT_LENGTH`` (#36401) > > >> - Fix get_leaves calculation for teardown in nested group (#36456) > > >> - Stop serializing timezone-naive datetime to timezone-aware datetime > > with > > >> UTC tz (#36379) > > >> - Make ``kubernetes`` decorator type annotation consistent with > operator > > >> (#36405) > > >> - Fix Webserver returning 500 for POST requests to > ``api/dag/*/dagrun`` > > >> from anonymous user (#36275) > > >> - Fix the required access for get_variable endpoint (#36396) > > >> - Fix datetime reference in ``DAG.is_fixed_time_schedule`` (#36370) > > >> - Fix AirflowSkipException message raised by BashOperator (#36354) > > >> - Allow PythonVirtualenvOperator.skip_on_exit_code to be zero (#36361) > > >> - Increase width of execution_date input in trigger.html (#36278) > > >> - Fix logging for pausing DAG (#36182) > > >> - Stop deserializing pickle when enable_xcom_pickling is False > (#36255) > > >> - Check DAG read permission before accessing DAG code (#36257) > > >> - Enable mark task as failed/success always (#36254) > > >> - Create latest log dir symlink as relative link (#36019) > > >> - Fix Python-based decorators templating (#36103) > > >> > > >> *Miscellaneous* > > >> - Rename concurrency label to max active tasks (#36691) > > >> - Restore function scoped ``httpx`` import in file_task_handler for > > >> performance (#36753) > > >> - Add support of Pendulum 3 (#36281) > > >> - Standardize airflow build process and switch to Hatchling build > > backend > > >> (#36537) > > >> - Get rid of ``pyarrow-hotfix`` for ``CVE-2023-47248`` (#36697) > > >> - Make ``graphviz`` dependency optional (#36647) > > >> - Announce MSSQL support end in Airflow 2.9.0, add migration script > > hints > > >> (#36509) > > >> - Set min ``pandas`` dependency to 1.2.5 for all providers and airflow > > >> (#36698) > > >> - Bump follow-redirects from 1.15.3 to 1.15.4 in ``/airflow/www`` > > (#36700) > > >> - Provide the logger_name param to base hook in order to override the > > >> logger name (#36674) > > >> - Fix run type icon alignment with run type text (#36616) > > >> - Follow BaseHook connection fields method signature in FSHook > (#36444) > > >> - Remove redundant ``docker`` decorator type annotations (#36406) > > >> - Straighten typing in workday timetable (#36296) > > >> - Use ``batch_is_authorized_dag`` to check if user has permission to > > read > > >> DAGs (#36279) > > >> - Replace deprecated get_accessible_dag_ids and use get_readable_dags > in > > >> get_dag_warnings (#36256) > > >> > > >> *Doc Only Changes* > > >> - Metrics tagging documentation (#36627) > > >> - In docs use logical_date instead of deprecated execution_date > (#36654) > > >> - Add section about live-upgrading Airflow (#36637) > > >> - Replace ``numpy`` example with practical exercise demonstrating > > top-level > > >> code (#35097) > > >> - Improve and add more complete description in the architecture > diagrams > > >> (#36513) > > >> - Improve the error message displayed when there is a webserver error > > >> (#36570) > > >> - Update ``dags.rst`` with information on DAG pausing (#36540) > > >> - Update installation prerequisites after upgrading to Debian Bookworm > > >> (#36521) > > >> - Add description on the ways how users should approach DB monitoring > > >> (#36483) > > >> - Add branching based on mapped task group example to > > >> dynamic-task-mapping.rst (#36480) > > >> - Add further details to replacement documentation (#36485) > > >> - Use cards when describing priority weighting methods (#36411) > > >> - Update ``metrics.rst`` for param ``dagrun.schedule_delay`` (#36404) > > >> - Update admonitions in Python operator doc to reflect sentiment > > (#36340) > > >> - Improve audit_logs.rst (#36213) > > >> - Remove Redshift mention from the list of managed Postgres backends > > >> (#36217) > > >> > > >> Cheers, > > >> Ephraim > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > >
