I separate it out, because it seems that despite the efforts to explain and document how our releases work It's not clear even for the PMC chair, so likely it warrants a separate thread - also it will be easier to find it in the archives this way.
I think this is an important topic that all maintainers should be aware of, so let me take a task to explain it in a longer email (and separate thread). I created it in a form of FAQ, because it seems that similar questions might be asked by others. *Do we have a process defined here?* Answering Bolke's question - who was apparently confused what our process is: > I see that some improvements to Airflow.io (two weeks ago) were not included and some provider updates neither. Haven't checked anything else yet. Apparently there is some confusion about the process, but yes we have a well defined and well tested (pretty much 4 years now) process that we follow., We follow it since I remember actually - it's been also done the same in 1.10 - but with some variations, Likely we do it the same way since the beginning of 2.0, but it has been refined and improved over time - by those who volunteered their time in the release process (a lot of ad-hoc discussion have been traditionally happening in #release-management slack channel) and as of few months ago we even documented it (It was in November 2023) - with this PR https://github.com/apache/airflow/pull/35245 It is currently described in a prominent place in our most important (and over the last year or so the README, we made the README pretty short and contains only super-important information on how Airflow is developed) README.md under *"What goes into the next release?"* chapter: https://github.com/apache/airflow?tab=readme-ov-file#what-goes-into-the-next-release . *How does the selection process for cherry-picking work?* In short (and this is the most important thing that every maintainer should be aware of): *those maintainers who think that issue should be included should mark it with the next (in this case 2.8.1) milestone*. It's up to individual maintainers who want to include certain changes to take care about it and mark the issues they think are bug fixes, to go into the next release This is the only thing that the maintainer has to do to get the PR proposed to be considered in the next patchlevel release. Sometimes - if controversial - maintainers discuss the proposals in #release-management channel, sometimes in #development or in the PR itself (especially if the release manager decides to not include it and changes the milestone (and explains why). *What's the release manager's role ?* Release manager's job is purely mechanical (as also mandated by the Apache Software Foundation release manager role description) to assess cherry-pick ability of those changes. Release manager - at the sole discretion and individual decision (this is the only place in the whole ASF setup where a single person has such power to make individual decisions) can reject some of those who other maintainers think should be included. But the release manager on his own does not make proposals on what should be included. *Is this process following the ASF rules?* I believe yes, The release manager's role is nicely described here: https://infra.apache.org/release-publishing.html#releasemanager. And there is a far more complete description here that describes the whole process https://www.apache.org/legal/release-policy.html#management - also mentioning that it's the PMC's responsibility (and particularly PMC chair's) to adhere to the process. *What's the role of individual maintainers?* The role of maintainers (collectively) to propose things for the next release. In our case it happens with setting the milestone on a PR. *When proposed PRs are rejected?* There are various reasons to reject those - if too complex to cherry-pick or when the release manager assesses it's a new feature, not a bugfix. Essentially (according to semver) when it comes to the user-facing changes, the PATCHLEVEL release should contain only bugfixes. and may contain docs changes if they are fixing/improving docs (not about new features) and also environment/build script changes (so non-user-facing changes) as they are pretty much always needed to keep the things nicely building - those are usually skipped from the changelog as non-user facing). *Why are provider changes not cherry-picked?* In our case - basically none of the provider changes are cherry-picked - unless they are needed to make the builds work well (sometimes happen). Providers are ALWAYS released from the latest main code, not from the v2-8-stable branch. In fact all the tests and ci checks for providers are skipped in the non-main (v2* branches). So yes - not seeing provider changes cherry-picked is absolutely expected. *Do we skip over some changes when releasing a patchlevel release? What's the purpose of patch-level releases?* Answering Bolke's question: > Is that intentional? Ie. is that the purpose of this release. Other big(ger) and more recent changes have been included hence asking. The purpose of that release is as described in SemVer - to give the users bugfix-only release that has no new features. Of course it's sometimes debatable whether changes are features/bugfixes, but we usually use #release-management to quickly chat about it, and eventually the release manager always makes a comment in the PR when the milestone is changed and explains the reasoning. Skipping is not intentional because we never "skip" things when cherry-picking, It's *reverse* - those maintainer who think that certain bug fixes (or internal changes or sometimes even feature changes that we classify really as "bugfix" SHOULD intentionally mark those PRs they want with 2.8.1 (or basically next patch-level) to be *included. * So there is no skipping, if maintainer did not deliberately mark PR as upcoming milestone, it will just not be included *Where do some maintainers know about it **from ? * Because the maintainers who actively participate - either acting as release managers (those who raised their hand and acted as release managers generally speaking) and those who wanted their changes to be part of the past releases have been doing it for years. For years this has been simply followed and discussed in #release-management channel and #development (and in devlist whenever there were new releases) and generally the maintainers who took part in those discussions and release process are aware of that - also many maintainers know the process as it "soaked" in when they were just watching. However recently we've made an attempt to document it (the PR above). So you could learn it by reading it (even if it does not have the whole context above). *Why do some people not know about it?* Not sure. Maybe because they were not interested and never asked? Maybe because there was never a long email about it at our devlist, or the documentation about it in README was too succinct? Or maybe we need a better way of communicating it - I am not sure. But I hope this email will clarify a lot of it, and will prove valuable when searching in devlist. Maybe even someone who manages to read it all will update the README description of ours to explain it better :) and maybe create a nice FAQ that we can put in our docs? I really hope this mail - even if long - will make people a bit more aware of the process, and if someone has an idea how the "collective" awareness can be improved - I think it's a good idea to send PR or email (or maybe even record a video :) ??) that will be a better way of communicating it. J. On Wed, Jan 17, 2024 at 9:03 AM Bolke de Bruin <bdbr...@gmail.com> wrote: > Just checking: > > I see that some improvements to Airflow.io (two weeks ago) were not > included and some provider updates neither. Haven't checked anything else > yet. > > Is that intentional? Ie. is that the purpose of this release. Other > big(ger) and more recent changes have been included hence asking. > > B. > > Sent from my iPhone > > > On 16 Jan 2024, at 20:18, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > +1 (binding): Checked all my changes, I ran airflow in a few > combinations > > (MySQL / Postgres Local/Celery executor. It looks and works well - run a > > few dags and navigated through a number of screens. Checked licences, > > signatures, checksums, performed a "reproducible build" check and it > worked > > (with a small glitch explained below). > > > > BTW. The new "hatchling-built" package looks good and it nicely installs > > airflow + extras (and it has a far better and cleaner set of extras - > > finally you can reproducibly install airflow with `all` extra if you > > ***REALLY*** want :). > > > > Reproducibility (also for other PMC members doing the check): I found a > > small glitch in the "reproducible" part of verifying the packages. While > > wheel and sdist packages are exactly the same binary-wise, the > > source-tarball was binarry different for me. I decompressed it and > compared > > the content - they are identical - but there is one difference which I > > overlooked - the group permissions for files in Ephraim's tarball are > > different from mine. I have totally forgotten about the fact that umask > > might set different group/other permissions when checking out the files > > from git. Fix will be coming shortly - in the meantime I recommend anyone > > who will be doing the comparison to uncompress both tarballs and compare > > the contents with `diff -r`. > > > > > > > >> On Tue, Jan 16, 2024 at 11:30 AM Ephraim Anierobi < > >> ephraimanier...@apache.org> wrote: > >> > >> Hey fellow Airflowers, > >> > >> I have cut Airflow 2.8.1rc1. This email is calling a vote on the > release, > >> which will last at least 72 hours, from Tuesday, January 16, 2024 at > 10:30 > >> am UTC > >> until Friday, January 19, 2024, at 10:30 am UTC > >> < > >> > https://www.timeanddate.com/worldclock/fixedtime.html?msg=8&iso=20240119T1030&p1=1440 > >>> , > >> and until 3 binding +1 votes have been received. > >> > >> Status of testing of the release is kept at > >> https://github.com/apache/airflow/issues/36808 > >> > >> Consider this my (binding) +1. > >> > >> Airflow 2.8.1rc1 is available at: > >> https://dist.apache.org/repos/dist/dev/airflow/2.8.1rc1/ > >> > >> *apache-airflow-2.8.1-source.tar.gz* is a source release that comes with > >> INSTALL instructions. > >> *apache-airflow-2.8.1.tar.gz* is the binary Python "sdist" release. > >> *apache_airflow-2.8.1-py3-none-any.whl* is the binary Python wheel > "binary" > >> release. > >> > >> Public keys are available at: > >> https://dist.apache.org/repos/dist/release/airflow/KEYS > >> > >> Please vote accordingly: > >> > >> [ ] +1 approve > >> [ ] +0 no opinion > >> [ ] -1 disapprove with the reason > >> > >> Only votes from PMC members are binding, but all members of the > community > >> are encouraged to test the release and vote with "(non-binding)". > >> > >> The test procedure for PMC members is described in: > >> > >> > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-pmc-members > >> > >> The test procedure for and Contributors who would like to test this RC > is > >> described in: > >> > >> > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md\#verify-the-release-candidate-by-contributors > >> > >> > >> Please note that the version number excludes the `rcX` string, so it's > now > >> simply 2.8.1. This will allow us to rename the artifact without > modifying > >> the artifact checksums when we actually release. > >> > >> Release Notes: > >> https://github.com/apache/airflow/blob/2.8.1rc1/RELEASE_NOTES.rst > >> > >> Changes since 2.8.0: > >> > >> *Significant Changes* > >> > >> *Target version for core dependency ``pendulum`` package set to 3 > >> (#36281).* > >> Support for pendulum 2.1.2 will be saved for a while, presumably until > the > >> next feature version of Airflow. > >> It is advised to upgrade user code to use pendulum 3 as soon as > possible. > >> > >> *Airflow packaging specification follows modern Python packaging > standards > >> (#36537).* > >> We standardized Airflow dependency configuration to follow latest > >> development in Python packaging by > >> using ``pyproject.toml``. Airflow is now compliant with those accepted > >> PEPs: > >> > >> * `PEP-440 Version Identification and Dependency Specification < > >> https://www.python.org/dev/peps/pep-0440/>`__ > >> * `PEP-517 A build-system independent format for source trees < > >> https://www.python.org/dev/peps/pep-0517/>`__ > >> * `PEP-518 Specifying Minimum Build System Requirements for Python > Projects > >> <https://www.python.org/dev/peps/pep-0518/>`__ > >> * `PEP-561 Distributing and Packaging Type Information < > >> https://www.python.org/dev/peps/pep-0561/>`__ > >> * `PEP-621 Storing project metadata in pyproject.toml < > >> https://www.python.org/dev/peps/pep-0621/>`__ > >> * `PEP-660 Editable installs for pyproject.toml based builds (wheel > based) > >> < > >> https://www.python.org/dev/peps/pep-0660/>`__ > >> * `PEP-685 Comparison of extra names for optional distribution > dependencies > >> <https://www.python.org/dev/peps/pep-0685/>`__ > >> > >> Also we implement multiple license files support coming from Draft, not > yet > >> accepted (but supported by hatchling) PEP: > >> * `PEP 639 Improving License Clarity with Better Package Metadata < > >> https://peps.python.org/pep-0639/>`__ > >> > >> This has almost no noticeable impact on users if they are using modern > >> Python packaging and development tools, generally > >> speaking Airflow should behave as it did before when installing it from > >> PyPI and it should be much easier to install > >> it for development purposes using ``pip install -e ".[devel]"``. > >> > >> The differences from the user side are: > >> > >> * Airflow extras now get extras normalized to ``-`` (following PEP-685) > >> instead of ``_`` and ``.`` > >> (as it was before in some extras). When you install airflow with such > >> extras (for example ``dbt.core`` or > >> ``all_dbs``) you should use ``-`` instead of ``_`` and ``.``. > >> > >> In most modern tools this will work in backwards-compatible way, but in > >> some old version of those tools you might need to > >> replace ``_`` and ``.`` with ``-``. You can also get warnings that the > >> extra you are installing does not exist - but usually > >> this warning is harmless and the extra is installed anyway. It is, > however, > >> recommended to change to use ``-`` in extras in your dependency > >> specifications for all Airflow extras. > >> > >> * Released airflow package does not contain ``devel``, ``devel-*``, > ``doc`` > >> and ``doc-gen`` extras. > >> Those extras are only available when you install Airflow from sources > in > >> ``--editable`` mode. This is > >> because those extras are only used for development and documentation > >> building purposes and are not needed > >> when you install Airflow for production use. Those dependencies had > >> unspecified and varying behaviour for > >> released packages anyway and you were not supposed to use them in > >> released packages. > >> > >> * The ``all`` and ``all-*`` extras were not always working correctly > when > >> installing Airflow using constraints > >> because they were also considered as development-only dependencies. > With > >> this change, those dependencies are > >> now properly handling constraints and they will install properly with > >> constraints, pulling the right set > >> of providers and dependencies when constraints are used. > >> > >> *Graphviz dependency is now an optional one, not required one (#36647).* > >> The ``graphviz`` dependency has been problematic as Airflow required > >> dependency - especially for > >> ARM-based installations. Graphviz packages require binary graphviz > >> libraries - which is already a > >> limitation, but they also require to install graphviz Python bindings > to be > >> build and installed. > >> This does not work for older Linux installation but - more importantly - > >> when you try to install > >> Graphviz libraries for Python 3.8, 3.9 for ARM M1 MacBooks, the packages > >> fail to install because > >> Python bindings compilation for M1 can only work for Python 3.10+. > >> > >> This is not a breaking change technically - the CLIs to render the DAGs > is > >> still there and IF you > >> already have graphviz installed, it will continue working as it did > before. > >> The only problem when it > >> does not work is where you do not have graphviz installed it will raise > an > >> error and inform that you need it. > >> > >> Graphviz will remain to be installed for most users: > >> > >> * the Airflow Image will still contain graphviz library, because > >> it is added there as extra > >> * when previous version of Airflow has been installed already, then > >> graphviz library is already installed there and Airflow will > >> continue working as it did > >> > >> The only change will be a new installation of new version of Airflow > from > >> the scratch, where graphviz will > >> need to be specified as extra or installed separately in order to enable > >> DAG rendering option. > >> > >> *Bug Fixes* > >> - Fix airflow-scheduler exiting with code 0 on exceptions (#36800) > >> - Fix Callback exception when a removed task is the last one in the > >> ``taskinstance`` list (#36693) > >> - Allow anonymous user edit/show resource when set > >> ``AUTH_ROLE_PUBLIC=admin`` (#36750) > >> - Better error message when sqlite URL uses relative path (#36774) > >> - Explicit string cast required to force integer-type run_ids to be > passed > >> as strings instead of integers (#36756) > >> - Add log lookup exception for empty ``op`` subtypes (#35536) > >> - Remove unused index on task instance (#36737) > >> - Fix check on subclass for ``typing.Union`` in > ``_infer_multiple_outputs`` > >> for Python 3.10+ (#36728) > >> - Make sure ``multiple_outputs`` is inferred correctly even when using > >> ``TypedDict`` (#36652) > >> - Add back FAB constant in legacy security manager (#36719) > >> - Fix AttributeError when using ``Dagrun.update_state`` (#36712) > >> - Do not let ``EventsTimetable`` schedule past events if > ``catchup=False`` > >> (#36134) > >> - Support encryption for triggers parameters (#36492) > >> - Fix the type hint for ``tis_query`` in ``_process_executor_events`` > >> (#36655) > >> - Redirect to index when user does not have permission to access a page > >> (#36623) > >> - Avoid using dict as default value in ``call_regular_interval`` > (#36608) > >> - Remove option to set a task instance to running state in UI (#36518) > >> - Fix details tab not showing when using dynamic task mapping (#36522) > >> - Raise error when ``DagRun`` fails while running ``dag test`` (#36517) > >> - Refactor ``_manage_executor_state`` by refreshing TIs in batch > (#36502) > >> - Add flask config: ``MAX_CONTENT_LENGTH`` (#36401) > >> - Fix get_leaves calculation for teardown in nested group (#36456) > >> - Stop serializing timezone-naive datetime to timezone-aware datetime > with > >> UTC tz (#36379) > >> - Make ``kubernetes`` decorator type annotation consistent with operator > >> (#36405) > >> - Fix Webserver returning 500 for POST requests to ``api/dag/*/dagrun`` > >> from anonymous user (#36275) > >> - Fix the required access for get_variable endpoint (#36396) > >> - Fix datetime reference in ``DAG.is_fixed_time_schedule`` (#36370) > >> - Fix AirflowSkipException message raised by BashOperator (#36354) > >> - Allow PythonVirtualenvOperator.skip_on_exit_code to be zero (#36361) > >> - Increase width of execution_date input in trigger.html (#36278) > >> - Fix logging for pausing DAG (#36182) > >> - Stop deserializing pickle when enable_xcom_pickling is False (#36255) > >> - Check DAG read permission before accessing DAG code (#36257) > >> - Enable mark task as failed/success always (#36254) > >> - Create latest log dir symlink as relative link (#36019) > >> - Fix Python-based decorators templating (#36103) > >> > >> *Miscellaneous* > >> - Rename concurrency label to max active tasks (#36691) > >> - Restore function scoped ``httpx`` import in file_task_handler for > >> performance (#36753) > >> - Add support of Pendulum 3 (#36281) > >> - Standardize airflow build process and switch to Hatchling build > backend > >> (#36537) > >> - Get rid of ``pyarrow-hotfix`` for ``CVE-2023-47248`` (#36697) > >> - Make ``graphviz`` dependency optional (#36647) > >> - Announce MSSQL support end in Airflow 2.9.0, add migration script > hints > >> (#36509) > >> - Set min ``pandas`` dependency to 1.2.5 for all providers and airflow > >> (#36698) > >> - Bump follow-redirects from 1.15.3 to 1.15.4 in ``/airflow/www`` > (#36700) > >> - Provide the logger_name param to base hook in order to override the > >> logger name (#36674) > >> - Fix run type icon alignment with run type text (#36616) > >> - Follow BaseHook connection fields method signature in FSHook (#36444) > >> - Remove redundant ``docker`` decorator type annotations (#36406) > >> - Straighten typing in workday timetable (#36296) > >> - Use ``batch_is_authorized_dag`` to check if user has permission to > read > >> DAGs (#36279) > >> - Replace deprecated get_accessible_dag_ids and use get_readable_dags in > >> get_dag_warnings (#36256) > >> > >> *Doc Only Changes* > >> - Metrics tagging documentation (#36627) > >> - In docs use logical_date instead of deprecated execution_date (#36654) > >> - Add section about live-upgrading Airflow (#36637) > >> - Replace ``numpy`` example with practical exercise demonstrating > top-level > >> code (#35097) > >> - Improve and add more complete description in the architecture diagrams > >> (#36513) > >> - Improve the error message displayed when there is a webserver error > >> (#36570) > >> - Update ``dags.rst`` with information on DAG pausing (#36540) > >> - Update installation prerequisites after upgrading to Debian Bookworm > >> (#36521) > >> - Add description on the ways how users should approach DB monitoring > >> (#36483) > >> - Add branching based on mapped task group example to > >> dynamic-task-mapping.rst (#36480) > >> - Add further details to replacement documentation (#36485) > >> - Use cards when describing priority weighting methods (#36411) > >> - Update ``metrics.rst`` for param ``dagrun.schedule_delay`` (#36404) > >> - Update admonitions in Python operator doc to reflect sentiment > (#36340) > >> - Improve audit_logs.rst (#36213) > >> - Remove Redshift mention from the list of managed Postgres backends > >> (#36217) > >> > >> Cheers, > >> Ephraim > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >