Thanks Ash, we did not cover this use case in the dev call last week. This is a good catch!
Vikram On Tue, Aug 4, 2026 at 2:28 PM Jarek Potiuk <[email protected]> wrote: > Just landed (and I have internet and Github access it seems :) . > > > Yeah. I am fine with this - that was my initial idea too :) .. > > I do not think we have to do some **more** messaging. How it worked - the > version I had initially was that it could handle both in new version > airflow, it just would fail if we go back to earlier one - and there - we > actually can't change the error message ... That ship had sailed :) > > J > > On Tue, Aug 4, 2026, 20:37 Rahul Vats <[email protected]> wrote: > > > Thanks Ash, for the detailed explanation. It makes more sense now to > > revert 70791 > > and rebase+ merge 70558. > > > > Regards, > > Rahul Vats > > > > > > > > On Tue, 4 Aug 2026 at 18:01, Amogh Desai <[email protected]> wrote: > > > > > Thanks for spotting this and explaining it so clearly, appreciate it, > > Ash. > > > > > > Agreed, no objection. After your explanation, I realised the fault. > It's > > > more than just > > > "pandas 3 doesn't work with xcom" - people could've been using pandas 3 > > > functionally > > > and not for xcom. > > > > > > Forcing pandas 2 on people already running pandas 3 fine on 3.3.0 is > > worse > > > than the > > > rollback risk we were worried about. So let's revert 70791 and merge > > 70558 > > > (after a review). > > > > > > The cross version error message is probably a separate issue or we fold > > it > > > in 70558. > > > > > > Thanks & Regards, > > > Amogh Desai > > > > > > > > > On Tue, Aug 4, 2026 at 4:47 PM Ash Berlin-Taylor <[email protected]> > wrote: > > > > > > > So my issue is 70791 is that it itself is (or will be) a regression > on > > > > 3.3.0 > > > > > > > > By limiting Pandas to < 3 in this change in a point release, we will > > end > > > > up breaking any dag that was using Pandas 3.0 features (because 3.0.5 > > > will > > > > happily install with 3.3.0), but with the proposed version > limitation, > > it > > > > would get forcibly downgraded. Either this will result in > unsatisfiable > > > > constraints if the users has `pandas>=3.0`, or perhaps worse it would > > > > silently downgrade. > > > > > > > > And as for the worry about downgrade breaking things, yes, but Pandas > > 3.0 > > > > was released on 21st January 2026. By forcing a downgrade now we are > > > > re-breaking anything people may have already dealt with this. To me > > it's > > > > not really any different to "I changed my dag to push something new > to > > > > xcom, if I roll back the dag I can't use those XComs anymore”. > > > > > > > > In this regard Airflow should behave as a library, not an > application — > > > > and be lenient in the versions we work with. > > > > > > > > If we’d applied this version constraint 6 months ago/just after > pandas > > > 3.0 > > > > was released it’s fine, but almost 7 months after the fact it seems > > > > actively harmful for compat/expectations. > > > > > > > > I think we should revert 70791 and merge > > > > https://github.com/apache/airflow/pull/70558/ in instead. If > everyone > > > > agrees/doesn’t disagree, then I can handle these changes as you are > > > > travelling Jarek. Perhaps we need to extend it with. “When attempting > > to > > > > load Pandas2 df when Pandas3 is the active version throw a nice > error”. > > > > TBD? (Unless anyone already knows how this will behave?) > > > > > > > > -ash > > > > > > > > > On 4 Aug 2026, at 08:04, Jarek Potiuk <[email protected]> wrote: > > > > > > > > > >> A (temporary) -1 on anything including > > > > > https://github.com/apache/airflow/pull/70791 please — I want to > > catch > > > > up > > > > > with the logic and thinking there (I understand this was discussed > on > > > the > > > > > dev call) > > > > > > > > > > FYI Ash - I think this is important to mention that it also impacts > > > > Airlfow > > > > > -> not only providers, so if we have other proposals - I think it > > would > > > > be > > > > > great to get it sorted before RC. I am flying in about few hours, > and > > > > then > > > > > i will also be available "sometimes" so it would be great to know > if > > we > > > > > need to do some follow-ups - might delay 3.1.1 as currently there > is > > a > > > > > runtime check in Airflow preventing pandas3 to be used **when** > > > > > serialization is used. > > > > > > > > > > > > > > > J. > > > > > > > > > > > > > > > On Tue, Aug 4, 2026 at 6:16 AM Steve Ahn <[email protected]> > > > wrote: > > > > > > > > > >> +1 (non-binding) on airbyte 6.0.1rc1. > > > > >> > > > > >> Tested against the RC wheel on 3.3.0: a cancelled Airbyte sync now > > > > yields a > > > > >> "cancelled" event rather than falling through the success branch > > > > (#69786). > > > > >> > > > > >> I also verified my changes in amazon 9.34.0rc1 (#69676, deferred > > > > Redshift > > > > >> statement > > > > >> is cancelled on kill) and common.sql 2.1.0rc1 (#69736, value-check > > > > >> tolerance no > > > > >> longer inverts for negative expected values) — both behave > correctly > > > > >> against their > > > > >> RC wheels. Not voting on those two since they carry #70791, which > > Ash > > > > has > > > > >> asked to > > > > >> hold. > > > > >> > > > > >> -- > > > > >> Steve Ahn > > > > >> > > > > >> On Mon, Aug 3, 2026 at 4:17 AM Jarek Potiuk <[email protected]> > > wrote: > > > > >> > > > > >>> Correction: (We discussed these at the dev call and in the > > > > PRs/issues—not > > > > >>> on the dev list, though.) Perhaps it's a good idea that we are > > doing > > > > this > > > > >>> now. But if you have any thoughts, perhaps it's better to discuss > > > them > > > > >> in a > > > > >>> separate thread :). > > > > >>> > > > > >>> On Mon, Aug 3, 2026 at 1:15 PM Jarek Potiuk <[email protected]> > > > wrote: > > > > >>> > > > > >>>> Sure. Take your time Ash. > > > > >>>> > > > > >>>> Just to give you more context (we can discuss this on Slack if > you > > > > have > > > > >>>> questions or doubts, rather than here). > > > > >>>> > > > > >>>> I think there are no "perfect" solutions here due to how > "pandas" > > is > > > > >>> built > > > > >>>> into our XCom processing and potential migration issues. There > are > > > > >>> **only** > > > > >>>> trade-offs. We discussed a few of those at the devlist. I'm also > > not > > > > >>>> entirely happy with what we came up with, but I think the issue > > here > > > > is > > > > >>>> choosing the "lesser evil". > > > > >>>> > > > > >>>> Unless of course we can find a solution that covers all > potential > > > > >> issues > > > > >>>> here: > > > > >>>> > > > > >>>> a) risk of accidental migration issues to Pandas 3 by people > > without > > > > >>>> realising that it breaks both XCom **and** incompatibilities > with > > > > >> Pandas > > > > >>> 3 > > > > >>>> b) risk of "no easy downgrade" path when people - even > knowingly - > > > > >>> migrate > > > > >>>> to Pandas 3, only to realize later they need to downgrade. > > > > >>>> c) risk of not supporting Pandas 3 "today" for those who need it > > > > >>>> > > > > >>>> We must weigh all these risks. The trade-off we developed > > considers > > > > >> risks > > > > >>>> (a) and (b) more impactful than (c). Our approach is to > > deliberately > > > > >> add > > > > >>>> support for Pandas 3 in Airflow 3.4 while temporarily mitigating > > (a) > > > > >> and > > > > >>>> (b). > > > > >>>> But other trade-offs here are possible of course. > > > > >>>> > > > > >>>> J, > > > > >>>> > > > > >>>> > > > > >>>> On Mon, Aug 3, 2026 at 12:33 PM Ash Berlin-Taylor < > [email protected] > > > > > > > >>> wrote: > > > > >>>> > > > > >>>>> A (temporary) -1 on anything including > > > > >>>>> https://github.com/apache/airflow/pull/70791 please — I want > to > > > > >> catch > > > > >>>>> up with the logic and thinking there (I understand this was > > > discussed > > > > >> on > > > > >>>>> the dev call) > > > > >>>>> > > > > >>>>> -ash > > > > >>>>> > > > > >>>>> > > > > >>>>>> On 3 Aug 2026, at 08:43, Andrew Chang <[email protected]> > > > > >> wrote: > > > > >>>>>> > > > > >>>>>> Thanks for preparing the release, Jarek! > > > > >>>>>> > > > > >>>>>> +1 (non-binding) > > > > >>>>>> > > > > >>>>>> Tested apache-airflow-providers-microsoft-azure 14.1.0rc1 on > > > > >> released > > > > >>>>>> Airflow 3.3.0 with breeze --use-airflow-version. I focused on > my > > > > >>> change > > > > >>>>>> #70301, which registers the wasb remote logging scheme. > > > > >>>>>> > > > > >>>>>> The wasb scheme resolves to WasbRemoteLogIO through provider > > > > >> dispatch, > > > > >>>>> and > > > > >>>>>> the legacy discovery path is not called. from_config() builds > > the > > > IO > > > > >>>>> from > > > > >>>>>> the [logging] section as expected. I also ran a real upload > and > > > read > > > > >>>>>> round-trip against Azurite, and the log content matched. > > > > >>>>>> > > > > >>>>>> Best, > > > > >>>>>> Andrew > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Jarek Potiuk <[email protected]> 於 2026年8月3日週一 上午8:37寫道： > > > > >>>>>> > > > > >>>>>>> Hey all, > > > > >>>>>>> > > > > >>>>>>> I have just cut the new wave Airflow Providers packages with > > > > >> release > > > > >>>>>>> preparation date 2026-08-01. This email is calling a vote on > > the > > > > >>>>>>> release, > > > > >>>>>>> which will last for 72 hours - which means that it will end > on > > > > >>>>>>> 2026-08-06 00:40 UTC and until 3 binding +1 votes have been > > > > >> received. > > > > >>>>>>> > > > > >>>>>>> Consider this my (binding) +1. > > > > >>>>>>> > > > > >>>>>>> This wave contains 48 providers. There are no breaking > changes > > in > > > > >> any > > > > >>>>>>> of them - all bumps are minor or patch level. > > > > >>>>>>> > > > > >>>>>>> Four of the providers are re-cuts of release candidates that > > were > > > > >>>>>>> prepared in the 2026-07-22 wave but never published, so they > > are > > > at > > > > >>>>>>> rc2 rather than rc1: > > > > >>>>>>> > > > > >>>>>>> - apache-airflow-providers-clickhousedb 1.0.1rc2 > > > > >>>>>>> - apache-airflow-providers-common-ai 0.7.0rc2 > > > > >>>>>>> - apache-airflow-providers-google 22.3.0rc2 > > > > >>>>>>> - apache-airflow-providers-openlineage 2.20.0rc2 > > > > >>>>>>> > > > > >>>>>>> For those four, the testing status issue carries over the > check > > > > >> marks > > > > >>>>>>> from the previous wave, so only the genuinely new commits are > > > left > > > > >>>>>>> unchecked. > > > > >>>>>>> > > > > >>>>>>> Airflow Providers are available at: > > > > >>>>>>> > > > > >> > https://dist.apache.org/repos/dist/dev/airflow/providers/2026-08-01 > > > > >>>>>>> > > > > >>>>>>> *apache_airflow_providers-2026-08-01-source.tar.gz* is the > full > > > > >>> source > > > > >>>>>>> tarball of airflow repo - snapshot taken at the moment of > > > > >> provider's > > > > >>>>>>> release. > > > > >>>>>>> > > > > >>>>>>> *apache-airflow-providers-<PROVIDER>-*.tar.gz* are the > > > convenience > > > > >>>>>>> python "sdist" distributions that we publish in PyPI > > > > >>>>>>> > > > > >>>>>>> *apache_airflow_providers_<PROVIDER>-*.whl are the > convenience > > > > >> Python > > > > >>>>>>> "wheel" distributions that we publish in PyPI. > > > > >>>>>>> > > > > >>>>>>> The test procedure for PMC members is described in > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-pmc-members > > > > >>>>>>> > > > > >>>>>>> The test procedure for and Contributors who would like to > test > > > this > > > > >>> RC > > > > >>>>>>> is described in: > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-contributors > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> Public keys are available at: > > > > >>>>>>> https://dist.apache.org/repos/dist/release/airflow/KEYS > > > > >>>>>>> > > > > >>>>>>> Please vote accordingly: > > > > >>>>>>> > > > > >>>>>>> [ ] +1 approve > > > > >>>>>>> [ ] +0 no opinion > > > > >>>>>>> [ ] -1 disapprove with the reason > > > > >>>>>>> > > > > >>>>>>> Only votes from PMC members are binding, but members of the > > > > >> community > > > > >>>>> are > > > > >>>>>>> encouraged to test the release and vote with "(non-binding)". > > > > >>>>>>> > > > > >>>>>>> Please note that the version number excludes the 'rcX' > string. > > > > >>>>>>> This will allow us to rename the artifact without modifying > > > > >>>>>>> the artifact checksums when we actually release it. > > > > >>>>>>> > > > > >>>>>>> The status of testing the providers by the community is kept > > > here: > > > > >>>>>>> https://github.com/apache/airflow/issues/70953 > > > > >>>>>>> > > > > >>>>>>> The issue is also the easiest way to see important PRs > included > > > in > > > > >>> the > > > > >>>>>>> RC candidates. > > > > >>>>>>> Detailed changelog for the providers will be published in the > > > > >>>>>>> documentation after the > > > > >>>>>>> RC candidates are released. > > > > >>>>>>> > > > > >>>>>>> You can find the RC packages in PyPI following these links: > > > > >>>>>>> > > > > >>>>>>> > > > > >> > https://pypi.org/project/apache-airflow-providers-airbyte/6.0.1rc1/ > > > > >>>>>>> > > > > >> > https://pypi.org/project/apache-airflow-providers-alibaba/3.4.0rc1/ > > > > >>>>>>> > > > > >> > https://pypi.org/project/apache-airflow-providers-amazon/9.34.0rc1/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-anthropic/0.2.1rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > > > https://pypi.org/project/apache-airflow-providers-apache-hdfs/4.12.2rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >> > > > > https://pypi.org/project/apache-airflow-providers-apache-hive/9.6.1rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > https://pypi.org/project/apache-airflow-providers-apache-kafka/1.15.1rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >> > > > > https://pypi.org/project/apache-airflow-providers-apache-livy/4.6.0rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > > > https://pypi.org/project/apache-airflow-providers-apache-spark/6.3.1rc1/ > > > > >>>>>>> > > > > >> > > https://pypi.org/project/apache-airflow-providers-arangodb/2.9.6rc1/ > > > > >>>>>>> > > > > >> > https://pypi.org/project/apache-airflow-providers-celery/3.23.1rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > > > https://pypi.org/project/apache-airflow-providers-clickhousedb/1.0.1rc2/ > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/10.21.0rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-cohere/1.6.7rc1/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-common-ai/0.7.0rc2/ > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > https://pypi.org/project/apache-airflow-providers-common-compat/1.18.0rc1/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-common-sql/2.1.0rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >> > > > > https://pypi.org/project/apache-airflow-providers-databricks/7.18.1rc1/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-dbt-cloud/4.9.3rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-docker/4.5.9rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-edge3/4.3.0rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > https://pypi.org/project/apache-airflow-providers-elasticsearch/6.9.0rc1/ > > > > >>>>>>> > > > > >> > https://pypi.org/project/apache-airflow-providers-exasol/4.10.5rc1/ > > > > >>>>>>> > > https://pypi.org/project/apache-airflow-providers-fab/3.8.0rc1/ > > > > >>>>>>> > > https://pypi.org/project/apache-airflow-providers-git/0.4.2rc1/ > > > > >>>>>>> > > > > >> > https://pypi.org/project/apache-airflow-providers-google/22.3.0rc2/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-hashicorp/4.8.0rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-imap/3.12.1rc1/ > > > > >>>>>>> > > > > >> > > https://pypi.org/project/apache-airflow-providers-keycloak/0.9.0rc1/ > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > https://pypi.org/project/apache-airflow-providers-microsoft-azure/14.1.0rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-neo4j/3.12.1rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-openai/1.8.2rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >>> > > > > > > https://pypi.org/project/apache-airflow-providers-openlineage/2.20.0rc2/ > > > > >>>>>>> > > > > >>>>> > > > > >> > > > > https://pypi.org/project/apache-airflow-providers-opensearch/1.11.2rc1/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-papermill/3.13.2rc1/ > > > > >>>>>>> > > > > >> > > https://pypi.org/project/apache-airflow-providers-postgres/7.0.1rc1/ > > > > >>>>>>> > > > > >> > https://pypi.org/project/apache-airflow-providers-presto/5.12.1rc1/ > > > > >>>>>>> > > > > >>>>> > > > > >> > > > > https://pypi.org/project/apache-airflow-providers-salesforce/5.14.2rc1/ > > > > >>>>>>> > > https://pypi.org/project/apache-airflow-providers-sftp/6.0.1rc1/ > > > > >>>>>>> > > https://pypi.org/project/apache-airflow-providers-smtp/3.0.3rc1/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-snowflake/6.16.0rc1/ > > > > >>>>>>> > > https://pypi.org/project/apache-airflow-providers-ssh/6.0.1rc1/ > > > > >>>>>>> > > > > >>> > > > https://pypi.org/project/apache-airflow-providers-standard/1.17.0rc1/ > > > > >>>>>>> > > > > >> > > https://pypi.org/project/apache-airflow-providers-telegram/4.9.6rc1/ > > > > >>>>>>> > > > > >> > > https://pypi.org/project/apache-airflow-providers-teradata/3.6.2rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-trino/6.6.1rc1/ > > > > >>>>>>> > > > > >> > > https://pypi.org/project/apache-airflow-providers-weaviate/3.4.1rc1/ > > > > >>>>>>> > > > https://pypi.org/project/apache-airflow-providers-yandex/4.5.1rc1/ > > > > >>>>>>> > > > > >>>>>>> Cheers, > > > > >>>>>>> Jarek Potiuk > > > > >>>>>>> > > > > >>>>>>> > > > > >> > > --------------------------------------------------------------------- > > > > >>>>>>> To unsubscribe, e-mail: [email protected] > > > > >>>>>>> For additional commands, e-mail: [email protected] > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>> > > > > >>>>> > > > > >>> > > > > >> > > > > > > > > > > > > > >
