Thanks Ash, for the detailed explanation. It makes more sense now to revert 70791 and rebase+ merge 70558.
Regards, Rahul Vats On Tue, 4 Aug 2026 at 18:01, Amogh Desai <[email protected]> wrote: > Thanks for spotting this and explaining it so clearly, appreciate it, Ash. > > Agreed, no objection. After your explanation, I realised the fault. It's > more than just > "pandas 3 doesn't work with xcom" - people could've been using pandas 3 > functionally > and not for xcom. > > Forcing pandas 2 on people already running pandas 3 fine on 3.3.0 is worse > than the > rollback risk we were worried about. So let's revert 70791 and merge 70558 > (after a review). > > The cross version error message is probably a separate issue or we fold it > in 70558. > > Thanks & Regards, > Amogh Desai > > > On Tue, Aug 4, 2026 at 4:47 PM Ash Berlin-Taylor <[email protected]> wrote: > > > So my issue is 70791 is that it itself is (or will be) a regression on > > 3.3.0 > > > > By limiting Pandas to < 3 in this change in a point release, we will end > > up breaking any dag that was using Pandas 3.0 features (because 3.0.5 > will > > happily install with 3.3.0), but with the proposed version limitation, it > > would get forcibly downgraded. Either this will result in unsatisfiable > > constraints if the users has `pandas>=3.0`, or perhaps worse it would > > silently downgrade. > > > > And as for the worry about downgrade breaking things, yes, but Pandas 3.0 > > was released on 21st January 2026. By forcing a downgrade now we are > > re-breaking anything people may have already dealt with this. To me it's > > not really any different to "I changed my dag to push something new to > > xcom, if I roll back the dag I can't use those XComs anymore”. > > > > In this regard Airflow should behave as a library, not an application — > > and be lenient in the versions we work with. > > > > If we’d applied this version constraint 6 months ago/just after pandas > 3.0 > > was released it’s fine, but almost 7 months after the fact it seems > > actively harmful for compat/expectations. > > > > I think we should revert 70791 and merge > > https://github.com/apache/airflow/pull/70558/ in instead. If everyone > > agrees/doesn’t disagree, then I can handle these changes as you are > > travelling Jarek. Perhaps we need to extend it with. “When attempting to > > load Pandas2 df when Pandas3 is the active version throw a nice error”. > > TBD? (Unless anyone already knows how this will behave?) > > > > -ash > > > > > On 4 Aug 2026, at 08:04, Jarek Potiuk <[email protected]> wrote: > > > > > >> A (temporary) -1 on anything including > > > https://github.com/apache/airflow/pull/70791 please — I want to catch > > up > > > with the logic and thinking there (I understand this was discussed on > the > > > dev call) > > > > > > FYI Ash - I think this is important to mention that it also impacts > > Airlfow > > > -> not only providers, so if we have other proposals - I think it would > > be > > > great to get it sorted before RC. I am flying in about few hours, and > > then > > > i will also be available "sometimes" so it would be great to know if we > > > need to do some follow-ups - might delay 3.1.1 as currently there is a > > > runtime check in Airflow preventing pandas3 to be used **when** > > > serialization is used. > > > > > > > > > J. > > > > > > > > > On Tue, Aug 4, 2026 at 6:16 AM Steve Ahn <[email protected]> > wrote: > > > > > >> +1 (non-binding) on airbyte 6.0.1rc1. > > >> > > >> Tested against the RC wheel on 3.3.0: a cancelled Airbyte sync now > > yields a > > >> "cancelled" event rather than falling through the success branch > > (#69786). > > >> > > >> I also verified my changes in amazon 9.34.0rc1 (#69676, deferred > > Redshift > > >> statement > > >> is cancelled on kill) and common.sql 2.1.0rc1 (#69736, value-check > > >> tolerance no > > >> longer inverts for negative expected values) — both behave correctly > > >> against their > > >> RC wheels. Not voting on those two since they carry #70791, which Ash > > has > > >> asked to > > >> hold. > > >> > > >> -- > > >> Steve Ahn > > >> > > >> On Mon, Aug 3, 2026 at 4:17 AM Jarek Potiuk <[email protected]> wrote: > > >> > > >>> Correction: (We discussed these at the dev call and in the > > PRs/issues—not > > >>> on the dev list, though.) Perhaps it's a good idea that we are doing > > this > > >>> now. But if you have any thoughts, perhaps it's better to discuss > them > > >> in a > > >>> separate thread :). > > >>> > > >>> On Mon, Aug 3, 2026 at 1:15 PM Jarek Potiuk <[email protected]> > wrote: > > >>> > > >>>> Sure. Take your time Ash. > > >>>> > > >>>> Just to give you more context (we can discuss this on Slack if you > > have > > >>>> questions or doubts, rather than here). > > >>>> > > >>>> I think there are no "perfect" solutions here due to how "pandas" is > > >>> built > > >>>> into our XCom processing and potential migration issues. There are > > >>> **only** > > >>>> trade-offs. We discussed a few of those at the devlist. I'm also not > > >>>> entirely happy with what we came up with, but I think the issue here > > is > > >>>> choosing the "lesser evil". > > >>>> > > >>>> Unless of course we can find a solution that covers all potential > > >> issues > > >>>> here: > > >>>> > > >>>> a) risk of accidental migration issues to Pandas 3 by people without > > >>>> realising that it breaks both XCom **and** incompatibilities with > > >> Pandas > > >>> 3 > > >>>> b) risk of "no easy downgrade" path when people - even knowingly - > > >>> migrate > > >>>> to Pandas 3, only to realize later they need to downgrade. > > >>>> c) risk of not supporting Pandas 3 "today" for those who need it > > >>>> > > >>>> We must weigh all these risks. The trade-off we developed considers > > >> risks > > >>>> (a) and (b) more impactful than (c). Our approach is to deliberately > > >> add > > >>>> support for Pandas 3 in Airflow 3.4 while temporarily mitigating (a) > > >> and > > >>>> (b). > > >>>> But other trade-offs here are possible of course. > > >>>> > > >>>> J, > > >>>> > > >>>> > > >>>> On Mon, Aug 3, 2026 at 12:33 PM Ash Berlin-Taylor <[email protected]> > > >>> wrote: > > >>>> > > >>>>> A (temporary) -1 on anything including > > >>>>> https://github.com/apache/airflow/pull/70791 please — I want to > > >> catch > > >>>>> up with the logic and thinking there (I understand this was > discussed > > >> on > > >>>>> the dev call) > > >>>>> > > >>>>> -ash > > >>>>> > > >>>>> > > >>>>>> On 3 Aug 2026, at 08:43, Andrew Chang <[email protected]> > > >> wrote: > > >>>>>> > > >>>>>> Thanks for preparing the release, Jarek! > > >>>>>> > > >>>>>> +1 (non-binding) > > >>>>>> > > >>>>>> Tested apache-airflow-providers-microsoft-azure 14.1.0rc1 on > > >> released > > >>>>>> Airflow 3.3.0 with breeze --use-airflow-version. I focused on my > > >>> change > > >>>>>> #70301, which registers the wasb remote logging scheme. > > >>>>>> > > >>>>>> The wasb scheme resolves to WasbRemoteLogIO through provider > > >> dispatch, > > >>>>> and > > >>>>>> the legacy discovery path is not called. from_config() builds the > IO > > >>>>> from > > >>>>>> the [logging] section as expected. I also ran a real upload and > read > > >>>>>> round-trip against Azurite, and the log content matched. > > >>>>>> > > >>>>>> Best, > > >>>>>> Andrew > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> Jarek Potiuk <[email protected]> 於 2026年8月3日週一 上午8:37寫道： > > >>>>>> > > >>>>>>> Hey all, > > >>>>>>> > > >>>>>>> I have just cut the new wave Airflow Providers packages with > > >> release > > >>>>>>> preparation date 2026-08-01. This email is calling a vote on the > > >>>>>>> release, > > >>>>>>> which will last for 72 hours - which means that it will end on > > >>>>>>> 2026-08-06 00:40 UTC and until 3 binding +1 votes have been > > >> received. > > >>>>>>> > > >>>>>>> Consider this my (binding) +1. > > >>>>>>> > > >>>>>>> This wave contains 48 providers. There are no breaking changes in > > >> any > > >>>>>>> of them - all bumps are minor or patch level. > > >>>>>>> > > >>>>>>> Four of the providers are re-cuts of release candidates that were > > >>>>>>> prepared in the 2026-07-22 wave but never published, so they are > at > > >>>>>>> rc2 rather than rc1: > > >>>>>>> > > >>>>>>> - apache-airflow-providers-clickhousedb 1.0.1rc2 > > >>>>>>> - apache-airflow-providers-common-ai 0.7.0rc2 > > >>>>>>> - apache-airflow-providers-google 22.3.0rc2 > > >>>>>>> - apache-airflow-providers-openlineage 2.20.0rc2 > > >>>>>>> > > >>>>>>> For those four, the testing status issue carries over the check > > >> marks > > >>>>>>> from the previous wave, so only the genuinely new commits are > left > > >>>>>>> unchecked. > > >>>>>>> > > >>>>>>> Airflow Providers are available at: > > >>>>>>> > > >> https://dist.apache.org/repos/dist/dev/airflow/providers/2026-08-01 > > >>>>>>> > > >>>>>>> *apache_airflow_providers-2026-08-01-source.tar.gz* is the full > > >>> source > > >>>>>>> tarball of airflow repo - snapshot taken at the moment of > > >> provider's > > >>>>>>> release. > > >>>>>>> > > >>>>>>> *apache-airflow-providers-<PROVIDER>-*.tar.gz* are the > convenience > > >>>>>>> python "sdist" distributions that we publish in PyPI > > >>>>>>> > > >>>>>>> *apache_airflow_providers_<PROVIDER>-*.whl are the convenience > > >> Python > > >>>>>>> "wheel" distributions that we publish in PyPI. > > >>>>>>> > > >>>>>>> The test procedure for PMC members is described in > > >>>>>>> > > >>>>>>> > > >>>>> > > >>> > > >> > > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-pmc-members > > >>>>>>> > > >>>>>>> The test procedure for and Contributors who would like to test > this > > >>> RC > > >>>>>>> is described in: > > >>>>>>> > > >>>>>>> > > >>>>> > > >>> > > >> > > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDERS.md#verify-the-release-candidate-by-contributors > > >>>>>>> > > >>>>>>> > > >>>>>>> Public keys are available at: > > >>>>>>> https://dist.apache.org/repos/dist/release/airflow/KEYS > > >>>>>>> > > >>>>>>> Please vote accordingly: > > >>>>>>> > > >>>>>>> [ ] +1 approve > > >>>>>>> [ ] +0 no opinion > > >>>>>>> [ ] -1 disapprove with the reason > > >>>>>>> > > >>>>>>> Only votes from PMC members are binding, but members of the > > >> community > > >>>>> are > > >>>>>>> encouraged to test the release and vote with "(non-binding)". > > >>>>>>> > > >>>>>>> Please note that the version number excludes the 'rcX' string. > > >>>>>>> This will allow us to rename the artifact without modifying > > >>>>>>> the artifact checksums when we actually release it. > > >>>>>>> > > >>>>>>> The status of testing the providers by the community is kept > here: > > >>>>>>> https://github.com/apache/airflow/issues/70953 > > >>>>>>> > > >>>>>>> The issue is also the easiest way to see important PRs included > in > > >>> the > > >>>>>>> RC candidates. > > >>>>>>> Detailed changelog for the providers will be published in the > > >>>>>>> documentation after the > > >>>>>>> RC candidates are released. > > >>>>>>> > > >>>>>>> You can find the RC packages in PyPI following these links: > > >>>>>>> > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-airbyte/6.0.1rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-alibaba/3.4.0rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-amazon/9.34.0rc1/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-anthropic/0.2.1rc1/ > > >>>>>>> > > >>>>> > > >>> > > https://pypi.org/project/apache-airflow-providers-apache-hdfs/4.12.2rc1/ > > >>>>>>> > > >>>>> > > >> > https://pypi.org/project/apache-airflow-providers-apache-hive/9.6.1rc1/ > > >>>>>>> > > >>>>> > > >>> > > >> > > > https://pypi.org/project/apache-airflow-providers-apache-kafka/1.15.1rc1/ > > >>>>>>> > > >>>>> > > >> > https://pypi.org/project/apache-airflow-providers-apache-livy/4.6.0rc1/ > > >>>>>>> > > >>>>> > > >>> > > https://pypi.org/project/apache-airflow-providers-apache-spark/6.3.1rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-arangodb/2.9.6rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-celery/3.23.1rc1/ > > >>>>>>> > > >>>>> > > >>> > > https://pypi.org/project/apache-airflow-providers-clickhousedb/1.0.1rc2/ > > >>>>>>> > > >>>>>>> > > >>>>> > > >>> > > >> > > > https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/10.21.0rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-cohere/1.6.7rc1/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-common-ai/0.7.0rc2/ > > >>>>>>> > > >>>>> > > >>> > > >> > > > https://pypi.org/project/apache-airflow-providers-common-compat/1.18.0rc1/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-common-sql/2.1.0rc1/ > > >>>>>>> > > >>>>> > > >> > https://pypi.org/project/apache-airflow-providers-databricks/7.18.1rc1/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-dbt-cloud/4.9.3rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-docker/4.5.9rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-edge3/4.3.0rc1/ > > >>>>>>> > > >>>>> > > >>> > > >> > > > https://pypi.org/project/apache-airflow-providers-elasticsearch/6.9.0rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-exasol/4.10.5rc1/ > > >>>>>>> https://pypi.org/project/apache-airflow-providers-fab/3.8.0rc1/ > > >>>>>>> https://pypi.org/project/apache-airflow-providers-git/0.4.2rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-google/22.3.0rc2/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-hashicorp/4.8.0rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-imap/3.12.1rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-keycloak/0.9.0rc1/ > > >>>>>>> > > >>>>>>> > > >>>>> > > >>> > > >> > > > https://pypi.org/project/apache-airflow-providers-microsoft-azure/14.1.0rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-neo4j/3.12.1rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-openai/1.8.2rc1/ > > >>>>>>> > > >>>>> > > >>> > > https://pypi.org/project/apache-airflow-providers-openlineage/2.20.0rc2/ > > >>>>>>> > > >>>>> > > >> > https://pypi.org/project/apache-airflow-providers-opensearch/1.11.2rc1/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-papermill/3.13.2rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-postgres/7.0.1rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-presto/5.12.1rc1/ > > >>>>>>> > > >>>>> > > >> > https://pypi.org/project/apache-airflow-providers-salesforce/5.14.2rc1/ > > >>>>>>> https://pypi.org/project/apache-airflow-providers-sftp/6.0.1rc1/ > > >>>>>>> https://pypi.org/project/apache-airflow-providers-smtp/3.0.3rc1/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-snowflake/6.16.0rc1/ > > >>>>>>> https://pypi.org/project/apache-airflow-providers-ssh/6.0.1rc1/ > > >>>>>>> > > >>> > https://pypi.org/project/apache-airflow-providers-standard/1.17.0rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-telegram/4.9.6rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-teradata/3.6.2rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-trino/6.6.1rc1/ > > >>>>>>> > > >> https://pypi.org/project/apache-airflow-providers-weaviate/3.4.1rc1/ > > >>>>>>> > https://pypi.org/project/apache-airflow-providers-yandex/4.5.1rc1/ > > >>>>>>> > > >>>>>>> Cheers, > > >>>>>>> Jarek Potiuk > > >>>>>>> > > >>>>>>> > > >> --------------------------------------------------------------------- > > >>>>>>> To unsubscribe, e-mail: [email protected] > > >>>>>>> For additional commands, e-mail: [email protected] > > >>>>>>> > > >>>>>>> > > >>>>> > > >>>>> > > >>> > > >> > > > > >
