This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 415cb70f877 Limit pandas to < 3 for DataFrame XComs (#70791)
415cb70f877 is described below
commit 415cb70f87784cc074d6db7844a07579587b5f0f
Author: Jarek Potiuk <[email protected]>
AuthorDate: Sat Aug 1 14:18:03 2026 +0200
Limit pandas to < 3 for DataFrame XComs (#70791)
* Limit pandas to < 3 for DataFrame XComs
Under pandas 3 a DataFrame does not round trip through an XCom the way it
does
under pandas 2. The public classes moved to the `pandas` namespace, so the
same
object qualifies as `pandas.DataFrame` rather than
`pandas.core.frame.DataFrame`
and the serde registry -- keyed on the latter -- does not dispatch to the
pandas
serializer at all. The caller gets serde's generic "cannot serialize object
of
type" with nothing pointing at pandas. Beyond that, the dtypes differ: an
`object` column reads back as `str` and missing values as `nan` rather than
`None`, so the reader's pandas version, not the writer's, decides what a
task
receives.
Declare `pandas<3` where Airflow declares pandas, and back it with a runtime
check, since pandas is an optional dependency and the constraint alone
cannot be
relied on -- a deployment may install pandas 3 directly or pull it in
through
another package.
The check needs the pandas 3 qualname registered to be reachable at all:
without `pandas.DataFrame` in the registry the request never reaches this
module.
It is registered for that reason only, to refuse the value with a message
that
names pandas and says what to do, not to support it.
Reads are refused as well as writes. A payload written under pandas 2 comes
back
with pandas 3 dtypes, so accepting it would hand the task different data
than was
pushed, with no signal.
Moving from pandas 2 to pandas 3 is a deliberate migration for users to
make.
Support for it is not dropped, only deferred.
Generated-by: Claude Opus 5 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
* Sync uv.lock with the pandas < 3 constraint
* Add significant newsfragment for the pandas < 3 limit
* Apply the pandas < 3 limit to the remaining providers
The XCom limit only binds where Airflow declares pandas. Thirteen providers
pin
pandas independently of common-sql, so a deployment installing any of them
could
still resolve pandas 3 and reach the runtime refusal rather than being held
on a
supported version by the constraint.
pandas-gbq is deliberately untouched -- a different package, unaffected by
this.
Provider READMEs and uv.lock are regenerated to match.
* Regenerate provider docs index for the pandas < 3 pins
The update-providers-build-files hook renders the extras table in each
provider's docs/index.rst from its pyproject dependencies. Fourteen
providers
show the pandas extra, so the pin change widens those tables.
* Explain the pandas upper bound at every site and drop a vacuous test
contributing-docs/13_airflow_dependencies_and_extras.rst asks for a comment
saying why whenever a dependency is upper-bound, and AGENTS.md asks for the
tracking URL in the file rather than only in a PR comment. All 45 specifiers
across 15 files landed without either. Add the rationale and a pointer to
the
pandas 3 support PR above each block, so whoever decides when the cap comes
off
does not have to reconstruct it.
test_pandas_2_dataframe_xcom_still_round_trips passed unchanged on main:
nothing
read pd.__version__ before this PR, so monkeypatching it to a 2.x value
while the
installed pandas is already 2.x changed nothing observable, and the round
trip is
already covered by test_pandas_serializers. Dropped.
---------
Co-authored-by: Rahul Vats <[email protected]>
---
airflow-core/newsfragments/70791.significant.rst | 35 +++++++++
providers/amazon/docs/index.rst | 8 +-
providers/amazon/pyproject.toml | 15 +++-
providers/apache/hdfs/README.rst | 12 +--
providers/apache/hdfs/docs/index.rst | 12 +--
providers/apache/hdfs/pyproject.toml | 15 +++-
providers/apache/hive/README.rst | 12 +--
providers/apache/hive/docs/index.rst | 12 +--
providers/apache/hive/pyproject.toml | 15 +++-
providers/common/sql/docs/index.rst | 8 +-
providers/common/sql/pyproject.toml | 15 +++-
providers/databricks/README.rst | 12 +--
providers/databricks/docs/index.rst | 12 +--
providers/databricks/pyproject.toml | 15 +++-
providers/exasol/README.rst | 12 +--
providers/exasol/docs/index.rst | 12 +--
providers/exasol/pyproject.toml | 15 +++-
providers/google/README.rst | 12 +--
providers/google/docs/index.rst | 12 +--
providers/google/pyproject.toml | 15 +++-
providers/papermill/README.rst | 12 +--
providers/papermill/docs/index.rst | 12 +--
providers/papermill/pyproject.toml | 15 +++-
providers/postgres/docs/index.rst | 8 +-
providers/postgres/pyproject.toml | 15 +++-
providers/presto/README.rst | 12 +--
providers/presto/docs/index.rst | 12 +--
providers/presto/pyproject.toml | 15 +++-
providers/salesforce/README.rst | 12 +--
providers/salesforce/docs/index.rst | 12 +--
providers/salesforce/pyproject.toml | 15 +++-
providers/snowflake/README.rst | 6 +-
providers/snowflake/docs/index.rst | 6 +-
providers/snowflake/pyproject.toml | 15 +++-
providers/trino/README.rst | 12 +--
providers/trino/docs/index.rst | 12 +--
providers/trino/pyproject.toml | 15 +++-
providers/weaviate/README.rst | 12 +--
providers/weaviate/docs/index.rst | 12 +--
providers/weaviate/pyproject.toml | 15 +++-
task-sdk/pyproject.toml | 15 +++-
.../src/airflow/sdk/serde/serializers/pandas.py | 45 +++++++++++
task-sdk/tests/task_sdk/serde/test_serializers.py | 28 +++++++
uv.lock | 90 +++++++++++-----------
44 files changed, 471 insertions(+), 228 deletions(-)
diff --git a/airflow-core/newsfragments/70791.significant.rst
b/airflow-core/newsfragments/70791.significant.rst
new file mode 100644
index 00000000000..827f22bc68f
--- /dev/null
+++ b/airflow-core/newsfragments/70791.significant.rst
@@ -0,0 +1,35 @@
+pandas is limited to ``< 3``; DataFrame XComs are refused on pandas 3
+
+A ``pandas.DataFrame`` does not survive an XCom round trip under pandas 3 the
way it does
+under pandas 2, so this version of Airflow constrains pandas to ``< 3`` and
refuses DataFrame
+XComs at runtime when a newer pandas is installed anyway.
+
+Two things change under pandas 3. Its public classes are exposed from the
``pandas``
+namespace, so a DataFrame qualifies as ``pandas.DataFrame`` rather than
+``pandas.core.frame.DataFrame`` and the XCom serializer registry -- keyed on
the latter --
+never dispatches to the pandas serializer. And the dtypes differ: an
``object`` column reads
+back as ``str``, and missing values as ``nan`` rather than ``None``, so the
*reader's* pandas
+version decides what a task receives rather than the writer's.
+
+Because pandas is an optional dependency, the ``< 3`` constraint alone cannot
be relied on --
+a deployment may install pandas 3 directly, or pull it in through another
package. It is
+therefore backed by a runtime check.
+
+**Behaviour changes:**
+
+- With pandas 3 or newer installed, pushing **or** pulling a ``DataFrame``
through XCom now
+ raises ``RuntimeError`` naming the installed pandas version and the
supported range.
+ Reads are refused as well as writes, because a payload written under pandas
2 reads back
+ with pandas 3 dtypes and would otherwise hand the task different data than
was pushed,
+ with no signal.
+- Previously the same situation produced an opaque
+ ``TypeError: cannot serialize object of type <class 'pandas.DataFrame'>``
from the
+ serializer registry, with nothing identifying pandas as the cause.
+- Deployments that pass DataFrames through XCom must pin ``pandas < 3`` in the
environment
+ that runs their tasks. Deployments that do not use DataFrame XComs are
unaffected at
+ runtime.
+- Nothing changes for pandas 2 users.
+
+**This defers pandas 3 support rather than dropping it.** Moving from pandas 2
to pandas 3
+is a deliberate migration for users to make, given the dtype changes it brings
to data that
+has already been written. Support for pandas 3 will be enabled in a future
Airflow release.
diff --git a/providers/amazon/docs/index.rst b/providers/amazon/docs/index.rst
index 6a01e3506d6..446a2e4df51 100644
--- a/providers/amazon/docs/index.rst
+++ b/providers/amazon/docs/index.rst
@@ -171,9 +171,9 @@ Install them when installing from PyPI. For example:
pip install apache-airflow-providers-amazon[aiobotocore]
-====================
============================================================================================================================================================
+====================
=====================================================================================================================================================================
Extra Dependencies
-====================
============================================================================================================================================================
+====================
=====================================================================================================================================================================
``aiobotocore`` ``aiobotocore>=3.0.0``
``cncf.kubernetes`` ``apache-airflow-providers-cncf-kubernetes>=7.2.0``
``s3fs`` ``s3fs>=2023.10.0``
@@ -186,14 +186,14 @@ Extra Dependencies
``imap`` ``apache-airflow-providers-imap``
``microsoft.azure`` ``apache-airflow-providers-microsoft-azure``
``mongo`` ``apache-airflow-providers-mongo``
-``pandas`` ``pandas>=2.1.2; python_version <"3.13"``,
``pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"``,
``pandas>=2.3.3; python_version >="3.14"``
+``pandas`` ``pandas>=2.1.2,<3; python_version <"3.13"``,
``pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"``,
``pandas>=2.3.3,<3; python_version >="3.14"``
``openlineage`` ``apache-airflow-providers-openlineage>=2.3.0``
``salesforce`` ``apache-airflow-providers-salesforce``
``ssh`` ``apache-airflow-providers-ssh``
``standard`` ``apache-airflow-providers-standard``
``common.messaging`` ``apache-airflow-providers-common-messaging>=2.0.0``
``sqlalchemy`` ``sqlalchemy>=1.4.54``
-====================
============================================================================================================================================================
+====================
=====================================================================================================================================================================
Downloading official packages
-----------------------------
diff --git a/providers/amazon/pyproject.toml b/providers/amazon/pyproject.toml
index a8a4224b7d0..e8200c48494 100644
--- a/providers/amazon/pyproject.toml
+++ b/providers/amazon/pyproject.toml
@@ -130,9 +130,18 @@ dependencies = [
"apache-airflow-providers-mongo"
]
"pandas" = [
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
]
"openlineage" = [
"apache-airflow-providers-openlineage>=2.3.0"
diff --git a/providers/apache/hdfs/README.rst b/providers/apache/hdfs/README.rst
index 6326ee5eeda..7841ffb143d 100644
--- a/providers/apache/hdfs/README.rst
+++ b/providers/apache/hdfs/README.rst
@@ -51,19 +51,19 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
==================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
==================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``hdfs[avro,dataframe,kerberos]`` ``>=2.5.4; python_version <
"3.12"``
``hdfs[avro,dataframe,kerberos]`` ``>=2.7.3; python_version >=
"3.12"``
``fastavro`` ``>=1.10.0; python_version >=
"3.13" and python_version < "3.14"``
``fastavro`` ``>=1.12.1; python_version >=
"3.14"``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
==================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
The changelog for the provider package can be found in the
`changelog
<https://airflow.apache.org/docs/apache-airflow-providers-apache-hdfs/4.12.1/changelog.html>`_.
diff --git a/providers/apache/hdfs/docs/index.rst
b/providers/apache/hdfs/docs/index.rst
index 6b6979fe7b4..1656ba5093d 100644
--- a/providers/apache/hdfs/docs/index.rst
+++ b/providers/apache/hdfs/docs/index.rst
@@ -85,19 +85,19 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
==================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
==================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``hdfs[avro,dataframe,kerberos]`` ``>=2.5.4; python_version <
"3.12"``
``hdfs[avro,dataframe,kerberos]`` ``>=2.7.3; python_version >=
"3.12"``
``fastavro`` ``>=1.10.0; python_version >=
"3.13" and python_version < "3.14"``
``fastavro`` ``>=1.12.1; python_version >=
"3.14"``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
==================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
Downloading official packages
-----------------------------
diff --git a/providers/apache/hdfs/pyproject.toml
b/providers/apache/hdfs/pyproject.toml
index 2907dd03aba..fa9baec7f7e 100644
--- a/providers/apache/hdfs/pyproject.toml
+++ b/providers/apache/hdfs/pyproject.toml
@@ -65,9 +65,18 @@ dependencies = [
'hdfs[avro,dataframe,kerberos]>=2.7.3;python_version>="3.12"',
'fastavro>=1.10.0; python_version>="3.13" and python_version<"3.14"',
'fastavro>=1.12.1; python_version>="3.14"',
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
]
[dependency-groups]
diff --git a/providers/apache/hive/README.rst b/providers/apache/hive/README.rst
index e07ecc84053..c2daf9e4d44 100644
--- a/providers/apache/hive/README.rst
+++ b/providers/apache/hive/README.rst
@@ -50,19 +50,19 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
``hmsclient`` ``>=0.1.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``pyhive[hive_pure_sasl]`` ``>=0.7.0``
``jmespath`` ``>=0.7.0``
-==========================================
=================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/apache/hive/docs/index.rst
b/providers/apache/hive/docs/index.rst
index 58da4e77be1..83982387686 100644
--- a/providers/apache/hive/docs/index.rst
+++ b/providers/apache/hive/docs/index.rst
@@ -99,19 +99,19 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
``hmsclient`` ``>=0.1.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``pyhive[hive_pure_sasl]`` ``>=0.7.0``
``jmespath`` ``>=0.7.0``
-==========================================
=================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/apache/hive/pyproject.toml
b/providers/apache/hive/pyproject.toml
index 34e41892cfb..50f395028e8 100644
--- a/providers/apache/hive/pyproject.toml
+++ b/providers/apache/hive/pyproject.toml
@@ -63,9 +63,18 @@ dependencies = [
"apache-airflow-providers-common-compat>=1.12.0",
"apache-airflow-providers-common-sql>=1.32.0",
"hmsclient>=0.1.0",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
"pyhive[hive_pure_sasl]>=0.7.0",
"jmespath>=0.7.0",
]
diff --git a/providers/common/sql/docs/index.rst
b/providers/common/sql/docs/index.rst
index ed9ac1aa46f..ec33b3329e5 100644
--- a/providers/common/sql/docs/index.rst
+++ b/providers/common/sql/docs/index.rst
@@ -141,10 +141,10 @@ Install them when installing from PyPI. For example:
pip install apache-airflow-providers-common-sql[pandas]
-==================
=======================================================================================================================================================================
+==================
================================================================================================================================================================================
Extra Dependencies
-==================
=======================================================================================================================================================================
-``pandas`` ``pandas[sql-other]>=2.1.2; python_version <"3.13"``,
``pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"``,
``pandas>=2.3.3; python_version >="3.14"``
+==================
================================================================================================================================================================================
+``pandas`` ``pandas[sql-other]>=2.1.2,<3; python_version <"3.13"``,
``pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"``,
``pandas>=2.3.3,<3; python_version >="3.14"``
``openlineage`` ``apache-airflow-providers-openlineage``
``polars`` ``polars>=1.26.0``
``sqlalchemy`` ``sqlalchemy>=1.4.54``
@@ -152,7 +152,7 @@ Extra Dependencies
``datafusion`` ``datafusion>=50.0.0,<52.0.0``
``pyiceberg-core`` ``pyiceberg-core>=0.8.0``
``apache.iceberg`` ``apache-airflow-providers-apache-iceberg``
-==================
=======================================================================================================================================================================
+==================
================================================================================================================================================================================
Downloading official packages
-----------------------------
diff --git a/providers/common/sql/pyproject.toml
b/providers/common/sql/pyproject.toml
index 5d93bb5d12a..d0d4541d587 100644
--- a/providers/common/sql/pyproject.toml
+++ b/providers/common/sql/pyproject.toml
@@ -72,11 +72,20 @@ dependencies = [
# Any change in the dependencies is preserved when the file is regenerated
[project.optional-dependencies]
"pandas" = [
- 'pandas[sql-other]>=2.1.2; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas[sql-other]>=2.1.2,<3; python_version <"3.13"',
# Technically - we should add "sql-other" here as well, but this will only
be possible when we move
# to sqlalchemy 2+ TODO(potiuk): do so.
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
]
"openlineage" = [
"apache-airflow-providers-openlineage"
diff --git a/providers/databricks/README.rst b/providers/databricks/README.rst
index 293fc4609b7..494cd6be7e4 100644
--- a/providers/databricks/README.rst
+++ b/providers/databricks/README.rst
@@ -50,9 +50,9 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
==================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
==================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.13.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
@@ -60,13 +60,13 @@ PIP package Version required
``databricks-sql-connector`` ``>=4.4.0``
``aiohttp`` ``>=3.14.0,<4``
``mergedeep`` ``>=1.3.4``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``pyarrow`` ``>=16.1.0; python_version <
"3.13"``
``pyarrow`` ``>=18.0.0; python_version >=
"3.13" and python_version < "3.14"``
``pyarrow`` ``>=22.0.0; python_version >=
"3.14"``
-==========================================
==================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/databricks/docs/index.rst
b/providers/databricks/docs/index.rst
index cbfc357c735..55835fb43c0 100644
--- a/providers/databricks/docs/index.rst
+++ b/providers/databricks/docs/index.rst
@@ -98,9 +98,9 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
==================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
==================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.13.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
@@ -108,13 +108,13 @@ PIP package Version
required
``databricks-sql-connector`` ``>=4.4.0``
``aiohttp`` ``>=3.14.0,<4``
``mergedeep`` ``>=1.3.4``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``pyarrow`` ``>=16.1.0; python_version <
"3.13"``
``pyarrow`` ``>=18.0.0; python_version >=
"3.13" and python_version < "3.14"``
``pyarrow`` ``>=22.0.0; python_version >=
"3.14"``
-==========================================
==================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/databricks/pyproject.toml
b/providers/databricks/pyproject.toml
index 40fac68ee2b..74594af076d 100644
--- a/providers/databricks/pyproject.toml
+++ b/providers/databricks/pyproject.toml
@@ -66,9 +66,18 @@ dependencies = [
"databricks-sql-connector>=4.4.0",
"aiohttp>=3.14.0, <4",
"mergedeep>=1.3.4",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
"pyarrow>=16.1.0; python_version < '3.13'",
"pyarrow>=18.0.0; python_version >= '3.13' and python_version < '3.14'",
"pyarrow>=22.0.0; python_version >= '3.14'",
diff --git a/providers/exasol/README.rst b/providers/exasol/README.rst
index c74b2a48367..424978788c4 100644
--- a/providers/exasol/README.rst
+++ b/providers/exasol/README.rst
@@ -50,17 +50,17 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
``pyexasol`` ``>=0.26.0,<2``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
=================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
Optional dependencies
----------------------
diff --git a/providers/exasol/docs/index.rst b/providers/exasol/docs/index.rst
index 13cb541eb12..3c01a40238b 100644
--- a/providers/exasol/docs/index.rst
+++ b/providers/exasol/docs/index.rst
@@ -95,17 +95,17 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
``pyexasol`` ``>=0.26.0,<2``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
=================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
Optional dependencies
---------------------
diff --git a/providers/exasol/pyproject.toml b/providers/exasol/pyproject.toml
index 636109b9840..7c30e390498 100644
--- a/providers/exasol/pyproject.toml
+++ b/providers/exasol/pyproject.toml
@@ -65,9 +65,18 @@ dependencies = [
# Capped to 1.x: pyexasol 2.x ships stricter types that break the exasol
hook.
# Remove the cap after migrating; tracked at
https://github.com/apache/airflow/issues/69123
"pyexasol>=0.26.0,<2",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
]
# The optional dependencies should be modified in place in the generated file
diff --git a/providers/google/README.rst b/providers/google/README.rst
index 015137fe2d6..bd1c048e1a7 100644
--- a/providers/google/README.rst
+++ b/providers/google/README.rst
@@ -57,9 +57,9 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
==================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
==================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.13.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
@@ -125,9 +125,9 @@ PIP package Version required
``httpx`` ``>=0.25.0``
``looker-sdk`` ``>=22.4.0,!=24.18.0``
``pandas-gbq`` ``>=0.7.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``proto-plus`` ``>=1.26.0``
``pyarrow`` ``>=18.0.0; python_version <
"3.14"``
``pyarrow`` ``>=22.0.0; python_version >=
"3.14"``
@@ -138,7 +138,7 @@ PIP package Version required
``tenacity`` ``>=8.3.0``
``immutabledict`` ``>=4.2.0``
``types-protobuf`` ``>=5.27.0,!=5.29.1.20250402``
-==========================================
==================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/google/docs/index.rst b/providers/google/docs/index.rst
index 2863bdaaea8..05c764f3c1c 100644
--- a/providers/google/docs/index.rst
+++ b/providers/google/docs/index.rst
@@ -110,9 +110,9 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
==================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
==================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.13.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
@@ -178,9 +178,9 @@ PIP package Version required
``httpx`` ``>=0.25.0``
``looker-sdk`` ``>=22.4.0,!=24.18.0``
``pandas-gbq`` ``>=0.7.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``proto-plus`` ``>=1.26.0``
``pyarrow`` ``>=18.0.0; python_version <
"3.14"``
``pyarrow`` ``>=22.0.0; python_version >=
"3.14"``
@@ -191,7 +191,7 @@ PIP package Version required
``tenacity`` ``>=8.3.0``
``immutabledict`` ``>=4.2.0``
``types-protobuf`` ``>=5.27.0,!=5.29.1.20250402``
-==========================================
==================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/google/pyproject.toml b/providers/google/pyproject.toml
index 1e9a606d237..e750891904d 100644
--- a/providers/google/pyproject.toml
+++ b/providers/google/pyproject.toml
@@ -134,9 +134,18 @@ dependencies = [
# See https://github.com/looker-open-source/sdk-codegen/issues/1518
"looker-sdk>=22.4.0,!=24.18.0",
"pandas-gbq>=0.7.0",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
"proto-plus>=1.26.0",
# Used to write parquet files by BaseSqlToGCSOperator
"pyarrow>=18.0.0; python_version < '3.14'",
diff --git a/providers/papermill/README.rst b/providers/papermill/README.rst
index 83033141bd4..d694e6cbcf7 100644
--- a/providers/papermill/README.rst
+++ b/providers/papermill/README.rst
@@ -50,19 +50,19 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.8.0``
``papermill[all]`` ``>=2.6.0``
``scrapbook[all]`` ``>=0.5.0``
``ipykernel`` ``>=6.29.4``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``nbconvert`` ``>=7.16.1``
-==========================================
=================================================================
+==========================================
====================================================================
The changelog for the provider package can be found in the
`changelog
<https://airflow.apache.org/docs/apache-airflow-providers-papermill/3.13.1/changelog.html>`_.
diff --git a/providers/papermill/docs/index.rst
b/providers/papermill/docs/index.rst
index 72c162cf6fd..00618cd5b01 100644
--- a/providers/papermill/docs/index.rst
+++ b/providers/papermill/docs/index.rst
@@ -97,19 +97,19 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.8.0``
``papermill[all]`` ``>=2.6.0``
``scrapbook[all]`` ``>=0.5.0``
``ipykernel`` ``>=6.29.4``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``nbconvert`` ``>=7.16.1``
-==========================================
=================================================================
+==========================================
====================================================================
Downloading official packages
-----------------------------
diff --git a/providers/papermill/pyproject.toml
b/providers/papermill/pyproject.toml
index 37d6fb5ab28..32ed9c96a91 100644
--- a/providers/papermill/pyproject.toml
+++ b/providers/papermill/pyproject.toml
@@ -64,9 +64,18 @@ dependencies = [
"papermill[all]>=2.6.0",
"scrapbook[all]>=0.5.0",
"ipykernel>=6.29.4",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
"nbconvert>=7.16.1",
]
diff --git a/providers/postgres/docs/index.rst
b/providers/postgres/docs/index.rst
index a236c0a4afc..59a91c40b19 100644
--- a/providers/postgres/docs/index.rst
+++ b/providers/postgres/docs/index.rst
@@ -143,18 +143,18 @@ Install them when installing from PyPI. For example:
pip install apache-airflow-providers-postgres[amazon]
-===================
============================================================================================================================================================
+===================
=====================================================================================================================================================================
Extra Dependencies
-===================
============================================================================================================================================================
+===================
=====================================================================================================================================================================
``amazon`` ``apache-airflow-providers-amazon>=2.6.0``
``asyncpg`` ``asyncpg>=0.30.0``
``microsoft.azure`` ``apache-airflow-providers-microsoft-azure>=12.8.0``
``openlineage`` ``apache-airflow-providers-openlineage``
-``pandas`` ``pandas>=2.1.2; python_version <"3.13"``,
``pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"``,
``pandas>=2.3.3; python_version >="3.14"``
+``pandas`` ``pandas>=2.1.2,<3; python_version <"3.13"``,
``pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"``,
``pandas>=2.3.3,<3; python_version >="3.14"``
``polars`` ``polars>=1.26.0``
``psycopg2`` ``psycopg2-binary>=2.9.9; python_version < '3.13'``,
``psycopg2-binary>=2.9.10; python_version >= '3.13'``
``sqlalchemy`` ``sqlalchemy>=1.4.54``
-===================
============================================================================================================================================================
+===================
=====================================================================================================================================================================
Downloading official packages
-----------------------------
diff --git a/providers/postgres/pyproject.toml
b/providers/postgres/pyproject.toml
index 1898cd5dc40..cf16e503e15 100644
--- a/providers/postgres/pyproject.toml
+++ b/providers/postgres/pyproject.toml
@@ -93,9 +93,18 @@ dependencies = [
"apache-airflow-providers-openlineage"
]
"pandas" = [
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
]
"polars" = [
"polars>=1.26.0"
diff --git a/providers/presto/README.rst b/providers/presto/README.rst
index 1d4639e04a3..c03a6a6f85e 100644
--- a/providers/presto/README.rst
+++ b/providers/presto/README.rst
@@ -50,19 +50,19 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
``presto-python-client`` ``>=0.8.4``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``psycopg2-binary`` ``>=2.9.9; python_version <
"3.13"``
``psycopg2-binary`` ``>=2.9.10; python_version >=
"3.13"``
-==========================================
=================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/presto/docs/index.rst b/providers/presto/docs/index.rst
index 5730e6a1ab4..444e957c410 100644
--- a/providers/presto/docs/index.rst
+++ b/providers/presto/docs/index.rst
@@ -98,19 +98,19 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
``presto-python-client`` ``>=0.8.4``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``psycopg2-binary`` ``>=2.9.9; python_version <
"3.13"``
``psycopg2-binary`` ``>=2.9.10; python_version >=
"3.13"``
-==========================================
=================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/presto/pyproject.toml b/providers/presto/pyproject.toml
index a00507c407e..5ca2ef6590a 100644
--- a/providers/presto/pyproject.toml
+++ b/providers/presto/pyproject.toml
@@ -63,9 +63,18 @@ dependencies = [
"apache-airflow-providers-common-compat>=1.12.0",
"apache-airflow-providers-common-sql>=1.32.0",
"presto-python-client>=0.8.4",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
'psycopg2-binary>=2.9.9; python_version < "3.13"',
'psycopg2-binary>=2.9.10; python_version >= "3.13"',
]
diff --git a/providers/salesforce/README.rst b/providers/salesforce/README.rst
index 38e0e084e0b..4fe9041a2f7 100644
--- a/providers/salesforce/README.rst
+++ b/providers/salesforce/README.rst
@@ -50,16 +50,16 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.10.1``
``simple-salesforce`` ``>=1.0.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
=================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
The changelog for the provider package can be found in the
`changelog
<https://airflow.apache.org/docs/apache-airflow-providers-salesforce/5.14.1/changelog.html>`_.
diff --git a/providers/salesforce/docs/index.rst
b/providers/salesforce/docs/index.rst
index 6e541d4d407..89040f6fe16 100644
--- a/providers/salesforce/docs/index.rst
+++ b/providers/salesforce/docs/index.rst
@@ -97,16 +97,16 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.10.1``
``simple-salesforce`` ``>=1.0.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
=================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
Downloading official packages
-----------------------------
diff --git a/providers/salesforce/pyproject.toml
b/providers/salesforce/pyproject.toml
index d16ff50b679..c5dd164ed4c 100644
--- a/providers/salesforce/pyproject.toml
+++ b/providers/salesforce/pyproject.toml
@@ -62,9 +62,18 @@ dependencies = [
"apache-airflow>=2.11.0",
"apache-airflow-providers-common-compat>=1.10.1",
"simple-salesforce>=1.0.0",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
]
[dependency-groups]
diff --git a/providers/snowflake/README.rst b/providers/snowflake/README.rst
index 35393f30deb..c0ad40bbbab 100644
--- a/providers/snowflake/README.rst
+++ b/providers/snowflake/README.rst
@@ -56,9 +56,9 @@ PIP package Version required
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``pyarrow`` ``>=16.1.0; python_version <
"3.13"``
``pyarrow`` ``>=18.0.0; python_version >=
"3.13" and python_version < "3.14"``
``pyarrow`` ``>=22.0.0; python_version >=
"3.14"``
diff --git a/providers/snowflake/docs/index.rst
b/providers/snowflake/docs/index.rst
index c8e5bac2553..41384f3146f 100644
--- a/providers/snowflake/docs/index.rst
+++ b/providers/snowflake/docs/index.rst
@@ -105,9 +105,9 @@ PIP package Version required
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``pyarrow`` ``>=16.1.0; python_version <
"3.13"``
``pyarrow`` ``>=18.0.0; python_version >=
"3.13" and python_version < "3.14"``
``pyarrow`` ``>=22.0.0; python_version >=
"3.14"``
diff --git a/providers/snowflake/pyproject.toml
b/providers/snowflake/pyproject.toml
index 2387daeaf74..0a012407064 100644
--- a/providers/snowflake/pyproject.toml
+++ b/providers/snowflake/pyproject.toml
@@ -62,9 +62,18 @@ dependencies = [
"apache-airflow>=2.11.0",
"apache-airflow-providers-common-compat>=1.12.0",
"apache-airflow-providers-common-sql>=1.32.0",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
"pyarrow>=16.1.0; python_version < '3.13'",
"pyarrow>=18.0.0; python_version >= '3.13' and python_version < '3.14'",
"pyarrow>=22.0.0; python_version >= '3.14'",
diff --git a/providers/trino/README.rst b/providers/trino/README.rst
index 18560ebd45a..de67e5360ed 100644
--- a/providers/trino/README.rst
+++ b/providers/trino/README.rst
@@ -50,17 +50,17 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``trino`` ``>=0.319.0``
-==========================================
=================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/trino/docs/index.rst b/providers/trino/docs/index.rst
index fc9ae7cbd8e..0dc95488ffd 100644
--- a/providers/trino/docs/index.rst
+++ b/providers/trino/docs/index.rst
@@ -98,17 +98,17 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.12.0``
``apache-airflow-providers-common-sql`` ``>=1.32.0``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
``trino`` ``>=0.319.0``
-==========================================
=================================================================
+==========================================
====================================================================
Optional cross provider package dependencies
--------------------------------------------
diff --git a/providers/trino/pyproject.toml b/providers/trino/pyproject.toml
index f550d172bcc..1a3b384b2cd 100644
--- a/providers/trino/pyproject.toml
+++ b/providers/trino/pyproject.toml
@@ -62,9 +62,18 @@ dependencies = [
"apache-airflow>=2.11.0",
"apache-airflow-providers-common-compat>=1.12.0",
"apache-airflow-providers-common-sql>=1.32.0",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
"trino>=0.319.0",
]
diff --git a/providers/weaviate/README.rst b/providers/weaviate/README.rst
index 637d0ddbc14..60f119c1d55 100644
--- a/providers/weaviate/README.rst
+++ b/providers/weaviate/README.rst
@@ -50,17 +50,17 @@ The package supports the following python versions:
3.10,3.11,3.12,3.13,3.14
Requirements
------------
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.8.0``
``httpx`` ``>=0.25.0``
``weaviate-client`` ``>=4.16.0,!=4.16.7``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
=================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
The changelog for the provider package can be found in the
`changelog
<https://airflow.apache.org/docs/apache-airflow-providers-weaviate/3.4.0/changelog.html>`_.
diff --git a/providers/weaviate/docs/index.rst
b/providers/weaviate/docs/index.rst
index 06a443cdb59..ea7b313b7cf 100644
--- a/providers/weaviate/docs/index.rst
+++ b/providers/weaviate/docs/index.rst
@@ -99,17 +99,17 @@ Requirements
The minimum Apache Airflow version supported by this provider distribution is
``2.11.0``.
-==========================================
=================================================================
+==========================================
====================================================================
PIP package Version required
-==========================================
=================================================================
+==========================================
====================================================================
``apache-airflow`` ``>=2.11.0``
``apache-airflow-providers-common-compat`` ``>=1.8.0``
``httpx`` ``>=0.25.0``
``weaviate-client`` ``>=4.16.0,!=4.16.7``
-``pandas`` ``>=2.1.2; python_version <
"3.13"``
-``pandas`` ``>=2.2.3; python_version >=
"3.13" and python_version < "3.14"``
-``pandas`` ``>=2.3.3; python_version >=
"3.14"``
-==========================================
=================================================================
+``pandas`` ``>=2.1.2,<3; python_version <
"3.13"``
+``pandas`` ``>=2.2.3,<3; python_version >=
"3.13" and python_version < "3.14"``
+``pandas`` ``>=2.3.3,<3; python_version >=
"3.14"``
+==========================================
====================================================================
Downloading official packages
-----------------------------
diff --git a/providers/weaviate/pyproject.toml
b/providers/weaviate/pyproject.toml
index f71f91ace82..4152b48fc6f 100644
--- a/providers/weaviate/pyproject.toml
+++ b/providers/weaviate/pyproject.toml
@@ -63,9 +63,18 @@ dependencies = [
"apache-airflow-providers-common-compat>=1.8.0",
"httpx>=0.25.0",
"weaviate-client>=4.16.0,!=4.16.7",
- 'pandas>=2.1.2; python_version <"3.13"',
- 'pandas>=2.2.3; python_version >="3.13" and python_version <"3.14"',
- 'pandas>=2.3.3; python_version >="3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.1.2,<3; python_version <"3.13"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.2.3,<3; python_version >="3.13" and python_version <"3.14"',
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ 'pandas>=2.3.3,<3; python_version >="3.14"',
]
[dependency-groups]
diff --git a/task-sdk/pyproject.toml b/task-sdk/pyproject.toml
index bb8f7e2c3f4..6c540ee3bc9 100644
--- a/task-sdk/pyproject.toml
+++ b/task-sdk/pyproject.toml
@@ -224,9 +224,18 @@ dev = [
"apache-airflow-providers-common-sql",
"apache-airflow-providers-standard",
"apache-airflow-devel-common",
- "pandas>=2.1.2; python_version <\"3.13\"",
- "pandas>=2.2.3; python_version >=\"3.13\" and python_version <\"3.14\"",
- "pandas>=2.3.3; python_version >=\"3.14\""
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ "pandas>=2.1.2,<3; python_version <\"3.13\"",
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ "pandas>=2.2.3,<3; python_version >=\"3.13\" and python_version <\"3.14\"",
+ # pandas 3 changes the dtypes a DataFrame reads back with, so DataFrame
XComs do not
+ # round trip; capped until pandas 3 support lands.
+ # Tracked at https://github.com/apache/airflow/pull/70558
+ "pandas>=2.3.3,<3; python_version >=\"3.14\""
]
docs = [
"apache-airflow-devel-common[docs]",
diff --git a/task-sdk/src/airflow/sdk/serde/serializers/pandas.py
b/task-sdk/src/airflow/sdk/serde/serializers/pandas.py
index 72a2e818a11..a3562f5af47 100644
--- a/task-sdk/src/airflow/sdk/serde/serializers/pandas.py
+++ b/task-sdk/src/airflow/sdk/serde/serializers/pandas.py
@@ -22,11 +22,49 @@ from typing import TYPE_CHECKING
from airflow.sdk.module_loading import qualname
# lazy loading for performance reasons
+#
+# ``pandas.core.frame.DataFrame`` is what a DataFrame qualifies as under
pandas 2. Under
+# pandas 3 the public classes moved to the ``pandas`` namespace, so the same
object
+# qualifies as ``pandas.DataFrame``. That name is registered too, but *only*
so that the
+# registry dispatches here and this module can refuse it with an actionable
message --
+# without it the caller gets serde's generic "cannot serialize object of type"
instead.
+# See ``_reject_pandas_3`` below.
serializers = [
"pandas.core.frame.DataFrame",
+ "pandas.DataFrame",
]
deserializers = serializers
+# First pandas major version whose DataFrame XComs this version of Airflow
does not support.
+_FIRST_UNSUPPORTED_PANDAS_MAJOR = 3
+
+
+def _reject_pandas_3(pd) -> None:
+ """
+ Refuse a DataFrame XCom when the installed pandas is 3 or newer.
+
+ pandas is an optional dependency, so the ``<3`` constraint Airflow
declares cannot be
+ relied on: a deployment may install pandas 3 directly, or pull it in
through another
+ package. This is the runtime half of that constraint.
+
+ The failure is deliberate rather than best-effort. Under pandas 3 a
DataFrame round
+ trips through parquet with different dtypes than under pandas 2 -- an
``object`` column
+ comes back as ``str`` and missing values as ``nan`` rather than ``None``
-- so silently
+ accepting the value would hand tasks data that differs from what was
written, with no
+ signal. Failing on the write is the louder and more recoverable half of
that.
+ """
+ major = int(pd.__version__.split(".")[0])
+ if major < _FIRST_UNSUPPORTED_PANDAS_MAJOR:
+ return
+ raise RuntimeError(
+ f"DataFrame XComs are not supported with pandas {pd.__version__}: this
version of "
+ f"Airflow supports pandas < {_FIRST_UNSUPPORTED_PANDAS_MAJOR} for XCom
serialization. "
+ "Pin pandas < 3 in the environment that runs your tasks, or avoid
passing DataFrames "
+ "through XCom. Moving to pandas 3 is a deliberate migration -- its
DataFrames read "
+ "back with different dtypes -- and Airflow will enable it in a future
release."
+ )
+
+
if TYPE_CHECKING:
import pandas as pd
@@ -43,6 +81,8 @@ def serialize(o: object) -> tuple[U, str, int, bool]:
if not isinstance(o, pd.DataFrame):
return "", "", 0, False
+ _reject_pandas_3(pd)
+
# for now, we *always* serialize into in memory
# until we have a generic backend that manages
# sinks
@@ -62,6 +102,11 @@ def deserialize(cls: type, version: int, data: object) ->
pd.DataFrame:
if cls is not pd.DataFrame:
raise TypeError(f"do not know how to deserialize {qualname(cls)}")
+ # Reading is refused too, not just writing: a payload written under pandas
2 comes back
+ # with pandas 3 dtypes, so the task would silently receive different data
than was
+ # pushed. An error is the safer outcome.
+ _reject_pandas_3(pd)
+
if not isinstance(data, str):
raise TypeError(f"serialized {qualname(cls)} has wrong data type
{type(data)}")
diff --git a/task-sdk/tests/task_sdk/serde/test_serializers.py
b/task-sdk/tests/task_sdk/serde/test_serializers.py
index 7e2ade64ebd..8ef21272194 100644
--- a/task-sdk/tests/task_sdk/serde/test_serializers.py
+++ b/task-sdk/tests/task_sdk/serde/test_serializers.py
@@ -286,6 +286,34 @@ class TestSerializers:
assert serialize(123) == ("", "", 0, False)
+ def test_pandas_3_dataframe_name_is_registered(self):
+ """The pandas 3 qualname must be registered, or the guard below is
unreachable.
+
+ Under pandas 3 a DataFrame qualifies as ``pandas.DataFrame``. serde
dispatches on
+ that name, so if it is absent from the registry the caller gets the
generic
+ "cannot serialize object of type" from serde and never reaches this
module.
+ """
+ from airflow.sdk.serde.serializers.pandas import deserializers,
serializers
+
+ assert "pandas.DataFrame" in serializers
+ assert "pandas.core.frame.DataFrame" in deserializers
+
+ @pytest.mark.parametrize("version", ["3.0.0", "3.0.5", "4.1.2"])
+ def test_pandas_3_dataframe_xcom_is_refused(self, monkeypatch, version):
+ """pandas >= 3 must fail loudly on both write and read, not round trip
silently."""
+ import pandas as pd
+
+ from airflow.sdk.serde.serializers import pandas as pandas_serializer
+
+ monkeypatch.setattr(pd, "__version__", version)
+ df = pd.DataFrame(data={"col1": [1, 2], "col2": [3, 4]})
+
+ with pytest.raises(RuntimeError, match=rf"DataFrame XComs are not
supported with pandas {version}"):
+ pandas_serializer.serialize(df)
+
+ with pytest.raises(RuntimeError, match=r"DataFrame XComs are not
supported with pandas"):
+ pandas_serializer.deserialize(pd.DataFrame, 1, "")
+
@pytest.mark.parametrize(
("klass", "version", "data", "msg"),
[
diff --git a/uv.lock b/uv.lock
index 61999ed455d..b61f10965f7 100644
--- a/uv.lock
+++ b/uv.lock
@@ -3180,9 +3180,9 @@ requires-dist = [
{ name = "jsonpath-ng", specifier = ">=1.5.3" },
{ name = "lxml", marker = "python_full_version < '3.13' and extra ==
'python3-saml'", specifier = ">=6.0.0" },
{ name = "marshmallow", specifier = ">=3" },
- { name = "pandas", marker = "python_full_version == '3.13.*' and extra ==
'pandas'", specifier = ">=2.2.3" },
- { name = "pandas", marker = "python_full_version < '3.13' and extra ==
'pandas'", specifier = ">=2.1.2" },
- { name = "pandas", marker = "python_full_version >= '3.14' and extra ==
'pandas'", specifier = ">=2.3.3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*' and extra ==
'pandas'", specifier = ">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version < '3.13' and extra ==
'pandas'", specifier = ">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14' and extra ==
'pandas'", specifier = ">=2.3.3,<3" },
{ name = "pyathena", specifier = ">=3.10.0" },
{ name = "python3-saml", marker = "python_full_version < '3.13' and extra
== 'python3-saml'", specifier = ">=1.16.0" },
{ name = "redshift-connector", specifier = ">=2.1.3" },
@@ -3488,9 +3488,9 @@ requires-dist = [
{ name = "fastavro", marker = "python_full_version >= '3.14'", specifier =
">=1.12.1" },
{ name = "hdfs", extras = ["avro", "dataframe", "kerberos"], marker =
"python_full_version < '3.12'", specifier = ">=2.5.4" },
{ name = "hdfs", extras = ["avro", "dataframe", "kerberos"], marker =
"python_full_version >= '3.12'", specifier = ">=2.7.3" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
]
[package.metadata.requires-dev]
@@ -3578,9 +3578,9 @@ requires-dist = [
{ name = "hmsclient", specifier = ">=0.1.0" },
{ name = "jmespath", specifier = ">=0.7.0" },
{ name = "kerberos", marker = "sys_platform != 'win32' and extra ==
'gssapi'", specifier = ">=1.3.0" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "pyhive", extras = ["hive-pure-sasl"], specifier = ">=0.7.0" },
{ name = "sqlalchemy", marker = "extra == 'sqlalchemy'", specifier =
">=1.4.54" },
{ name = "winkerberos", marker = "sys_platform == 'win32' and extra ==
'gssapi'", specifier = ">=0.7.0" },
@@ -4763,9 +4763,9 @@ requires-dist = [
{ name = "datafusion", marker = "extra == 'datafusion'", specifier =
">=50.0.0,<52.0.0" },
{ name = "methodtools", specifier = ">=0.4.7" },
{ name = "more-itertools", specifier = ">=9.0.0" },
- { name = "pandas", marker = "python_full_version == '3.13.*' and extra ==
'pandas'", specifier = ">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14' and extra ==
'pandas'", specifier = ">=2.3.3" },
- { name = "pandas", extras = ["sql-other"], marker = "python_full_version <
'3.13' and extra == 'pandas'", specifier = ">=2.1.2" },
+ { name = "pandas", marker = "python_full_version == '3.13.*' and extra ==
'pandas'", specifier = ">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14' and extra ==
'pandas'", specifier = ">=2.3.3,<3" },
+ { name = "pandas", extras = ["sql-other"], marker = "python_full_version <
'3.13' and extra == 'pandas'", specifier = ">=2.1.2,<3" },
{ name = "polars", marker = "extra == 'polars'", specifier = ">=1.26.0" },
{ name = "pyiceberg-core", marker = "extra == 'pyiceberg-core'", specifier
= ">=0.8.0" },
{ name = "sqlalchemy", marker = "extra == 'sqlalchemy'", specifier =
">=1.4.54" },
@@ -4877,9 +4877,9 @@ requires-dist = [
{ name = "fastavro", marker = "python_full_version >= '3.14' and extra ==
'avro'", specifier = ">=1.12.1" },
{ name = "fastavro", marker = "python_full_version < '3.14' and extra ==
'avro'", specifier = ">=1.9.0" },
{ name = "mergedeep", specifier = ">=1.3.4" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "pyarrow", marker = "python_full_version < '3.13'", specifier =
">=16.1.0" },
{ name = "pyarrow", marker = "python_full_version == '3.13.*'", specifier
= ">=18.0.0" },
{ name = "pyarrow", marker = "python_full_version >= '3.14'", specifier =
">=22.0.0" },
@@ -5241,9 +5241,9 @@ requires-dist = [
{ name = "apache-airflow", editable = "." },
{ name = "apache-airflow-providers-common-compat", editable =
"providers/common/compat" },
{ name = "apache-airflow-providers-common-sql", editable =
"providers/common/sql" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "pyexasol", specifier = ">=0.26.0,<2" },
{ name = "sqlalchemy", marker = "extra == 'sqlalchemy'", specifier =
">=1.4.54" },
]
@@ -5759,9 +5759,9 @@ requires-dist = [
{ name = "httpx", specifier = ">=0.25.0" },
{ name = "immutabledict", specifier = ">=4.2.0" },
{ name = "looker-sdk", specifier = ">=22.4.0,!=24.18.0" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "pandas-gbq", specifier = ">=0.7.0" },
{ name = "plyvel", marker = "python_full_version < '3.13' and extra ==
'leveldb'", specifier = ">=1.5.1" },
{ name = "proto-plus", specifier = ">=1.26.0" },
@@ -7085,9 +7085,9 @@ requires-dist = [
{ name = "apache-airflow-providers-common-compat", editable =
"providers/common/compat" },
{ name = "ipykernel", specifier = ">=6.29.4" },
{ name = "nbconvert", specifier = ">=7.16.1" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "papermill", extras = ["all"], specifier = ">=2.6.0" },
{ name = "scrapbook", extras = ["all"], specifier = ">=0.5.0" },
]
@@ -7253,9 +7253,9 @@ requires-dist = [
{ name = "apache-airflow-providers-openlineage", marker = "extra ==
'openlineage'", editable = "providers/openlineage" },
{ name = "asyncpg", specifier = ">=0.30.0" },
{ name = "asyncpg", marker = "extra == 'asyncpg'", specifier = ">=0.30.0"
},
- { name = "pandas", marker = "python_full_version == '3.13.*' and extra ==
'pandas'", specifier = ">=2.2.3" },
- { name = "pandas", marker = "python_full_version < '3.13' and extra ==
'pandas'", specifier = ">=2.1.2" },
- { name = "pandas", marker = "python_full_version >= '3.14' and extra ==
'pandas'", specifier = ">=2.3.3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*' and extra ==
'pandas'", specifier = ">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version < '3.13' and extra ==
'pandas'", specifier = ">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14' and extra ==
'pandas'", specifier = ">=2.3.3,<3" },
{ name = "polars", marker = "extra == 'polars'", specifier = ">=1.26.0" },
{ name = "psycopg", extras = ["binary"], marker = "python_full_version <
'3.14'", specifier = ">=3.2.9" },
{ name = "psycopg", extras = ["binary"], marker = "python_full_version >=
'3.14'", specifier = ">=3.3.3" },
@@ -7326,9 +7326,9 @@ requires-dist = [
{ name = "apache-airflow-providers-common-compat", editable =
"providers/common/compat" },
{ name = "apache-airflow-providers-common-sql", editable =
"providers/common/sql" },
{ name = "apache-airflow-providers-google", marker = "extra == 'google'",
editable = "providers/google" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "presto-python-client", specifier = ">=0.8.4" },
{ name = "psycopg2-binary", marker = "python_full_version < '3.13'",
specifier = ">=2.9.9" },
{ name = "psycopg2-binary", marker = "python_full_version >= '3.13'",
specifier = ">=2.9.10" },
@@ -7459,9 +7459,9 @@ docs = [
requires-dist = [
{ name = "apache-airflow", editable = "." },
{ name = "apache-airflow-providers-common-compat", editable =
"providers/common/compat" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "simple-salesforce", specifier = ">=1.0.0" },
]
@@ -7824,9 +7824,9 @@ requires-dist = [
{ name = "apache-airflow-providers-common-sql", editable =
"providers/common/sql" },
{ name = "apache-airflow-providers-microsoft-azure", marker = "extra ==
'microsoft-azure'", editable = "providers/microsoft/azure" },
{ name = "apache-airflow-providers-openlineage", marker = "extra ==
'openlineage'", editable = "providers/openlineage" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "pyarrow", marker = "python_full_version < '3.13'", specifier =
">=16.1.0" },
{ name = "pyarrow", marker = "python_full_version == '3.13.*'", specifier
= ">=18.0.0" },
{ name = "pyarrow", marker = "python_full_version >= '3.14'", specifier =
">=22.0.0" },
@@ -8159,9 +8159,9 @@ requires-dist = [
{ name = "apache-airflow-providers-common-sql", editable =
"providers/common/sql" },
{ name = "apache-airflow-providers-google", marker = "extra == 'google'",
editable = "providers/google" },
{ name = "apache-airflow-providers-openlineage", marker = "extra ==
'openlineage'", editable = "providers/openlineage" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "trino", specifier = ">=0.319.0" },
]
provides-extras = ["google", "openlineage"]
@@ -8297,9 +8297,9 @@ requires-dist = [
{ name = "apache-airflow", editable = "." },
{ name = "apache-airflow-providers-common-compat", editable =
"providers/common/compat" },
{ name = "httpx", specifier = ">=0.25.0" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
{ name = "weaviate-client", specifier = ">=4.16.0,!=4.16.7" },
]
@@ -9070,9 +9070,9 @@ dev = [
{ name = "apache-airflow-devel-common", editable = "devel-common" },
{ name = "apache-airflow-providers-common-sql", editable =
"providers/common/sql" },
{ name = "apache-airflow-providers-standard", editable =
"providers/standard" },
- { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2" },
- { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3" },
- { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3" },
+ { name = "pandas", marker = "python_full_version < '3.13'", specifier =
">=2.1.2,<3" },
+ { name = "pandas", marker = "python_full_version == '3.13.*'", specifier =
">=2.2.3,<3" },
+ { name = "pandas", marker = "python_full_version >= '3.14'", specifier =
">=2.3.3,<3" },
]
docs = [{ name = "apache-airflow-devel-common", extras = ["docs"], editable =
"devel-common" }]
mypy = [{ name = "apache-airflow-devel-common", extras = ["mypy"], editable =
"devel-common" }]
