jedcunningham opened a new pull request, #62686:
URL: https://github.com/apache/airflow/pull/62686
XCom values containing float('nan'), float('inf'), or float('-inf') caused
the database migration to silently corrupt data or fail outright when
upgrading. Three bugs were present across backends:
- Consecutive tokens (e.g. [NaN, NaN]) were only partially replaced, leaving
bare NaN/Infinity in the output and breaking the JSON cast.
- Infinity and -Infinity were not handled at all — only NaN was.
- Bare top-level values (a single NaN or Infinity, not inside a list or
dict) were not matched and passed through unconverted.
MySQL also had two bugs in the replacement query that caused it to produce
the wrong output (one of these was pre-existing from #57866).
An example of a failing Infinity in xcom:
```
2026-03-02T03:23:06.876232Z [info ] Running upgrade 9fc3fc5de720 ->
eed27faa34e3, Remove pickled data from xcom table. [alembic.runtime.migration]
loc=m
igration.py:621
Traceback (most recent call last):
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py",
line 1967, in _exec_single_context
self.dialect.do_execute(
File
"/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line
952, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.InvalidTextRepresentation: invalid input syntax for type json
DETAIL: Token "Infinity" is invalid.
CONTEXT: JSON data, line 1: Infinity
…
sqlalchemy.exc.DataError: (psycopg2.errors.InvalidTextRepresentation)
invalid input syntax for type json
DETAIL: Token "Infinity" is invalid.
CONTEXT: JSON data, line 1: Infinity
[SQL:
ALTER TABLE xcom
ALTER COLUMN value TYPE JSONB
USING CASE
WHEN value IS NOT NULL THEN CAST(CONVERT_FROM(value, 'UTF8')
AS JSONB)
ELSE NULL
END
]
(Background on this error at: https://sqlalche.me/e/20/9h9h)
```
A Dag that can be used to test:
```
from __future__ import annotations
try:
from airflow.sdk import DAG, task
except ImportError:
from airflow import DAG
from airflow.decorators import task
@task
def push_nan_values():
return {
"nan_value": float("nan"),
"nested": {
"stats": {
"count": 1.0,
"mean": 0.196,
"std": float("nan"),
},
"list_of_nans": [float("nan"), float("nan"), float("nan")],
},
"normal_value": 42,
}
@task
def push_inf_values():
return {
"inf_value": float("inf"),
"neg_inf_value": float("-inf"),
"nested": {
"stats": {
"count": 1.0,
"mean": 0.196,
"min": float("-inf"),
"max": float("inf"),
},
"list_of_infs": [float("inf"), float("-inf"), float("inf"),
float("-inf")],
},
"normal_value": 42,
}
@task
def push_top_level_nan():
return float("nan")
@task
def push_top_level_inf():
return float("inf")
with DAG(dag_id="nan_xcom", schedule=None):
push_nan_values()
with DAG(dag_id="infinity_xcom", schedule=None):
push_inf_values()
with DAG(dag_id="top_level_nan_xcom", schedule=None):
push_top_level_nan()
with DAG(dag_id="top_level_inf_xcom", schedule=None):
push_top_level_inf()
```
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes
Generated-by: Cursor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
