ldacey opened a new issue, #53593:
URL: https://github.com/apache/airflow/issues/53593
### Apache Airflow version
3.0.3
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
I have many DAGs which incrementally download data by passing a `max_value`
XCom (modified timestamp for files, updated_at timestamp etc).
When I upgraded to Airflow 3.0.3 I noticed that this stopped working and
downgrading to 3.0.2 worked again. I don't have many details other than a few
logs:
Run 1:
> INFO - Pushing 'extracted_files' XCom with 10 files
> INFO - Pushing 'max_value' XCom: 2025-07-20 20:00:27+00:00 - dag_id:
campaign_extract_raw, task_id: extract_source_to_raw,
> run_id: scheduled__2025-07-10T04:02:00+00:00
Run 2 (XCom is pulling the same DAG ID and task ID)
> Attempting XCom pull - dag_id: campaign_extract_raw, task_id:
extract_source_to_raw
> INFO - XCom pull result (current run) for max_value: None
> INFO - XCom pull result (with prior dates) for max_value
> INFO - All XCom values for task: None
---
Another example where I logged anything I could think of to try to narrow
down the issue. The DAG was correctly pulling from the same DAG ID and task ID.
None was returned and the DAG fell back to the start_date (so attempted to
redownload the entire table...). This DAG is returning the max value from a
taskflow task, so the key is `return_value` instead:
> INFO - Cloning repository: bundle_name="gitlab-legacy": version=null:
> INFO - Filling up the DagBag from
/opt/airflow/dag_bundles/gitlab-legacy/tracking_repo/src/dags/
> INFO - get_previous_max_value called with task_id=extract_source_to_raw,
key=return_value:
> INFO - DAG params: {'since': None, 'until': None}:
> INFO - Current DAG ID: example_extract_raw:
> INFO - Using DAG ID: example_extract_raw, task ID: extract_source_to_raw:
> INFO - Attempting XCom pull for dag_id=example_extract_raw,
task_id=extract_source_to_raw, key=return_value, include_prior_dates=True:
> INFO - XCom pull result: None (type: <class 'NoneType'>):
> INFO - XCom pull returned None or empty value:
> INFO - Using DAG start date as fallback: 2017-12-31T18:30:00+00:00:
The next run After downgrading to Airflow 3.0.2 correctly found the XCom:
> INFO - Using DAG ID: example_extract_raw, task ID: extract_source_to_raw
> INFO - Attempting XCom pull for dag_id=example_extract_raw,
task_id=extract_source_to_raw, key=return_value, include_prior_dates=True
> INFO - XCom pull result: 2025-07-20T00:00:00 (type: <class 'str'>)
> INFO - Found previous max value: 2025-07-20T00:00:00
### What you think should happen instead?
`include_prior_dates` should find the XCom from within the same DAG. I
checked the xcom DB table and they are definitely there.
### How to reproduce
Upgrade to 3.0.3 and run a DAG with `include_prior_dates` retrieving a key
from within the same DAG.
### Operating System
Debian bookworm
### Versions of Apache Airflow Providers
+ apache-airflow-providers-cncf-kubernetes==10.6.1
+ apache-airflow-providers-common-compat==1.7.2
+ apache-airflow-providers-common-io==1.6.1
+ apache-airflow-providers-common-sql==1.27.3
+ apache-airflow-providers-fab==2.3.0
+ apache-airflow-providers-ftp==3.13.1
+ apache-airflow-providers-google==16.1.0
+ apache-airflow-providers-http==5.3.2
+ apache-airflow-providers-microsoft-azure==12.5.0
+ apache-airflow-providers-odbc==4.10.1
+ apache-airflow-providers-openlineage==2.5.0
+ apache-airflow-providers-postgres==6.2.1
+ apache-airflow-providers-salesforce==5.11.1
+ apache-airflow-providers-sftp==5.3.2
+ apache-airflow-providers-smtp==2.1.1
+ apache-airflow-providers-ssh==4.1.1
+ apache-airflow-providers-standard==1.4.1
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
I used the 1.18.0 Airflow Helm chart and tested locally with Docker compose.
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
