Berenger-Wooclap opened a new issue, #34754:
URL: https://github.com/apache/airflow/issues/34754
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
We are using Apache Airflow version 2.5.3, and we have a DAG in which
multiple tasks are executed sequentially, with the output of the first task
serving as input for the subsequent tasks. All these tasks are set to
TriggerRule.NONE_FAILED, meaning they should execute even if one of them fails.
However, we've encountered an issue where if the second task, which is
dynamic, is 'skipped' (i.e., it does not run), the following task does not
receive the XCom data from the first task. We have implemented a temporary
workaround by preventing the second task from being 'skipped,' but we are
looking for a permanent solution where the third task always receives XCom data
from the first task, even if the second task is 'skipped.'
### What you think should happen instead
I don't really know what is happening but i think the xcom is not propagate
for any reason.
### How to reproduce
Here is the part of the dag that create the tasks :
` tables_to_export = PythonOperator(
task_id="show_configuration",
python_callable=get_tables_to_export_from_params,
on_failure_callback=slack_fail_alert,
provide_context=True,
)
mongodb_exporter = WooclapKubernetesPodOperator.partial(
config=config,
image_pull_policy="Always",
service_account_name=service_account_name,
default_image_name=default_image_name,
on_failure_callback=slack_fail_alert,
name="mongodb-exporter-pod",
task_id="mongodb-exporter-task",
pool="mongo-db-pool-limit",
).expand(expand_args=tables_to_export.output)
mongo_dump_to_pg = WooclapKubernetesPodOperator.partial(
config=config,
image_pull_policy="Always",
service_account_name=service_account_name,
default_image_name=default_image_name,
on_failure_callback=slack_fail_alert,
name="mongo-dump-to-pg-pod",
task_id="mongo-dump-to-pg-task",
pool="woostats-db-pool-limit",
).expand(expand_args=tables_to_export.output)`
### Operating System
Airflow 2.5.3 deploy on kubernetes
### Versions of Apache Airflow Providers
Airflow 2.5.3
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
The xcom is done with postgres db if i'm not wrong
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]