Berenger-Wooclap opened a new issue, #34754:
URL: https://github.com/apache/airflow/issues/34754

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   We are using Apache Airflow version 2.5.3, and we have a DAG in which 
multiple tasks are executed sequentially, with the output of the first task 
serving as input for the subsequent tasks. All these tasks are set to 
TriggerRule.NONE_FAILED, meaning they should execute even if one of them fails.
   
   However, we've encountered an issue where if the second task, which is 
dynamic, is 'skipped' (i.e., it does not run), the following task does not 
receive the XCom data from the first task. We have implemented a temporary 
workaround by preventing the second task from being 'skipped,' but we are 
looking for a permanent solution where the third task always receives XCom data 
from the first task, even if the second task is 'skipped.'
   
   ### What you think should happen instead
   
   I don't really know what is happening but i think the xcom is not propagate 
for any reason.
   
   ### How to reproduce
   
   Here is the part of the dag that create the tasks : 
   `  tables_to_export = PythonOperator(
           task_id="show_configuration",
           python_callable=get_tables_to_export_from_params,
           on_failure_callback=slack_fail_alert,
           provide_context=True,
       )
   
       mongodb_exporter = WooclapKubernetesPodOperator.partial(
           config=config,
           image_pull_policy="Always",
           service_account_name=service_account_name,
           default_image_name=default_image_name,
           on_failure_callback=slack_fail_alert,
           name="mongodb-exporter-pod",
           task_id="mongodb-exporter-task",
           pool="mongo-db-pool-limit",
       ).expand(expand_args=tables_to_export.output)
   
       mongo_dump_to_pg = WooclapKubernetesPodOperator.partial(
           config=config,
           image_pull_policy="Always",
           service_account_name=service_account_name,
           default_image_name=default_image_name,
           on_failure_callback=slack_fail_alert,
           name="mongo-dump-to-pg-pod",
           task_id="mongo-dump-to-pg-task",
           pool="woostats-db-pool-limit",
       ).expand(expand_args=tables_to_export.output)`
   
   ### Operating System
   
   Airflow 2.5.3 deploy on kubernetes
   
   ### Versions of Apache Airflow Providers
   
   Airflow 2.5.3 
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   The xcom is done with postgres db if i'm not wrong
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to