kacpermuda opened a new issue, #48927:
URL: https://github.com/apache/airflow/issues/48927

   ### Apache Airflow version
   
   main (development)
   
   ### If "Other Airflow 2 version" selected, which one?
   
   _No response_
   
   ### What happened?
   
   II've noticed that OpenLineage's listener is sending two FAIL events when 
task has some retries left but the problem is not OpenLineage-related - the 
listener_manager gets called twice. 
   
   The problem is that 
[this](https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/taskinstance.py#L3099)
 get_listener_manager().hook.on_task_instance_failed listener's call is done on 
the scheduler, exactly 
[here](https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/jobs/scheduler_job_runner.py#L890).
 This is because the task is falling into the if for tasks killed_externally, 
and the ti.handle_failure is called (and it has the listener's call inside).
   
   ### What you think should happen instead?
   
   I think that this simple task execution should not be treated as external 
task state change and for sure, the listener should be called once.
   
   ### How to reproduce
   
   1. Run this DAG on latest main.
   2. Look in the scheduler logs, you should find ERROR logs `Executor 
CeleryExecutor ... reported that the task instance ... finished with state 
success, but the task instance's state attribute is running` and then some logs 
about listener's on_task_instance_failed being called (on DEBUG level)
   
   ```
   import datetime as dt
   
   from airflow import DAG
   from airflow.providers.standard.operators.bash import BashOperator
   
   with DAG(
       dag_id="dag_failure_wait",
       start_date=dt.datetime(2024, 7, 3),
       schedule=None,
       catchup=False,
   ) as dag:
       task_failure = BashOperator(
           task_id="task_failure",
           bash_command="sleep 2 && exit 1;",
           retry_delay=1,
           retries=1,
       )
   ```
   
   ### Operating System
   
   MacOS
   
   ### Versions of Apache Airflow Providers
   
   latest main
   
   ### Deployment
   
   Virtualenv installation
   
   ### Deployment details
   
   Breeze, with LocalExecutor / CeleryExecutor (tested both).
   `breeze start-airflow --integration openlineage -b postgres`
   `breeze start-airflow --integration openlineage -b postgres 
--executor=CeleryExecutor`
   
   ### Anything else?
   
   Logs for CeleryExecutor run:
   
[log_celery_20250408_105902.txt](https://github.com/user-attachments/files/19647875/log_celery_20250408_105902.txt)
   
[log_scheduler_20250408_105917.txt](https://github.com/user-attachments/files/19647876/log_scheduler_20250408_105917.txt)
   
   Logs for LocalExecutor run:
   
[log_scheduler_20250408_110833.txt](https://github.com/user-attachments/files/19647906/log_scheduler_20250408_110833.txt)
   
   Logs for LocalExecutor without the OpenLineage integration:
   
[log_scheduler_20250408_111105.txt](https://github.com/user-attachments/files/19647938/log_scheduler_20250408_111105.txt)
   
   OL events received for the run (notice two FAIL events for first task run):
   
   
![Image](https://github.com/user-attachments/assets/7e264997-ddb7-4bbb-8e91-19b241c6486b)
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to