vlieven opened a new issue, #36809:
URL: https://github.com/apache/airflow/issues/36809

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### If "Other Airflow 2 version" selected, which one?
   
   2.7.3
   
   ### What happened?
   
   If the `python_callable` used by a ShortCircuitOperator does not properly 
return a boolean, but instead crashes, the retry and downstream behavior is not 
deterministic. Either:
   - the task properly retries, succeeds and downstream tasks are triggered
   - the task does not properly retry, downstream tasks are marked as 
upstream_failed
   - the task does retry, is not marked as success, but downstream tasks are 
triggered
   
   ### What you think should happen instead?
   
   In case the callable crashes, it should be treated as a failure, and retry 
logic should be properly executed until the task completes successfully.
   
   ### How to reproduce
   
   As the behavior is not deterministic, I've created 10 parallel flows to 
demonstrate the issue. A crash is simulated by `exit(-1)`.
   
   This DAG file should demonstrate the broken behavior:
   
   ```python
   from airflow import DAG
   from airflow.models.taskinstance import TaskInstance
   from airflow.operators.empty import EmptyOperator
   from airflow.operators.python import ShortCircuitOperator
   from datetime import datetime, timedelta
   
   
   default_args = {
       "owner": "Test",
       "depends_on_past": False,
       "start_date": datetime.now() - timedelta(days=2),
       "retries": 1,
       "retry_delay": timedelta(seconds=5),
   }
   
   
   def python_callable(ti: TaskInstance):
       if ti.try_number % 2 == 1:
           exit(-1)
       else:
           return True
   
   
   with DAG(
       "test_shortcircuit",
       default_args=default_args,
       schedule_interval="@daily",
   ) as test_dag:
       for i in range(10):
           ShortCircuitOperator(
               task_id=f"short-{i}", python_callable=python_callable,
           ) >> EmptyOperator(
               task_id=f"empty-{i}"
           )
   
   ```
   
   The resulting grid view is the following:
   
   <img width="514" alt="Screenshot 2024-01-16 at 09 14 08" 
src="https://github.com/apache/airflow/assets/16362535/69fa9e39-ec0f-4daf-a7e7-163b670fc65c";>
   
   
   ### Operating System
   
   Amazon Linux 2
   
   ### Versions of Apache Airflow Providers
   
   This is using only the built-in operators
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   Running on a Kubernetes cluster
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to