raphaelauv commented on PR #41047: URL: https://github.com/apache/airflow/pull/41047#issuecomment-2269253971
`But we should assume that most people are at 2.7.1 + now` so most people suffer from this silent bug and should not trust soft_fail ( I had to remove soft_fail to all my sensors and add a ShortCircuitOperator checking the status of previous task and check if it's a `AirflowSensorTimeout` ) -> ```python from airflow.operators.empty import EmptyOperator from airflow.sensors.python import PythonSensor from airflow.operators.python import ShortCircuitOperator from airflow.utils.trigger_rule import TriggerRule from pendulum import today from airflow import DAG def a(): # return False # task_c will run raise ValueError("toto") # task_c will not run def b(error: str | None): if error == "AirflowSensorTimeout": return True else: return False def sav_error(context): exception_name = context["exception"].__class__.__name__ context.get('task_instance').xcom_push("error", exception_name) with DAG( dag_id='example', schedule_interval='0 0 * * *', start_date=today("UTC").add(days=-5)): a = PythonSensor(task_id="a", python_callable=a, on_failure_callback=sav_error, timeout=5, poke_interval=5) b = ShortCircuitOperator(task_id="b", python_callable=b, trigger_rule=TriggerRule.ALL_DONE, op_kwargs={ 'error': "{{ ti.xcom_pull(task_ids='a', key='error') }}"}) c = EmptyOperator(task_id="c") a >> b >> c ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org