yuqian90 commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098#issuecomment-582219010 > @yuqian90 I agree with what @ChrisEverling said [here](https://github.com/apache/airflow/pull/4182#discussion_r362862918), and with the folks who created the issue, "none failed" includes "all skipped", and the documentation backs that up. It took some debugging to find this extra condition, and I still don't understand the rationale behind it. > > Why not make the current trigger rule behave as documented, and add a new one to preserve the unexpected "none failed and not all skipped" behavior? @jared-martin Sure. I think continuing what @TV4Fun was originally doing in this PR is also fine. However, I was pointing out in my previous comments the side-effect of making this change when e.g. when join tasks with `none_failed` trigger_rule are nested in branching logic. If people think it's better to proceed with fixing the existing `none_failed` to match what the doc says, I would suggest we update the parts in the doc about using `none_failed` task to create join for `BranchPythonOperator`. It can no longer be a `none_failed` `DummyOperator`. It needs to be something like the following, where the join task checks the state of its parent `BranchPythonOperator` and skip itself if the parent is skipped. I would highly recommend making `create_join` a free function inside `airflow/operators/python.py` to make it more convenient for users of `BranchPythonOperator` and `ShortCircuitOperator`. ```python from airflow.models import DAG from airflow.operators.dummy_operator import DummyOperator from airflow.operators.python import BranchPythonOperator, PythonOperator from airflow.utils.dates import days_ago def create_join(branch_task_id, *args, **kwargs): """ :param branch_task_id: The task_id of the parent BranchPythonOperator task that created the branching logic. :type branch_task_id: str """ def python_callable(ti, **_): from airflow.utils.session import create_session from airflow.exceptions import AirflowSkipException from airflow.utils.state import State from airflow.models import TaskInstance with create_session() as session: branch_ti = session.query(TaskInstance).filter(TaskInstance.dag_id == ti.dag_id, TaskInstance.task_id == branch_task_id, TaskInstance.execution_date == ti.execution_date ).first() if not branch_ti: return if branch_ti.state == State.SKIPPED: raise AirflowSkipException("Skipping {} because parent BranchPythonOperator {} is skipped." .format(ti.task_id, branch_task_id)) return PythonOperator(trigger_rule="none_failed", python_callable=python_callable, *args, **kwargs) with DAG(dag_id="example_nested_branch_dag", start_date=days_ago(2), schedule_interval="@daily") as dag: branch_1 = BranchPythonOperator(task_id="branch_1", python_callable=lambda: "false_1") true_1 = DummyOperator(task_id="true_1") false_1 = DummyOperator(task_id="false_1") branch_2 = BranchPythonOperator(task_id="branch_2", python_callable=lambda: "true_2") true_2 = DummyOperator(task_id="true_2") false_2 = DummyOperator(task_id="false_2") join_2 = create_join(task_id="join_2", branch_task_id=branch_2.task_id) true_3 = DummyOperator(task_id="true_3") join_1 = create_join(task_id="join_1", branch_task_id=branch_1.task_id) branch_1 >> [true_1, false_1] true_1 >> branch_2 >> [true_2, false_2] >> join_2 >> true_3 >> join_1 false_1 >> join_1 ``` You can see the DAG now behaves the same as what it should have been before this PR: 
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
