yuqian90 commented on a change in pull request #4182: [AIRFLOW-3336] Add new TriggerRule that will consider skipped ancestors as success URL: https://github.com/apache/airflow/pull/4182#discussion_r361067775
########## File path: airflow/ti_deps/deps/trigger_rule_dep.py ########## @@ -152,6 +152,11 @@ def _evaluate_trigger_rule( elif tr == TR.ONE_FAILED: if upstream_done and not (failed or upstream_failed): ti.set_state(State.SKIPPED, session) + elif tr == TR.NONE_FAILED: + if upstream_failed or failed: + ti.set_state(State.UPSTREAM_FAILED, session) + elif skipped == upstream: Review comment: @rmn36 @kaxil @ashb @Fokko I'm trying to understand the rationale behind this line. E.g. for a dag that looks like this, with just op1 >> op2. When we run the DAG, both op1 and op2 become skipped. The reason is op1 raises `AirflowSkipException`, so op1 is skipped. op2 is set to `none_failed`. But op2 is also skipped because this line checks `skipped == upstream` and marks op2 as skipped. However, the doc for "Trigger Rule" says: ```none_failed: all parents have not failed (failed or upstream_failed) i.e. all parents have succeeded or been skipped``` So if I interpret this correctly, when all of op2's upstream tasks are skipped, none of them have failed. So op2 should be success rather than skip. This may be a feature or a bug. If it's a feature, we need to update the doc. Or if it's a bug we should fix this line. What do you think? ``` with DAG("skip_exception", schedule_interval=None, start_date=pendulum.parse("20190101")) as dag: def raise_skip(): raise AirflowSkipException op1 = PythonOperator(task_id="op1", python_callable=raise_skip) op2 = DummyOperator(task_id="op2", trigger_rule="none_failed") op1 >> op2 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services