[ https://issues.apache.org/jira/browse/AIRFLOW-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei-Alin Popescu reassigned AIRFLOW-2680: -------------------------------------------- Assignee: Andrei-Alin Popescu > Don't automatically percolate skipped stae > ------------------------------------------ > > Key: AIRFLOW-2680 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2680 > Project: Apache Airflow > Issue Type: Improvement > Reporter: Andrei-Alin Popescu > Assignee: Andrei-Alin Popescu > Priority: Minor > > Dear Airflow Maintainers, > > As part of our workflow, we have cases where all the upstream of a certain > task A can be skipped. In this case, airflow seems to automatically mark A as > skipped. > However, this does not quite work for us, since there are changes external to > the DAG which A needs to process, regardless of whether its upstream ran or > not. Additionally, we require A to get into an "upstream_failed" state and > not run if any its upstream tasks failed. > I don't see a trigger rule to cover this, so what would be the best way to > achieve this? I was thinking we could attach a DummyOperator as an upstream > to A, which in a way marks the fact that A depends on some external data and > needs to run anyway, but this can get really ugly for big DAGs. > I was also thinking we could have a new trigger_rule, e.g. "no_failure", > which would only trigger tasks if no upstream has failed. It differs from > "all_success" in that it will also trigger if all upstream has been skipped, > rather than percolating the skipped state on. > I'd really appreciate your feedback on this, and I'd like to know if in fact > there is already a good way of doing this with airflow that I don't know of. -- This message was sent by Atlassian JIRA (v7.6.3#76005)