yuqian90 commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG URL: https://github.com/apache/airflow/pull/6392#issuecomment-548423260 > You could separate them into 2 DAGs, and use TriggerDagRunOperator after task K in DAG 1 to trigger DAG 2 after sensor passes True. > > You can have BranchPythonOperator in DAG2: to decide if it needs to just run L & M or run the other branch where 1st task can be TriggerDagRunOperator to run Dag 1. But note that this can end up in an always True loops so make sure that your BranchPythonOperator in Dag 2 has the correct logic. That's an interesting idea. I'll think a bit more and see if I can adapt the idea for our use. Separating into two DAGs have some other implications for us. E.g. `A, C, E, ..., J` is sometimes not the only chain we need to rerun in the entire workflow (although it is in the simplified example I put up). If every task rerun splits the DAG into two, some of our workflows may end up as four different DAGs because there are two such "cycles" in the entire workflow. One use case we often have is someone may need to manually rerun tasks that have finished historically for the past month to pick up changes made to some tasks or to external einvironment. If we separate the workflow on multiple DAGs, that manual rerun becomes harder. That said, the `TriggerDagRunOperator + BranchPythonOperator` idea is definitely something I'll look into a bit more. Thanks again.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
