yuqian90 commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG URL: https://github.com/apache/airflow/pull/6392#issuecomment-555353514 > @yuqian90 Yes if you use pool, SubDagOperator can become problematic. The changes to SubDagOperator may be released in Airflow 2.0 Hi @kaxil thank you for you suggestions and comments. For this particular problem I described in [AIRFLOW-5648](https://issues.apache.org/jira/browse/AIRFLOW-5648), I ended up dealing with it using a simple approach. I created a function `duplicate_sub_dag()` that makes a copy of the portion of the DAG that I need to rerun. This works great for now. If anyone is interested in what `duplicate_sub_dag()` I can share the code. The problem in [AIRFLOW-5648](https://issues.apache.org/jira/browse/AIRFLOW-5648) could be easily solved like this: ``` A >> C >> E >> G >> H >> I >> J >> K >> L >> M >> Finish B >> D >> F >> G Sensor >> duplicate_sub_dag(dag, A.task_id, J.task_id) >> K ``` However, there's another type of problem that I still can't find a good solution for. And I think several other people in the Airflow community are having similar needs: Whenever we use `ExternalTaskSensor` to set up a dependency on a different DAG, and then someone clears the task that the `ExternalTaskSensor` depends on, there's no way for Airflow to clear the `ExternalTaskSensor` automatically. A quick search found someone trying to ask about this problem in [AIRFLOW-2279](https://issues.apache.org/jira/browse/AIRFLOW-2279). The approach in this PR can be easily adapted to solve that problem although some more discussions and opinions are probably needed to make it fit Airflow better. I'm going to put up a separate PR to propose a solution for [AIRFLOW-2279](https://issues.apache.org/jira/browse/AIRFLOW-2279).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
