I'm not super familiar with this part of the scheduler. What exactly are the implications of doing this mid-loop vs at scheduler termination? Is there a use case where DAGs hit this besides having been deleted?
The deactivate_stale_dags call doesn't appear to be super expensive or anything like that. This seems like a reasonable idea to me. *Taylor Edmiston* Blog <https://blog.tedmiston.com/> | CV <https://stackoverflow.com/cv/taylor> | LinkedIn <https://www.linkedin.com/in/tedmiston/> | AngelList <https://angel.co/taylor> | Stack Overflow <https://stackoverflow.com/users/149428/taylor-edmiston> On Wed, Aug 22, 2018 at 2:32 PM Dan Davydov <ddavy...@twitter.com.invalid> wrote: > I see some PRs creating endpoints to delete DAGs and other things related > to manually deleting DAGs from the DB, but is there a good reason why we > can't just move the deactivating DAG logic into the main scheduler loop? > > The scheduler already has some code like this, but it only runs when the > Scheduler terminates: > if all_files_processed: > self.log.info( > "Deactivating DAGs that haven't been touched since %s", > execute_start_time.isoformat() > ) > models.DAG.deactivate_stale_dags(execute_start_time) >
