Hi, I'm trying to debug an issue where a task in my dag is being set to REMOVED state, and the tasks after are running waiting for preceding tasks to succeed/fail.
If I delete the removed tasks from the meta database, the tasks will start running, move to succeeded state, but then promptly move to REMOVED state again. A change that may have caused this was to change the dag from: latest_only >> cleanup_hdfs >> run_mr run_mr >> refresh_tables >> refresh_done_file run_mr >> refresh_global_table >> refresh_done_file refresh_done_file >> reap_old to: latest_only >> cleanup_hdfs >> run_mr run_mr >> refresh_tables >> *compute_stats_orgs* >> refresh_done_file run_mr >> refresh_global_table >> *compute_stats_global* >> refresh_done_file refresh_done_file >> reap_old Where the highlighted tasks were added. Any idea what would cause this? I've tried clearing all task instances for the dag, restarting all services, no luck. I'm on 1.9.0 Thanks! - Nasron
