theister commented on issue #18501:
URL: https://github.com/apache/airflow/issues/18501#issuecomment-984785114
Short update:
I was able to reproduce the problem using the following minimal example on
`2.0.2`
```
from datetime import datetime
from airflow import DAG
from airflow.operators.dummy import DummyOperator
from airflow.operators.bash import BashOperator
with DAG(
"debug-long-replay-dag",
schedule_interval="0 0 * * *",
start_date=datetime(2020, 1, 1),
max_active_runs=10,
catchup=True,
) as long_dag:
initial_task = DummyOperator(task_id="initial-task")
sleep_task = BashOperator(bash_command="sleep 300",
task_id="sleep-5-mins")
initial_task >> sleep_task
with DAG(
"debug-frequent-dag",
schedule_interval="*/1 * * * *",
start_date=datetime(2021, 1, 1),
catchup=False,
) as hf_dag:
sleep_task = BashOperator(bash_command="sleep 30",
task_id="sleep-30-sec")
```
We're currently upgrading our instances to 2.2.2, and I can no longer
reproduce it on the new version 🚀
Clearing task instances for a large number of dag runs now sets all DagRuns
to the newly introduced `"queued"` DagRun status, and the scheduler no longer
is starved out.
@vlieven @leonsmith please check if 2.2.2 solves it for you, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
