Asquator commented on PR #55537: URL: https://github.com/apache/airflow/pull/55537#issuecomment-3325641701
### BENCHMARK: Control group #### Description **DAGs: 1 Tasks in DAG: 4000, all mapped No limits whatsoever Operator: `BashOperator` Weight rule: downstream (default)** -> [See DAG code](https://gist.github.com/Asquator/9dfa052595cf16121f4bd21c0797d50d) <-- <details> <summary>Technical details</summary> DB: Postgres 13 CPU: 16 cores - D16s v3 (Azure) RAM: 64GB Executor: `LocalExecutor` Configurations: ``` AIRFLOW__CORE__PARALLELISM=128 AIRFLOW__SCHEDULER__MAX_TIS_PER_QUERY=16 AIRFLOW__CELERY__WORKER_AUTOSCALE=48,16 AIRFLOW__CELERY__OPERATION_TIMEOUT=60 ``` </details> #### Results | Selector strategy ➡️ <br> Measurement ⬇️| Old | New | Meaning |----------|----------|----------|----------| | Total iterations | 250 | 250 | Not affected | | Average scheduler loop duration | 1482[ms] | 1418[ms] | Barely affected | | Total time| ~371[sec] | ~345[sec] | Throughput barely affected On the left: **old** selector On the right: **new** selector <img width="2473" height="692" alt="bench3" src="https://github.com/user-attachments/assets/78d36244-35ef-4d20-a781-6cef3c4cf872"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
