potiuk commented on issue #36920: URL: https://github.com/apache/airflow/issues/36920#issuecomment-1979783880
> Hi, the only config I changed was AIRFLOW__DATABASE__SQL_ALCHEMY_POOL_SIZE, which was set to 180. My Airflow has 1200 active running dags so this number was set at a rather high value. This is a total misunderstanding of how SQL Alchemy Pool works. SQL Alchemy Pool has exactly `0` impact on running DAGs. Each task is run in their own process (every task is a forked, new process and establishes their own DB connnections) and sql alchemy pool can only be really useful if the connections are shared within the same process. SQL Alchemy Pool is only used in scheduler to do schedulling and setting huge value for the POOL like that makes completely no sense, because None of our processed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
