potiuk commented on issue #35267: URL: https://github.com/apache/airflow/issues/35267#issuecomment-1786255118
I think indeed we need more details an investigation and answers (from your side @a-meledin ) . * Is it only with mapped tasks in that number you see the problem? * Have you checked if you have enough resources to run your database with this setup (I.e. handling 150 opened connections) ? * Have you try to set lower/higher values? What happens them? Generally speaking when you allow PGBouncer to allow 150 connections to your database, then it will open all 150 connections and Postgres creates a separate process for each connection. So if your postress does not have enough resources (for example memory - but it can be CPU or other resources ) to run all your processes, it will slow down to a crawl as those process will compete for those resources. You need to look at the resource usage and see what your bottleneck are. Finally the last question and request: * Did you see a regression comparing to other airflow versions or is it the only version of Airflow you tested it with? Can you try with 2.7.1 and upcoming (will be released tomorrow likely 2.7.3rc) and see if there are any differences. There were some changes in 2.7.2 that **should** decrease the number of connections used comparing to 2.7.1 but I wonder if this did not have some side effects, so if you can check those, that would be great. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org