GitHub user abhipalsingh added a comment to the discussion: Intermittent Airflow 3.1.2 API Timeouts Under High Load Causing Task Failures (KubernetesExecutor)
Assuming you are using postges backend db. Try increasing the capacity on pgbouncer service. Either scale-up the connection pool or scale-out and have more pgbouncer pods to handle the additional traffic. With Airflow upgrade from 2.x to 3.x, the new design have fundamentally increased the need to query Airflow database. When you scale-up/scale-out pgbouncer, also check if your backend db server/cluster can handle the additional connection load pgbouncer will now start sending. GitHub link: https://github.com/apache/airflow/discussions/58453#discussioncomment-17299312 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
