antruigon opened a new pull request, #71237:
URL: https://github.com/apache/airflow/pull/71237
## What
Adds `pgbouncer.terminationGracePeriodSeconds` to the Helm chart, rendered
on the PgBouncer Deployment's pod spec. Defaults to **120** to match the drain
window of the chart's default preStop hook.
## Why
The chart ships a default PgBouncer preStop hook that drains client
connections for up to 120 seconds:
```yaml
containerLifecycleHooks:
preStop:
exec:
# Allow existing queries clients to complete within 120 seconds
command: ["/bin/sh", "-c", "killall -INT pgbouncer && sleep 120"]
```
…but the Deployment never sets `terminationGracePeriodSeconds`, so the
Kubernetes default of **30s** SIGKILLs the pod mid-drain. On any node drain or
eviction (cluster upgrades, autoscaler scale-down, AMI rolls) in-flight client
connections are cut instead of drained — workers see
`psycopg2.OperationalError: server closed the connection unexpectedly`, and
tasks that were mid-query fail. We hit this in production during a routine EKS
node-group AMI roll: one evicted PgBouncer replica cut ~880 in-flight
connections and failed 21 zero-retry DAG runs.
Every other long-running component in the chart already exposes this knob
(`scheduler`, `workers.celery`, `triggerer`, `dagProcessor`, `statsd`, `redis`,
`otelCollector`); PgBouncer was the only one missing it, and it is arguably the
component that needs it most since it holds everyone's DB connections.
## Default choice
`120`, matching the shipped preStop. Note this is an upper bound, not a
fixed wait: PgBouncer receives SIGINT immediately, stops accepting new
connections, and exits as soon as the last client connection is released — so
pods still terminate quickly under low load. Users who prefer the old behaviour
can set it back to `30`.
## Changes
- `chart/values.yaml` — new `pgbouncer.terminationGracePeriodSeconds: 120`
next to the preStop hook it pairs with
- `chart/values.schema.json` — schema entry (`integer`, default 120,
Kubernetes docs section)
- `chart/templates/pgbouncer/pgbouncer-deployment.yaml` — render the field
on the pod spec
- `chart/tests/helm_tests/other/test_pgbouncer.py` — parametrized test for
default + override (mirrors `test_statsd.py`)
All 86 tests in `test_pgbouncer.py` + `test_pdb_pgbouncer.py` pass locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
