serhii-karelov opened a new issue, #65712:
URL: https://github.com/apache/airflow/issues/65712
### Under which category would you file this issue?
Providers
### Apache Airflow version
3.2.1
### What happened and how to reproduce it?
When have tens of tasks in-flight, and while surfing the UI, occasionally we
get
1. Queued up hertbeat UPDATE queries in the airflow api
2. Connection pool exhaustion
3. API response times to the heartbeats is high, causing workers to timeout,
and retry.
4. Some tasks terminated and put into a retry state
### What you think should happen instead?
_No response_
### Operating System
_No response_
### Deployment
Other Docker-based deployment
### Apache Airflow Provider(s)
celery
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
Some query prevents the heartbeat UPDATEs
```
usename state xact_duration query_duration query
db_user idle in transaction 00:00:22.17721 00:00:00.06658 SELECT
task_instance.state AS task_instance_state
FROM
task_instance
WHERE
task_instance.id = '0<zzzyyy>54'::uuid::UUID
db_user active 00:00:22.123495 00:00:22.120761 UPDATE
task_instance SET
last_heartbeat_at='2026-04-23T09:03:18.258390+00:00'::timestamptz,
updated_at='2026-04-23T09:03:18.261693+00:00'::timestamptz WHERE
task_instance.id
db_user active 00:00:22.058237 00:00:22.054038 UPDATE
task_instance SET
last_heartbeat_at='2026-04-23T09:03:18.320462+00:00'::timestamptz,
updated_at='2026-04-23T09:03:18.324080+00:00'::timestamptz WHERE
task_instance.id
db_user active 00:00:21.901896 00:00:21.89822 UPDATE
task_instance SET
last_heartbeat_at='2026-04-23T09:03:18.479643+00:00'::timestamptz,
updated_at='2026-04-23T09:03:18.483318+00:00'::timestamptz WHERE
task_instance.id
db_user active 00:00:21.537818 00:00:21.534485 UPDATE
task_instance SET
last_heartbeat_at='2026-04-23T09:03:18.844164+00:00'::timestamptz,
updated_at='2026-04-23T09:03:18.847447+00:00'::timestamptz WHERE
task_instance.id
...
```
Slow endpoints
| # | Duration (s) | Status | Method | Endpoint |
|---|---|---|---|---|
| 1 | 72.02 | 200 | GET |
/api/v2/dags/XYZ/dagRuns/~/hitlDetails?state=deferred&response_received=false |
| 2 | 71.88 | 200 | GET | /ui/grid/runs/QAZ?limit=25&order_by=-run_after |
| 3 | 71.05 | 204 | PATCH | /execution/task-instances/ZAQ/state |
| 4 | 71.01 | 200 | GET | /api/v2/dags/QWE/details |
| 5 | 70.91 | 200 | GET | /ui/dags/EWQ/latest_run |
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
