MCprotein opened a new issue, #75716:
URL: https://github.com/apache/airflow/issues/75716
### Apache Airflow version
3.3.1 (the same code is present on `main`)
### What happened?
Opening the **Browse → Dag Runs** list sends one `GET
/api/v2/dags/{dag_id}/details` request for every distinct Dag in the visible
rows, all at the same time. No dialog is opened.
In our deployment (1 API server process, default `sql_alchemy_pool_size=5` /
`max_overflow=10`), a list of 50 runs triggered 42 concurrent `/details`
requests. The API server exhausted its DB connection pool,
`/api/v2/monitor/health` stopped responding, and the liveness probe restarted
the pod. The whole UI was unavailable for about 80 seconds.
### Cause
- `pages/DagRuns/DagRuns.tsx` renders `<ClearRunButton dagRun={row.original}
/>` for every row.
- `ClearRunButton` always mounts `<ClearRunDialog dagRun={dagRun}
onClose={onClose} open={open} />`, even when `open` is false.
- `ClearRunDialog` calls `useDagServiceGetDagDetails({ dagId })` without
`enabled: open`, and also calls `useAutoRefresh({ dagId })`, which calls
`useDagServiceGetDagDetails` again.
As a result, every row issues a details query on mount. React Query
deduplicates by Dag ID, so the number of requests equals the number of distinct
Dags in the page.
`ClearTaskInstanceButton` already avoids this on the Task Instances list by
letting the page own a single dialog (`onOpen` prop). The Dag Runs list does
not use the same approach.
### What you think should happen instead?
Opening the Dag Runs list should not request Dag details for each row.
Details should be fetched only when the Clear dialog is opened.
Possible fixes:
1. Mount the dialog only while it is open in `ClearRunButton`: `{open ?
<ClearRunDialog ... /> : undefined}`.
2. Or gate both queries in `ClearRunDialog` on `open`: pass `enabled: open`
to `useDagServiceGetDagDetails`, and pass `dagId` to `useAutoRefresh` only when
open.
3. Or use a single page-level dialog, the same way the Task Instances list
does.
### How to reproduce
1. Create about 40 Dags and run each one once.
2. Open Browse → Dag Runs with page size 50.
3. Observe one `GET /api/v2/dags/{dag_id}/details` request per distinct Dag
in the network tab or the API server access log.
### Operating System
Debian bookworm (official `apache/airflow:slim-3.3.1-python3.12` image),
Kubernetes (EKS)
### Deployment
Official Apache Airflow Helm Chart (1.22.0)
### Anything else?
As a local workaround we patch the built bundle so that `ClearRunButton`
mounts the dialog only when it is open. We also increased the API server
replicas and its DB pool size.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
