ephraimbuddy opened a new pull request, #77254: URL: https://github.com/apache/airflow/pull/77254
`test_migration_stairway` fails on every MySQL and Postgres DB-core job on main. The stairway runs out of connections partway through (`FATAL: sorry, too many clients already` on Postgres, `Too many connections` on MySQL), fails after its retries, and leaves the database mid-migration for the tests that follow. The stairway fixture asked for NullPool with `configure_orm(disable_connection_pool=True)`. When `upgradedb()` leaves `_single_connection_pool()`, it rebuilds the ORM from config, so from the first step on every engine got the default pool. The session and migration-lock connection that `upgradedb()` keeps using stay bound to the engine that rebuild disposed. Using them afterwards re-creates a pool on that engine, and nothing disposes it again. Each step leaves about one idle connection that closes only when a full garbage collection frees the engine. In the long-running "Other" test process, full collections are rare enough that the stairway reaches the server's connection limit. Setting NullPool through `[database] sql_alchemy_pool_enabled` makes every rebuild keep it, so those connections close as soon as they are returned. The per-revision NullPool re-apply in the test loop is no longer needed. The leak predates the commits after which CI started failing; those commits only changed when garbage collection happens to run in that process. Run with `uv run` against local Postgres 14 and MySQL 8.0 servers, with full garbage collection suppressed to match CI, and on SQLite: | Backend | Before | After | |---|---|---| | Postgres 14 | idle connections grow by ~1 per step; fails at step ~95 with the CI error | flat at 3 for all 267 steps; passes | | MySQL 8.0 | 63 idle connections at step 60 | flat at 3 for all 267 steps; passes | | SQLite | not run | passes | --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
