hkc-8010 opened a new pull request, #66854:
URL: https://github.com/apache/airflow/pull/66854
Closes #66853
## Summary
- In `AssetManager.register_asset_change()`, replace ORM `.append()` on the
`asset_events`
relationship with a direct `INSERT INTO asset_alias_asset_event`. The
`.append()` lazy-loads
the entire existing events collection (thousands of rows on long-running
deployments) while
the `task_instance` row lock is held, causing idle-in-transaction pile-up
that exhausts API
server memory and triggers OOMKill.
- In `ti_update_state()`, add `session.commit()` after the TI state UPDATE
and Log entry to
release the `task_instance` row lock before asset registration runs.
Registration failures
after the commit are logged and swallowed -- the task outcome is already
durable.
Note on `session.commit()` in a session-parameter function: this
intentionally deviates from
the project convention (CLAUDE.md line 136). No code after the commit relies
on rollback; the
subsequent `session.get()` re-loads fresh state. Alternative approaches
(second session,
background task) were considered and rejected for higher operational
complexity with no
correctness benefit.
## Changes
- `airflow-core/src/airflow/assets/manager.py`: direct INSERT replaces ORM
lazy-load append
-
`airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py`:
early commit + deferred asset registration
- `airflow-core/tests/unit/assets/test_manager.py`: regression test for
no-lazy-load on alias-event association
-
`airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_task_instances.py`:
regression test for 204 on post-commit asset registration failure
## PR Checklist
- [x] My PR is targeted at the `main` branch
- [x] Tests added/updated
- [ ] Newsfragment added under `airflow-core/newsfragments/`
- [x] `prek` hooks all pass (pre-commit and pre-push)
- [x] Breeze integration tests pass (Python 3.10, SQLite, `--db-reset`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
