sjyangkevin opened a new pull request, #76891:
URL: https://github.com/apache/airflow/pull/76891
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull
request.
Write a good git commit message following this guide:
https://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the
documentation.
For user-facing UI changes, please attach before/after screenshots (or a
short
screen recording) so reviewers can assess the visual impact.
Feel free to ping (in general) for the review if you do not see reaction for
a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we
sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
closes: #73303
## Why
A running task talks to the Execution API with a short-lived token. So that
long tasks keep working, the server hands back a fresh token whenever the
current one is close to expiring.
Until now the server renewed the token without checking whether the task was
still running. A token that belonged to a finished task could therefore be kept
alive. This change binds the reissue path to the lifecycle of the workloads the
token was minted for, so a token **WILL NOT** be renewed after the workload it
belongs to has reached a terminal state.
## What changed
Before renewing a token, the server now asks one more question: "is the work
this token was issued for still unfinished?" It renews only in these cases:
- the token belongs to a task try that is the current try and has not
reached a terminal state,
- the token belongs to a callback that has not reached a terminal state,
- the token is the asset watcher token, which has no lifecycle.
In every other case (the task finished, the try was archived, the row was
deleted, or the ID is unknown) the request is still served as usual, but no new
token is returned. The old token then stops working when it expires on its own.
Only renewal changes. Requests are not rejected, so a finished task can
still run its `on_success_callback` with the token it already holds. The
database lookup happens only when a token is actually due for renewal, which is
about once per token lifetime per task. Responses keep the same shape, so no
Execution API version change is needed and older workers keep working.
#73303 talks about task instances. Deadline callbacks get the same kind of
token and have their own clear end state, so I applied the same rule to them.
## Before and after
I reproduced this end to end in Breeze with a 60-second token lifetime: a
real task ran and succeeded, and a script then kept calling a read-only
endpoint with the task's last token.
Before this change, the task finished and its token was due to expire 45
seconds later. Each call inside the renewal window returned a new token, even a
heartbeat call that the server rejected with `409`. After four renewals it was
valid until more than three minutes after the task finished, and it could have
been kept alive.
After this change, the calls made after the task finished were still served,
but none of them returned a new token. Once the token expired, the next call
was rejected with `403 Invalid auth token`. While the task was still running,
renewal worked exactly as before.
## Tests
- New unit tests cover each case of the rule through a real HTTP request: a
running task, every terminal task state, an archived try, an unknown ID, a
running and a finished callback, and the asset watcher.
- Two existing renewal tests used a made-up ID with no database row. They
test timing, so they now stub the new check, and one of them also asserts that
the lookup is skipped when a token is not due for renewal.
- The whole Execution API unit test directory passes in Breeze (1425 passed,
13 skipped).
- Static checks and `mypy-airflow-core` pass.
- Build docs
The docs in `jwt_token_authentication.rst` and `security_model.rst` are
updated to describe the new behaviour.
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR,
please
change below checkbox to `[X]` followed by the name of the tool, uncomment
the "Generated-by".
-->
- [X] Yes (please specify the tool below)
Generated-by: [Claude Code] following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
