1fanwang opened a new pull request, #73931:
URL: https://github.com/apache/airflow/pull/73931
A client can't change just the slots of a pool. `PATCH
/api/v2/pools/{pool_name}?update_mask=slots` with `{"slots": 8}` returns 422
because the pool name and `include_deferred` are still required. A bulk update
with the same mask fails too, and only `default_pool` honours the mask. So
clients resend `include_deferred` every time, and `airflowctl pools update
--pool etl --slots 16` sends false, which turns deferred slots off on a pool
that had them on.
With this change, a PATCH or bulk update with a mask needs only the masked
fields and keeps the rest. Masked fields are still validated (`{"slots": null}`
is still a 422), and a PATCH without a mask still needs the full body.
The service validated the body against the full pool schema before applying
the mask. Now it validates only the fields it will write and takes the other
required ones from the stored pool.
closes: https://github.com/apache/airflow/issues/73768
related: https://github.com/apache/airflow/pull/71220 edits the same
validation block. They are compatible; the second to merge needs a rebase there.
### Testing Done
The reproducer below drives a real API server (SQLite, simple auth manager)
run with `airflow` from `uv sync --project airflow-core`, once at main 18fe1c6
and once at this branch.
Before, main at 18fe1c6:
```
$ bash pool-mask-e2e.sh .venv/bin/airflow
$ airflow version # the server
3.4.0
$ curl -X POST http://127.0.0.1:64263/api/v2/pools -d '{"name": "etl",
"slots": 4, "description": "nightly loads", "include_deferred": true}'
{"name":"etl","slots":4,"description":"nightly
loads","include_deferred":true,"occupied_slots":0,"running_slots":0,"queued_slots":0,"scheduled_slots":0,"open_slots":4,"deferred_slots":0,"team_name":null}
HTTP 201
$ curl -X PATCH 'http://127.0.0.1:64263/api/v2/pools/etl?update_mask=slots'
-d '{"slots": 8}'
{"detail":[{"type":"missing","loc":["pool"],"msg":"Field
required","input":{"slots":8},"url":"https://errors.pydantic.dev/2.13/v/missing"},{"type":"missing","loc":["include_deferred"],"msg":"Field
required","input":{"slots":8},"url":"https://errors.pydantic.dev/2.13/v/missing"}]}
HTTP 422
$ curl -X PATCH http://127.0.0.1:64263/api/v2/pools -d '{"actions":
[{"action": "update", "entities": [{"name": "etl", "slots": 12}],
"update_mask": ["slots"]}]}'
{"detail":[{"type":"missing","loc":["include_deferred"],"msg":"Field
required","input":{"slots":12,"pool":"etl"},"url":"https://errors.pydantic.dev/2.13/v/missing"}]}
HTTP 422
$ curl http://127.0.0.1:64263/api/v2/pools/etl
{"name":"etl","slots":4,"description":"nightly
loads","include_deferred":true,"occupied_slots":0,"running_slots":0,"queued_slots":0,"scheduled_slots":0,"open_slots":4,"deferred_slots":0,"team_name":null}
HTTP 200
$ curl -X PATCH 'http://127.0.0.1:64263/api/v2/pools/etl?update_mask=slots'
-d '{"slots": null}' # a masked field is still validated
{"detail":[{"type":"missing","loc":["pool"],"msg":"Field
required","input":{"slots":null},"url":"https://errors.pydantic.dev/2.13/v/missing"},{"type":"int_type","loc":["slots"],"msg":"Input
should be a valid
integer","input":null,"url":"https://errors.pydantic.dev/2.13/v/int_type"},{"type":"missing","loc":["include_deferred"],"msg":"Field
required","input":{"slots":null},"url":"https://errors.pydantic.dev/2.13/v/missing"}]}
HTTP 422
$ curl -X PATCH http://127.0.0.1:64263/api/v2/pools/etl -d '{"slots": 8}'
# without update_mask the full body is still required
{"detail":[{"type":"missing","loc":["pool"],"msg":"Field
required","input":{"slots":8},"url":"https://errors.pydantic.dev/2.13/v/missing"},{"type":"missing","loc":["include_deferred"],"msg":"Field
required","input":{"slots":8},"url":"https://errors.pydantic.dev/2.13/v/missing"}]}
HTTP 422
```
After, this branch:
```
$ bash pool-mask-e2e.sh .venv/bin/airflow
$ airflow version # the server
3.4.0
$ curl -X POST http://127.0.0.1:64318/api/v2/pools -d '{"name": "etl",
"slots": 4, "description": "nightly loads", "include_deferred": true}'
{"name":"etl","slots":4,"description":"nightly
loads","include_deferred":true,"occupied_slots":0,"running_slots":0,"queued_slots":0,"scheduled_slots":0,"open_slots":4,"deferred_slots":0,"team_name":null}
HTTP 201
$ curl -X PATCH 'http://127.0.0.1:64318/api/v2/pools/etl?update_mask=slots'
-d '{"slots": 8}'
{"name":"etl","slots":8,"description":"nightly
loads","include_deferred":true,"occupied_slots":0,"running_slots":0,"queued_slots":0,"scheduled_slots":0,"open_slots":8,"deferred_slots":0,"team_name":null}
HTTP 200
$ curl -X PATCH http://127.0.0.1:64318/api/v2/pools -d '{"actions":
[{"action": "update", "entities": [{"name": "etl", "slots": 12}],
"update_mask": ["slots"]}]}'
{"create":null,"update":{"success":["etl"],"errors":[]},"delete":null}
HTTP 200
$ curl http://127.0.0.1:64318/api/v2/pools/etl
{"name":"etl","slots":12,"description":"nightly
loads","include_deferred":true,"occupied_slots":0,"running_slots":0,"queued_slots":0,"scheduled_slots":0,"open_slots":12,"deferred_slots":0,"team_name":null}
HTTP 200
$ curl -X PATCH 'http://127.0.0.1:64318/api/v2/pools/etl?update_mask=slots'
-d '{"slots": null}' # a masked field is still validated
{"detail":[{"type":"int_type","loc":["slots"],"msg":"Input should be a valid
integer","input":null,"url":"https://errors.pydantic.dev/2.13/v/int_type"}]}
HTTP 422
$ curl -X PATCH http://127.0.0.1:64318/api/v2/pools/etl -d '{"slots": 8}'
# without update_mask the full body is still required
{"detail":[{"type":"missing","loc":["pool"],"msg":"Field
required","input":{"slots":8},"url":"https://errors.pydantic.dev/2.13/v/missing"},{"type":"missing","loc":["include_deferred"],"msg":"Field
required","input":{"slots":8},"url":"https://errors.pydantic.dev/2.13/v/missing"}]}
HTTP 422
```
<details>
<summary>Reproducer source: pool-mask-e2e.sh</summary>
```bash
#!/usr/bin/env bash
# Partial pool PATCH requests with and without update_mask against a real
Airflow API server (SQLite, simple auth manager).
# Usage: pool-mask-e2e.sh <airflow executable for the server>
set -u
AIRFLOW=${1:?usage: pool-mask-e2e.sh <airflow>}
PORT=$(python3 -c 'import socket; s = socket.socket(); s.bind(("127.0.0.1",
0)); print(s.getsockname()[1])')
URL="http://127.0.0.1:$PORT";
SERVER_HOME="$(mktemp -d)"
SERVER_ENV=(
AIRFLOW_HOME="$SERVER_HOME"
AIRFLOW__CORE__LOAD_EXAMPLES=False
AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS=admin:admin
)
say() { printf '\n$ %s\n' "$1"; }
env "${SERVER_ENV[@]}" "$AIRFLOW" db migrate > "$SERVER_HOME/setup.log" 2>&1
|| { cat "$SERVER_HOME/setup.log"; exit 1; }
env "${SERVER_ENV[@]}" "$AIRFLOW" api-server --port "$PORT" --workers 1 >
"$SERVER_HOME/api-server.log" 2>&1 &
SERVER_PID=$!
trap 'kill "$SERVER_PID" 2>/dev/null; wait "$SERVER_PID" 2>/dev/null; rm -rf
"$SERVER_HOME"' EXIT
PASSWORDS="$SERVER_HOME/simple_auth_manager_passwords.json.generated"
for _ in $(seq 90); do
curl -sf "$URL/api/v2/monitor/health" > /dev/null && [ -s "$PASSWORDS" ]
&& break
sleep 1
done
PASSWORD=$(python3 -c 'import json, sys;
print(json.load(open(sys.argv[1]))["admin"])' "$PASSWORDS")
TOKEN=$(curl -s -X POST "$URL/auth/token" -H 'Content-Type:
application/json' \
-d "{\"username\": \"admin\", \"password\": \"$PASSWORD\"}" | python3 -c
'import json, sys; print(json.load(sys.stdin)["access_token"])')
api() { curl -s -w '\nHTTP %{http_code}\n' -H "Authorization: Bearer $TOKEN"
-H 'Content-Type: application/json' "$@"; }
say 'airflow version # the server'
env "${SERVER_ENV[@]}" "$AIRFLOW" version 2> /dev/null
CREATE='{"name": "etl", "slots": 4, "description": "nightly loads",
"include_deferred": true}'
say "curl -X POST $URL/api/v2/pools -d '$CREATE'"
api -X POST "$URL/api/v2/pools" -d "$CREATE"
say "curl -X PATCH '$URL/api/v2/pools/etl?update_mask=slots' -d '{\"slots\":
8}'"
api -X PATCH "$URL/api/v2/pools/etl?update_mask=slots" -d '{"slots": 8}'
BULK='{"actions": [{"action": "update", "entities": [{"name": "etl",
"slots": 12}], "update_mask": ["slots"]}]}'
say "curl -X PATCH $URL/api/v2/pools -d '$BULK'"
api -X PATCH "$URL/api/v2/pools" -d "$BULK"
say "curl $URL/api/v2/pools/etl"
api "$URL/api/v2/pools/etl"
say "curl -X PATCH '$URL/api/v2/pools/etl?update_mask=slots' -d '{\"slots\":
null}' # a masked field is still validated"
api -X PATCH "$URL/api/v2/pools/etl?update_mask=slots" -d '{"slots": null}'
say "curl -X PATCH $URL/api/v2/pools/etl -d '{\"slots\": 8}' # without
update_mask the full body is still required"
api -X PATCH "$URL/api/v2/pools/etl" -d '{"slots": 8}'
```
</details>
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: GitHub Copilot CLI (Claude Opus 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
