yevsh opened a new issue, #60461:
URL: https://github.com/apache/airflow/issues/60461
### Official Helm Chart version
1.18.0 (latest released)
### Apache Airflow version
3.1.5
### Kubernetes Version
v1.33.5
### Helm Chart configuration
airflow-config configmap
[api]
auth_backends =
airflow.api.auth.backend.session,airflow.api.auth.backend.basic_auth
jwt_audience = apache-airflow
[api_auth]
backend = airflow.api.auth.backend.default
jwt_audience = apache-airflow
[celery]
flower_url_prefix =
worker_concurrency = 16
[celery_kubernetes_executor]
kubernetes_queue = kubernetes
[core]
auth_manager =
airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
colored_console_log = False
dags_folder = /opt/airflow/dags
execution_api_server_url = http://airflow-api-server:8080/execution/
executor = KubernetesExecutor
load_examples = False
max_active_runs_per_dag = 1
max_active_tasks_per_dag = 20
parallelism = 50
remote_logging = False
### Docker Image customizations
apache/airflow:3.1.5-python3.12
### What happened
_No response_
### What you think should happen instead
When worker pod is started it fails on error:
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/execute_workload.py",
line 125, in <module>
main()
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/execute_workload.py",
line 121, in main
execute_workload(workload)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/execute_workload.py",
line 66, in execute_workload
supervise(
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py",
line 1984, in supervise
process = ActivitySubprocess.start(
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py",
line 955, in start
proc._on_child_started(ti=what, dag_rel_path=dag_rel_path,
bundle_info=bundle_info)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py",
line 966, in _on_child_started
ti_context = self.client.task_instances.start(ti.id, self.pid,
start_date)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/api/client.py",
line 215, in start
resp = self.client.patch(f"task-instances/{id}/run",
content=body.model_dump_json())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py",
line 1218, in patch
return self.request(
^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
338, in wrapped_f
return copy(f, *args, **kw)
^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
477, in __call__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
378, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
400, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 449,
in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 401,
in __get_result
raise self._exception
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
480, in __call__
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/api/client.py",
line 885, in request
return super().request(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py",
line 825, in request
return self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py",
line 914, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py",
line 942, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py",
line 999, in _send_handling_redirects
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py",
line 982, in _send_handling_redirects
hook(response)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/api/client.py",
line 186, in raise_on_4xx_5xx_with_note
return get_json_error(response) or response.raise_for_status()
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_models.py",
line 829, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
**httpx.HTTPStatusError: Client error '403 Forbidden' for url
'http://airflow-api-server:8080/execution/task-instances/019bb800-269d-7a22-b7dd-a0b77ed26871/run'**
For more information check:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403
### How to reproduce
If I try to do inside api-server container
curl -sS -X POST "http://localhost:8080/auth/token"; -H "Content-Type:
application/json" -d '{"username":"admin","password":"admin"}'
curl -sS -i -X PATCH \
"http://localhost:8080/execution/task-instances/019bb73b-0b68-7236-891b-50b2fc85fd4c/run";
\
-H "Authorization: Bearer token" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d '{}'
i fail on:
HTTP/1.1 403 Forbidden
date: Tue, 13 Jan 2026 15:35:28 GMT
server: uvicorn
content-length: 55
content-type: application/json
airflow-api-version: 2025-11-05
vary: Accept-Encoding
{"detail":"Invalid auth token: Audience doesn't match"}
but if I decode the jwt i see inside:
{
...
"aud": "apache-airflow",
"exp": 1768402852,
}
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
