xvega commented on code in PR #73399:
URL: https://github.com/apache/airflow/pull/73399#discussion_r4442143065
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/hooks/kubernetes.py:
##########
@@ -921,16 +922,12 @@ async def _load_config(self):
"""
Load Kubernetes configuration.
- For static auth (token, certificate), configuration is loaded once per
hook instance
- and cached. For exec-based auth (EKS, GKE), the config is reloaded on
every call so
- that short-lived tokens are always refreshed.
+ Reload credentials on every access while retaining the client for
non-exec auth.
Review Comment:
I ran a local benchmark with AI assistance Codex (GPT-6): I used 15 hooks,
five repeated `get_conn()` calls per hook after warm-up, and took the median of
three alternating upstream/PR runs:
| Configuration | Upstream | This PR |
|---|---:|---:|
| `kube_config_path` | 3.74 ms | 0.48 ms |
| Default kubeconfig | 3.72 ms | 0.48 ms |
| `kube_config` extra | 3.71 ms | 0.65 ms |
This includes loading and parsing the kubeconfig, but doesn’t make any
Kubernetes API calls. I used a single-context kubeconfig, so the timings only
cover getting the client, not the whole trigger.
Upstream created an SSL context on each of the 75 calls in each run. With
this PR, the hooks reused the contexts created during warm-up. Even with the
config reload, getting the client was around 6–8× faster in this local test.
Given those results, I’ve left the per-call reload in place so we still pick
up credential changes without needing another cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
