seanghaeli opened a new pull request, #73014: URL: https://github.com/apache/airflow/pull/73014
Adds an optional `[kubernetes_executor] client_factory` setting: the import path of a function that returns the Kubernetes client the KubernetesExecutor uses. It is unset by default, and when unset nothing changes. Opening as a draft to agree the shape before I polish it. ## Why Today the executor can build its client two ways: an in-cluster service account token, or a kubeconfig. If your cluster hands out short-lived credentials, there is no point at which you can give the executor a client you built yourself, so you end up refreshing credentials underneath a client you do not own. On the operator side of this provider, where the same gap exists, that has cost us repeated bugs: stale cached config that breaks exec-based token refresh ([#61737](https://github.com/apache/airflow/issues/61737), still open), credential refresh errors on long-running pods ([#57585](https://github.com/apache/airflow/issues/57585), fixed), and a rejected token when the exec plugin does not carry the assumed-role identity ([#64657](https://github.com/apache/airflow/issues/64657), fixed). The two that were fixed were each patched on their own terms, and each patch made the refresh logic a little more intricate. Those reports all run through `KubernetesHook`, which builds its own clients and never calls the function changed here, so this change does not fix them. It changes the executor path only, where every client is already built in one function. If the shape is right, extending the same seam to the hook path is a reasonable follow-up. ## This is not about reaching EKS The exec-plugin kubeconfig already works for that, and nothing here changes or replaces it. The gap is about who builds the client. ## On coupling providers together Raised already, so briefly: the setting holds an import path, so `cncf.kubernetes` gains no dependency on or knowledge of any other provider, and nothing in this diff mentions AWS. It affects only people who set it. ## Why an import path and not a callable `KubernetesJobWatcher` runs as a separate process, so under the `spawn` start method it shares no objects with the scheduler. Storing a path lets each process resolve the factory for itself from configuration. ## When it is set The factory owns client construction, so `in_cluster`, `cluster_context`, `config_file`, `verify_ssl`, `ssl_ca_cert`, `enable_tcp_keepalive` and `api_client_retry_configuration` stop applying. That is documented on the option. ## Tests The default in-cluster and kubeconfig paths when the option is unset, a configured factory replacing default construction, an unimportable path raising `ImportError`, and the factory being resolved in a separate interpreter, which is the watcher case. The three factory tests fail without this change and pass with it. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Fable 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
