Hi All,

I would like to propose a change to how the task SDK loads secrets
backends, and get objections (if any).

Getting a connection or a variable today constructs a brand new backend
instance. Any authenticated client
the backend holds is discarded and rebuilt each time a secret is accessed.

For an external backend, that means re-authenticating per secret. With
HashiCorp Vault, one AppRole login for
every variable a task reads. I measured two variables fetched in one
process producing two logins.

I propose memoising the backend list where it is constructed, keyed on the
search path. Keyed rather than a single global
because core's cache only ever serves the server chain. Every other search
path takes its rebuild branch, and the task runner
always asks for the worker chain, so copying core's shape verbatim would
leave worker lookups rebuilding exactly as they do now.

This change means that holding an instance for the life of a task process
will mean that an expiring credential has to refresh. I went through
all seven external backends in our providers:
- amazon, google, microsoft/azure: auth is delegated to boto3, google-auth
and azure-identity, all of which refresh credentials internally.
- akeyless: keeps its own token cache with an explicit TTL and refreshes.
- yandex: the Yandex SDK refreshes from a service account key, and raw
OAuth tokens are long-lived.
- hashicorp: covered by a provider change that just went in, which drops
two redundant auth/token/lookup-self calls per lookup and
re-authenticates reactively on a 403 instead. (PR:
https://github.com/apache/airflow/pull/71701)
- cncf/kubernetes: not covered. It calls load_incluster_config() inside a
cached_property, so the projected service account token is read once.
Bound tokens expire and kubelet rotates the file, and the Python client
does not re-read it.

The Kubernetes gap already exists on the server side, where core has cached
backend instances for the life of the process for years, so this is not
new.
It does become reachable in a worker whose task outlives the token. If
someone is interested in fixing it like how Vault did it, be my guest.

*What am I asking?*
Any objection to backend instances living for the life of the process on
the worker path, given that is already the case on the server path? And
does anyone
know of a backend, in-tree or out, that would not tolerate it? Out-of-tree
backends are the case I cannot audit.

Thanks & Regards,
Amogh Desai

Reply via email to