potiuk commented on code in PR #70681:
URL: https://github.com/apache/airflow/pull/70681#discussion_r3873767794
##########
airflow-core/src/airflow/configuration.py:
##########
@@ -740,6 +741,55 @@ def get_custom_secret_backend(worker_mode: bool = False)
-> BaseSecretsBackend |
return conf._get_custom_secret_backend(worker_mode=worker_mode)
+class Backend(enum.Enum):
+ """Known secrets backends."""
+
+ ENVIRONMENT_VARIABLE = "environment_variable"
+ EXECUTION_API = "execution_api"
+ CUSTOM = "custom"
+ METASTORE = "metastore"
+
+ @classmethod
+ def from_module(cls, default_backend: str) -> Backend:
Review Comment:
These three literals are exactly the ones
`scripts/ci/prek/check_secrets_search_path_sync.py` exists to keep in sync
(registered at `.pre-commit-config.yaml:588`) — but that hook only extracts and
compares the two list constants, `DEFAULT_SECRETS_SEARCH_PATH` in core and
`_SERVER_DEFAULT_SECRETS_SEARCH_PATH` in task-sdk. It has no visibility into
`from_module`.
Restoring the constants addressed the first half of my earlier comment, but
the coupling moved rather than went away — and it now exists in two copies.
This block is duplicated verbatim in
`task-sdk/src/airflow/sdk/configuration.py` (lines 272-321): 50 lines identical
except `from airflow.configuration import conf` vs `from
airflow.sdk.configuration import conf`.
The concrete failure: add a fourth backend to `DEFAULT_SECRETS_SEARCH_PATH`.
The hook stays green, because it only checks that the two constants match each
other and they both moved. Then `initialize_secrets_backends` raises
`ValueError: Unknown module provided: <class path>` on every start, in both
core and the SDK.
`shared/configuration/` is the natural home for `Backend`, `from_module` and
`_get_secrets_backend_order` — one definition instead of two, and the
class-path mapping ends up next to the constants the hook already guards.
##########
task-sdk/src/airflow/sdk/configuration.py:
##########
@@ -267,6 +269,55 @@ def get_custom_secret_backend(worker_mode: bool = False):
return conf._get_custom_secret_backend(worker_mode=worker_mode)
+class Backend(enum.Enum):
Review Comment:
This block is identical to `airflow-core/src/airflow/configuration.py` lines
744-793 apart from the `conf` import — I diffed the two. See the comment there
for the drift hazard; moving this into `shared/configuration/` would let both
sides import a single definition.
##########
airflow-core/src/airflow/config_templates/config.yml:
##########
@@ -1589,6 +1589,22 @@ secrets:
sensitive: true
example: ~
default: ""
+ backends_order:
+ description: |
+ .. note:: |experimental|
+
+ Comma-separated list of secret backends. These backends will be used
in the order they are specified.
+ Please note that the `environment_variable` and `metastore` are
required values and cannot be removed
Review Comment:
Single backticks are RST's default role rather than literal markup, so these
render as italics instead of code. The `workers.backends_order` description
immediately below uses double backticks for the same two values.
```suggestion
Please note that the ``environment_variable`` and ``metastore`` are
required values and cannot be removed
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
