junyeong0619 commented on PR #66022:
URL: https://github.com/apache/airflow/pull/66022#issuecomment-4342824455
After investigating #61595, I've identified a significant inconsistency in
the Variable API that could lead to user confusion regarding data integrity.
Currently, Variable.get() provides a consistent lookup experience by
traversing the Cache, Secrets Backend, and Metadata DB. However, due to the
lack of an abstracted list method across all secret backends, Variable.list()
(and the FastAPI get_variables endpoint) only queries the Metadata DB.
This creates a functional inconsistency:
Variable.get("secret_key") returns a value from AWS Secrets Manager.
Variable.list() does not show "secret_key" because it isn't stored in the DB.
To prevent user confusion regarding data integrity, I suggest making this
behavior explicit. Renaming the internal or public method to something like
list_from_db() would clearly communicate that the results are limited to the
metastore.
Providing this explicit naming will help users immediately understand why
their secret-backed variables are missing from the list without needing to dive
into the core source code.
I would love to hear your thoughts on this direction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
