potiuk commented on PR #43040: URL: https://github.com/apache/airflow/pull/43040#issuecomment-2434739027
> @amoghrajesh , shouldn't the list of sensitive_config_values be updated with the list of keys that need to redact ? Line: https://github.com/amoghrajesh/airflow/blob/840ea3efb9533837e9f36b75fa527a0fbafeb23a/airflow/configuration.py#L121 Not really @saurabhb-dev , It's even better to mask values in this case, rather than keys. The secrets_masker works in two modes: * for known structures it can mask sensitive key values (for example if a key in connection is "password" - it will mask the value when it prints the structure (when it is aware of the structure and acts on it) * but this is impossible when you log messages when the structures or values have been already converted to log string - by the time secrets masker in logger (installed as filter) gets the message, the structure is gone already (converted to string representation) and we are not aware that particular key corresponds to particular value. So our secrets masker has the second mode - where it can mask specific values. It will scan the string for all the values that are registered upfront and mask them if it finds any of them in the string. This is what happens here - we retrieve all the secret config values (whether they come by env vars or by other means) and we add values (i.e. actual secrets) to be masked. This way when any secret is printed anywhere where secrets_masker is used, it will automatically mask it - regardless if it is a structure (secrets_masker checks values of dicts for example) or whether it's already converted to string. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
