Hello,
Recent work on cleaning and standardising Fernet Key parsing introduced
some breaking changes. For example, empty Fernet keys ('') were raising
exceptions, especially in integration/end-to-end tests. Previously, even
though comments were specific to certain use cases, empty Fernet keys were
allowed and handled by NullFernet. This approach makes the system
unencrypted, similar to having just an empty password while logging in to a
system. We have reverted the breaking changes to enable room for discussion.
We should have a generic, agreed-upon approach for Fernet key handling.
Where appropriate, big warnings and documentation should be added or
updated. For example, our Helm chart generates a Fernet key for deployment
if not provided (empty), whereas Docker Compose does not.
There is also a PR related to providing a Fernet key for integration tests,
where tests should always have a key to run reliably. This may not cover
all cases, but it’s the first step toward making Fernet handling resilient
and consistent.
Below is the approach discussed with a small group of people (Jarek Potiuk,
Jens Scheffler and me):
Proposed Fernet Key parsing approach:
-
Specified but empty ('')* →* use NullFernet (log a big warning that it’s
unencrypted).
-
Not specified* →* attempt to generate and persist; fail if unable (e.g.,
due to permissions).
-
Specified and non-empty* →* attempt to use it; fail if format is invalid.
It would be great to collect any feedback or ideas. You can find some
useful links below.
Which part of the code mentioned:
https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/crypto.py
Adding Fernet Key as default (followed by Amogh Desai with a good idea of
generating the fernet dynamically in the code for integration tests, which
will be updated): https://github.com/apache/airflow/pull/58112
Kind regards,
--
Bugra Ozturk
