Hi all, I support the proposed approach. It improves security by default and makes disabling encryption an explicit option.
I noticed a few things: In get_fernet() (airflow-core/src/airflow/models/crypto.py <https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/crypto.py#L97>), we need to distinguish: - Not specified: Key missing from all sources (env vars, config file, defaults) - Explicitly empty: Key exists but is "" (e.g., fernet_key = in config or `AIRFLOW__CORE__FERNET_KEY="") Currently, conf.get("core", "FERNET_KEY") may return "" in both cases if the default template expands to empty. We should check config sources directly, such as using conf.has_option() or inspecting the lookup sequence, to differentiate these cases. For the unspecified case, we need to handle key generation, either in get_fernet() or during initialisation of the configuration. Thanks for driving this forward. Best regards, Ankit Chaurasia On Tue, Nov 18, 2025 at 1:18 AM Buğra Öztürk <[email protected]> wrote: > Hey all, > > It would be great to collect more minds on this. > Otherwise, I will move forward with a lazy consensus in the following days. > It is important to have some level of changes including warnings, please > let us know your thoughts. :) > > Bugra Ozturk > > On Wed, 12 Nov 2025, 09:53 Jarek Potiuk, <[email protected]> wrote: > > > Yep. Also to add - a little bit on `whys`. > > > > The root cause was that we had some inconsistency in behaviour of code > > duplication that Jens tried to sort out and it broke our tests. And > > for me the consistent approach described by Buğra is something we can > > nicely explain to users. Currently the behaviour of a few border > > conditions is not specified in the docs: > > > > > https://airflow.apache.org/docs/apache-airflow/stable/security/secrets/fernet.html > > > > Essentially, the proposal is that you can only disable fernet > > encryption by specifying an empty key - explicitly (but also warn). We > > believe it's a rather common producti case that airflow.cfg is either > > explicitly provided or the configuration folder is read-only. And > > there we could fall-back to no-encryption even if default behaviour is > > "generate unique key by default" - so we propose to opt for explicit > > failure, to avoid surprises and people not being aware that their > > sensitive data is **not** encrypted. > > > > The reason we raise it here is that this is a bit of a behaviour > > change. It's a "security hardening" and not intentional behaviour so > > we are **good** with SemVer and could do it in 3.2, however maybe > > there are some considerations that we have not thought about so wanted > > to get wider feedback. > > > > J. > > > > On Wed, Nov 12, 2025 at 1:37 AM Buğra Öztürk <[email protected]> > > wrote: > > > > > > Hello, > > > > > > Recent work on cleaning and standardising Fernet Key parsing introduced > > > some breaking changes. For example, empty Fernet keys ('') were raising > > > exceptions, especially in integration/end-to-end tests. Previously, > even > > > though comments were specific to certain use cases, empty Fernet keys > > were > > > allowed and handled by NullFernet. This approach makes the system > > > unencrypted, similar to having just an empty password while logging in > > to a > > > system. We have reverted the breaking changes to enable room for > > discussion. > > > > > > We should have a generic, agreed-upon approach for Fernet key handling. > > > Where appropriate, big warnings and documentation should be added or > > > updated. For example, our Helm chart generates a Fernet key for > > deployment > > > if not provided (empty), whereas Docker Compose does not. > > > > > > There is also a PR related to providing a Fernet key for integration > > tests, > > > where tests should always have a key to run reliably. This may not > cover > > > all cases, but it’s the first step toward making Fernet handling > > resilient > > > and consistent. > > > > > > Below is the approach discussed with a small group of people (Jarek > > Potiuk, > > > Jens Scheffler and me): > > > > > > Proposed Fernet Key parsing approach: > > > > > > > > > - > > > > > > Specified but empty ('')* →* use NullFernet (log a big warning that > > it’s > > > unencrypted). > > > - > > > > > > Not specified* →* attempt to generate and persist; fail if unable > > (e.g., > > > due to permissions). > > > - > > > > > > Specified and non-empty* →* attempt to use it; fail if format is > > invalid. > > > > > > > > > It would be great to collect any feedback or ideas. You can find some > > > useful links below. > > > Which part of the code mentioned: > > > > > > https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/models/crypto.py > > > Adding Fernet Key as default (followed by Amogh Desai with a good idea > of > > > generating the fernet dynamically in the code for integration tests, > > which > > > will be updated): https://github.com/apache/airflow/pull/58112 > > > > > > > > > Kind regards, > > > > > > -- > > > Bugra Ozturk > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
