This is an automated email from the ASF dual-hosted git repository.

Miretpl pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 9e6126a19c1 Allow mounting the Airflow config as a directory to avoid 
the subPath mount race (#70180)
9e6126a19c1 is described below

commit 9e6126a19c1bb15958219156f205be6cf01867f4
Author: Diego Álvarez <[email protected]>
AuthorDate: Mon Nov 16 22:52:46 2026 +0100

    Allow mounting the Airflow config as a directory to avoid the subPath mount 
race (#70180)
    
    * Allow mounting the Airflow config as a directory to avoid the subPath 
mount race
    
    The chart mounts airflow.cfg (and airflow_local_settings.py) into every 
component and
    the KubernetesExecutor pod template via a subPath bind-mount of the config 
ConfigMap.
    subPath + ConfigMap hits a known Kubernetes race: on node/volume lifecycle 
events
    (kubelet volume reconstruction, node autoscaling) the pre-computed subPath 
source inode
    goes stale and runc fails the mount at container init with a StartError and 
no
    application log. It is intermittent, cross-component, and 
runtime-version-independent.
    
    Add an opt-in mountConfigAsDir (default false, existing behaviour 
unchanged). When true,
    the config ConfigMap is mounted as a directory at {airflowHome}/config and 
AIRFLOW_CONFIG
    is set to {airflowHome}/config/airflow.cfg. A directory-projected ConfigMap 
volume has no
    pre-computed subPath bind source, so it is immune to the race. 
airflow_local_settings.py
    keeps its existing path.
    
    Refs: #33788
    
    Co-authored-by: diego15ag <[email protected]>
    
    * Name newsfragment after PR number
    
    * Fix docs spellcheck in mountConfigAsDir description
    
    Sphinx spellcheck (chart docs) flagged "inode" and "runc". Reword
    "stale-inode" to "stale-mount" and backtick ``runc`` so both are accepted,
    keeping the description precise.
    
    Co-authored-by: diego15ag <[email protected]>
    
    * Format test with ruff
    
    * Reuse airflow_config_dir helper for the AIRFLOW_CONFIG env var
    
    The helper and the env var both derived {airflowHome}/config independently.
    Make the helper return the unquoted path and quote at the two call sites, so
    the config subdirectory is encoded once. Rendered output is unchanged.
    
    * Describe mountConfigAsDir in the values docs instead of justifying it
    
    Use the reviewer's wording for the values.yaml comment and the schema
    description; the rationale lives in the PR description. Drop the
    newsfragment as not needed.
    
    * Address review on the chart config-mount tests
    
    Cover every component that includes the config mount helper, not only the
    core five, and branch on kind for the CronJob pod path. Express the mount
    and env assertions as jmespath filters and drop the length assertion that
    checked nothing. Split the default-mount test so the default case renders
    with no values, and move the airflowLocalSettings case to its own test.
    
    ---------
    
    Co-authored-by: diego15ag <[email protected]>
---
 chart/templates/_helpers.yaml                      | 14 ++++
 .../helm_tests/airflow_aux/test_airflow_common.py  | 79 ++++++++++++++++++++++
 chart/values.schema.json                           |  6 ++
 chart/values.yaml                                  |  4 ++
 4 files changed, 103 insertions(+)

diff --git a/chart/templates/_helpers.yaml b/chart/templates/_helpers.yaml
index 1f0074b3c66..4ddbeecafcb 100644
--- a/chart/templates/_helpers.yaml
+++ b/chart/templates/_helpers.yaml
@@ -63,6 +63,10 @@ If release name contains chart name it will be used as a 
full name.
   # Hard Coded Airflow Envs
   - name: AIRFLOW_HOME
     value: {{ .Values.airflowHome }}
+  {{- if .Values.mountConfigAsDir }}
+  - name: AIRFLOW_CONFIG
+    value: {{ printf "%s/airflow.cfg" (include "airflow_config_dir" .) | quote 
}}
+  {{- end }}
   {{- if .Values.enableBuiltInSecretEnvVars.AIRFLOW__CORE__FERNET_KEY }}
   - name: AIRFLOW__CORE__FERNET_KEY
     valueFrom:
@@ -736,7 +740,16 @@ server_tls_key_file = /etc/pgbouncer/server.key
   {{- printf "%s-config" (include "airflow.fullname" .) }}
 {{- end }}
 
+{{- define "airflow_config_dir" -}}
+  {{- printf "%s/config" .Values.airflowHome }}
+{{- end }}
+
 {{- define "airflow_config_mount" -}}
+{{- if .Values.mountConfigAsDir }}
+- name: config
+  mountPath: {{ include "airflow_config_dir" . | quote }}
+  readOnly: true
+{{- else }}
 - name: config
   mountPath: {{ template "airflow_config_path" . }}
   subPath: airflow.cfg
@@ -748,6 +761,7 @@ server_tls_key_file = /etc/pgbouncer/server.key
   readOnly: true
   {{- end }}
 {{- end }}
+{{- end }}
 
 {{/* Helper for service account name generation */}}
 {{- define "_serviceAccountNameGen" -}}
diff --git a/chart/tests/helm_tests/airflow_aux/test_airflow_common.py 
b/chart/tests/helm_tests/airflow_aux/test_airflow_common.py
index 8c3418f7559..0b9560d4860 100644
--- a/chart/tests/helm_tests/airflow_aux/test_airflow_common.py
+++ b/chart/tests/helm_tests/airflow_aux/test_airflow_common.py
@@ -652,6 +652,85 @@ class TestAirflowCommon:
         for doc in docs:
             assert expected_mount in 
jmespath.search("spec.template.spec.initContainers[0].volumeMounts", doc)
 
+    def test_mount_config_as_dir(self):
+        components = [
+            "templates/scheduler/scheduler-deployment.yaml",
+            "templates/workers/worker-deployment.yaml",
+            "templates/api-server/api-server-deployment.yaml",
+            "templates/triggerer/triggerer-deployment.yaml",
+            "templates/dag-processor/dag-processor-deployment.yaml",
+            "templates/flower/flower-deployment.yaml",
+            "templates/cleanup/cleanup-cronjob.yaml",
+            "templates/database-cleanup/database-cleanup-cronjob.yaml",
+            "templates/jobs/migrate-database-job.yaml",
+            "templates/jobs/create-user-job.yaml",
+        ]
+        docs = render_chart(
+            values={
+                "mountConfigAsDir": True,
+                "airflowLocalSettings": "# local settings",
+                "executor": "CeleryExecutor,KubernetesExecutor",
+                "flower": {"enabled": True},
+                "cleanup": {"enabled": True},
+                "databaseCleanup": {"enabled": True},
+            },
+            show_only=components,
+        )
+        dir_mount = {"name": "config", "mountPath": "/opt/airflow/config", 
"readOnly": True}
+        for doc in docs:
+            pod = "spec.jobTemplate.spec.template.spec" if doc["kind"] == 
"CronJob" else "spec.template.spec"
+            assert 
jmespath.search(f"{pod}.containers[0].volumeMounts[?name=='config' && 
!subPath]", doc) == [
+                dir_mount
+            ]
+            assert (
+                jmespath.search(
+                    f"{pod}.containers[0].volumeMounts[?subPath=='airflow.cfg' 
|| subPath=='airflow_local_settings.py']",
+                    doc,
+                )
+                == []
+            )
+            assert 
jmespath.search(f"{pod}.containers[0].env[?name=='AIRFLOW_CONFIG'].value", doc) 
== [
+                "/opt/airflow/config/airflow.cfg"
+            ]
+
+    def test_config_mounted_via_subpath_by_default(self):
+        docs = 
render_chart(show_only=["templates/scheduler/scheduler-deployment.yaml"])
+        mounts = "spec.template.spec.containers[0].volumeMounts"
+        assert jmespath.search(f"{mounts}[?subPath=='airflow.cfg']", docs[0]) 
== [
+            {
+                "name": "config",
+                "mountPath": "/opt/airflow/airflow.cfg",
+                "subPath": "airflow.cfg",
+                "readOnly": True,
+            }
+        ]
+        assert 
jmespath.search(f"{mounts}[?subPath=='airflow_local_settings.py']", docs[0]) == 
[]
+        assert jmespath.search(f"{mounts}[?name=='config' && !subPath]", 
docs[0]) == []
+        assert 
jmespath.search("spec.template.spec.containers[0].env[?name=='AIRFLOW_CONFIG']",
 docs[0]) == []
+
+    def test_local_settings_mounted_via_subpath_when_set(self):
+        docs = render_chart(
+            values={"airflowLocalSettings": "# local settings"},
+            show_only=["templates/scheduler/scheduler-deployment.yaml"],
+        )
+        mounts = "spec.template.spec.containers[0].volumeMounts"
+        assert jmespath.search(f"{mounts}[?subPath=='airflow.cfg']", docs[0]) 
== [
+            {
+                "name": "config",
+                "mountPath": "/opt/airflow/airflow.cfg",
+                "subPath": "airflow.cfg",
+                "readOnly": True,
+            }
+        ]
+        assert 
jmespath.search(f"{mounts}[?subPath=='airflow_local_settings.py']", docs[0]) == 
[
+            {
+                "name": "config",
+                "mountPath": "/opt/airflow/config/airflow_local_settings.py",
+                "subPath": "airflow_local_settings.py",
+                "readOnly": True,
+            }
+        ]
+
     def test_priority_class_name(self):
         docs = render_chart(
             values={
diff --git a/chart/values.schema.json b/chart/values.schema.json
index ad7fd9aebd6..244f11b7012 100644
--- a/chart/values.schema.json
+++ b/chart/values.schema.json
@@ -691,6 +691,12 @@
             "x-docsSection": "Common",
             "default": "See values.yaml"
         },
+        "mountConfigAsDir": {
+            "description": "Mount the Airflow config ConfigMap as a directory 
(at ``{airflowHome}/config``) and set accordingly, instead of mounting 
``airflow.cfg`` via ``subPath``.",
+            "type": "boolean",
+            "x-docsSection": "Kubernetes",
+            "default": false
+        },
         "rbac": {
             "description": "Enable RBAC (default on most clusters these 
days).",
             "type": "object",
diff --git a/chart/values.yaml b/chart/values.yaml
index dab6066525f..a551f664c29 100644
--- a/chart/values.yaml
+++ b/chart/values.yaml
@@ -313,6 +313,10 @@ airflowConfigAnnotations: {}
 # 'airflow_local_settings' file as a string (templated)
 airflowLocalSettings: ~
 
+# Mount the Airflow config ConfigMap as a directory (at 
``{airflowHome}/config``) and set
+# accordingly, instead of mounting ``airflow.cfg`` via ``subPath``.
+mountConfigAsDir: false
+
 # Enable RBAC (default on most clusters these days)
 rbac:
   # Specifies whether RBAC resources should be created

Reply via email to