This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v3-0-test by this push:
new 917674b6ad0 [v3-0-test] Common airflow.cfg files across all containers
in default docker-compose.yaml (#49681) (#49775)
917674b6ad0 is described below
commit 917674b6ad08029600cacf03675c2e98f95189df
Author: Jarek Potiuk <[email protected]>
AuthorDate: Fri Apr 25 13:02:30 2025 +0200
[v3-0-test] Common airflow.cfg files across all containers in default
docker-compose.yaml (#49681) (#49775)
* Mount airflow project dir to containers airflow home
* Remove dags/logs/config/plugin mounts
They are no longer needed as we are directly mounting AIRFLOW_HOME
* Set AIRFLOW_CFG in common env
* Update docs and airflow init step to align with common-env
* Fix spell check and static checks
---------
(cherry picked from commit ec49402dd4cc87088fd58565a226d0b44dbd2fde)
Co-authored-by: Dheeraj Turaga <[email protected]>
Co-authored-by: Amogh Desai <[email protected]>
---
.../docs/howto/docker-compose/docker-compose.yaml | 9 +++------
airflow-core/docs/howto/docker-compose/index.rst | 22 +++++++++++++++-------
2 files changed, 18 insertions(+), 13 deletions(-)
diff --git a/airflow-core/docs/howto/docker-compose/docker-compose.yaml
b/airflow-core/docs/howto/docker-compose/docker-compose.yaml
index cb99c6085db..603d373e4fa 100644
--- a/airflow-core/docs/howto/docker-compose/docker-compose.yaml
+++ b/airflow-core/docs/howto/docker-compose/docker-compose.yaml
@@ -71,8 +71,7 @@ x-airflow-common:
# for other purpose (development, test and especially production usage)
build/extend Airflow image.
_PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}
# The following line can be used to set a custom config file, stored in
the local config folder
- # If you want to use it, outcomment it and replace airflow.cfg with the
name of your config file
- # AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'
+ AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'
volumes:
- ${AIRFLOW_PROJ_DIR:-.}/dags:/opt/airflow/dags
- ${AIRFLOW_PROJ_DIR:-.}/logs:/opt/airflow/logs
@@ -252,8 +251,8 @@ services:
echo "
https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html#before-you-begin";
echo
fi
- mkdir -p /sources/logs /sources/dags /sources/plugins
- chown -R "${AIRFLOW_UID}:0" /sources/{logs,dags,plugins}
+ mkdir -p /opts/airflow/{logs,dags,plugins,config}
+ chown -R "${AIRFLOW_UID}:0" /opts/airflow/{logs,dags,plugins,config}
exec /entrypoint airflow version
# yamllint enable rule:line-length
environment:
@@ -264,8 +263,6 @@ services:
_AIRFLOW_WWW_USER_PASSWORD: ${_AIRFLOW_WWW_USER_PASSWORD:-airflow}
_PIP_ADDITIONAL_REQUIREMENTS: ''
user: "0:0"
- volumes:
- - ${AIRFLOW_PROJ_DIR:-.}:/sources
airflow-cli:
<<: *airflow-common
diff --git a/airflow-core/docs/howto/docker-compose/index.rst
b/airflow-core/docs/howto/docker-compose/index.rst
index 1e60262da14..270dd70e935 100644
--- a/airflow-core/docs/howto/docker-compose/index.rst
+++ b/airflow-core/docs/howto/docker-compose/index.rst
@@ -125,8 +125,8 @@ Setting the right Airflow user
------------------------------
On **Linux**, the quick-start needs to know your host user id and needs to
have group id set to ``0``.
-Otherwise the files created in ``dags``, ``logs`` and ``plugins`` will be
created with ``root`` user ownership.
-You have to make sure to configure them for the docker-compose:
+Otherwise the files created in ``dags``, ``logs``, ``config`` and ``plugins``
will be created with
+``root`` user ownership. You have to make sure to configure them for the
docker-compose:
.. code-block:: bash
@@ -143,6 +143,17 @@ safely ignore it. You can also manually create an ``.env``
file in the same fold
AIRFLOW_UID=50000
+Initialize airflow.cfg (Optional)
+---------------------------------
+
+If you want to initialize ``airflow.cfg`` with default values before launching
the airflow service, run.
+
+.. code-block:: bash
+
+ docker compose run airflow-cli airflow config list
+
+This will seed ``airflow.cfg`` with default values in ``config`` folder.
+
Initialize the database
-----------------------
@@ -346,12 +357,9 @@ Special case - Adding a custom config file
If you have a custom config file and wish to use it in your Airflow instance,
you need to perform the following steps:
-1) Remove comment from the ``AIRFLOW_CONFIG:
'/opt/airflow/config/airflow.cfg'`` line
- in the ``docker-compose.yaml`` file.
-
-2) Place your custom ``airflow.cfg`` file in the local config folder.
+1) Replace the auto-generated ``airflow.cfg`` file in the local config folder
with your custom config file.
-3) If your config file has a different name than ``airflow.cfg``, adjust the
filename in
+2) If your config file has a different name than ``airflow.cfg``, adjust the
filename in
``AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'``
Networking
