This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 8c93a257030 Move DAG file processing docs into admin/deployment
section (#45873)
8c93a257030 is described below
commit 8c93a257030566f2c84360da8ec8f7a8c2aa73b5
Author: Jed Cunningham <[email protected]>
AuthorDate: Tue Jan 21 16:31:24 2025 -0700
Move DAG file processing docs into admin/deployment section (#45873)
This topic is more of admin/deployment than it is a
authoring/scheduler.
(There are a bunch more docs changes necessary, but
I'm going to do it in chunks so it's easier to review.)
---
.../dagfile-processing.rst | 0
docs/apache-airflow/administration-and-deployment/index.rst | 1 +
.../apache-airflow/administration-and-deployment/scheduler.rst | 9 ---------
docs/apache-airflow/authoring-and-scheduling/index.rst | 1 -
docs/apache-airflow/core-concepts/overview.rst | 10 ++++------
docs/apache-airflow/redirects.txt | 3 ++-
6 files changed, 7 insertions(+), 17 deletions(-)
diff --git
a/docs/apache-airflow/authoring-and-scheduling/dagfile-processing.rst
b/docs/apache-airflow/administration-and-deployment/dagfile-processing.rst
similarity index 100%
rename from docs/apache-airflow/authoring-and-scheduling/dagfile-processing.rst
rename to
docs/apache-airflow/administration-and-deployment/dagfile-processing.rst
diff --git a/docs/apache-airflow/administration-and-deployment/index.rst
b/docs/apache-airflow/administration-and-deployment/index.rst
index 5ba79a5439b..1d1fbdf5f9a 100644
--- a/docs/apache-airflow/administration-and-deployment/index.rst
+++ b/docs/apache-airflow/administration-and-deployment/index.rst
@@ -31,6 +31,7 @@ This section contains information about deploying DAGs into
production and the a
dag-serialization
modules_management
scheduler
+ dagfile-processing
pools
cluster-policies
priority-weight
diff --git a/docs/apache-airflow/administration-and-deployment/scheduler.rst
b/docs/apache-airflow/administration-and-deployment/scheduler.rst
index d3f05738e11..f3e8adbf6b7 100644
--- a/docs/apache-airflow/administration-and-deployment/scheduler.rst
+++ b/docs/apache-airflow/administration-and-deployment/scheduler.rst
@@ -67,15 +67,6 @@ In the UI, it appears as if Airflow is running your tasks a
day **late**
For more read about that you can reference `this GitHub discussion
<https://github.com/apache/airflow/discussions/28809>`__.
-DAG File Processing
--------------------
-
-You can have the Airflow Scheduler be responsible for starting the process
that turns the Python files contained in the DAGs folder into DAG objects
-that contain tasks to be scheduled.
-
-Refer to :doc:`../authoring-and-scheduling/dagfile-processing` for details on
how this can be achieved
-
-
Triggering DAG with Future Date
-------------------------------
diff --git a/docs/apache-airflow/authoring-and-scheduling/index.rst
b/docs/apache-airflow/authoring-and-scheduling/index.rst
index 28cc46ab061..aabec6dbbb4 100644
--- a/docs/apache-airflow/authoring-and-scheduling/index.rst
+++ b/docs/apache-airflow/authoring-and-scheduling/index.rst
@@ -28,7 +28,6 @@ It's recommended that you first review the pages in
:doc:`core concepts </core-c
plugins
deferring
- dagfile-processing
serializers
connections
dynamic-task-mapping
diff --git a/docs/apache-airflow/core-concepts/overview.rst
b/docs/apache-airflow/core-concepts/overview.rst
index e10cf9d5785..19f0775f9f8 100644
--- a/docs/apache-airflow/core-concepts/overview.rst
+++ b/docs/apache-airflow/core-concepts/overview.rst
@@ -49,6 +49,10 @@ A minimal Airflow installation consists of the following
components:
a configuration property of the *scheduler*, not a separate component and
runs within the scheduler
process. There are several executors available out of the box, and you can
also write your own.
+* A *dag processor*, which parses DAG files and serializes them into the
+ *metadata database*. More about processing DAG files can be found in
+ :doc:`/administration-and-deployment/dagfile-processing`
+
* A *webserver*, which presents a handy user interface to inspect, trigger and
debug the behaviour of
DAGs and tasks.
@@ -74,12 +78,6 @@ performance in your Airflow:
where deferred tasks are not used, a triggerer is not necessary. More about
deferring tasks can be
found in :doc:`/authoring-and-scheduling/deferring`.
-* Optional *dag processor*, which parses DAG files and serializes them into the
- *metadata database*. By default, the *dag processor* process is part of the
scheduler, but it can
- be run as a separate component for scalability and security reasons. If *dag
processor* is present
- *scheduler* does not need to read the *DAG files* directly. More about
- processing DAG files can be found in
:doc:`/authoring-and-scheduling/dagfile-processing`
-
* Optional folder of *plugins*. Plugins are a way to extend Airflow's
functionality (similar to installed
packages). Plugins are read by the *scheduler*, *dag processor*, *triggerer*
and *webserver*. More about
plugins can be found in :doc:`/authoring-and-scheduling/plugins`.
diff --git a/docs/apache-airflow/redirects.txt
b/docs/apache-airflow/redirects.txt
index 8f5bac18abf..4dc01dce75c 100644
--- a/docs/apache-airflow/redirects.txt
+++ b/docs/apache-airflow/redirects.txt
@@ -135,7 +135,6 @@ concepts/xcoms.rst core-concepts/xcoms.rst
concepts/scheduler.rst administration-and-deployment/scheduler.rst
concepts/pools.rst administration-and-deployment/pools.rst
concepts/priority-weight.rst administration-and-deployment/priority-weight.rst
-concepts/dagfile-processing.rst authoring-and-scheduling/dagfile-processing.rst
concepts/deferring.rst authoring-and-scheduling/deferring.rst
concepts/datasets.rst authoring-and-scheduling/datasets.rst
concepts/cluster-policies.rst
administration-and-deployment/cluster-policies.rst
@@ -150,6 +149,8 @@ executor/sequential.rst
core-concepts/executor/sequential.rst
upgrading-from-1-10/upgrade-check.rst
howto/upgrading-from-1-10/upgrade-check.rst
core-concepts/dag-run.rst authoring-and-scheduling/cron.rst
core-concepts/executor/debug.rst core-concepts/debug.rst
+concepts/dagfile-processing.rst
administration-and-deployment/dagfile-processing.rst
+authoring-and-scheduling/dagfile-processing.rst
administration-and-deployment/dagfile-processing.rst
# Moving provider executor docs to providers
## The ``../`` indicates that it will move to the root of the docs directory,
unlike the rest of
