o-nikolas commented on code in PR #73958:
URL: https://github.com/apache/airflow/pull/73958#discussion_r4150104500
##########
airflow-core/docs/core-concepts/multi-team.rst:
##########
@@ -663,6 +685,41 @@ Team filtering and queue filtering are orthogonal — they
combine as AND condit
remain unassigned until one starts — the same applies to every queue when
``--queues`` is used. If you
combine ``--team-name`` and ``--queues``, this requirement extends to each
team-and-queue combination.
+.. _multi-team-dag-processor:
+
+Team-scoped Dag Processing
+--------------------------
+
+The Dag processor parses Dag files from your configured Dag bundles. Unlike
the triggerer, it is not
+scoped with ``--team-name`` directly; it is scoped by **bundle** using the
``--bundle-name`` CLI argument
+(which may be passed more than once). Because each bundle is owned by at most
one team, scoping a
+processor to a team's bundles scopes it to that team. The teams a processor
serves are derived from the
Review Comment:
nit suggestion
```suggestion
processor to a team's bundle(s) scopes it to that team. The teams a
processor serves are derived from the
```
##########
airflow-core/docs/core-concepts/multi-team.rst:
##########
@@ -663,6 +685,41 @@ Team filtering and queue filtering are orthogonal — they
combine as AND condit
remain unassigned until one starts — the same applies to every queue when
``--queues`` is used. If you
combine ``--team-name`` and ``--queues``, this requirement extends to each
team-and-queue combination.
+.. _multi-team-dag-processor:
+
+Team-scoped Dag Processing
+--------------------------
+
+The Dag processor parses Dag files from your configured Dag bundles. Unlike
the triggerer, it is not
+scoped with ``--team-name`` directly; it is scoped by **bundle** using the
``--bundle-name`` CLI argument
+(which may be passed more than once). Because each bundle is owned by at most
one team, scoping a
+processor to a team's bundles scopes it to that team. The teams a processor
serves are derived from the
+:ref:`Dag bundle config <multi-team-dag-bundles>`.
+
+.. code-block:: bash
+
+ # Process only team_a's bundle(s)
+ airflow dag-processor --bundle-name team_a_dags
+
+ # Process several bundles (for example, all of team_b's bundles)
+ airflow dag-processor --bundle-name team_b_dags --bundle-name
team_b_extra_dags
+
+ # Process every configured bundle (all teams and global bundles)
+ airflow dag-processor
+
+Running **one Dag processor per team** (scoped to that team's bundles) is
recommended, so that each
+team's Dag code is parsed in a separate process and teams stay isolated.
Unlike the triggerer, however,
+this is not required for coverage: a global triggerer only picks up teamless
triggers, whereas a Dag
+processor started without ``--bundle-name`` parses *every* configured bundle.
A single global Dag
+processor therefore covers every team's Dags as well as global (teamless)
bundles, which is possible but
+gives up the per-team parsing isolation.
+
+.. note::
Review Comment:
Can you add another note, or maybe somewhere in the paragraph above about
further isolating the team dag processors not just by bundle but on separate
team owned compute (separate vms, docker containers, servers, etc). This isn't
mandatory but it is a way to provide the tightest boundary for dag processing
for teams.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
