vincbeck commented on code in PR #73995: URL: https://github.com/apache/airflow/pull/73995#discussion_r4155818366
########## airflow-core/docs/administration-and-deployment/plugins.rst: ########## @@ -514,6 +521,167 @@ Right-to-left languages are handled automatically: the UI derives text direction code (via the browser's locale data, e.g. Persian ``fa`` or Urdu ``ur``), so a custom RTL language flips the whole UI to right-to-left without any extra configuration. +.. _plugins-multi-team: + +Multi-team deployments +---------------------- + +.. versionadded:: 3.4.0 + +A plugin can name the team that owns it by setting ``team_name``. Airflow then offers what the +plugin contributes to that team only, instead of to the whole deployment: + +.. code-block:: python + + from airflow.plugins_manager import AirflowPlugin + + from my_package.payments import PaymentWindowTimetable, settlement_date + + + class PaymentsPlugin(AirflowPlugin): + name = "payments" + # Only this team's Dags, tasks and users get the pieces below. + team_name = "payments" + macros = [settlement_date] + timetables = [PaymentWindowTimetable] + +``team_name`` is part of the plugin's code, so the plugin author decides it; there is no +deployment-time override. Leaving it unset (the default) makes the plugin **global**: everything +it contributes is available to every team, which is how plugins written before multi-team support +behaved. + +``team_name`` only takes effect when :doc:`multi-team mode </core-concepts/multi-team>` is +enabled. With ``[core] multi_team = False`` it is ignored and every plugin is global. + +The team must already exist in the metadata database (``airflow teams create <team_name>``). The Review Comment: Should we mention `airflow teams sync` as well? ########## airflow-core/docs/administration-and-deployment/plugins.rst: ########## @@ -514,6 +521,167 @@ Right-to-left languages are handled automatically: the UI derives text direction code (via the browser's locale data, e.g. Persian ``fa`` or Urdu ``ur``), so a custom RTL language flips the whole UI to right-to-left without any extra configuration. +.. _plugins-multi-team: + +Multi-team deployments +---------------------- + +.. versionadded:: 3.4.0 + +A plugin can name the team that owns it by setting ``team_name``. Airflow then offers what the +plugin contributes to that team only, instead of to the whole deployment: + +.. code-block:: python + + from airflow.plugins_manager import AirflowPlugin + + from my_package.payments import PaymentWindowTimetable, settlement_date + + + class PaymentsPlugin(AirflowPlugin): + name = "payments" + # Only this team's Dags, tasks and users get the pieces below. + team_name = "payments" + macros = [settlement_date] + timetables = [PaymentWindowTimetable] + +``team_name`` is part of the plugin's code, so the plugin author decides it; there is no +deployment-time override. Leaving it unset (the default) makes the plugin **global**: everything +it contributes is available to every team, which is how plugins written before multi-team support +behaved. + +``team_name`` only takes effect when :doc:`multi-team mode </core-concepts/multi-team>` is +enabled. With ``[core] multi_team = False`` it is ignored and every plugin is global. + +The team must already exist in the metadata database (``airflow teams create <team_name>``). The +API server checks this when it loads plugins for the API: a plugin naming an unknown team is +recorded as a plugin import error (surfaced under *Admin → Plugins* and at +``GET /api/v2/plugins/importErrors``) and logged as a warning. It is deliberately not raised, so +one misconfigured plugin does not stop the API server, or the other plugins, from starting. This is +cached for the life of the process, so creating the team afterwards does not clear +the error until the API server restarts. The plugin still loads, and everything it scopes to the +nonexistent team is unusable in the meantime. Its scheduling classes are refused for every Dag, +its macros resolve for no task, and its extra links are shown nowhere. + +Team validation of plugins runs in the API server. Plugin loading in the other components +does no team lookup at all, and the subprocess that imports Dag files has no database session. + +Plugin *discovery* is unchanged: every Airflow component still loads every installed plugin, and +``team_name`` decides who is offered what. ``airflow plugins`` lists each plugin's ``team_name``. + +.. list-table:: + :header-rows: 1 + :widths: 34 66 + + * - Plugin attribute + - Effect of ``team_name`` + * - ``fastapi_apps`` + - App is mounted behind a team check; only that team's users may call it. + * - ``fastapi_root_middlewares`` + - Skipped, with a warning. A root middleware cannot be scoped to one team. + * - ``external_views``, ``react_apps`` + - Only offered in the UI to that team's users. + * - ``macros`` + - Only resolvable when rendering templates for that team's tasks. + * - ``global_operator_extra_links``, ``operator_extra_links`` + - Only shown on task instances of that team's Dags. + * - ``timetables``, ``partition_mappers``, ``windows``, ``deadline_references``, + ``priority_weight_strategies`` + - Only usable by that team's Dags. + * - ``listeners`` + - None. Every listener receives events for all teams. + * - ``ui_translations``, ``hook_lineage_readers``, ``flask_blueprints``, + ``appbuilder_views``, ``appbuilder_menu_items``, ``admin_views``, ``menu_links`` + - None. These stay global. + +Listeners are deployment-wide by design: every listener, including one a team plugin registers, +receives events for every team's Dags. Their hooks fire in shared components such as the scheduler +and the API server, so choosing which listeners run is the Deployment Manager's responsibility. + +API endpoints +^^^^^^^^^^^^^ + +A team plugin's ``fastapi_apps`` are mounted with a middleware that resolves the caller (bearer +token or UI session cookie, exactly as the core API does) and asks the auth manager whether that +user is authorized for the team. A caller the middleware cannot authenticate gets the same +``401``/``403`` the core API returns for that token; an authenticated caller who is not authorized +for the team gets ``403``. The request's HTTP method is mapped onto one of Airflow's resource +methods (``GET``/``HEAD``/``OPTIONS`` → ``GET``, ``POST`` → ``POST``, ``PUT``/``PATCH`` → ``PUT``, +``DELETE`` → ``DELETE``) before it is passed on, so an auth manager that distinguishes methods can +grant a team read-only access to its own plugin. Auth managers that only check membership can +ignore it. + +This is the only authorization Airflow adds to a plugin app. Whether the caller may perform a +given action *within* the team is still the plugin's decision, and a **global** plugin's app gets +no authentication and no team check at all — see the warning in :ref:`the example above +<plugin-example>`. + +``fastapi_root_middlewares`` are not scoped. A root middleware wraps every request to the API +server, including core routes and other teams' plugins, so one declared by a team plugin is +skipped and a warning is logged. A team plugin that needs middleware should apply it inside its +own FastAPI app, where it only sees that app's requests. + +UI elements +^^^^^^^^^^^ + +The UI builds its navigation items, external views and React apps from ``GET /api/v2/plugins``, +which returns global plugins plus the plugins of teams the caller is authorized for. A team +plugin's UI pieces are therefore only offered to that team's users. The endpoint still requires +the existing *Plugins* view permission; team scoping narrows what that permission returns rather +than introducing a separate one. + +``GET /api/v2/plugins/importErrors`` is **not** filtered by team: anyone with the *Plugins* view +permission sees the import errors of all plugins, including their source paths and error text. A +plugin load failure is deployment-level information that the person debugging it needs, so it is +reported the same way to everyone who may see the plugins page at all. Review Comment: We definitely can do it as a follow-up but we might want to improve that? I should not see erro from a plugin I dont have access? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
