jason810496 opened a new pull request, #74042: URL: https://github.com/apache/airflow/pull/74042
STACKLINE related: #71929 Part of the native Dag e2e stack, split out of #74035. Builds on P2 ([layer diff](https://github.com/apache/airflow/compare/jason/lang-sdk-e2e/03b-serialized-dag-validation...jason/lang-sdk-e2e/03c-coordinator-dag-parsing)). Task SDK only. ## Why To parse native Dags, a coordinator has to say which Dag bundles it parses, and the Dag importer registry has to use its importer for them. ## What changes - `BaseCoordinator.get_dag_importer_class()` names the class of the Dag importer a coordinator hands out, and `get_dag_importer()` returns it. Both return `None` by default, so a coordinator opts in. - `SubprocessCoordinator.serves_bundle()` and `get_parsed_bundles(kwargs)`: a coordinator parses its `dag_bundle_name` bundle, or every bundle when it sets neither that nor an explicit root. - `DagImporterRegistry.from_config(bundle_name)` registers the importers of the coordinators that parse the bundle. - `CoordinatorManager.from_config` rejects two coordinators whose importers claim one extension in a bundle, before either is built. - A Lang-SDK runtime gets the resolved `[api] base_url`, `[operators] default_deferrable` and `[triggerer] queues_enabled` in its environment, like the log levels. A coordinator opts in by naming its importer class, and parses the bundle set in its config: ```python class MyCoordinator(SubprocessCoordinator): @classmethod def get_dag_importer_class(cls): return MyDagImporter # supported_extensions = [".jar"] ``` ```ini [sdk] coordinators = {"mine": {"classpath": "my_pkg.MyCoordinator", "kwargs": {"dag_bundle_name": "jar-dags"}}} ``` ## How to test ```bash uv run --project task-sdk pytest task-sdk/tests/task_sdk/coordinators task-sdk/tests/task_sdk/importers task-sdk/tests/task_sdk/execution_time/test_coordinator.py -xvs ``` --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code Opus 5.5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
