kaxil opened a new pull request, #74496: URL: https://github.com/apache/airflow/pull/74496
`mask_secret()` imported `airflow.sdk.execution_time.task_runner` only to check whether `SUPERVISOR_COMMS` is set. `settings.initialize()` calls `conf.mask_secrets()` at startup, which calls `mask_secret()` for each sensitive config value, so every process that imports `airflow` loaded the whole task runtime: the operator base classes, the API client, the coordinators, and the SDK configuration with a second full provider discovery. That covers every CLI command and every scheduler, API server, triggerer and dag-processor start. Outside a task process none of it is used. `SUPERVISOR_COMMS` is now read from `sys.modules`. That loses nothing: every place that sets it (the task runner's `main()`, the dag-processor, the triggerer, the callback supervisor, in-process `dag.test`) assigns it on the imported `airflow.sdk.execution_time.task_runner` module, so the module is always in `sys.modules` when comms exist. The module itself only declares the name as an annotation, so importing it never produced a value in a process where nothing had set one. Measured in the CI image (Python 3.11, all providers installed, bytecode cached), median of 9 runs, alternating with `main`: | | `main` | This PR | |---|---|---| | `python -c "import airflow"` | 1.36s | **0.80s** | | `airflow version` | 1.37s | **1.01s** | | `airflow --help` | 1.41s | **1.03s** | | `airflow config get-value core executor` | 1.35s | **1.04s** | | Shell completion of `airflow dags list --o` | 1.24s | **0.91s** | The new subprocess test fails on `main`, because `task_runner` ends up in `sys.modules` after a single `mask_secret()` call. The two forwarding tests keep the task-execution path covered for both `mask_secret` and `amask_secret`. --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
