amoghrajesh opened a new pull request, #54449:
URL: https://github.com/apache/airflow/pull/54449
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
closes: https://github.com/apache/airflow/issues/54397
## Why
As part of https://github.com/apache/airflow/issues/51545, we are trying to
avoid code duplication between `airflow-core` and `task-sdk`. The secrets
masking functionality is widely used so that it can be moved to a shared
library. This follows the pattern set by the timezone utilities
migration(https://github.com/apache/airflow/pull/53149) and ensures a single
source of truth for secrets masking logic across Airflow codebase.
## High level changes made
### 1. Created shared library structure
- **New directory**: `shared/secrets_masker/` with standard shared library
layout
- **Project configuration**: `shared/secrets_masker/pyproject.toml` defining
`apache-airflow-shared-secrets_masker`
- **Source code**: Moved from
`task-sdk/src/airflow/sdk/execution_time/secrets_masker.py` to
`shared/secrets_masker/src/airflow_shared/secrets_masker/secrets_masker.py`
### 2. Updated consuming packages
- **Workspace configuration**: Added `shared/secrets_masker` to root
`pyproject.toml` workspace members
- **Public API modules**: Created
`task-sdk/src/airflow/sdk/secrets_masker.py` that re-exports shared
functionality
- **Import statements**: Updated imports across airflow-core and task-sdk to
use new shared paths
### 4. How the updates were made:
- **Pre-commit hooks**: Leveraged existing
`check-shared-distributions-structure` and `check-shared-distributions-usage`
hooks
- **Symlinks**: Automatic creation of symlinks from
`airflow/_shared/secrets_masker` and `airflow/sdk/_shared/secrets_masker`
- **Force-include**: Automated addition to `pyproject.toml` files for wheel
packaging
## Sanity tests
### Symlink verification
```bash
# Verify symlinks exist and point to correct locations
(airflow) ➜ airflow git:(move-secrets-masker-to-shared) ✗ ls -la
airflow-core/src/airflow/_shared/secrets_masker
ls -la task-sdk/src/airflow/sdk/_shared/secrets_masker
lrwxr-xr-x@ 1 amoghdesai staff 67 Aug 13 15:59
airflow-core/src/airflow/_shared/secrets_masker ->
../../../../shared/secrets_masker/src/airflow_shared/secrets_masker
lrwxr-xr-x@ 1 amoghdesai staff 70 Aug 13 15:59
task-sdk/src/airflow/sdk/_shared/secrets_masker ->
../../../../../shared/secrets_masker/src/airflow_shared/secrets_masker
# Check symlink targets
(airflow) ➜ airflow git:(move-secrets-masker-to-shared) ✗ readlink
airflow-core/src/airflow/_shared/secrets_masker
readlink task-sdk/src/airflow/sdk/_shared/secrets_masker
../../../../shared/secrets_masker/src/airflow_shared/secrets_masker
../../../../../shared/secrets_masker/src/airflow_shared/secrets_masker
```
### Shared library test in isolation
```bash
# Test shared library works independently
(airflow) ➜ airflow git:(move-secrets-masker-to-shared) ✗ cd
shared/secrets_masker
PYTHONPATH=src python -c "
from airflow_shared.secrets_masker.secrets_masker import SecretsMasker
filt = SecretsMasker()
filt.add_mask('password')
print('✅ Shared library works in isolation')
"
✅ Shared library works in isolation
```
### Import validation
```bash
# Test imports work from consuming packages
(airflow) ➜ secrets_masker git:(move-secrets-masker-to-shared) ✗python -c
"from airflow.sdk.secrets_masker import SecretsMasker; print('✅ task-sdk import
works')""
python -c "from airflow._shared.secrets_masker.secrets_masker import
SecretsMasker; print('✅ airflow-core import works')"
✅ task-sdk import works
✅ airflow-core import works
```
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
