amoghrajesh commented on PR #54449:
URL: https://github.com/apache/airflow/pull/54449#issuecomment-3205508675
> What we also need (following my comment) - is a separate test GitHub
Action job that will be running "secrets_masker" and generally (later) - any
other shared code in isolation.
>
> Basically what that job should do is:
>
> ```
> cd shared/secrets_masker
> uv run pytest
> ```
>
> This will make sure that the tests are passing and also automatically do
"isolation" test - which means that it will automatically test if we are
actually not using something in secrets_masker that we have not declared as
dependency.
Yep I totally agree. That made me wonder if it would work at all.
Tnx to your comment @potiuk I checked in detail some failing tests and now
it works as expected:
```
➜ airflow git:(move-secrets-masker-to-shared) ✗ cd shared/secrets_masker uv
run pytest
warning: The package `pandas==2.1.4` does not have an extra named `postgres`
Built apache-airflow-shared-secrets-masker @
file:///Users/amoghdesai/Documents/OSS/repos/airflow/shared/secrets_masker
Uninstalled 1 package in 1ms
Installed 1 package in 1ms
=============================================================================
test session starts
==============================================================================
platform darwin -- Python 3.13.3, pytest-8.4.1, pluggy-1.6.0 --
/Users/amoghdesai/Documents/OSS/repos/airflow/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/amoghdesai/Documents/OSS/repos/airflow
configfile: pyproject.toml
plugins: unordered-0.7.0, instafail-0.5.0, timeouts-1.2.1, asyncio-1.1.0,
xdist-3.8.0, custom-exit-code-0.3.0, anyio-4.10.0, icdiff-0.9,
rerunfailures-15.1, cov-6.2.1, kgb-7.2, mock-3.14.1, time-machine-2.17.0,
requests-mock-1.12.1
asyncio: mode=Mode.STRICT, asyncio_default_fixture_loop_scope=function,
asyncio_default_test_loop_scope=function
setup timeout: 0.0s, execution timeout: 0.0s, teardown timeout: 0.0s
collected 103 items
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_message
PASSED
[ 0%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_args
PASSED
[ 1%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_extra
PASSED
[ 2%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_exception
PASSED
[ 3%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_exception_not_raised
PASSED
[ 4%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_exc_tb
XFAIL (Cannot filter secrets in traceback source)
[ 5%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_masking_in_implicit_context_exceptions
PASSED [ 6%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_masking_in_explicit_context_exceptions
PASSED [ 7%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_mask_secret[None-secret-expected_mask0]
PASSED [ 8%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_mask_secret[apikey-secret-expected_mask1]
PASSED [ 9%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_mask_secret[None-value2-expected_mask2]
PASSED [ 10%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_mask_secret[None-value3-expected_mask3]
PASSED [ 11%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_mask_secret[api_key-value4-expected_mask4]
PASSED [ 12%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_mask_secret[None-value5-expected_mask5]
PASSED [ 13%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_mask_secret[None--expected_mask6]
PASSED [ 14%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact[patterns0-None-secret-***]
PASSED [ 15%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact[patterns1-None-value1-expected1]
PASSED [ 16%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact[patterns2-None-value2-expected2]
PASSED [ 17%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact[patterns3-None-value3-expected3]
PASSED [ 18%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact[patterns4-None-value4-expected4]
PASSED [ 19%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact[patterns5-api_key-value5-expected5]
PASSED [ 20%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact[patterns6-env-value6-expected6]
PASSED [ 21%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_replacement[api_key-pass-*\ufe0f\u20e3*\ufe0f\u20e3*\ufe0f\u20e3]
PASSED [ 22%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_replacement[api_key-value1-expected1]
PASSED [ 23%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_replacement[None-value2-expected2]
PASSED [ 24%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_replacement[None-value3-expected3]
PASSED [ 25%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_replacement[api_key-value4-expected4]
PASSED [ 26%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_filehandles
PASSED
[ 27%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_max_depth[val0-expected0-None]
PASSED [ 28%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_max_depth[val1-expected1-1]
PASSED [ 29%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_max_depth[val2-expected2-None]
PASSED [ 30%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_max_depth[val3-expected3-None]
PASSED [ 31%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_max_depth[val4-expected4-None]
PASSED [ 32%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_max_depth[val5-expected5-1]
PASSED [ 33%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_with_str_type
PASSED
[ 33%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_state_enum[success-success]
PASSED [ 34%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_state_enum[failed-failed]
PASSED [ 35%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_state_enum[running-running]
PASSED [ 36%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_state_enum[state3-expected3]
PASSED [ 37%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_state_enum[state4-expected4]
PASSED [ 38%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_state_enum[state5-expected5]
PASSED [ 39%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_redact_state_enum[testvalue-testvalue]
PASSED [ 40%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_reset_secrets_masker
PASSED
[ 41%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_defaults[-False]
PASSED [ 42%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_defaults[None-False]
PASSED [ 43%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_defaults[key-False]
PASSED [ 44%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_defaults[google_api_key-True]
PASSED [ 45%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_defaults[GOOGLE_API_KEY-True]
PASSED [ 46%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_defaults[GOOGLE_APIKEY-True]
PASSED [ 47%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_defaults[1-False]
PASSED [ 48%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[key-TRELLO_KEY-True]
PASSED [ 49%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[key-TRELLO_API_KEY-True]
PASSED [ 50%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[key-GITHUB_APIKEY-True]
PASSED [ 51%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[key,
token-TRELLO_TOKEN-True] PASSED [ 52%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[mysecretword,
mysensitivekey-GITHUB_mysecretword-True] PASSED [ 53%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[None-TRELLO_API-False]
PASSED [ 54%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[token-TRELLO_KEY-False]
PASSED [ 55%]
tests/secrets_masker/test_secrets_masker.py::TestShouldHideValueForKey::test_hiding_config[token,
mysecretword-TRELLO_KEY-False] PASSED [ 56%]
tests/secrets_masker/test_secrets_masker.py::TestRedactedIO::test_redacts_from_print
PASSED
[ 57%]
tests/secrets_masker/test_secrets_masker.py::TestRedactedIO::test_write
PASSED
[ 58%]
tests/secrets_masker/test_secrets_masker.py::TestRedactedIO::test_input_builtin
PASSED
[ 59%]
tests/secrets_masker/test_secrets_masker.py::TestMaskSecretAdapter::test_calling_mask_secret_adds_adaptations_for_returned_str
PASSED [ 60%]
tests/secrets_masker/test_secrets_masker.py::TestMaskSecretAdapter::test_calling_mask_secret_adds_adaptations_for_returned_iterable
PASSED [ 61%]
tests/secrets_masker/test_secrets_masker.py::TestMaskSecretAdapter::test_calling_mask_secret_not_set
PASSED [ 62%]
tests/secrets_masker/test_secrets_masker.py::TestMaskSecretAdapter::test_add_mask_short_secrets_and_skip_keywords[abc-False-True-short
secret with first warning] PASSED [ 63%]
tests/secrets_masker/test_secrets_masker.py::TestMaskSecretAdapter::test_add_mask_short_secrets_and_skip_keywords[def-False-False-short
secret with no warning] PASSED [ 64%]
tests/secrets_masker/test_secrets_masker.py::TestMaskSecretAdapter::test_add_mask_short_secrets_and_skip_keywords[airflow-False-False-keyword
that should be skipped] PASSED [ 65%]
tests/secrets_masker/test_secrets_masker.py::TestMaskSecretAdapter::test_add_mask_short_secrets_and_skip_keywords[valid_secret-True-False-valid
secret that should be masked] PASSED [ 66%]
tests/secrets_masker/test_secrets_masker.py::TestStructuredVsUnstructuredMasking::test_structured_sensitive_fields_always_masked
PASSED [ 66%]
tests/secrets_masker/test_secrets_masker.py::TestStructuredVsUnstructuredMasking::test_unstructured_text_min_length_enforced
PASSED [ 67%]
tests/secrets_masker/test_secrets_masker.py::TestContainerTypesRedaction::test_kubernetes_env_var_redaction
PASSED [ 68%]
tests/secrets_masker/test_secrets_masker.py::TestContainerTypesRedaction::test_deeply_nested_mixed_structures
PASSED [ 69%]
tests/secrets_masker/test_secrets_masker.py::TestEdgeCases::test_circular_references
PASSED
[ 70%]
tests/secrets_masker/test_secrets_masker.py::TestEdgeCases::test_regex_special_chars_in_secrets
PASSED [ 71%]
tests/secrets_masker/test_secrets_masker.py::TestDirectMethodCalls::test_redact_all_directly
PASSED [
72%]
tests/secrets_masker/test_secrets_masker.py::TestMixedDataScenarios::test_mixed_structured_unstructured_data
PASSED [ 73%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_simple_strings[***-original_secret-password-original_secret]
PASSED [ 74%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_simple_strings[new_secret-original_secret-password-new_secret]
PASSED [ 75%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_simple_strings[***-original_value-normal_field-***]
PASSED [ 76%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_simple_strings[new_value-original_value-normal_field-new_value]
PASSED [ 77%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_simple_strings[***-original_value-None-***]
PASSED [ 78%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_simple_strings[new_value-original_value-None-new_value]
PASSED [ 79%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_dictionaries[old_data0-new_data0-expected0]
PASSED [ 80%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_dictionaries[old_data1-new_data1-expected1]
PASSED [ 81%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data0-new_data0-None-expected0]
PASSED [ 82%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data1-new_data1-None-expected1]
PASSED [ 83%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data2-new_data2-password-expected2]
PASSED [ 84%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data3-new_data3-normal_list-expected3]
PASSED [ 85%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data4-new_data4-None-expected4]
PASSED [ 86%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data5-new_data5-None-expected5]
PASSED [ 87%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data6-new_data6-password-expected6]
PASSED [ 88%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data7-new_data7-normal_tuple-expected7]
PASSED [ 89%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data8-new_data8-None-expected8]
PASSED [ 90%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data9-new_data9-None-expected9]
PASSED [ 91%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data10-new_data10-password-expected10]
PASSED [ 92%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_collections[old_data11-new_data11-normal_tuple-expected11]
PASSED [ 93%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_mismatched_types
PASSED [ 94%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_with_missing_keys
PASSED [ 95%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_complex_redacted_structures
PASSED [ 96%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_partially_redacted_structures
PASSED [ 97%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_max_depth
PASSED
[ 98%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_enum_values
PASSED [
99%]
tests/secrets_masker/test_secrets_masker.py::TestSecretsMaskerMerge::test_merge_round_trip
PASSED
[100%]
===========================================================================
short test summary info
============================================================================
XFAIL
tests/secrets_masker/test_secrets_masker.py::TestSecretsMasker::test_exc_tb -
Cannot filter secrets in traceback source
================================================================== 102
passed, 1 xfailed, 1 warning in 1.28s
===================================================================
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
