juan-pablo-guereca opened a new pull request, #77232:
URL: https://github.com/apache/airflow/pull/77232

   A secret that contains spaces or special characters was left unmasked when a 
formatter printed it differently from how it was registered, for example when 
the repr of a Kubernetes model is logged through `pprint`. `pprint` splits a 
string that does not fit the line width at its spaces into adjacent literals 
(`'abc '` / `'def'`), and `repr` and `pprint` print backslashes, mixed quote 
kinds and non-printable characters as escape sequences, so the registered text 
never appeared in the output. This extends the multi-line handling from #75651 
to those cases.
   
   Two commits:
   
   1. **Spaces:** `add_mask` registers one extra pattern per secret with 
spaces, built from its words joined by a separator that also matches the quote, 
line break and quote `pprint` puts between pieces, and the escaped forms of 
whitespace (`\t`, `\r`, `\xNN`, `\uNNNN`). Masking each word separately was 
rejected: it redacts every long word globally and leaks a short word left alone 
on its own literal.
   2. **Escapes:** `_adaptations` also yields the secret as `repr` prints it, 
and the words of the wrapped pattern are matched as `repr` prints them. The 
replacer alternation is ordered by `_get_pattern_length`, which counts the 
separator as the single escaped space it stands for, so a wrapped pattern does 
not sort ahead of the pattern of a longer secret that contains it.
   
   ### Existing tests changed
   
   * `test_calling_mask_secret_not_set`: `"a secret"` now registers its wrapped 
pattern as well.
   * `test_secret_with_trailing_newline_registers_stripped_value`: the `repr` 
form of `token-value\n` masks the escaped newline too, so the output is `'***'` 
instead of `'***\n'`. The value is still fully masked.
   
   ### Known limits
   
   Each is pinned by a strict `xfail` test and listed in 
`mask-sensitive-values.rst`:
   
   * A secret with leading or trailing whitespace that `pprint` splits.
   * A secret with both quote kinds in different words when `pprint` splits it, 
because `pprint` escapes quotes per piece.
   * The lines of a multi-line secret that contain special characters or are 
long enough to be split further.
   * A longer secret that continues a shorter one only modulo the kind of 
whitespace can lose to the wrapped pattern of the shorter one.
   
   ### Performance
   
   `shared/secrets_masker` runs for every log line, so I measured the base 
commit and this change alternately, 5 runs each, same environment, medians. The 
control (`json.dumps`, untouched by the change) moved 0%.
   
   | Case | Base | This PR |
   |---|---|---|
   | Register 300 tokens without spaces | 2.5 ms | 4.5 ms |
   | Register 300 phrases of 5 words | 5.1 ms | 57.9 ms |
   | Redact 5,000 log lines, 100 token secrets | 113 ms | 115 ms |
   | Redact 5,000 log lines, 50 token and 50 phrase secrets | 112 ms | 161 ms |
   
   Secrets without spaces cost a few microseconds more to register and nothing 
more to redact. A phrase adds a second, longer pattern to the alternation, 
which is where the slower redaction comes from, and a registration recompiles 
the combined regex once per secret. I think that is acceptable for correctness 
on values that were leaking, but I am happy to look at a cheaper structure if 
reviewers prefer.
   
   ### Testing
   
   * New tests cover the whitespace separators (every character `str.isspace()` 
accepts, through real `pprint`), `repr` escapes, wrapped secrets with 
backslashes, non-printable characters and mixed quote kinds, and the ordering 
fix; each fails on the previous source.
   * Also checked manually in an Airflow 3 dev environment with a patched image.
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (Claude Code, Sonnet 5.5)
   
   Generated-by: Claude Code (Sonnet 5.5) following [the 
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   
   ---
   
   * Read the **[Pull Request 
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
 for more information. Note: commit author/co-author name and email in commits 
become permanently public when merged.
   * For fundamental code changes, an Airflow Improvement Proposal 
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
 is needed.
   * When adding dependency, check compliance with the [ASF 3rd Party License 
Policy](https://www.apache.org/legal/resolved.html#category-x).
   * For significant user-facing changes create newsfragment: 
`{pr_number}.significant.rst`, in 
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
 You can add this file in a follow-up commit after the PR is created so you 
know the PR number.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to