ddreyer opened a new pull request, #19555: URL: https://github.com/apache/pinot/pull/19555
## Problem `AuditRequestProcessor.captureRequestPayload` serializes the request entity, the query parameters and the allow-listed headers straight into the audit event. There is no redaction anywhere in `org.apache.pinot.common.audit`, so **no configuration can make payload capture safe**: any credential a caller sends to an audited endpoint is written verbatim to stdout and to the rotating audit file, then shipped wherever cluster logs go. Observed on a controller with `capture.request.payload.enabled` set: every `POST /connections/browse` line carried the full body, exposing AWS access-key/secret-key pairs in plaintext next to the requesting principal. It is not specific to that endpoint — table creates carrying Kafka SASL passwords, warehouse passwords and tier-storage keys leak the same way. Anyone with log or node access can read live credentials, including log-pipeline operators who have no access to the API itself. Capture is off by default (`capture.request.payload.enabled`, `AuditConfig`), but it is a supported feature that operators do turn on. ## Change Adds `AuditRedactor`, through which everything captured from the caller now passes: - **Credential-bearing keys are masked**, reusing `Obfuscator` with a pattern list extended past its suffix-anchored defaults. Those defaults miss the AWS `accessKey` / `accessKeyId` pair, which is exactly what leaked. - **A body that is not parseable JSON is recorded by size only** (`[redacted: unparseable payload, N bytes]`). Secrets can only be located inside a structure that is understood, so an opaque payload is not kept at all. This includes bodies truncated at `request.payload.size.max.bytes`, whose trailing fragment is no longer valid JSON. - **Redaction is unconditional** and has no configuration key, because every setting that disables it is a credential leak. Key matching deliberately errs towards over-redaction: masking a harmless `tokenUri` costs an audit reader nothing, while missing one `accessKey` is the bug this prevents. Query parameters and headers are redacted too — note that query parameters are captured regardless of `capture.request.payload.enabled`. ## Behavior change Two existing tests asserted that an allow-listed `Authorization` / `X-Password` header was captured verbatim, while their own comments said it "should be filtered out". They now assert the mask. Allow-listing a credential-bearing header now records *that it was sent*, not what it said. ## Tests - New `AuditRedactorTest`: data-driven coverage of sensitive vs. benign keys, nested objects, secrets inside arrays, top-level arrays, unparseable and truncated bodies, byte-vs-character sizing, and non-mutation of the input map. - New cases in `AuditRequestProcessorTest` covering the end-to-end capture path, including that redaction does not disturb the entity stream handed to the resource method. ## Notes This builds on the array-traversal fix already in `Obfuscator` on master — secrets nested inside JSON arrays (e.g. Kafka credentials in `streamConfigMaps`, a list of maps) depend on it. The test suite was run green on an earlier base; it has not been re-run since rebasing onto current master. Marked draft pending that CI run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
