OIiveirra opened a new pull request, #69478:
URL: https://github.com/apache/doris/pull/69478
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary: When a Hudi JNI scan uses explicit S3A credentials,
enabling scanner DEBUG logging prints every parameter value, including access
keys, secret keys and session tokens. Log parameter names without values and
record the query ID separately so scans remain traceable. The Hadoop
configuration used to read data is unchanged.
Add a regression test that captures the real scanner's DEBUG output, checks
that parameter names and the query ID remain visible, rejects ordinary and
bucket-specific credentials and custom tokens, and verifies that Hadoop
properties retain their original values.
### Release note
Hudi JNI scanner debug logs no longer include storage credential values and
continue to include query IDs for diagnostics.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test.
Validation with JDK 17:
- `./run-fe-ut.sh --run
'org.apache.doris.hudi.HadoopHudiJniScannerLogTest,org.apache.doris.hudi.HadoopHudiPluginTest,org.apache.doris.hudi.HadoopHudiColumnValueTest'`:
16 tests passed.
- The new log test fails on the original code because parameter values
appear in DEBUG output and passes after the fix.
- The existing plugin test writes and reads a local Hudi table through the
Java scanner.
- Maven `validate` / Checkstyle for
`be-java-extensions/hadoop-hudi-scanner` and its dependencies, plus `git diff
--check`: passed.
- The logging regression is covered directly by JUnit. FE/BE + S3
end-to-end regression was not run locally; SQL planning, credential propagation
and data-reading logic are unchanged.
- Behavior changed:
- [ ] No.
- [x] Yes. DEBUG logs retain parameter names and query IDs without
printing other parameter values.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
