zy-kkk opened a new pull request, #69232:
URL: https://github.com/apache/doris/pull/69232
### What problem does this PR solve?
Issue Number: N/A
Related PR: #68862
Problem Summary:
After #68862, FE production and test sources fail to compile:
- `S3EventSourceOffsetProvider` still imports `S3EventNotification` and
`Jackson` from AWS SDK v1 after the `aws-java-sdk-s3` dependency was removed.
- `DorisHttpTestCase` still imports `ServletWebServerApplicationContext`
from its old package after the Spring Boot 4 upgrade.
Replace the AWS SDK v1 notification model with private DTOs parsed by the
existing Jackson dependency. Model only the fields consumed by Doris, and
retain the existing Joda time parser, scalar coercion, single-pass UTF-8 URL
decoding, trailing-JSON rejection and SQS acknowledgement boundaries. This
removes the remaining v1 imports without restoring the removed dependency or
adding a notification library.
Unknown and unused notification metadata is ignored. A malformed unused
field such as `userIdentity` no longer discards a message whose consumed fields
are valid; malformed consumed fields retain their existing handling.
Update the HTTP test import to
`org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext`
for Spring Boot 4.
### Release note
Fix FE production and test compilation failures after the AWS SDK dependency
removal and Spring Boot 4 upgrade.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- Passed 200 tests across 17 S3 offset and streaming test classes,
including all 44 tests in `S3EventSourceOffsetProviderTest`.
- Added three tests and extended existing cases for notification
metadata, comments, large object sizes, scalar conversion, encoded keys,
optional timestamps and malformed consumed fields.
- Command: `FE_UT_PARALLEL=1 bash run-fe-ut.sh --run
'org.apache.doris.job.offset.s3.*Test,org.apache.doris.job.extensions.insert.streaming.*Test'`.
- [x] Manual test (add detailed scripts or steps below)
- Compared parsing behavior with the previous AWS SDK v1
implementation.
- Passed `FE_MAVEN_THREADS=1 bash build.sh --fe`, including the UI
and FE packaging.
- FE Core Checkstyle passed for 6,195 files. Verified that the
packaged FE JAR contains the notification DTOs and the existing runtime
dependencies.
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [x] Yes. Restores FE compilation and ignores malformed metadata fields
that Doris does not consume.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
