mrhhsg opened a new pull request, #69505:
URL: https://github.com/apache/doris/pull/69505
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Spill part files had no file-level format identifier. The
reader decoded footer block counts and offsets before confirming that a part
used a supported layout. Add an eight-byte trailer (little-endian version 1 and
`DSPL` magic) to every completed part, and validate it before interpreting
footer metadata. Unmarked or unsupported parts now return an error. Existing
block and footer fields remain unchanged, and the trailer uses the existing
append and accounting path for local and S3 spill.
### Release note
Spill part files include a versioned format marker. Reads reject unmarked or
unsupported part formats.
### Check List (For Author)
- Test: Unit Test
- `BUILD_TYPE=ASAN ./build.sh --be -j 32` (passed).
- `./run-be-ut.sh --run --filter='SpillFileTest.*:SpillFileS3Test.*' -j
32` (85/85 passed in a focused test build with file cache disabled).
- `build-support/clang-format.sh`, `build-support/check-format.sh`,
`build-support/check-build-hygiene.sh`, and changed-line `clang-tidy` (passed).
- Full BE unit suite and SQL regression were not completed.
- Behavior changed: Yes (unmarked or unsupported spill part formats are
rejected).
- Does this need documentation: No (internal temporary spill file format).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
