Z-SWEI opened a new pull request, #67604:
URL: https://github.com/apache/doris/pull/67604
### What problem does this PR solve?
Problem Summary:
For queries whose results are pumped through `MemoryScratchSink` into
`ResultQueueMgr` (the external scan context path:
`BaseBackendService::get_next` -> `ResultQueueMgr::fetch_result`), when the
fragment instance fails -- e.g. query timeout or memory limit exceeded --
`MemoryScratchSinkLocalState::close()` only puts the `nullptr` eos sentinel
into the record batch queue and never propagates the failure status to the
`ResultQueueMgr`.
A fetcher that already passed the queue status check and is blocked in
`blocking_get()` wakes up on the sentinel, treats it as a normal end-of-stream,
and returns `eos` to the caller. The client stops polling once it sees eos, so
the query silently returns **truncated results** instead of the real error
(`TIMEOUT`, `MEM_LIMIT_EXCEEDED`, ...).
This patch:
1. Publishes the fragment failure status to the result queue before putting
the eos sentinel in `MemoryScratchSinkLocalState::close()`. It must be done
before the sentinel: a fetcher blocked in `blocking_get` wakes up on the
sentinel and re-checks the queue status.
2. Re-checks the queue status when `fetch_result` observes the sentinel, so
the failure is surfaced on this last fetch instead of a silent truncated eos.
3. Adds a unit test (`fetch_result_failure_before_eos_sentinel`) covering
the race: a fetch thread blocked in `blocking_get` while the sink thread
publishes the failure status and then puts the sentinel.
### Release note
Fix silently truncated query results when a fragment served by
`MemoryScratchSink` fails -- the query now returns the real failure status
(e.g. TIMEOUT, MEM_LIMIT_EXCEEDED) instead of an apparent normal end-of-stream.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [ ] No.
- [x] Yes. A failed fragment served by `MemoryScratchSink` now surfaces
its real failure status to the caller instead of a silent truncated eos.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
