Baymine opened a new pull request, #66312:
URL: https://github.com/apache/doris/pull/66312
### What problem does this PR solve?
Issue Number: close #66311
Related PR: (none)
Problem Summary:
On the FE, `ProfileManager.loadProfilesFromStorageIfFirstTime(boolean sync)`
spawns a new `profile-loader` thread on every call with no in-flight guard, and
only sets `isProfileLoaded = true` inside the `try` block after the batch loop
completes. Because this method is reached from the periodic scheduler:
- if the load throws, or
- if the `profileIOExecutor` thread pool is saturated so the per-batch
`future.get()` calls never return,
then `isProfileLoaded` is never set, and a fresh `profile-loader` thread is
spawned on every scheduler tick — an unbounded thread leak that keeps growing
until the FE restarts.
This PR:
- adds an `AtomicBoolean isProfileLoading` and guards the load entry with
`compareAndSet(false, true)`, so only one loader runs at a time; a synchronous
caller that loses the CAS blocks on `waitForProfileLoadFinish()` until the
in-flight load finishes;
- extracts the batch-read body into `loadProfilesFromStorage()` and wraps it
in `try/finally`, where the `finally` calls `markProfileLoaded()` and clears
`isProfileLoading` **even on failure**, so a saturated/failed load does not
respawn a loader every second (a WARN documents that the load will not be
retried until the FE restarts);
- makes the async loader thread a daemon, and runs the load inline when
`sync == true`.
No behavior change on the success path; the set of loaded profiles is
identical.
### Release note
None
### Check List (For Author)
- Test <PASS>
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test.
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
