zhiqiang-hhhh opened a new pull request, #61313: URL: https://github.com/apache/doris/pull/61313
## Summary This PR fixes ANN index build OpenMP thread budgeting and adds a BE unit test for the concurrency cap. ## Problem `ScopedOmpThreadBudget` computed available budget from `config::omp_threads_limit` directly. With the default `omp_threads_limit = -1`, each builder effectively reserved 1 thread and did not follow the documented auto behavior (80% of CPU cores). In addition, concurrent builders could overrun the intended global limit because there was no wait/coordination when the budget was exhausted. ## Root Cause In `faiss_ann_index.cpp`: - The constructor used `config::omp_threads_limit` directly instead of the auto-resolved limit from `get_omp_threads_limit()`. - No blocking mechanism existed when all OpenMP budget was already in use. ## Fix - Use `get_omp_threads_limit()` as the actual global limit for budgeting. - Add a condition variable to block builders until at least one OpenMP slot is available. - Keep the existing policy of reserving up to half of remaining budget (minimum 1). - Add concise comments to explain wait/wakeup behavior. ## Test - Added `VectorSearchTest.OmpThreadBudgetNeverExceedsLimit` in `be/test/storage/index/ann/faiss_vector_index_test.cpp`. - The test sets `config::omp_threads_limit = 1`, runs multiple concurrent ANN `add()` builds, samples `ann_index_build_index_threads`, and asserts peak usage never exceeds 1 and finally returns to 0. ## Validation - Local compilation/test run was intentionally skipped per request. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
