adasari opened a new pull request, #19167:
URL: https://github.com/apache/pinot/pull/19167
Problems:
1. The `BinaryWorkloadScheduler` dispatch thread polls
SecondaryWorkloadQueue every 1 ms (default) regardless of whether any secondary
queries exist. Secondary queries are not frequent, So on a server with no
secondary queries, there are ~1000 needless wakeups per min, per server, on a
Thread.MAX_PRIORITY daemon thread.
2. Also, SecondaryWorkloadQueue removing expired queries without ever
completing its result future. so the server never sent a response for it and
leaked the entry tracking it.
Changes
1. Problem#1:
* Untimed wait when the queue is empty. put() already signals
_queryReaderCondition under _queueLock, so nothing else can make the queue
non-empty without waking the reader. This eliminates 100% of idle wakeups.
* Signal when reserved threads are released.
* Timed wait retained when the queue is non-empty. The timed branch keeps
that sweep running and now only ticks in the rare state where secondary queries
are queued and the group is at its thread limit.
2. Problem#2:
* SchedulerGroup.trimExpired now returns the removed queries so the
caller can complete them.
* SecondaryWorkloadQueue takes a Consumer<SchedulerQueryContext> expiry
handler and invokes it for each.
Testing:
Unit tests `SecondaryWorkloadQueueTest`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
