[
https://issues.apache.org/jira/browse/SOLR-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17376641#comment-17376641
]
Mark Robert Miller commented on SOLR-15512:
-------------------------------------------
I have something that should be ready for a PR here.
Idea is to reuse a single executor, default sized at the number of detected
cores, across auxiliary search features that use threads (initially,
multi-threaded faceting feature and here).
We use a more efficient executor design for this shared thread pool.
* LinkedTransferQueue is much more efficient than SynchronousQueue as can be
found in online performance results as well as practical results (I've pushed
through almost the entire Solr code base before and compared).
* There are some issues with using an unbounded queue with Java executors -
namely that they don't fire up threads when the queue is willing to accept the
task first. Not good for the unbounded transfer queue and kind of odd in many
cases if you partially fill the queue.
{noformat}
return new LinkedTransferQueue<>() {
@Override public boolean offer(Runnable r) {
return tryTransfer(r);
}
};{noformat}
Since offer is what determines if we start a thread, we return false unless the
task can be directly transferred to a waiting thread.
Then we add an rejection policy that will add to the queue on reject.
{noformat}
setRejectedExecutionHandler(new AbortPolicy() {
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
try {
getQueue().put(r);
} catch (InterruptedException e1) {
// reinterrupt
}
}
});{noformat}
Resulting in a more efficient pool with generally better behavior in terms of
thread count and queuing behavior.
Regardless of the query load, making this a shared pool will manage the thread
pool of a size around what can actually be executed on while queuing up
additional work instead of allowing for an unknown, unbounded barrage of
threads under load.
> Add support for passing IndexSearcher an executor for multi-threaded
> multi-segment and possible future multi-threaded per segment search.
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-15512
> URL: https://issues.apache.org/jira/browse/SOLR-15512
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Mark Robert Miller
> Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
