[
https://issues.apache.org/jira/browse/SOLR-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated SOLR-3585:
-----------------------------------
Attachment: SOLR-3585.patch
Folks, why are you so skeptic? Don't you need utilize your boxes to get
indexing happen faster?
The also potential usage which I see is returning multithreading to DIH back by
keeping the old core single threaded, then putting this multithread consumer
behind and pushing into from improved datasources which will have buffer and
separate threads.
ok. here is java doc from the recent patch. most update is correct and
comprehensive halting implementation.
/**
* Forks incoming adds into multiple threads.
* Instead of regular chaining it chains on it's own with chain specified in
* "backing.chain" update parameter.
* supports two solrconfig.xml parameters <ul>
* <li>
* "bufferSize" length of the buffer for add commands
* </li>
* <li>"pipesNumber" number of parallel threads for handling updates</li>
* </ul>
* it spawns own cached thread pool, and shutdown it on core closing.
* AddUpdate commands are queued for background execution by spawned processors.
* All other commands awaits until all queued tasks are completed, and then are
* handled in the calling thread (request).
* if one of the update processors threads (aka pipes) catches exception, all
other pipes
* are stopped synchronously, but not in the same moment, and the root cause is
propagated
* into request thread.
* for queuing spin-delay pattern is used to prevent dead lock when buffer is
full, and
* queuing thread is blocked, but buffer can't be processed due to dead pipes.
* */
> processing updates in multiple threads
> --------------------------------------
>
> Key: SOLR-3585
> URL: https://issues.apache.org/jira/browse/SOLR-3585
> Project: Solr
> Issue Type: Improvement
> Components: update
> Affects Versions: 4.0
> Reporter: Mikhail Khludnev
> Priority: Minor
> Attachments: SOLR-3585.patch, multithreadupd.patch
>
>
> Hello,
> I'd like to contribute update processor which forks many threads which
> concurrently process the stream of commands. It may be beneficial for users
> who streams many docs through single request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
