Ah, multi-threaded highlighting. I implemented that once as a precursor to
ultimately other better things -- the UnifiedHighlighter.
Your ExecutorService ought to be a field on the handler. In inform() you
can call SolrCore.addCloseHook to ensure this executor is shut down.
I suggest looking at this presentation from a few years ago I did with
Bloomberg at Lucene/Solr Revolution:
https://www.youtube.com/watch?v=tv5qKDKW8kk&t=14s
The UnifiedHighlighter is not enabled by default. See the documentation:
https://builds.apache.org/job/Solr-reference-guide-master/javadoc/highlighting.html
Still... there is perhaps some value in multi-threading the highlighting
for huge docs, but I think we ultimately found no need after re-engineering
the highlighter.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Wed, Aug 28, 2019 at 10:36 AM SOLR4189 wrote:
> Hi all.
>
> In our team we thought about some tricky solution for queries with long
> time
> highlighting. For example, highlighting that takes more than 25 seconds.
> So,
> we created our component that wraps highlighting component of SOLR in this
> way:
>
> public void inform(SolrCore core) {
> . . . .
> subSearchComponent = core.getSearchComponent("highlight");
> . . . .
> }
>
> public void process(ResponseBuilder rb) throws Exception {
> long timeout = 25000;
> ExecutorService exec = null:
> try {
> exec = Executors.newSingleThreadExecutor();
> Future future = exec.submit(() -> {
> try {
> subSearchComponent.process(rb);
> } catch (IOException e) {
> return e;
> }
> return null;
> });
> Exception ex = future.get(timeout, TimeUnit.MILLISECONDS);
> if (ex != null) {
> throw ex;
> }
> } catch ( TimeoutException toe) {
> . . . .
> } catch (Exception e) {
>throw new IOException(e);
> } finally {
> if (exec != null) {
> exec.shutdownNow();
> }
> }
> }
>
> This solution works, but sometime we see that searchers stay open and as a
> result our RAM usage is pretty high (like a memory leak of
> SolrIndexSearcher
> objects). And only after a SOLR service restart they disappear.
>
> What do you think about this solution?
> Maybe exists some built-in function for it?
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>