[
https://issues.apache.org/jira/browse/SOLR-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220136#comment-15220136
]
Jeff Wartes commented on SOLR-8922:
-----------------------------------
Ok, so after a little more than 12 hours on one of my production nodes, there
was no noticeable change in CPU usage.
Running before/after GC logs through GCViewer, it's a little hard to compare
rate, since the logs were for different intervals and the "after" log included
startup. That said, "Freed mem/minute" was down by 44%, and "Throughput" went
from 87% to 93%. I also see noticeably reduced average pause time, and
increased average pause interval. All positive signs.
The only irritation I'm finding here is that it looks like the CMS collector is
running more often. I expect that's simply because I changed the footing of a
fairly tuned set of GC parameters though.
> DocSetCollector can allocate massive garbage on large indexes
> -------------------------------------------------------------
>
> Key: SOLR-8922
> URL: https://issues.apache.org/jira/browse/SOLR-8922
> Project: Solr
> Issue Type: Improvement
> Reporter: Jeff Wartes
> Attachments: SOLR-8922.patch
>
>
> After reaching a point of diminishing returns tuning the GC collector, I
> decided to take a look at where the garbage was coming from. To my surprise,
> it turned out that for my index and query set, almost 60% of the garbage was
> coming from this single line:
> https://github.com/apache/lucene-solr/blob/94c04237cce44cac1e40e1b8b6ee6a6addc001a5/solr/core/src/java/org/apache/solr/search/DocSetCollector.java#L49
> This is due to the simple fact that I have 86M documents in my shards.
> Allocating a scratch array big enough to track a result set 1/64th of my
> index (1.3M) is also almost certainly excessive, considering my 99.9th
> percentile hit count is less than 56k.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
