Found the reason for many evictions (bug in our code), please ignore the specific question on filter cache. All other questions (in bold) are still very relevant.
From: Esther Goldbraich/Haifa/IBM To: solr-user@lucene.apache.org Date: 27/08/2015 01:13 PM Subject: Re: Solr 5.2.1 versus Solr 4.7.0 performance :) Yes, 64 collections / 1 shard is compared to 1 collection / 64 shards (with router.name=compositeId) on Solr 5. Quering with "_route_" should not eliminate distributed search overhead? What is the difference in distribution mechanism between Solr 4 & Solr 5? Especially is there any change in filter cache management? We see very low hit ratio, with many evictions. Seems that specific shard holds entries from other shards (based on _route_ field). Why this can happen? Yes, we have facets, not in all queries. Using regular facets, haven't upgraded to Json facets yet. Appreciate your help, Esther From: Toke Eskildsen <t...@statsbiblioteket.dk> To: <solr-user@lucene.apache.org> Date: 27/08/2015 12:27 PM Subject: Re: Solr 5.2.1 versus Solr 4.7.0 performance On Thu, 2015-08-27 at 11:23 +0300, Esther Goldbraich wrote: > We are using GC tuning options: Xgcpolicy:gencon , verbose:gc. > RAM: 64GB > Solr heap: -Xms512m -Xmx32768m > Index per server: 500G Expecting "Your RAM size should be equal to index size"-posts to arrive in 3, 2, 1... > Surprisingly, running different setup on same machines, 64 collections / 1 > shard per collection gives significantly better results. > Any ideas? Compared to what? If your other scenario is to have 1 collection split over 64 shards, then the difference boils down to distributed search vs. single-shard search. There is a non-trivial overhead with doing distributed search, so if a collection fits well into a single shard (replicas does not count as distribution here), that is preferable. Come to think about it, distribution in itself might account for the difference between 4 & 5 that you are observing. Are you doing faceting as part of your test? - Toke Eskildsen, State and University Library, Denmark
