I monitored swap activities for the query using vmstat. The *so* and *si* shows 0 till the completion of query. Also the top showed 0 against swap. This means there was no scarcity of physical memory. Swap activity seems not to be a bottleneck. Kindly note that this I ran on 8 node cluster with 30 gb RAM and 140 gb of index on each node.
Regards, Modassar On Mon, Nov 2, 2015 at 5:27 PM, Modassar Ather <modather1...@gmail.com> wrote: > Okay. I guess your observation of 400% for a single core is with top and > looking at that core's entry? If so, the 400% can be explained by > excessive garbage collection. You could turn GC-logging on to check > that. With a bit of luck GC would be the cause of the slow down. > > Yes it is with top command. I will check GC activities and try to relate > with CPU usage. > > The query q=network se* is quick enough in our system too. It takes around > 3-4 seconds for around 8 million records. > The problem is with the same query as phrase. q="network se*". > Can you please share your experience with such query where the wild card > expansion is huge like in the query above? > > I changed my SolrCloud setup from 12 shard to 8 shard and given each shard > 30 GB of RAM on the same machine with same index size (re-indexed) but > could not see the significant improvement for the query given. > > I will check the swap activity. > > Also can you please share your experiences with respect to RAM, GC, solr > cache setup etc as it seems by your comment that the SolrCloud environment > you have is kind of similar to the one I work on? > > Regards, > Modassar > > On Mon, Nov 2, 2015 at 5:20 PM, Toke Eskildsen <t...@statsbiblioteket.dk> > wrote: > >> On Mon, 2015-11-02 at 16:25 +0530, Modassar Ather wrote: >> > The remaining size after you removed the heap usage should be reserved >> for >> > the index (not only the other system activities). >> > I am not able to get the above point. So when I start Solr with 28g >> RAM, >> > for all the activities related to Solr it should not go beyond 28g. And >> the >> > remaining heap will be used for activities other than Solr. Please help >> me >> > understand. >> >> It is described here: >> https://wiki.apache.org/solr/SolrPerformanceProblems#OS_Disk_Cache >> >> I will be quick to add that I do not agree with Shawn (the primary >> author of the page) on the stated limits and find that the page in >> general ignores that performance requirements differ a great deal. >> Nevertheless, it is very true that Solr performance is tied to the >> amount of OS disk cache: >> >> You can have a machine with 10TB of RAM, but Solr performance will still >> be poor if you use it all for JVMs. >> >> Practically all modern operating system uses free memory for disk cache. >> Free memory is the memory not used for JVMs or other programs. It might >> be that you have a lot less than 30-40GB free: If you are on a Linux >> server, try calling 'top' and see what is says under 'cached'. >> >> Related, I support jim's suggestion to inspect the swap activity: >> In the past we had problem with a machine that insisted on swapping >> excessively, although there were high IO and free memory. >> >> > The disks are SSDs. >> >> That makes your observations stranger still. >> >> >> - Toke Eskildsen, State and University Library, Denmark >> >> >> >
