The fieldCollapseCache should not be used as it is now, it uses too much memory. It stores any information relevant for a field collapse search. Like document collapse counts, collapsed document ids / fields, collapsed docset and uncollapsed docset (everything per unique search). So the memory usage will grow for each unique query (and fast with all this information). So its best I think to disable this cache for now.
Martijn On 8 June 2010 19:05, Jean-Sebastien Vachon <js.vac...@videotron.ca> wrote: > Hi All, > > I've been running some tests using 6 shards each one containing about 1 > millions documents. > Each shard is running in its own virtual machine with 7 GB of ram (5GB > allocated to the JVM). > After about 1100 unique queries the shards start to struggle and run out of > memory. I've reduced all > other caches without significant impact. > > When I remove completely the fieldCollapseCache, the server can keep up for > hours > and use only 2 GB of ram. (I'm even considering returning to a 32 bits JVM) > > The size of the fieldCollapseCache was set to 5000 items. How can 5000 items > eat 3 GB of ram? > > Can someone tell me what is put in this cache? Has anyone experienced this > kind of problem? > > I am running Solr 1.4.1 with patch 236. All requests are collapsing on a > single field (pint) and > collapse.maxdocs set to 200 000. > > Thanks for any hints... > >
