Searching and sorting is now done on a per-segment basis, meaning that the FieldCache entries used for sorting and for function queries are created and used per-segment and can be reused for segments that don't change between index updates. While generally beneficial, this can lead to increased memory usage over 1.3 in certain scenarios: 1) A single valued field that was used for both sorting and faceting in 1.3 would have used the same top level FieldCache entry. In 1.4, sorting will use entries at the segment level while faceting will still use entries at the top reader level, leading to increased memory usage. 2) Certain function queries such as ord() and rord() require a top level FieldCache instance and can thus lead to increased memory usage. Consider replacing ord() and rord() with alternatives, such as function queries based on ms() for date boosting.
http://svn.apache.org/repos/asf/lucene/solr/branches/branch-1.4/CHANGES.txt On Wednesday 09 February 2011 16:07:01 Rachita Choudhary wrote: > Hi Solr Users, > > We are in the process of upgrading from Solr 1.3 to Solr 1.4.1. > While performing stress test on Solr 1.4.1 to measure the performance > improvement in Query times (QTime) and no more blocked threads, we ran into > memory issues with Solr 1.4.1. > > Test Setup details: > - 2 identical hosts running Solr 1.3 and Solr 1.4.1 individually. > - 3 cores with index sizes : 10 GB, 2 GB, 1 GB. > - JVM Max RAM : 3GB ( Xmx3072m) , Total RAM : 4GB > - No other application/service running on the servers. > - For querying solr servers, we are using wget queries from a standalone > host. > > For the same index data and same set of queries, Solr 1.3 is hovering > between 1.5 to 2.2 GB, whereas with about 20K requests Solr 1.4.1 is > reaching its 3 GB limit and performing FULL GC after almost every query. > The Full GC is also not freeing up any memory. > > Has anyone also faced similar issues with Solr 1.4.1 ? > > Also why is Solr 1.4.1 using more memory for the same amount of processing > compared to Solr 1.3 ? > > Is there any particular configuration that needs to be done to avoid this > high memory usage ? > > Thanks, > Rachita -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350
