Erik, I understand that caching is going to improve performance. Infact we did a PSR run with caches enabled and we got awesome results. But these wouldn't be really representative because the PSR scripts will be doing the same searches again and again. These would be cached and there would be virtually no evictions. This is not a practical case.
My hardware (in the PSR environment where I am testing) is pretty good - 12 CPU, 24 G RAM, Ultrasparc III 1.2 GHz processors, Solaris 10. We have allocated 3.2 GB RAM for Weblogic (JVM). This is the maximum that I am able to allocate for one JVM. I think I need to go back and check if I am not using all the fields in the query. I understand that setting indexed=false alone will not ensure that all fields don't participate in the query. Thanks a lot for your response. Regards Rahul On Fri, Jul 31, 2009 at 3:33 PM, Erik Hatcher <e...@ehatchersolutions.com>wrote: > > On Jul 31, 2009, at 2:35 AM, Rahul R wrote: > > Hello, >> We are trying to get Solr to work for a really huge parts database. >> Details >> of the database >> - 55 million parts >> - Totally 3700 properties (facets). But each record will not have value >> for >> all properties. >> - Most of these facets are defined as dynamic fields within the Solr Index >> >> We were getting really unacceptable timing while doing faceting/searches >> on >> an index created with this database. >> > > Were you accounting for cache warming? Were your caches sized > appropriately? What kind of hardware and RAM were you using? What were the > JVM settings? > > And certainly not least important - what version of Solr are you running? > The difference in faceting performance and scalability between Solr 1.3 and > what will be Solr 1.4 is quite dramatic. > > We thought that by limiting the number of properties that are available for >> faceting, the performance can be improved. To test this, we enabled only 6 >> properties for faceting by setting indexed=true (in schema.xml) for only >> these properties. All other properties which are defined as dynamic >> properties had indexed=false. >> > > These settings won't matter - what matters in this case is what facets you > request, not what is actually in the index. > > > My questions: >> - Will reducing the number of facets improve faceting and search >> performance ? >> > > Reducing what fields you request will, of course. But what you actually > index has no effect on performance until you request it. > > - Is there a better way to reduce the number of facets ? >> > > Hard to say without doing a deeper analysis of your needs. > > - Will having a large number of properties defined as dynamic fields, >> reduce >> performance ? >> > > Dynamic fields versus statically named fields have no effect on > performance. > > Erik > >
