How often does the index change? Can you use an HTTP cache and do this once for each new index?
wunder On 1/31/08 9:09 AM, "Andy Blower" <[EMAIL PROTECTED]> wrote: > > Actually I do need all facets for a field, although I've just realised that > the tests are limited to only 100. Ooops. So it should be worse in > reality... erk. > > Since that's what we do with our current search engine, Solr has to be able > to compete with this. The fields are a mix of non-multi, non-tokenized and > others which are. I've yet to experiment with this. > > Thanks. > > > shalinmangar wrote: >> >> I can't give you a definitive answer based on the data you've provided. >> However, do you really need to get *all* facets? Can't you limit them with >> facet.limit field? Are you planning to run multiple *:* queries with all >> facets turned on a 58GB index in a live system? I don't think that's a >> good >> idea. >> >> As for the 125 seconds, I think it is probably because of paging issues. >> Are >> you faceting on multivalued or tokenized fields? In that case, Solr uses >> field queries which consume a lot of memory if the number of unique terms >> are large. >> >> On Jan 31, 2008 9:13 PM, Andy Blower <[EMAIL PROTECTED]> wrote: >> >>> >>> I'm evaluating SOLR/Lucene for our needs and currently looking at >>> performance >>> since 99% of the functionality we're looking for is provided. The index >>> contains 18.4 Million records and is 58Gb in size. Most queries are >>> acceptably quick, once the filters are cached. The filters select one or >>> more of three subsets of the data and then intersect from around 15 other >>> subsets of data depending on a user subscription. >>> >>> We're returning facets on several fields, and sometimes a blank (q=*:*) >>> query is run purely to get the facets for all of the data that the user >>> can >>> access. This information is turned into browse information and can be >>> different for each user. >>> >>> Running performance tests using jMeter sequentially with a single user, >>> these blank queries are slower than the normal queries, but still in the >>> 1-4sec range. Unfortunately if I increase the number of test threads so >>> that >>> more than one of the blank queries is submitted while one is already >>> being >>> processed, everything grinds to a halt and the responses to these blank >>> queries can take up to 125secs to be returned! >>> >>> This surprises me because the filter query submitted has usually already >>> been submitted along with a normal query, and so should be cached in the >>> filter cache. Surely all solr needs to do is return a handful of fields >>> for >>> the first 100 records in the list from the cache - or so I thought. >>> >>> Can anyone tell me what might be causing this dramatic slowdown? Any >>> suggestions for solutions would be gratefully received. >>> >>> >>> Thans >>> Andy. >>> -- >>> View this message in context: >>> http://www.nabble.com/Slow-response-times-using-*%3A*-tp15206563p15206563.ht >>> ml >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >>
