: We will take this approach in our production environment but meanwhile I am : curious if this issue will be addressed: it seems the new/first searchers do : not really buy any performance benefits because it uses so much memory, : especially at core loading time.
There's nothing inheriently wrong with using newSearcher/firstSearcher -- for many people they do in fact provide a perf improvement for "real" users (at the cost of some initial time spent warming before those users ever get access to the searcher) As i udnerstand it from this thread, your issue is not actually the firstSearcher/newSearcher -- your issue (per yonik's ocmments) is that with per Segment sorting in 1.4, the FieldCache for some of your fields requires a lot more ram in 1.4 then it would have been for Solr 1.3 -- which caused GC thrashing during initialization. Even w/o using firstSearcher/newSearcher, all that RAM is still going to be used if/when you do sort on those fields -- all removing the firstSearcher/newSearcher queries on those fields has done for you is delay when the time spent initializing those FieldCaches happens and when that RAM first starts getting used. It's posisbly you never actual sort on those fields, in which case removing those warming queries completely is definitely the way to go -- but if you do sort on them then the warming queries can still legitimately be helpful (in thta they pay the cost up front before a real user issues queries) As yonik mentioned the real "fix" for the amount of memory being used is to switch to the TrieDateFields which use much more efficient FieldCache's for sorting -- with that change you can probably start using the warming queries again. (Depending on how you tested, you may not have noticed much advantage to having them because you'll really only see the advantages on the initial queries that do sorting -- those should show huge outlier times w/o the warming queries, but once those poor unlucky users have paid the price for initializing hte FieldCache, every one elses sorts should be fast) -Hoss
