Anyone?
On Mon, Mar 17, 2014 at 12:03 PM, Salman Akram < salman.ak...@northbaysolutions.net> wrote: > Below is one of the sample slow query that takes mins! > > ((stock or share*) w/10 (sale or sell* or sold or bought or buy* or > purchase* or repurchase*)) w/10 (executive or director) > > If a filter is used it comes in fq but what can be done about plain > keyword search? > > > On Sun, Mar 16, 2014 at 4:37 AM, Erick Erickson > <erickerick...@gmail.com>wrote: > >> What are our complex queries? You >> say that your app will very rarely see the >> same query thus you aren't using caches... >> But, if you can move some of your >> clauses to fq clauses, then the filterCache >> might well be used to good effect. >> >> >> >> On Thu, Mar 13, 2014 at 7:22 AM, Salman Akram >> <salman.ak...@northbaysolutions.net> wrote: >> > 1- SOLR 4.6 >> > 2- We do but right now I am talking about plain keyword queries just >> sorted >> > by date. Once this is better will start looking into caches which we >> > already changed a little. >> > 3- As I said the contents are not stored in this index. Some other >> metadata >> > fields are but with normal queries its super fast so I guess even if I >> > change there it will be a minor difference. We have SSD and quite fast >> too. >> > 4- That's something we need to do but even in low workload those queries >> > take a lot of time >> > 5- Every 10 mins and currently no auto warming as user queries are >> rarely >> > same and also once its fully warmed those queries are still slow. >> > 6- Nops. >> > >> > On Thu, Mar 13, 2014 at 5:38 PM, Dmitry Kan <solrexp...@gmail.com> >> wrote: >> > >> >> 1. What is your solr version? In 4.x family the proximity searches have >> >> been optimized among other query types. >> >> 2. Do you use the filter queries? What is the situation with the cache >> >> utilization ratios? Optimize (= i.e. bump up the respective cache >> sizes) if >> >> you have low hitratios and many evictions. >> >> 3. Can you avoid storing some fields and only index them? When the >> field is >> >> stored and it is retrieved in the result, there are couple of disk >> seeks >> >> per field=> search slows down. Consider SSD disks. >> >> 4. Do you monitor your system in terms of RAM / cache stats / GC? Do >> you >> >> observe STW GC pauses? >> >> 5. How often do you commit & do you have the autowarming / external >> warming >> >> configured? >> >> 6. If you use faceting, consider storing DocValues for facet fields. >> >> >> >> some solr wiki docs: >> >> >> >> >> https://wiki.apache.org/solr/SolrPerformanceProblems?highlight=%28%28SolrPerformanceFactors%29%29 >> >> >> >> >> >> >> >> >> >> >> >> On Thu, Mar 13, 2014 at 8:52 AM, Salman Akram < >> >> salman.ak...@northbaysolutions.net> wrote: >> >> >> >> > Well some of the searches take minutes. >> >> > >> >> > Below are some stats about this particular index that I am talking >> about: >> >> > >> >> > Index size = 400GB (Using CommonGrams so without that the index is >> around >> >> > 180GB) >> >> > Position File = 280GB >> >> > Total Docs = 170 million (just indexed for searching - for >> highlighting >> >> > contents are stored in another index) >> >> > Avg Doc Size = Few hundred KBs >> >> > RAM = 384GB (it has other indexes too but still OS cache can have >> 60-80% >> >> of >> >> > the total index cached) >> >> > >> >> > Phrase queries run pretty fast with CG but complex versions of >> wildcard >> >> and >> >> > proximity queries can be really slow. I know using CG will make them >> slow >> >> > but they just take too long. By default sorting is on date but users >> have >> >> > few other parameters too on which they can sort. >> >> > >> >> > I wanted to avoid creating multiple indexes (maybe based on years) >> but >> >> > seems that to search on partial data that's the only feasible way. >> >> > >> >> > >> >> > >> >> > >> >> > On Wed, Mar 12, 2014 at 2:47 PM, Dmitry Kan <solrexp...@gmail.com> >> >> wrote: >> >> > >> >> > > As Hoss pointed out above, different projects have different >> >> > requirements. >> >> > > Some want to sort by date of ingestion reverse, which means that >> having >> >> > > posting lists organized in a reverse order with the early >> termination >> >> is >> >> > > the way to go (no such feature in Solr directly). Some other >> projects >> >> > want >> >> > > to collect all docs matching a query, and then sort by rank, but >> you >> >> > cannot >> >> > > guarantee, that the most recently inserted document is the most >> >> relevant >> >> > in >> >> > > terms of your ranking. >> >> > > >> >> > > >> >> > > Do your current searches take too long? >> >> > > >> >> > > >> >> > > On Tue, Mar 11, 2014 at 11:51 AM, Salman Akram < >> >> > > salman.ak...@northbaysolutions.net> wrote: >> >> > > >> >> > > > Its a long video and I will definitely go through it but it seems >> >> this >> >> > is >> >> > > > not possible with SOLR as it is? >> >> > > > >> >> > > > I just thought it would be quite a common issue; I mean >> generally for >> >> > > > search engines its more important to show the first page results, >> >> > rather >> >> > > > than using timeAllowed which might not even return a single >> result. >> >> > > > >> >> > > > Thanks! >> >> > > > >> >> > > > >> >> > > > -- >> >> > > > Regards, >> >> > > > >> >> > > > Salman Akram >> >> > > > >> >> > > >> >> > > >> >> > > >> >> > > -- >> >> > > Dmitry >> >> > > Blog: http://dmitrykan.blogspot.com >> >> > > Twitter: http://twitter.com/dmitrykan >> >> > > >> >> > >> >> > >> >> > >> >> > -- >> >> > Regards, >> >> > >> >> > Salman Akram >> >> > >> >> >> >> >> >> >> >> -- >> >> Dmitry >> >> Blog: http://dmitrykan.blogspot.com >> >> Twitter: http://twitter.com/dmitrykan >> >> >> > >> > >> > >> > -- >> > Regards, >> > >> > Salman Akram >> > > > > -- > Regards, > > Salman Akram > > -- Regards, Salman Akram
