How old is 'older'? I'm pretty sure I'm still getting much faster performance on an optimized index in Solr 1.4.
This could be due to the nature of my index and queries (which include some medium sized stored fields, and extensive facetting -- facetting on up to a dozen fields in every request, where each field can include millions of unique values. Amazing I can do this with good performance at all!). It's also possible i'm wrong about that faster performance, i haven't done robustly valid benchmarking on a clone of my production index yet. But it really looks like that way to me, from what investigation I have done. If the answer is that optimization is believed no longer neccesary on versions LATER than 1.4, that might be the simplest explanation. ________________________________________ From: Pierre GOSSE [pierre.go...@arisem.com] Sent: Friday, July 22, 2011 10:23 AM To: solr-user@lucene.apache.org Subject: RE: commit time and lock Hi Mark I've read that in a thread title " Weird optimize performance degradation", where Erick Erickson states that "Older versions of Lucene would search faster on an optimized index, but this is no longer necessary.", and more recently in a thread you initiated a month ago "Question about optimization". I'll also be very interested if anyone had a more precise idea/datas of benefits and tradeoff of optimize vs merge ... Pierre -----Message d'origine----- De : Marc SCHNEIDER [mailto:marc.schneide...@gmail.com] Envoyé : vendredi 22 juillet 2011 15:45 À : solr-user@lucene.apache.org Objet : Re: commit time and lock Hello, Pierre, can you tell us where you read that? "I've read here that optimization is not always a requirement to have an efficient index, due to some low level changes in lucene 3.xx" Marc. On Fri, Jul 22, 2011 at 2:10 PM, Pierre GOSSE <pierre.go...@arisem.com>wrote: > Solr will response for search during optimization, but commits will have to > wait the end of the optimization process. > > During optimization a new index is generated on disk by merging every > single file of the current index into one big file, so you're server will be > busy, especially regarding disk access. This may alter your response time > and has very negative effect on the replication of index if you have a > master/slave architecture. > > I've read here that optimization is not always a requirement to have an > efficient index, due to some low level changes in lucene 3.xx, so maybe you > don't really need optimization. What version of solr are you using ? Maybe > someone can point toward a relevant link about optimization other than solr > wiki > http://wiki.apache.org/solr/SolrPerformanceFactors#Optimization_Considerations > > Pierre > > > -----Message d'origine----- > De : Jonty Rhods [mailto:jonty.rh...@gmail.com] > Envoyé : vendredi 22 juillet 2011 12:45 > À : solr-user@lucene.apache.org > Objet : Re: commit time and lock > > Thanks for clarity. > > One more thing I want to know about optimization. > > Right now I am planning to optimize the server in 24 hour. Optimization is > also time taking ( last time took around 13 minutes), so I want to know > that > : > > 1. when optimization is under process that time will solr server response > or > not? > 2. if server will not response then how to do optimization of server fast > or > other way to do optimization so our user will not have to wait to finished > optimization process. > > regards > Jonty > > > > On Fri, Jul 22, 2011 at 2:44 PM, Pierre GOSSE <pierre.go...@arisem.com > >wrote: > > > Solr still respond to search queries during commit, only new indexations > > requests will have to wait (until end of commit?). So I don't think your > > users will experience increased response time during commits (unless your > > server is much undersized). > > > > Pierre > > > > -----Message d'origine----- > > De : Jonty Rhods [mailto:jonty.rh...@gmail.com] > > Envoyé : jeudi 21 juillet 2011 20:27 > > À : solr-user@lucene.apache.org > > Objet : Re: commit time and lock > > > > Actually i m worried about the response time. i k commiting around 500 > > docs in every 5 minutes. as i know,correct me if i m wrong; at the > > time of commiting solr server stop responding. my concern is how to > > minimize the response time so user not need to wait. or any other > > logic will require for my case. please suggest. > > > > regards > > jonty > > > > On Tuesday, June 21, 2011, Erick Erickson <erickerick...@gmail.com> > wrote: > > > What is it you want help with? You haven't told us what the > > > problem you're trying to solve is. Are you asking how to > > > speed up indexing? What have you tried? Have you > > > looked at: http://wiki.apache.org/solr/FAQ#Performance? > > > > > > Best > > > Erick > > > > > > On Tue, Jun 21, 2011 at 2:16 AM, Jonty Rhods <jonty.rh...@gmail.com> > > wrote: > > >> I am using solrj to index the data. I have around 50000 docs indexed. > As > > at > > >> the time of commit due to lock server stop giving response so I was > > >> calculating commit time: > > >> > > >> double starttemp = System.currentTimeMillis(); > > >> server.add(docs); > > >> server.commit(); > > >> System.out.println("total time in commit = " + > > (System.currentTimeMillis() - > > >> starttemp)/1000); > > >> > > >> It taking around 9 second to commit the 5000 docs with 15 fields. > > However I > > >> am not confirm the lock time of index whether it is start > > >> since server.add(docs); time or server.commit(); time only. > > >> > > >> If I am changing from above to following > > >> > > >> server.add(docs); > > >> double starttemp = System.currentTimeMillis(); > > >> server.commit(); > > >> System.out.println("total time in commit = " + > > (System.currentTimeMillis() - > > >> starttemp)/1000); > > >> > > >> then commit time becomes less then 1 second. I am not sure which one > is > > >> right. > > >> > > >> please help. > > >> > > >> regards > > >> Jonty > > >> > > > > > >
