Question about the purpose of reindexing
I've been trying to find a concise explanation of this, and seem to so far have missed it. (Google, etc). What is the purpose/need to reindex a solr index? How do you determine what provides the best performance? What detrimental affects occur if you operate off of delta indexes? Aaron Chmelik Web Designer & Programmer email: aaron.chme...@gmail.com website: http://webdesign.aaronchmelik.com phone: 651.757.5979
Re: Question about the purpose of reindexing
Define reindexing. Every new document is indexed and existing documents are deleted and indexed as if it is a new document. Completely reindexing from scratch is only required if breaking changes are made to the schema or if you upgrade to a new version that uses another format and isn't able to convert. To my knowledge and experience, it is perfectly sane to never do a complete reindex and still accept incremental updates. Just do an optimize once in a while and you're good to go. > I've been trying to find a concise explanation of this, and seem to so far > have missed it. (Google, etc). What is the purpose/need to reindex a solr > index? How do you determine what provides the best performance? What > detrimental affects occur if you operate off of delta indexes? > > Aaron Chmelik > Web Designer & Programmer > email: aaron.chme...@gmail.com > website: http://webdesign.aaronchmelik.com > phone: 651.757.5979
Re: Question about the purpose of reindexing
One more question - what does optimization do? Maybe to be a little more precise - what happens to the index that requires optimizaion (what is the problem and how does optimization solve it). Aaron Chmelik Web Designer & Programmer email: aaron.chme...@gmail.com website: http://webdesign.aaronchmelik.com phone: 651.757.5979 On Thu, May 26, 2011 at 4:27 PM, Aaron Chmelik wrote: > Thanks! coming from Sphinx, with seperate delta index files. Thanks for the > info! > > Aaron Chmelik > Web Designer & Programmer > email: aaron.chme...@gmail.com > website: http://webdesign.aaronchmelik.com > phone: 651.757.5979 > > > On Thu, May 26, 2011 at 4:21 PM, Markus Jelsma > wrote: > >> Define reindexing. Every new document is indexed and existing documents >> are >> deleted and indexed as if it is a new document. Completely reindexing from >> scratch is only required if breaking changes are made to the schema or if >> you >> upgrade to a new version that uses another format and isn't able to >> convert. >> >> To my knowledge and experience, it is perfectly sane to never do a >> complete >> reindex and still accept incremental updates. Just do an optimize once in >> a >> while and you're good to go. >> >> > I've been trying to find a concise explanation of this, and seem to so >> far >> > have missed it. (Google, etc). What is the purpose/need to reindex a >> solr >> > index? How do you determine what provides the best performance? What >> > detrimental affects occur if you operate off of delta indexes? >> > >> > Aaron Chmelik >> > Web Designer & Programmer >> > email: aaron.chme...@gmail.com >> > website: http://webdesign.aaronchmelik.com >> > phone: 651.757.5979 >> > >
Re: Question about the purpose of reindexing
Optimizing an index forces segments to merge. Usually, segments are merged automatically based on your mergeFactor setting. During a merge documents flagged for deletion are really purged and the number of segments is reduces which improves search performance. There are some good pages on mergeFactor around. Haven't got a link at hand. > One more question - what does optimization do? Maybe to be a little more > precise - what happens to the index that requires optimizaion (what is the > problem and how does optimization solve it). > > Aaron Chmelik > Web Designer & Programmer > email: aaron.chme...@gmail.com > website: http://webdesign.aaronchmelik.com > phone: 651.757.5979 > > On Thu, May 26, 2011 at 4:27 PM, Aaron Chmelik wrote: > > Thanks! coming from Sphinx, with seperate delta index files. Thanks for > > the info! > > > > Aaron Chmelik > > Web Designer & Programmer > > email: aaron.chme...@gmail.com > > website: http://webdesign.aaronchmelik.com > > phone: 651.757.5979 > > > > > > On Thu, May 26, 2011 at 4:21 PM, Markus Jelsma > > > > > > wrote: > >> Define reindexing. Every new document is indexed and existing documents > >> are > >> deleted and indexed as if it is a new document. Completely reindexing > >> from scratch is only required if breaking changes are made to the > >> schema or if you > >> upgrade to a new version that uses another format and isn't able to > >> convert. > >> > >> To my knowledge and experience, it is perfectly sane to never do a > >> complete > >> reindex and still accept incremental updates. Just do an optimize once > >> in a > >> while and you're good to go. > >> > >> > I've been trying to find a concise explanation of this, and seem to so > >> > >> far > >> > >> > have missed it. (Google, etc). What is the purpose/need to reindex a > >> > >> solr > >> > >> > index? How do you determine what provides the best performance? What > >> > detrimental affects occur if you operate off of delta indexes? > >> > > >> > Aaron Chmelik > >> > Web Designer & Programmer > >> > email: aaron.chme...@gmail.com > >> > website: http://webdesign.aaronchmelik.com > >> > phone: 651.757.5979
Re: Question about the purpose of reindexing
Thanks. I think I can take it form there! Aaron Chmelik Web Designer & Programmer email: aaron.chme...@gmail.com website: http://webdesign.aaronchmelik.com phone: 651.757.5979 On Thu, May 26, 2011 at 4:51 PM, Markus Jelsma wrote: > Optimizing an index forces segments to merge. Usually, segments are merged > automatically based on your mergeFactor setting. During a merge documents > flagged for deletion are really purged and the number of segments is > reduces > which improves search performance. There are some good pages on mergeFactor > around. Haven't got a link at hand. > > > One more question - what does optimization do? Maybe to be a little more > > precise - what happens to the index that requires optimizaion (what is > the > > problem and how does optimization solve it). > > > > Aaron Chmelik > > Web Designer & Programmer > > email: aaron.chme...@gmail.com > > website: http://webdesign.aaronchmelik.com > > phone: 651.757.5979 > > > > On Thu, May 26, 2011 at 4:27 PM, Aaron Chmelik > wrote: > > > Thanks! coming from Sphinx, with seperate delta index files. Thanks for > > > the info! > > > > > > Aaron Chmelik > > > Web Designer & Programmer > > > email: aaron.chme...@gmail.com > > > website: http://webdesign.aaronchmelik.com > > > phone: 651.757.5979 > > > > > > > > > On Thu, May 26, 2011 at 4:21 PM, Markus Jelsma > > > > > > > > > wrote: > > >> Define reindexing. Every new document is indexed and existing > documents > > >> are > > >> deleted and indexed as if it is a new document. Completely reindexing > > >> from scratch is only required if breaking changes are made to the > > >> schema or if you > > >> upgrade to a new version that uses another format and isn't able to > > >> convert. > > >> > > >> To my knowledge and experience, it is perfectly sane to never do a > > >> complete > > >> reindex and still accept incremental updates. Just do an optimize once > > >> in a > > >> while and you're good to go. > > >> > > >> > I've been trying to find a concise explanation of this, and seem to > so > > >> > > >> far > > >> > > >> > have missed it. (Google, etc). What is the purpose/need to reindex a > > >> > > >> solr > > >> > > >> > index? How do you determine what provides the best performance? What > > >> > detrimental affects occur if you operate off of delta indexes? > > >> > > > >> > Aaron Chmelik > > >> > Web Designer & Programmer > > >> > email: aaron.chme...@gmail.com > > >> > website: http://webdesign.aaronchmelik.com > > >> > phone: 651.757.5979 >
