Not sure how it plays with segment merging and optimizing, but have you considered DocValues for your price fields? On the horizon there's work being done to allow them to be independently updated (although that won't help you now of course). It's not clear at this point how that will play when lots and lots and lots of updates happen though.
Of course an optimize will purge your deletes, but I'm sure you already know that I'm sure. On Wed, Dec 3, 2014 at 5:45 AM, Alexey Kozhemiakin <alexey_kozhemia...@epam.com> wrote: > Hi Alexandre, our rebuilds are not like 'full rebuilds' - it's a constant > massive flow of price updates in ecommerce marketplace. Unfortunately > "substitution" option is not working for us :( > > -----Original Message----- > From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] > Sent: Wednesday, December 3, 2014 16:39 > To: solr-user > Subject: Re: SegmentInfos exposed to /admin/luke > > You can't use grouping aliases and do full rebuilds on a separate core > + substitutions? Might be a better strategy for nearly complete > replacement. > > Regards, > Alex. > P.s. But I like your proposal anyway. > > Personal: http://www.outerthoughts.com/ and @arafalov > Solr resources and newsletter: http://www.solr-start.com/ and @solrstart > Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 > > > On 3 December 2014 at 06:35, Alexey Kozhemiakin > <alexey_kozhemia...@epam.com> wrote: >> Dear All, >> >> We have a high percentage of deleted docs which do not go away because there >> are several huge ancient segments that do not merge with anything else >> naturally. Our use case in constant reindexing of same data - ~100 gb, 12 >> 000 000 real records, 20 000 000 total records in index, which is ~80% >> deletes. >> >> We plan to deal with situation by playing with mergeFactor, >> reclaimDeletesWeight and maxSegmentSizeMB settings to optimize for our >> re-indexing rate and data size. >> And in order to do it with eyes-opened we want to see a picture similar to >> http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html >> with columns of segment size and %of deletes. >> The plan is to expose SegmentInfos via /admin/luke handler and draw column >> bars in Solr admin. >> >> Is there an easier way to achieve that? Even in raw Luke we didn't' found >> these data. >> >> We'd be happy to push the changes to Solr afterwards. >> >> >> Thank you, >> Alexey Kozhemiakin >>
