Looks to me like if you’re using TieredMergePolicy, forceMergeDeletesPctAllowed is simply a way of including a segment as a merge candidate that would not have otherwise been a candidate (due to being a different tier). Setting it to zero means never include a segment as a merge candidate due to delete pct, setting it to 100 means any segment with a delete is a candidate.
Related, TieredMergePolicy .reclaimDeletesWeight alters the likelihood that two candidate segments will be selected for a merge based on delete ratio. Related, doing optimize with expungeDeletes=true simply causes TieredMergePolicy to use the forceMergeDeletesPctAllowed. Nothing here really answers Shawn’s use case though, since in all cases you’re still required to merge segments to remove deletes. Clearly, we could set a very high forceMergeDeletesPctAllowed and reclaimDeletesWeight, but also clearly, if we really wanted to remove all deletes, an optimize with maxSegments=1 would do the job. It’s just expensive, and it’ll be that expensive again as soon as you get more deletes. In my mind, I’d want to say “if deletes are more than x% of a given segment’s size, just re-write the segment while filtering out the deletes. The only reason I might care about the merge policy is that I might prefer to do this only on the larger/longer-lived segments that don’t get merged often. From: David Smiley <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Monday, August 15, 2016 at 8:56 PM To: "[email protected]" <[email protected]> Subject: Re: How hard would a "wipe all deletes" operation be? Shawn: https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers Search for "expungeDeletes". So it's 10% by default. I did some digging and I see this 10% figure is settable on the TieredMergePolicy. So you could modify solrconfig.xml and set <forceMergeDeletesPctAllowed>0</forceMergeDeletesPctAllowed> as a setting on the merge policy. On Thu, Aug 11, 2016 at 2:59 PM Shawn Heisey <[email protected]<mailto:[email protected]>> wrote: On 8/11/2016 10:58 AM, David Smiley wrote: > Note there is a threshold to expungeDeletes such that if there aren't > enough deletes in a segment relative to the docs in that segments, it > won't do any expunging. What sort of request do I need to send to force expunging *all* deleted documents, even if there's only one in a segment? Is that possible? That's the end goal of all this scheming. Thanks, Shawn --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]> -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
