Yeah I figured that was too many deleteddocs. It could just be that our max segments is set too high though.
The reason I asked is because our optimize requests have started failing. Or at least,they are appearing to fail because the optimize request returns a non 200. The optimize seems to go ahead successfully regardless though. Before trying to find out if I can asynchronously request and poll for success (doesn't appear to be possible yet) or a better way of determining success, I thought I'd check if the whole thing was necessary to begin with. Hopefully it doesn't involve polling the core status until deleteddocs goes below a certain level :/. Cheers for info. On Mon, Aug 8, 2016 at 2:58 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 8/8/2016 3:10 AM, Callum Lamb wrote: > > How true is this claim? Is optimizing still a good idea for the > > general case? > > For the general case, optimizing is not recommended. If there are a > very large number of deleted documents, which does describe your > situation, then there is definitely a benefit. > > In cases where there are a lot of deleted documents, scoring can be > affected by the presence of the deleted documents, and the drop in index > size after an optimize can result in a large performance boost. For the > general case where there are not many deletes, there *is* a performance > benefit to optimizing down to a single segment, but it is nowhere near > as dramatic as it was in the 1.x/3.x days. > > The problem with optimizes in the general case is this: The performance > hit that the optimize operation itself causes may not be worth the small > performance improvement. > > If you have a time where your index is quiet enough that the optimize > itself won't be disruptive, then you should certainly take advantage of > that time and do the optimize, even if there aren't many deletes. > > There is another benefit to optimizes that doesn't get mentioned often: > It can make subsequent normal merging operations during indexing faster, > because there will not be as many large segments. > > Thanks, > Shawn > > -- Mintel Group Ltd | 11 Pilgrim Street | London | EC4V 6RN Registered in England: Number 1475918. | VAT Number: GB 232 9342 72 Contact details for our other offices can be found at http://www.mintel.com/office-locations. This email and any attachments may include content that is confidential, privileged or otherwise protected under applicable law. Unauthorised disclosure, copying, distribution or use of the contents is prohibited and may be unlawful. If you have received this email in error, including without appropriate authorisation, then please reply to the sender about the error and delete this email and any attachments.
