[ https://issues.apache.org/jira/browse/LUCENE-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230302#comment-14230302 ]
Michael McCandless commented on LUCENE-5987: -------------------------------------------- I'll try to tackle this (make abort a tragedy): it's a ridiculous situation, today, that IW can throw an exception and it turns out that exception silently just deleted tons of previously indexed documents and when you close/commit the IW, they are gone. > Make indexwriter a mere mortal when exceptions strike > ----------------------------------------------------- > > Key: LUCENE-5987 > URL: https://issues.apache.org/jira/browse/LUCENE-5987 > Project: Lucene - Core > Issue Type: Task > Reporter: Robert Muir > Assignee: Michael McCandless > > IndexWriter's exception handling is overly complicated. Every method in > general reads like this: > {code} > try { > try { > try { > ... > // lock order: COMPLICATED > synchronized(this or that) { > } > ... > } finally { > if (!success5) { > deleter.deleteThisFileOrThat(); > } > ... > } > } > {code} > Part of the problem is it acts like its an invincible superhero, e.g. can > take a disk full on merge or flush to the face and just keep on trucking, and > you can somehow fix the root cause and then just go about making commits on > the same instance. > But we have a hard enough time ensuring exceptions dont do the wrong thing > (e.g. cause corruption), and I don't think we really test this crazy behavior > anywhere: e.g. making commits AFTER hitting disk full and so on. > It would probably be simpler if when such things happen, IW just considered > them "tragic" just like OOM and rolled itself back, instead of doing all > kinds of really scary stuff to try to "keep itself healthy" (like the little > dance it plays with IFD in mergeMiddle manually deleting CFS files). > Besides, without something like a WAL, Indexwriter isn't really fit to be a > superhero anyway: it can't prevent you from losing data in such situations. > It just doesn't have the right tools for the job. > edit: just to be clear I am referring to abort (low level exception during > flush) and exceptions during merge. For simple non-aborting cases like > analyzer errors, of course we can deal with this. We already made great > progress on turning a lot of BS exceptions that would cause aborts into > non-aborting ones recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org