[
https://issues.apache.org/jira/browse/LUCENE-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-5987:
--------------------------------
Description:
IndexWriter's exception handling is overly complicated. Every method in general
reads like this:
{code}
try {
try {
try {
...
// lock order: COMPLICATED
synchronized(this or that) {
}
...
} finally {
if (!success5) {
deleter.deleteThisFileOrThat();
}
...
}
}
{code}
Part of the problem is it acts like its an invincible superhero, e.g. can take
a disk full on merge or flush to the face and just keep on trucking, and you
can somehow fix the root cause and then just go about making commits on the
same instance.
But we have a hard enough time ensuring exceptions dont do the wrong thing
(e.g. cause corruption), and I don't think we really test this crazy behavior
anywhere: e.g. making commits AFTER hitting disk full and so on.
It would probably be simpler if when such things happen, IW just considered
them "tragic" just like OOM and rolled itself back, instead of doing all kinds
of really scary stuff to try to "keep itself healthy" (like the little dance it
plays with IFD in mergeMiddle manually deleting CFS files).
Besides, without something like a WAL, Indexwriter isn't really fit to be a
superhero anyway: it can't prevent you from losing data in such situations. It
just doesn't have the right tools for the job.
edit: just to be clear I am referring to abort (low level exception during
flush) and exceptions during merge. For simple non-aborting cases like analyzer
errors, of course we can deal with this. We already made great progress on
turning a lot of BS exceptions that would cause aborts into non-aborting ones
recently.
was:
IndexWriter's exception handling is overly complicated. Every method in general
reads like this:
{code}
try {
try {
try {
...
// lock order: COMPLICATED
synchronized(this or that) {
}
...
} finally {
if (!success5) {
deleter.deleteThisFileOrThat();
}
...
}
}
{code}
Part of the problem is it acts like its an invincible superhero, e.g. can take
a disk full on merge or flush to the face and just keep on trucking, and you
can somehow fix the root cause and then just go about making commits on the
same instance.
But we have a hard enough time ensuring exceptions dont do the wrong thing
(e.g. cause corruption), and I don't think we really test this crazy behavior
anywhere: e.g. making commits AFTER hitting disk full and so on.
It would probably be simpler if when such things happen, IW just considered
them "tragic" just like OOM and rolled itself back, instead of doing all kinds
of really scary stuff to try to "keep itself healthy" (like the little dance it
plays with IFD in maybeMerge manually deleting CFS files).
Besides, without something like a WAL, Indexwriter isn't really fit to be a
superhero anyway: it can't prevent you from losing data in such situations. It
just doesn't have the right tools for the job.
> Make indexwriter a mere mortal when exceptions strike
> -----------------------------------------------------
>
> Key: LUCENE-5987
> URL: https://issues.apache.org/jira/browse/LUCENE-5987
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Robert Muir
>
> IndexWriter's exception handling is overly complicated. Every method in
> general reads like this:
> {code}
> try {
> try {
> try {
> ...
> // lock order: COMPLICATED
> synchronized(this or that) {
> }
> ...
> } finally {
> if (!success5) {
> deleter.deleteThisFileOrThat();
> }
> ...
> }
> }
> {code}
> Part of the problem is it acts like its an invincible superhero, e.g. can
> take a disk full on merge or flush to the face and just keep on trucking, and
> you can somehow fix the root cause and then just go about making commits on
> the same instance.
> But we have a hard enough time ensuring exceptions dont do the wrong thing
> (e.g. cause corruption), and I don't think we really test this crazy behavior
> anywhere: e.g. making commits AFTER hitting disk full and so on.
> It would probably be simpler if when such things happen, IW just considered
> them "tragic" just like OOM and rolled itself back, instead of doing all
> kinds of really scary stuff to try to "keep itself healthy" (like the little
> dance it plays with IFD in mergeMiddle manually deleting CFS files).
> Besides, without something like a WAL, Indexwriter isn't really fit to be a
> superhero anyway: it can't prevent you from losing data in such situations.
> It just doesn't have the right tools for the job.
> edit: just to be clear I am referring to abort (low level exception during
> flush) and exceptions during merge. For simple non-aborting cases like
> analyzer errors, of course we can deal with this. We already made great
> progress on turning a lot of BS exceptions that would cause aborts into
> non-aborting ones recently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
