> No, this doesn't make sense. The OS detects a disk full on accepting > the write into the write cache, not [later] on flushing the write > cache to disk. If the OS accepts the write, then disk is not full (ie > flushing the cache will succeed, unless some other not-disk-full > problem happens). > > Hmmm, at least, normally. What OS/IO system were you on when you saw > corruption due to disk full when fsync is disabled? > > I'm still skeptical that disk full even with fsync disabled can lead > to corruption.... I'd like to see some concrete proof :)
Linux 2.6.30-1-amd64, ext3, simple scsi drive I checked with our resident DB brainiac, he says such things are possible. Okay, I'm not 100% sure this is the cause of my corruptions. It just happened that when the index got corrupted, disk space was also used up - several times. I had that silent-fail-to-write theory and checked it up with some knowledgeable people. Even if they are right, I can be mistaken and the root cause is different. > You're mixing up terminology a bit here -- you can't "hold on to the > latest commit then switch to it". A commit (as sent to the deletion > policy) means a *real* commit (ie, IW.commit or IW.close was called). > So I think your BG thread would simply be calling IW.commit every N > seconds? Under "hold on to" I meant - keep from being deleted, like SnapshotDP does. >> I'm just playing around with stupid idea. I'd like to have NRT >> look-alike without binding readers and writers. :) > I see... well binding durability & visibility will always be costly. > This is why Lucene decouples them (by making NRT readers available). My experiments do the same, essentially. But after I understood that to perform deletions IW has to load term indexes anyway, I'm almost ready to give up and go for intertwined IW/IR mess :) > BTW, if you know your OS/IO system always persists cached writes w/in > N seconds, a safe way to avoid fsync is to use a by-time expiring > deletion policy. Ie, a commit stays alive as long as its age is less > than X... DP's unit test has such a policy. But you better really > know for sure that the OS/IO system guarantee that :) Yeah. I thought of it, but it is even more shady :) -- Kirill Zakharenko/Кирилл Захаренко (ear...@gmail.com) Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423 ICQ: 104465785 --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
