Incidentally.. this is what TerminateProcess does say from MSDN: TerminateProcess initiates termination and returns immediately. This stops execution of all threads within the process and requests cancellation of all pending I/O. The terminated process cannot exit until all pending I/O has been completed or canceled.
So I am not sure if this is a problem or not.. if a background merge is happening while TerminateProcess() is called, then potentially, the Lucene libraries may have written some data, but I wonder if the O/S can "cancel" it before it is committed? You would hope if fsync() equivalent is called, that the pending I/O operations can't be cancelled, but I do wonder? On 28 July 2010 15:04, David Sitsky <s...@nuix.com> wrote: > Hi Mike, > > Definitely no OOME (or other critical errors), and if there were, we > would have terminated the program straight away. > > Cheers, > David > > On 28 July 2010 01:44, Michael McCandless <luc...@mikemccandless.com> wrote: >> Were there any exceptions during indexing, before the >> TerminateProcess() call? EG OOME? >> >> Mike >> >> On Tue, Jul 27, 2010 at 8:43 AM, Michael McCandless >> <luc...@mikemccandless.com> wrote: >>> On Mon, Jul 26, 2010 at 7:10 PM, David Sitsky <s...@nuix.com> wrote: >>>> During processing.. there might be a number of reasons why we need to >>>> shutdown the indexing process, but perhaps what is unusual is we call >>>> the win32 API TerminateProcess() call rather than System.exit(), for >>>> slightly obscure reasons. When calling exit(), this still calls a >>>> large body of code (for example dll shutdown hooks) which in some >>>> situations, we found could "hang" the exiting process, which was a >>>> problem for us. >>>> >>>> In a sense, this should be no different to killing the process under >>>> Windows using the task manager, or kill -9 on unix systems. >>> >>> OK. Lucene should be fine after a kill -9 or a TerminateProcess, >>> assuming Windows really does act like Unix and any "committed" IO >>> operations done by the process prior to the kill are in fact committed >>> to the filesystem. >>> >>>> At no time did the machine itself crash, and the disk involved (I'm >>>> told) was a local RAID filesystem. I am guessing the disk has write >>>> caching enabled, but given the machine didn't crash, this shouldn't >>>> matter. >>> >>> Right, the write caching shouldn't matter since the machine didn't go down. >>> >>>> Something else that is slightly unusual is we explicitly call commit() >>>> at certain times to flush indexing work to disk. >>> >>> OK that's fine. >>> >>>> Its interesting in both instances, CheckIndex said there was 1 broken >>>> segment containing 1 document. >>> >>> Yeah that is curious.... I'll try to mull. >>> >>> Any chance you could run with an infoStream set on IndexWriter? Then >>> if this happens again I can pour over that... >>> >>> Mike >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> > > > > -- > Cheers, > David > > Nuix Pty Ltd > Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699 > Web: http://www.nuix.com Fax: +61 2 9212 6902 > -- Cheers, David Nuix Pty Ltd Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699 Web: http://www.nuix.com Fax: +61 2 9212 6902 --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
