It's also exceptionally strange that the size of your fdx files, when you hit the exception, is not even, and so out of whack (103, 8457 bytes) given that you have 1 doc.
Are there virus checkers or any other filesystem spying type apps running on this one machine? Or... is it possible two IWs were accidentally opened on the same directory? Are you changing the lock factory for your FSDirectory? Mike On Thu, Apr 15, 2010 at 11:24 AM, jm <jmugur...@gmail.com> wrote: > not sure if it matters, but I have to correct my statment, where this > has happened was both times win2008 R1 64bits, local filesystem. > > I am trying to reproduce in my dev workstation but unable so far. > > On Thu, Apr 15, 2010 at 10:11 AM, jm <jmugur...@gmail.com> wrote: >> Hi Mike >> >> I have a server side, exposes a webservice with operations (this is >> simplified ignoring things non lucene related): >> - addToIndex(doc, date) >> - flushAllIndexes() >> >> addToIndex() adds the doc to the index, the index is chosen based on >> the date (simplified it's one index per day). To speed this up the >> webservice keeps a cache of IndexWriters. If the index is already >> open it uses it, otherwise adds it to the cache (the index is created >> if it does not exist). I have a listener on the cache expiration so >> the index is closed when it is removed from the cache (the cache has >> only 10 items). >> >> flushAllIndexes() just gets all indexes open on the cache and closes them >> all. >> >> The webservice can serve many requests at the same time, but all >> operations dealing with lucene are synchonized. I guess this is most >> likely a bug in my synching code etc so I have verified it many times. >> Even trying to reproduce the issue by stopping threads in a debugger >> worked fine, synchronized methods were working as expected (I >> understand am probably missing the case that triggers the issue). >> >> The last time this happened there was a single thread (in the client) >> calling addToIndex(doc, date) many times and then it called >> flushAllIndexes() The next second I got the error. The client >> developer assures me the call to flushAllIndexes() as in the same >> thread as addToIndex(doc, date) (anyway, it should work even if it was >> different threads of course). >> >> yes, local filesystem under Vista Business sp1 32b. >> >> Hope this helps. I am just now developing a client that pounds the >> server with some threads calling the methods all the time to try to >> reproduce the issue. >> >> javier >> >> On Thu, Apr 15, 2010 at 1:47 AM, Michael McCandless >> <luc...@mikemccandless.com> wrote: >>> Not good! >>> >>> Can you describe how your threads work with Lucene? >>> >>> Is this just a local filesystem (disk) under vista? >>> >>> Mike >>> >>> On Wed, Apr 14, 2010 at 7:41 AM, jm <jmugur...@gmail.com> wrote: >>>> Hi, >>>> >>>> I am trying to chase an issue in our code and it is being quite >>>> difficult. We have seen two instances (see below) where we get the >>>> same error. I have been trying to reproduce but it has been impossible >>>> so far.I have several threads, some might be creating indices and >>>> adding documents, others closing indices etc. In the list this >>>> message appear several times, in some cases it was a lucene bug (not >>>> my case, I dont have millions of docs here) and in other it seems bad >>>> multithreading. I have verified my multithreaded code for the 5th time >>>> and all seems well synchronized. >>>> >>>> So I am asking if somebody can give me any hint, maybe the fact that >>>> the first number is always a 1 means something?? I suspect the issue >>>> is when a new index is just created maybe. >>>> >>>> java.lang.RuntimeException: after flush: fdx size mismatch: 1 docs vs >>>> 103 length in bytes of _2.fdx file exists?=true >>>> org.apache.lucene.index.StoredFieldsWriter.closeDocStore(StoredFieldsWriter.java:97) >>>> org.apache.lucene.index.DocFieldProcessor.closeDocStore(DocFieldProcessor.java:50) >>>> org.apache.lucene.index.DocumentsWriter.closeDocStore(DocumentsWriter.java:380) >>>> org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:581) >>>> org.apache.lucene.index.IndexWriter.doFlushInternal(IndexWriter.java:4294) >>>> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:4192) >>>> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:4183) >>>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:2190) >>>> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2153) >>>> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2117) >>>> >>>> >>>> java.lang.RuntimeException: after flush: fdx size mismatch: 1 docs vs >>>> 8457 length in bytes of _0.fdx file exists?=true ... >>>> >>>> My env is vista, jdk1.6, lucene 2.9.1 >>>> thanks in advance. >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >>> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
