I tried it again (rm -rf /solr/index and post all the docs again) but this time, I get the error (I also switched to the Sun JVM to see if that helped):
15-Aug-08 4:57:08 PM org.apache.solr.core.SolrCore execute INFO: webapp=/solr path=/update params={} status=500 QTime=4576 15-Aug-08 4:57:08 PM org.apache.solr.common.SolrException log SEVERE: javax.xml.stream.XMLStreamException: required string: "field" at gnu.xml.stream.XMLParser.error(libgcj.so.8rh) at gnu.xml.stream.XMLParser.require(libgcj.so.8rh) at gnu.xml.stream.XMLParser.readEndElement(libgcj.so.8rh) at gnu.xml.stream.XMLParser.next(libgcj.so.8rh) at org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:323) at org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:197) at org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:125) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:128) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1143) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) 2008-08-15 16:57:08.440::WARN: EXCEPTION java.lang.NullPointerException at org.mortbay.io.bio.SocketEndPoint.getRemoteAddr(SocketEndPoint.java:116) at org.mortbay.jetty.Request.getRemoteAddr(Request.java:746) at org.mortbay.jetty.NCSARequestLog.log(NCSARequestLog.java:230) at org.mortbay.jetty.handler.RequestLogHandler.handle(RequestLogHandler.java:51) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) On Fri, Aug 15, 2008 at 8:26 AM, Doug Steigerwald <[EMAIL PROTECTED]> wrote: > We actually have this same exact issue on 5 of our cores. We're just going > to wipe the index and reindex soon, but it isn't actually causing any > problems for us. We can update the index just fine, there's just no merging > going on. > > Ours happened when I reloaded all of our cores for a schema change. I don't > do that any more ;). > > Doug > > On Aug 14, 2008, at 11:08 PM, Yonik Seeley wrote: > >> Since this looks like more of a lucene issue, I've replied in >> [EMAIL PROTECTED] >> >> -Yonik >> >> On Thu, Aug 14, 2008 at 10:18 PM, Ian Connor <[EMAIL PROTECTED]> wrote: >>> >>> I seem to be able to reproduce this very easily and the data is >>> medline (so I am sure I can share it if needed with a quick email to >>> check). >>> >>> - I am using fedora: >>> %uname -a >>> Linux ghetto5.projectlounge.com 2.6.23.1-42.fc8 #1 SMP Tue Oct 30 >>> 13:18:33 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux >>> %java -version >>> java version "1.7.0" >>> IcedTea Runtime Environment (build 1.7.0-b21) >>> IcedTea 64-Bit Server VM (build 1.7.0-b21, mixed mode) >>> - single core (will use shards but each machine just as one HDD so >>> didn't see how cores would help but I am new at this) >>> - next run I will keep the output to check for earlier errors >>> - very and I can share code + data if that will help >>> >>> On Thu, Aug 14, 2008 at 4:23 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote: >>>> >>>> Yikes... not good. This shouldn't be due to anything you did wrong >>>> Ian... it looks like a lucene bug. >>>> >>>> Some questions: >>>> - what platform are you running on, and what JVM? >>>> - are you using multicore? (I fixed some index locking bugs recently) >>>> - are there any exceptions in the log before this? >>>> - how reproducible is this? >>>> >>>> -Yonik >>>> >>>> On Thu, Aug 14, 2008 at 2:47 PM, Ian Connor <[EMAIL PROTECTED]> >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I have rebuilt my index a few times (it should get up to about 4 >>>>> Million but around 1 Million it starts to fall apart). >>>>> >>>>> Exception in thread "Lucene Merge Thread #0" >>>>> org.apache.lucene.index.MergePolicy$MergeException: >>>>> java.lang.IndexOutOfBoundsException: Index: 105, Size: 33 >>>>> at >>>>> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:323) >>>>> at >>>>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:300) >>>>> Caused by: java.lang.IndexOutOfBoundsException: Index: 105, Size: 33 >>>>> at java.util.ArrayList.rangeCheck(ArrayList.java:572) >>>>> at java.util.ArrayList.get(ArrayList.java:350) >>>>> at >>>>> org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:260) >>>>> at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:188) >>>>> at >>>>> org.apache.lucene.index.SegmentReader.document(SegmentReader.java:670) >>>>> at >>>>> org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:349) >>>>> at >>>>> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134) >>>>> at >>>>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3998) >>>>> at >>>>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3650) >>>>> at >>>>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:214) >>>>> at >>>>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:269) >>>>> >>>>> >>>>> When this happens, the disk usage goes right up and the indexing >>>>> really starts to slow down. I am using a Solr build from about a week >>>>> ago - so my Lucene is at 2.4 according to the war files. >>>>> >>>>> Has anyone seen this error before? Is it possible to tell which Array >>>>> is too large? Would it be an Array I am sending in or another internal >>>>> one? >>>>> >>>>> Regards, >>>>> Ian Connor >>>>> >>>> >>> >>> >>> >>> -- >>> Regards, >>> >>> Ian Connor >>> > > -- Regards, Ian Connor 1 Leighton St #605 Cambridge, MA 02141 Direct Line: +1 (978) 6333372 Call Center Phone: +1 (714) 239 3875 (24 hrs) Mobile Phone: +1 (312) 218 3209 Fax: +1(770) 818 5697 Suisse Phone: +41 (0) 22 548 1664 Skype: ian.connor