At root, it's an OOM: Caused by: java.lang.OutOfMemoryError: Map failed at I'm guessing that you're optimizing after the import? What are the JVM settings you're using? The standard response is "increase the amount of memory available to the JVM", but it's expensive to change it and only find out you're running over the limit *after* a billion docs.
The "standard" advice is to allow the JVM about half the memory available on the machine, leaving the rest for the op system to use as it sees fit, but that's just a starting point. Hope that helps Erick On Fri, Dec 24, 2010 at 1:19 AM, Rok Rejc <rokrej...@gmail.com> wrote: > Hi all, > > I have created a new index (using Solr trunk version from 17th December, > running on Windows 7 & Tomcat 6, 64 bit JVM) with around 1.1 billion of > documents (index size around 550GB, mergeFactor=20). > > After the (csv) import I have commited the data and got this error: > > HTTP Status 500 - Severe errors in solr configuration. Check your log files > for more detailed information on what may be wrong. > ------------------------------------------------------------- > java.lang.RuntimeException: java.io.IOException: Map failed at > org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1095) at > org.apache.solr.core.SolrCore.<init>(SolrCore.java:587) at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:660) at > org.apache.solr.core.CoreContainer.load(CoreContainer.java:412) at > org.apache.solr.core.CoreContainer.load(CoreContainer.java:294) at > > org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:243) > at > org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:86) > at > > org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:295) > at > > org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:422) > at > > org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:115) > at > > org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4001) > at > org.apache.catalina.core.StandardContext.start(StandardContext.java:4651) > at > > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:546) at > > org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:637) > at > > org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:563) > at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:498) > at > org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277) at > org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321) > at > > org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119) > at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) at > org.apache.catalina.core.StandardHost.start(StandardHost.java:785) at > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) at > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:445) at > org.apache.catalina.core.StandardService.start(StandardService.java:519) at > org.apache.catalina.core.StandardServer.start(StandardServer.java:710) at > org.apache.catalina.startup.Catalina.start(Catalina.java:581) at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at > java.lang.reflect.Method.invoke(Unknown Source) at > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by: > java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(Unknown > Source) at > > org.apache.lucene.store.MMapDirectory$MultiMMapIndexInput.<init>(MMapDirectory.java:327) > at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:209) > at > > org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:68) > at > > org.apache.lucene.index.SegmentReader$CoreReaders.openDocStores(SegmentReader.java:208) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:529) at > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:504) at > org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:123) at > org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:91) > at > > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:623) > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86) at > org.apache.lucene.index.IndexReader.open(IndexReader.java:437) at > org.apache.lucene.index.IndexReader.open(IndexReader.java:316) at > > org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38) > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1084) ... 33 > more > Caused by: java.lang.OutOfMemoryError: Map failed at > sun.nio.ch.FileChannelImpl.map0(Native Method) ... 48 more > > I can see that the error is going down to lucene and java, but I don't have > a clue what should I do... Any suggestions? > > Thanks and merry christmas:) > > Rok >
