Hmm, it may also be a memory problem. You have both Nutch and Tomcat + Solr running on the same machine with limited RAM? 4GB allocated to Nutch and how much to Tomcat?
Remeber that file descriptors take memory too, it adds up significantly if there are many. Both Tomcat + Solr and Nutch can open of a lot. > Are you using any non default or experimental JVM options? I've never seen > this happening anywhere with standard SUN JVM's. > > > Hi, > > > > My problem might not be suitable for the nutch mailing list but I > > asked on java mailing lists but to no avail and wonder if someone here > > has experienced the same. > > > > I am trying to crawl several hosts using Nutch(1.4) and storing > > content on Solr with one host per index(core). I had posted this > > problem earlier at > > http://lucene.472066.n3.nabble.com/Nutch-Crawl-to-Solr-with-separate-core > > s- for-hosts-td3447260.html and could get SolrWriter to create host > > specific cores. > > > > Unfortunately while this works for a sample crawl on local machine, it > > gets stuck (and crashes with the JRE) on an EC2 instance (with JRE > > version: 6.0_27-b07) producing an error dump posted at > > http://paste.pocoo.org/show/501326/. > > > > Has anybody faced similar problem or has clue about what might be > > going wrong or what diagnostics to do? Please let me know if I can > > provide any further information that might be useful. > > > > Best regards, > > > > --Sudip.
