Sounds like a memory issue. Can you check my other reply in this thread?
> No. This is standard Sun (Oracle) JVM (Java version 1.6.0_27). I even > tried with 1.6.0_24 but with same effect. Only the time it takes for > the crawler to hang and jvm to crash varies. But then, it varies even > between different runs. > > On Wed, Nov 2, 2011 at 2:05 AM, Markus Jelsma > > <[email protected]> wrote: > > Are you using any non default or experimental JVM options? I've never > > seen this happening anywhere with standard SUN JVM's. > > > >> Hi, > >> > >> My problem might not be suitable for the nutch mailing list but I > >> asked on java mailing lists but to no avail and wonder if someone here > >> has experienced the same. > >> > >> I am trying to crawl several hosts using Nutch(1.4) and storing > >> content on Solr with one host per index(core). I had posted this > >> problem earlier at > >> http://lucene.472066.n3.nabble.com/Nutch-Crawl-to-Solr-with-separate-cor > >> es- for-hosts-td3447260.html and could get SolrWriter to create host > >> specific cores. > >> > >> Unfortunately while this works for a sample crawl on local machine, it > >> gets stuck (and crashes with the JRE) on an EC2 instance (with JRE > >> version: 6.0_27-b07) producing an error dump posted at > >> http://paste.pocoo.org/show/501326/. > >> > >> Has anybody faced similar problem or has clue about what might be > >> going wrong or what diagnostics to do? Please let me know if I can > >> provide any further information that might be useful. > >> > >> Best regards, > >> > >> --Sudip.
