Hi Markus, Thanks for the help. I also suspected it to be a memory issue but after wrangling with the problem for days I just tried it on a different EC2 instance, with same jdk and lo behold! it worked without the slightest of problems.
I have no clue what was causing the trouble and this is frustrating but glad that it finally did, albeit on a different machine. Thanks, --Sudip. On Wed, Nov 2, 2011 at 2:14 AM, Markus Jelsma <[email protected]> wrote: > Hmm, it may also be a memory problem. You have both Nutch and Tomcat + Solr > running on the same machine with limited RAM? 4GB allocated to Nutch and how > much to Tomcat? > > Remeber that file descriptors take memory too, it adds up significantly if > there are many. Both Tomcat + Solr and Nutch can open of a lot. > > > > >> Are you using any non default or experimental JVM options? I've never seen >> this happening anywhere with standard SUN JVM's. >> >> > Hi, >> > >> > My problem might not be suitable for the nutch mailing list but I >> > asked on java mailing lists but to no avail and wonder if someone here >> > has experienced the same. >> > >> > I am trying to crawl several hosts using Nutch(1.4) and storing >> > content on Solr with one host per index(core). I had posted this >> > problem earlier at >> > http://lucene.472066.n3.nabble.com/Nutch-Crawl-to-Solr-with-separate-core >> > s- for-hosts-td3447260.html and could get SolrWriter to create host >> > specific cores. >> > >> > Unfortunately while this works for a sample crawl on local machine, it >> > gets stuck (and crashes with the JRE) on an EC2 instance (with JRE >> > version: 6.0_27-b07) producing an error dump posted at >> > http://paste.pocoo.org/show/501326/. >> > >> > Has anybody faced similar problem or has clue about what might be >> > going wrong or what diagnostics to do? Please let me know if I can >> > provide any further information that might be useful. >> > >> > Best regards, >> > >> > --Sudip. >
