Hi All, I am running Nutch on Hadoop2.3.0 and I get Hadoop Spill Exception even though the disk space that is being utilized is only 30% of the available.
I am looking at the syslog file on the userlogs directory however , I do not see much information there except the following line. How do I know whats causing this spill exception as disk space does not seem to be the problem. I see the following entry in the syslog file. 2014-04-17 12:17:28,517 INFO [FetcherThread] org.apache.hadoop.mapred.MapTask: kvstart = 6159120(24636480); kvend = 6152164(24608656); length = 6957/6553600 2014-04-17 12:17:28,528 INFO [FetcherThread] org.apache.hadoop.mapred.MapTask: (EQUATOR) 3661729 kvi 915428(3661712) 2014-04-17 12:17:28,529 ERROR [FetcherThread] org.apache.nutch.fetcher.Fetcher: fetcher caught:java.io.IOException: Spill failed Thanks for your help.