Re: MapRed Job Completes; Output Ceases Mid-Job

2009-10-08 Thread Geoffry Roberts
Jason, Quite possibly, here's what I did: I upped "dfs.datanode.max.xcievers" to 512, which is a doubling, and the full set of output files are created correctly. Thanks for responding. Learning, learning the ins and outs of Hadoop. On Thu, Oct 8, 2009 at 6:01 AM, Jason Venner wrote: > Are yo

Re: MapRed Job Completes; Output Ceases Mid-Job

2009-10-08 Thread Jason Venner
Are you perhaps creating large numbers of files, and running out of file descriptors in your tasks. On Wed, Oct 7, 2009 at 1:52 PM, Geoffry Roberts wrote: > All, > > I have a MapRed job that ceases to produce output about halfway through. > The obvious question is why? > > This job reads a file a