Reduce shuffle data transfer takes excessively long

Hello, I have been working on profiling the performance of certain parts of Hadoop 0.20.203.0. For this reason, I have set up a simple cluster that uses one node as the Namenode/Jobtracker, and one node as the sole Datanode/tasktracker. In this experiment, I run a job consisting of a single

Re: Writing large output kills job with timeout _ need ideas

An earlier reply at http://search-hadoop.com/m/e9dM3rw9IP1 may help you get over the idle task issue, if you're idle due to processing and not a real freeze. On Thu, Jan 26, 2012 at 8:45 PM, Radim Kolar wrote: > Any bright ideas? > > > call status update or Progress every 600 seconds or less -

Re: Writing large output kills job with timeout _ need ideas

Any bright ideas? call status update or Progress every 600 seconds or less