If you jstack the process in the middle of one of these pauses, can you see
where it's sticking?
- Aaron

On Fri, Mar 13, 2009 at 6:51 AM, Doug Cook <[email protected]> wrote:

>
> Hi folks,
>
> I've been debugging a severe performance problems with a Hadoop-based
> application (a highly modified version of Nutch). I've recently upgraded to
> Hadoop 0.19.1 from a much, much older version, and a reduce that used to
> work just fine is now running orders of magnitude more slowly.
>
> From the logs I can see that progress of my reduce stops for periods that
> average almost exactly 10 seconds (with a very narrow distribution around
> 10
> seconds), and it does so in various places in my code, but more or less in
> proportion to how much time I'd expect the task would normally spend in
> that
> particular place in the code, i.e. the behavior seems like my code is
> randomly being interrupted for 10 seconds at a time.
>
> I'm planning to keep digging, but thought that these symptoms might sound
> familiar to someone on this list. Ring any bells? Your help much
> appreciated.
>
> Thanks!
>
> Doug Cook
> --
> View this message in context:
> http://www.nabble.com/Reduce-task-going-away-for-10-seconds-at-a-time-tp22496810p22496810.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

Reply via email to