For what it's worth, I was seeing mysterious hangs, but it went away when
upgrading from spark1.2 to 1.2.1.I don't know if this is your problem.Also, I'm
using AWS EMR images, which were also "upgraded".
Anyway, that's my experience.
-Mike
From: Manas Kar <[email protected]>
To: "[email protected]" <[email protected]>
Sent: Friday, February 27, 2015 3:50 PM
Subject: How to debug a Hung task
Hi, I have a spark application that hangs on doing just one task (Rest 200-300
task gets completed in reasonable time)I can see in the Thread dump which
function gets stuck how ever I don't have a clue as to what value is causing
that behaviour.Also, logging the inputs before the function is executed does
not help as the actual message gets buried in logs.
How do one go about debugging such case?Also, is there a way I can wrap my
function inside some sort of timer based environment and if it took too long I
would throw a stack trace or some sort.
ThanksManas
