Harsh: thanks for the quick response. we often see an error response such as "Failed(Query returned non-zero code: 2, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask)" and then go through all the logs to figure out what happened. I use the jobtracker UI to go to the error logs and see what happened.
I was thinking a log parsing tool with a good UI to go through the distributed-logs and help you find errors, get stats on similar effors in prev runs etc will be useful. HADOOP-9861 might help in getting good info, but might be still not very easy for quick debugging. Has anybody faced similar issues as part of their development? Are there any better ways to pin point the cause of error? Thx Gopi | www.wignite.com On Tue, Aug 27, 2013 at 10:42 AM, Harsh J <ha...@cloudera.com> wrote: > We set a part of the failure reason as the diagnostic message for a > failed task that a JobClient API retrieves/can retrieve: > > http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/RunningJob.html#getTaskDiagnostics(org.apache.hadoop.mapred.TaskAttemptID) > . > Often this is > 'useless' given the stack trace's top part isn't always carrying the > most relevant information, so perhaps HADOOP-9861 may help here once > it is checked in. > > On Tue, Aug 27, 2013 at 10:34 AM, Gopi Krishna M <mgo...@gmail.com> wrote: > > Hi > > > > We are seeing our map-reduce jobs crashing once in a while and have to go > > through the logs on all the nodes to figure out what went wrong. > Sometimes > > it is low resources and sometimes it is a programming error which is > > triggered on specific inputs.. Same is true for some of our hive > queries. > > > > Are there any tools (free/paid) which help us to do this debugging > quickly? > > I am planning to write a debugging tool for sifting through the > distributed > > logs of hadoop but wanted to check if there are already any useful tools > for > > this. > > > > Thx > > Gopi | www.wignite.com > > > > -- > Harsh J >
