In our case, which seems to be the same problem, the web UI does not show anything useful except the first line of the stack trace:
2012-01-03 21:16:27,256 WARN org.apache.hadoop.mapred.TaskLog: Failed to retrieve stdout log for task: attempt_201201031651_0008_m_000233_0 Only the task tracker log shows a full stack trace. This happened on 1.0.0 and 0.20.205.0 but not 0.20.203.0. 2012-01-03 21:16:27,256 WARN org.apache.hadoop.mapred.TaskLog: Failed to retrieve stdout log for task: attempt_201201031651_0008_m_000233_0 java.io.FileNotFoundException: /opt/hadoop/hadoop-0.20.205.0/libexec/../logs/userlogs/job_201201031651_0008/attempt_201201031651_0008_m_000233_0/log.index (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(SecureIOUtils.java:102) at org.apache.hadoop.mapred.TaskLog.getAllLogsFileDetails(TaskLog.java:187) at org.apache.hadoop.mapred.TaskLog$Reader.(TaskLogServlet.java:81) at org.apache.hadoop.mapred.TaskLogServlet.doGet(TaskLogServlet.java:296) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:835) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) > Actually, all that is telling you is that the task failed and the > job-client couldn't display the logs. > > Can you check the JT web-ui and see why the task failed ? > > If you don't see anything there, you can try see the TaskTracker logs on > the node on which the task ran. > > Arun > > On Jan 31, 2012, at 3:21 AM, Marcin Cylke wrote: > > Hi > > > > I've upgraded my hadoop cluster to version 1.0.0. The upgrade process > > went relatively smoothly but it rendered the cluster inoperable due to > > errors in jobtrackers operation: > > > > # in job output > > Error reading task > > outputhttp://hadoop4:50060/tasklog?plaintext=true&attemptid=attempt_20120 > > 1311241_0003_m_000004_2&filter=stdout > > > > # in each of the jobtrackers' logs > > WARN org.apache.hadoop.mapred.TaskLog: Failed to retrieve stderr log for > > task: attempt_201201311241_0003_r_000000_1 > > java.io.FileNotFoundException: > > /usr/lib/hadoop-1.0.0/libexec/../logs/userlogs/job_201201311241_0003/atte > > mpt_201201311241_0003_r_000000_1/log.index (No such file or directory) > > > > at java.io.FileInputStream.open(Native Method) > > > > These errors seem related to this two problems: > > > > http://grokbase.com/t/hadoop.apache.org/mapreduce-user/2012/01/error-read > > ing-task-output-and-log-filenotfoundexceptions/03mjwctewcnxlgp2jkcrhvsgep > > 4e > > > > https://issues.apache.org/jira/browse/MAPREDUCE-2846 > > > > But I've looked into the source code and the fix from MAPREDUCE-2846 is > > there. Perhaps there is some other reason? > > > > Regards > > Marcin > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/
