[
https://issues.apache.org/jira/browse/MAPREDUCE-5600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820550#comment-13820550
]
Kousuke Saruta commented on MAPREDUCE-5600:
-------------------------------------------
I think branch-1 has the same issue.
When TaskTracker#getTasksFromRunningJobs is called, we'll get a iterator of
"runningJobs"
{code}
/**
* Get the list of tasks from running jobs on this task tracker.
* @return a copy of the list of TaskStatus objects
*/
synchronized List<TaskStatus> getTasksFromRunningJobs() {
List<TaskStatus> result = new ArrayList<TaskStatus>(tasks.size());
for (Map.Entry <JobID, RunningJob> item : runningJobs.entrySet()) {
...
}
}
return result;
}
{code}
On the other hand, TaskTracker#addTaskToJob can be called during iteration.
addTaskToJob method modifies "runningJob" so it can cause
ConcurrentModificationException.
{code}
private RunningJob addTaskToJob(JobID jobId,
TaskInProgress tip) {
synchronized (runningJobs) {
...
runningJobs.put(jobId, rJob);
...
}
}
{code}
When we call getTasksFromRunningJobs, we get a synchronized monitor for the
instance of TaskTracker, but, when we call addTaskToJob,
we get a synchronized monitor for "runningJob". So, we may need to modify
TaskTracker to get an appropriate monitor when calling getTasksFromRunningJobs.
> ConcurrentModificationException on /tasktracker.jsp
> ---------------------------------------------------
>
> Key: MAPREDUCE-5600
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5600
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Environment: Found this in the MR1 bundled with
> hadoop-2.0.0-mr1-cdh4.1.3 (which I think is based on some 0.20 version).
> Reporter: Benoit Sigoure
>
> If you request {{/tasktracker.jsp}} frequently on a TaskTracker that's busy,
> every once in a while you'll get this:
> {code}
> 2013-10-29 13:25:55,524 ERROR org.mortbay.log: /tasktracker.jsp
> java.util.ConcurrentModificationException
> at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1117)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1153)
> at java.util.TreeMap$EntryIterator.next(TreeMap.java:1148)
> at
> org.apache.hadoop.mapred.TaskTracker.getTasksFromRunningJobs(TaskTracker.java:3991)
> at
> org.apache.hadoop.mapred.tasktracker_jsp._jspService(tasktracker_jsp.java:98)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1056)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
