Sangjin Lee created MAPREDUCE-5751: -------------------------------------- Summary: MR app master fails to start in some cases if mapreduce.job.classloader is true Key: MAPREDUCE-5751 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5751 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Sangjin Lee Assignee: Sangjin Lee
If mapreduce.job.classloader is set to true, and the MR client includes a jetty jar in its libjars or job jar, the MR app master fails to start. A typical stack trace we get is as follows: {noformat} java.lang.ClassCastException: org.mortbay.jetty.webapp.WebInfConfiguration cannot be cast to org.mortbay.jetty.webapp.Configuration at org.mortbay.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:890) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:462) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) at org.mortbay.jetty.Server.doStart(Server.java:224) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.apache.hadoop.http.HttpServer.start(HttpServer.java:676) at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:208) at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:151) at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1040) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1307) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1303) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1259) {noformat} This happens because as part of the MR app master start the jetty classes are loaded normally through the app classloader, but WebAppContext tries to load the specific Configuration class via the thread context classloader (which had been set to the user job classloader). -- This message was sent by Atlassian JIRA (v6.1.5#6160)