[ https://issues.apache.org/jira/browse/MAPREDUCE-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897994#comment-13897994 ]
Sangjin Lee commented on MAPREDUCE-5751: ---------------------------------------- Yes, I think org.mortbay. should be added to the system classes by default. In fact, I think I'll do both changes: (1) adding org.mortbay. to the system classes, and (2) delay setting the classloaders until just before startJobs(). I think (2) still has value in case any other library that comes between these points happens to use the thread context classloader. I'll submit a patch shortly. > MR app master fails to start in some cases if mapreduce.job.classloader is > true > ------------------------------------------------------------------------------- > > Key: MAPREDUCE-5751 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5751 > Project: Hadoop Map/Reduce > Issue Type: Bug > Affects Versions: 2.2.0 > Reporter: Sangjin Lee > Assignee: Sangjin Lee > > If mapreduce.job.classloader is set to true, and the MR client includes a > jetty jar in its libjars or job jar, the MR app master fails to start. A > typical stack trace we get is as follows: > {noformat} > java.lang.ClassCastException: org.mortbay.jetty.webapp.WebInfConfiguration > cannot be cast to org.mortbay.jetty.webapp.Configuration > at > org.mortbay.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:890) > at > org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:462) > at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) > at > org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) > at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at > org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) > at org.mortbay.jetty.Server.doStart(Server.java:224) > at > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) > at org.apache.hadoop.http.HttpServer.start(HttpServer.java:676) > at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:208) > at > org.apache.hadoop.mapreduce.v2.app.client.MRClientService.start(MRClientService.java:151) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:1040) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1307) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1303) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1259) > {noformat} > This happens because as part of the MR app master start the jetty classes are > loaded normally through the app classloader, but WebAppContext tries to load > the specific Configuration class via the thread context classloader (which > had been set to the user job classloader). -- This message was sent by Atlassian JIRA (v6.1.5#6160)