[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ]
Ajith S edited comment on SPARK-27122 at 3/14/19 4:07 AM: ---------------------------------------------------------- The problem seems to be shading of jetty package. When we run test, the class path seems to be made from the classes folder(resource-managers/yarn/target/scala-2.12/classes) instead of jar (resource-managers/yarn/target/spark-yarn_2.12-3.0.0-SNAPSHOT.jar) . Check attachment. And here is the javap command which shows the difference between org.apache.spark.scheduler.cluster.YarnSchedulerBackend present in jar folder and classes folder !image-2019-03-14-09-35-23-046.png! was (Author: ajithshetty): The problem seems to be shading of jetty package. When we run test, the class path seems to be made from the classes folder(resource-managers/yarn/target/scala-2.12/classes) instead of jar (resource-managers/yarn/target/spark-yarn_2.12-3.0.0-SNAPSHOT.jar) And here is the javap command which shows the difference between org.apache.spark.scheduler.cluster.YarnSchedulerBackend present in jar folder and classes folder !image-2019-03-14-09-35-23-046.png! > YARN test failures in Java 9+ > ----------------------------- > > Key: SPARK-27122 > URL: https://issues.apache.org/jira/browse/SPARK-27122 > Project: Spark > Issue Type: Sub-task > Components: YARN > Affects Versions: 3.0.0 > Reporter: Sean Owen > Priority: Major > Attachments: image-2019-03-14-09-34-20-592.png, > image-2019-03-14-09-35-23-046.png > > > Currently on Java 11: > {code} > YarnSchedulerBackendSuite: > - RequestExecutors reflects node blacklist and is serializable > - Respect user filters when adding AM IP filter *** FAILED *** > java.lang.ClassCastException: > org.spark_project.jetty.servlet.ServletContextHandler cannot be cast to > org.eclipse.jetty.servlet.ServletContextHandler > at > scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237) > at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) > at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) > at scala.collection.TraversableLike.map(TraversableLike.scala:237) > at scala.collection.TraversableLike.map$(TraversableLike.scala:230) > at scala.collection.AbstractTraversable.map(Traversable.scala:108) > at > org.apache.spark.scheduler.cluster.YarnSchedulerBackend.$anonfun$addWebUIFilter$2(YarnSchedulerBackend.scala:183) > at > org.apache.spark.scheduler.cluster.YarnSchedulerBackend.$anonfun$addWebUIFilter$2$adapted(YarnSchedulerBackend.scala:174) > at scala.Option.foreach(Option.scala:274) > ... > {code} > This looks like a classpath issue, probably ultimately related to the same > classloader issues in https://issues.apache.org/jira/browse/SPARK-26839 -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org