[
https://issues.apache.org/jira/browse/SPARK-45897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arnaud Nauwynck updated SPARK-45897:
------------------------------------
Attachment: test-springboot-spark.zip
> java.lang.NoClassDefFoundError: javax/servlet/Servlet incompatibilities
> upgrading springboot-dependencies 2.7 to 3.* (package javax.* -> jakarta.*)
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-45897
> URL: https://issues.apache.org/jira/browse/SPARK-45897
> Project: Spark
> Issue Type: Dependency upgrade
> Components: Web UI
> Affects Versions: 3.2.4, 3.3.3, 3.4.0, 3.4.1, 3.5.0
> Reporter: Arnaud Nauwynck
> Priority: Major
> Attachments: test-springboot-spark.zip
>
>
> updating springboot-dependencies 2.7 to 3.1 breaks spark with springboot
> compatibilities.
> {noformat}
> <dependencyManagement>
> <dependencies>
> <dependency>
> <groupId>org.springframework.boot</groupId>
> <artifactId>spring-boot-dependencies</artifactId>
> <version>2.7.14</version> <!-- OK 2.7.* works fine, but 3.*
> ERROR -->
> <type>pom</type>
> <scope>import</scope>
> </dependency>
> </dependencies>
> </dependencyManagement>
> {noformat}
> Internally, everything compile ok, but fail at runtime while creating the
> SparkSession, with SparkUI:
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: javax/servlet/Servlet
> at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:239)
> ~[spark-core_2.13-3.5.0.jar:3.5.0]
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:503)
> ~[spark-core_2.13-3.5.0.jar:3.5.0]
> at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2888)
> ~[spark-core_2.13-3.5.0.jar:3.5.0]
> at
> org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:1099)
> ~[spark-sql_2.13-3.5.0.jar:3.5.0]
> at scala.Option.getOrElse(Option.scala:201)
> ~[scala-library-2.13.8.jar:na]
> at
> org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:1093)
> ~[spark-sql_2.13-3.5.0.jar:3.5.0]
> ....
> Caused by: java.lang.ClassNotFoundException: javax.servlet.Servlet
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
> ~[na:na]
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
> ~[na:na]
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> ~[na:na]
> ... 49 common frames omitted
> {noformat}
> Root cause: major change with springboot 3: 'javax.servlet.Servlet' class is
> NO more defined in package 'javax.servlet.', but migrated to
> 'jakarta.servlet.'
> The compiled dependency from spark-core to javax-servlet-api is replaced by
> maven dependencyManagement
> It "might be" possible to get rid of maven dependencyManagement or to
> override several artifact versions, but it is NOT PRACTICAL.
> Here is a version that work, much more complex with springboot3 than
> springboot2.
> It requires few of forced versions, and exclusions in maven.
>
> {noformat}
> <properties>
> <spark.version>3.5.0</spark.version>
> <scala.version>2.13</scala.version>
> </properties>
> <dependencyManagement>
> <dependencies>
> <dependency>
> <groupId>org.springframework.boot</groupId>
> <artifactId>spring-boot-dependencies</artifactId>
> <version>3.1.2</version>
> <type>pom</type>
> <scope>import</scope>
> </dependency>
> <dependency>
> <groupId>org.glassfish.jersey.containers</groupId>
> <artifactId>jersey-container-servlet-core</artifactId>
> <version>2.41</version>
> </dependency>
> <dependency>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-api</artifactId>
> <version>2.0.9</version>
> </dependency>
> </dependencies>
> </dependencyManagement>
> <dependencies>
> <dependency>
> <groupId>org.springframework.boot</groupId>
> <artifactId>spring-boot-starter-web</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_${scala.version}</artifactId>
> <version>${spark.version}</version>
> <exclusions>
> <exclusion>
> <groupId>org.apache.logging.log4j</groupId>
> <artifactId>log4j-slf4j2-impl</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> <dependency>
> <groupId>jakarta.servlet</groupId>
> <artifactId>jakarta.servlet-api</artifactId>
> <version>4.0.4</version>
> </dependency>
> </dependencies>
> {noformat}
> there are several examples showing errors and fixed for servlet, log4j,
> glassfish incompatibilities here:
> https://github.com/Arnaud-Nauwynck/test-snippets/tree/master/test-springboot-spark
> A possible long term solution is to migrate to servlet in package
> "jakarta.servlet." instead of "javax.servlet." as springboot already did, so
> there would "not (?)" be incompatibilities between very old and much more
> recent jars.
> Any roadmap for this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
