Team, Do we have any updates when spark 4.x version will release in order to address below issues related to > java.lang.NoClassDefFoundError: javax/servlet/Servlet
Thanks and Regards, Guru On 2023/10/05 17:19:51 Angshuman Bhattacharya wrote: > Thanks Ahmed. I am trying to bring this up with Spark DE community > > On Thu, Oct 5, 2023 at 12:32 PM Ahmed Albalawi < > ahmed.albal...@capitalone.com> wrote: > > > Hello team, > > > > We are in the process of upgrading one of our apps to Spring Boot 3.x > > while using Spark, and we have encountered an issue with Spark > > compatibility, specifically with Jakarta Servlet. Spring Boot 3.x uses > > Jakarta Servlet while Spark uses Javax Servlet. Can we get some guidance on > > how to upgrade to Spring Boot 3.x while continuing to use Spark. > > > > The specific error is listed below: > > > > java.lang.NoClassDefFoundError: javax/servlet/Servlet > > at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:239) > > at org.apache.spark.SparkContext.<init>(SparkContext.scala:503) > > at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2888) > > at org.apache.spark.SparkContext.getOrCreate(SparkContext.scala) > > > > The error comes up when we try to run a mvn clean install, and the issue is in our test cases. This issue happens specifically when we build our spark session. The line of code it traces down to is as follows: > > > > *session = SparkSession.builder().sparkContext(SparkContext.getOrCreate(sparkConf)).getOrCreate();* > > > > What we have tried: > > > > - We noticed according to this post < https://urldefense.com/v3/__https://stackoverflow.com/questions/75690210/apache-spark-with-spring-boot-failed-to-start-exception-factory-method-javasp__;!!EFVe01R3CjU!fdH40mQbStFzaUfVJ7JR-ZDD8DFiYsPseqp-yRu1BAFs8Zc3PZGVzaqF39BbL5vGOwm3mXNcTF1_O_mk_uj9FaBlEplW-4Oh7jceadvuqg$>, there are no compatible versions of spark using version 5 of the Jakarta Servlet API > > > > - We've tried < https://urldefense.com/v3/__https://stackoverflow.com/questions/76618374/spark-3-4-1-jakarta-servlet-6-0-0-compatibility-issue__;!!EFVe01R3CjU!fdH40mQbStFzaUfVJ7JR-ZDD8DFiYsPseqp-yRu1BAFs8Zc3PZGVzaqF39BbL5vGOwm3mXNcTF1_O_mk_uj9FaBlEplW-4Oh7jeV6Df7lQ$> using the maven shade plugin to use jakarta instead of javax, but are running into some other issues with this. > > - We've also looked at the following < https://urldefense.com/v3/__https://stackoverflow.com/questions/75350944/dependecy-conflict-apache-spark-and-spring-boot__;!!EFVe01R3CjU!fdH40mQbStFzaUfVJ7JR-ZDD8DFiYsPseqp-yRu1BAFs8Zc3PZGVzaqF39BbL5vGOwm3mXNcTF1_O_mk_uj9FaBlEplW-4Oh7jeiYCkybw$> to use jakarta 4.x with jersey 2.x and still have an issue with the servlet > > > > > > Please let us know if there are any solutions to this issue. Thanks! > > > > > > -- > > *Ahmed Albalawi* > > > > Senior Associate Software Engineer • EP2 Tech - CuRE > > > > 571-668-3911 • 1680 Capital One Dr. > > > > ______________________________________________________________________ > > > > The information contained in this e-mail may be confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer. > > > >
