[jira] [Commented] (SPARK-43592) NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372

[Shivam Kasat (Jira)]

[ 
https://issues.apache.org/jira/browse/SPARK-43592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724773#comment-17724773
 ] 

Shivam Kasat commented on SPARK-43592:
--

When I use JDK11 image in spark dockerfile and maven target and source are kept 
1.8 -> It works fine
When I use JDK8 image in spark dockerfile and maven target and source jdk 1.8 
-> it throws error

> NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372
> ---
>
> Key: SPARK-43592
> URL: https://issues.apache.org/jira/browse/SPARK-43592
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 3.4.0
> Environment: JDK: JDK8u362, JDK8u372
> Kubernetes
> Spark 3.4
>Reporter: Shivam Kasat
>Priority: Critical
>  Labels: JDK1.8, java, jdk11
>
> My project was on spark 3.3 with JDK8u362 and I tried updating it to spark 
> 3.4, Official documentation of spark 3.4 says it works with JDK8u362 and 
> above but when I tried upgrading docker base image of spark to JDK8u362 and 
> JDK8u372 it is failing at runtime with below error, For JDK8u362 it throws 
> error for Java.nio.CharBuffer.position method and for JDK8u372 it throws 
> error for java.nio.ByteBuffer.flip method. But when I run with JDK11 image in 
> spark Docker file it works fine. Am I missing anything or how to fix this 
> issue as I want to run it with JDK8.
> {code:java}
> ava.lang.NoSuchMethodError: 
> java.nio.CharBuffer.position(I)Ljava/nio/CharBuffer;
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.unescapeSQLString(ParserUtils.scala:220)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.string(ParserUtils.scala:95)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$createString$2(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach$(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map$(TraversableLike.scala:279) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractTraversable.map(Traversable.scala:108) 
> ~[scala-library-2.12.17.jar:?]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.createString(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitStringLiteral$1(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:160)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:58)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$StringLiteralContext.accept(SqlBaseParser.java:19511)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitConstantDefault(SqlBaseParserBaseVisitor.java:1735)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$ConstantDefaultContext.accept(SqlBaseParser.java:18373)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitValueExpressionDefault(SqlBaseParserBaseVisitor.java:1567)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> 

[jira] [Commented] (SPARK-43592) NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372

[Shivam Kasat (Jira)]

[ 
https://issues.apache.org/jira/browse/SPARK-43592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724768#comment-17724768
 ] 

Shivam Kasat commented on SPARK-43592:
--

yes! when i tried building with JDK11. it works fine

> NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372
> ---
>
> Key: SPARK-43592
> URL: https://issues.apache.org/jira/browse/SPARK-43592
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 3.4.0
> Environment: JDK: JDK8u362, JDK8u372
> Kubernetes
> Spark 3.4
>Reporter: Shivam Kasat
>Priority: Critical
>  Labels: JDK1.8, java, jdk11
>
> My project was on spark 3.3 with JDK8u362 and I tried updating it to spark 
> 3.4, Official documentation of spark 3.4 says it works with JDK8u362 and 
> above but when I tried upgrading docker base image of spark to JDK8u362 and 
> JDK8u372 it is failing at runtime with below error, For JDK8u362 it throws 
> error for Java.nio.CharBuffer.position method and for JDK8u372 it throws 
> error for java.nio.ByteBuffer.flip method. But when I run with JDK11 image in 
> spark Docker file it works fine. Am I missing anything or how to fix this 
> issue as I want to run it with JDK8.
> {code:java}
> ava.lang.NoSuchMethodError: 
> java.nio.CharBuffer.position(I)Ljava/nio/CharBuffer;
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.unescapeSQLString(ParserUtils.scala:220)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.string(ParserUtils.scala:95)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$createString$2(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach$(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map$(TraversableLike.scala:279) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractTraversable.map(Traversable.scala:108) 
> ~[scala-library-2.12.17.jar:?]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.createString(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitStringLiteral$1(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:160)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:58)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$StringLiteralContext.accept(SqlBaseParser.java:19511)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitConstantDefault(SqlBaseParserBaseVisitor.java:1735)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$ConstantDefaultContext.accept(SqlBaseParser.java:18373)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitValueExpressionDefault(SqlBaseParserBaseVisitor.java:1567)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$ValueExpressionDefaultContext.accept(SqlBaseParser.java:17491)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> 

[jira] [Commented] (SPARK-43592) NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372

[Shivam Kasat (Jira)]

[ 
https://issues.apache.org/jira/browse/SPARK-43592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724763#comment-17724763
 ] 

Shivam Kasat commented on SPARK-43592:
--

by maven compiler version is fine & [Spark 
Documentation|https://spark.apache.org/docs/latest/] says that spark 3.4 works 
on JDK8u362 and above.

> NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372
> ---
>
> Key: SPARK-43592
> URL: https://issues.apache.org/jira/browse/SPARK-43592
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 3.4.0
> Environment: JDK: JDK8u362, JDK8u372
> Kubernetes
> Spark 3.4
>Reporter: Shivam Kasat
>Priority: Critical
>  Labels: JDK1.8, java, jdk11
>
> My project was on spark 3.3 with JDK8u362 and I tried updating it to spark 
> 3.4, Official documentation of spark 3.4 says it works with JDK8u362 and 
> above but when I tried upgrading docker base image of spark to JDK8u362 and 
> JDK8u372 it is failing at runtime with below error, For JDK8u362 it throws 
> error for Java.nio.CharBuffer.position method and for JDK8u372 it throws 
> error for java.nio.ByteBuffer.flip method. But when I run with JDK11 image in 
> spark Docker file it works fine. Am I missing anything or how to fix this 
> issue as I want to run it with JDK8.
> {code:java}
> ava.lang.NoSuchMethodError: 
> java.nio.CharBuffer.position(I)Ljava/nio/CharBuffer;
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.unescapeSQLString(ParserUtils.scala:220)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.string(ParserUtils.scala:95)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$createString$2(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach$(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map$(TraversableLike.scala:279) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractTraversable.map(Traversable.scala:108) 
> ~[scala-library-2.12.17.jar:?]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.createString(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitStringLiteral$1(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:160)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:58)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$StringLiteralContext.accept(SqlBaseParser.java:19511)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitConstantDefault(SqlBaseParserBaseVisitor.java:1735)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$ConstantDefaultContext.accept(SqlBaseParser.java:18373)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitValueExpressionDefault(SqlBaseParserBaseVisitor.java:1567)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$ValueExpressionDefaultContext.accept(SqlBaseParser.java:17491)
>  

[jira] [Commented] (SPARK-43592) NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372

[Shivam Kasat (Jira)]

[ 
https://issues.apache.org/jira/browse/SPARK-43592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724256#comment-17724256
 ] 

Shivam Kasat commented on SPARK-43592:
--

Posted the same on stackoverflow: 
https://stackoverflow.com/questions/76286857/nosuchmethoderror-in-spark-3-4-with-jdk8u362-jdk8u372

> NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372
> ---
>
> Key: SPARK-43592
> URL: https://issues.apache.org/jira/browse/SPARK-43592
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Core
>Affects Versions: 3.4.0
> Environment: JDK: JDK8u362, JDK8u372
> Kubernetes
> Spark 3.4
>Reporter: Shivam Kasat
>Priority: Critical
>  Labels: JDK1.8, java, jdk11
>
> My project was on spark 3.3 with JDK8u362 and I tried updating it to spark 
> 3.4, Official documentation of spark 3.4 says it works with JDK8u362 and 
> above but when I tried upgrading docker base image of spark to JDK8u362 and 
> JDK8u372 it is failing at runtime with below error, For JDK8u362 it throws 
> error for Java.nio.CharBuffer.position method and for JDK8u372 it throws 
> error for java.nio.ByteBuffer.flip method. But when I run with JDK11 image in 
> spark Docker file it works fine. Am I missing anything or how to fix this 
> issue as I want to run it with JDK8.
> {code:java}
> ava.lang.NoSuchMethodError: 
> java.nio.CharBuffer.position(I)Ljava/nio/CharBuffer;
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.unescapeSQLString(ParserUtils.scala:220)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.string(ParserUtils.scala:95)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$createString$2(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.Iterator.foreach$(Iterator.scala:943) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map(TraversableLike.scala:286) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.TraversableLike.map$(TraversableLike.scala:279) 
> ~[scala-library-2.12.17.jar:?]
> at scala.collection.AbstractTraversable.map(Traversable.scala:108) 
> ~[scala-library-2.12.17.jar:?]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.createString(AstBuilder.scala:2632)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitStringLiteral$1(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:160)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:2618)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:58)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$StringLiteralContext.accept(SqlBaseParser.java:19511)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitConstantDefault(SqlBaseParserBaseVisitor.java:1735)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$ConstantDefaultContext.accept(SqlBaseParser.java:18373)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitValueExpressionDefault(SqlBaseParserBaseVisitor.java:1567)
>  ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
> at 
> org.apache.spark.sql.catalyst.parser.SqlBaseParser$ValueExpressionDefaultContext.accept(SqlBaseParser.java:17491)
>  

[jira] [Created] (SPARK-43592) NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372

[Shivam Kasat (Jira)]

Shivam Kasat created SPARK-43592:


 Summary: NoSuchMethodError in Spark 3.4 with JDK8u362 & JDK8u372
 Key: SPARK-43592
 URL: https://issues.apache.org/jira/browse/SPARK-43592
 Project: Spark
  Issue Type: Bug
  Components: Spark Core
Affects Versions: 3.4.0
 Environment: JDK: JDK8u362, JDK8u372
Kubernetes
Spark 3.4
Reporter: Shivam Kasat


My project was on spark 3.3 with JDK8u362 and I tried updating it to spark 3.4, 
Official documentation of spark 3.4 says it works with JDK8u362 and above but 
when I tried upgrading docker base image of spark to JDK8u362 and JDK8u372 it 
is failing at runtime with below error, For JDK8u362 it throws error for 
Java.nio.CharBuffer.position method and for JDK8u372 it throws error for 
java.nio.ByteBuffer.flip method. But when I run with JDK11 image in spark 
Docker file it works fine. Am I missing anything or how to fix this issue as I 
want to run it with JDK8.
{code:java}
ava.lang.NoSuchMethodError: java.nio.CharBuffer.position(I)Ljava/nio/CharBuffer;
at 
org.apache.spark.sql.catalyst.parser.ParserUtils$.unescapeSQLString(ParserUtils.scala:220)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.ParserUtils$.string(ParserUtils.scala:95) 
~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$createString$2(AstBuilder.scala:2632)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286) 
~[scala-library-2.12.17.jar:?]
at scala.collection.Iterator.foreach(Iterator.scala:943) 
~[scala-library-2.12.17.jar:?]
at scala.collection.Iterator.foreach$(Iterator.scala:943) 
~[scala-library-2.12.17.jar:?]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) 
~[scala-library-2.12.17.jar:?]
at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
~[scala-library-2.12.17.jar:?]
at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
~[scala-library-2.12.17.jar:?]
at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
~[scala-library-2.12.17.jar:?]
at scala.collection.TraversableLike.map(TraversableLike.scala:286) 
~[scala-library-2.12.17.jar:?]
at scala.collection.TraversableLike.map$(TraversableLike.scala:279) 
~[scala-library-2.12.17.jar:?]
at scala.collection.AbstractTraversable.map(Traversable.scala:108) 
~[scala-library-2.12.17.jar:?]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.createString(AstBuilder.scala:2632)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitStringLiteral$1(AstBuilder.scala:2618)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:160)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:2618)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.visitStringLiteral(AstBuilder.scala:58)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.SqlBaseParser$StringLiteralContext.accept(SqlBaseParser.java:19511)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitConstantDefault(SqlBaseParserBaseVisitor.java:1735)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.SqlBaseParser$ConstantDefaultContext.accept(SqlBaseParser.java:18373)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:73)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.SqlBaseParserBaseVisitor.visitValueExpressionDefault(SqlBaseParserBaseVisitor.java:1567)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.SqlBaseParser$ValueExpressionDefaultContext.accept(SqlBaseParser.java:17491)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.typedVisit(AstBuilder.scala:63) 
~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.expression(AstBuilder.scala:1630)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$withPredicate$1(AstBuilder.scala:1870)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at 
org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:160)
 ~[spark-catalyst_2.12-3.4.0.jar:3.4.0]
at