[
https://issues.apache.org/jira/browse/HUDI-7513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-7513:
---------------------------------
Labels: pull-request-available (was: )
> Add jackson-module-scala to spark bundle
> ----------------------------------------
>
> Key: HUDI-7513
> URL: https://issues.apache.org/jira/browse/HUDI-7513
> Project: Apache Hudi
> Issue Type: Bug
> Affects Versions: 0.14.1
> Reporter: xi chaomin
> Priority: Major
> Labels: pull-request-available
>
> When I do spark stream read, get NoClassDefFoundError.
> {code:java}
> // code placeholder
> 24/03/18 14:15:37 ERROR MicroBatchExecution: Query [id =
> 2f7a09fb-03f2-4b21-bf72-79295daeda5a, runId =
> 5d627c7a-e469-4f91-bd84-c012e393cd60] terminated with error
> java.lang.NoClassDefFoundError:
> org/apache/hudi/com/fasterxml/jackson/module/scala/DefaultScalaModule$
> at
> org.apache.spark.sql.hudi.streaming.HoodieStreamSource.$anonfun$getBatch$1(HoodieStreamSource.scala:142)
> at scala.Option.map(Option.scala:230)
> at
> org.apache.spark.sql.hudi.streaming.HoodieStreamSource.getBatch(HoodieStreamSource.scala:142)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$populateStartOffsets$4(MicroBatchExecution.scala:354)
> at scala.collection.Iterator.foreach(Iterator.scala:943)
> at scala.collection.Iterator.foreach$(Iterator.scala:943)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
> at scala.collection.IterableLike.foreach(IterableLike.scala:74)
> at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
> at
> org.apache.spark.sql.execution.streaming.StreamProgress.foreach(StreamProgress.scala:27)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.populateStartOffsets(MicroBatchExecution.scala:351)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:224)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> at
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:375)
> at
> org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:373)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:68)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:219)
> at
> org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:67)
> at
> org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:213)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution.$anonfun$runStream$1(StreamExecution.scala:307)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:285)
> at
> org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:208)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hudi.com.fasterxml.jackson.module.scala.DefaultScalaModule$
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
