[
https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raymond Xu updated HUDI-4549:
-----------------------------
Reviewers: Raymond Xu
> hive sync bundle causes class loader issue
> ------------------------------------------
>
> Key: HUDI-4549
> URL: https://issues.apache.org/jira/browse/HUDI-4549
> Project: Apache Hudi
> Issue Type: Bug
> Components: dependencies
> Reporter: Raymond Xu
> Assignee: Sagar Sumit
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.12.1
>
>
> A weird classpath issue i found: when testing deltastreamer using
> hudi-utilities-slim-bundle, if i put --jars
> hudi-hive-sync-bundle.jar,hudi-spark-bundle.jar then i’ll get this error when
> writing
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hudi.avro.MercifulJsonConverter.convert(Ljava/lang/String;Lorg/apache/avro/Schema;)Lorg/apache/avro/generic/GenericRecord;
> at
> org.apache.hudi.utilities.sources.helpers.AvroConvertor.fromJson(AvroConvertor.java:86)
> at
> org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)
> {code}
> if i put the spark bundle before the hive sync bundle, then no issue. Without
> hive-sync-bundle, also no issue. So hive-sync-bundle somehow messes up with
> classpath? not sure why it reports a hudi-common API not found… caused by
> shading avro?
> the same behavior i observed with aws-bundle, which makes sense, as it’s a
> superset of hive-sync-bundle
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
