[ https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sagar Sumit updated HUDI-4549: ------------------------------ Story Points: 2 (was: 3) > hive sync bundle causes class loader issue > ------------------------------------------ > > Key: HUDI-4549 > URL: https://issues.apache.org/jira/browse/HUDI-4549 > Project: Apache Hudi > Issue Type: Bug > Components: dependencies > Reporter: Raymond Xu > Assignee: Sagar Sumit > Priority: Blocker > Labels: pull-request-available > Fix For: 0.12.1 > > > A weird classpath issue i found: when testing deltastreamer using > hudi-utilities-slim-bundle, if i put --jars > hudi-hive-sync-bundle.jar,hudi-spark-bundle.jar then i’ll get this error when > writing > {code:java} > Caused by: java.lang.NoSuchMethodError: > org.apache.hudi.avro.MercifulJsonConverter.convert(Ljava/lang/String;Lorg/apache/avro/Schema;)Lorg/apache/avro/generic/GenericRecord; > at > org.apache.hudi.utilities.sources.helpers.AvroConvertor.fromJson(AvroConvertor.java:86) > at > org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070) > {code} > if i put the spark bundle before the hive sync bundle, then no issue. Without > hive-sync-bundle, also no issue. So hive-sync-bundle somehow messes up with > classpath? not sure why it reports a hudi-common API not found… caused by > shading avro? > the same behavior i observed with aws-bundle, which makes sense, as it’s a > superset of hive-sync-bundle -- This message was sent by Atlassian Jira (v8.20.10#820010)