[
https://issues.apache.org/jira/browse/FLINK-39429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18073867#comment-18073867
]
peiyu commented on FLINK-39429:
-------------------------------
I referenced the Fluss project, which is compatible with both Flink 1.x and
Flink 2.x versions. For Guava, it implements a separate, specific shade package.
see: https://github.com/apache/fluss-shaded
The flink-cdc project uses flink-shaded, but the versions of flink-shaded are
different in flink 1.x and flink 2.x, and the package names are also different.
Can we also provide a specific shaded package to solve this problem, similar to
flink-cdc-shaded?
> NoClassDefFoundError for flink-shaded-guava31 in
> IncrementalSourceStreamFetcher (Flink 2.2 / CDC 3.6.0)
> -------------------------------------------------------------------------------------------------------
>
> Key: FLINK-39429
> URL: https://issues.apache.org/jira/browse/FLINK-39429
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Reporter: João Boto
> Priority: Major
>
> When running a Flink CDC pipeline on Flink 2.2 using Flink CDC 3.6.0, the job
> crashes with a java.lang.NoClassDefFoundError:
> org/apache/flink/shaded/guava31/com/google/common/util/concurrent/ThreadFactoryBuilder.
> This occurs specifically when scan.incremental.snapshot.enabled is set to
> true, as the IncrementalSourceStreamFetcher and IncrementalSourceScanFetcher
> are strictly looking for the guava31 shade, which seems to be missing or
> incompatible with the default Flink 2.2 classpath environment.
> *Environment:*
> Flink version: 2.2
> Flink CDC version: 3.6.0
> Database / Connector: PostgreSQL
> Java version: 21
> Deployment mode: Kubernetes
> *Steps to reproduce*:
> Configure a CDC source using Flink CDC 3.6.0.
> Ensure incremental snapshots are enabled:
> this.debeziumProps.setProperty("scan.incremental.snapshot.enabled", "true");
> Deploy the job to a Flink 2.2 cluster.
> The job fails immediately during the fetcher initialization.
> *Expected behavior:*
> The Flink CDC connector should successfully initialize the
> IncrementalSourceStreamFetcher and perform lock-free incremental snapshots
> without missing shaded Guava dependencies out of the box. If a specific
> shaded Guava jar is required for Flink 2.2 compatibility, it should be
> bundled or explicitly documented.
> *Actual behavior / Stack trace:*
> {code:java}
> java.lang.RuntimeException: One or more fetchers have encountered exception
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:307)
> at
> org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:269)
> at
> org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:181)
> ...
> at java.base/java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/flink/shaded/guava31/com/google/common/util/concurrent/ThreadFactoryBuilder
> at
> org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceStreamFetcher.<init>(IncrementalSourceStreamFetcher.java:77)
> at
> org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.getStreamFetcher(IncrementalSourceSplitReader.java:272)
> at
> org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.submitStreamSplit(IncrementalSourceSplitReader.java:254)
> at
> org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.pollSplitRecords(IncrementalSourceSplitReader.java:153)
> at
> org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.fetch(IncrementalSourceSplitReader.java:98)
> at
> org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58)
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:193)
> ...
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.shaded.guava31.com.google.common.util.concurrent.ThreadFactoryBuilder
> ... 13 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
