XiaodongHuan created FLINK-40473:
------------------------------------
Summary: YARN application mode fails to locate flink-cdc-dist jar
when the jar name carries a Flink version suffix
Key: FLINK-40473
URL: https://issues.apache.org/jira/browse/FLINK-40473
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.6.0, cdc-3.7.0
Reporter: XiaodongHuan
Attachments: image-2026-08-26-15-45-49-268.png
h3. Problem
Submitting a YAML pipeline with \{{-t yarn-application}} using the official
flink-cdc-3.6.0-1.20 binary distribution always fails:
{code}
java.io.FileNotFoundException: Failed to fetch Flink CDC dist jar from path:
/path/to/flink-cdc-3.6.0-1.20/lib
at
org.apache.flink.cdc.composer.flink.deployment.YarnApplicationDeploymentExecutor.getFlinkCDCDistJarFromEnv(YarnApplicationDeploymentExecutor.java:151)
at
org.apache.flink.cdc.composer.flink.deployment.YarnApplicationDeploymentExecutor.deploy(YarnApplicationDeploymentExecutor.java:76)
at org.apache.flink.cdc.cli.CliExecutor.run(CliExecutor.java:75)
{code}
The \{{lib}} directory does contain the dist jar. The message is misleading: the
directory is fine, but no file in it matches the expected naming pattern.
h3. Root cause
The dist jar is located by name matching
(YarnApplicationDeploymentExecutor.java:62):
{code:java}
"^flink-cdc-dist-(\\d+(\\.\\d+)*)(-SNAPSHOT)?\\.jar$"
{code}
After the version this only accepts \{{-SNAPSHOT}} or \{{.jar}}. Since 3.6.0 the
distribution is released per Flink minor version, and the release script sets
{\{revision}} to \{{<release-version>-<flink-version>}}
(tools/releasing/deploy_staging_jars.sh:83), so the shipped jar is
{\{flink-cdc-dist-3.6.0-1.20.jar}}. The \{{-1.20}} segment never matches.
Not caught earlier because a plain \{{mvn package}} on master keeps
\{{revision}}
at \{{3.7-SNAPSHOT}}, which does match. The code is identical between
release-3.6.0 and master, so 3.7.0 will reproduce this once released.
h3. Workaround
Set \{{pipeline.jars}} explicitly to skip the lookup:
{code}
bash ./bin/flink-cdc.sh -t yarn-application \
-Dpipeline.jars=file:///path/to/lib/flink-cdc-dist-3.6.0-1.20.jar job.yaml
{code}
h3. Proposed fix
Allow an optional Flink version suffix, and include the pattern plus the actual
directory listing in the exception message:
{code:java}
"^flink-cdc-dist-(\\d+(\\.\\d+)*)(-\\d+(\\.\\d+)*)?(-SNAPSHOT)?\\.jar$"
{code}
!image-2026-08-26-15-45-49-268.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
