llphxd opened a new pull request, #4514: URL: https://github.com/apache/flink-cdc/pull/4514
## What is the purpose of this pull request? Fixes [FLINK-40473](https://issues.apache.org/jira/browse/FLINK-40473). Submitting a YAML pipeline with `-t yarn-application` using the official `flink-cdc-3.6.0-1.20` binary distribution always fails with: java.io.FileNotFoundException: Failed to fetch Flink CDC dist jar from path: /path/to/flink-cdc-3.6.0-1.20/lib The `lib` directory does contain the dist jar. `YarnApplicationDeploymentExecutor` locates it by matching file names against `^flink-cdc-dist-(\d+(\.\d+)*)(-SNAPSHOT)?\.jar$`, which only accepts `-SNAPSHOT` or `.jar` after the version number. Since 3.6.0 the binary distribution is released once per Flink minor version, and the release script sets `revision` to `<release-version>-<flink-version>`. The `-1.20` segment never matches the pattern. ## Brief change log - Match the dist jar by its `flink-cdc-dist-` prefix and `.jar` suffix instead of a version pattern, so the lookup no longer depends on how the version is formatted. - Extract the lib directory scan into `getFlinkCDCDistJar(Path)` so it can be unit tested without mutating the `FLINK_CDC_HOME` environment variable. - Add `YarnApplicationDeploymentExecutorTest`. The set of names accepted by the old pattern is a strict subset of what the new predicate accepts, so no previously working setup is affected. --- ## Verifying this change This change added tests and can be verified as follows: - Added unit tests in `flink-cdc-composer/src/test/java/org/apache/flink/cdc/composer/flink/deployment/YarnApplicationDeploymentExecutorTest.java`: - `testFindDistJarWhateverVersionItCarries` covers `flink-cdc-dist-3.5.0.jar`, `flink-cdc-dist-3.7-SNAPSHOT.jar`, `flink-cdc-dist-3.6.0-1.20.jar` and `flink-cdc-dist-3.7.0-2.0.jar`. The last two fail without this fix. A connector jar and a directory named like a dist jar are placed in the same lib directory to verify neither is picked up. - `testDistJarNotFound` verifies that a lib directory holding only connector jars still raises `FileNotFoundException`, i.e. the prefix match does not over-match. The failure happens in the first statement of `deploy()`, before any interaction with YARN, so the unit test covers the full code path that was broken. ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? <!-- If generative AI tooling has been used in the process of authoring this PR, please change the checkbox below to `[X]` followed by the name of the tool, and uncomment the "Generated-by" line. See the ASF Generative Tooling Guidance for details: https://www.apache.org/legal/generative-tooling.html You are responsible for the quality and correctness of every change in this PR regardless of the tooling used. Low-effort AI-generated PRs will be closed. --> - [ ] Yes (please specify the tool below) <!-- Generated-by: [Tool Name and Version] --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
