Admaing opened a new pull request, #18667:
URL: https://github.com/apache/dolphinscheduler/pull/18667
<!--Thanks very much for contributing to Apache DolphinScheduler, we are
happy that you want to help us improve DolphinScheduler! -->
## Was this PR generated or assisted by AI?
<!--(Please answer YES or NO. If YES, please specify which parts were
generated or assisted by AI)-->
## Purpose of the pull request
Fix #16789. Stopping a Flink task killed only the client process tree, so
the Flink job could keep running afterwards:
- batch `FlinkTask` never issued `flink cancel`;
- `FlinkStreamTask` did, but with an unresolved `${FLINK_HOME}` — the cancel
command is executed by `ProcessBuilder`, which does not expand shell variables
— so it always failed with `Cannot run program "${FLINK_HOME}/bin/flink"`.
## Brief change log
- resolve `${FLINK_HOME}` for the cancel / savepoint commands (task params
first, then the `FLINK_HOME` environment variable)
- move the cancel logic from `FlinkStreamTask` into `FlinkTask`: cancel by
the JobID printed by `flink run`, and fall back to the previous behaviour (kill
the process tree, cancel the YARN/K8s application) when the task was submitted
to a resource manager, when there is no JobID, or when the CLI fails
- wait for the cancel command to finish instead of fire-and-forget, so a
failure can be detected
- Note: this changes the stop path of batch Flink tasks — they now attempt
`flink cancel` before the process kill
## Verify this pull request
This change added tests and can be verified as follows:
- `FlinkArgsUtilsTest`: `${FLINK_HOME}` is resolved in the cancel /
savepoint command
- `FlinkTaskTest`: cancel via the CLI using the JobID from the task log;
fallback when the CLI fails; fallback when no JobID is found; no CLI call for
YARN/K8s submissions
- Manually verified with the real classes in a Linux container: a batch task
(`deployMode=cluster`, detached submit) and a stream task both execute `flink
cancel <jobId>` now, where before the batch task never issued it and the stream
task always threw
## Pull Request Notice
[Pull Request
Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
