DanielLeens opened a new pull request, #12143:
URL: https://github.com/apache/seatunnel/pull/12143
### Purpose of this pull request
Adds an engine E2E regression test for the second failure site closed by
#11812 (issue #11808): a `TaskGroup` whose tasks all deserialize correctly but
whose deployment then fails inside `TaskExecutionService#deployLocalTask`
before the `TaskGroupContext` is published.
### Background: two failure sites, one fix
`TaskExecutionService#deployTask` acquires one classloader reference per
task while deserializing the group. Ownership is only transferred to the
worker's regular bookkeeping when `deployLocalTask` publishes the
`TaskGroupContext` into `executionContexts`. #11812 closed two distinct places
where a failed deployment used to drop those references:
| | Site A | Site B |
|---|---|---|
| Where | deserialization inside `deployTask`'s per-task loop, before the
`TaskGroup` exists | inside `deployLocalTask`, after the `TaskGroup` exists and
before `executionContexts.put(...)` |
| `deployTask` result | throws, returns `TaskDeployState.failed(...)`
synchronously | returns `TaskDeployState.success()` synchronously; the failure
only surfaces asynchronously via `notifyTaskStatusToMaster` |
| Released by | `deployTask`'s outer `catch (Throwable)` | the
`onFailureBeforeContextPublished` callback wired by `deployTask` |
| Existing coverage | `TaskDeploymentClassLoaderLeakIT` (#12028), end to end
| only
`TaskExecutionServiceTest#testDeployTaskHandlesFailureBeforeContextPublication`,
which stubs `notifyTaskStatusToMaster` with Mockito |
Site B had no coverage through the real path: nothing ran the asynchronous
`FAILED` report against a real master.
### What the new test does
`TaskDeploymentPrePublicationClassLoaderLeakIT` sits next to
`TaskDeploymentClassLoaderLeakIT` and uses the same `SeaTunnelServerStarter`
master + worker harness, with classloader cache mode disabled so eviction is a
second leak signal next to the reference count.
1. **Control deployment**: a valid single-task group whose task blocks on a
gate. While it runs, the published context owns the acquired classloader
(reference count 1, live classloader count baseline + 1). Opening the gate lets
the group finish through `taskDone`: reference count 0, classloader evicted,
context moved to the finished contexts, and `FINISHED` reported to the master
through the real `NotifyTaskStatusOperation`.
2. **Five Site B attempts**: a two-task group where both tasks deserialize
(two classloaders acquired) and the second task throws from
`Task#setTaskExecutionContext`, the only per-task hook `deployLocalTask`
invokes between `TaskGroup` construction and publication. Per attempt the test
asserts:
- `deployTask` returned success synchronously (the Site B signature);
- no active or finished context was ever published for the location;
- both references are already 0 and both classloaders evicted when
`deployTask` returns, and the live classloader count is back to baseline (no
accumulation across attempts);
- asynchronously (bounded Awaitility, 60s): the worker completed the
group with `FAILED` carrying the injected message, the master's
`CoordinatorService` received `FAILED` for that exact location, and the
worker's retry loop terminated on the master's `JobNotFoundException` (the
production branch for a task group whose job the master does not track).
The asynchronous report is observed at both ends by a Log4j2 appender
attached to the worker's `TaskExecutionService` logger and the master's
`CoordinatorService` logger (both nodes share the test JVM's Log4j2 context);
nothing is mocked or stubbed.
Why `setTaskExecutionContext` and not `init()`: `Task#init()` is only called
by `BlockingWorker#run` / `CooperativeTaskWorker` after publication, so an
`init()` failure takes the ordinary post-publication path and never reaches
Site B. It is the same hook #11812's own unit test used.
### Does this PR introduce _any_ user-facing change?
No. Test-only change: no `src/main` and no `pom.xml` changes.
### How was this patch tested?
Test-only change. Verified locally on JDK 8 (`jdk1.8.0_171`) in a fresh
worktree on `apache/dev` (`10206bc7aa6`), after a full dependency build:
```
./mvnw install -pl
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base -am -DskipTests
-Dskip.ui=true -nsu -Dmaven.gitcommitid.skip=true -T 3C
```
(BUILD SUCCESS, 67 reactor modules, 8:57; `seatunnel-engine-ui` skipped via
`-Dskip.ui=true` since this change does not touch it.)
Three consecutive runs of
```
./mvnw verify -pl
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base -DskipUT=true
-DskipIT=false -Dit.test=TaskDeploymentPrePublicationClassLoaderLeakIT
```
| Run | Result | Test time |
|---|---|---|
| 1 | Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 | 12.1 s |
| 2 | Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 | 11.6 s |
| 3 | Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 | 12.2 s |
What the engine logged in run 1 (worker on `:5803`, master on `:5802`) for
one of the five Site B attempts, in order:
```
TaskExecutionService - deploying TaskGroup
TaskGroupLocation{jobId=1788670228489002, pipelineId=1, taskGroupId=1} init
success
TaskExecutionService - Task TaskGroupLocation{jobId=1788670228489002,
pipelineId=1, taskGroupId=1} complete with error
java.lang.IllegalStateException: simulated failure before TaskGroupContext
publication, attempt 0
TaskExecutionService - Task TaskGroupLocation{jobId=1788670228489002,
pipelineId=1, taskGroupId=1} complete with state FAILED
CoordinatorService - Received task end from execution
TaskGroupLocation{jobId=1788670228489002, pipelineId=1, taskGroupId=1}, state
FAILED
TaskExecutionService - send notify task status failed because can't find job
... JobNotFoundException: Job 1788670228489002 not running
```
All five attempts passed `taskGroup.init()` and failed at
`setTaskExecutionContext`; only the control group logged the post-publication
`deploying TaskGroup ... success` and was then reported as `FINISHED` at both
ends. Six `can't find job` warnings (control + five attempts) and zero retry /
restore-in-progress lines: every `notifyTaskStatusToMaster` loop terminated on
the master's `JobNotFoundException` on its first round trip.
`./mvnw spotless:apply` was run on the module; no `src/main` or `pom.xml`
files are touched.
### Check list
* [x] Code changed are covered with tests, or it does not need tests for
reason: test-only change
* [x] Change does not need documentation
* [x] Ran `./mvnw spotless:apply` on the changed module
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
