joseluisll opened a new pull request, #8681:
URL: https://github.com/apache/hadoop/pull/8681
### Description of PR
Make the two federation mock processes shut their services down when they are
terminated, so that `TestFederationSubCluster`-based suites leave nothing
running behind them.
`TestMockSubCluster` and `TestMockRouter` are named like tests but are not
tests — they declare no `@Test` methods. They are process entry points:
`TestFederationSubCluster` launches each as its own JVM through `JavaProcess`
and tears it down in `JavaProcess#stop`, which calls `Process#destroy`
(SIGTERM
on Linux and macOS) followed by `waitFor`. Neither registered a shutdown
hook,
so each JVM exited with its services still running — a `MiniYARNCluster` of
one
ResourceManager and three NodeManagers in the sub-cluster processes, the
Router's own services in the other. No `serviceStop` was ever called: RPC
servers, web apps and state-store connections were torn down by process death
rather than by the service lifecycle.
Not a mini-cluster leak (cf. YARN-11983, which covers those, in different
files
in this module): these clusters are meant to outlive the method that starts
them. The defect is that their termination is unclean.
Both entry points now register a `CompositeServiceShutdownHook` before
`init`/`start`, so a failure part-way through startup still tears down
whatever
came up. `Router#main` already does exactly this (`Router.java:317-324`);
`TestMockRouter` starts the Router directly and so skipped it. Each uses the
hook priority of the daemon it actually runs — `ResourceManager`'s for the
sub-cluster, `Router`'s for the router — which every YARN daemon defines as
30.
Teardown is bounded by `hadoop.service.shutdown.timeout`, 30s per hook by
default.
`TestMockRouter` also closes the ZooKeeper-backed `FederationStateStore` it
hands to `FederationStateStoreFacade`, which never closes it. That hook is
registered at a lower priority than the Router's; `ShutdownHookManager` runs
the
highest priority first, so the Router is fully stopped before the store it
may
still be using goes away.
`TestMockRouter`'s class javadoc claimed "Tests `Router`" — the reason these
are
easy to mistake for tests in the first place. It now says what the class is.
Both files are in `hadoop-yarn-server-router`, package
`org.apache.hadoop.yarn.server.router.subcluster`. No production code
changes;
no new dependencies.
### How was this patch tested?
- `hadoop-yarn-server-router` test-compiles cleanly:
`mvn -B -o test-compile -P '!native-win,!shelltest' -DskipTests -pl
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router`
- The end-to-end path is the SIGTERM teardown itself, which only happens
when a
`TestFederationSubCluster` suite runs and destroys the child JVMs. It was
not
exercised locally — this was developed on Windows, where `Process#destroy`
maps to `TerminateProcess` and runs no shutdown hooks at all, so the
behaviour cannot be observed there.
`TestYarnFederationWithCapacityScheduler` and
`TestYarnFederationWithFairScheduler` exercise it in CI.
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the
endpoint
declared according to the connector-specific documentation? *Note:
Automated CI
testing doesn't cover all cases so manual testing with cloud storage
is still
required.*
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
### AI Tooling
If an AI tool was used:
- [x] The PR includes the phrase "Contains content generated by <tool>"
where <tool> is the name of the AI tool used.
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
Contains content generated by Claude Code (Claude Opus 5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
