yandrey321 opened a new pull request, #11719:
URL: https://github.com/apache/ozone/pull/11719
## What changes were proposed in this pull request?
**Problem.** The first datanode client in a process calls
`XceiverClientMetrics.create()`, which runs
`DefaultMetricsSystem.initialize("XceiverClientMetrics")`. That starts a
full `MetricsSystemImpl`: it reads the
metrics configuration through Commons Configuration, starts a timer thread,
and registers JMX MBeans for the system
and every source. That costs about 100 ms per cold client, and in
short-lived processes nothing reads the result.
Examples are YARN `ContainerLocalizer` JVMs, CLI commands, job submitters,
and clients in other applications.
MapReduce AppMasters and tasks start the metrics system themselves
(`MRAppMaster`, `YarnChild`), so this change
doesn't affect them. HDDS-1721 added the call so that a client with a
configured metrics sink publishes its metrics.
**Implementation idea.** A new client setting, `ozone.client.metrics.mode`:
- `ALWAYS`: start the metrics system as before;
- `AUTO` (default): start it only if a sink is configured for the
`xceiverclientmetrics` prefix;
- `NEVER`: never start it.
All metrics sources are still registered in every mode. Registering with a
metrics system that hasn't started is
cheap: Hadoop builds the source and adds a callback, so the source goes live
if the system starts later. Only the
`initialize()` call became conditional.
**Key decisions.**
- **Keep registration unconditional.** `XceiverClientMetrics`,
`CacheMetrics` and `ContainerClientMetrics` behave as
before. Every Ozone service (datanode, SCM, OM, Recon, S3 Gateway, HttpFS,
Freon) starts the metrics system itself,
so clients embedded in a service keep publishing their metrics in any mode.
- **`AUTO` uses the same sink lookup as Hadoop's `MetricsConfig`.** It reads
the first of
`hadoop-metrics2-xceiverclientmetrics.properties` and
`hadoop-metrics2.properties` found on the classpath. It looks
for `xceiverclientmetrics.sink.<instance>.<option>` keys; `*.sink.*` keys
only supply defaults. The file is read with
`java.util.Properties`, so the check itself doesn't load Commons
Configuration. If the file can't be read, the
client starts the metrics system, as before.
- **The mode is static.** `XceiverClientCreator` sets it from the client's
configuration. The callers of
`XceiverClientManager.getXceiverClientMetrics()` are static and have no
configuration, and the metrics system is
started at most once per JVM, as before.
**Side effects.**
- With the default `AUTO`, a client process that has no sink configured no
longer exposes its metrics over JMX
(`/jmx`, jconsole). Set `ozone.client.metrics.mode=ALWAYS` to keep that.
Worth a release note.
- Inside a service that already started the metrics system, the client no
longer calls `initialize()` again. That
call used to log "metrics system already initialized!".
Generated-by: Claude Code (Claude Opus 5.5)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17351
## How was this patch tested?
Unit and Integration tests.
- **Measurement.** Same build, `ALWAYS` against the default `AUTO`, on JDK
17 (17.0.20.1) and JDK 25. No metrics
sink is configured; the dist's `hadoop-metrics2.properties` has only
`*.sink.file.class` and `*.period`.
- `XceiverClientMetrics.create()` in a fresh JVM, 9 runs: on JDK 17, 129
ms → 57 ms (−72 ms), and 1,112 → 391 extra
classes loaded; on JDK 25, 148 ms → 57 ms (−91 ms). The remaining 57 ms
is the source itself (registry,
per-operation counters and latency metrics).
- Cold `ofs://` client on the secure `ozonesecure` compose cluster (create
the client, getFileStatus, read 64 KB,
close), 10 interleaved rounds per arm, medians:
| JDK | mode | total | range | first read |
|---|---|---|---|---|
| 17 | `ALWAYS` (before) | 1,285 ms | 1,254–1,353 ms | 432 ms |
| 17 | `AUTO` (default) | **1,230 ms (−55 ms)** | 1,205–1,248 ms | 380
ms (−52 ms) |
| 25 | `ALWAYS` (before) | 1,317 ms | 1,267–1,423 ms | 431 ms |
| 25 | `AUTO` (default) | **1,227 ms (−90 ms)** | 1,205–1,263 ms | 368
ms (−63 ms) |
On both JDKs the ranges don't overlap. The `ALWAYS` arm sets the mode on
the client's configuration, which loads the
configuration files earlier. That shifts about 30–40 ms from client
creation to the configuration step, so the
total is the comparison to look at.
- TestDFSIO on secure YARN, against the secure compose Ozone cluster: 3
NodeManagers with
`LinuxContainerExecutor`, 4 files, write and read, 6 interleaved rounds
per arm, 24 of 24 jobs successful. The run
was done twice: on the stock Hadoop image (JDK 11), and on that image
with JDK 17. In the `ALWAYS` arm the mode
was set on the job and in the NodeManagers' `ozone-site.xml`, which the
localizers read.
- Localizer JVMs: their localization time drops by **53–89 ms on JDK
17** (the main band of 61 localizations at
about 1.6 s: 1.600 → 1.547 s median, standard error about 0.015 s). On
JDK 11 it drops by 41–69 ms (main band at
about 2 s: 1.980 → 1.934 s). A job has two localizations on its
critical path, so that's about 0.1 s per job.
- MapReduce AppMaster and task start-up: no change on either JDK (within
±55 ms of noise), because those JVMs
already start the metrics system themselves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
