Jerry Chung created SOLR-18407:
----------------------------------
Summary: Solr 10 emits duplicate JVM metrics from both
runtime-telemetry-java8 and runtime-telemetry-java17 scopes on Java 17
Key: SOLR-18407
URL: https://issues.apache.org/jira/browse/SOLR-18407
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: metrics
Affects Versions: 10.0
Reporter: Jerry Chung
When running Solr 10 on Java 17, the {{/solr/admin/metrics?wt=prometheus}}
endpoint emits JVM metrics (e.g. {{{}jvm_gc_duration_seconds_count{}}},
{{{}jvm_memory_used_bytes{}}}) from two OTEL instrumentation scopes
simultaneously:
* {{io.opentelemetry.runtime-telemetry-java17}} (JFR-based)
* {{io.opentelemetry.runtime-telemetry-java8}} (JMX-based)
{{OtelRuntimeJvmMetrics}} only calls {{.enableAllFeatures()}} without
{{{}.enableExperimentalJmxTelemetry(){}}}, so the java8/JMX scope should not
activate. However, the bundled version of
{{opentelemetry-runtime-telemetry-java17}} ({{{}2.22.0-alpha{}}}) implicitly
activates JMX metrics alongside JFR, producing duplicate series with
conflicting values for the same pool (e.g. G1 Eden Space {{used}} differs
between scopes by an order of magnitude).
*Impact:* Metric consumers receive duplicate series with different values for
the same metric, making it impossible to reliably use these metrics in
dashboards or alerts without scope-specific filtering.
*Expected behavior:* Only {{runtime-telemetry-java17}} (JFR) metrics should be
emitted on Java 17, or there should be a configuration to select
*Workaround:* None available without patching Solr or upgrading the bundled
OTEL library to 2.13.x+ where the implicit JMX activation was fixed.
*Related:* SOLR-17806 /
[apache/solr#3494|https://github.com/apache/solr/issues/3494],
opentelemetry-java-instrumentation#13332,
opentelemetry-java-instrumentation#13337
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
