Jackie-Jiang commented on code in PR #19872: URL: https://github.com/apache/pinot/pull/19872#discussion_r4288363217
########## pinot-plugins/pinot-metrics/pinot-yammer/README.md: ########## @@ -0,0 +1,61 @@ +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> + +# Pinot Yammer metrics (deprecated) + +`pinot-yammer` is deprecated since Pinot 1.6.0. Use the existing `pinot-dropwizard` metrics backend instead. + +Dropwizard is the default when no metrics factory is configured. Explicit Yammer factory settings still select +Yammer, which remains bundled. A removal release will be decided separately; see +[#19871](https://github.com/apache/pinot/issues/19871). + +## Migrate to Dropwizard + +1. Ensure `pinot-dropwizard` and its runtime dependencies are available to each affected service. Both metrics + plugins ship in the standard Pinot binary distribution; custom builds, classpaths, and plugin filters may + need to be updated. +2. Remove explicit Yammer factory settings to use the Dropwizard default, or replace them with the applicable + property in each service's configuration file: + + ```properties + pinot.broker.metrics.factory.className=org.apache.pinot.plugin.metrics.dropwizard.DropwizardMetricsFactory + pinot.controller.metrics.factory.className=org.apache.pinot.plugin.metrics.dropwizard.DropwizardMetricsFactory + pinot.server.metrics.factory.className=org.apache.pinot.plugin.metrics.dropwizard.DropwizardMetricsFactory + pinot.minion.metrics.factory.className=org.apache.pinot.plugin.metrics.dropwizard.DropwizardMetricsFactory + ``` + +3. To publish a Prometheus scrape endpoint, enable the existing JMX exporter Java agent with + `configs-dropwizard/<role>.yml`, where `<role>` is `broker`, `controller`, `server`, or `minion`. + If the agent is already enabled, change its YAML path from `configs/<role>.yml` when switching from Yammer. + See the + [agent startup and scrape configuration](../../../docker/images/pinot/etc/jmx_prometheus_javaagent/README.md#expose-a-scrape-endpoint-with-dropwizard). +4. Restart each affected service to load the new factory and exporter configuration. Validate the migration in + a test environment before rolling it out to production. + +The backend-specific role configurations retain the Prometheus metric names and labels for standard Pinot metrics. +Those timers are reported in milliseconds and meter rates per second with either backend. +Native JMX names and attributes can differ, as can sampled timer distributions. Validate custom JMX consumers, Review Comment: Document the change from lifetime to sampled timer aggregates There is a more specific semantic change than sampled distributions differing: Yammer's timer `Min`, `Max`, `Mean`, and `StdDev` are aggregates over all recorded observations, whereas Dropwizard's JMX reporter computes those attributes from its decaying reservoir snapshot. Matching the 1,028-sample capacity and decay factor does not preserve these aggregate semantics. For example, the reported maximum can decrease as an old observation leaves the sample, and the mean is weighted toward recent observations. These values keep the same exported series names, so existing dashboards and alerts can silently change meaning even after the YAML migration. Please explicitly document this in the migration/release notes and validate the intended behavior with nonconstant, time-separated observations. The shared exporter tests check names and labels, and a constant-duration smoke test cannot distinguish these semantics. ########## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ########## @@ -49,9 +49,7 @@ private CommonConstants() { public static final String CONFIG_OF_BROKER_EVENT_LISTENER_CLASS_NAME = "factory.className"; public static final String CONFIG_OF_REQUEST_CONTEXT_TRACKED_HEADER_KEYS = "request.context.tracked.header.keys"; public static final String DEFAULT_METRICS_FACTORY_CLASS_NAME = - //"org.apache.pinot.plugin.metrics.compound.CompoundPinotMetricsFactory"; - "org.apache.pinot.plugin.metrics.yammer.YammerMetricsFactory"; - //"org.apache.pinot.plugin.metrics.dropwizard.DropwizardMetricsFactory"; + "org.apache.pinot.plugin.metrics.dropwizard.DropwizardMetricsFactory"; Review Comment: Address the exporter upgrade path before changing the default An existing deployment with no explicit metrics factory setting and the current `configs/<role>.yml` switches to Dropwizard after upgrading, while retaining exporter rules written for Yammer. Standard Pinot series then stop matching. The new configurations preserve names and labels only after the operator also changes the Java-agent YAML path; direct JMX consumers must adapt to the new ObjectNames/types as well. The README documents this migration, but an unchanged deployment configuration is still incompatible. Please either provide a compatible exporter transition or stage the default change after a migration/deprecation period. Explicitly retaining Yammer remains a useful rollback path, but existing users must know to configure it before upgrading. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
