divijvaidya commented on code in PR #14133: URL: https://github.com/apache/kafka/pull/14133#discussion_r1280583213
########## core/src/main/java/kafka/log/remote/RemoteStorageMetrics.java: ########## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package kafka.log.remote; + +import com.yammer.metrics.core.MetricName; +import kafka.server.BrokerTopicStats; +import org.apache.kafka.server.metrics.KafkaYammerMetrics; + +import java.util.HashSet; +import java.util.Set; + +import static org.apache.kafka.storage.internals.log.RemoteStorageThreadPool.AVG_IDLE_PERCENT; +import static org.apache.kafka.storage.internals.log.RemoteStorageThreadPool.TASK_QUEUE_SIZE; + +public class RemoteStorageMetrics { + final static MetricName REMOTE_BYTES_OUT_PER_SEC = getMetricName( + "kafka.server", "BrokerTopicMetrics", BrokerTopicStats.RemoteBytesOutPerSec()); + final static MetricName REMOTE_BYTES_IN_PER_SEC = getMetricName( + "kafka.server", "BrokerTopicMetrics", BrokerTopicStats.RemoteBytesInPerSec()); + final static MetricName REMOTE_READ_REQUESTS_PER_SEC = getMetricName( + "kafka.server", "BrokerTopicMetrics", BrokerTopicStats.RemoteReadRequestsPerSec()); + final static MetricName REMOTE_WRITE_REQUESTS_PER_SEC = getMetricName( + "kafka.server", "BrokerTopicMetrics", BrokerTopicStats.RemoteWriteRequestsPerSec()); + final static MetricName FAILED_REMOTE_READ_REQUESTS_PER_SEC = getMetricName( + "kafka.server", "BrokerTopicMetrics", BrokerTopicStats.FailedRemoteReadRequestsPerSec()); + final static MetricName FAILED_REMOTE_WRITE_REQUESTS_PER_SEC = getMetricName( + "kafka.server", "BrokerTopicMetrics", BrokerTopicStats.FailedRemoteWriteRequestsPerSec()); + final static MetricName REMOTE_LOG_MANAGER_TASKS_AVG_IDLE_PERCENT = getMetricName( + "kafka.log.remote", "RemoteLogManager", RemoteLogManager.REMOTE_LOG_MANAGER_TASKS_AVG_IDLE_PERCENT); + final static MetricName REMOTE_LOG_READER_TASK_QUEUE_SIZE = getMetricName( + "org.apache.kafka.storage.internals.log", "RemoteStorageThreadPool", RemoteLogManager.REMOTE_LOG_READER_METRICS_NAME_PREFIX + TASK_QUEUE_SIZE); Review Comment: Apologies that I wasn't clear in my previous comments. I was hoping that RemoteStorageMetrics act as the source of truth for all RemoteStorage related metrics, i.e. instead of `BrokerTopicStats.FailedRemoteWriteRequestsPerSec` we have `RemoteStorageMetrics.FailedRemoteWriteRequestsPerSec` and this constant is used in BrokerTopicStats. This is also consistent with how we are defining metrics in newer code such as QuorumControllerMetrics. As an example for REMOTE_LOG_MANAGER_TASKS_AVG_IDLE_PERCENT: we have a constant ``` public final String REMOTE_LOG_MANAGER_TASKS_AVG_IDLE_PERCENT_METRIC_NAME = "RemoteLogManagerTasksAvgIdlePercent"; final static MetricName REMOTE_LOG_MANAGER_TASKS_AVG_IDLE_PERCENT = getMetricName( "kafka.log.remote", "RemoteLogManager", RemoteStorageMetrics.REMOTE_LOG_MANAGER_TASKS_AVG_IDLE_PERCENT); ``` At RemoteLogManager, we use the interface of MetricsRegistry which takes MetricNames as an argument: ``` public <T> Gauge<T> newGauge(MetricName metricName, Gauge<T> metric) ``` and use it as: ``` metricsGroup.newGauge(RemoteStorageMetrics.REMOTE_LOG_MANAGER_TASKS_AVG_IDLE_PERCENT, new Gauge<Double>() { @Override public Double value() { return rlmScheduledThreadPool.getIdlePercent(); } }); ``` The advantages of this approach is that: 1. we don't have metrics constants defined at two places, e.g. MetricsNames in RemoteStorageMetrics and String metric names in respective files. 2. when someone adds a new metric RemoteStorage based metric, they will notice that all metrics are at one place and will remember to add their new metric in this file. 3. we use a consistent approach across the code base such as in `QuorumControllerMetrics` and `ControllerMetadataMetrics` 4. Later the scope of responsibility of this class could be expanded to close all RemoteStorage metrics at one place simultaneously. what do you think? ########## core/src/main/java/kafka/server/builders/KafkaApisBuilder.java: ########## @@ -38,6 +38,7 @@ import java.util.Collections; import java.util.Optional; + Review Comment: nit Extra line -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
