[
https://issues.apache.org/jira/browse/HADOOP-19737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18045488#comment-18045488
]
ASF GitHub Bot commented on HADOOP-19737:
-----------------------------------------
bhattmanish98 commented on code in PR #8135:
URL: https://github.com/apache/hadoop/pull/8135#discussion_r2622799467
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##########
@@ -1338,7 +1338,11 @@ public AbfsRestOperation read(final String path,
AbfsReadResourceUtilizationMetrics readResourceUtilizationMetrics =
retrieveReadResourceUtilizationMetrics();
// If metrics are available, record them in the tracing context for
diagnostics or logging.
if (readResourceUtilizationMetrics != null) {
-
tracingContext.setResourceUtilizationMetricResults(readResourceUtilizationMetrics.toString());
+ String readMetrics = readResourceUtilizationMetrics.toString();
Review Comment:
With this logic, we can send same metrics data in two different read call,
is this expected?
Should we move these lines under synchronized block?
```
String readMetrics = readResourceUtilizationMetrics.toString();
tracingContext.setResourceUtilizationMetricResults(readMetrics);
if (!readMetrics.isEmpty()) {
readResourceUtilizationMetrics.markPushed();
}
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferManagerV2.java:
##########
@@ -116,7 +116,7 @@ public final class ReadBufferManagerV2 extends
ReadBufferManager {
/* Tracks the last scale direction applied, or empty if none. */
private volatile String lastScaleDirection = EMPTY_STRING;
/* Maximum CPU utilization observed during the monitoring interval. */
- private volatile double maxJvmCpuUtilization = 0.0;
+ private volatile long maxJvmCpuUtilization = 0L;
Review Comment:
Is there a specific reason for this change? Using double seems more logical
for CPU utilization.
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/JvmIdProvider.java:
##########
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.azurebfs;
+
+import java.util.Random;
+
+/**
+ * Provides a JVM-scoped identifier.
+ *
+ * <p>The identifier is generated once when the class is loaded and remains
+ * constant for the lifetime of the JVM. It is derived using a combination of
+ * the current system time and random entropy to reduce the likelihood of
+ * collisions across JVM instances.</p>
+ *
+ * <p>The identifier is intended for lightweight JVM-level identification,
+ * such as tagging metrics or log entries. It provides best-effort uniqueness
+ * and is not guaranteed to be globally unique.</p>
+ *
+ * <p>This class is utility-only and cannot be instantiated.</p>
+ */
+public final class JvmIdProvider {
Review Comment:
Is calling it JvmIdProvider correct? Here we are generating our own id which
is not JVM Id
> ABFS: Add metrics to identify improvements with read and write aggressiveness
> -----------------------------------------------------------------------------
>
> Key: HADOOP-19737
> URL: https://issues.apache.org/jira/browse/HADOOP-19737
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.5.0, 3.4.2
> Reporter: Anmol Asrani
> Assignee: Anmol Asrani
> Priority: Major
> Labels: pull-request-available
>
> Introduces new performance metrics in the ABFS driver to monitor and evaluate
> the effectiveness of read and write aggressiveness tuning. These metrics help
> in understanding how thread pool behavior, CPU utilization, and heap
> availability impact overall I/O throughput and latency. By capturing detailed
> statistics such as active thread count, pool size, and system resource
> utilization, this enhancement enables data-driven analysis of optimizations
> made to improve ABFS read and write performance under varying workloads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
