[
https://issues.apache.org/jira/browse/HADOOP-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853609#comment-17853609
]
ASF GitHub Bot commented on HADOOP-18325:
-----------------------------------------
anmolanmol1234 opened a new pull request, #6878:
URL: https://github.com/apache/hadoop/pull/6878
We have introduced support for metric collection at the filesystem instance
level.
Metrics are pushed to the store upon the closure of a filesystem instance,
encompassing all operations that utilized that specific instance.
Collected Metrics:
Number of successful requests without any retries.
Count of requests that succeeded after a specified number of retries (x
retries).
3.Request count subjected to throttling.
4.Number of requests that failed despite exhausting all retry attempts. etc.
Implementation Details:
Incorporated logic in the AbfsClient to facilitate metric pushing through an
additional request.
This occurs in scenarios where no requests are sent to the backend for a
defined idle period.
By implementing these enhancements, we ensure comprehensive monitoring and
analysis of filesystem interactions, enabling a deeper understanding of success
rates, retry scenarios, throttling instances, and exhaustive failure scenarios.
Additionally, the AbfsClient logic ensures that metrics are proactively pushed
even during idle periods, maintaining a continuous and accurate representation
of filesystem performance.
> ABFS: Add correlated metric support for ABFS operations
> -------------------------------------------------------
>
> Key: HADOOP-18325
> URL: https://issues.apache.org/jira/browse/HADOOP-18325
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.3.3
> Reporter: Anmol Asrani
> Assignee: Anmol Asrani
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> Add metrics related to a particular job, specific to number of total
> requests, retried requests, retry count and others
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
