Ahmar Suhail created HADOOP-19364:
-------------------------------------
Summary: Support IoStatistics for new analytics-accelerator stream
Key: HADOOP-19364
URL: https://issues.apache.org/jira/browse/HADOOP-19364
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Ahmar Suhail
S3A provides InputStream statistics:
[https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/statistics/S3AInputStreamStatistics.java]
This helps track things like how many bytes were read from a stream etc.
The current integration does not currently implement statistics. To start off
with we should identify which of these statistics makes sense for us track in
the new stream. Some examples are:
1/ bytesRead
2/ readOperationStarted
3/ initiateGetRequest
Some of these (1 and 2) are more straightforward, and should not require any
changes to analytics-accelerator-s3, but tracking GET requests will require
this.
We should also add tests that make assertions on these statistics. See
ITestS3APrefetchingInputStream for an example to do this.
And see https://issues.apache.org/jira/browse/HADOOP-18190 for how this was
done on the prefetching stream, and PR:
https://github.com/apache/hadoop/pull/4458
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
