Steve Loughran created HADOOP-17273:
---------------------------------------
Summary: FSDataInput/Output Streams to automatically collect
IOStatistics
Key: HADOOP-17273
URL: https://issues.apache.org/jira/browse/HADOOP-17273
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 3.3.1
Reporter: Steve Loughran
the fs input/output streams to automatically collect stream IO statistics even
if the inner classes don't; count invocations and durations
An issue here becomes "how would you aggregate with the inner stream,
*efficiently*. Not sure there, except maybe
* have some interface to access an updatable IOStatisticsStore which, if the
stream implements, says "here is something you can update"
* wrapper class updates that with counts, durations
* we extend DynamicIOStatistics for each counter/gauge etc to have a factory
which can add new entries on demand (new AtomicLong, etc)
* so wrapper classes just update their stats, which updates existing stats or
triggers off an on-demand creation of a new entry
Not so informative in terms of low level details (HTTP/IPC requests and
latency, errors, bytes discarded) but would give callers benefits of using the
API for HDFS, Ozone, GCS
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
