Steve Loughran created HADOOP-17273: ---------------------------------------
Summary: FSDataInput/Output Streams to automatically collect IOStatistics Key: HADOOP-17273 URL: https://issues.apache.org/jira/browse/HADOOP-17273 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 3.3.1 Reporter: Steve Loughran the fs input/output streams to automatically collect stream IO statistics even if the inner classes don't; count invocations and durations An issue here becomes "how would you aggregate with the inner stream, *efficiently*. Not sure there, except maybe * have some interface to access an updatable IOStatisticsStore which, if the stream implements, says "here is something you can update" * wrapper class updates that with counts, durations * we extend DynamicIOStatistics for each counter/gauge etc to have a factory which can add new entries on demand (new AtomicLong, etc) * so wrapper classes just update their stats, which updates existing stats or triggers off an on-demand creation of a new entry Not so informative in terms of low level details (HTTP/IPC requests and latency, errors, bytes discarded) but would give callers benefits of using the API for HDFS, Ozone, GCS -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org