mehakmeet commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1192316937
Pushed the changes, makes sense in case of null IOStatisticsContext. There
is one more issue, in case we have `fs.thread.level.iostatistics.enabled=false`
which means empty counters, hen
mehakmeet commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1190642189
added your latest commit + Testing for full suite done for both enabled and
disabled `fs.s3a.committer.experimental.collect.iostatistics` too.
--
This is an automated message from the
mehakmeet commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1186772913
Cherry-picked S3A committers integration and lining up with spark build.
Some javadocs would need modification, adding that in a commit. Need to see how
to test this on a spark build loc
mehakmeet commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1181759094
Changed the way streams are accessing the thread IOStatistics, now we would
directly get them from the current active context in the stream's constructor
rather than pass them around thr
mehakmeet commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1164037866
More merge conflicts due to vectored IO merging.
Interesting bit of info on the new test, seems to fail in maven verify but
works fine in intellij, I remember Steve saying something a
mehakmeet commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1162654911
Had to force push to resolve conflicts.
Some changes in the latest commit:
- Moved thread context initialization and config for enabling in
hadoop-common.
- Weak ref thread m
mehakmeet commented on PR #4352:
URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1157264920
Thanks for the review @steveloughran, sorry couldn't address anything until
now(got little ill)
> main comment is that the thread's statistic aggregator should be
fetched/stored in co
