[ https://issues.apache.org/jira/browse/MAPREDUCE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15741494#comment-15741494 ]
Vinayakumar B edited comment on MAPREDUCE-6486 at 12/12/16 9:49 AM: -------------------------------------------------------------------- Since ViewFS is just a wrapper over real filesystems, it would be better just ignore these stats and dont show it in final counters list. {{FileSystem.getAllStatistics()}} can filter stats for all filesystems with child filesystems, or just viewfs. What do you think [~cnauroth]? was (Author: vinayrpet): Since ViewFS is just a wrapper over real filesystems, it would be better just ignore these stats and dont show it in final counters list. Since these counters are already updated for resolved filesystems. {{FileSystem.getAllStatistics()}} can filter stats for all filesystems with child filesystems, or just viewfs. What do you think [~cnauroth]? > File{Input|Output}Format counters not updated when using viewfs > --------------------------------------------------------------- > > Key: MAPREDUCE-6486 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6486 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, task > Reporter: Laurent Goujon > > MapReduce tasks rely on filesystem stats to update FileInputFormatCounter and > FileOutputFormatCounter bytes counters. > But if the filesystem is viewfs, stats are not updated and counters are zero. > However the resolved filesystem for the input and output paths have their > stats updated, and should be used by the framework (which could be done by > resolving the path first). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org