[
https://issues.apache.org/jira/browse/MAPREDUCE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15741494#comment-15741494
]
Vinayakumar B edited comment on MAPREDUCE-6486 at 12/12/16 9:49 AM:
--------------------------------------------------------------------
Since ViewFS is just a wrapper over real filesystems, it would be better just
ignore these stats and dont show it in final counters list.
{{FileSystem.getAllStatistics()}} can filter stats for all filesystems with
child filesystems, or just viewfs.
What do you think [~cnauroth]?
was (Author: vinayrpet):
Since ViewFS is just a wrapper over real filesystems, it would be better just
ignore these stats and dont show it in final counters list.
Since these counters are already updated for resolved filesystems.
{{FileSystem.getAllStatistics()}} can filter stats for all filesystems with
child filesystems, or just viewfs.
What do you think [~cnauroth]?
> File{Input|Output}Format counters not updated when using viewfs
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-6486
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6486
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, task
> Reporter: Laurent Goujon
>
> MapReduce tasks rely on filesystem stats to update FileInputFormatCounter and
> FileOutputFormatCounter bytes counters.
> But if the filesystem is viewfs, stats are not updated and counters are zero.
> However the resolved filesystem for the input and output paths have their
> stats updated, and should be used by the framework (which could be done by
> resolving the path first).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
