[
https://issues.apache.org/jira/browse/HADOOP-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645475#action_12645475
]
Sharad Agarwal commented on HADOOP-4188:
----------------------------------------
The Task.FileSystemCounter enum has the read/write counters for all concrete
filesystems. So if we want Task to be completely agnostic to filesystems, the
framework counters also somehow needs to be dynamically created. Currently
there is a static mapping between these - Task_FileSystemCounter.properties
We would then need to associate counter names based on filesystem URI schemes,
which is not quite possible if using current FileSystem#statisticsTable.
We can add Map<String, Statistics> statsByUriScheme to FileSystem as suggested
by Doug and use that. This map can be populated in createFileSystem(URI uri,
Configuration conf) call as :
statsByUriScheme.put(uri.getScheme(), fs.statistics);
Other very straightforward alternative which looks good to me:
Just break the compile time dependency on the concrete file system and use
conf.getClassByName; as anyway Task is aware of the concrete filesystems in
Task.FileSystemCounter. Making it fully agnostic would require refactoring of
Task.FileSystemCounter etc.
> Remove Task's dependency on concrete file systems
> -------------------------------------------------
>
> Key: HADOOP-4188
> URL: https://issues.apache.org/jira/browse/HADOOP-4188
> Project: Hadoop Core
> Issue Type: Sub-task
> Reporter: Tom White
> Assignee: Sharad Agarwal
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
