[
https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731901#action_12731901
]
Vinod K V commented on MAPREDUCE-476:
-------------------------------------
Some more comments:
TaskTracker.initialize()
- A new DistributedCacheManager is created every time, so old files will not
be deleted by the subsequent purgeCache. Either we should have only one
cacheManager for a TaskTracker or DistributedCacheManager.cachedArchives should
be static.
- The same problem exists with the deprecated purgeCache() method in
DistributedCache.java
JobClient.java
- What happens to the code in here? Should this be refactored too/ are we
going to do something about it?
DistributedCache.java
- getLocalCache() (+190) should be indirected to
DistributedCacheManager.getLocalCache(). Otherwise there is a stack overflow
error - the method calls itself as of now.
- I think most of the getter/setter methods are deprecable and moveable to
DistributedCacheManager. Or at least we should give some thought about it. For
most of them, I do see from the javadoc that they are only for internal use
anyways.
> extend DistributedCache to work locally (LocalJobRunner)
> --------------------------------------------------------
>
> Key: MAPREDUCE-476
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-476
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: sam rash
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-2914-v1-full.patch,
> HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch,
> MAPREDUCE-476-v2.patch, MAPREDUCE-476.patch
>
>
> The DistributedCache does not work locally when using the outlined recipe at
> http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html
>
> Ideally, LocalJobRunner would take care of populating the JobConf and copying
> remote files to the local file sytem (http, assume hdfs = default fs = local
> fs when doing local development.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
