[
https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732474#action_12732474
]
Vinod K V commented on MAPREDUCE-476:
-------------------------------------
Looked at the remaining parts of the patch. Some more comments.
DistributedCacheHandle.makeClassLoader
- use File.toURI.toURL() instead of File.toURL() directly. File.toURL() is
deprecated.
LocalJobRunner.java
- TaskRunner.setupWorkDir needs to be fixed to have a workDir arg to help the
code in LocalJobRunner. Compilation(ant binary) breaks now.
- TODO: Test pipes with LocalJobRunner and make sure that it works, i.e.
verify HADOOP-5174.
TestDistributedCacheManager
- Code related to setFileStamps in JobClient.java (+636 to +656) and
testManagerFlow() (+71 to +74) can be refactored into an internal-only method
in DistributedCacheManager.
- Minor: assertNonNull(+90) check can be moved after +91 and use
localCacheFiles for the null check
TestMRWithDistributedCache
- Just documenting a TODO: Will need to fix this class's javadoc once
MAPREDUCE-711 is in.
- Fix the comment at +84. It is actually two archives.
- Please put a comment for the class saying that it is NOT A FAST test,
keeping in view the recent efforts to have a fast test target.
> extend DistributedCache to work locally (LocalJobRunner)
> --------------------------------------------------------
>
> Key: MAPREDUCE-476
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-476
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: sam rash
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-2914-v1-full.patch,
> HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch,
> MAPREDUCE-476-v2.patch, MAPREDUCE-476.patch
>
>
> The DistributedCache does not work locally when using the outlined recipe at
> http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html
>
> Ideally, LocalJobRunner would take care of populating the JobConf and copying
> remote files to the local file sytem (http, assume hdfs = default fs = local
> fs when doing local development.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
