[
https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Zeyliger updated MAPREDUCE-476:
--------------------------------------
Attachment: MAPREDUCE-476-v7.patch
Fixing the test failures, or so I hope.
The "workDir" handling in LocalJobRunner was confused, so I've fixed that. In
general, the symlinking stuff is a bit of a complex mess: you can symlink
things when you do "foo#bar", and also, if you set the config variable,
everything gets symlinked into the local working dir. This is used by
streaming. It seems like the latter layer should be done by streaming on its
own, but, alas, it may be too late to do that.
Note that I've attached a v6-vs-v7.patch file to make it easier to see what the
latest changes were.
-- Philip
> extend DistributedCache to work locally (LocalJobRunner)
> --------------------------------------------------------
>
> Key: MAPREDUCE-476
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-476
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: sam rash
> Assignee: Philip Zeyliger
> Priority: Minor
> Attachments: HADOOP-2914-v1-full.patch,
> HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch,
> MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt,
> MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch,
> MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch,
> MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch,
> MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch,
> MAPREDUCE-476.patch, v6-to-v7.patch
>
>
> The DistributedCache does not work locally when using the outlined recipe at
> http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html
>
> Ideally, LocalJobRunner would take care of populating the JobConf and copying
> remote files to the local file sytem (http, assume hdfs = default fs = local
> fs when doing local development.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
