[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod K V updated MAPREDUCE-476: Release Note: Extended DistributedCache to work with LocalJobRunner. extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Fix For: 0.21.0 Attachments: 476.20S-2.patch, HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch, MAPREDUCE-476-v8.patch, MAPREDUCE-476-v9.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Status: Patch Available (was: Open) extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch, MAPREDUCE-476-v8.patch, MAPREDUCE-476-v9.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Attachment: MAPREDUCE-476-v9.patch Well-spotted, Tom. I've restored the missing test. extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch, MAPREDUCE-476-v8.patch, MAPREDUCE-476-v9.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Status: Open (was: Patch Available) extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Status: Patch Available (was: Open) extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch, MAPREDUCE-476-v8.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[
https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Zeyliger updated MAPREDUCE-476:
--
Attachment: MAPREDUCE-476-v8.patch
The test failure was spurious
(org.apache.hadoop.mapred.TestRecoveryManager.testRestartCount is failing
elsewhere also, and has nothing to do with this patch).
But the FindBugs errors were reasonable. New patch fixes the three reported,
and pasted in below.
{quote}
Bad practice Warnings
CodeWarning
DP
org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager.makeClassLoader(ClassLoader)
creates a java.net.URLClassLoader classloader, which should be performed
within a doPrivileged block
Bug type DP_CREATE_CLASSLOADER_INSIDE_DO_PRIVILEGED (click for details)
In class org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager
In method
org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager.makeClassLoader(ClassLoader)
In class java.net.URLClassLoader
At TaskDistributedCacheManager.java:[line 235]
RV org.apache.hadoop.mapred.TaskRunner.setupWorkDir(JobConf, File) ignores
exceptional return value of java.io.File.mkdir()
Bug type RV_RETURN_VALUE_IGNORED_BAD_PRACTICE (click for details)
In class org.apache.hadoop.mapred.TaskRunner
In method org.apache.hadoop.mapred.TaskRunner.setupWorkDir(JobConf, File)
Called method java.io.File.mkdir()
At TaskRunner.java:[line 630]
Performance Warnings
CodeWarning
UrF Unread field:
org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager$CacheFile.localClassPath
{quote}
extend DistributedCache to work locally (LocalJobRunner)
Key: MAPREDUCE-476
URL: https://issues.apache.org/jira/browse/MAPREDUCE-476
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: sam rash
Assignee: Philip Zeyliger
Priority: Minor
Attachments: HADOOP-2914-v1-full.patch,
HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch,
MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt,
MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch,
MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch,
MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch,
MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch,
MAPREDUCE-476-v8.patch, MAPREDUCE-476.patch, v6-to-v7.patch
The DistributedCache does not work locally when using the outlined recipe at
http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html
Ideally, LocalJobRunner would take care of populating the JobConf and copying
remote files to the local file sytem (http, assume hdfs = default fs = local
fs when doing local development.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Attachment: v6-to-v7.patch extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Attachment: MAPREDUCE-476-v7.patch Fixing the test failures, or so I hope. The workDir handling in LocalJobRunner was confused, so I've fixed that. In general, the symlinking stuff is a bit of a complex mess: you can symlink things when you do foo#bar, and also, if you set the config variable, everything gets symlinked into the local working dir. This is used by streaming. It seems like the latter layer should be done by streaming on its own, but, alas, it may be too late to do that. Note that I've attached a v6-vs-v7.patch file to make it easier to see what the latest changes were. -- Philip extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Status: Open (was: Patch Available) extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476-v7.patch, MAPREDUCE-476.patch, v6-to-v7.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod K V updated MAPREDUCE-476: Attachment: MAPREDUCE-476-20090818.txt MAPREDUCE-711 is done on mapreduce side. So, no more blockers for this issue. Not sure what was wrong with the earlier patch. In any case, it was broken again by changes from MAPREDUCE-478. Attached is the updated patch. Can you please check the latest patch and see if anything is missing or if you need any more changes? If you are fine with it, can you ask someone to commit this, perhaps Tom? Already this patch is broken by a couple of patches, so I think the earlier this gets checked in, the better. extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod K V updated MAPREDUCE-476: Status: Patch Available (was: Open) extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Attachment: MAPREDUCE-476-v5-requires-MR711.patch Latest patch. Thanks Vinod for the review! extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Attachment: MAPREDUCE-476-v4-requires-MR711.patch MAPREDUCE-476-v2-vs-v4.txt extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Attachment: MAPREDUCE-476-v2.patch In light of MAPREDUCE-711, I generated a new patch. I applied MAPREDUCE-711-20090709-mapreduce.1.txt first, so this shouldn't be submitted to Hudson until after that gets checked in. I generated the patch by applying bq. cat HADOOP-2914-v3.patch | sed -e 's%src/core/%src/java/%g' | sed -e 's%src/mapred/%src/java/%g' | sed -e 's%src/test/core%src/test/mapred%g' | patch -p0 I had to clean up DistributedCache.java a tiny bit (there were 2 rejects) because some Javadoc links were removed in the project move; I've reinstated them. (I think they were removed because they pointed to MR from Common, but that's no longer an issue with MAPREDUCE-711.) extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-v2.patch, MAPREDUCE-476.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Zeyliger updated MAPREDUCE-476: -- Attachment: MAPREDUCE-476.patch Regenerated patch after project split. I used: bq. cat /Users/philip/Downloads/HADOOP-2914-v3.patch | sed -e 's,src/mapred/,src/java/,' | patch -p0 extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
