The distributed cache does not behave as expected in PsuedoDistributed mode.
Since all of the tasks run within the same jvm, there is no easy way to alter the class path to add distributed cache items, in addition there is no way to change the working directory of the jvm to the 'per task directory' that the normal task runner creates, where the distributed cache items made available. On Sat, Oct 10, 2009 at 10:48 AM, Alberto Luengo Cabanillas < [email protected]> wrote: > Hi everyone! I´m trying to distribute some common data between my nodes, > but > I´m unable to do it. I have some questions about this: > > 1.- I´m working in PseudoDistributed mode in Windows (under Cygwin) and my > DFS server is running on a virtual machine (Ubuntu server). May I use > DistributedCache normally with this configuration? > 2.-In the case of the Mapper nodes, in the Driver class I add these two > lines: > > URI uriMapper =new URI("/user/hadoop-user/data/mapper_configuration.dat"); > (I also verify this file already exists in the DFS). > DistributedCache.addCacheFile(uriMapper,conf); > > 3.- In the Mapper class, I have this: > > @Override > protected void setup(Context cont) { > ... > String configureCacheName = new > Path("/user/hadoop-user/data/mapper_configuration.dat").getName(); > Path [] cacheFiles = > DistributedCache.getLocalCacheFiles(cont.getConfiguration()); > System.out.println("The file is: "+cacheFiles.toString()); > ... > } > > Does anyone have an idea about what I´m doing wrong? (I´m going crazy, > actually...) > > Thanks a lot in advance. > > -- > Alberto > -- Pro Hadoop, a book to guide you from beginner to hadoop mastery, http://www.amazon.com/dp/1430219424?tag=jewlerymall www.prohadoopbook.com a community for Hadoop Professionals
