Map tasks allocation in reduce slots?

2012-12-29 Thread Pedro Sá da Costa
MapReduce framework has map and reduce slots, that are used to track which tasks are running. When map tasks are just running, the reduce slots that the job have will be filled by map tasks? -- Best regards,

Re: Map tasks allocation in reduce slots?

2012-12-29 Thread Harsh J
MR1 has the concept of map and reduce slots (i.e. different types of slots) - but it does not run maps in reducer allocated slots - they are exclusive for reducers of any job to run. YARN, on which MR2 runs, has no notion of dedicated slots (for M or R types). On Sat, Dec 29, 2012 at 6:51 PM,

RE: Fastest way to transfer files

2012-12-29 Thread David Parks
Here’s an example of running distcp (actually in this case s3distcp, but it’s about the same, just new DistCp()) from java: ToolRunner.run(getConf(), new S3DistCp(), new String[] { --src, /src/dir/, --srcPattern, .*(itemtable)-r-[0-9]*.*, --dest,