Bryan, The relative ./tmp directory is under the work-directory of the task's attempt on a node. I believe it should be sufficient to use that as a base, to create directories or files under? Or are you simply looking for a way to expand this relative directory name to a fully absolute one (which should be doable using simple Java SE APIs)?
On Thu, May 5, 2011 at 12:48 AM, Bryan Keller <brya...@gmail.com> wrote: > Right. What I am struggling with is how to retrieve the path/drive that the > reducer is using, so I can use the same path for local temp files. > > On May 4, 2011, at 9:03 AM, Robert Evans wrote: > >> Bryan, >> >> I believe that map/reduce gives you a single drive to write to so that your >> reducer has less of an impact on other reducers/mappers running on the same >> box. If you want to write to more drives I thought the idea would then be >> to increase the number of reducers you have and let mapred assign each to a >> drive to use, instead of having one reducer eating up I/O bandwidth from all >> of the drives. >> >> --Bobby Evans >> >> On 5/4/11 7:11 AM, "Bryan Keller" <brya...@gmail.com> wrote: >> >> I too am looking for the best place to put local temp files I create during >> reduce processing. I am hoping there is a variable or property someplace >> that defines a per-reducer temp directory. The "mapred.child.tmp" property >> is by default simply the relative directory "./tmp" so it isn't useful on >> it's own. -- Harsh J
