What happens when the reducer task gets invoked more than once? My guess is once a reducer task finishes writing the data for a particular key to HDFS, it won't somehow get re-executed again for the same key right?
On Mon, Nov 3, 2008 at 11:28 AM, Miles Osborne <[EMAIL PROTECTED]> wrote: > you can't guarantee that a reducer (or mapper for that matter) will be > executed exactly once unless you turn-off preemptive scheduling. but, > a distinct key gets sent to a single reducer, so yes, only one reducer > will see a particulat key + associated values > > Miles > > 2008/11/3 Ryan LeCompte <[EMAIL PROTECTED]>: >> Hello, >> >> Is it safe to assume that only one reduce task will ever operate on >> values for a particular key? Or is it possible that more than one >> reduce task can work on values for the same key? The reason I ask is >> because I want to ensure that a piece of code that I write at the end >> of my reducer method will only ever be executed once after all values >> for a particular key are aggregated/summed. >> >> Thanks, >> Ryan >> > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. >
