Another approach would be to use a custom InputFormat implementation,
with the flag as a property of the input split . Consider wrapping your
InputFormat with something like 'InputFormatWithFlag', that returns
splits that combine the wrapped InputFormat's splits with your flag.
Since InputFormat#getSplits is run in one process, your custom
InputFormat can safely ensure that only one flag is set.
Brien
On 1/2/2010 11:08 PM, Mark Kerzner wrote:
I think you need some kind of semaphore that you can turn on by the first
reducer. For example, allocating a file in HDFS would work - if you could
guarantee that it is an atomic operation (create-if-does-not-exist).
Mark
On Sat, Jan 2, 2010 at 10:04 PM, bharath v<
bharathvissapragada1...@gmail.com> wrote:
Hi,
I want a particular "section of code" to run only in any "ONE" of the
mappers . So I employed the following procedure.
Main-Class
{
public boolean flag = true;
Map-Class
{
if(flag)
{
flag=false;
/* section of code */
}
}
I am running this code on in pseudo-distributed mode and its working fine .
I doubt whether this runs correctly in distributed mode because , mappers
on
other systems have to notified of the changed "flag" .. Any Comments ? If
this is wrong , any suggestions on what method I must follow to achieve
this
functionality in D-mode .
Thanks
