What's the proper way to use hadoop task side-effect files?

[Zhang, jian]

Hi,

I was new to hadoop. Sorry for my novice question.
I got some problem while I was trying to use task side-effect files.
Since there is no code example in wiki, I tried this way:

I override cofigure method in reducer to create a side file,

         public void configure(JobConf conf){
         logger.info("Tring to create sideFiles inside reducer.!");

         Path workpath=conf.getOutputPath();
         Path sideFile= new Path(workpath,"SideFile.txt");
         try {
                   FileSystem fs = FileSystem.get(conf);
                   out= fs.create(sideFile);
         } catch (IOException e) {
                   logger.error("Failed to create sidefile!");
         }        
         }
And try to use it in reducer.

But I got some strange problems,
Even If the method is in reducer Class, mapper tasks are creating the
side files.
Mapper tasks hang because there are tring to recreate the file.

org.apache.hadoop.dfs.AlreadyBeingCreatedException:
 failed to create file
/data/input/MID06/_temporary/_task_200804112315_0001_m_000008_0/SideFile
.txt for DFSClient_task_200804112315_0001_m_000008_0 on client
192.168.0.203 because current leaseholder is trying to recreate file.
         at
org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:9
74)
         at
org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:931)
         at org.apache.hadoop.dfs.NameNode.create(NameNode.java:281)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
         at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:585)
         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)


Can anybody help me on this, how to use side-effect files? 



Best Regards

Jian Zhang