Have you taken a look into extending the FileOutputFormat class and
overriding the OutputCommitter API functionality?
Regards,
Shahab
On Mon, Jun 3, 2013 at 5:11 PM, samir das mohapatra <samir.help...@gmail.com
> wrote:
> Dear All,
>
> Is there any way to copy the intermediate output file of the mapper
> into local folder after each map task complete.
>
> Right now I am using
>
> FileSystem.copyToLocalFile(hdfsLocation,localLocation);
> indiste the cleanup of mapper task , but it is failing .
>
> Exception file not found.
>
> But if I am giving same statement after the job complete in driver class
> ,it is working fine. that i dont want.
>
> protected void cleanup(Context context){
> FileSystem.copyToLocalFile(hdfsLocation,localLocation);//failed
> }
>
> Note: I need to copy the inter mediate output of the mapper to local file
> system just after each map task complete. I dont want any reducer.
>
> If this is not the right solution then how to solve this type of
> scenario.
>
> Any help.............
>
> regards,
> samir.
>
