Sharad, Sqoop handles this by the following arguments:
--staging-table <staging-table-name> The table in which data will be staged before being inserted into the destination table. --clear-staging-table Indicates that any data present in the staging table can be deleted. For more information: http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_syntax_3 Note that only some connectors support staging tables. Regards, Kate On Tue, Nov 29, 2011 at 8:29 PM, Sharad Agarwal <[email protected]>wrote: > How do we ensure that map/reduce tasks are idempotent. To be more clear: if > there are more than 1 map/reduce attempts for an export job, how do we make > sure that data from only single task attempt is committed ? > > Normally this is handled by OutputCommitter, however I see > NullOutputCommitter which doesn't do anything. > > Thanks > Sharad >
