It says that the output directory exists, can you verify if thats the case?
Sqoop merge takes 3 directory parameters: --new-data - directory with the new data set to merge (in case of duplicates we keep the values from this directory) --onto - directory with existing data set to merge (some of the values here will be replaced by new values) --target-dir - this directory should not exist before running Sqoop merge. The results will go here. Gwen On Mon, Aug 4, 2014 at 9:51 AM, <[email protected]> wrote: > Hi, > > > > I am getting a weird error this morning! When running sqoop merge command > > > > Deleted s3://xxx-xx-xxx/data/xxxx/xxxx/xxxx/merged > > merge folder exists so deleted 0 > > > > Next, > > > > Merge command fails with the error > > > > > > 14/08/04 11:39:24 ERROR security.UserGroupInformation: > PriviledgedActionException as:hadoop (auth:SIMPLE) > cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory > s3://xx-xx-xx/xx/xxxx/xxx/xxx/merged already exists > > 14/08/04 11:39:24 ERROR tool.MergeTool: Encountered IOException running > import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output > directory s3://xx-xx-xx/xx/xxxx/xxxx/xxxx/merged already exists > > at > org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146) > > at > org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456) > > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342) > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) > > at > org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286) > > at > org.apache.sqoop.mapreduce.JobBase.runJob(JobBase.java:312) > > at > org.apache.sqoop.mapreduce.MergeJob.runMergeJob(MergeJob.java:138) > > at org.apache.sqoop.tool.MergeTool.run(MergeTool.java:57) > > at org.apache.sqoop.Sqoop.run(Sqoop.java:145) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) > > at org.apache.sqoop.Sqoop.main(Sqoop.java:238) > > > > 11:39:24: sqoopMergeStatus 1 > > 11:39:24: 0 > > 11:39:24: sqoop Merge Failed for XXXX > > > > Any idea, how to fix it? > > > ________________________________ > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise confidential information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. Where allowed by > local law, electronic communications with Accenture and its affiliates, > including e-mail and instant messaging (including content), may be scanned > by our systems for the purposes of information security and assessment of > internal compliance with Accenture policy. > ______________________________________________________________________________________ > > www.accenture.com
