Yep, permission problem. Weird though it seems to be moving a file within the same dir. Thanks for the update! - Douglas From: amit kumar <ak3...@gmail.com<mailto:ak3...@gmail.com>> Reply-To: <user@hive.apache.org<mailto:user@hive.apache.org>> Date: Tue, 5 May 2015 04:40:18 +0530 To: <user@hive.apache.org<mailto:user@hive.apache.org>> Subject: Re: Unable to move files on Hive/Hdfs
Hi Doug, I have use CDH 5.2.1 I performed the below task, and getting the error, but after rolling back the below changes issue has been resolved itself. Disable ACLs on Name Nodes Set Enable Access Control Lists = False Save Changes Restart Hadoop Cluster Thanks, On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak3...@gmail.com<mailto:ak3...@gmail.com>> wrote: Hi Doug, I have use CDH 5.2.1 Disable ACLs on Name Nodes Set Enable Access Control Lists = False Save Changes Restart Hadoop Cluster Stack trace: 2015-05-04 10:38:18,820 INFO [main]: exec.Task (SessionState.java:printInfo(537)) - Moving data to: hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000 from hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002 2015-05-04 10:38:18,857 ERROR [main]: exec.Task (SessionState.java:printError(546)) - Failed with exception Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000 org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000 at org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269) at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89) at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553) After rolling those same changes out, the problem resolved itself. On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <douglas.mo...@thinkbiganalytics.com<mailto:douglas.mo...@thinkbiganalytics.com>> wrote: Hi Amit, We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2 release. There are lots of reasons for this though. Can you provide more details about the stack trace and version so we can compare? For our problem we've seen some relief with SET hive.metastore.client.socket.timeout=60s but the problem still happens from time to time. Thanks, Douglas From: amit kumar <ak3...@gmail.com<mailto:ak3...@gmail.com>> Reply-To: <user@hive.apache.org<mailto:user@hive.apache.org>> Date: Tue, 5 May 2015 03:12:15 +0530 To: <user@hive.apache.org<mailto:user@hive.apache.org>> Subject: Unable to move files on Hive/Hdfs While moving the data from hive/hdfs we get below error, Please suggest on this. Moving data to: hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000 Failed with exception Unable to move sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\ 235-1/-ext-10002 to destination hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\ 10000 FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask MapReduce Jobs Launched: Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS Write: 489704 SUCCESS Total MapReduce CPU Time Spent: 5 seconds 830 msec Error (1). Execution Failed. 2015-05-04 10:03:13 ERROR (1) in run_hive Thanks,
