[
https://issues.apache.org/jira/browse/HDFS-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu reassigned HDFS-14190:
-------------------------------
Assignee: Aihua Xu
> Copying folders containing = - characters between hdfs (using webhdfs) does
> not work in distcp
> ----------------------------------------------------------------------------------------------
>
> Key: HDFS-14190
> URL: https://issues.apache.org/jira/browse/HDFS-14190
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
> Affects Versions: 3.1.1
> Reporter: yinsong
> Assignee: Aihua Xu
> Priority: Major
>
> Copying folders containing = - characters between hdfs (using webhdfs) does
> not work in distcp
> for example:
> src:hadoop2.7 target:hadoop3.1.1
> (1)
> hadoop distcp \
> -pugp \
> -i \
> webhdfs://1.1.1.1:50070/sudiyi_datawarehouse
> webhdfs://2.2.2.2:50070/sudiyi_datawarehouse
> ERROR tools.SimpleCopyListing: FileNotFoundException exception in listStatus:
> File /sudiyi_datawarehouse/st_device_standard_ds/date_time%3D2018-10-10 does
> not exist
>
> (2)
> hadoop distcp \
> -Dmapreduce.framework.name=yarn \
> -pugp \
> -i \
> webhdfs://1.1.1.1:50070/druid webhdfs://2.2.2.2:50070/druid
> Error: java.io.IOException: File copy failed:
> webhdfs://10.26.93.65:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800
> -->
> webhdfs://10.27.234.198:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800
> at
> org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:259)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:217)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:48)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> Caused by: java.io.IOException: Couldn't run retriable-command: Copying
> webhdfs://10.26.93.65:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800
> to
> webhdfs://10.27.234.198:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800
> at
> org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:101)
> at
> org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:256)
> ... 10 more
> Caused by: java.io.IOException: Failed to promote
> tmp-file:webhdfs://10.27.234.198:50070/druid/.distcp.tmp.attempt_1545990837043_0016_m_000015_2
> to:
> webhdfs://10.27.234.198:50070/druid/indexing-logs/kill_task-myapp_V1-2018-04-26T16_20_55+0800
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.promoteTmpToTarget(RetriableFileCopyCommand.java:250)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:140)
> at
> org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:99)
> at
> org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
