[
https://issues.apache.org/jira/browse/HADOOP-19309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-19309.
-------------------------------------
Fix Version/s: 3.5.0
Resolution: Fixed
> S3A CopyFromLocalFile operation fails when the source file does not contain
> file scheme.
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-19309
> URL: https://issues.apache.org/jira/browse/HADOOP-19309
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.4.0, 3.3.6
> Reporter: Syed Shameerur Rahman
> Assignee: Syed Shameerur Rahman
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> When the sourcePath does not contain any file scheme information, S3A
> CopyFromLocalFile operation fails with the following exception stack trace.
> {code:java}
> at
> org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.getFinalPath(CopyFromLocalOperation.java:360)
> at
> org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.uploadSourceFromFS(CopyFromLocalOperation.java:222)
> at
> org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.execute(CopyFromLocalOperation.java:169)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$copyFromLocalFile$23(S3AFileSystem.java:4217)
> at
> org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
> at
> org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
> at
> org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2871)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2890)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.copyFromLocalFile(S3AFileSystem.java:4209)
> {code}
> Additionally the failure is seen only when
> {color:#172b4d}*fs.s3a.optimized.copy.from.local.enabled* is enabled (which
> is by default). This happens only when the local source file is given without
> any file scheme for example : /tmp/file.txt instead of file:///tmp/file.txt.
> {color}
>
> {color:#172b4d}The proposal here is to add file scheme to the source if the
> source path does not contain the same.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
