[
https://issues.apache.org/jira/browse/HADOOP-19309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17888625#comment-17888625
]
ASF GitHub Bot commented on HADOOP-19309:
-----------------------------------------
shameersss1 opened a new pull request, #7113:
URL: https://github.com/apache/hadoop/pull/7113
### Description of PR
When the sourcePath does not contain any file scheme information, S3A
CopyFromLocalFile operation fails. Additionally the failure is seen only when
the config `fs.s3a.optimized.copy.from.local.enabled `is enabled (which is by
default). This happens only when the local source file is given without any
file scheme for example : `/tmp/file.txt` instead of` file:///tmp/file.txt.`
### How was this patch tested?
Added a test case to validate the issue and the fix. Ran the test`
ITestS3ACopyFromLocalFile ` in us-east-1
### For code changes:
- [x ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [x] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> S3A CopyFromLocalFile operation fails when the source file does not contain
> file scheme.
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-19309
> URL: https://issues.apache.org/jira/browse/HADOOP-19309
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.4.0, 3.3.6
> Reporter: Syed Shameerur Rahman
> Assignee: Syed Shameerur Rahman
> Priority: Major
>
> When the sourcePath does not contain any file scheme information, S3A
> CopyFromLocalFile operation fails with the following exception stack trace.
> {code:java}
> at
> org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.getFinalPath(CopyFromLocalOperation.java:360)
> at
> org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.uploadSourceFromFS(CopyFromLocalOperation.java:222)
> at
> org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.execute(CopyFromLocalOperation.java:169)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$copyFromLocalFile$23(S3AFileSystem.java:4217)
> at
> org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
> at
> org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
> at
> org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2871)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2890)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.copyFromLocalFile(S3AFileSystem.java:4209)
> {code}
> Additionally the failure is seen only when
> {color:#172b4d}*fs.s3a.optimized.copy.from.local.enabled* is enabled (which
> is by default). This happens only when the local source file is given without
> any file scheme for example : /tmp/file.txt instead of file:///tmp/file.txt.
> {color}
>
> {color:#172b4d}The proposal here is to add file scheme to the source if the
> source path does not contain the same.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
