[
https://issues.apache.org/jira/browse/HADOOP-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957579#comment-16957579
]
Gopal Vijayaraghavan commented on HADOOP-16629:
-----------------------------------------------
bq. we are adding a new API call which should be implementable by all object
stores.
This should be implementable by any object store which has independent buckets
over a single data volume (so Azure, S3, GCS and Ozone).
The equivalents would be the underpinnings of S3 sync, Azcopy and gsutil copy -
all of these work presently, but require us to leave hadoop tooling to use
them, which has serious issues as you mentioned with tokens and s3guard in
particular.
As [~aengineer] might be able to say for sure, but Ozone also can support the
native copying within itself because of the separation between the namespace
and blockspace.
Going back to an more "original implementation", this mirrors a federated
namenode with a common blockpool which resembles the same split between data
storage and namespaces. If /tmp was a different namenode than /user, then the
fact that there are paths is purely coincidental and the actual movement is a
namespace exchange between two different namenodes.
bq. That includes encryption, s3guard, delegation tokens and other advanced
features.
S3guard is one of good reasons I think this API needs to be in Hadoop rather
than forking a process out to run "s3 sync". The encryption problems are not
specific to this API, because it is equally applicable here.
Those particular problems aren't solved by ignoring them, but they are also not
solved by forcing a ViewFS + path mounts as a workaround for what you propose.
> support copyFile in s3a filesystem
> ----------------------------------
>
> Key: HADOOP-16629
> URL: https://issues.apache.org/jira/browse/HADOOP-16629
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.1
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
