[
https://issues.apache.org/jira/browse/HADOOP-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819373#comment-16819373
]
Gabor Bota commented on HADOOP-16259:
-------------------------------------
Thanks for working on this [~PrakashGopalsamy]!
I took a look at the docs, and noticed that your plan includes changes in s3:
and s3n:. Please be aware that the S3A filesystem client with prefix s3a://, is
the S3 client undergoing active development and maintenance, so s3 and s3n are
deprecated.
Ref.:
https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html#Overview
> Distcp to set S3 Storage Class
> ------------------------------
>
> Key: HADOOP-16259
> URL: https://issues.apache.org/jira/browse/HADOOP-16259
> Project: Hadoop Common
> Issue Type: New Feature
> Components: hadoop-aws
> Affects Versions: 2.8.4
> Reporter: Prakash Gopalsamy
> Priority: Minor
> Labels: aws-s3, distcp
> Attachments: ENHANCE_HADOOP_DISTCP_FOR_CUSTOM_S3_STORAGE_CLASS.docx
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Hadoop distcp implementation doesn’t have properties to override Storage
> class while transferring data to Amazon S3 storage. Hadoop distcp doesn’t set
> any storage class while transferring data to Amazon S3 storage. Due to this
> all the objects moved from cluster to S3 using Hadoop Distcp are been stored
> in the default storage class “STANDARD”. By providing a new feature to
> override the default S3 storage class through configuration properties will
> be helpful to upload objects in other storage classes. I have come up with a
> design to implement this feature in a design document and uploaded the same
> in the JIRA. Kindly review and let me know for your suggestions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
