[ https://issues.apache.org/jira/browse/HADOOP-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-13282: ------------------------------------ Status: Open (was: Patch Available) > S3 blob etags to be made visible in status/getFileChecksum() calls > ------------------------------------------------------------------ > > Key: HADOOP-13282 > URL: https://issues.apache.org/jira/browse/HADOOP-13282 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 2.9.0 > Reporter: Steve Loughran > Assignee: Steve Loughran > Priority: Minor > Attachments: HADOOP-13282-001.patch, HADOOP-13282-002.patch > > > If the etags of blobs were exported via {{getFileChecksum()}}, it'd be > possible to probe for a blob being in sync with a local file. Distcp could > use this to decide whether to skip a file or not. > Now, there's a problem there: distcp needs source and dest filesystems to > implement the same algorithm. It'd only work out the box if you were copying > between S3 instances. There are also quirks with encryption and multipart: > [s3 > docs|http://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html]. > At the very least, it's something which could be used when indexing the FS, > to check for changes later. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org