[ https://issues.apache.org/jira/browse/HADOOP-13261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15705095#comment-15705095 ]
Steve Loughran commented on HADOOP-13261: ----------------------------------------- it'd be expensive to query though, as you do not get this information back on a LIST; you'd need to do a HEAD on the file —way too expensive to use in split calculation. Where it could be used is in copy and perhaps an s3-specific distcp, where the partition size could be propagated > save partition split size on multipart uploads > ---------------------------------------------- > > Key: HADOOP-13261 > URL: https://issues.apache.org/jira/browse/HADOOP-13261 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 2.8.0 > Reporter: Steve Loughran > Priority: Minor > > On multipart uploads, save the split size as a metadata value. This would > allow split calculation optimized for the partitions to be performed in some > bulk operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org