[ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-8545: ----------------------------------- Attachment: HADOOP-8545-23.patch Updated patch with lots of improvements, including # delete reworked to be less recursive and more efficient on network IO. (one GET + many DELETE) # throttling option for bulk side-effecting HTTP ops in {{delete()}} and {{rename()}} -as recursive delete/rename operations can encounter throttling on some OpenStack clusters. # duration counters on all HTTP operations, with the ability to extract these from the {{SwiftNativeFileSystem()}} and so identify high-cost operations. (HEAD, apparently). Partitioned file uploads adds # partitioned upload of multi-block data added in a single {{write()}} call; # support for >10 partitions in a file # ability to probe #of partitions uploaded (for tests). # delete logic that distinguishes partitioned file (non-zero file with 1+ multiple children) from directory (zero-byte file with 0+ children) and so permits delete of partitioned file even on a non-recursive delete. # tests for all of this. There's two big things outstanding # Rename to support partitioned files. The OpenStack COPY operation copies the metadata that represents a renamed file, but as it refers to the original blocks, which are deleted during the rename, the aggregate file goes away, even though the partitions are successfully moved. # HP Cloud auth. Dmitry has a patch for this that is yet to be integrated. > Filesystem Implementation for OpenStack Swift > --------------------------------------------- > > Key: HADOOP-8545 > URL: https://issues.apache.org/jira/browse/HADOOP-8545 > Project: Hadoop Common > Issue Type: New Feature > Components: fs > Affects Versions: 2.0.3-alpha, 1.1.2 > Reporter: Tim Miller > Assignee: Dmitry Mezhensky > Labels: hadoop, patch > Attachments: HADOOP-8545-10.patch, HADOOP-8545-11.patch, > HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch, > HADOOP-8545-15.patch, HADOOP-8545-16.patch, HADOOP-8545-17.patch, > HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-1.patch, > HADOOP-8545-20.patch, HADOOP-8545-21.patch, HADOOP-8545-22.patch, > HADOOP-8545-23.patch, HADOOP-8545-2.patch, HADOOP-8545-3.patch, > HADOOP-8545-4.patch, HADOOP-8545-5.patch, HADOOP-8545-6.patch, > HADOOP-8545-7.patch, HADOOP-8545-8.patch, HADOOP-8545-9.patch, > HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, HADOOP-8545.patch > > > ,Add a filesystem implementation for OpenStack Swift object store, similar to > the one which exists today for S3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira