[Openstack] [Swift] Use JSON in object xattr
Hello, Now in Swift object server (swift.object.server.DiskFile) for serialization objects meta data used python cPickle module. So the question is why not use other portable serialization format for this, for example JSON? Thanks, Victor ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Swift] Unresolved reference in swift: test/unit/proxy/test_server.py
Hello I found unresolved reference for exception MemcacheLockError (test/unit/proxy/test_server.py line 3580) in Swift unit test for proxy server. I try to find this exception class on swift code, but can't find it. https://github.com/openstack/swift/blob/master/test/unit/proxy/test_server.py#L3580 Thanks, Victor ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] LFS patch (Ia32c9c34)
Hello I think, that changes that I made can be used by you ZaitcevFS all you need for this is just add support for it to LFS middleware or build your own middleware. And your middleware maybe will require special hooks (setup_partition) for prepare file system for files. 17.07.2012 20:40, Pete Zaitcev пишет: On Mon, 16 Jul 2012 22:45:48 + Victor Rodionovvictor.rodio...@nexenta.com wrote: Most of patch code was restructured, most of logic was moved to middleware level and use hooks in Swift code. I create separate project (LFS middleware https://github.com/nexenta/lfs) for now there are only 2 supported file system types (XFS and ZFS) there. Also this middleware provide API for getting file system status information (for example, for ZFS it's current pool status, etc). It sounds strange to make Swift aware of the specific LFS. One day I come up with ZaitcevFS and what happens then? Patching Swift again I presume. I wish Gluster people would stop delaying their competing LFS proposal and finally submit it. Their code was out for a year, but they apparently are content to carry a fork forever: http://review.gluster.com/805 for 1.4.5 http://review.gluster.com/3118 for 1.4.8 -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] LFS patch (Ia32c9c34)
Hello I think, that changes that I made can be used by you ZaitcevFS all you need for this is just add support for it to LFS middleware or build your own middleware. And your middleware may use special hooks (setup_partition) for prepare file system for files. 17.07.2012 20:40, Pete Zaitcev пишет: On Mon, 16 Jul 2012 22:45:48 + Victor Rodionovvictor.rodio...@nexenta.com wrote: Most of patch code was restructured, most of logic was moved to middleware level and use hooks in Swift code. I create separate project (LFS middleware https://github.com/nexenta/lfs) for now there are only 2 supported file system types (XFS and ZFS) there. Also this middleware provide API for getting file system status information (for example, for ZFS it's current pool status, etc). It sounds strange to make Swift aware of the specific LFS. One day I come up with ZaitcevFS and what happens then? Patching Swift again I presume. I wish Gluster people would stop delaying their competing LFS proposal and finally submit it. Their code was out for a year, but they apparently are content to carry a fork forever: http://review.gluster.com/805 for 1.4.5 http://review.gluster.com/3118 for 1.4.8 -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Swift] LFS patch (Ia32c9c34)
Hello I've submit patch (https://review.openstack.org/#/c/7101/), that help Swift use special features of file system on that it working. One of the changes in this patch is for reduce number of network replicas of partition if user use self-repairing mirrored device. For this user should add mirror_copies parameter to each device. By default mirror_copies for all devices is 1, so changes of code don't take any effect for current Swift deployments. For almost all systems three singleton replicas can be replaced by two mirrored replicas. So if all user devices is mirrored (mirror_copies = 2), then number of network copies of most partition will be reduced, and then for operation like PUT and POST we will make less request. The definition of mirroring specifically requires the local file system detect the bad replica on its own, such as by calculating checksums of the content, and automatically repairing data defects when discovered. So if one of devices fail recovery will be done by file system without coping data from other device. This changes was made in ring builder and take effect if mirror_copies 1, so this code is not danger for current Swift users, but for other users can provide new possibility. Also this patch add hooks, that can be used for manipulation with file system, when Swift operate with account, container or object files. This hooks used by middleware that is separate project, so if user don't install it this changes will not take effect. This feature only enabled by customers that have chosen to install the enabling software and turn it on and it is easy to test that this patches have no impact on the generic deployments. Most of patch code was restructured, most of logic was moved to middleware level and use hooks in Swift code. I create separate project (LFS middleware https://github.com/nexenta/lfs) for now there are only 2 supported file system types (XFS and ZFS) there. Also this middleware provide API for getting file system status information (for example, for ZFS it's current pool status, etc). Further the Nexenta side-project is not the only local file system that could provide this form of local replication and data protection.Trading off between network replication and local replication is a valid performance decision. Insisting on a fixed amount of network replication without regard to the degree of local protection provided against data loss would effectively bias the system towards network replication only. Why pay for local data protection if you cannot reduce the amount you pay for network replication? This patch enables solutions that widen the range of choices available to users as to how they protect their data. Thanks, Victor Rodionov ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Swift] Problem with merging
Hello I have patch https://review.openstack.org/#/c/7565/ (Patch for Swift Solaris (illumos) compatibility), that was approved, but during merging something goes wrong. It seems it's jenkins preblem, can someone whe responsible for this, start merging again please. Thanks, Victor ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Swift] S3 like ACL for Swift
Hello I have working implementation of S3 like ACL API for Swift, for this changes I need to store ACL on object and container server, then I need to change container and object servers code. So my question, if this changes will be interesting for Swift community or no? Thanks, Victor ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] S3 like ACL for Swift
Hello My implementation need only remote user header from authorization system with user name (comma separated list of username (roles)) or put key REMOTE_USER to WSGI env. So it will work ok with tempauth, because it add REMOTE_USER to env. For keystone no much changes will be needed. Unfortunately, as I said before, I don't have any idea how it can be implemented as middleware. BTW, I also have changes for swift3 middleware for support S3 API to change ACL, so users can use S3 clients (I test this with Cyberduck) for manipulation with container and object ACL. Also, I want ask do you think it's good idea to store object ACL in object metadata? 20.06.2012, в 19:17, John Dickinson написал(а): Yes, this could be good for swift. ACLs in swift do need to be stored in swift (for scale reasons), but their implementation is dependent on the particular auth system that you are using. The auth middleware is responsible for determining if a request is granted access to a particular swift entity. How does your implementation work with the current ACL support provided by tempauth and swauth? Are your ACLs compatible with the RBAC work being done in keystone? I would suggest that general, full-featured ACL support should be done in conjunction with the work done in keystone and the swift-keystone middleware. If your implementation is simply more full-featured S3 compatibility, I'd suggest patching the 3rd party swift3 middleware. --John On Jun 20, 2012, at 9:38 AM, Victor Rodionov wrote: Hello I have working implementation of S3 like ACL API for Swift, for this changes I need to store ACL on object and container server, then I need to change container and object servers code. So my question, if this changes will be interesting for Swift community or no? Thanks, Victor ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Running review requests code on jenkins.openstack.org
Hello How I can run my code on Openstack jenkins server, for verifie it? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Error with Swift test.functional tests
Hello I faced with error in test.functional, this a console log vito@vito:~/etalon/swift/test$ nosetests functional .E...E... == ERROR: testMetadataNumberLimit (test.functional.tests.TestFile) -- Traceback (most recent call last): File /home/vito/etalon/swift/test/functional/tests.py, line 1017, in testMetadataNumberLimit self.assert_(file.write()) File /home/vito/etalon/swift/test/functional/swift.py, line 731, in write raise ResponseError(self.conn.response) ResponseError: 503: Internal Server Error == ERROR: testMetadataNumberLimit (test.functional.tests.TestFileUTF8) -- Traceback (most recent call last): File /home/vito/etalon/swift/test/functional/tests.py, line 1017, in testMetadataNumberLimit self.assert_(file.write()) File /home/vito/etalon/swift/test/functional/swift.py, line 731, in write raise ResponseError(self.conn.response) ResponseError: 503: Internal Server Error -- Ran 153 tests in 67.563s FAILED (errors=2) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Code contribution
Hello How code in gerrit becomes verified? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] HTTP status value naming normalization
Hello It's very cool. I send review request to review.openstack.org this a link to it https://review.openstack.org/#/c/6775/ Thank you, Victor 2012/4/23 Doug Hellmann doug.hellm...@dreamhost.com: This sounds like a good candidate to live in openstack-common, rather than being limited to Swift. On Sat, Apr 21, 2012 at 12:22 PM, John Dickinson m...@not.mn wrote: I like what you are trying to do here. Can you please submit this as a patch through gerrit so we can get the rest of the core devs to look at it? --John On Apr 20, 2012, at 12:14 PM, Victor Rodionov wrote: There are many place in Swift code where used hard coded values, such as response statuses (200, 201, 404, ...) which can replaced with constants HTTP_OK, HTTP_CREATED, HTTP_NOT_FOUND. Also there is widlly used idiom 200 = status 300, that can be replaced as well with something like this is_success(status). I want add modules for defining all required constants (Swift, HTTP). So I think this changes will improve Swift code readability. PS: this is an initial changes in github https://github.com/vitoordaz/swift/commit/7163d5df13ceaf8fc7b53ba812fe16bd7dd31131 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] HTTP status value naming normalization
There are many place in Swift code where used hard coded values, such as response statuses (200, 201, 404, ...) which can replaced with constants HTTP_OK, HTTP_CREATED, HTTP_NOT_FOUND. Also there is widlly used idiom 200 = status 300, that can be replaced as well with something like this is_success(status). I want add modules for defining all required constants (Swift, HTTP). So I think this changes will improve Swift code readability. PS: this is an initial changes in github https://github.com/vitoordaz/swift/commit/7163d5df13ceaf8fc7b53ba812fe16bd7dd31131 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
