Re: [Openstack] Improved browser-based access to Swift

2013-07-18 Thread John Dickinson
Those both sound pretty cool. If you submit a patch (via gerrit) to the https://github.com/openstack/swift/blob/master/doc/source/associated_projects.rst document, then you can have these projects listed on http://docs.openstack.org/developer/swift/associated_projects.html --John On Jul

Re: [Openstack] [Swift] Breakpoint resume with tempurl

2013-07-11 Thread John Dickinson
Swift supports Range requests, so you are able to make a GET request with the Range header starting with the first byte you didn't fetch the first time. --John On Jul 10, 2013, at 8:10 PM, Jonathan Lu jojokur...@gmail.com wrote: Hi, all stackers, I have realized downloading large

Re: [Openstack] Debug level / Swift

2013-07-03 Thread John Dickinson
You can set log_level = DEBUG in the [DEFAULT] section of your config files. https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L19 --John On Jul 3, 2013, at 7:37 AM, CHABANI Mohamed El Hadi chabani.mohamed.h...@gmail.com wrote: Hi guys, I want to active the

Re: [Openstack] Availability of metrics from SWIFT - Object Storage

2013-06-27 Thread John Dickinson
Swift itself also reports well over 130 unique metrics about itself via StatsD, including the bandwidth for each client request (like ceilometer does). You can monitor these with any standard statsd listener, and monitoring this data doesn't require integration with any other openstack project

Re: [Openstack] [OpenStack] SWIFT Object Store spanning multiple data centers

2013-06-16 Thread John Dickinson
The global clusters feature is Swift is very new and just now being finished up. We are finishing up the last part of it and will have it completed in our next release (tentatively scheduled for June 27). The last part is the affinity write (ie don't write to a WAN region). The regions concept

Re: [Openstack] Load Balancers for Swift with nginx and pound

2013-06-14 Thread John Dickinson
Also, just as general info, don't use nginx in front of Swift. nginx buffers request bodies, and that can become very problematic when uploading content into the Swift cluster (especially since the body could be up to 5GB--not too many of those requests and you'll overload your nginx box).

Re: [Openstack] Folsom Keystone and Grizzly Swift

2013-06-11 Thread John Dickinson
Yes it will work, but I would not recommend it for production systems. There were some changes in Keystone during the Grizzly release that are pretty much required for Keystone to be usable with Swift (specifically around Keystone's ability to support token caching). My recommendation would be

Re: [Openstack] Folsom Keystone and Grizzly Swift

2013-06-11 Thread John Dickinson
and should be compatible with folsom and have the fix you need for caching (the cache=swift.cache parameter to have in auth_token). You will need a recent version of swift tho to get everything working well. On Tue, Jun 11, 2013 at 4:39 PM, John Dickinson m...@not.mn wrote: Yes it will work

Re: [Openstack] [Swift] Upgrade Swift from ver1.4 to ver1.8

2013-06-10 Thread John Dickinson
This is quite possible to do, and in fact one thing that we keep in mind with every Swift release: deployers must be able to upgrade without any lapse in availability. The basic steps are: - stop background processes - upgrade packages (system and/or swift) - restart processes - start

Re: [Openstack] curl with swift

2013-06-10 Thread John Dickinson
Sure, easy to do (curl is what I normally use anyway). To auth (for auth v1. v2 and keystone will be different): curl -i -H X-Auth-User: foo -H X-Auth-Key: bar http://swift/auth/v1.0 The 2 headers you need to look for are X-Storage-URL and X-Auth-Token. After that, use the X-Auth-Token to talk

Re: [Openstack] [Swift] Swift load balancing

2013-06-07 Thread John Dickinson
The given options (DNS, SW load balancer, and HW load balancer) are all things I've seen people use in production Swift clusters. As mentioned in another reply, DNSRR isn't really load balancing, but it can be used if nothing else is available. One thing to consider when choosing a load

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread John Dickinson
On May 31, 2013, at 4:53 PM, Luse, Paul E paul.e.l...@intel.com wrote: I’m looking at tacking this item: https://blueprints.launchpad.net/swift/+spec/support-storage-server-redirects and wanted to get some feedback on the following observations/thoughts: 1) This is a capability that

Re: [Openstack] Understanding Failure Recovery in Swift

2013-05-09 Thread John Dickinson
Replication will be logged and you can look for a Object replication complete log message. For a more detailed look at how swift handles failures, you can watch this video: http://mirror.linux.org.au/linux.conf.au/2013/mp4/Playing_with_OpenStack_Swift.mp4 (I need to post my blog post that has

Re: [Openstack] Configuring tempurl in Openstack Swift

2013-04-29 Thread John Dickinson
There are no plans to change the current implementation of tempurl. In order to generate a tempurl that can be validated without relying on a network call to a centralized authority service, the owner and the service must have a shared secret. In the tempurl feature, this shared secret is set

Re: [Openstack] [DevStack] Does a Swift/Keystone only install require AMQP?

2013-02-19 Thread John Dickinson
nothing in swift requires rabbit, qpid, or zeromq --john On Feb 19, 2013, at 4:53 PM, Everett Toews everett.to...@rackspace.com wrote: Hi All, When I was doing a Swift/Keystone only install with DevStack I used the following in my localrc disable_all_services enable_service key

Re: [Openstack] [SWIFT] code reading

2013-02-13 Thread John Dickinson
This part of the code gets the account info, but if the account isn't found and account autocreation is set (eg for tempauth and keystone), then we need to create the account first before returning the info. However, since there are multiple replicas of the account in the cluster, we can't

Re: [Openstack] [swift]

2013-02-11 Thread John Dickinson
This is the result of `python ./setup.py develop`. That command sets up the scripts to reference your local source so that active dev work is immediately reflected. `python ./setup.py install` would actually copy the scripts to the right locations. --John On Feb 11, 2013, at 9:02 AM, Kun

[Openstack] [Swift] Swift 1.7.6 and python-swiftclient 1.3.0 released

2013-01-24 Thread John Dickinson
I'm happy to announce that Swift 1.7.6 has been released. This release is the work of twenty five contributors. As always, you can upgrade to this release with no downtime for your clients. You can find download links at https://launchpad.net/swift and the Launchpad release tracking at

[Openstack] [Swift] Candidate Swift 1.7.6 cut

2013-01-18 Thread John Dickinson
We've cut he milestone-proposed branch for Swift 1.7.6. It's scheduled to be released next Thursday January 24. Please take a look and let us know of any issues you find ASAP. Candidate tarball: http://tarballs.openstack.org/swift/swift-milestone-proposed.tar.gz The full proposed changelog

Re: [Openstack] [SWIFT] Change the partition power to recreate the RING

2013-01-14 Thread John Dickinson
graycol.gifJohn Dickinson ---01/11/2013 04:28:47 PM---If effect, this would be a complete replacement of your rings, and that is essentially a whole new c From: John Dickinson m...@not.mn To: Alejandro Comisario alejandro.comisa...@mercadolibre.com, Cc: openstack-operat

Re: [Openstack] [SWIFT] Change the partition power to recreate the RING

2013-01-11 Thread John Dickinson
If effect, this would be a complete replacement of your rings, and that is essentially a whole new cluster. All of the existing data would need to be rehashed into the new ring before it is available. There is no process that rehashes the data to ensure that it is still in the correct

[Openstack] [Swift] new regular meeting for Swift

2013-01-08 Thread John Dickinson
I've scheduled a bi-weekly Swift team meeting for Swift contributors. Starting tomorrow, we will meet in #openstack-meeting every other Wednesday at 11am Pacific, 1pm Central, 1900 UTC. For tomorrow's meeting, we will be discussing Swift's next release and the current outstanding patches

Re: [Openstack] swift -- object layout on storage

2013-01-04 Thread John Dickinson
It's pretty simple. Swift uses the underlying filesystem to store the data on disk, and so you can use normal FS tools to find and inspect your data. For the object server, the magic happens here: https://github.com/openstack/swift/blob/master/swift/obj/server.py#L117 The end result is that

Re: [Openstack] Error while enabling s3 support in swift

2012-11-28 Thread John Dickinson
. -- Shashank Sahni On Tue, Nov 20, 2012 at 12:55 PM, John Dickinson m...@not.mn wrote: check out the README at https://github.com/fujita/swift3 for the correct proxy server config section. --john On Nov 19, 2012, at 11:22 PM, Shashank Sahni shredde...@gmail.com wrote: Hi, I'm

Re: [Openstack] PoC Swift

2012-11-19 Thread John Dickinson
You can run your swift cluster with 2 replicas. This is set when the ring is first created. You can use `swift-ring-builder` (ie with no arguments) to get usage help. Note that there is not (yet) an easy way to change your replica count once the ring has been created. If you have more than one

Re: [Openstack] Error while enabling s3 support in swift

2012-11-19 Thread John Dickinson
check out the README at https://github.com/fujita/swift3 for the correct proxy server config section. --john On Nov 19, 2012, at 11:22 PM, Shashank Sahni shredde...@gmail.com wrote: Hi, I'm trying to install swift 1.7.4 on Ubuntu 12.04. In order to enable the s3 support, I added the

Re: [Openstack] [Swift] account-level and container-level usage information

2012-11-13 Thread John Dickinson
You could use a project like slogging (http://github.com/notmyname/slogging) to run inside the cluster and aggregate that information from account dbs and logs. --John On Nov 13, 2012, at 12:15 PM, Pete Zaitcev zait...@redhat.com wrote: On Mon, 12 Nov 2012 20:48:35 -0800 Ning Zhang

Re: [Openstack] Running other services on swift nodes

2012-11-04 Thread John Dickinson
From the Swift perspective, there isn't any reason other services can't be run on the Swift boxes. I'd check for a few things, though. 1) Make sure dependencies aren't in conflict. Thanks to the work of the CI team, this should be mostly sane. 2) Obviously, monitor your systems and don't

Re: [Openstack] [SWIFT] Rack-awareness

2012-11-01 Thread John Dickinson
This is already supported in Swift with the concept of availability zones. Swift will place each replica in different availability zones, if possible. If you only have one zone, Swift will place the replicas on different machines. If you only have one machine, Swift will place the replicas on

Re: [Openstack] [SWIFT] Rack-awareness

2012-11-01 Thread John Dickinson
anything like it? If not, im wondering how, for example rackspace handles this kind of issues ( ignoring all the CDN thing ) Cheers. --- Alejandrito On Thu, Nov 1, 2012 at 12:55 PM, John Dickinson m...@not.mn wrote: This is already supported in Swift with the concept

Re: [Openstack] Nova middleware for enabling CORS?

2012-10-30 Thread John Dickinson
Since the CORS support in Swift allows the preflight OPTIONS response to be different on a per-container basis (which is correct in a multi-tenant system), the CORS support was added directly into Swift's proxy server rather than as middleware. In order to fulfill the OPTIONS request, container

Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers

2012-10-29 Thread John Dickinson
, John Dickinson m...@not.mn wrote: A 507 is returned by the object servers in 2 situations: 1) the drives are full or 2) the drives have been unmounted because of disk error. It's highly likely that you simply have full drives. Remember that the usable space in your cluster is 1/N where N

Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers

2012-10-27 Thread John Dickinson
A 507 is returned by the object servers in 2 situations: 1) the drives are full or 2) the drives have been unmounted because of disk error. It's highly likely that you simply have full drives. Remember that the usable space in your cluster is 1/N where N = replica count. As an example, with 3

[Openstack] Swift 1.7.5 release plan

2012-10-25 Thread John Dickinson
Fast on the heels of a productive summit in San Diego, we are getting ready to release Swift 1.7.5. Our current schedule is to cut the QA release on November 5 and, assuming it passes all QA tests, prepare the final release on November 8. This is quite a solid release with a ton of bug fixes

Re: [Openstack] [SWIFT] Proxies Sizing for 90.000 / 200.000 RPM

2012-10-24 Thread John Dickinson
Sorry for the delay. You've got an interesting problem, and we were all quite busy last week with the summit. First, the standard caveat: Your performance is going to be highly dependent on your particular workload and your particular hardware deployment. 3500 req/sec in two different

Re: [Openstack] [SWIFT] Proxies Sizing for 90.000 / 200.000 RPM

2012-10-24 Thread John Dickinson
are so i can know where to tune in case i need to. Thanks ! --- Alejandrito On Wed, Oct 24, 2012 at 3:28 PM, John Dickinson m...@not.mn wrote: Sorry for the delay. You've got an interesting problem, and we were all quite busy last week with the summit. First, the standard caveat: Your

Re: [Openstack] [openstack-dev] [Swift] community meeting Oct 1

2012-10-01 Thread John Dickinson
Reminder for today's meeting. On Sep 24, 2012, at 10:59 PM, John Dickinson m...@not.mn wrote: As we finish up Folsom and head into the Grizzly summit, I'd like to have a Swift community meeting. Who: The Swift community (users, deployers, contributors, core devs) When: October 1, 2012

Re: [Openstack] Swift reliability

2012-09-26 Thread John Dickinson
The 404s on object PUTs are probably related to the timeout errors you are seeing on the container servers. This may be because of IO contention on your hardware (eg overtaxed drives). How does the disk IO look on your physical hardware? The disk full errors may be because you are running out

[Openstack] [Swift] community meeting Oct 1

2012-09-25 Thread John Dickinson
As we finish up Folsom and head into the Grizzly summit, I'd like to have a Swift community meeting. Who: The Swift community (users, deployers, contributors, core devs) When: October 1, 2012 at 8pm (UTC), 3pm (Central), 1pm (Pacific) Where: #openstack-meeting on freenode (IRC) Agenda:

Re: [Openstack] [swift] make swift.common.utils.streq_const_time more efficient

2012-09-13 Thread John Dickinson
The intended purpose of this string comparison is to explicitly compare every character. Doing it this way guards against timing attacks (http://en.wikipedia.org/wiki/Timing_attack). --John On Sep 13, 2012, at 12:06 AM, Mike Green iasy...@gmail.com wrote: def streq_const_time(s1, s2):

[Openstack] [swift] 1.7.0-final pushed

2012-09-12 Thread John Dickinson
the Essex release, but I wanted to publicly thank everyone who contributed to Swift during the Folsom release cycle. The following people have code contributions in Swift during the Folsom release cycle. Thanks again! Greg Holt John Dickinson Darrell Bishop Samuel Merritt Florian Hines David Goetz

Re: [Openstack] [OpenStack][Swift][Replicator] Does object replicator push exist object to handoff node while a node/disk/network fails ?

2012-09-06 Thread John Dickinson
you can force a replicator to push to a handoff node by unmounting the drive one of the primary replicas is on. --John On Sep 6, 2012, at 9:00 AM, Kuo Hugo tonyt...@gmail.com wrote: Hi folks , John and Chmouel , I did post a question about this long time ago. And my test result is match

[Openstack] Swift PTL candidacy

2012-09-04 Thread John Dickinson
I am running to continue my position as Swift PTL. I have been involved with swift since the project started. I am an active contributor, reviewer, and community participant. I have lead meetups about swift, given conference presentations on swift, and am active in IRC helping those who have

[Openstack] [swift] important upgrade note for swift deployers

2012-08-28 Thread John Dickinson
Swift 1.7 = The next release of Swift will be version 1.7. This will be our release for OpenStack Folsom, and is scheduled land mid-September. There is an important change for deployers in this release. This email has the details so you can begin planning your upgrade path. What's the

Re: [Openstack] Question about undo delete in Swift with Object Version

2012-08-22 Thread John Dickinson
With the current implementation of versioning in Swift, this isn't possible. It's better to think of the feature as versioned writes. --John On Aug 22, 2012, at 12:13 AM, ZHOU Yuan dunk...@gmail.com wrote: Hi stackers, I'm trying to understand the object version feature in Swift. In the

[Openstack] [Swift] Where do we go from here?

2012-08-14 Thread John Dickinson
Swift has many exciting features coming in the OpenStack Folsom Release this fall, but where do we go from here? What's next for Swift in grizzly? I've got some ideas. I'd like to mention them and see where you the community will take them. I've written up most of them into quick one- line

[Openstack] [Swift] community update and what's coming in Folsom

2012-08-13 Thread John Dickinson
We just released Swift 1.6.0 last Monday ( https://lists.launchpad.net/openstack/msg15505.html ). We've got a lot of great features and improvements in it, and I wanted to take some time to update the wider community about where Swift is. Swift 1.4.8 was included with the last OpenStack realease

Re: [Openstack] Swift + keystone integration

2012-08-11 Thread John Dickinson
Make sure that the endpoint stored in keystone is returning the right hostname/domain name and port (8080 based on your config). --John On Aug 11, 2012, at 12:58 PM, Miguel Alejandro González maggo...@gmail.com wrote: Hello I have 3 nodes with ubuntu 12.04 server and installed openstack

[Openstack] [swift] Operational knowledge sharing

2012-08-10 Thread John Dickinson
In a standard swift deployment, the proxy server is running behind a load balancer and/or an SSL terminator. At SwiftStack, we discovered an issue that may arise from some config parameters in this layer, and we'd like to share it with other swift deployers. Symptom: Users updating metadata

[Openstack] Swift 1.6.0 released

2012-08-06 Thread John Dickinson
I'm happy to announce that Swift 1.6.0 has been released. You can get the tarball at https://launchpad.net/swift/folsom/1.6.0. As always, you can upgrade your production Swift clusters to this new version with no downtime to your clients. The complete changelog for this release is at

[Openstack] next swift release just around the corner

2012-07-23 Thread John Dickinson
The next swift release is scheduled for public release next Monday (July 30). That means we've got a little bit of work to do this week to get it ready. In order to allow Cloud Files QA time to check it, we need to have packages built by the middle of the day Wednesday. This means all

Re: [Openstack] [Swift] LFS patch (Ia32c9c34)

2012-07-18 Thread John Dickinson
Nexenta's LFS patch (https://review.openstack.org/#/c/7524/) has languished for a while, and I'd like to address that. First, thank you for your patch submission. This patch adds new functionality that potentially can allow swift to be deployed in more places. The original version of the

Re: [Openstack] Statsd on SWIFT 1.4.8

2012-07-05 Thread John Dickinson
statsd integration was added in swift 1.5.0 On Jul 5, 2012, at 9:07 AM, Leandro Reox wrote: On swift essex stable 1.4.8, is the capacity to send statistics to a statsd server available? Its not in the docs Regards ___ Mailing list:

Re: [Openstack] [Swift] [Storage node] Lots of timeouts in load test after several hours around 1, 000, 0000 operations

2012-07-01 Thread John Dickinson
I hope you are able to get an answer. I'm traveling this week, so I won't have a chance to look in to it. I hope some of the other core devs will have a chance to help you find an answer. --John On Jul 1, 2012, at 2:03 PM, Kuo Hugo tonyt...@gmail.com wrote: Hi all , I did several

Re: [Openstack] Swift Probetests

2012-06-26 Thread John Dickinson
The probe tests are internal whole-system type of tests that test functionality never exposed through normal integration testing. They exist at the level between unit tests and functional tests. For example, one of the probetests makes sure that asynchronous container updates actually happen.

Re: [Openstack] [Swift] S3 like ACL for Swift

2012-06-20 Thread John Dickinson
Yes, this could be good for swift. ACLs in swift do need to be stored in swift (for scale reasons), but their implementation is dependent on the particular auth system that you are using. The auth middleware is responsible for determining if a request is granted access to a particular swift

Re: [Openstack] [Swift] S3 like ACL for Swift

2012-06-20 Thread John Dickinson
On Jun 20, 2012, at 11:02 AM, Victor Rodionov wrote: Also, I want ask do you think it's good idea to store object ACL in object metadata? I'd suggest looking at container-level ACLs rather than object-level. But either way, the data does need to be stored in the metadata in swift itself.

Re: [Openstack] [Swift] Interested in implementing swift ring builder server

2012-06-19 Thread John Dickinson
It looks like Florian (at Rackspace) is working on that blueprint. He just assigned it to himself. I'm happy to hear that you have some extra devs for swift work. I'd love to help coordinate some swift goals with you. Off the top of my head, here are a few things that could be worked on: 1)

Re: [Openstack] Opening up bug triaging rights

2012-05-14 Thread John Dickinson
On May 14, 2012, at 10:06 AM, Thierry Carrez wrote: Hello everyone, Currently the bug triaging rights for a given PROJECT (ability to set status and importance of bugs, but also ability to nominate a bug for a past series) is restricted to the corresponding PROJECT-bugs team, which is

[Openstack] [Swift] swift news and plans

2012-05-04 Thread John Dickinson
TL;DR: removing code from swift, associated projects doc, swift 1.5.0 I want to let the openstack community know of some recent changes within swift and how those changes will affect the next version of swift. Swift has a growing developer community and a rapidly expanding deployed base. While

Re: [Openstack-poc] 3rd Party APIs

2012-05-03 Thread John Dickinson
On May 3, 2012, at 1:16 PM, Jay Pipes wrote: The term recommended comes with a lot of baggage :) I don't want plugins to be recommended or suggested -- at least by the community; companies should feel free to recommend or suggest whatever they feel is best for their distro or deployment.

Re: [Openstack] [Openstack-operators] Question Regarding Swift Distribution Statistics

2012-05-02 Thread John Dickinson
On your proxy server, use swift-get-nodes to see which servers your object is on. With no arguments or --help you will get usage info. --John On May 2, 2012, at 3:48 PM, Duncan McGreggor wrote: cc'ing openstack list On Wed, May 2, 2012 at 4:45 PM, Richard Raseley rich...@raseley.com

Re: [Openstack] HTTP status value naming normalization

2012-04-21 Thread John Dickinson
I like what you are trying to do here. Can you please submit this as a patch through gerrit so we can get the rest of the core devs to look at it? --John On Apr 20, 2012, at 12:14 PM, Victor Rodionov wrote: There are many place in Swift code where used hard coded values, such as response

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

2012-04-12 Thread John Dickinson
Swift keeps total bytes, container, and object count (eventually) up-to-date in the account metadata. There are also log processing tools (like slogging - http://github.com/notmyname/slogging) that can provide usage information (including bandwidth) based on swift logs. While I think that it's

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

2012-04-12 Thread John Dickinson
I should also mention the summit session talking about this very topic led by Everett Toews. It's (currently) scheduled for 9am on wednesday. http://summit.openstack.org/sessions/view/81 --John On Apr 12, 2012, at 8:51 PM, John Dickinson wrote: Swift keeps total bytes, container

Re: [Openstack] Swift: NAS or DAS?

2012-03-16 Thread John Dickinson
Generally, you would introduce latency in the storage system by using a NAS attached to a storage drive. Also, at scale, your costs will be dominated by drive, so you will want to optimize the storage nodes for dense, cheap storage. --John On Mar 16, 2012, at 8:32 AM, Michaël Van de Borne

Re: [Openstack] swift and two data-centres - hierarchical zones?

2012-03-12 Thread John Dickinson
On Mar 10, 2012, at 5:58 AM, John Leach wrote: I think what I need here is hierarchical zones - I'd define one parent zone per data-centre, and then multiple child zones within each (representing racks or whatever). Swift would be configured to write 3 replicas in 3 child zones, aiming

Re: [Openstack] Swift concept architecture

2012-03-12 Thread John Dickinson
On Mar 11, 2012, at 12:16 AM, Dmitry Ukov wrote: Hi all, I want to introduce some ideas about Swift. Let’s assume we have huge amount of data stored in Swift (e.g. 10Pb). This data are dynamically changed by users. So we need to reduce network load caused by replication and intensive

[Openstack] Swift patches for the essex release

2012-03-07 Thread John Dickinson
The final swift release for openstack essex is coming quickly. Swift 1.4.8 is tentatively scheduled for March 22nd. To get your patches into this release, please submit them for review by next week so we have enough time to review and QA them. --John smime.p7s Description: S/MIME

Re: [Openstack] [Swift] Cannot create container.

2012-02-24 Thread John Dickinson
a 507 response means that the drive was unmounted. If you are running this in a VM (like the SAIO), then you need to disable to mount check. Docs for this are in the SAIO docs. --John On Feb 24, 2012, at 9:13 AM, Leander Bessa wrote: Hello, I'm trying to set up a custom swift server in a

[Openstack] swift authors list

2012-02-24 Thread John Dickinson
Stefano has asked that swift start keeping email addresses in our AUTHORS file. I'll be adding these soon, but some contributors have more than one email. If you have a particular email address you'd like to have me add to your name, please let me know. Otherwise, I will use one from git log.

Re: [Openstack] Swift container ACLs and container visibility question

2012-02-23 Thread John Dickinson
It all depends on the auth system you are using. Below is for swauth and tempauth: Are the users using the same shared storage? If so, set them up as .admin users with the same storage endpoint. If they are not using the same shared storage, then you may be stuck. The ACL support in swauth and

Re: [Openstack] swprobe: swift middleware for sending metrics to graphite using statsd

2012-02-21 Thread John Dickinson
That's great. Have you by any chance seen https://github.com/pandemicsyn/swift-informant? It's something similar that we've been playing with at Rackspace. --John On Feb 21, 2012, at 10:36 AM, Jasper Capel wrote: Hi all, I'm announcing a piece of Swift middleware, swprobe [1], designed

Re: [Openstack] python-swiftclient?

2012-02-13 Thread John Dickinson
On Feb 13, 2012, at 8:29 AM, Chmouel Boudjnah wrote: What do you think if we : - split swift.common.client to its own. - have bin/swift import that package and shipped with it. - have a comprehensive test suite covering the CLI and the library. - have some proper PIP release for all

Re: [Openstack] Doubts about virtual CPUs and Swift storage capacity.

2012-02-11 Thread John Dickinson
On Feb 11, 2012, at 7:29 AM, Jorge Luiz Corrêa wrote: 2) About Swift, how do I determine the total usable storage capacity of the system? For example: I have 3 nodes with 5 HDs of 1 TB each one. Straightly I have 15 TB of space. If I use raid 1 I can say that I'm going to have 7,5 TB of

[Openstack] Swift 1.4.6 release

2012-02-10 Thread John Dickinson
I'm happy to announce that swift 1.4.6 has been released today. We've added some great new features in this release and fixed several outstanding bugs. The full changelog is below, but I'd like to highlight a few key points. Swift 1.4.6 includes new middleware that adds the ability to upload

Re: [Openstack] Swift Container Name Restrictions

2012-01-24 Thread John Dickinson
On Jan 24, 2012, at 5:22 PM, Matthew Wodrich wrote: Hi Folks, I'm trying to write some scripts to work with Swift containers, but I don't actually know what the restrictions on container names are. Does anyone know what the specification is, or where I can read up on it? For example:

Re: [Openstack] Swift supported file system

2012-01-23 Thread John Dickinson
That functionality is left up to a client. For example, you could use FUSE to spoof swift as a filesystem or you could use a client like Cyberduck or even write your own. Last week someone on this mailing list talked about adding webDAV support to swift. All of these work in that they present

Re: [Openstack] Swift supported file system

2012-01-23 Thread John Dickinson
The storage volumes referenced in the ring are identified by an IP, port, and mount point. So, it is possible to use network attached storage for swift (as long as it still supports xattrs). However, I don't know if this has ever really been tried (especially in production), and I'd be

Re: [Openstack] (no subject)

2012-01-19 Thread John Dickinson
look in syslog on your proxy server to see what caused the error. --John On Jan 19, 2012, at 6:28 PM, Khaled Ben Bahri wrote: Hi all, I tryed to install OpenStack swift, after creating and configuring all nodes, when i want to check that swift works, I execute this command : swift

[Openstack] swift essex status update

2012-01-11 Thread John Dickinson
more exciting things to come before our final essex release is made. As always, patches welcome! John Dickinson Swift Project Technical Lead notmyname on IRC smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net

Re: [Openstack] Using Gerrit to verify the CLA

2012-01-09 Thread John Dickinson
https://rackspace.echosign.com/verifier On Jan 9, 2012, at 8:08 AM, Mark McLoughlin wrote: Hey, On Thu, 2012-01-05 at 10:02 -0800, James E. Blair wrote: This change is in place; membership in openstack-cla is required in order to submit changes to Gerrit. All of the -core groups have

Re: [Openstack] Several questions about HOW SWIFT WORKS

2012-01-06 Thread John Dickinson
The best, technical description of the ring was written by the person who had the biggest role in writing it for swift: http://www.tlohg.com/p/building-consistent-hashing-ring.html --John smime.p7s Description: S/MIME cryptographic signature ___

[Openstack] swift release 1.4.5

2012-01-04 Thread John Dickinson
It's time again for a swift release. We have cut the swift 1.4.5 release and it's headed to QA. We expect it to be validated by the end of this week and it should land for public use early next week. Below is the changelog for this release. The highlights are the swift-orphans and swift-oldies

Re: [Openstack] Several questions about HOW SWIFT WORKS

2012-01-03 Thread John Dickinson
Answers inline. On Jan 3, 2012, at 11:32 AM, Alejandro Comisario wrote: So, lets get down to business. # 1 we have memcache service running on each proxy, so as far as we know, memcache actually caches keystone tokens and object paths as the request ( PUT , GET) enters the proxy, but

Re: [Openstack] [RFC] Common config options module

2011-12-06 Thread John Dickinson
Overall, I think it's a great thing to have commonality between the projects on option names and environment variables. I think it's worthwhile to push for that in the swift cli tool in the essex timeframe. On the topic of common config libraries, though, I think the differences are less

Re: [Openstack] rpms for rhel5.x to install Open stack Object Storage

2011-11-21 Thread John Dickinson
I suspect there is a communication gap somewhere, but this is certainly not the case. Openstack Object Storage (swift) is not deprecated. Glance provides a bridge between nova and swift, but all three are important, active projects. Sudhaker, I know that rpms exist for swift, but I don't know

Re: [Openstack] Update Container Metadata.

2011-11-14 Thread John Dickinson
Updating existing container metadata with a POST is entirely supported and intended behavior. (The same also applies to adding/updating metadata on an account.) --John On Nov 14, 2011, at 8:10 PM, easco wrote: Anne, Thank you for your help, and you're correct, that is the case I am

Re: [Openstack] Tutorials of how to install openstack swift into centos 6

2011-11-09 Thread John Dickinson
Awesome. Thanks for putting this together. I know a lot of people have been interested in getting swift running on non-ubuntu systems. Thanks for sharing this with everyone. --John On Nov 9, 2011, at 12:00 AM, pf shineyear wrote: openstack swift install on centos 6 1. proxy install

Re: [Openstack] [Openstack-poc] proposal for policy around and management of client libraries

2011-11-08 Thread John Dickinson
On Nov 8, 2011, at 10:54 AM, Thierry Carrez wrote: With solution (2), if you look at the issue from Gerrit, GitHub, Launchpad or Jenkins, those will be separate projects though. The fact that they share the same PTL is not enough to make them one. For example, they will have separate bug

Re: [Openstack] proposal for policy around and management of client libraries

2011-11-07 Thread John Dickinson
In general, I support the idea of good, supported client libraries. I have a few questions about this particular proposal. 1) Would a distinct python client (and the associated project) be required for each core openstack project that exposes an API? 2) Why does the PPB need to vote? Actually,

Re: [Openstack] proposal for policy around and management of client libraries

2011-11-07 Thread John Dickinson
On Nov 7, 2011, at 4:23 PM, Monty Taylor wrote: 2) Why does the PPB need to vote? Actually, what would the PPB be voting on (assuming the answer to #1 is no)? Well, it would be effectively promoting several existing projects which are managed in a few different places (rackspace, 4P, etc)

Re: [Openstack-poc] [Openstack] proposal for policy around and management of client libraries

2011-11-07 Thread John Dickinson
On Nov 7, 2011, at 4:23 PM, Monty Taylor wrote: 2) Why does the PPB need to vote? Actually, what would the PPB be voting on (assuming the answer to #1 is no)? Well, it would be effectively promoting several existing projects which are managed in a few different places (rackspace, 4P, etc)

Re: [Openstack] +1, All services should have WADLs

2011-10-28 Thread John Dickinson
I am concerned about some of the implications that are being discussed. 1) A WADL is part of documentation of an API. Nobody is going to object to more documentation. 2) Being an open-source project, if somebody wants to commit to creating and maintaining a WADL for a particular part of

Re: [Openstack] +1, All services should have WADLs

2011-10-28 Thread John Dickinson
On Oct 28, 2011, at 10:04 AM, Ed Leafe wrote: Swift had the advantage of starting out as a closed source project that only had to serve a single master, and thus didn't need external orchestration to keep it on track. Nova, OTOH, as a community development effort, essentially had to

Re: [Openstack] New git-review tool ready for people to try

2011-10-14 Thread John Dickinson
I'd much prefer storing data in git config rather than in a dot file. On Oct 14, 2011, at 7:33 AM, Julien Danjou wrote: On Fri, Oct 14 2011, James E. Blair wrote: Another idea though is to add a small dotfile into each repository indicating the canonical location of that repo's gerrit.

Re: [Openstack] New git-review tool ready for people to try

2011-10-14 Thread John Dickinson
, at 8:04 AM, Julien Danjou wrote: On Fri, Oct 14 2011, John Dickinson wrote: I'd much prefer storing data in git config rather than in a dot file. A dot file is commit-able. Git config is not. So in this case, providing the dot file wins. Think about like a .gitignore file. -- Julien

Re: [Openstack] Creating openstack-dev mailing list

2011-10-07 Thread John Dickinson
Jesse Andrews (anotherjesse)(Rackspace) Jonathan Bryce (jbryce)(Rackspace) Devin Carlen (devcamcar)(Nebula) Thierry Carrez (ttx)(Rackspace) John Dickinson (notmyname)(Rackspace)) Vish Ishaya (vishy)(Rackspace) Josh Kearney (jk0)(Rackspace) Joshua McKenty (jmckenty)(Piston) Ewan Mellor

Re: [Openstack] Swift git differences between launchpad vs github milestone-proposed branch history

2011-09-29 Thread John Dickinson
/9ab33970b58b8219245bfd89e2ad9442c0e94f17) $ git log -n1 9ab33970b58b8219245bfd89e2ad9442c0e94f17 commit 9ab33970b58b8219245bfd89e2ad9442c0e94f17 Merge: 8526098 c4f0b55 Author: John Dickinson john.dickin...@rackspace.commailto:john.dickin...@rackspace.com Date: Tue Jun 14 16:37:02 2011 +

Re: [Openstack] 55PB storage cloud hosted using Swift

2011-09-29 Thread John Dickinson
clarification: 5.5PB On Sep 29, 2011, at 3:00 PM, Brian Schott wrote: In case you missed it, SDSC is hosting a commercial 55 petabyte storage cloud using Swift: http://arstechnica.com/business/news/2011/09/supercomputing-center-targets-55-petabyte-storage-at-academics-students.ars

  1   2   >