Re: [Openstack] [RFC] OpenStack Programming Model Framework

2011-01-03 Thread John Dickinson
On Jan 3, 2011, at 6:13 PM, John Purrier wrote: API/Service Protocol: RESTful interfaces specific to each service. Each service will present a Public API, a Management API, and optionally, a Notification API. The set and format of the service interface calls define the Service Protocol.

Re: [Openstack] Problems with path, prefix and delimiter with Object Storage

2011-03-17 Thread John Dickinson
A path query requires directory marker objects to be present (http://docs.openstack.org/openstack-object-storage/developer/content/ch03s02.html#d5e514) I would recommend using prefix+delimiter over path. Path query parameters are in swift to support backwards compatibility with the original

Re: [Openstack] Request help choosing a topic to be developed on cloud computing

2011-04-04 Thread John Dickinson
Hello, I do not know if you can help me, but if this is not the case, I apologize in advance. my name is Khalid abbou, Moroccan, I am currently in 3rd yearengineering student at the National Institute of Posts andTelecommunications (INPT), and as part of my final project

Re: [Openstack] [team-entarch] Eto

2011-06-09 Thread John Dickinson
We're all glad you're feeling better. Now get back to work! We've got a cloud to build, and it's not going to build itself. ;-) On Jun 9, 2011, at 12:18 PM, Bryan Taylor wrote: Doh! Sorry for the spam openstack. o now autocompletes to this instead of my project team's list and I didn't

[Openstack] swift 1.4.1 release

2011-06-20 Thread John Dickinson
Swift 1.4.1 has been released. This release contains the following changes: * st renamed to swift * swauth was separated froms swift. It is now its own project and can be found at https://github.com/gholt/swauth. * tempauth middleware added as an extremely limited auth system for dev work. *

Re: [Openstack] [Keystone] [Swift] Keystone Tenant vs Swift Account

2011-07-19 Thread John Dickinson
to Swift? Who or how would the account tombstones be set? Thanks Divyesh -Original Message- From: John Dickinson [mailto:john.dickin...@rackspace.com] Sent: Monday, July 18, 2011 5:03 PM To: Khandeshi, Divyesh Cc: 'openstack@lists.launchpad.net' Subject: Re: [Openstack] [Keystone

[Openstack] swift status (August 1)

2011-08-01 Thread John Dickinson
Just a quick note on swift's status and what we're working on. Last week I gave a talk at OSCON about swift. There was a good turnout, and I hope questions were answered. The slides can be found at http://d.not.mn/swift_overview_oscon2011.pdf. Also, we are working on: 1) Reliability

Re: [Openstack] How to Create Object Folders/Directories

2011-08-05 Thread John Dickinson
You only need a -H 'Content-Length: 0' in that command and it should work. For more infor on nested virtual directories, see http://programmerthoughts.com/programming/nested-folders-in-cloud-files/ --John On Aug 5, 2011, at 4:24 AM, 홍성희 wrote: Hi I now want to create a virtual folder. I

[Openstack] swift update

2011-08-12 Thread John Dickinson
A quick update from the swift team: Since my last update, we added some additional quarantine checks for corrupted objects and a nice ops tool called recon. Recon is middleware that is loaded on the object server and monitors a storage node for various things like async pending files, load,

Re: [Openstack] AuthZ functionality in Keystone - Re: [WAS]OpenStack Identity: Keystone API Proposal

2011-08-18 Thread John Dickinson
On Aug 18, 2011, at 5:50 PM, Vishvananda Ishaya wrote: On Aug 18, 2011, at 3:45 PM, Somik Behera wrote: Hi Vish, That would be one very reasonable way to do it, but in that case we are fragmenting AuthZ in multiple services instead of Keystone taking care of AuthZ across all

[Openstack] swift at the essex summit

2011-08-25 Thread John Dickinson
I beginning to think about the things that need to be covered at the design summit about swift specifically. I've started an etherpad, and I welcome all input. http://etherpad.openstack.org/swift-essex --John ___ Mailing list:

Re: [Openstack] New nova service proposal

2011-08-26 Thread John Dickinson
On Aug 26, 2011, at 6:10 PM, Monsyne Dragon wrote: The proper way, IMHO, for this to work is that a request generates a workorder with a set of tasks. This gets sent to something (scheduler, probably) which looks at the first uncompleted task on the workorder, makes the decision on

[Openstack] swift 1.4.3 (diablo) fast approaches

2011-08-29 Thread John Dickinson
We are fast approaching the next swift release, version 1.4.3. This is the version of swift that will be in the diablo release of openstack. Since swift 1.4.3 is scheduled for next week, all patches that will be included need to be proposed now. We will be doing final dev and QA work this week

[Openstack] Openstack swift code now on github

2011-09-06 Thread John Dickinson
://wiki.openstack.org/GerritWorkflow#Project_Setup). If you have any questions, please join #openstack-dev on freenode and ask. --John Dickinson notmyname ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe

Re: [Openstack] X-Copy-From and Large Object Support

2011-09-07 Thread John Dickinson
A server-side copy will consolidate the file to one object. The new target object has the same limitations as other objects in the system (5GB). --John On Sep 7, 2011, at 5:34 PM, Caitlin Bestler wrote: What is the expected behavior when an object uploaded using Large Object Support is

[Openstack] swift 1.4.3 (diablo) released

2011-09-12 Thread John Dickinson
We have released swift 1.4.3 today. The changelog is below. Baring something we missed in testing, this release is the swift version that will be released as Openstack Diablo. The launchpad landing page for this release is https://launchpad.net/swift/diablo/1.4.3. swift (1.4.3) *

Re: [Openstack] [Swift] [Keystone] Account migration

2011-09-13 Thread John Dickinson
Swift treats the hash or tenantid part of the account as an opaque string. (Technically, the first one isn't a hash but a uuid.) I don't think there is any migration path because there is nothing to migrate. But I may be missing something. --John On Sep 13, 2011, at 2:55 PM, Nguyen, Liem

Re: [Openstack] [Swift] [Keystone] Account migration

2011-09-13 Thread John Dickinson
*accidentally* have 2 Swift accounts instead of one? Ideally, if I can migrate all the old account Ids to use the new format, that would guarantee consistency between the old accounts and any newly-created accounts. Am I missing something here? Liem -Original Message- From: John

Re: [Openstack] [Swift] [Keystone] Account migration

2011-09-13 Thread John Dickinson
@lists.launchpad.net] On Behalf Of John Dickinson Sent: Tuesday, September 13, 2011 4:11 PM To: Nguyen, Liem Manh Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [Swift] [Keystone] Account migration Swift treats the hash or tenantid part of the account as an opaque string. (Technically, the first

Re: [Openstack] Essex sessions

2011-09-16 Thread John Dickinson
Matt, I've been busy with some other work-related things this week, but I hope to focus some more on the summit sessions next week. If you have some ideas of things you'd like to see discussed, please add them. --John On Sep 16, 2011, at 10:51 AM, Matt Colton wrote: I've been looking over

Re: [Openstack] OpenStack Satellite

2011-09-27 Thread John Dickinson
What benefits does an openstack-satellite project bring? Other than all using some openstack component, what do these projects have in common that justifies grouping them? For example, I know of many open source projects that use swift (swauth, slogging, cyberduck, all the rackspace language

Re: [Openstack] Improving meetbot reports (was Re: Community metrics, developers' engagement)

2011-09-27 Thread John Dickinson
process transparent and accessible. The current setup meets the letter of the law, but perhaps not the spirit. --John On Sep 27, 2011, at 7:08 PM, Stefano Maffulli wrote: On Mon, 2011-09-26 at 12:51 -0500, John Dickinson wrote: I too would like to see official community and PPB meeting summaries

Re: [Openstack] OpenStack Satellite

2011-09-27 Thread John Dickinson
So you're looking for a place to point marketing? We already have http://openstack.org/projects/ (which has a long list on it) and the wiki has plenty of space to add new pages, if necessary. However, the original proposal is not about a marketing landing page to promote an openstack ecosystem.

Re: [Openstack] Why does object server's Put have a container method?

2011-09-28 Thread John Dickinson
Remember that there are 3 copies of each container, too. The proxy is setting a header in each PUT to the object server. The object server reads the header to know which container server to update. Failure scenarios are handled better this way. All ok: proxy sends PUT to 3 object servers. All

Re: [Openstack] Swift git differences between launchpad vs github milestone-proposed branch history

2011-09-29 Thread John Dickinson
/9ab33970b58b8219245bfd89e2ad9442c0e94f17) $ git log -n1 9ab33970b58b8219245bfd89e2ad9442c0e94f17 commit 9ab33970b58b8219245bfd89e2ad9442c0e94f17 Merge: 8526098 c4f0b55 Author: John Dickinson john.dickin...@rackspace.commailto:john.dickin...@rackspace.com Date: Tue Jun 14 16:37:02 2011 +

Re: [Openstack] 55PB storage cloud hosted using Swift

2011-09-29 Thread John Dickinson
clarification: 5.5PB On Sep 29, 2011, at 3:00 PM, Brian Schott wrote: In case you missed it, SDSC is hosting a commercial 55 petabyte storage cloud using Swift: http://arstechnica.com/business/news/2011/09/supercomputing-center-targets-55-petabyte-storage-at-academics-students.ars

Re: [Openstack] Creating openstack-dev mailing list

2011-10-07 Thread John Dickinson
Jesse Andrews (anotherjesse)(Rackspace) Jonathan Bryce (jbryce)(Rackspace) Devin Carlen (devcamcar)(Nebula) Thierry Carrez (ttx)(Rackspace) John Dickinson (notmyname)(Rackspace)) Vish Ishaya (vishy)(Rackspace) Josh Kearney (jk0)(Rackspace) Joshua McKenty (jmckenty)(Piston) Ewan Mellor

Re: [Openstack] New git-review tool ready for people to try

2011-10-14 Thread John Dickinson
I'd much prefer storing data in git config rather than in a dot file. On Oct 14, 2011, at 7:33 AM, Julien Danjou wrote: On Fri, Oct 14 2011, James E. Blair wrote: Another idea though is to add a small dotfile into each repository indicating the canonical location of that repo's gerrit.

Re: [Openstack] New git-review tool ready for people to try

2011-10-14 Thread John Dickinson
, at 8:04 AM, Julien Danjou wrote: On Fri, Oct 14 2011, John Dickinson wrote: I'd much prefer storing data in git config rather than in a dot file. A dot file is commit-able. Git config is not. So in this case, providing the dot file wins. Think about like a .gitignore file. -- Julien

Re: [Openstack] +1, All services should have WADLs

2011-10-28 Thread John Dickinson
I am concerned about some of the implications that are being discussed. 1) A WADL is part of documentation of an API. Nobody is going to object to more documentation. 2) Being an open-source project, if somebody wants to commit to creating and maintaining a WADL for a particular part of

Re: [Openstack] +1, All services should have WADLs

2011-10-28 Thread John Dickinson
On Oct 28, 2011, at 10:04 AM, Ed Leafe wrote: Swift had the advantage of starting out as a closed source project that only had to serve a single master, and thus didn't need external orchestration to keep it on track. Nova, OTOH, as a community development effort, essentially had to

Re: [Openstack] proposal for policy around and management of client libraries

2011-11-07 Thread John Dickinson
In general, I support the idea of good, supported client libraries. I have a few questions about this particular proposal. 1) Would a distinct python client (and the associated project) be required for each core openstack project that exposes an API? 2) Why does the PPB need to vote? Actually,

Re: [Openstack] proposal for policy around and management of client libraries

2011-11-07 Thread John Dickinson
On Nov 7, 2011, at 4:23 PM, Monty Taylor wrote: 2) Why does the PPB need to vote? Actually, what would the PPB be voting on (assuming the answer to #1 is no)? Well, it would be effectively promoting several existing projects which are managed in a few different places (rackspace, 4P, etc)

Re: [Openstack] [Openstack-poc] proposal for policy around and management of client libraries

2011-11-08 Thread John Dickinson
On Nov 8, 2011, at 10:54 AM, Thierry Carrez wrote: With solution (2), if you look at the issue from Gerrit, GitHub, Launchpad or Jenkins, those will be separate projects though. The fact that they share the same PTL is not enough to make them one. For example, they will have separate bug

Re: [Openstack] Tutorials of how to install openstack swift into centos 6

2011-11-09 Thread John Dickinson
Awesome. Thanks for putting this together. I know a lot of people have been interested in getting swift running on non-ubuntu systems. Thanks for sharing this with everyone. --John On Nov 9, 2011, at 12:00 AM, pf shineyear wrote: openstack swift install on centos 6 1. proxy install

Re: [Openstack] Update Container Metadata.

2011-11-14 Thread John Dickinson
Updating existing container metadata with a POST is entirely supported and intended behavior. (The same also applies to adding/updating metadata on an account.) --John On Nov 14, 2011, at 8:10 PM, easco wrote: Anne, Thank you for your help, and you're correct, that is the case I am

Re: [Openstack] rpms for rhel5.x to install Open stack Object Storage

2011-11-21 Thread John Dickinson
I suspect there is a communication gap somewhere, but this is certainly not the case. Openstack Object Storage (swift) is not deprecated. Glance provides a bridge between nova and swift, but all three are important, active projects. Sudhaker, I know that rpms exist for swift, but I don't know

Re: [Openstack] [RFC] Common config options module

2011-12-06 Thread John Dickinson
Overall, I think it's a great thing to have commonality between the projects on option names and environment variables. I think it's worthwhile to push for that in the swift cli tool in the essex timeframe. On the topic of common config libraries, though, I think the differences are less

Re: [Openstack] Several questions about HOW SWIFT WORKS

2012-01-03 Thread John Dickinson
Answers inline. On Jan 3, 2012, at 11:32 AM, Alejandro Comisario wrote: So, lets get down to business. # 1 we have memcache service running on each proxy, so as far as we know, memcache actually caches keystone tokens and object paths as the request ( PUT , GET) enters the proxy, but

[Openstack] swift release 1.4.5

2012-01-04 Thread John Dickinson
It's time again for a swift release. We have cut the swift 1.4.5 release and it's headed to QA. We expect it to be validated by the end of this week and it should land for public use early next week. Below is the changelog for this release. The highlights are the swift-orphans and swift-oldies

Re: [Openstack] Several questions about HOW SWIFT WORKS

2012-01-06 Thread John Dickinson
The best, technical description of the ring was written by the person who had the biggest role in writing it for swift: http://www.tlohg.com/p/building-consistent-hashing-ring.html --John smime.p7s Description: S/MIME cryptographic signature ___

Re: [Openstack] Using Gerrit to verify the CLA

2012-01-09 Thread John Dickinson
https://rackspace.echosign.com/verifier On Jan 9, 2012, at 8:08 AM, Mark McLoughlin wrote: Hey, On Thu, 2012-01-05 at 10:02 -0800, James E. Blair wrote: This change is in place; membership in openstack-cla is required in order to submit changes to Gerrit. All of the -core groups have

[Openstack] swift essex status update

2012-01-11 Thread John Dickinson
more exciting things to come before our final essex release is made. As always, patches welcome! John Dickinson Swift Project Technical Lead notmyname on IRC smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net

Re: [Openstack] (no subject)

2012-01-19 Thread John Dickinson
look in syslog on your proxy server to see what caused the error. --John On Jan 19, 2012, at 6:28 PM, Khaled Ben Bahri wrote: Hi all, I tryed to install OpenStack swift, after creating and configuring all nodes, when i want to check that swift works, I execute this command : swift

Re: [Openstack] Swift supported file system

2012-01-23 Thread John Dickinson
That functionality is left up to a client. For example, you could use FUSE to spoof swift as a filesystem or you could use a client like Cyberduck or even write your own. Last week someone on this mailing list talked about adding webDAV support to swift. All of these work in that they present

Re: [Openstack] Swift supported file system

2012-01-23 Thread John Dickinson
The storage volumes referenced in the ring are identified by an IP, port, and mount point. So, it is possible to use network attached storage for swift (as long as it still supports xattrs). However, I don't know if this has ever really been tried (especially in production), and I'd be

Re: [Openstack] Swift Container Name Restrictions

2012-01-24 Thread John Dickinson
On Jan 24, 2012, at 5:22 PM, Matthew Wodrich wrote: Hi Folks, I'm trying to write some scripts to work with Swift containers, but I don't actually know what the restrictions on container names are. Does anyone know what the specification is, or where I can read up on it? For example:

[Openstack] Swift 1.4.6 release

2012-02-10 Thread John Dickinson
I'm happy to announce that swift 1.4.6 has been released today. We've added some great new features in this release and fixed several outstanding bugs. The full changelog is below, but I'd like to highlight a few key points. Swift 1.4.6 includes new middleware that adds the ability to upload

Re: [Openstack] Doubts about virtual CPUs and Swift storage capacity.

2012-02-11 Thread John Dickinson
On Feb 11, 2012, at 7:29 AM, Jorge Luiz Corrêa wrote: 2) About Swift, how do I determine the total usable storage capacity of the system? For example: I have 3 nodes with 5 HDs of 1 TB each one. Straightly I have 15 TB of space. If I use raid 1 I can say that I'm going to have 7,5 TB of

Re: [Openstack] python-swiftclient?

2012-02-13 Thread John Dickinson
On Feb 13, 2012, at 8:29 AM, Chmouel Boudjnah wrote: What do you think if we : - split swift.common.client to its own. - have bin/swift import that package and shipped with it. - have a comprehensive test suite covering the CLI and the library. - have some proper PIP release for all

Re: [Openstack] swprobe: swift middleware for sending metrics to graphite using statsd

2012-02-21 Thread John Dickinson
That's great. Have you by any chance seen https://github.com/pandemicsyn/swift-informant? It's something similar that we've been playing with at Rackspace. --John On Feb 21, 2012, at 10:36 AM, Jasper Capel wrote: Hi all, I'm announcing a piece of Swift middleware, swprobe [1], designed

Re: [Openstack] Swift container ACLs and container visibility question

2012-02-23 Thread John Dickinson
It all depends on the auth system you are using. Below is for swauth and tempauth: Are the users using the same shared storage? If so, set them up as .admin users with the same storage endpoint. If they are not using the same shared storage, then you may be stuck. The ACL support in swauth and

Re: [Openstack] [Swift] Cannot create container.

2012-02-24 Thread John Dickinson
a 507 response means that the drive was unmounted. If you are running this in a VM (like the SAIO), then you need to disable to mount check. Docs for this are in the SAIO docs. --John On Feb 24, 2012, at 9:13 AM, Leander Bessa wrote: Hello, I'm trying to set up a custom swift server in a

[Openstack] swift authors list

2012-02-24 Thread John Dickinson
Stefano has asked that swift start keeping email addresses in our AUTHORS file. I'll be adding these soon, but some contributors have more than one email. If you have a particular email address you'd like to have me add to your name, please let me know. Otherwise, I will use one from git log.

[Openstack] Swift patches for the essex release

2012-03-07 Thread John Dickinson
The final swift release for openstack essex is coming quickly. Swift 1.4.8 is tentatively scheduled for March 22nd. To get your patches into this release, please submit them for review by next week so we have enough time to review and QA them. --John smime.p7s Description: S/MIME

Re: [Openstack] swift and two data-centres - hierarchical zones?

2012-03-12 Thread John Dickinson
On Mar 10, 2012, at 5:58 AM, John Leach wrote: I think what I need here is hierarchical zones - I'd define one parent zone per data-centre, and then multiple child zones within each (representing racks or whatever). Swift would be configured to write 3 replicas in 3 child zones, aiming

Re: [Openstack] Swift concept architecture

2012-03-12 Thread John Dickinson
On Mar 11, 2012, at 12:16 AM, Dmitry Ukov wrote: Hi all, I want to introduce some ideas about Swift. Let’s assume we have huge amount of data stored in Swift (e.g. 10Pb). This data are dynamically changed by users. So we need to reduce network load caused by replication and intensive

Re: [Openstack] Swift: NAS or DAS?

2012-03-16 Thread John Dickinson
Generally, you would introduce latency in the storage system by using a NAS attached to a storage drive. Also, at scale, your costs will be dominated by drive, so you will want to optimize the storage nodes for dense, cheap storage. --John On Mar 16, 2012, at 8:32 AM, Michaël Van de Borne

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

2012-04-12 Thread John Dickinson
Swift keeps total bytes, container, and object count (eventually) up-to-date in the account metadata. There are also log processing tools (like slogging - http://github.com/notmyname/slogging) that can provide usage information (including bandwidth) based on swift logs. While I think that it's

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

2012-04-12 Thread John Dickinson
I should also mention the summit session talking about this very topic led by Everett Toews. It's (currently) scheduled for 9am on wednesday. http://summit.openstack.org/sessions/view/81 --John On Apr 12, 2012, at 8:51 PM, John Dickinson wrote: Swift keeps total bytes, container

Re: [Openstack] HTTP status value naming normalization

2012-04-21 Thread John Dickinson
I like what you are trying to do here. Can you please submit this as a patch through gerrit so we can get the rest of the core devs to look at it? --John On Apr 20, 2012, at 12:14 PM, Victor Rodionov wrote: There are many place in Swift code where used hard coded values, such as response

Re: [Openstack] [Openstack-operators] Question Regarding Swift Distribution Statistics

2012-05-02 Thread John Dickinson
On your proxy server, use swift-get-nodes to see which servers your object is on. With no arguments or --help you will get usage info. --John On May 2, 2012, at 3:48 PM, Duncan McGreggor wrote: cc'ing openstack list On Wed, May 2, 2012 at 4:45 PM, Richard Raseley rich...@raseley.com

[Openstack] [Swift] swift news and plans

2012-05-04 Thread John Dickinson
TL;DR: removing code from swift, associated projects doc, swift 1.5.0 I want to let the openstack community know of some recent changes within swift and how those changes will affect the next version of swift. Swift has a growing developer community and a rapidly expanding deployed base. While

Re: [Openstack] Opening up bug triaging rights

2012-05-14 Thread John Dickinson
On May 14, 2012, at 10:06 AM, Thierry Carrez wrote: Hello everyone, Currently the bug triaging rights for a given PROJECT (ability to set status and importance of bugs, but also ability to nominate a bug for a past series) is restricted to the corresponding PROJECT-bugs team, which is

Re: [Openstack] [Swift] Interested in implementing swift ring builder server

2012-06-19 Thread John Dickinson
It looks like Florian (at Rackspace) is working on that blueprint. He just assigned it to himself. I'm happy to hear that you have some extra devs for swift work. I'd love to help coordinate some swift goals with you. Off the top of my head, here are a few things that could be worked on: 1)

Re: [Openstack] [Swift] S3 like ACL for Swift

2012-06-20 Thread John Dickinson
Yes, this could be good for swift. ACLs in swift do need to be stored in swift (for scale reasons), but their implementation is dependent on the particular auth system that you are using. The auth middleware is responsible for determining if a request is granted access to a particular swift

Re: [Openstack] [Swift] S3 like ACL for Swift

2012-06-20 Thread John Dickinson
On Jun 20, 2012, at 11:02 AM, Victor Rodionov wrote: Also, I want ask do you think it's good idea to store object ACL in object metadata? I'd suggest looking at container-level ACLs rather than object-level. But either way, the data does need to be stored in the metadata in swift itself.

Re: [Openstack] Swift Probetests

2012-06-26 Thread John Dickinson
The probe tests are internal whole-system type of tests that test functionality never exposed through normal integration testing. They exist at the level between unit tests and functional tests. For example, one of the probetests makes sure that asynchronous container updates actually happen.

Re: [Openstack] [Swift] [Storage node] Lots of timeouts in load test after several hours around 1, 000, 0000 operations

2012-07-01 Thread John Dickinson
I hope you are able to get an answer. I'm traveling this week, so I won't have a chance to look in to it. I hope some of the other core devs will have a chance to help you find an answer. --John On Jul 1, 2012, at 2:03 PM, Kuo Hugo tonyt...@gmail.com wrote: Hi all , I did several

Re: [Openstack] Statsd on SWIFT 1.4.8

2012-07-05 Thread John Dickinson
statsd integration was added in swift 1.5.0 On Jul 5, 2012, at 9:07 AM, Leandro Reox wrote: On swift essex stable 1.4.8, is the capacity to send statistics to a statsd server available? Its not in the docs Regards ___ Mailing list:

Re: [Openstack] [Swift] LFS patch (Ia32c9c34)

2012-07-18 Thread John Dickinson
Nexenta's LFS patch (https://review.openstack.org/#/c/7524/) has languished for a while, and I'd like to address that. First, thank you for your patch submission. This patch adds new functionality that potentially can allow swift to be deployed in more places. The original version of the

[Openstack] next swift release just around the corner

2012-07-23 Thread John Dickinson
The next swift release is scheduled for public release next Monday (July 30). That means we've got a little bit of work to do this week to get it ready. In order to allow Cloud Files QA time to check it, we need to have packages built by the middle of the day Wednesday. This means all

[Openstack] Swift 1.6.0 released

2012-08-06 Thread John Dickinson
I'm happy to announce that Swift 1.6.0 has been released. You can get the tarball at https://launchpad.net/swift/folsom/1.6.0. As always, you can upgrade your production Swift clusters to this new version with no downtime to your clients. The complete changelog for this release is at

[Openstack] [swift] Operational knowledge sharing

2012-08-10 Thread John Dickinson
In a standard swift deployment, the proxy server is running behind a load balancer and/or an SSL terminator. At SwiftStack, we discovered an issue that may arise from some config parameters in this layer, and we'd like to share it with other swift deployers. Symptom: Users updating metadata

Re: [Openstack] Swift + keystone integration

2012-08-11 Thread John Dickinson
Make sure that the endpoint stored in keystone is returning the right hostname/domain name and port (8080 based on your config). --John On Aug 11, 2012, at 12:58 PM, Miguel Alejandro González maggo...@gmail.com wrote: Hello I have 3 nodes with ubuntu 12.04 server and installed openstack

[Openstack] [Swift] community update and what's coming in Folsom

2012-08-13 Thread John Dickinson
We just released Swift 1.6.0 last Monday ( https://lists.launchpad.net/openstack/msg15505.html ). We've got a lot of great features and improvements in it, and I wanted to take some time to update the wider community about where Swift is. Swift 1.4.8 was included with the last OpenStack realease

[Openstack] [Swift] Where do we go from here?

2012-08-14 Thread John Dickinson
Swift has many exciting features coming in the OpenStack Folsom Release this fall, but where do we go from here? What's next for Swift in grizzly? I've got some ideas. I'd like to mention them and see where you the community will take them. I've written up most of them into quick one- line

Re: [Openstack] Question about undo delete in Swift with Object Version

2012-08-22 Thread John Dickinson
With the current implementation of versioning in Swift, this isn't possible. It's better to think of the feature as versioned writes. --John On Aug 22, 2012, at 12:13 AM, ZHOU Yuan dunk...@gmail.com wrote: Hi stackers, I'm trying to understand the object version feature in Swift. In the

[Openstack] [swift] important upgrade note for swift deployers

2012-08-28 Thread John Dickinson
Swift 1.7 = The next release of Swift will be version 1.7. This will be our release for OpenStack Folsom, and is scheduled land mid-September. There is an important change for deployers in this release. This email has the details so you can begin planning your upgrade path. What's the

[Openstack] Swift PTL candidacy

2012-09-04 Thread John Dickinson
I am running to continue my position as Swift PTL. I have been involved with swift since the project started. I am an active contributor, reviewer, and community participant. I have lead meetups about swift, given conference presentations on swift, and am active in IRC helping those who have

Re: [Openstack] [OpenStack][Swift][Replicator] Does object replicator push exist object to handoff node while a node/disk/network fails ?

2012-09-06 Thread John Dickinson
you can force a replicator to push to a handoff node by unmounting the drive one of the primary replicas is on. --John On Sep 6, 2012, at 9:00 AM, Kuo Hugo tonyt...@gmail.com wrote: Hi folks , John and Chmouel , I did post a question about this long time ago. And my test result is match

[Openstack] [swift] 1.7.0-final pushed

2012-09-12 Thread John Dickinson
the Essex release, but I wanted to publicly thank everyone who contributed to Swift during the Folsom release cycle. The following people have code contributions in Swift during the Folsom release cycle. Thanks again! Greg Holt John Dickinson Darrell Bishop Samuel Merritt Florian Hines David Goetz

Re: [Openstack] [swift] make swift.common.utils.streq_const_time more efficient

2012-09-13 Thread John Dickinson
The intended purpose of this string comparison is to explicitly compare every character. Doing it this way guards against timing attacks (http://en.wikipedia.org/wiki/Timing_attack). --John On Sep 13, 2012, at 12:06 AM, Mike Green iasy...@gmail.com wrote: def streq_const_time(s1, s2):

[Openstack] [Swift] community meeting Oct 1

2012-09-25 Thread John Dickinson
As we finish up Folsom and head into the Grizzly summit, I'd like to have a Swift community meeting. Who: The Swift community (users, deployers, contributors, core devs) When: October 1, 2012 at 8pm (UTC), 3pm (Central), 1pm (Pacific) Where: #openstack-meeting on freenode (IRC) Agenda:

Re: [Openstack] Swift reliability

2012-09-26 Thread John Dickinson
The 404s on object PUTs are probably related to the timeout errors you are seeing on the container servers. This may be because of IO contention on your hardware (eg overtaxed drives). How does the disk IO look on your physical hardware? The disk full errors may be because you are running out

Re: [Openstack] [openstack-dev] [Swift] community meeting Oct 1

2012-10-01 Thread John Dickinson
Reminder for today's meeting. On Sep 24, 2012, at 10:59 PM, John Dickinson m...@not.mn wrote: As we finish up Folsom and head into the Grizzly summit, I'd like to have a Swift community meeting. Who: The Swift community (users, deployers, contributors, core devs) When: October 1, 2012

Re: [Openstack] [SWIFT] Proxies Sizing for 90.000 / 200.000 RPM

2012-10-24 Thread John Dickinson
Sorry for the delay. You've got an interesting problem, and we were all quite busy last week with the summit. First, the standard caveat: Your performance is going to be highly dependent on your particular workload and your particular hardware deployment. 3500 req/sec in two different

Re: [Openstack] [SWIFT] Proxies Sizing for 90.000 / 200.000 RPM

2012-10-24 Thread John Dickinson
are so i can know where to tune in case i need to. Thanks ! --- Alejandrito On Wed, Oct 24, 2012 at 3:28 PM, John Dickinson m...@not.mn wrote: Sorry for the delay. You've got an interesting problem, and we were all quite busy last week with the summit. First, the standard caveat: Your

[Openstack] Swift 1.7.5 release plan

2012-10-25 Thread John Dickinson
Fast on the heels of a productive summit in San Diego, we are getting ready to release Swift 1.7.5. Our current schedule is to cut the QA release on November 5 and, assuming it passes all QA tests, prepare the final release on November 8. This is quite a solid release with a ton of bug fixes

Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers

2012-10-27 Thread John Dickinson
A 507 is returned by the object servers in 2 situations: 1) the drives are full or 2) the drives have been unmounted because of disk error. It's highly likely that you simply have full drives. Remember that the usable space in your cluster is 1/N where N = replica count. As an example, with 3

Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers

2012-10-29 Thread John Dickinson
, John Dickinson m...@not.mn wrote: A 507 is returned by the object servers in 2 situations: 1) the drives are full or 2) the drives have been unmounted because of disk error. It's highly likely that you simply have full drives. Remember that the usable space in your cluster is 1/N where N

Re: [Openstack] Nova middleware for enabling CORS?

2012-10-30 Thread John Dickinson
Since the CORS support in Swift allows the preflight OPTIONS response to be different on a per-container basis (which is correct in a multi-tenant system), the CORS support was added directly into Swift's proxy server rather than as middleware. In order to fulfill the OPTIONS request, container

Re: [Openstack] [SWIFT] Rack-awareness

2012-11-01 Thread John Dickinson
This is already supported in Swift with the concept of availability zones. Swift will place each replica in different availability zones, if possible. If you only have one zone, Swift will place the replicas on different machines. If you only have one machine, Swift will place the replicas on

Re: [Openstack] [SWIFT] Rack-awareness

2012-11-01 Thread John Dickinson
anything like it? If not, im wondering how, for example rackspace handles this kind of issues ( ignoring all the CDN thing ) Cheers. --- Alejandrito On Thu, Nov 1, 2012 at 12:55 PM, John Dickinson m...@not.mn wrote: This is already supported in Swift with the concept

Re: [Openstack] Running other services on swift nodes

2012-11-04 Thread John Dickinson
From the Swift perspective, there isn't any reason other services can't be run on the Swift boxes. I'd check for a few things, though. 1) Make sure dependencies aren't in conflict. Thanks to the work of the CI team, this should be mostly sane. 2) Obviously, monitor your systems and don't

Re: [Openstack] [Swift] account-level and container-level usage information

2012-11-13 Thread John Dickinson
You could use a project like slogging (http://github.com/notmyname/slogging) to run inside the cluster and aggregate that information from account dbs and logs. --John On Nov 13, 2012, at 12:15 PM, Pete Zaitcev zait...@redhat.com wrote: On Mon, 12 Nov 2012 20:48:35 -0800 Ning Zhang

Re: [Openstack] PoC Swift

2012-11-19 Thread John Dickinson
You can run your swift cluster with 2 replicas. This is set when the ring is first created. You can use `swift-ring-builder` (ie with no arguments) to get usage help. Note that there is not (yet) an easy way to change your replica count once the ring has been created. If you have more than one

Re: [Openstack] Error while enabling s3 support in swift

2012-11-19 Thread John Dickinson
check out the README at https://github.com/fujita/swift3 for the correct proxy server config section. --john On Nov 19, 2012, at 11:22 PM, Shashank Sahni shredde...@gmail.com wrote: Hi, I'm trying to install swift 1.7.4 on Ubuntu 12.04. In order to enable the s3 support, I added the

Re: [Openstack] Error while enabling s3 support in swift

2012-11-28 Thread John Dickinson
. -- Shashank Sahni On Tue, Nov 20, 2012 at 12:55 PM, John Dickinson m...@not.mn wrote: check out the README at https://github.com/fujita/swift3 for the correct proxy server config section. --john On Nov 19, 2012, at 11:22 PM, Shashank Sahni shredde...@gmail.com wrote: Hi, I'm

Re: [Openstack] swift -- object layout on storage

2013-01-04 Thread John Dickinson
It's pretty simple. Swift uses the underlying filesystem to store the data on disk, and so you can use normal FS tools to find and inspect your data. For the object server, the magic happens here: https://github.com/openstack/swift/blob/master/swift/obj/server.py#L117 The end result is that

  1   2   >