Re: [Openstack] [Keystone][Swift] Problems with admin_user, admin_password, admin_tenant_name

[Alex Yang]

If you want to create a reseller_admin user with keystone, you should add
this user in keystone with admin role.
The admin role is the configuration of keysoneauth.

[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin, swiftoperator


2013/2/23 Kun Huang 

> Back to my first question, should I create admin_user, admin_password,
> admin_tenant_name by keystone command myself?
>
>
> On Fri, Feb 22, 2013 at 9:02 AM, Kun Huang wrote:
>
>> Dolph, thanks your advice, but in my environment, both below two case
>>
>> 1. setting admin_token,admin_user, admin_password, admin_tenant_name in
>> proxy.conf and setting admin_token in keystone.conf
>> 2. setting admin_user, admin_password, admin_tenant_name in proxy.conf
>> and setting admin_token in keystone.conf
>>
>> only admin_token in keystone.conf is valuable. (default ADMIN)
>> Should I create admin_user, admin_password, admin_tenant_name by keystone
>> command myself?
>>
>>
>>
>>
>> On Fri, Feb 22, 2013 at 1:45 AM, Dolph Mathews 
>> wrote:
>>
>>> Setting the admin_token config is overriding the admin_user,
>>> admin_password, and admin_tenant_name values by specifying a static token.
>>> I'd recommend removing it and using the auth credentials you've already
>>> provided.
>>>
>>> -Dolph
>>>
>>>
>>> On Thu, Feb 21, 2013 at 3:10 AM, Kun Huang wrote:
>>>
 Following another question:
 In document:
 http://docs.openstack.org/developer/keystone/middleware_architecture.html
 ,


 [filter:tokenauth]
 paste.filter_factory = keystone.middleware.auth_token:filter_factory
 auth_host = 127.0.0.1
 auth_port = 35357
 auth_protocol = http
 auth_uri = http://127.0.0.1:5000/
 admin_token = Super999Sekret888Password777
 admin_user = admin
 admin_password = SuperSekretPassword
 admin_tenant_name = service
 ;Uncomment next line and check ip:port to use memcached to cache tokens
 ;memcache_servers = 127.0.0.1:11211

 That admin token is not admin_token = ADMIN in keystone, right?





 On Thu, Feb 21, 2013 at 4:06 PM, Kun Huang wrote:

> I'm building environment with Swift + Keystone.
> I have set admin_user, admin_password, admin_tenant_name, but it
> doesn't work.
> I followed the document:
> http://docs.openstack.org/developer/swift/overview_auth.html and chmouel's
> blog:
> http://blog.chmouel.com/2011/11/24/swift-and-keystone-middleware-part1/
>
> My swift and keystone run well:
>
> [image: Inline image 1]
>
> But failed in curl:
> [image: Inline image 2]
>
> I guess the value of admin_user/admin_password/admin_tenant_name
> doesn't save in db.
>
> Which step do I missed? or some problems in my proxy config?
>
> Here is my current config.
> [DEFAULT]
> bind_port = 
> user = swift
>
> [pipeline:main]
> pipeline = catch_errors healthcheck cache authtoken keystoneauth
> proxy-server
>
> [app:proxy-server]
> use = egg:swift#proxy
> account_autocreate = true
>
> [filter:keystoneauth]
> use = egg:swift#keystoneauth
> operator_roles = admin, swiftoperator
>
> [filter:authtoken]
> # Delaying the auth decision is required to support token-less
> # usage for anonymous referrers ('.r:*').
> paste.filter_factory =
> keystoneclient.middleware.auth_token:filter_factory
> auth_host = keystonehost
> auth_port = 35357
> auth_protocol = http
> auth_uri = http://keystonehost:5000/
> admin_tenant_name = service
> admin_user = swift
> admin_password = password
>
> [filter:cache]
> use = egg:swift#memcache
> set log_name = cache
>
> [filter:catch_errors]
> use = egg:swift#catch_errors
>
> [filter:healthcheck]
> use = egg:swift#healthcheck
>


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


>>>
>>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Alex Yang]

2013/2/20 Chmouel Boudjnah 

> I haven't had the chance to do a full reviews but here is a few toughts :
>
> - the account quota goes along this review:
>
> https://review.openstack.org/#/c/21563/
>
> so we can get easily have account metadatas from middlewares.
>
Yes, I also want quota-level to be stored in memcache through the
account_info() function.

>
> - I am not sure I fancy much the json blob in the ini config (why not a
> new config file?)
>
It's a good suggestion.

>
> - I have already an implementation that mostly follow the same structure
> as (the already merged)  container quota middleware but I am waiting on the
> other review for account info metadata to send it for reviews.
>
> - Down the line but probably not for the v1 I like to be able to notify
> somewhere about the account being over quota ideally it would be an option
> to get to ceilometer or other plugged in system.
>

> Chmouel.
>
>
> On Wed, Feb 20, 2013 at 11:11 AM, Alex Yang  wrote:
>
>>
>>  Storage Quotas 
>> Design<https://docs.google.com/document/d/1b5hkT_E8PyzaAPjNImW0SF-yyh8vGrN_DNjMChol4_Q/edit>
>> This is the design draft of Storage Quota.
>> Implementation of this design is
>> https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota
>>
>>
>> --
>>   杨雨
>>   Email:   alex890...@gmail.com
>> GitHub:   https://github.com/AlexYangYu
>> Blog:http://alexyang.sinaapp.com
>>  Weibo:   http://www.weibo.com/alexyangyu
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Alex Yang]

Hi Schwede,
I have already read your codes. To implement the authentication of updating
metadata in the middleware is good for maintainance.
But the account usage is not accurate because of the eventual consistency.
So I do the usage quota based on container.
I will refactor my implementation collaborate with your work and Boudjnah's
work.

Thanks!
.


2013/2/20 Christian Schwede 

> Hi,
> you might also have a look at 
> https://github.com/cschwede/**swquota<https://github.com/cschwede/swquota>
>
> Account quota is stored in account metadata and set by a reseller account.
> I changed the code slightly to make it more consistent to the already
> merged container quota.
>
> @chmouel: I might create a pull request to swift itself - what do you
> think?
>
> Christian
>
>
> Am Mittwoch, den 20.02.2013, 11:11 +0100 schrieb Alex Yang <
> alex890...@gmail.com>:
>
>>  Storage Quotas Design [1]
>>
>> This is the design draft of Storage Quota.
>> Implementation of this design is
>> https://github.com/AlexYangYu/**StackLab-swift/tree/dev-quota<https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota>[2]
>>
>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift] Administration Interfaces/Tools for Swift

[Alex Yang]

You can try to use StatD wisth zabbix.


2013/1/31 Gui Maluf 

> Nice things guys!
> I think for the monitoring purpose swift is well served. There is many
> tools that make the work. But seems that there is a lack of a admin
> interface, right? What I've found on SwiftStack looks pretty, but
> unfortunately their tool is paid, and I couldnt get how much they want for
> that.
> I'll keep searching for something, if I find something I'll come back here!
>
> Thank you guys!
>
>
> On Tue, Jan 29, 2013 at 4:48 PM, Florian Hines wrote:
>
>>  Theres actually 2 diamond plugin's for swift now. I added the one we use
>> internally at Rackspace a week or two ago
>> https://github.com/BrightcoveOS/Diamond/wiki/collectors-OpenstackSwiftReconCollector.
>>  In addition to these and Zenoss theres also been nagios and collectd
>> plugins floating around.
>>
>> At least in our deployments Swifts built in statsd metrics become a bit
>> overwhelming (both in volume of metrics and what metrics are actually being
>> reported). This is probably just a side effect of the size of our clusters.
>> We ended up sticking with our original statsd/metric setup (a combo of some
>> proxy middleware, and translating a bunch of Swift's syslog lines to statsd
>> events).
>>
>> --
>> Florian Hines | @pandemicsyn
>> http://about.me/pandemicsyn
>>
>>  On Tuesday, January 29, 2013 at 12:16 PM, Dieter Plaetinck wrote:
>>
>> On Tue, 29 Jan 2013 15:24:31 -0200
>> Gui Maluf  wrote:
>>
>> Hi guys,
>> I'm searching for a Swift Interface that integrates Administration and
>> maybe monitoring.
>> I've checked SwiftStack  and
>> Gladinet<
>> http://gladinet.blogspot.com.br/2012/06/basic-layer-on-top-of-openstack-swift.html
>> >,
>> but both are not opensource and require money to be used.
>>
>> There is something available that could ease admins tasks on swift?
>>
>> Thanks in advance!
>>
>>
>> for monitoring, you can use
>> https://github.com/Dieterbe/graph-explorer (graphite dashboard)
>> which displays data (amongst others) from
>> https://github.com/BrightcoveOS/Diamond (which has a swift plugin)
>> as well as from the built in statsd support into swift
>>
>> i'm the author of graph-explorer and the diamond openstack swift plugin
>> (i wrote these things cause i also couldn't find anything existing)
>> i actually plan to write a blog post about these things soonish.
>>
>>
>> Dieter
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>
>
> --
> *guilherme* \n
> \t *maluf*
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [swift] how to configure NWR policy

[Alex Yang]

Hi, Zhang,
Just add 'X-Newest: Ture' in the HTTP GET request.


2012/12/11 Hua ZZ Zhang 

> Hi all,
>
> I have a question about swift configuration for NWR policy. According to
> some documents of Swift, NWR is configurable. The general configuration is:
> N=3, W=2, R=1 or 2.
> Swift can provide both models of consistency: strong and eventual. But I
> can't find where to configure this option.
>
> Any suggestion is appreciated!
>
> *Best Regards, *
>
> --
>
>*Edward Zhang(张华)*
>IBM China Software Development Lab
>
>
>
>
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
<<1F033383.gif>><>___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift] account-level and container-level usage information

[Alex Yang]

Hi，
The python-swiftclient(https://github.com/openstack/python-swiftclient) can
retrieve the account-level and container-level usage infomation.

To retrieve the account usage information:
$ swift -V 2 -A http://127.0.0.1:5000/v2.0 -U tenant1:swift_user -K
19561212 stat
   Account: AUTH_70b51a6d180f4f1da78d80316c69e85c
Containers: 10
   Objects: 0
 Bytes: 0
Meta Quota: L1
Accept-Ranges: bytes
X-Timestamp: 1352735608.55267
X-Trans-Id: tx72878dfcba9948298a6f4efb4e51e569

To retrieve the container usage information:
$ swift -V 2 -A http://127.0.0.1:5000/v2.0 -U tenant1:swift_user -K
19561212 stat c1
No handlers could be found for logger "keystoneclient.v2_0.client"
  Account: AUTH_70b51a6d180f4f1da78d80316c69e85c
Container: c1
  Objects: 0
Bytes: 0
 Read ACL:
Write ACL:
  Sync To:
 Sync Key:
Accept-Ranges: bytes
X-Timestamp: 1352735637.83676
X-Trans-Id: tx7968942b927b4f1fba0c40fb1372adba

You can also use the REST API,
http://docs.openstack.org/cli/quick-start/content/swift_client_commands.html

But the bytes and objects of account is not accurate.
You can get the accurate result by retrieve all the containers and add them
up.


2012/11/13 Ning Zhang 

> Hello All,
>
> Is there any Swift (GUI or command line) tool that can
> retrieve the account-level and
> container-level usage information (e.g. how large space
> has been used under an account, how large space has been
> used under a tenant) and also works with keystone?
>
> Thanks,
>
> Ning
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>



-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers

[Alex Yang]

There are any error about disk in the kern.log?

2012/10/30 Nathan Trueblood 

> Still no further clues.   I re-created all the volumes I'm using for
> Swift.  Plenty of Inodes free:
>
> lab@data02:~$ df -i
> FilesystemInodes IUsed IFree IUse% Mounted on
> /dev/sda2   12214272 39290  121749821% /
> none  107979   4821074971% /dev
> none  107979   2681077111% /run
> none  107979 21079771% /run/lock
> none  107979 11079781% /run/shm
> /dev/sda1  4915223 491291% /boot
> /dev/sda4  13404640037 1340463631% /srv/node/sda4
>
> I successfully upload a small object to container cont1, then cont2.
> When I upload to cont3, I see the following in the object-server log
> (data02)
>
> This seems to be the problematic sequence:
>
> Data02 has ip 192.168.1.202
> Data03 has ip 192.168.1.203
>
> 1. First the account server reports an HTTP 201 on the container from a
> different object server in a different zone.
> 2. Then the object server reports a 404 trying to HEAD the new object.
> 3. Then the object server reports a 507 trying to PUT the new object.
>
> From this point the operation eventually fails and the proxy reports a 503.
>
> Oct 29 17:58:20 data02 account-server 192.168.1.203 - -
> [30/Oct/2012:00:58:20 +] "PUT /sda4/116021/AUTH_system/cont3" 201 -
> "tx5a3ca6c845af41928e0ba6b7bc58d2da" "-" "-" 0.0082 ""
> Oct 29 17:58:20 data02 object-server 192.168.1.111 - -
> [30/Oct/2012:00:58:20 +] "HEAD
> /sda4/257613/AUTH_system/cont3/home/lab/bigfile3" 404 - "-"
> "tx5f21503ff12e45e39a80eb52f6757261" "-" 0.0011
> Oct 29 17:58:20 data02 object-server 192.168.1.111 - -
> [30/Oct/2012:00:58:20 +] "PUT
> /sda4/257613/AUTH_system/cont3/home/lab/bigfile3" 507 - "-"
> "tx425494dc372740e28d043a07d3a08b9a" "-" 0.0031
>
> In an earlier, successful transaction I noticed that between Steps 1 and 2
> above, there is a response from the container-server:
>
> Oct 29 17:57:59 data02 account-server 192.168.1.204 - -
> [30/Oct/2012:00:57:59 +] "PUT /sda4/116021/AUTH_system/cont2" 201 -
> "txb10d75886bf14e4eba14fcc52d81c5d9" "-" "-" 0.0182 ""
> Oct 29 17:57:59 data02 container-server 192.168.1.111 - -
> [30/Oct/2012:00:57:59 +] "PUT /sda4/122355/AUTH_system/cont2" 201 -
> "txb10d75886bf14e4eba14fcc52d81c5d9" "-" "-" 0.1554
> Oct 29 17:57:59 data02 object-server 192.168.1.111 - -
> [30/Oct/2012:00:57:59 +] "HEAD
> /sda4/226151/AUTH_system/cont2/home/lab/bigfile3" 404 - "-"
> "tx1c514850530849d1bfbfa716d9039b87" "-" 0.0012
> Oct 29 17:57:59 data02 container-server 192.168.1.204 - -
> [30/Oct/2012:00:57:59 +] "PUT
> /sda4/122355/AUTH_system/cont2/home/lab/bigfile3" 201 -
> "tx8130af5cae484e5f9c5a25541d1c87aa" "-" "-" 0.0041
> Oct 29 17:57:59 data02 object-server 192.168.1.111 - -
> [30/Oct/2012:00:57:59 +] "PUT
> /sda4/226151/AUTH_system/cont2/home/lab/bigfile3" 201 - "-"
> "tx8130af5cae484e5f9c5a25541d1c87aa" "-" 0.1716
>
>
> So maybe the container server is failing to create the new container?
> Maybe a bug in auto-create of containers?
>
> Definitely NOT a problem with the filesystem, but something is causing the
> object-server to think there is a problem with the filesystem.
>
> I suspect a bug in one of the underlying libraries.
>
> Any further suggestions on how to troubleshoot?
>
> Thanks.   When I finally find the solution, I'll post my results.
>
> -N
>
> On Fri, Oct 26, 2012 at 11:21 PM, John Dickinson  wrote:
>
>> A 507 is returned by the object servers in 2 situations: 1) the drives
>> are full or 2) the drives have been unmounted because of disk error.
>>
>> It's highly likely that you simply have full drives. Remember that the
>> usable space in your cluster is 1/N where N = replica count. As an example,
>> with 3 replicas and 5 nodes with a single 1TB drive each, you only have
>> about 1.6TB available for data.
>>
>> As Pete suggested in his response, how big are your drives, and what does
>> `df` tell you?
>>
>> --John
>>
>>
>> On Oct 26, 2012, at 5:26 PM, Nathan Trueblood 
>> wrote:
>>
>> > Hey folks-
>> >
>> > I'm trying to figure out what's going wrong with my Swift deployment on
>> a small cluster of "mini" servers.   I have a small test cluster (5 storage
>> nodes, 1 proxy) of mini-servers that are ARM-based.   The proxy is a
>> regular, Intel-based server with plenty of RAM.   The
>> object/account/container servers are relatively small, with 2GB of RAM per
>> node.
>> >
>> > Everything starts up fine, but now I'm trying to troubleshoot a strange
>> problem.   After I successfully upload a few test files, it seems like the
>> storage system stops responding and the proxy gives me a 503 error.
>> >
>> > Here's the test sequence I run on my proxy:
>> >
>> > lab@proxy01:~/bin$ ./swiftcl.sh stat
>> > swift -A http://proxy01:8080/auth/v1.0 -U system:root -K testpass stat
>> >Account: AUTH_system
>> > Containers: 5
>> >Objects: 4
>> >  Bytes: 4

Re: [Openstack] does swift support resume upload?

[Alex Yang]

Hi,
What is resume upload?

2012/10/13 符永涛 

> Dear swift experts,
>
> We're planning to use swift to implement an upload service. And we want to
> know if swift support resume upload feature? Thank you.
>
> --
> 符永涛
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [Swift]How to recovery the ring builder file?

[Alex Yang]

Hi all.
Yesterday, I made a mistake than I delete all my ring builder files.
How can I recovery the ring builder files?

-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Swift PTL candidacy

[Alex Yang]

John does a great job!

2012/9/5 Thierry Carrez 

> John Dickinson wrote:
> > I am running to continue my position as Swift PTL.
>
> As an election official, I can confirm that you are eligible.
>
> --
> Thierry Carrez (ttx)
> Release Manager, OpenStack
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>



-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack][Swift] How about move auth_toke and swift_auth middlewares into Swift?

[Alex Yang]

Thanks!

2012/8/30 Chmouel Boudjnah 

> Hello Alex,
>
> On Thu, Aug 30, 2012 at 11:52 AM, Alex Yang  wrote:
>
>> How to deal with the auth_token middleware?
>> Move it into swift or move it into openstack/common?
>>
>
> there was a whole discussion last week about it, not sure what was the
> decision if that was to go to openstack/common or to its own project but
> that's definitively a post folsom works.
>
>  Chmouel.
>



-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack][Swift] How about move auth_toke and swift_auth middlewares into Swift?

[Alex Yang]

Good job！
How to deal with the auth_token middleware?
Move it into swift or move it into openstack/common?

2012/8/30 Chmouel Boudjnah 

> Hi,
>
> The keystone middleware has been moved to swift already, it just
> wasn't removed from keystone, I haven't send a removal request from
> keystone since there is a massive works of documentation that needed
> to be done and it's a bit too late for this release timeline since
> distributors/operators may be not ready for it yet.
>
> Chmouel.
>
> On Thu, Aug 30, 2012 at 5:45 AM, Alex Yang  wrote:
> > Hi Swifters，
> > I use keystone for the authentication and authorization of swift. During
> the
> > benchmark test, I also encountered this kind of problems
> > https://bugs.launchpad.net/keystone/+bug/1020127.
> > I do some changes that moved the auth_toke and swift_auth middlewares
> into
> > swift and replace the python-memecache client with
> > swift.common.memcaced.MemcacheRing.
> > It's works well for my business environment and easy to setup.
> >
> > I am willing to contribute this changes to swift or keystone.
> > What's your advices?
> >
> __
> >   杨雨
> >   Email:   alex890...@gmail.com
> > GitHub:   https://github.com/AlexYangYu
> > Blog:http://alexyang.sinaapp.com
> >  Weibo:   http://www.weibo.com/alexyangyu
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
>



-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [OpenStack][Swift] How about move auth_toke and swift_auth middlewares into Swift?

[Alex Yang]

Hi Swifters，
I use keystone for the authentication and authorization of swift. During
the benchmark test, I also encountered this kind of problems
https://bugs.launchpad.net/keystone/+bug/1020127.
I do some changes that moved the auth_toke and swift_auth middlewares into
swift and replace the python-memecache client with
swift.common.memcaced.MemcacheRing.
It's works well for my business environment and easy to setup.

I am willing to contribute this changes to swift or keystone.
What's your advices?
__
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]Is the cloudfiles compatible with keystone v2.0?

[Alex Yang]

2012/7/6 Alex Yang 

> Hi,
> I have created a new branch named for_keystone2.0 in g...@github.com:
> AlexYangYu/python-cloudfiles.git.
> This branch works fine with keystone v2.0.
>
> import cloudfiles
>
> conn = cloudfiles.get_connection(
> username='service:swift', api_key='swift',
> authurl='http://127.0.0.1:5000/v2.0/tokens', auth_version='v2.0'
> )
>
> Why keystone encourage  migrating from port 5000 to port 35357?
> In my opinion, the 5000 is a public endpoint for get tokens, the 35357 is
> an admin endpoint.They may have different security strategy.
> I am familiar with keystone.
>
 I am not familiar with keystone.

>
> 2012/7/6 Pete Zaitcev 
>
>> On Thu, 5 Jul 2012 18:21:48 +0800
>> Alex Yang  wrote:
>>
>> > If i want use the cloudfiles with the keystone(the newest version), I
>> need
>> > change the code of cloudfiles by myself?
>>
>> Indeed, clients have to be adapted in cases like yours. However,
>> please look at python-swiftclient. It may be sufficiently similar
>> so you don't need to change much in your application that uses
>> python-cloudfiles.
>>
>> By the way, it was my understanding that Keystone people encourage
>> everyone to migrate from port 5000 to port 35357. The latter is
>> assigned with IANA.
>>
>> -- Pete
>>
>
>
>
> --
>   杨雨
>   Email:   alex890...@gmail.com
> GitHub:   https://github.com/AlexYangYu
> Blog:http://alexyang.sinaapp.com
>  Weibo:   http://www.weibo.com/alexyangyu
>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]Is the cloudfiles compatible with keystone v2.0?

[Alex Yang]

Hi,
I have created a new branch named for_keystone2.0 in g...@github.com:
AlexYangYu/python-cloudfiles.git.
This branch works fine with keystone v2.0.

import cloudfiles

conn = cloudfiles.get_connection(
username='service:swift', api_key='swift',
authurl='http://127.0.0.1:5000/v2.0/tokens', auth_version='v2.0'
)

Why keystone encourage  migrating from port 5000 to port 35357?
In my opinion, the 5000 is a public endpoint for get tokens, the 35357 is
an admin endpoint.They may have different security strategy.
I am familiar with keystone.

2012/7/6 Pete Zaitcev 

> On Thu, 5 Jul 2012 18:21:48 +0800
> Alex Yang  wrote:
>
> > If i want use the cloudfiles with the keystone(the newest version), I
> need
> > change the code of cloudfiles by myself?
>
> Indeed, clients have to be adapted in cases like yours. However,
> please look at python-swiftclient. It may be sufficiently similar
> so you don't need to change much in your application that uses
> python-cloudfiles.
>
> By the way, it was my understanding that Keystone people encourage
> everyone to migrate from port 5000 to port 35357. The latter is
> assigned with IANA.
>
> -- Pete
>



-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Swift: tempURL

[Alex Yang]

tempurl dose not work with keystone. you must follow the instruction of
tempurl to use it. the instruction is locate in the comment of
swift/common/middleware/tempurl.py
在 2012-5-15 上午2:06，"Suchi Sinha (susinha)" 写道：

>   I am trying to run swift  temp url  feature. We have keystone as
> identity service. 
>
> Does this feature works with  keystone? 
>
> ** **
>
> I am always getting “no such file or directory”.
>
> I am  following all  the steps   generate the tempURL.
>
> ** **
>
> I will  appreciate any  help.
>
> ** **
>
> Thanks.
>
> ~Suchi 
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

[Alex Yang]

In my view, the biggest problem of swift is not the new features but the
improvement of performance.

At first, we knew that container-updater, *-auditor, *-replicator will loop
all the files in the disk during the running interval. When the number of
files is very large, the process of replicator, updater and auditor will
spend much time. So, the time of eventual consistency is very long.

Our practice of  Sina Web Service Team https://launchpad.net/~sws:

total accounts:  121,961;

total containers:160,703;

total objects: 14,291,519;

total storage usage:  1.3T

account replication time:10 hours;

container replication time:  10 hours;

object replication time:   48 hours;

account audit time:   2 hours;

container audit time: 9 hours;

container update time:19 hours;

This is terrible If we develop quota upon account db. There is long time
for eventual consistency.

Secondly,  there is a vicious circle during replication. The replicator
will query the account-server, container-server and object-server to
compare the metadata, and determine whether to sync. When the number of
files is very large, the frequent query make the account-server,
container-server and object-server become a bottle neck. This will
influence the process of proxy-server to work with back-end servers. There
a lot of Timeout(10s) ERROR in proxy-server's log and the load-average is
very high.
So, some PUT, POST operations failed, and the replicator to sync, and fail
more, and sync more...
.
In my opinion, we need to improve the process of replication and container
update by using event drive framework or something else..
My leader may discuss this topic at Design Summit,
http://openstackconferencespring2012.sched.org/speaker/huicheng

2012/4/13 John Dickinson >

> Swift keeps total bytes, container, and object count (eventually)
> up-to-date in the account metadata. There are also log processing tools
> (like slogging - http://github.com/notmyname/slogging) that can provide
> usage information (including bandwidth) based on swift logs.
>
> While I think that it's appropriate for swift to generate the usage
> information (via internal processes or log processing), the appropriate
> place for quotas is in whatever system handles the concept of a user
> (normally the auth system). This way quotas are enforced by revoking or
> limiting access of the auth token.
>
> --John
>
>
> On Apr 12, 2012, at 11:53 AM, Frederik Van Hecke wrote:
>
> > Hi Kuo,
> >
> > One option would be to keep the usage information (num files, num bytes,
> etc) per container / account in an sqlite DB, just like it is done for
> account and container info.
> >
> > To avoid having to loop through all data at regular intervals (to update
> the info), additional logic could be added to the api methods to update the
> sqlite DB's when new files are added, files are deleted, etc. Such approach
> will require more lines of code, but will be far less stressful on
> performance.
> >
> > (the brute-force approach to loop through it at regular intervals will
> be hell on performance on large deployments..)
> >
> >
> > For data transfer billing based on download / upload amounts, a similar
> approach could be used.
> >
> > If no one else is looking into this, I would certainly be willing to
> help to help get this started.
> >
> >
> > Kind regards,
> > Frederik Van Hecke
> >
> > T:  +32487733713
> > E:  frede...@cluttr.be  'frede...@cluttr.be');>
> > W: www.cluttr.be
> >
> >
> >
> >
> >
> > This e-mail and any attachments thereto may contain information which is
> confidential and/or protected by intellectual property rights and are
> intended for the sole use of the recipient(s)named above. Any use of the
> information contained herein (including, but not limited to, total or
> partial reproduction, communication or distribution in any form) by persons
> other than the designated recipient(s) is prohibited. If you have received
> this e-mail in error, please notify the sender either by telephone or by
> e-mail and delete the material from any computer. Thank you for your
> cooperation.
> >
> >
> >
> > On Thu, Apr 12, 2012 at 17:45, Kuo Hugo 
> > >
> wrote:
> > Hi folks ,
> >
> > I'm thinking about the better approach to manage "an user" or "an
> account" space usage quota in swift.
> > Is  there any related blueprint or sub-project even an idea around ?
> > Any suggestion of benefits to be an external service or to be a
> middle-ware in swift-proxy ?
> >
> > I'm concerning about such feature will reduce the performance of entire
> Swift environment.
> >
> > Appreciate :>
> >
> >
> >
> > --
> > +Hugo Kuo+
> > tonyt...@gmail.com 
> > +886 935004793
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net  'openstack@lists.launchpad.net');>
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : http

[Openstack] The high CPU usage of container-server in swift

[Alex Yang]

Hi!
I faced a problem that the process of container-server performance a high
CPU usage.
I tried to profile the container-server via repoze.profile, but failed.
Any one else can give me some suggestion about profiling the
container-server?


  Thanks!
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp