Re: [ceph-users] num of placement groups created for default pools

[Snider, Tim]

Thanks for the explanation that makes sense.
Tim

-Original Message-
From: Tyler Brekke [mailto:tyler.bre...@inktank.com] 
Sent: Thursday, October 24, 2013 6:42 AM
To: Snider, Tim
Cc: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] num of placement groups created for default pools

Hey Tim,

If you deployed with ceph-deploy then your monitors started without knowledge 
of how many OSDs you will be adding to your cluster. You can add  
'osd_pool_default_pg_num' and 'osd_pool_default_pgp_num' to you ceph.conf 
before creating your monitors to have the default pools created with the proper 
number of placement groups.

I believe with the old mkcephfs script the number of osds was used to give a 
better default pg count. I don't think this is really necessary anymore as you 
can increase your placement group size now.

ceph osd pool set  pg_num  ceph osd pool set  pgp_num 


On Wed, Oct 23, 2013 at 6:13 AM, Snider, Tim  wrote:
> I have a newly created cluster with 68 osds and the default of 2 replicas. 
> The default pools  are created with 64 placement groups . The documentation 
> in http://ceph.com/docs/master/rados/operations/pools/ states  for osd pool 
> creation :
> "We recommend approximately 50-100 placement groups per OSD to balance out 
> memory and CPU requirements and per-OSD load. For a single pool of objects, 
> you can use the following formula: Total PGS = (osds *100)/Replicas"
>
> For this cluster pools should have  3200 pgs [ (64*100)/2] according to the 
> recommendation.
> Why isn't  the guideline followed for default pools?
> Maybe they're created prior to having all the osds activated?
> Maybe I'm reading the documentation incorrectly.
>
> /home/ceph/bin# ceph osd getmaxosd
> max_osd = 68 in epoch 219
> /home/ceph/bin# ceph osd getmaxosd
> max_osd = 68 in epoch 219
> /home/ceph/bin# ceph osd lspools
> 0 data,1 metadata,2 rbd,
> /home/ceph/bin# ceph osd pool get data pg_num
> pg_num: 64
> /home/ceph/bin# ceph osd pool get data size
> size: 2
>
> Thanks,
> Tim
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] num of placement groups created for default pools

[Snider, Tim]

I have a newly created cluster with 68 osds and the default of 2 replicas. The 
default pools  are created with 64 placement groups . The documentation in 
http://ceph.com/docs/master/rados/operations/pools/ states  for osd pool 
creation :
"We recommend approximately 50-100 placement groups per OSD to balance out 
memory and CPU requirements and per-OSD load. For a single pool of objects, you 
can use the following formula: Total PGS = (osds *100)/Replicas"

For this cluster pools should have  3200 pgs [ (64*100)/2] according to the 
recommendation.
Why isn't  the guideline followed for default pools? 
Maybe they're created prior to having all the osds activated? 
Maybe I'm reading the documentation incorrectly.

/home/ceph/bin# ceph osd getmaxosd
max_osd = 68 in epoch 219
/home/ceph/bin# ceph osd getmaxosd
max_osd = 68 in epoch 219
/home/ceph/bin# ceph osd lspools
0 data,1 metadata,2 rbd,
/home/ceph/bin# ceph osd pool get data pg_num
pg_num: 64
/home/ceph/bin# ceph osd pool get data size  
size: 2

Thanks,
Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] error: failed to add bucket placement

[Snider, Tim]

Can someone tell me what I'm missing. I have a radosgw user created.
I get the following complaint when I try to create a pool with or without the 
--uid parameter:

#radosgw-admin pool add --pool=radosPool --uid=rados
failed to add bucket placement: (2) No such file or directory

Thanks,
Tim

Timothy Snider
Strategic Planning & Architecture - Advanced Development
NetApp
316-636-8736 Direct Phone
316-213-0223 Mobile Phone
tim.sni...@netapp.com
netapp.com
 [Description: http://media.netapp.com/images/netapp-logo-sig-5.gif]

<>___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] changing from default journals to external journals

[Snider, Tim]

Your reply seems to contradict the reply from Sage:

> Sage -

> The journal device needs a file system created does that device need to 
be mounted?

Yes.. the mkjournal step needs to writes to the journal (wehther it's a 
file or block device).
Tim

From: Kurt Bauer [mailto:kurt.ba...@univie.ac.at]
Sent: Monday, October 21, 2013 10:16 AM
To: Snider, Tim
Cc: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] changing from default journals to external journals

Hi,

neither do you need a filesystem on the partition, nor does it have to be 
mounted. You can link the journal against the raw partition.

Best regards,
Kurt

Snider, Tim schrieb:

Sage -

The journal device needs a file system created does that device need to be 
mounted?

Tim



-Original Message-

From: Sage Weil [mailto:s...@inktank.com]

Sent: Thursday, October 17, 2013 11:02 AM

To: Snider, Tim

Cc: ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>

Subject: RE: [ceph-users] changing from default journals to external journals



On Thu, 17 Oct 2013, Snider, Tim wrote:

/var/lib/ceph/osd/ceph-NNN/journal is a "real" file on my system:

 ls -l  /var/lib/ceph/osd/ceph-0/journal

 -rw-r--r-- 1 root root 1073741824 Oct 17 06:47

/var/lib/ceph/osd/ceph-0/journal



Any problems with my proposed added steps (3 - 5)?



1.   stop a ceph-osd daemon

2.   ceph-osd --flush-journal -i NNN

3.   (+) cp /var/lib/ceph/osd/ceph-NNN/journal 
/var/lib/ceph/osd/ceph-NNN/journal.saved

4.   (+) create a new empty journal file:

 touch /mount/sdzNNN/journal

5.   (+) Create the symbolic link for Ceph:

 ln -s /mount/sdzNNN/journal var/lib/ceph/osd/ceph-NNN/journal



3. mv journal journal.old

4. ln -s /dev/whatever journal



(you're better off using a raw partition or other block device than a

file.)  or,



4. touch /new/path

5. ln -s /other/path journal



6.   ceph-osd --mkjournal -i NNN

7.   start ceph-osd



Tim

-Original Message-

From: Sage Weil [mailto:s...@inktank.com]

Sent: Wednesday, October 16, 2013 5:02 PM

To: Snider, Tim

Cc: ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>

Subject: Re: [ceph-users] changing from default journals to external

journals



On Wed, 16 Oct 2013, Snider, Tim wrote:

I configured my cluster using the default journal location for my

osds. Can I migrate the default journals to explicit separate

devices without a complete cluster teardown and reinstallation? How?

- stop a ceph-osd daemon, then

- ceph-osd --flush-journal -i NNN

- set/adjust the journal symlink at /var/lib/ceph/osd/ceph-NNN/journal to

  point wherever you want

- ceph-osd --mkjournal -i NNN

- start ceph-osd



This won't set up the udev magic on the journal device, but that doesn't really 
matter if you're not hotplugging devices.



sage





___

ceph-users mailing list

ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>

http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] changing from default journals to external journals

[Snider, Tim]

Sage - 
The journal device needs a file system created does that device need to be 
mounted?
Tim

-Original Message-
From: Sage Weil [mailto:s...@inktank.com] 
Sent: Thursday, October 17, 2013 11:02 AM
To: Snider, Tim
Cc: ceph-users@lists.ceph.com
Subject: RE: [ceph-users] changing from default journals to external journals

On Thu, 17 Oct 2013, Snider, Tim wrote:
> /var/lib/ceph/osd/ceph-NNN/journal is a "real" file on my system:
>  ls -l  /var/lib/ceph/osd/ceph-0/journal 
>  -rw-r--r-- 1 root root 1073741824 Oct 17 06:47 
> /var/lib/ceph/osd/ceph-0/journal
> 
> Any problems with my proposed added steps (3 - 5)?
> 
> 1.stop a ceph-osd daemon
> 2.ceph-osd --flush-journal -i NNN
> 3.(+) cp /var/lib/ceph/osd/ceph-NNN/journal 
> /var/lib/ceph/osd/ceph-NNN/journal.saved
> 4.(+) create a new empty journal file: 
>   touch /mount/sdzNNN/journal
> 5.(+) Create the symbolic link for Ceph:
>   ln -s /mount/sdzNNN/journal 
> var/lib/ceph/osd/ceph-NNN/journal

3. mv journal journal.old
4. ln -s /dev/whatever journal

(you're better off using a raw partition or other block device than a
file.)  or,

4. touch /new/path
5. ln -s /other/path journal

> 6.ceph-osd --mkjournal -i NNN
> 7.start ceph-osd
> 
> Tim
> -Original Message-
> From: Sage Weil [mailto:s...@inktank.com]
> Sent: Wednesday, October 16, 2013 5:02 PM
> To: Snider, Tim
> Cc: ceph-users@lists.ceph.com
> Subject: Re: [ceph-users] changing from default journals to external 
> journals
> 
> On Wed, 16 Oct 2013, Snider, Tim wrote:
> > I configured my cluster using the default journal location for my 
> > osds. Can I migrate the default journals to explicit separate 
> > devices without a complete cluster teardown and reinstallation? How?
> 
> - stop a ceph-osd daemon, then
> - ceph-osd --flush-journal -i NNN
> - set/adjust the journal symlink at /var/lib/ceph/osd/ceph-NNN/journal to
>   point wherever you want
> - ceph-osd --mkjournal -i NNN
> - start ceph-osd
> 
> This won't set up the udev magic on the journal device, but that doesn't 
> really matter if you're not hotplugging devices.
> 
> sage
> 
> 
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] changing from default journals to external journals

[Snider, Tim]

/var/lib/ceph/osd/ceph-NNN/journal is a "real" file on my system:
 ls -l  /var/lib/ceph/osd/ceph-0/journal 
 -rw-r--r-- 1 root root 1073741824 Oct 17 06:47 
/var/lib/ceph/osd/ceph-0/journal

Any problems with my proposed added steps (3 - 5)?

1.  stop a ceph-osd daemon
2.  ceph-osd --flush-journal -i NNN
3.  (+) cp /var/lib/ceph/osd/ceph-NNN/journal 
/var/lib/ceph/osd/ceph-NNN/journal.saved
4.  (+) create a new empty journal file: 
touch /mount/sdzNNN/journal
5.  (+) Create the symbolic link for Ceph:
ln -s /mount/sdzNNN/journal 
var/lib/ceph/osd/ceph-NNN/journal
6.  ceph-osd --mkjournal -i NNN
7.  start ceph-osd

Tim
-Original Message-
From: Sage Weil [mailto:s...@inktank.com] 
Sent: Wednesday, October 16, 2013 5:02 PM
To: Snider, Tim
Cc: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] changing from default journals to external journals

On Wed, 16 Oct 2013, Snider, Tim wrote:
> I configured my cluster using the default journal location for my 
> osds. Can I migrate the default journals to explicit separate devices 
> without a complete cluster teardown and reinstallation? How?

- stop a ceph-osd daemon, then
- ceph-osd --flush-journal -i NNN
- set/adjust the journal symlink at /var/lib/ceph/osd/ceph-NNN/journal to
  point wherever you want
- ceph-osd --mkjournal -i NNN
- start ceph-osd

This won't set up the udev magic on the journal device, but that doesn't really 
matter if you're not hotplugging devices.

sage
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] changing from default journals to external journals

[Snider, Tim]

I configured my cluster using the default journal location for my osds. Can I 
migrate the default journals to explicit separate devices without a complete 
cluster teardown and reinstallation? How?

Thanks,
Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Ceph cluster access using s3 api with curl

[Snider, Tim]

Rookie question: What's the curl command / URL / steps to get an authentication 
token from the cluster without using the swift debug command first.
Using the swift_key values should work but I haven't found the right 
combination /url.
Here's what I've done:
1: Get user info from ceph cluster:
# radosgw-admin user info --uid rados
2013-10-16 13:29:42.956578 7f166aeef780  0 WARNING: cannot read region 
map
{ "user_id": "rados",
  "display_name": "rados",
  "email": "n...@none.com",
  "suspended": 0,
  "max_buckets": 1000,
  "auid": 0,
  "subusers": [],
  "keys": [
{ "user": "rados",
  "access_key": "V92UJ5F24DF2CDGQINTK",
  "secret_key": "uzWaCMQnZ8uxyR3zte2Dthxbca\/H4qsm3p0QI29f"}],
  "swift_keys": [
{ "user": "rados:swift",
  "secret_key": "123"}],
  "caps": [],
  "op_mask": "read, write, delete",
  "default_placement": "",
  "placement_tags": []}

2:  Jump thru the (unnecessary) Swift deubg hoop. Debug truncated the http 
command that holds the key:
# swift --verbose --debug -V 1.0 -A http://10.113.193.189/auth -U 
rados:swift  -K 123 list
DEBUG:swiftclient:REQ: curl -i http://10.113.193.189/auth -X GET

DEBUG:swiftclient:RESP STATUS: 204

DEBUG:swiftclient:REQ: curl -i 
http://10.113.193.189/swift/v1?format=json -X GET -H "X-Auth-Token: 
AUTH_rgwtk0b007261646f733a73776966740ddca424fed74e69be4860524846912b0f99a7531ecda91ae47684ebd6b69e40f1dc6b45"

DEBUG:swiftclient:RESP STATUS: 200

DEBUG:swiftclient:RESP BODY: []

3: I should be able to pass user and password values from the user info 
command. I haven't found the correct url or path to use.  This command (and 
variations : auth/v1.0 ...) fails. Is the directory structure / URL to get an 
authentication token documented somewhere?
# curl -i http://10.113.193.189/auth -X GET -H 'X-Storage-User: 
rados:swift' -H 'X-Storage-Pass: 123'
HTTP/1.1 403 Forbidden
Date: Wed, 16 Oct 2013 20:33:31 GMT
Server: Apache/2.2.22 (Ubuntu)
Accept-Ranges: bytes
Content-Length: 23
Content-Type: application/json

{"Code":"AccessDenied"}

Thanks,
Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] xfs log device and osd journal specifications in ceph.conf

[Snider, Tim]

3 questions:

1.   I'd like to use xfs devices with a separate log device in a ceph cluster. 
What's the best way to do this?  Is it possible to specify xfs log devices in 
the [osd.x] sections of ceph.conf? 
E.G.:
[osd.0]
host = delta
devs = /dev/sdx
osd mkfs options xfs = -d su=131072,sw=8 -i size=1024 -l 
logdev=/dev/sdq1,su=131072

[osd.1]
host = epsilon
devs = /dev/sdy
osd mkfs options xfs = -d su=131072,sw=8 -i size=1024 -l 
logdev=/dev/sdq2,su=131072

2.Is this the correct syntax for the line without the log device options?
 osd mkfs options xfs = -d su=131072,sw=8 -i size=1024

3. For osd journal devices. I assume there's a 1:1 relationship between osds 
and journal devices.  The section in sample.ceph.conf seems to imply a single 
entry.
Should there be an osd journal entry in each [osd.x] section of ceph.conf? 

[osd]
; This is where the osd expects its data
osd data = /data/$name

; Ideally, make the journal a separate disk or partition.
; 1-10GB should be enough; more if you have fast or many
; disks.  You can use a file under the osd data dir if need be
; (e.g. /data/$name/journal), but it will be slower than a
; separate disk or partition.
; This is an example of a file-based journal.
osd journal = /data/$name/journal
osd journal size = 1000 ; journal size, in megabytes

On my cluster (deployed with ceph-deploy) the data is in /var/lib/ceph/osd. Not 
/data/$name as in the sample file. Directory organization on my cluster:
/var/lib/ceph/osd/:
ceph-0  ceph-10  ceph-12  ceph-14  ceph-16  ceph-18  ceph-2   ceph-21  
ceph-3  ceph-5  ceph-7  ceph-9
ceph-1  ceph-11  ceph-13  ceph-15  ceph-17  ceph-19  ceph-20  ceph-22  
ceph-4  ceph-6  ceph-8

/var/lib/ceph/osd/ceph-0:

/var/lib/ceph/osd/ceph-1:

ls /data
ls: cannot access /data: No such file or directory
Thanks,
Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ...-all-starter documentation available?

[Snider, Tim]

I was poking around on a node, found the following executables:
radosgw-all-starter
ceph-mds-all-starter
ceph-mon-all-starter
ceph-osd-all-starter
A  Ceph web page search yielded no results. Does documentation exist? Where?

Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] increase # of mons and use haproxy to increase ceph performance?

[Snider, Tim]

Does inceasing the number of monitors affect  Ceph cluster performance 
(everything else remaining ==)? If it does I hope it's positive.
And - will accessing Ceph monitors thru a haproxy server also improve 
performance?

Thanks,
Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph access using curl

[Snider, Tim]

The –debug command worked as described. Can anyone give me a synposis of how 
the authentication token is generated?
   Token generated: 
AUTH_rgwtk0b007261646f733a7377696674046eff2c9ac6a5041b00545248a7893b900677683adaaca1095128b6edf8fc378d7d49d8"

The first part looks like a header: 'AUTH_rgwtk', (rados gateway token)??  
length == 11  and a prefix == rados:swift
AUTH_rgwtk0b0072 61 64 6f 73 3a 73 77 69 66 74
Length = 0xb  r  a  d  o  s  :  s  w  i  f  t

What is the token body -- How is the token body encoded? It's not a straight 
match of any keys shown by the radowgw-admin command?
046eff2c 9ac6a504 1b00545 248a7893 b9006776 83adaaca 1095128 b6edf8fc 378d7d49 
d8

root@controller21:~/ssbench-0.2.16# radosgw-admin user info --uid=rados
2013-10-07 05:55:34.804639 7ff1c3f6c780  0 WARNING: cannot read region map
{ "user_id": "rados",
  "display_name": "rados",
  "email": "n...@none.com",
  "suspended": 0,
  "max_buckets": 1000,
  "auid": 0,
  "subusers": [
{ "id": "rados:swift",
  "permissions": "full-control"},
{ "id": "rados:swift1",
  "permissions": "full-control"}],
  "keys": [
{ "user": "rados",
  "access_key": "R5F0D2UCSK3618DJ829A",
  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
  "swift_keys": [
{ "user": "rados:swift",
  "secret_key": "77iJvemrxWvYk47HW7pxsL+eHdA53AtLl2T0OyuG"},
{ "user": "rados:swift1",
  "secret_key": "l9Xlg66JvbNvMmZAj91AeQByEiP8R8sBahCJeqAG"}],
  "caps": [],
  "op_mask": "read, write, delete",
  "default_placement": "",
  "placement_tags": []}

swift --debug -V 1.0 -A 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/auth -U rados:swift -K 
"77iJvemrxWvYk47HW7pxsL+eHdA53AtLl2T0OyuG"  list

DEBUG:swiftclient:REQ: curl -i 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/auth -X GET
DEBUG:swiftclient:RESP STATUS: 204
DEBUG:swiftclient:REQ: curl -i 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/swift/v1?format=json -X GET 
-H "X-Auth-Token: 
AUTH_rgwtk0b007261646f733a7377696674046eff2c9ac6a5041b00545248a7893b900677683adaaca1095128b6edf8fc378d7d49d8"

DEBUG:swiftclient:RESP STATUS: 200

DEBUG:swiftclient:RESP BODY: 
[{"name":"ssbench_00","count":832,"bytes":85196800},...{"name":"xxx","count":1,"bytes":604}]
DEBUG:swiftclient:REQ: curl -i 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/swift/v1?format=json&marker=xxx
 -X GET -H "X-Auth-Token: 
AUTH_rgwtk0b007261646f733a7377696674046eff2c9ac6a5041b00545248a7893b900677683adaaca1095128b6edf8fc378d7d49d8"
DEBUG:swiftclient:RESP STATUS: 200
DEBUG:swiftclient:RESP BODY: []

AUTH_rgwtk0b007261646f733a7377696674
  r a d o s : s w i f t
046eff2c 9ac6a504 1b00545 248a7893 b9006776 83adaaca 1095128 b6edf8fc 378d7d49 
d8

From: Darren Birkett [mailto:darren.birk...@gmail.com] 
Sent: Saturday, October 05, 2013 1:00 AM
To: Snider, Tim; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] ceph access using curl

Try using passing '--debug' to the swift command. It should output the 
equivalent curl command for you to use.

- Darren
"Snider, Tim"  wrote:
I'm having pilot error with getting the path correct using curl.
Bucket listing using "radosgw-admin bucket list" works as does the swift API.
Can someone point out my (obvious) error?
 
Bucket list works:
root@controller21:/home/ceph/my-cluster#  radosgw-admin bucket list  
2013-10-04 11:28:13.144065 7fe5f26fe780  0 WARNING: cannot read region map
[
    "xxx"]
 
Swift API to Ceph works
    swift -V 1.0 -A 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/auth -U rados:swift -K 
"xxx"  stat
   Account: v1
    Containers: 1
   Objects: 1
 Bytes: 604
  Vary: Accept-Encoding
    Server: Apache/2.2.22 (Ubuntu)
    X-Account-Bytes-Used-Actual: 4096
    Content-Type: text/plain; charset=utf-8
    root@controller21:/home/ceph/my-cluster# swift -V 1.0 -A 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/auth -U rados:swift -K 
"77iJvemrxWvYk47HW7pxsL+eHdA53AtLl2T0OyuG"  list
    xxx
 
However I can’t get the correct http path. Variations of auth/v1.0, auth/v1 … 
don’t work either:
    root@controller21:/home/ceph/my-cluster# curl -v -H 'X-Storage-User: 
rados:swift' -H 'X-Storage-Pass: xxx'  
http://ictp-R2C4-Cont

[ceph-users] ceph access using curl

[Snider, Tim]

I'm having pilot error with getting the path correct using curl.
Bucket listing using "radosgw-admin bucket list" works as does the swift API.
Can someone point out my (obvious) error?

Bucket list works:
root@controller21:/home/ceph/my-cluster#  radosgw-admin bucket list
2013-10-04 11:28:13.144065 7fe5f26fe780  0 WARNING: cannot read region map
[
"xxx"]

Swift API to Ceph works
swift -V 1.0 -A 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/auth -U rados:swift -K 
"xxx"  stat
   Account: v1
Containers: 1
   Objects: 1
 Bytes: 604
  Vary: Accept-Encoding
Server: Apache/2.2.22 (Ubuntu)
X-Account-Bytes-Used-Actual: 4096
Content-Type: text/plain; charset=utf-8
root@controller21:/home/ceph/my-cluster# swift -V 1.0 -A 
http://ictp-R2C4-Controller21.ict.englab.netapp.com/auth -U rados:swift -K 
"77iJvemrxWvYk47HW7pxsL+eHdA53AtLl2T0OyuG"  list
xxx

However I can't get the correct http path. Variations of auth/v1.0, auth/v1 ... 
don't work either:
root@controller21:/home/ceph/my-cluster# curl -v -H 'X-Storage-User: 
rados:swift' -H 'X-Storage-Pass: xxx'  
http://ictp-R2C4-Controller21.ict.englab.netapp.com/auth
* About to connect() to ictp-R2C4-Controller21.ict.englab.netapp.com 
port 80 (#0)
*   Trying 10.113.193.189... connected
> GET /auth HTTP/1.1
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 
OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: ictp-R2C4-Controller21.ict.englab.netapp.com
> Accept: */*
> X-Storage-User: rados:swift
> X-Storage-Pass: 77iJvemrxWvYk47HW7pxsL+eHdA53AtLl2T0OyuG
>
< HTTP/1.1 403 Forbidden
< Date: Fri, 04 Oct 2013 18:30:01 GMT
< Server: Apache/2.2.22 (Ubuntu)
< Accept-Ranges: bytes
< Content-Length: 23
< Content-Type: application/json
<
* Connection #0 to host ictp-R2C4-Controller21.ict.englab.netapp.com 
left intact
* Closing connection #0
{"Code":"AccessDenied"}root@controller21:/home/ceph/my-cluster#

Thanks,
Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rdosgw swift subuser creation

[Snider, Tim]

Thanks that worked - you were close: 
This is another document issue on http://ceph.com/docs/next/radosgw/config/ , 
--gen-secret parameter requirement isn't mentioned.
Enabling Swift Access
Allowing access to the object store with Swift (OpenStack 
Object Storage) compatible clients requires an additional step; 
namely, the creation of a subuser and a Swift access key.
  sudo radosgw-admin subuser create --uid=johndoe 
--subuser=johndoe:swift --access=full
  sudo radosgw-admin key create --subuser=johndoe:swift 
--key-type=swift

radosgw-admin key create   --subuser=rados:swift   --key-type=swift --gen-secret
2013-09-27 14:46:40.202708 7f25c5d70780  0 WARNING: cannot read region map
{ "user_id": "rados",
  "display_name": "rados",
  "email": "n...@none.com",
  "suspended": 0,
  "max_buckets": 1000,
  "auid": 0,
  "subusers": [
{ "id": "rados:swift",
  "permissions": "full-control"}],
  "keys": [
{ "user": "rados",
  "access_key": "R5F0D2UCSK3618DJ829A",
  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
  "swift_keys": [
{ "user": "rados:swift",
  "secret_key": "77iJvemrxWvYk47HW7pxsL+eHdA53AtLl2T0OyuG"}],
  "caps": [],
  "op_mask": "read, write, delete",
  "default_placement": "",
  "placement_tags": []}
-Original Message-
From: Matt McNulty [mailto:ma...@codero.com] 
Sent: Friday, September 27, 2013 4:40 PM
To: Snider, Tim; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] rdosgw swift subuser creation

Hi Tim,

Try adding --gen-key to your create command (you should be able to create a key 
for the subuser you already created).

Thanks,
Matt

On 9/27/13 4:35 PM, "Snider, Tim"  wrote:

>I created an rdosgw user and swift subuser and attempted to generate a 
>key for the swift user. Using the commands below. However the swift key 
>was empty when the command completed. What did I miss?
>
>root@controller21:/etc# radosgw-admin user create --uid=rados 
>--display-name=rados --email=n...@none.com
>2013-09-27 13:34:08.155162 7f984f0a5780  0 WARNING: cannot read region 
>map { "user_id": "rados",
>  "display_name": "rados",
>  "email": "n...@none.com",
>  "suspended": 0,
>  "max_buckets": 1000,
>  "auid": 0,
>  "subusers": [],
>  "keys": [
>{ "user": "rados",
>  "access_key": "R5F0D2UCSK3618DJ829A",
>  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
>  "swift_keys": [],
>  "caps": [],
>  "op_mask": "read, write, delete",
>  "default_placement": "",
>  "placement_tags": []}
>
>root@controller21:/etc# radosgw-admin subuser create  --uid=rados
>--subuser=rados:swift   --key-type=swift --access=full
>2013-09-27 13:34:58.761911 7f5307c04780  0 WARNING: cannot read region 
>map { "user_id": "rados",
>  "display_name": "rados",
>  "email": "n...@none.com",
>  "suspended": 0,
>  "max_buckets": 1000,
>  "auid": 0,
>  "subusers": [
>{ "id": "rados:swift",
>  "permissions": "full-control"}],
>  "keys": [
>{ "user": "rados",
>  "access_key": "R5F0D2UCSK3618DJ829A",
>  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
>  "swift_keys": [],
>  "caps": [],
>  "op_mask": "read, write, delete",
>  "default_placement": "",
>  "placement_tags": []}
>
>root@controller21:/etc# radosgw-admin key create   --subuser=rados:swift
> --key-type=swift
>2013-09-27 13:35:43.544005 7f599e672780  0 WARNING: cannot read region 
>map { "user_id": "rados",
>  "display_name": "rados",
>  "email": "n...@none.com",
>  "suspended": 0,
>  "max_buckets": 1000,
>  "auid": 0,
>  "subusers": [
>{ "id": "rados:swift",
>  "permissions": "full-control"}],
>  "keys": [
>{ "user": "rados",
>  "access_key": "R5F0D2UCSK3618DJ829A",
>  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
>  "swift_keys": [
>{ "user": "rados:swift",
>  "secret_key": ""}],
>  "caps": [],
>  "op_mask": "read, write, delete",
>  "default_placement": "",
>  "placement_tags": []}
>
>Thanks,
>Tim
>
>Timothy Snider
>Strategic Planning & Architecture - Advanced Development NetApp
>316-636-8736 Direct Phone
>316-213-0223 Mobile Phone
>tim.sni...@netapp.com
>netapp.com
> 
>
>
>___
>ceph-users mailing list
>ceph-users@lists.ceph.com
>http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] rdosgw swift subuser creation

[Snider, Tim]

I created an rdosgw user and swift subuser and attempted to generate a key for 
the swift user. Using the commands below. However the swift key was empty when 
the command completed. What did I miss? 

root@controller21:/etc# radosgw-admin user create --uid=rados 
--display-name=rados --email=n...@none.com
2013-09-27 13:34:08.155162 7f984f0a5780  0 WARNING: cannot read region map
{ "user_id": "rados",
  "display_name": "rados",
  "email": "n...@none.com",
  "suspended": 0,
  "max_buckets": 1000,
  "auid": 0,
  "subusers": [],
  "keys": [
{ "user": "rados",
  "access_key": "R5F0D2UCSK3618DJ829A",
  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
  "swift_keys": [],
  "caps": [],
  "op_mask": "read, write, delete",
  "default_placement": "",
  "placement_tags": []}

root@controller21:/etc# radosgw-admin subuser create  --uid=rados  
--subuser=rados:swift   --key-type=swift --access=full
2013-09-27 13:34:58.761911 7f5307c04780  0 WARNING: cannot read region map
{ "user_id": "rados",
  "display_name": "rados",
  "email": "n...@none.com",
  "suspended": 0,
  "max_buckets": 1000,
  "auid": 0,
  "subusers": [
{ "id": "rados:swift",
  "permissions": "full-control"}],
  "keys": [
{ "user": "rados",
  "access_key": "R5F0D2UCSK3618DJ829A",
  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
  "swift_keys": [],
  "caps": [],
  "op_mask": "read, write, delete",
  "default_placement": "",
  "placement_tags": []}

root@controller21:/etc# radosgw-admin key create   --subuser=rados:swift   
--key-type=swift
2013-09-27 13:35:43.544005 7f599e672780  0 WARNING: cannot read region map
{ "user_id": "rados",
  "display_name": "rados",
  "email": "n...@none.com",
  "suspended": 0,
  "max_buckets": 1000,
  "auid": 0,
  "subusers": [
{ "id": "rados:swift",
  "permissions": "full-control"}],
  "keys": [
{ "user": "rados",
  "access_key": "R5F0D2UCSK3618DJ829A",
  "secret_key": "PJR1rvV2+Xrzlwo+AZZKXextsDl45EaLljzopgjD"}],
  "swift_keys": [
{ "user": "rados:swift",
  "secret_key": ""}],
  "caps": [],
  "op_mask": "read, write, delete",
  "default_placement": "",
  "placement_tags": []}

Thanks,
Tim

Timothy Snider
Strategic Planning & Architecture - Advanced Development
NetApp
316-636-8736 Direct Phone
316-213-0223 Mobile Phone
tim.sni...@netapp.com
netapp.com
 


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph.conf changes and restarting ceph.

[Snider, Tim]

Is the form: auth cluster required = none or auth_cluster_required = none? 
("_"s as a word separator)

-Original Message-
From: John Wilkins [mailto:john.wilk...@inktank.com] 
Sent: Tuesday, September 24, 2013 11:43 AM
To: Aronesty, Erik
Cc: Snider, Tim; Gary Mazzaferro; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] ceph.conf changes and restarting ceph.

From your pastie details, it looks like you are using "auth supported = none".  
That's pre 0.51, as noted in the documentation. Perhaps I should omit the old 
usage or omit it entirely.

It should look like this:

auth cluster required = none
auth service required = none
auth client required = none

not

auth supported = none

On Tue, Sep 24, 2013 at 8:00 AM, Aronesty, Erik 
 wrote:
> I did the same thing, restarted with upstart, and I still need to use
> authentication.   Not sure why yet.   Maybe I didn’t change the /etc/ceph
> configs on all the nodes….
>
>
>
> From: ceph-users-boun...@lists.ceph.com 
> [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Snider, Tim
> Sent: Tuesday, September 24, 2013 9:15 AM
> To: Gary Mazzaferro; John Wilkins
> Cc: ceph-users@lists.ceph.com
>
>
> Subject: Re: [ceph-users] ceph.conf changes and restarting ceph.
>
>
>
> Authentication works. I was interested in trying it without authentication.
> I didn’t see the upstart link earlier.
>
> Is the plan to only use upstart and not service for Dumpling and beyond?
>
> Tim
>
>
>
> From: Gary Mazzaferro [mailto:ga...@oedata.com]
> Sent: Tuesday, September 24, 2013 1:16 AM
> To: John Wilkins
> Cc: Snider, Tim; ceph-users@lists.ceph.com
> Subject: Re: [ceph-users] ceph.conf changes and restarting ceph.
>
>
>
> Hi John
>
>
>
> Why ? do the 'service' scripts not work ? (sorry I don't have access 
> to the systems from my location) I used dumpling and ceph-deploy on debian.
>
>
>
> -gary
>
>
>
>
>
>
>
>
>
> On Mon, Sep 23, 2013 at 11:25 PM, John Wilkins 
> 
> wrote:
>
> I will update the Cephx docs. The usage in those docs for restarting 
> is for Debian/Ubuntu deployed with mkcephfs.  If you are using 
> Dumpling and deployed with ceph-deploy, you will need to use Upstart.
> See
> http://ceph.com/docs/master/rados/operations/operating/#running-ceph-w
> ith-upstart for details. If you are using Ceph on RHEL, CentOS, etc., 
> use sysvinit.
>
>
> On Mon, Sep 23, 2013 at 3:21 PM, Gary Mazzaferro  wrote:
>> Tim
>>
>> Did it work with authentication enabled  ?
>>
>> -gary
>>
>>
>> On Mon, Sep 23, 2013 at 2:10 PM, Snider, Tim 
>> wrote:
>>>
>>> I modified /etc/ceph.conf for no authentication and to specify both 
>>> private and public networks. /etc/ceph/ceph.conf was distributed to 
>>> all nodes in the cluster
>>>
>>> ceph was restarted on all nodes using  "service ceph -a restart".
>>>
>>> After that authentication is still required and no ports are open on 
>>> the cluster facing (192.168.10.0) network.
>>>
>>> Details in  http://pastie.org/8349534.
>>>
>>> What am I missing something?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Tim
>>>
>>>
>>> ___
>>> ceph-users mailing list
>>> ceph-users@lists.ceph.com
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>>
>>
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
> --
> John Wilkins
> Senior Technical Writer
> Intank
> john.wilk...@inktank.com
> (415) 425-9599
> http://inktank.com
>
>



--
John Wilkins
Senior Technical Writer
Intank
john.wilk...@inktank.com
(415) 425-9599
http://inktank.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph.conf changes and restarting ceph.

[Snider, Tim]

Authentication works. I was interested in trying it without authentication. I 
didn't see the upstart link earlier.
Is the plan to only use upstart and not service for Dumpling and beyond?
Tim

From: Gary Mazzaferro [mailto:ga...@oedata.com]
Sent: Tuesday, September 24, 2013 1:16 AM
To: John Wilkins
Cc: Snider, Tim; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] ceph.conf changes and restarting ceph.

Hi John

Why ? do the 'service' scripts not work ? (sorry I don't have access to the 
systems from my location) I used dumpling and ceph-deploy on debian.

-gary




On Mon, Sep 23, 2013 at 11:25 PM, John Wilkins 
mailto:john.wilk...@inktank.com>> wrote:
I will update the Cephx docs. The usage in those docs for restarting
is for Debian/Ubuntu deployed with mkcephfs.  If you are using
Dumpling and deployed with ceph-deploy, you will need to use Upstart.
See 
http://ceph.com/docs/master/rados/operations/operating/#running-ceph-with-upstart
for details. If you are using Ceph on RHEL, CentOS, etc., use
sysvinit.

On Mon, Sep 23, 2013 at 3:21 PM, Gary Mazzaferro 
mailto:ga...@oedata.com>> wrote:
> Tim
>
> Did it work with authentication enabled  ?
>
> -gary
>
>
> On Mon, Sep 23, 2013 at 2:10 PM, Snider, Tim 
> mailto:tim.sni...@netapp.com>> wrote:
>>
>> I modified /etc/ceph.conf for no authentication and to specify both
>> private and public networks. /etc/ceph/ceph.conf was distributed to all
>> nodes in the cluster
>>
>> ceph was restarted on all nodes using  "service ceph -a restart".
>>
>> After that authentication is still required and no ports are open on the
>> cluster facing (192.168.10.0) network.
>>
>> Details in  http://pastie.org/8349534.
>>
>> What am I missing something?
>>
>>
>>
>> Thanks,
>>
>> Tim
>>
>>
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>


--
John Wilkins
Senior Technical Writer
Intank
john.wilk...@inktank.com<mailto:john.wilk...@inktank.com>
(415) 425-9599
http://inktank.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ceph.conf changes and restarting ceph.

[Snider, Tim]

I modified /etc/ceph.conf for no authentication and to specify both private and 
public networks. /etc/ceph/ceph.conf was distributed to all nodes in the cluster
ceph was restarted on all nodes using  "service ceph -a restart".
After that authentication is still required and no ports are open on the 
cluster facing (192.168.10.0) network.
Details in  http://pastie.org/8349534.
What am I missing something?

Thanks,
Tim
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com