Re: v0.56 released

2013-01-03 Thread norbi
Hi List, after upgrading from 0.55.1 to 0.56 some MONs are crashing during the upgrade. I have 3 MONs with 0.55.1, mon.a, mon.b. and mon.c So now i am upgrading mon.a to 0.56, i restarted mon.a and see that mon.c is crashed... so i restarted mon.c and see, now mon.b is crashed, after

Re: status monitoring options for ceph cluster

2013-01-03 Thread Ugis
Hopefully there will be some place where community can share their ceph how-to, hope to see what Paul offers. Wiki could be suitable solution. As I see at present there seems no ready solution for fine grained ceph monitoring. Therefore I can express what I would imagine ceph monitoring look

Re: status monitoring options for ceph cluster

2013-01-03 Thread Wido den Hollander
On 01/03/2013 12:05 AM, Dan Mick wrote: (and of course it makes sense to build those on libraries that could serve other access methods) I'd prefer a library :-) This way you can build your own binaries, bindings for languages like Python, PHP and Java. The binaries could just link against

Re: v0.56 released

2013-01-03 Thread Joao Eduardo Luis
On 01/03/2013 08:28 AM, norbi wrote: Hi List, after upgrading from 0.55.1 to 0.56 some MONs are crashing during the upgrade. I have 3 MONs with 0.55.1, mon.a, mon.b. and mon.c So now i am upgrading mon.a to 0.56, i restarted mon.a and see that mon.c is crashed... so i restarted mon.c and see,

hit suicide timeout message after upgrade to 0.56

2013-01-03 Thread Wido den Hollander
Hi, I updated my 10 node 40 OSD cluster from 0.48 to 0.56 yesterday evening and found out this morning that I had 23 OSDs still up and in. Investigating some logs I found these messages: * -8 2013-01-02 21:13:40.528936

Looking to Use Ceph

2013-01-03 Thread emyr.james
Hi, I'm thinking of starting to use ceph initially for evaluation...seeing how it compares to our existing lustre file system. One thing that I would like confirmation of is how ceph stores large files. If I store a large file in CephFS is it automatically split up into chunks with the

Re: Looking to Use Ceph

2013-01-03 Thread Wido den Hollander
On 01/03/2013 01:29 PM, emyr.james wrote: Hi, I'm thinking of starting to use ceph initially for evaluation...seeing how it compares to our existing lustre file system. One thing that I would like confirmation of is how ceph stores large files. If I store a large file in CephFS is it

Re: Looking to Use Ceph

2013-01-03 Thread Mark Nelson
On 01/03/2013 06:29 AM, emyr.james wrote: Hi, I'm thinking of starting to use ceph initially for evaluation...seeing how it compares to our existing lustre file system. One thing that I would like confirmation of is how ceph stores large files. If I store a large file in CephFS is it

radosgw segfault in 0.56

2013-01-03 Thread Sylvain Munaut
Hi, I've just updated a test cluster to 0.56 and I'm getting a segfault when doing requests on radosgw : root@ceph /var/log/ceph # /usr/bin/radosgw -n client.radosgw.gateway -f *** Caught signal (Segmentation fault) ** in thread 7fee095f8700 ceph version 0.56

Re: status monitoring options for ceph cluster

2013-01-03 Thread Sage Weil
On Thu, 3 Jan 2013, Wido den Hollander wrote: On 01/03/2013 12:05 AM, Dan Mick wrote: (and of course it makes sense to build those on libraries that could serve other access methods) I'd prefer a library :-) This way you can build your own binaries, bindings for languages like Python, PHP

Re: hit suicide timeout message after upgrade to 0.56

2013-01-03 Thread Sage Weil
Hi Wido, On Thu, 3 Jan 2013, Wido den Hollander wrote: Hi, I updated my 10 node 40 OSD cluster from 0.48 to 0.56 yesterday evening and found out this morning that I had 23 OSDs still up and in. Investigating some logs I found these messages: This sounds quite a bit #3714. You might give

Re: kernel rbd format=2

2013-01-03 Thread Gregory Farnum
Alex has been doing a lot of work to support this lately, but I don't think he's sketched out the actual implementation timeline yet. -Greg On Mon, Dec 17, 2012 at 4:10 PM, Chris Dunlop ch...@onthe.net.au wrote: Hi, Format 2 images (and attendant layering support) are not yet supported by the

Re: kernel rbd format=2

2013-01-03 Thread Alex Elder
On 01/03/2013 11:12 AM, Gregory Farnum wrote: Alex has been doing a lot of work to support this lately, but I don't think he's sketched out the actual implementation timeline yet. -Greg On Mon, Dec 17, 2012 at 4:10 PM, Chris Dunlop ch...@onthe.net.au wrote: Hi, Format 2 images (and

[PATCH REPOST] libceph: reformat __reset_osd()

2013-01-03 Thread Alex Elder
Reformat __reset_osd() into three distinct blocks of code handling the three return cases. Signed-off-by: Alex Elder el...@inktank.com --- net/ceph/osd_client.c | 33 +++-- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/net/ceph/osd_client.c

[PATCH REPOST 0/4] rbd: four minor patches

2013-01-03 Thread Alex Elder
I'm re-posting my patch backlog, in chunks that may or may not match how they got posted before. This series contains some pretty fairly straightforward changes. -Alex [PATCH REPOST 1/4] rbd: document rbd_spec structure [PATCH REPOST 2/4] rbd: kill

[PATCH REPOST 1/4] rbd: document rbd_spec structure

2013-01-03 Thread Alex Elder
I promised Josh I would document whether there were any restrictions needed for accessing fields of an rbd_spec structure. This adds a big block of comments that documents the structure and how it is used--including the fact that we don't attempt to synchronize access to it. Signed-off-by: Alex

[PATCH REPOST 2/4] rbd: kill rbd_spec-image_name_len

2013-01-03 Thread Alex Elder
There may have been a benefit to hanging on to the length of an image name before, but there is really none now. The only time it's used is when probing for rbd images, so we can just compute the length then. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 19

[PATCH REPOST 3/4] rbd: kill rbd_spec-image_id_len

2013-01-03 Thread Alex Elder
There is no real benefit to keeping the length of an image id, so get rid of it. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 11 +++ 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index a002984..e01dbb1

[PATCH REPOST 4/4] rbd: use kmemdup()

2013-01-03 Thread Alex Elder
This replaces two kmalloc()/memcpy() combinations with a single call to kmemdup(). Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c |7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index e01dbb1..d97611e

[PATCH REPOST] ceph: define ceph_encode_8_safe()

2013-01-03 Thread Alex Elder
It's kind of a silly macro, but ceph_encode_8_safe() is the only one missing from an otherwise pretty complete set. It's not used, but neither are a couple of the others in this set. While in there, insert some whitespace to tidy up the alignment of the line-terminating backslashes in some of

[PATCH REPOST 0/4] rbd: add warnings

2013-01-03 Thread Alex Elder
This series adds a compact way of generating standardized warning messages in rbd, and then adds some code to issue warnings in a few spots. -Alex [PATCH REPOST 1/4] rbd: define and use rbd_warn() [PATCH REPOST 2/4] rbd: add warning messages for missing

[PATCH REPOST 1/4] rbd: define and use rbd_warn()

2013-01-03 Thread Alex Elder
Define a new function rbd_warn() that produces a boilerplate warning message, identifying in the resulting message the affected rbd device in the best way available. Use it in a few places that now use pr_warning(). Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 43

[PATCH REPOST 2/4] rbd: add warning messages for missing arguments

2013-01-03 Thread Alex Elder
Tell the user (via dmesg) what was wrong with the arguments provided via /sys/bus/rbd/add. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 24 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c

[PATCH REPOST 3/4] rbd: add a warning in bio_chain_clone_range()

2013-01-03 Thread Alex Elder
Add a warning in bio_chain_clone_range() to help a user determine what exactly might have led to a failure. There is only one; please say something if you disagree with the following reasoning. There are three places this can return abnormally: - Initially, if there is nothing to clone. It

[PATCH REPOST 4/4] rbd: add warnings to rbd_dev_probe_update_spec()

2013-01-03 Thread Alex Elder
Josh suggested adding warnings to this function to help users diagnose problems. Other than memory allocatino errors, there are two places where errors can be returned. Both represent problems that should have been caught earlier, and as such might well have been handled with BUG_ON() calls.

Re: radosgw segfault in 0.56

2013-01-03 Thread Lorieri
Hi Sylvain, happened to me using ubuntu packages. usually when you upgrade a package it calls all its dependencies, for ceph you have to update one by one. did you try that ? cheers On Thu, Jan 3, 2013 at 1:54 PM, Sylvain Munaut s.mun...@whatever-company.com wrote: Hi, I've just updated a

Re: hit suicide timeout message after upgrade to 0.56

2013-01-03 Thread Wido den Hollander
Hi, On 01/03/2013 05:52 PM, Sage Weil wrote: Hi Wido, On Thu, 3 Jan 2013, Wido den Hollander wrote: Hi, I updated my 10 node 40 OSD cluster from 0.48 to 0.56 yesterday evening and found out this morning that I had 23 OSDs still up and in. Investigating some logs I found these messages:

Re: hit suicide timeout message after upgrade to 0.56

2013-01-03 Thread Sage Weil
On Thu, 3 Jan 2013, Wido den Hollander wrote: Hi, On 01/03/2013 05:52 PM, Sage Weil wrote: Hi Wido, On Thu, 3 Jan 2013, Wido den Hollander wrote: Hi, I updated my 10 node 40 OSD cluster from 0.48 to 0.56 yesterday evening and found out this morning that I had 23 OSDs

Re: hit suicide timeout message after upgrade to 0.56

2013-01-03 Thread Wido den Hollander
On 01/03/2013 10:05 PM, Sage Weil wrote: On Thu, 3 Jan 2013, Wido den Hollander wrote: Hi, On 01/03/2013 05:52 PM, Sage Weil wrote: Hi Wido, On Thu, 3 Jan 2013, Wido den Hollander wrote: Hi, I updated my 10 node 40 OSD cluster from 0.48 to 0.56 yesterday evening and found out this

Re: When to use filestore xattr use omap = true

2013-01-03 Thread Mark Nelson
On 01/03/2013 03:16 PM, Roberto Aguilar wrote: Hello, I read the Filesystems section in the Hard disk and file system recommendations [1] documentation and was confused on whether or not I should be using the setting: filestore xattr use omap = true The section reads: [...] but the ext4

Re: When to use filestore xattr use omap = true

2013-01-03 Thread Roberto Aguilar
Thanks for clarifying, Mark. I made a quick change to the docs: https://github.com/rca/ceph/commit/37b57cdf0fdc5c03eeff3f5eb58ff4010ce581f6 Can I send you a pull request? Thanks, -Roberto. On Thu, Jan 3, 2013 at 1:20 PM, Mark Nelson mark.nel...@inktank.com wrote: On 01/03/2013 03:16 PM,

[PATCH REPOST 0/2] rbd: standardize some variable names

2013-01-03 Thread Alex Elder
This series just makes the names of variables for certain objects follow a consistent naming convention. -Alex [PATCH REPOST 1/2] rbd: standardize rbd_request variable names [PATCH REPOST 2/2] rbd: standardize ceph_osd_request variable names -- To

[PATCH REPOST 1/2] rbd: standardize rbd_request variable names

2013-01-03 Thread Alex Elder
There are two names used for items of rbd_request structure type: req and req_data. The former name is also used to represent items of pointers to struct ceph_osd_request. Change all variables that have these names so they are instead called rbd_req consistently. Signed-off-by: Alex Elder

[PATCH REPOST 2/2] rbd: standardize ceph_osd_request variable names

2013-01-03 Thread Alex Elder
There are spots where a ceph_osds_request pointer variable is given the name req. Since we're dealing with (at least) three types of requests (block layer, rbd, and osd), I find this slightly distracting. Change such instances to use osd_req consistently to make the abstraction represented a

[PATCH REPOST] rbd: be picky about osd request status type

2013-01-03 Thread Alex Elder
The result field in a ceph osd reply header is a signed 32-bit type, but rbd code often casually uses int to represent it. The following changes the types of variables that handle this result value to be s32 instead of int to be completely explicit about it. Only at the point we pass that result

[PATCH REPOST 0/2] rbd: simplify rbd_rq_fn()

2013-01-03 Thread Alex Elder
This series pulls the core processing out of rbd_rq_fn(), then simplifies the result a little further. -Alex [PATCH REPOST 1/2] rbd: encapsulate handling for a single request [PATCH REPOST 2/2] rbd: a little more cleanup of rbd_rq_fn() -- To unsubscribe

[PATCH REPOST 1/2] rbd: encapsulate handling for a single request

2013-01-03 Thread Alex Elder
In rbd_rq_fn(), requests are fetched from the block layer and each request is processed, looping through the request's list of bio's until they've all been consumed. Separate the handling for a single request into its own function to make it a bit easier to see what's going on. Signed-off-by:

[PATCH REPOST 2/2] rbd: a little more cleanup of rbd_rq_fn()

2013-01-03 Thread Alex Elder
Now that a big hunk in the middle of rbd_rq_fn() has been moved into its own routine we can simplify it a little more. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 50 +++--- 1 file changed, 23 insertions(+), 27 deletions(-)

Re: When to use filestore xattr use omap = true

2013-01-03 Thread Dan Mick
Thanks for the help, Roberto. Sure, send a pull request. On 01/03/2013 01:31 PM, Roberto Aguilar wrote: Thanks for clarifying, Mark. I made a quick change to the docs: https://github.com/rca/ceph/commit/37b57cdf0fdc5c03eeff3f5eb58ff4010ce581f6 Can I send you a pull request? Thanks,

[PATCH REPOST] rbd: end request on error in rbd_do_request() caller

2013-01-03 Thread Alex Elder
Only one of the three callers of rbd_do_request() provide a collection structure to aggregate status. If an error occurs in rbd_do_request(), have the caller take care of calling rbd_coll_end_req() if necessary in that one spot. Signed-off-by: Alex Elder el...@inktank.com ---

[PATCH REPOST 0/2] rbd: only get snap context for write requests

2013-01-03 Thread Alex Elder
This series stops acquiring the snap context for read requests. -Alex [PATCH REPOST 1/2] rbd: make exists flag atomic [PATCH REPOST 2/2] rbd: only get snap context for write requests -- To unsubscribe from this list: send the line unsubscribe ceph-devel in

[PATCH REPOST 1/2] rbd: make exists flag atomic

2013-01-03 Thread Alex Elder
The rbd_device-exists field can be updated asynchronously, changing from set to clear if a mapped snapshot disappears from the base image's snapshot context. Currently, value of the exists flag is only read and modified under protection of the header semaphore, but that will change with the next

[PATCH REPOST 2/2] rbd: only get snap context for write requests

2013-01-03 Thread Alex Elder
Right now we get the snapshot context for an rbd image (under protection of the header semaphore) for every request processed. There's no need to get the snap context if we're doing a read, so avoid doing so in that case. Note that we no longer need to hold the header semaphore to check the

[PATCH REPOST] rbd: separate layout init

2013-01-03 Thread Alex Elder
Pull a block of code that initializes the layout structure in an osd request into its own function so it can be reused. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 23 ++- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git

Re: [PATCH REPOST 0/4] rbd: four minor patches

2013-01-03 Thread David Zafman
I reviewed these. Reviewed-by: David Zafman david.zaf...@inktank.com David Zafman Senior Developer david.zaf...@inktank.com On Jan 3, 2013, at 11:04 AM, Alex Elder el...@inktank.com wrote: I'm re-posting my patch backlog, in chunks that may or may not match how they got posted before.

Re: [PATCH REPOST 1/2] rbd: standardize rbd_request variable names

2013-01-03 Thread Dan Mick
Reviewed-by: Dan Mick dan.m...@inktank.com On 01/03/2013 02:38 PM, Alex Elder wrote: There are two names used for items of rbd_request structure type: req and req_data. The former name is also used to represent items of pointers to struct ceph_osd_request. Change all variables that have these

Re: [PATCH REPOST 2/2] rbd: standardize ceph_osd_request variable names

2013-01-03 Thread Dan Mick
Reviewed-by: Dan Mick dan.m...@inktank.com On 01/03/2013 02:40 PM, Alex Elder wrote: There are spots where a ceph_osds_request pointer variable is given the name req. Since we're dealing with (at least) three types of requests (block layer, rbd, and osd), I find this slightly distracting.

[PATCH REPOST 0/5] rbd: drop some unneeded parameters

2013-01-03 Thread Alex Elder
This series cleans up some parameter lists, eliminating parameters that don't need to be used. -Alex [PATCH REPOST 1/5] rbd: drop oid parameters from ceph_osdc_build_request() [PATCH REPOST 2/5] rbd: drop snapid parameter from rbd_req_sync_read() [PATCH

[PATCH REPOST 1/5] rbd: drop oid parameters from ceph_osdc_build_request()

2013-01-03 Thread Alex Elder
The last two parameters to ceph_osd_build_request() describe the object id, but the values passed always come from the osd request structure whose address is also provided. Get rid of those last two parameters. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c |

[PATCH REPOST 2/5] rbd: drop snapid parameter from rbd_req_sync_read()

2013-01-03 Thread Alex Elder
There is only one caller of rbd_req_sync_read(), and it passes CEPH_NOSNAP as the snapshot id argument. Delete that parameter and just use CEPH_NOSNAP within the function. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c |6 ++ 1 file changed, 2 insertions(+), 4

[PATCH REPOST 3/5] rbd: drop flags parameter from rbd_req_sync_exec()

2013-01-03 Thread Alex Elder
All callers of rbd_req_sync_exec() pass CEPH_OSD_FLAG_READ as their flags argument. Delete that parameter and use CEPH_OSD_FLAG_READ within the function. If we find a need to support write operations we can add it back again. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c

[PATCH REPOST 4/5] rbd: kill rbd_req_sync_op() snapc and snapid parameters

2013-01-03 Thread Alex Elder
The snapc and snapid parameters to rbd_req_sync_op() always take the values NULL and CEPH_NOSNAP, respectively. So just get rid of them and use those values where needed. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 17 + 1 file changed, 5

[PATCH REPOST 5/5] rbd: don't bother setting snapid in rbd_do_request()

2013-01-03 Thread Alex Elder
For some reason, the snapid field of the osd request header is explicitly set to CEPH_NOSNAP in rbd_do_request(). Just a few lines later--with no code that would access this field in between--a call is made to ceph_calc_raw_layout() passing the snapid provided to rbd_do_request(), which encodes

[PATCH REPOST 0/2] libceph: embed r_trail struct in ceph_osd_request()

2013-01-03 Thread Alex Elder
This series simplifies some handling of osd client message handling by using an initialized ceph_pagelist structure to refer to the trail portion of a ceph_osd_request rather than using a null pointer to represent not there. -Alex [PATCH REPOST 1/2]

[PATCH REPOST 1/2] libceph: always allow trail in osd request

2013-01-03 Thread Alex Elder
An osd request structure contains an optional trail portion, which if present will contain data to be passed in the payload portion of the message containing the request. The trail field is a ceph_pagelist pointer, and if null it indicates there is no trail. A ceph_pagelist structure contains a

[PATCH REPOST 2/2] libceph: kill op_needs_trail()

2013-01-03 Thread Alex Elder
Since every osd message is now prepared to include trailing data, there's no need to check ahead of time whether any operations will make use of the trail portion of the message. We can drop the second argument to get_num_ops(), and as a result we can also get rid of op_needs_trail() which is no

Re: [PATCH REPOST] rbd: separate layout init

2013-01-03 Thread Dan Mick
...and implementing a bit of new safety about pool_id. Reviewed-by: Dan Mick dan.m...@inktank.com On 01/03/2013 02:55 PM, Alex Elder wrote: Pull a block of code that initializes the layout structure in an osd request into its own function so it can be reused. Signed-off-by: Alex Elder

Re: [PATCH REPOST 2/4] rbd: add warning messages for missing arguments

2013-01-03 Thread Dan Mick
Do you want to include in the message some kind of indication which operation/function is involved? (this is definitely better, but even better might be to add rbd add or rbd_add_parse_args to the msgs) On 01/03/2013 11:11 AM, Alex Elder wrote: Tell the user (via dmesg) what was wrong with

Re: [PATCH REPOST 1/4] rbd: define and use rbd_warn()

2013-01-03 Thread Dan Mick
Nifty. Reviewed-by: Dan Mick dan.m...@inktank.com On 01/03/2013 11:11 AM, Alex Elder wrote: Define a new function rbd_warn() that produces a boilerplate warning message, identifying in the resulting message the affected rbd device in the best way available. Use it in a few places that now use