[PATCH] mds: Clean up finished two phase commits

2012-09-19 Thread Yan, Zheng
From: "Yan, Zheng" When handling master request with slaves, the mds could crash after receiving all slaves' commit acknowledgement, but before journalling the ECommitted. Current MDS recovery code does not handle this case correctly, the request will be left in LogSegment's uncommitted_masters a

OSD-crash on 0.48.1argonout, error "void ReplicatedPG::recover_got(hobject_t, eversion_t)" not seen on list

2012-09-19 Thread Oliver Francke
Hi all, after adding a new node into our ceph-cluster yesterday, we had a crash of one OSD. I have found this kind of message in the bugtracker as being solved ( http://tracker.newdream.net/issues/2075), I will update this one for my convenience and attach the according log ( due to producti

Re: [PATCH 1/9] rbd: lay out header probe infrastructure

2012-09-19 Thread Josh Durgin
Reviewed-by: Josh Durgin On 09/07/2012 02:13 PM, Alex Elder wrote: This defines a new function rbd_dev_probe() as a top-level function for populating detailed information about an rbd device. It first checks for the existence of a format 2 rbd image id object. If it exists, the image is assume

Re: [PATCH 2/9] rbd: add code to get the size of a v2 rbd image

2012-09-19 Thread Josh Durgin
Reviewed-by: Josh Durgin On 09/07/2012 02:13 PM, Alex Elder wrote: The size of an rbd format 2 image is fetched from the server using a "get_size" method. The same method is used for getting the size of a snapshot, so structure this addition with a generic helper routine that we can get this i

Re: [PATCH 3/9] rbd: get the object prefix for a v2 rbd image

2012-09-19 Thread Josh Durgin
One minor nit below. Otherwise looks good to me. Reviewed-by: Josh Durgin On 09/07/2012 02:13 PM, Alex Elder wrote: The object prefix of an rbd format 2 image is fetched from the server using a "get_object_prefix" method. Signed-off-by: Alex Elder --- drivers/block/rbd.c | 47 +++

Re: [PATCH 4/9] rbd: get image features for a v2 image

2012-09-19 Thread Josh Durgin
On 09/07/2012 02:13 PM, Alex Elder wrote: The features values for an rbd format 2 image are fetched from the server using a "get_features" method. The same method is used for getting the features for a snapshot, so structure this addition with a generic helper routine that can get this informati

Re: [PATCH 5/9] rbd: get the snapshot context for a v2 image

2012-09-19 Thread Josh Durgin
Reviewed-by: Josh Durgin On 09/07/2012 02:13 PM, Alex Elder wrote: Fetch the snapshot context for an rbd format 2 image by calling the "get_snapcontext" method on its header object. Signed-off-by: Alex Elder --- drivers/block/rbd.c | 85 +++

Re: [PATCH 6/9] rbd: get snapshot name for a v2 image

2012-09-19 Thread Josh Durgin
A couple small things, otherwise: Reviewed-by: Josh Durgin On 09/07/2012 02:14 PM, Alex Elder wrote: Define rbd_dev_v2_snap_name() to fetch the name for a particular snapshot in a format 2 rbd image. Define rbd_dev_v2_snap_info() to to be a wrapper for getting the name, size, and features for

Re: [PATCH 7/9] rbd: update remaining header fields for v2

2012-09-19 Thread Josh Durgin
On 09/07/2012 02:15 PM, Alex Elder wrote: There are three fields that are not yet updated for format 2 rbd image headers: the version of the header object; the encryption type; and the compression type. There is no interface defined for fetching the latter two, so just initialize them explicitl

Re: [PATCH 9/9] rbd: activate v2 image support

2012-09-19 Thread Josh Durgin
On 09/19/2012 12:56 PM, Josh Durgin wrote: Until layering is implemented, this should report an error if features is not 0. Layering is feature bit 1. You can copy the userspace header from ceph.git in src/include/librbd/features.h to use the same feature macros. err, that's src/include/rbd/fe

Re: [PATCH 9/9] rbd: activate v2 image support

2012-09-19 Thread Josh Durgin
Until layering is implemented, this should report an error if features is not 0. Layering is feature bit 1. You can copy the userspace header from ceph.git in src/include/librbd/features.h to use the same feature macros. On 09/07/2012 02:15 PM, Alex Elder wrote: Now that v2 images support is fu

Re: Memory usage of ceph-mds

2012-09-19 Thread Sage Weil
On Wed, 19 Sep 2012, Tren Blackburn wrote: > Hey List; > > I'm in the process of rsyncing in about 7TB of data to Ceph across > approximately 58565475 files (okay, so I guess that's not so > approximate). It's only managed to copy a small portion of this so far > (about 35GB) and the server that c

Re: Memory usage of ceph-mds

2012-09-19 Thread Gregory Farnum
On Wed, Sep 19, 2012 at 1:48 PM, Sage Weil wrote: > On Wed, 19 Sep 2012, Tren Blackburn wrote: >> Hey List; >> >> I'm in the process of rsyncing in about 7TB of data to Ceph across >> approximately 58565475 files (okay, so I guess that's not so >> approximate). It's only managed to copy a small po

Re: [PATCH 8/9] rbd: define rbd_dev_v2_snapc_refresh()

2012-09-19 Thread Josh Durgin
On 09/07/2012 02:15 PM, Alex Elder wrote: Define a new function rbd_dev_v2_snapc_refresh() to update/refresh the snapshot context for a format version 2 rbd image. Update rbd_refresh_header() so it selects which function to use based on the image format. Rename __rbd_refresh_header() to be rbd

Re: Memory usage of ceph-mds

2012-09-19 Thread Tren Blackburn
On Wed, Sep 19, 2012 at 1:52 PM, Gregory Farnum wrote: > On Wed, Sep 19, 2012 at 1:48 PM, Sage Weil wrote: >> On Wed, 19 Sep 2012, Tren Blackburn wrote: >>> Hey List; >>> >>> I'm in the process of rsyncing in about 7TB of data to Ceph across >>> approximately 58565475 files (okay, so I guess that

Re: Memory usage of ceph-mds

2012-09-19 Thread Gregory Farnum
On Wed, Sep 19, 2012 at 2:05 PM, Tren Blackburn wrote: > Greg: It's difficult to tell you that. I'm rsyncing 2 volumes from our > filers. Each base directory on each filer mount has approximate 213 > directories, and then each directory under that has approximately > anywhere from 3000 - 5000 dir

Re: Memory usage of ceph-mds

2012-09-19 Thread Sage Weil
On Wed, 19 Sep 2012, Tren Blackburn wrote: > On Wed, Sep 19, 2012 at 1:52 PM, Gregory Farnum wrote: > > On Wed, Sep 19, 2012 at 1:48 PM, Sage Weil wrote: > >> On Wed, 19 Sep 2012, Tren Blackburn wrote: > >>> Hey List; > >>> > >>> I'm in the process of rsyncing in about 7TB of data to Ceph across

Re: Memory usage of ceph-mds

2012-09-19 Thread Tren Blackburn
On Wed, Sep 19, 2012 at 2:12 PM, Sage Weil wrote: > On Wed, 19 Sep 2012, Tren Blackburn wrote: >> On Wed, Sep 19, 2012 at 1:52 PM, Gregory Farnum wrote: >> > On Wed, Sep 19, 2012 at 1:48 PM, Sage Weil wrote: >> >> On Wed, 19 Sep 2012, Tren Blackburn wrote: >> >>> Hey List; >> >>> >> >>> I'm in t

Re: Memory usage of ceph-mds

2012-09-19 Thread Tren Blackburn
On Wed, Sep 19, 2012 at 2:12 PM, Gregory Farnum wrote: > On Wed, Sep 19, 2012 at 2:05 PM, Tren Blackburn wrote: > >> Greg: It's difficult to tell you that. I'm rsyncing 2 volumes from our >> filers. Each base directory on each filer mount has approximate 213 >> directories, and then each director

Re: Memory usage of ceph-mds

2012-09-19 Thread Sage Weil
On Wed, 19 Sep 2012, Tren Blackburn wrote: > On Wed, Sep 19, 2012 at 2:12 PM, Gregory Farnum wrote: > > On Wed, Sep 19, 2012 at 2:05 PM, Tren Blackburn > > wrote: > > > >> Greg: It's difficult to tell you that. I'm rsyncing 2 volumes from our > >> filers. Each base directory on each filer mount

Re: Memory usage of ceph-mds

2012-09-19 Thread Tren Blackburn
On Wed, Sep 19, 2012 at 2:33 PM, Sage Weil wrote: > On Wed, 19 Sep 2012, Tren Blackburn wrote: >> On Wed, Sep 19, 2012 at 2:12 PM, Gregory Farnum wrote: >> > On Wed, Sep 19, 2012 at 2:05 PM, Tren Blackburn >> > wrote: >> > >> >> Greg: It's difficult to tell you that. I'm rsyncing 2 volumes from

Re: Memory usage of ceph-mds

2012-09-19 Thread Tren Blackburn
On Wed, Sep 19, 2012 at 2:45 PM, Tren Blackburn wrote: > On Wed, Sep 19, 2012 at 2:33 PM, Sage Weil wrote: >> On Wed, 19 Sep 2012, Tren Blackburn wrote: >>> On Wed, Sep 19, 2012 at 2:12 PM, Gregory Farnum wrote: >>> > On Wed, Sep 19, 2012 at 2:05 PM, Tren Blackburn >>> > wrote: >>> > >>> >> Gr

Re: Why does mkcephfs take approximately 30 seconds per osd on ceph 0.51?

2012-09-19 Thread Josh Durgin
On 09/18/2012 04:47 PM, Tren Blackburn wrote: On Tue, Sep 18, 2012 at 4:32 PM, Josh Durgin wrote: On 09/18/2012 02:23 PM, Tren Blackburn wrote: On Tue, Sep 18, 2012 at 2:11 PM, Tren Blackburn wrote: On Tue, Sep 18, 2012 at 1:58 PM, Sage Weil wrote: On Tue, 18 Sep 2012, Tren Blackburn wr

Re: Memory usage of ceph-mds

2012-09-19 Thread Sage Weil
On Wed, 19 Sep 2012, Tren Blackburn wrote: > On Wed, Sep 19, 2012 at 2:45 PM, Tren Blackburn wrote: > > On Wed, Sep 19, 2012 at 2:33 PM, Sage Weil wrote: > >> On Wed, 19 Sep 2012, Tren Blackburn wrote: > >>> On Wed, Sep 19, 2012 at 2:12 PM, Gregory Farnum wrote: > >>> > On Wed, Sep 19, 2012 at 2

Re: Memory usage of ceph-mds

2012-09-19 Thread Tren Blackburn
On Wed, Sep 19, 2012 at 4:30 PM, Sage Weil wrote: > On Wed, 19 Sep 2012, Tren Blackburn wrote: >> On Wed, Sep 19, 2012 at 2:45 PM, Tren Blackburn wrote: >> > On Wed, Sep 19, 2012 at 2:33 PM, Sage Weil wrote: >> >> On Wed, 19 Sep 2012, Tren Blackburn wrote: >> >>> On Wed, Sep 19, 2012 at 2:12 PM,

[PATCH 02/27] userns: net: Call key_alloc with GLOBAL_ROOT_UID, GLOBAL_ROOT_GID instead of 0, 0

2012-09-19 Thread Eric W. Biederman
From: "Eric W. Biederman" In net/dns_resolver/dns_key.c and net/rxrpc/ar-key.c make them work with user namespaces enabled where key_alloc takes kuids and kgids. Pass GLOBAL_ROOT_UID and GLOBAL_ROOT_GID instead of bare 0's. Cc: Sage Weil Cc: ceph-devel@vger.kernel.org Cc: David Howells Cc: Dav