Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-09 Thread Paul Evans
Congrats Chris and nice save on that RBD! -- Paul On Apr 9, 2015, at 11:11 AM, Chris Kitzmiller ckitzmil...@hampshire.edu wrote: Success! Hopefully my notes from the process will help: In the event of multiple disk failures the cluster could lose PGs. Should this occur it is best to

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-09 Thread Chris Kitzmiller
Success! Hopefully my notes from the process will help: In the event of multiple disk failures the cluster could lose PGs. Should this occur it is best to attempt to restart the OSD process and have the drive marked as up+out. Marking the drive as out will cause data to flow off the drive to

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-07 Thread Chris Kitzmiller
I'm not having much luck here. Is there a possibility that the imported PGs aren't being picked up because the MONs think that they're older than the empty PGs I find on the up OSDs? I feel that I'm so close to *not* losing my RBD volume because I only have two bad PGs and I've successfully

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-06 Thread Craig Lewis
In that case, I'd set the crush weight to the disk's size in TiB, and mark the osd out: ceph osd crush reweight osd.OSDID weight ceph osd out OSDID Then your tree should look like: -9 *2.72* host ithome 30 *2.72* osd.30 up *0* An OSD can be UP

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-06 Thread Chris Kitzmiller
On Apr 6, 2015, at 1:49 PM, Craig Lewis cle...@centraldesktop.com wrote: In that case, I'd set the crush weight to the disk's size in TiB, and mark the osd out: ceph osd crush reweight osd.OSDID weight ceph osd out OSDID Then your tree should look like: -9 2.72 host

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-06 Thread Chris Kitzmiller
On Apr 3, 2015, at 12:37 AM, LOPEZ Jean-Charles jelo...@redhat.com wrote: according to your ceph osd tree capture, although the OSD reweight is set to 1, the OSD CRUSH weight is set to 0 (2nd column). You need to assign the OSD a CRUSH weight so that it can be selected by CRUSH: ceph osd

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-04 Thread Chris Kitzmiller
On Apr 3, 2015, at 12:37 AM, LOPEZ Jean-Charles jelo...@redhat.com wrote: according to your ceph osd tree capture, although the OSD reweight is set to 1, the OSD CRUSH weight is set to 0 (2nd column). You need to assign the OSD a CRUSH weight so that it can be selected by CRUSH: ceph osd

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

2015-04-02 Thread LOPEZ Jean-Charles
Hi Chris, according to your ceph osd tree capture, although the OSD reweight is set to 1, the OSD CRUSH weight is set to 0 (2nd column). You need to assign the OSD a CRUSH weight so that it can be selected by CRUSH: ceph osd crush reweight osd.30 x.y (where 1.0=1TB) Only when this is done