Re: [ceph-users] Placement groups on a 216 OSD cluster with multiple pools

2013-11-15 Thread Andrey Korolyov
Of course, but it means that in case of failure you can no longer trust your data consistency and should recheck it against separately stored checksums or so. I`m leaving aside such fact that Ceph will not probably recover pool properly with replication number lower than 2 in many cases. So

Re: [ceph-users] HDD bad sector, pg inconsistent, no object remapping

2013-11-15 Thread Mihály Árva-Tóth
Hello, I think you misunderstood me. I known how can I replace bad HDD, thanks. My problem is the following: Object replica number is 3. Objects that in 11.15d PG which store data on osd.0 bad sectors place inter alia. Ceph should know objects are in 11.15d on osd.0 is bad (because deep-scrub

Re: [ceph-users] Ceph performance

2013-11-15 Thread Dnsbed Ops
we didn't have. On 2013-11-15 18:16, James Pearce wrote: Do you have journals on separate disks (SSD, preferably)? On 2013-11-15 10:14, Dnsbed Ops wrote: Hello, We have the plan to run ceph as block storage for openstack, but from test we found the IOPS is slow. Our apps primarily use the

[ceph-users] integration of radosgw and keystone

2013-11-15 Thread Gao, Wei M
Hi, all I am a beginner of ceph and tried to integrate radosgw and keystone according to the guide here(http://ceph.com/docs/master/radosgw/config/#integrating-with-openstack- keystone). My ceph version is v0.67.4. I was able to run: swift -V 1.0 -A http://10.239.149.9:8000/auth -U test:swift

[ceph-users] ceph-deploy disk zap fails but succeeds on retry

2013-11-15 Thread Gruher, Joseph R
Using ceph-deploy 1.3.2 with ceph 0.72.1. Ceph-deploy disk zap will fail and exit with error, but then on retry will succeed. This is repeatable as I go through each of the OSD disks in my cluster. See output below. I am guessing the first attempt to run changes something about the initial

[ceph-users] v0.72.1 released

2013-11-15 Thread Samuel Just
This point release addresses issue #6761 (http://tracker.ceph.com/issues/6761). Upgrades to v0.72 can cause reads to begin returning ENFILE (Too many open files in system). Changes: * osd: fix upgrade issue with object_info_t encodings * ceph_filestore_tool: add tool to repair osd stores

[ceph-users] Today I’ve encountered multiple OSD down and multiple OSD won’t start and OSD disk access “Input/Output” error”

2013-11-15 Thread Aquino, BenX O
Procedure when an OSD is down or Error encountered during Ceph status checks : Ceph version 0.67.4 1).Is the Cluster just started and has not complete starting OSD’s. 2).Ensure continues Hard Access to the Ceph Node: -either via HW serial console server and serial console redirect. -by

Re: [ceph-users] Today I’ve encountered multiple OSD down and multiple OSD won’t start and OSD disk access “Input/Output” error”

2013-11-15 Thread Aquino, BenX O
Thanks Kyle, --I'll look into and try out udev and upstart. -- yes on set noout, definitely a good idea, until for sure that osd is gone for good. If osd disk is totally gone, Then down-n'-out. Remove from crushmap/Update crushmap. Verify crushmap Then used ceph-deploy to add a replacement

Re: [ceph-users] HDD bad sector, pg inconsistent, no object remapping

2013-11-15 Thread David Zafman
Replication does not occur until the OSD is “out.” This creates a new mapping in the cluster of where the PGs should be and thus data begins to move and/or create sufficient copies. This scheme lets you control how and when you want the replication to occur. If you have plenty of space and