Hi, Recently I tried adding a new node (OSD) to ceph cluster using ceph-deploy tool. Since I was experimenting with tool and ended up deleting OSD nodes on new server couple of times.
Now since ceph OSDs are running on new server cluster PGs seems to be inactive (10-15%) and they are not recovering or rebalancing. Not sure what to do. I tried shutting down OSDs on new server. Status: [root@fre105 ~]# ceph -s 2019-01-03 18:56:42.867081 7fa0bf573700 -1 asok(0x7fa0b80017a0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph-guests/ceph-client.admin.4018644.140328258509136.asok': (2) No such file or directory cluster: id: adb9ad8e-f458-4124-bf58-7963a8d1391f health: HEALTH_ERR 3 pools have many more objects per pg than average 373907/12391198 objects misplaced (3.018%) 2 scrub errors 9677 PGs pending on creation Reduced data availability: 7145 pgs inactive, 6228 pgs down, 1 pg peering, 2717 pgs stale Possible data damage: 2 pgs inconsistent Degraded data redundancy: 178350/12391198 objects degraded (1.439%), 346 pgs degraded, 1297 pgs undersized 52486 slow requests are blocked > 32 sec 9287 stuck requests are blocked > 4096 sec too many PGs per OSD (2968 > max 200) services: mon: 3 daemons, quorum ceph-mon01,ceph-mon02,ceph-mon03 mgr: ceph-mon03(active), standbys: ceph-mon01, ceph-mon02 osd: 39 osds: 36 up, 36 in; 51 remapped pgs rgw: 1 daemon active data: pools: 18 pools, 54656 pgs objects: 6050k objects, 10941 GB usage: 21727 GB used, 45308 GB / 67035 GB avail pgs: 13.073% pgs not active 178350/12391198 objects degraded (1.439%) 373907/12391198 objects misplaced (3.018%) 46177 active+clean 5054 down 1173 stale+down 1084 stale+active+undersized 547 activating 201 stale+active+undersized+degraded 158 stale+activating 96 activating+degraded 46 stale+active+clean 42 activating+remapped 34 stale+activating+degraded 23 stale+activating+remapped 6 stale+activating+undersized+degraded+remapped 6 activating+undersized+degraded+remapped 2 activating+degraded+remapped 2 active+clean+inconsistent 1 stale+activating+degraded+remapped 1 stale+active+clean+remapped 1 stale+remapped 1 down+remapped 1 remapped+peering io: client: 0 B/s rd, 208 kB/s wr, 28 op/s rd, 28 op/s wr Thanks -- Arun Poonia
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com