Hi,
We had a disk on the cluster that was not responding properly and
causing 'slow requests'. The osd on the disk was stopped and the osd was
marked down and then out. Rebalancing succeeded but (some?) pgs from
that osd are now stuck in stale+active+clean state, which is not being
resolved (see below for query results).
My question: is it better to mark this osd as "lost" (i.e. 'ceph osd
lost 14') or to remove the osd as detailed here:
https://www.sebastien-han.fr/blog/2015/12/11/ceph-properly-remove-an-osd/
Thanks,
David
/
$ ceph health detail
HEALTH_ERR 17 pgs are stuck inactive for more than 300 seconds; 17 pgs
stale; 17 pgs stuck stale
pg 7.f3 is stuck stale for 6138.330316, current state
stale+active+clean, last acting [14]
pg 7.bd is stuck stale for 6138.330365, current state
stale+active+clean, last acting [14]
pg 7.b6 is stuck stale for 6138.330374, current state
stale+active+clean, last acting [14]
pg 7.c5 is stuck stale for 6138.330363, current state
stale+active+clean, last acting [14]
pg 7.ac is stuck stale for 6138.330385, current state
stale+active+clean, last acting [14]
pg 7.5b is stuck stale for 6138.330678, current state
stale+active+clean, last acting [14]
pg 7.1b4 is stuck stale for 6138.330409, current state
stale+active+clean, last acting [14]
pg 7.182 is stuck stale for 6138.330445, current state
stale+active+clean, last acting [14]
pg 7.1f8 is stuck stale for 6138.330720, current state
stale+active+clean, last acting [14]
pg 7.53 is stuck stale for 6138.330697, current state
stale+active+clean, last acting [14]
pg 7.1d2 is stuck stale for 6138.330663, current state
stale+active+clean, last acting [14]
pg 7.70 is stuck stale for 6138.330742, current state
stale+active+clean, last acting [14]
pg 7.14f is stuck stale for 6138.330585, current state
stale+active+clean, last acting [14]
pg 7.23 is stuck stale for 6138.330610, current state
stale+active+clean, last acting [14]
pg 7.153 is stuck stale for 6138.330600, current state
stale+active+clean, last acting [14]
pg 7.cc is stuck stale for 6138.330409, current state
stale+active+clean, last acting [14]
pg 7.16b is stuck stale for 6138.330509, current state
stale+active+clean, last acting [14]
$ ceph pg dump_stuck stale//
//ok//
//pg_stat state up up_primary acting acting_primary//
//7.f3 stale+active+clean [14] 14 [14] 14//
//7.bd stale+active+clean [14] 14 [14] 14//
//7.b6 stale+active+clean [14] 14 [14] 14//
//7.c5 stale+active+clean [14] 14 [14] 14//
//7.ac stale+active+clean [14] 14 [14] 14//
//7.5b stale+active+clean [14] 14 [14] 14//
//7.1b4 stale+active+clean [14] 14 [14] 14//
//7.182 stale+active+clean [14] 14 [14] 14//
//7.1f8 stale+active+clean [14] 14 [14] 14//
//7.53 stale+active+clean [14] 14 [14] 14//
//7.1d2 stale+active+clean [14] 14 [14] 14//
//7.70 stale+active+clean [14] 14 [14] 14//
//7.14f stale+active+clean [14] 14 [14] 14//
//7.23 stale+active+clean [14] 14 [14] 14//
//7.153 stale+active+clean [14] 14 [14] 14//
//7.cc stale+active+clean [14] 14 [14] 14//
//7.16b stale+active+clean [14] 14 [14] 14/
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
