Hello, After a major network outage our ceph cluster ended up with an inactive PG:
# ceph health detail HEALTH_WARN 1 pgs incomplete; 1 pgs stuck inactive; 1 pgs stuck unclean; 1 requests are blocked > 32 sec; 1 osds have slow requests pg 3.367 is stuck inactive for 912263.766607, current state incomplete, last acting [28,35,2] pg 3.367 is stuck unclean for 912263.766688, current state incomplete, last acting [28,35,2] pg 3.367 is incomplete, acting [28,35,2] 1 ops are blocked > 268435 sec 1 ops are blocked > 268435 sec on osd.28 1 osds have slow requests # ceph -s cluster 6713d1b8-83da-11e6-aa79-525400d98c5a health HEALTH_WARN 1 pgs incomplete 1 pgs stuck inactive 1 pgs stuck unclean 1 requests are blocked > 32 sec monmap e3: 3 mons at {tv-dl360-1=10.12.193.73:6789/0,tv-dl360-2=10.12.193.74:6789/0,tv-dl360-3=10.12.193.75:6789/0} election epoch 72, quorum 0,1,2 tv-dl360-1,tv-dl360-2,tv-dl360-3 osdmap e60609: 72 osds: 72 up, 72 in pgmap v3670252: 4864 pgs, 11 pools, 134 GB data, 23778 objects 490 GB used, 130 TB / 130 TB avail 4863 active+clean 1 incomplete client io 0 B/s rd, 38465 B/s wr, 2 op/s ceph pg repair doesn't change anything. What should I try to recover it? Attached is the result of ceph pg query on the problem PG. Thank you, Laszlo
pg_3.367_query.gz
Description: application/gzip
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com