I've also seen this effect a few times since Mimic (never happened in Luminous). It always resolved itself but the HEALTH_ERROR can be confusing to users.
Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Fri, Jan 25, 2019 at 8:33 AM Gregory Farnum <gfar...@redhat.com> wrote: > > This doesn’t look familiar to me. Is the cluster still doing recovery so we > can at least expect them to make progress when the “out” OSDs get removed > from the set? > On Tue, Jan 22, 2019 at 2:44 PM Wido den Hollander <w...@42on.com> wrote: >> >> Hi, >> >> I've got a couple of PGs which are stuck in backfill_toofull, but none >> of them are actually full. >> >> "up": [ >> 999, >> 1900, >> 145 >> ], >> "acting": [ >> 701, >> 1146, >> 1880 >> ], >> "backfill_targets": [ >> "145", >> "999", >> "1900" >> ], >> "acting_recovery_backfill": [ >> "145", >> "701", >> "999", >> "1146", >> "1880", >> "1900" >> ], >> >> I checked all these OSDs, but they are all <75% utilization. >> >> full_ratio 0.95 >> backfillfull_ratio 0.9 >> nearfull_ratio 0.9 >> >> So I started checking all the PGs and I've noticed that each of these >> PGs has one OSD in the 'acting_recovery_backfill' which is marked as out. >> >> In this case osd.1880 is marked as out and thus it's capacity is shown >> as zero. >> >> [ceph@ceph-mgr ~]$ ceph osd df|grep 1880 >> 1880 hdd 4.54599 0 0 B 0 B 0 B 0 0 27 >> [ceph@ceph-mgr ~]$ >> >> This is on a Mimic 13.2.4 cluster. Is this expected or is this a unknown >> side-effect of one of the OSDs being marked as out? >> >> Thanks, >> >> Wido >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
