solved it: the warning is of course generated by ceph-mgr and not ceph-mon.
So for my problem that means: should have injected the option in ceph-mgr. That's why it obviously worked when setting it on the pool... The solution for you is to simply put the option under global and restart ceph-mgr (or use daemon config set; it doesn't support changing config via ceph tell for some reason) Paul On Mon, Dec 9, 2019 at 8:32 PM Paul Emmerich <paul.emmer...@croit.io> wrote: > > > On Mon, Dec 9, 2019 at 5:17 PM Robert LeBlanc <rob...@leblancnet.us> > wrote: > >> I've increased the deep_scrub interval on the OSDs on our Nautilus >> cluster with the following added to the [osd] section: >> > > should have read the beginning of your email; you'll need to set the > option on the mons as well because they generate the warning. So your > problem might be completely different from what I'm seeing here > > > > Paul > > >> >> osd_deep_scrub_interval = 2600000 >> >> And I started seeing >> >> 1518 pgs not deep-scrubbed in time >> >> in ceph -s. So I added >> >> mon_warn_pg_not_deep_scrubbed_ratio = 1 >> >> since the default would start warning with a whole week left to scrub. >> But the messages persist. The cluster has been running for a month with >> these settings. Here is an example of the output. As you can see, some of >> these are not even two weeks old, no where close to the 75% of 4 weeks. >> >> pg 6.1f49 not deep-scrubbed since 2019-11-09 23:04:55.370373 >> pg 6.1f47 not deep-scrubbed since 2019-11-18 16:10:52.561204 >> pg 6.1f44 not deep-scrubbed since 2019-11-18 15:48:16.825569 >> pg 6.1f36 not deep-scrubbed since 2019-11-20 05:39:00.309340 >> pg 6.1f31 not deep-scrubbed since 2019-11-27 02:48:45.347680 >> pg 6.1f30 not deep-scrubbed since 2019-11-11 21:34:15.795622 >> pg 6.1f2d not deep-scrubbed since 2019-11-24 11:37:39.502829 >> pg 6.1f27 not deep-scrubbed since 2019-11-25 07:38:58.689315 >> pg 6.1f25 not deep-scrubbed since 2019-11-20 00:13:43.048569 >> pg 6.1f1a not deep-scrubbed since 2019-11-09 15:08:43.516666 >> pg 6.1f19 not deep-scrubbed since 2019-11-25 10:24:47.884332 >> 1468 more pgs... >> Mon Dec 9 08:12:01 PST 2019 >> >> There is very little data on the cluster, so it's not a problem of >> deep-scrubs taking too long: >> >> $ ceph df >> RAW STORAGE: >> CLASS SIZE AVAIL USED RAW USED %RAW USED >> hdd 6.3 PiB 6.1 PiB 153 TiB 154 TiB 2.39 >> nvme 5.8 TiB 5.6 TiB 138 GiB 197 GiB 3.33 >> TOTAL 6.3 PiB 6.2 PiB 154 TiB 154 TiB 2.39 >> >> POOLS: >> POOL ID STORED OBJECTS USED >> %USED MAX AVAIL >> .rgw.root 1 3.0 KiB 7 3.0 KiB >> 0 1.8 PiB >> default.rgw.control 2 0 B 8 0 B >> 0 1.8 PiB >> default.rgw.meta 3 7.4 KiB 24 7.4 KiB >> 0 1.8 PiB >> default.rgw.log 4 11 GiB 341 11 GiB >> 0 1.8 PiB >> default.rgw.buckets.data 6 100 TiB 41.84M 100 TiB >> 1.82 4.2 PiB >> default.rgw.buckets.index 7 33 GiB 574 33 GiB >> 0 1.8 PiB >> default.rgw.buckets.non-ec 8 8.1 MiB 22 8.1 MiB >> 0 1.8 PiB >> >> Please help me figure out what I'm doing wrong with these settings. >> >> Thanks, >> Robert LeBlanc >> ---------------- >> Robert LeBlanc >> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
