On Tue, Jul 11, 2017 at 12:46 PM, Ashley Merrick <ash...@amerrick.co.uk> wrote: > Hello, > > Perfect thanks that fixed my issue! > > Still seems to be a bug on the ceph pg dump unless it has been moved out of > the PG and directly into the OSD?
I am looking into this issue and have been trying to bisect it but it has proved challenging. Please feel free to open a tracker as we have discussed or I will open one once I have more information. Edward, if you feel this could be improved please also open a separate tracker for the ratio ordering issue outlining the steps involved and noting the issues encountered. > > ,Ashley > > -----Original Message----- > From: Edward R Huyer [mailto:erh...@rit.edu] > Sent: Tuesday, 11 July 2017 7:53 AM > To: Ashley Merrick <ash...@amerrick.co.uk> > Cc: ceph-users@lists.ceph.com > Subject: [ceph-users] OSD Full Ratio Luminous - Unset > > I just now ran into the same problem you did, though I managed to get it > straightened out. > > It looks to me like the "ceph osd set-{full,nearfull,backfillfull}-ratio" > commands *do* work, with two caveats. > > Caveat 1: "ceph pg dump" doesn't reflect the change for some reason. > Caveat 2: There doesn't seem to be much of a safety measure to prevent you > from entering incoherent values. In particular, if backfillfull-ratio is > smaller than nearfull-ratio, you will continue to have the ERR health > message. Backfillfull must be greater than (or equal to?) nearfull. So, for > instance, my cluster at the moment has a full ratio of .9, a backfillfull > ratio of 0.85, and a nearfull ratio of 0.8. > > So there appears to be a bug when upgrading from Jewel where the full ratios > don't carry over properly, a display bug of some kind with "ceph pg dump", > and a general issue with lack of information about full ratios being out of > order. Oh, and the set ratio commands allow incoherent values, though that > might be intentional. > > ----- > Edward Huyer > School of Interactive Games and Media > Rochester Institute of Technology > Golisano 70-2373 > 152 Lomb Memorial Drive > Rochester, NY 14623 > 585-475-6651 > mailto:erh...@rit.edu > > Obligatory Legalese: > The information transmitted, including attachments, is intended only for the > person(s) or entity to which it is addressed and may contain confidential > and/or privileged material. Any review, retransmission, dissemination or > other use of, or taking of any action in reliance upon this information by > persons or entities other than the intended recipient is prohibited. If you > received this in error, please contact the sender and destroy any copies of > this information. > > So using the commands given I checked all the mon's and a couple of OSD's > that are not backfilling due to saying they are backfill_full. > > However when running the command the value's I have been trying to set via > the set commands are correctly set and reporting via the admin socket. > > However "ceph pg dump | head" still shows 0 for them all and the PG's still > aren't back filling, so apart from ceph pg dump showing incorrect values the > admin socket shows correct however cluster is still stuck in non-backfilling > state / ERR_health due to messages. > > ,Ashley > > ________________________________ > From: Brad Hubbard <bhubbard at redhat.com> > Sent: 07 July 2017 10:31:01 > To: Ashley Merrick > Cc: ceph-users at ceph.com > Subject: Re: [ceph-users] OSD Full Ratio Luminous - Unset > > On Fri, Jul 7, 2017 at 4:49 PM, Ashley Merrick <ashley at amerrick.co.uk> > wrote: >> After looking into this further it seem's none of the : >> >> >> ceph osd set-{full,nearfull,backfillfull}-ratio >> >> >> Commands seem to be taking any effect on the cluster including the >> backfillfull ratio, this command looks to have been added/changed >> since Jewel, and a different way of setting the above. However does >> not seem to be giving the expected results. > > Hi Ashley, > > Please do open a tracker for this including commands used and resulting > output so we can investigate this fully. > > In the meantime you should be able to view and manipulate individual full > ratio values via the admin socket. > > ceph daemon osd.0 config show|grep ratio ceph daemon mon.a config show|grep > ratio ceph daemon osd.0 help|grep "config set " > >> >> >> ,Ashley >> >> ________________________________ >> From: ceph-users <ceph-users-bounces at lists.ceph.com> on behalf of >> Ashley Merrick <ashley at amerrick.co.uk> >> Sent: 06 July 2017 12:44:09 >> To: ceph-users at ceph.com >> Subject: Re: [ceph-users] OSD Full Ratio Luminous - Unset >> >> Anyone have some feedback on this? Happy to log a bug ticket if it is >> one, but want to make sure not missing something Luminous change related. >> >> ,Ashley >> >> Sent from my iPhone >> >> On 4 Jul 2017, at 3:30 PM, Ashley Merrick <ashley at amerrick.co.uk> wrote: >> >> Okie noticed their is a new command to set these. >> >> >> Tried these and still showing as 0 and error on full ratio out of >> order "ceph osd set-{full,nearfull,backfillfull}-ratio" >> >> >> ,Ashley >> >> ________________________________ >> From: ceph-users <ceph-users-bounces at lists.ceph.com> on behalf of >> Ashley Merrick <ashley at amerrick.co.uk> >> Sent: 04 July 2017 05:55:10 >> To: ceph-users at ceph.com >> Subject: [ceph-users] OSD Full Ratio Luminous - Unset >> >> >> Hello, >> >> >> On a Luminous upgraded from Jewel I am seeing the following in ceph -s : >> "Full ratio(s) out of order" >> >> >> and >> >> >> ceph pg dump | head >> dumped all >> version 44281 >> stamp 2017-07-04 05:52:08.337258 >> last_osdmap_epoch 0 >> last_pg_scan 0 >> full_ratio 0 >> nearfull_ratio 0 >> >> I have tried to inject the values however makes no effect, these where >> previously non 0 values and the issue only showed after running "ceph >> osd require-osd-release luminous" >> >> >> Thanks, >> >> Ashley >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Cheers, Brad _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
