[ceph-users] Re: Power outage recovery

Recovery from OSDs loses the mds and rgw keys they use to authenticate with cephx. You need to get those set up again by using the auth commands. I don’t have them handy but it is discussed in the mailing list archives. -Greg On Thu, Sep 15, 2022 at 3:28 PM Jorge Garcia wrote: > Yes, I tried

[ceph-users] Re: Power outage recovery

Yes, I tried restarting them and even rebooting the mds machine. No joy. If I try to start ceph-mds by hand, it returns: 2022-09-15 15:21:39.848 7fc43dbd2700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2] failed to fetch mon config

[ceph-users] Re: Power outage recovery

Having the quorum / monitors back up may change the MDS and RGW's ability to start and stay running. Have you tried just restarting the MDS / RGW daemons again? Respectfully, *Wes Dillingham* w...@wesdillingham.com LinkedIn On Thu, Sep 15, 2022 at

[ceph-users] Re: Power outage recovery

OK, I'll try to give more details as I remember them. 1. There was a power outage and then power came back up. 2. When the systems came back up, I did a "ceph -s" and it never returned. Further investigation revealed that the ceph-mon processes had not started in any of the 3 monitors. I

[ceph-users] Re: Power outage recovery

What does "ceph status" "ceph health detail" etc show, currently? Based on what you have said here my thought is you have created a new monitor quorum and as such all auth details from the old cluster are lost including any and all mgr cephx auth keys, so what does the log for the mgr say? How

[ceph-users] Re: Power outage recovery

The data only seems to be gone (if you mean what I think you mean) because the MGRs are not running and the OSDs can’t report their status. But are all MONs and OSDs up? What is the ceph status? What do the MGRs log when trying to start them? Zitat von Jorge Garcia : We have a Nautilus

[ceph-users] Re: Power outage recovery

> (particularly the "Recovery using OSDs" section). I got it so the mon > processes would start, but then the ceph-mgr process died, and would not > restart. Not sure how to recover so both ceph-mgr and ceph-mon processes > run. In the meantime, all the data is gone. Any suggestions? All the data