[ceph-users] ceph-radosgw: Initialization timeout, failed to initialize
We have a Red Hat installation of Luminuous (full packages version: 12.2.8-128.1). We're experiencing an issue where the ceph-radosgw service will timeout during initialization and cycle through attempts every five minutes until it seems to just give up. Every other ceph service starts successfully. I tried looking at the health of the cluster, but anytime I run a command, whether ceph or radosgw-admin just to see a list of users, it seems to time out as well. I've used strace when attempting to start radosgw directly and was presented with a missing keyring error. I would be inclined to think that might be the problem, but wouldn't that also impact all of the other services? I haven't been able to find anything in the logs that would lead me down any paths. Everything I've looked at (journalctl, /var/log/messages, /var/log/ceph/ceph-rgw-server.log) all just say the same thing: the service attempted to start, it failed to initialize, entered a failed state, service stopped. This repeats. I only get pulled in to look at Ceph every now and then so I don't have enough knowledge to know how the various components interact or if anything external is having an impact. Is there anywhere that might be holding a bit of information? Thank you, Mathew Sent with [ProtonMail](https://protonmail.com) Secure Email. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: ceph-radosgw: Initialization timeout, failed to initialize
I increased the debug level to 20. There isn't anything additional being written: 2021-02-23 16:26:38.736642 7£2c45£3700 -1 Initialization timeout, failed to initialize 2021-02-23 16:26:38.931400 7f4d7bf4a000 0 deferred set uid:gid to 167:167 (ceph:ceph) 2021-02-23 16:26:38.931707 7f4d7bf4a000 0 ceph version 12.2.8-128.1. TEST. bz1742993.el7cp (87ba41fa7b3bcb79d916fb0ca41a9dd90eb877f8) luminous (stable), process radosgw, pid 7169 2021-02-23 16:31:38.931973 7f4d6b8e5700 -1 Initialization timeout, failed to initialize 2021-02-23 16:31:39.191898 7fa2fae31000 deferred set uid:gid to 167:167 (ceph:ceph) 2021-02-23 16:31:39.192119 7fa2fae31000 0 ceph version 12.2.8-128.1. TEST.bz1742993.el7cp (87ba41fa7b3bcb79d916fb0ca41a9dd90eb877f8) luminous (stable), process radosgw, pid 7204 Thank you, Mathew Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, February 23, 2021 10:57 AM, Janne Johansson wrote: > Den tis 23 feb. 2021 kl 16:53 skrev Mathew Snyder > mathew.sny...@protonmail.com: > > > We have a Red Hat installation of Luminuous (full packages version: > > 12.2.8-128.1). We're experiencing an issue where the ceph-radosgw service > > will timeout during initialization and cycle through attempts every five > > minutes until it seems to just give up. Every other ceph service starts > > successfully. > > I tried looking at the health of the cluster, but anytime I run a command, > > whether ceph or radosgw-admin just to see a list of users, it seems to time > > out as well. > > I've used strace when attempting to start radosgw directly and was > > presented with a missing keyring error. I would be inclined to think that > > might be the problem, but wouldn't that also impact all of the other > > services? > > No, a missing rgw key would stop only it, and the radosgw-admin > command (if run on a box without a global admin key) > > > I haven't been able to find anything in the logs that would lead me down > > any paths. Everything I've looked at (journalctl, /var/log/messages, > > /var/log/ceph/ceph-rgw-server.log) all just say the same thing: the service > > attempted to start, it failed to initialize, entered a failed state, > > service stopped. This repeats. > > See if you can bump the debug log level of the radosgw when starting it. > > https://access.redhat.com/solutions/2085183 > and > https://docs.ceph.com/en/latest/rados/troubleshooting/log-and-debug/#rados-gateway > > --- > > May the most significant bit of your life be positive. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] radosgw-admin error: "could not fetch user info: no user info saved"
I'm running into a problem that I've found around the Internet, but for which
I'm unable to find a solution:
$ sudo radosgw-admin user info
could not fetch user info: no user info saved
I've seen others report this issue, but the troubleshooting has either failed
to pinpoint the issue, or has gone from "I have a problem" to "I solved it"
with no information as to how it was solved. Unfortunately, due to the nature
of our network, I'm unable to provide debug information en masse. I can try to
provide individual lines of output if someone knows what I should look for.
I am able to run the user creation command:
$ sudo radosgw-admin user create --uid="testuser2"
--display-name="testuser2"
{
"user_id": "testuser2",
"display_name": "testuser2",
"email": "",
"suspended": 0,
"max_buckets": 1000,
...
}
After doing this, I'm still unable to see the user information instead getting
the same "no user info saved" output. That said, I know the user is added:
$ sudo radosgw-admin metadata list user
[
"devtest1",
"testuser",
"testuser2",
..
]
I'm pretty new to this system and Ceph in general. I looked at
/etc/ceph/ceph.conf but I don't see anything that would have anything to do
with users. I also don't have an account on the Ansible server that controls
the configurations so I can't see if anything is defined there.
I know this isn't particularly detailed in its description of the issue, but I
can't find any other useful help via search.
Thank you,
Mathew
Sent with [ProtonMail](https://protonmail.com) Secure Email.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: radosgw-admin error: "could not fetch user info: no user info saved"
As simple as that, eh? Cripes. Thank you, Mathew Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On Thursday, April 16, 2020 4:40 AM, Janne Johansson wrote: > Den ons 15 apr. 2020 kl 21:01 skrev Mathew Snyder > : > >> I'm running into a problem that I've found around the Internet, but for >> which I'm unable to find a solution: >> $ sudo radosgw-admin user info >> could not fetch user info: no user info saved > > radosgw-admin user info --uid="name of user goes here" > > -- > May the most significant bit of your life be positive. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: radosgw-admin error: "could not fetch user info: no user info saved"
Was able to do what I needed to do. Thank you, Mathew Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On Thursday, April 16, 2020 4:40 AM, Janne Johansson wrote: > Den ons 15 apr. 2020 kl 21:01 skrev Mathew Snyder > : > >> I'm running into a problem that I've found around the Internet, but for >> which I'm unable to find a solution: >> $ sudo radosgw-admin user info >> could not fetch user info: no user info saved > > radosgw-admin user info --uid="name of user goes here" > > -- > May the most significant bit of your life be positive. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
