[ceph-users] Re: Reconstructing an OSD server when the boot OS is corrupted

eph ceph 3 Feb 7 12:12 require_osd_release > > -rw--- 1 ceph ceph 10 Feb 7 12:12 type > > -rw--- 1 ceph ceph 3 Feb 7 12:12 whoami > > [root@rook-ceph-osd-36-6876cdb479-4764r ceph-36]# > > > -----

[ceph-users] Reconstructing an OSD server when the boot OS is corrupted

Dear Ceph Community We have 5 OSD servers running Ceph v15.2.17. The host operating system is Ubuntu 20.04. One of the servers has suffered corruption to its boot operating system. Using a system rescue disk it is possible to mount the root filesystem but it is not possible to boot the operating

[ceph-users] Re: MDS crash on FAILED ceph_assert(cur->is_auth())

Hi Emmaneul It was a while ago, but as I recall I evicted all clients and that allowed me to restart the MDS servers. There was something clearly "broken" in how at least one of the clients was interacting with the system. Peter On Thu, 4 May 2023 at 07:18, Emmanuel Jaep wrote: > Hi, > > did

[ceph-users] Re: Corrupt bluestore after sudden reboot (17.2.5)

I am trying to do this, but the log file is 26 GB and growing. Is there perhaps a subset of the logs that would be useful? Peter On Mon, 16 Jan 2023 at 18:42, wrote: > Hi Peter, > > Could you add debug_bluestore = 20 to your ceph.conf and restart the OSD, > then send the log after it crashes?

[ceph-users] Re: OSD logs missing from Centralised Logging

the daemons solved the problem. Peter On Thu, 9 Feb 2023 at 16:27, Tarrago, Eli (RIS-BCT) < eli.tarr...@lexisnexisrisk.com> wrote: > Please include your promtai; logs, loki logs, promtail configuration, and > your loki configuration. > > > > *From: *Peter van Heus

[ceph-users] OSD logs missing from Centralised Logging

Hi there I am running Ceph version 17.2.5 and have deployed centralised logging as per this guide: https://ceph.io/en/news/blog/2022/centralized_logging/ The logs from the OSDs are not, however, showing up in the Grafana dashboard, as per this screenshot: [image: image.png] The Promtail

[ceph-users] Corrupt bluestore after sudden reboot (17.2.5)

Due to the ongoing South African energy crisis our datacenter experienced sudden power loss. We are running ceph 17.2.5 deployed with cephadm. Two of our OSDs did not start correctly, with the error: # ceph-bluestore-tool fsck --path

[ceph-users] Re: BlueFS spillover warning gone after upgrade to Quincy

t; upgrade so the spillover would have been corrected (temporarily)? Do > you know how much spillover you had before? And how big was the db > when you had the warnings? > > Regards, > Eugen > > Zitat von Peter van Heusden : > > > Hello everyone > > > > I have a

[ceph-users] BlueFS spillover warning gone after upgrade to Quincy

Hello everyone I have a Ceph installation where some of the OSDs were misconfigured to use 1GB SSD partitions for rocksdb. This caused a spillover ("BlueFS *spillover* detected"). I recently upgraded to quincy using cephadm (17.2.5) the spillover warning vanished. This is despite

[ceph-users] Re: MDS crash on FAILED ceph_assert(cur->is_auth())

a ticket. Peter On Fri, 6 Aug 2021 at 10:00, Yann Dupont wrote: > > Le 28/06/2021 à 10:52, Peter van Heusden a écrit : > > I am running Ceph 15.2.13 on CentOS 7.9.2009 and recently my MDS servers > > have started failing with the error message > > > > In function 'v

[ceph-users] Re: MDS crash on FAILED ceph_assert(cur->is_auth())

Yes it keeps crashing in a loop. I ran again with debug set to 20 and the last 100,000 lines of that log are here: https://gist.github.com/pvanheus/33eb22b179a9cbd68a460984de8ef24a On Mon, 28 Jun 2021 at 15:29, Stefan Kooman wrote: > On 6/28/21 10:52 AM, Peter van Heusden wrote: >

[ceph-users] MDS crash on FAILED ceph_assert(cur->is_auth())

I am running Ceph 15.2.13 on CentOS 7.9.2009 and recently my MDS servers have started failing with the error message In function 'void Server::handle_client_open(MDRequestRef&)' thread 7f0ca9908700 time 2021-06-28T09:21:11.484768+0200