[ceph-users] Re: mon stuck in probing

some logs here, 2024-03-13T11:13:34.083+0800 7f6984a95640 4 mon.memb4@3(probing) e6 probe_timeout 0x5650c19d6100 2024-03-13T11:13:34.083+0800 7f6984a95640 10 mon.memb4@3(probing) e6 bootstrap 2024-03-13T11:13:34.083+0800 7f6984a95640 10 mon.memb4@3(probing) e6 sync_reset_requester 2024-03-13T11:13

[ceph-users] Re: CephFS space usage

Igor, Etienne, Bogdan, The system is a four node cluster. Each node has 12 3.8TB SSDs, and each SSD is an OSD. I have not defined any separate DB / WAL devices - this cluster is mostly at cephadm defaults. Everything is currently configured to have x3 replicas. The system also does various

[ceph-users] Re: 18.2.2 dashboard really messed up.

Thanks!  Oddly, all the dashboard checks you suggest appear normal, yet the result remains broken. Before I used your instruction about the dashboard, I have this result: root@noc3:~# ceph dashboard get-prometheus-api-host http://noc3.1.quietfountain.com:9095 root@noc3:~# netstat -6nlp | grep 9

[ceph-users] Re: CephFS space usage

Hi Thorn, could you please share the output of "ceph df detail" command representing the problem? And please give an overview of your OSD layout - amount of OSDs, shared or dedicated DB/WAL, main and DB volume sizes. Thanks, Igor On 3/13/2024 5:58 AM, Thorne Lawler wrote: Hi everyone!

[ceph-users] Re: bluestore_min_alloc_size and bluefs_shared_alloc_size

Hi Joel, generally speaking you need OSD redeployment to apply 64K to 4K min_alloc_size downgrade for block device only. Other improvements (including supporting 4K units for BlueFS) are applied to existing OSDs automatically when relevant Ceph release is installed. So yes - if you have octo

[ceph-users] Re: ceph osd different size to create a cluster for Openstack : asking for advice

NVMe (hope those are enterprise not client) drives aren't likely to suffer the same bottlenecks as HDDs or even SATA SSDs. And a 2:1 size ratio isn't the largest I've seen. So I would just use all 108 OSDs as a single device class and spread the pools across all of them. That way you won't ru

[ceph-users] ceph osd different size to create a cluster for Openstack : asking for advice

Hi, I need some guidance from you folks... I am going to deploy a ceph cluster in HCI mode for an openstack platform. My hardware will be : - 03 control nodes : - 27 osd nodes : each node has 03x3.8To nvme + 01x1.9To nvme disks (those disks will all be used as OSDs) In my Openstack I will be cr

[ceph-users] ceph osd crush reweight rounding issue

Hi, After some tests with OSD crush weight I wanted to restore the original weight of the OSD. But that proves to be difficult. See the following example: Situation before OSD crush reweight has taken place ceph osd tree ID CLASS WEIGHT TYPE NAMESTATUS REWEIGHT PRI-AFF -11

[ceph-users] mon stuck in probing

Hello, The problem is a mon stucked in probing state. The env is ceph 18.2.1 on ubuntu22.04 with rdma, 5 mons. One mon memb4 is out of quorum. The debug log is attached. Thanks. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send

[ceph-users] Re: CephFS space usage

Hi, Not sure if it was mentioned but also you could check the following: 1. Snapshots Snapshots can consume a significant amount of space without being immediately obvious. They preserve the state of the filesystem at various points in time. List Snapshots: Use the "*ceph fs subvolume snapshot ls

[ceph-users] Re: CephFS space usage

Hi, Check your replication/EC configuration. How do you get your different sizes/usages? Étienne From: Thorne Lawler Sent: Wednesday, 13 March 2024 03:58 To: ceph-users@ceph.io Subject: [ceph-users] CephFS space usage [Some people who received this message do