This is currently reef but we started with it on quincy Yes disks replacement works
Am Di., 30. Sept. 2025 um 17:21 Uhr schrieb Eugen Block <[email protected]>: > That’s really surprising, did you also already have to replace drives > and they were redeployed as expected? > I must admit, it’s been some time since I last tried something similar > (maybe pacific), and when it failed I decided to do it differently. > Which version is this? > > Zitat von Stephan Hohn <[email protected]>: > > > In our deployments it works like this. > > > > It grabs what you get from "ceph orch ls" from top to bottom. > > That's why we use the numbering (osd_1... osd_2... osd_3...). > > Also db_slots work quite well. > > > > > > > > Am Di., 30. Sept. 2025 um 15:57 Uhr schrieb Eugen Block <[email protected]>: > > > >> That's not going to work as you expect: if the same host is applicable > >> to multiple osd specs, only one of the specs will be applied (I think > >> it's the last one in the list of 'ceph orch ls osd --export'). > >> > >> I would also recommend to not use device paths, those can change after > >> a reboot (although cephadm works with LVM and labels). > >> > >> The config option "db_slots" has never worked reliably, it's mentioned > >> in the docs somewhere, so it most likely won't work either as you > >> expect. > >> > >> If you can distinguish your OSDs using models, sizes or some other > >> parameter, you could use several osd specs. If you can't, you might > >> need to create the daemons manually like this: > >> > >> ceph01:~ # ceph orch daemon add osd > >> > ceph01:data_devices=/dev/vdb,/dev/vdc,/dev/vdd,db_devices=/dev/vde,block_db_size=64G > >> (I'm not sure about that last parameter, writing this from the top of my > >> head) > >> > >> But note that these OSDs will be displayed as "unmanaged" in 'ceph > >> orch ls osd' output. > >> > >> > >> Zitat von "GLE, Vivien" <[email protected]>: > >> > >> > Hi, > >> > > >> > > >> > For testing purpose we need to deploy : > >> > > >> > > >> > - 1 pool of 6 SSD OSD > >> > > >> > - 1 pool of 6 HDD OSD > >> > > >> > - 1 pool of 6 HDD OSD with 2 SSD for DB+WAL > >> > > >> > > >> > I tried to orch apply this yaml but it doesn't work as expected > >> > > >> > > >> > Osd part of the yaml -> > >> > > >> > > >> > service_type: osd > >> > service_id: osd_spec > >> > placement: > >> > hosts: > >> > - host1 > >> > - host2 > >> > - host3 > >> > - host4 > >> > - host5 > >> > - host6 > >> > data_devices: > >> > paths: > >> > - /dev/sda > >> > - /dev/sdc > >> > spec: > >> > data_devices: > >> > all: true > >> > filter_logic: AND > >> > objectstore: bluestore > >> > --- > >> > service_type: osd > >> > service_id: osd_spec_wall > >> > placement: > >> > hosts: > >> > - host1 > >> > - host2 > >> > - host3 > >> > - host4 > >> > - host5 > >> > - host6 > >> > > >> > spec: > >> > data_devices: > >> > paths: > >> > - /dev/sdf > >> > db_devices: > >> > paths: > >> > - /dev/sde > >> > limit: 2 > >> > db_slots: 3 > >> > > >> > > >> > > >> > Only 1 db on /dev/sde from host1 has been created and this OSD > >> > showed up as half full at his creation: > >> > > >> > ceph osd df | grep 25 > >> > > >> > 25 hdd 0.63669 1.00000 652 GiB 373 GiB 1.5 MiB 1 KiB 38 > >> > MiB 279 GiB 57.15 35.93 0 up > >> > > >> > ceph-volume lvm list > >> > > >> > ====== osd.25 ====== > >> > > >> > [block] > >> > > >> > /dev/ceph-23d9297a-d0e1-47be-8650-5c8ccae4fe0e/osd-block-2f009760-fc2b-46d5-984d-e8200dfd9d9d > >> > > >> > block device > >> > > >> > /dev/ceph-23d9297a-d0e1-47be-8650-5c8ccae4fe0e/osd-block-2f009760-fc2b-46d5-984d-e8200dfd9d9d > >> > block uuid vqJGf8-V5g0-S1cA-BAcN-Qm9D-9VTx-xsc8wk > >> > cephx lockbox secret > >> > cluster fsid id > >> > cluster name ceph > >> > crush device class > >> > db device > >> > > >> > /dev/ceph-9c39b87c-a39c-413f-b1ef-07881195fcb8/osd-db-feee5095-b5e7-47a0-ae87-8e5039512661 > >> > db uuid Rtj5KA-Qxjk-IFmY-3ffQ-gSAu-Snte-uku8HC > >> > encrypted 0 > >> > osd fsid 2f009760-fc2b-46d5-984d-e8200dfd9d9d > >> > osd id 25 > >> > osdspec affinity osd_spec_wall > >> > type block > >> > vdo 0 > >> > with tpm 0 > >> > devices /dev/sdf > >> > > >> > [db] > >> > > >> > /dev/ceph-9c39b87c-a39c-413f-b1ef-07881195fcb8/osd-db-feee5095-b5e7-47a0-ae87-8e5039512661 > >> > > >> > block device > >> > > >> > /dev/ceph-23d9297a-d0e1-47be-8650-5c8ccae4fe0e/osd-block-2f009760-fc2b-46d5-984d-e8200dfd9d9d > >> > block uuid vqJGf8-V5g0-S1cA-BAcN-Qm9D-9VTx-xsc8wk > >> > cephx lockbox secret > >> > cluster fsid id > >> > cluster name ceph > >> > crush device class > >> > db device > >> > > >> > /dev/ceph-9c39b87c-a39c-413f-b1ef-07881195fcb8/osd-db-feee5095-b5e7-47a0-ae87-8e5039512661 > >> > db uuid Rtj5KA-Qxjk-IFmY-3ffQ-gSAu-Snte-uku8HC > >> > encrypted 0 > >> > osd fsid 2f009760-fc2b-46d5-984d-e8200dfd9d9d > >> > osd id 25 > >> > osdspec affinity osd_spec_wall > >> > type db > >> > vdo 0 > >> > with tpm 0 > >> > devices /dev/sde > >> > > >> > > >> > > >> > The other /dev/sde device did show up as data_device instead of > >> > db_device (example here on host2) : > >> > > >> > ceph-volume lvm list > >> > > >> > ====== osd.17 ====== > >> > > >> > [block] > >> > > >> > /dev/ceph-629f98b0-5ed4-4e75-81b9-e85ca76afb15/osd-block-5d43d683-1f7f-4dc1-935e-6a79745252f9 > >> > > >> > block device > >> > > >> > /dev/ceph-629f98b0-5ed4-4e75-81b9-e85ca76afb15/osd-block-5d43d683-1f7f-4dc1-935e-6a79745252f9 > >> > block uuid HQpp1l-x7IB-kA2W-6gWO-BGlM-VN2k-vYf43R > >> > cephx lockbox secret > >> > cluster fsid id > >> > cluster name ceph > >> > crush device class > >> > encrypted 0 > >> > osd fsid 5d43d683-1f7f-4dc1-935e-6a79745252f9 > >> > osd id 17 > >> > osdspec affinity osd_spec > >> > type block > >> > vdo 0 > >> > with tpm 0 > >> > devices /dev/sde > >> > > >> > Thx for your help > >> > Vivien > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > _______________________________________________ > >> > ceph-users mailing list -- [email protected] > >> > To unsubscribe send an email to [email protected] > >> > >> > >> _______________________________________________ > >> ceph-users mailing list -- [email protected] > >> To unsubscribe send an email to [email protected] > >> > > > > _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
