Hi guys I have installed Proxmox 6 and activate 3 servers with PVE 6 and Ceph. In this 3 server, I have 4 HDD: 3 SAS 500GB 1 SAS 2 TB However, we need to remove this 3 500GB of eache server... So I do out and stop and I am waiting for rebalance, but is took too long... Get this message: Reduced data availability: 2 pgs inactive, 2 pgs down pg 1.3a is down, acting [11,9,10] pg 1.23a is down, acting [11,9,10] (This 11,9,10 it's the 2 TB SAS HDD) And too many PGs per OSD (571 > max 250) I already tried decrease the number of PG to 256 ceph osd pool set VMS pg_num 256 but it seem no effect att all: ceph osd pool get VMS pg_num pg_num: 571
Now, the sitution is that: ceph -s cluster: id: 93c55c6b-ce64-4e1a-92bc-0bc529d695f2 health: HEALTH_WARN Reduced data availability: 2 pgs inactive, 2 pgs down Degraded data redundancy: 6913/836472 objects degraded (0.826%), 18 pgs degraded, 19 pgs undersized too many PGs per OSD (571 > max 250) services: mon: 5 daemons, quorum pve3,pve4,pve5,pve7,pve6 (age 51m) mgr: pve3(active, since 39h), standbys: pve5, pve7, pve6, pve4 osd: 12 osds: 3 up (since 16m), 3 in (since 16m); 19 remapped pgs data: pools: 1 pools, 571 pgs objects: 278.82k objects, 1.1 TiB usage: 2.9 TiB used, 2.5 TiB / 5.5 TiB avail pgs: 0.350% pgs not active 6913/836472 objects degraded (0.826%) 550 active+clean 17 active+undersized+degraded+remapped+backfill_wait 2 down 1 active+undersized+degraded+remapped+backfilling 1 active+undersized+remapped+backfill_wait io: client: 15 KiB/s rd, 1.0 MiB/s wr, 3 op/s rd, 102 op/s wr recovery: 15 MiB/s, 3 objects/s progress: Rebalancing after osd.2 marked out [=============================.] Rebalancing after osd.7 marked out [============================..] Rebalancing after osd.6 marked out [==========....................] Do I need to do something or just leave Ceph do this work?? Thanks a lot! Cheers _______________________________________________ pve-user mailing list pve-user@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user