[ceph-users] How to reset compat weight-set changes caused by PG balancer module?
Apparently the PG balancer crush-compat mode adds some crush bucket weights. Those cause major havoc in our cluster, our PG distribution is all over the place. Seeing things like this:... 97 hdd 9.09470 1.0 9.1 TiB 6.3 TiB 6.3 TiB 32 KiB 17 GiB 2.8 TiB 69.03 1.08 28 up 98 hdd 9.09470 1.0 9.1 TiB 4.5 TiB 4.5 TiB 96 KiB 11 GiB 4.6 TiB 49.51 0.77 20 up 99 hdd 9.09470 1.0 9.1 TiB 7.0 TiB 6.9 TiB 80 KiB 18 GiB 2.1 TiB 76.47 1.20 31 up Filling rates are from 50 - 90%. Unfortunately reweighing doesn't seem to help and I suspect it's because of bucket weights which are WEIRD bucket_id -42 weight_set [ [ 7.846 11.514 9.339 9.757 10.173 8.900 9.164 6.759 ] I disabled the module already but the rebalance is broken now. Do I have to hand reset this and push a new crush map? This is a sensitive production cluster, I don't feel pretty good about that. Thanks for any ideas.. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] How to reset compat weight-set changes caused by PG balancer module?
Apparently the PG balancer crush-compat mode adds some crush bucket weights. Those cause major havoc in our cluster, our PG distribution is all over the place. Seeing things like this:... 97 hdd 9.09470 1.0 9.1 TiB 6.3 TiB 6.3 TiB 32 KiB 17 GiB 2.8 TiB 69.03 1.08 28 up 98 hdd 9.09470 1.0 9.1 TiB 4.5 TiB 4.5 TiB 96 KiB 11 GiB 4.6 TiB 49.51 0.77 20 up 99 hdd 9.09470 1.0 9.1 TiB 7.0 TiB 6.9 TiB 80 KiB 18 GiB 2.1 TiB 76.47 1.20 31 up Filling rates are from 50 - 90%. Unfortunately reweighing doesn't seem to help and I suspect it's because of bucket weights which are WEIRD bucket_id -42 weight_set [ [ 7.846 11.514 9.339 9.757 10.173 8.900 9.164 6.759 ] I disabled the module already but the rebalance is broken now. Do I have to hand reset this and push a new crush map? This is a sensitive production cluster, I don't feel pretty good about that. Thanks for any ideas.. `osd crush weight-set rm-compat` and use upmap mode instead. k ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
