I agree wrt making the nodes weights uniform. When mixing drive sizes, be careful that the larger ones don’t run afoul of the pg max — they will receive more pgs than the smaller ones, and if you lose a node that might be enough to send some over the max. ‘ceph OSD df’ and look at the PG counts.
This can also degrade performance since IO is not spread uniformly. Primary affinity hops can mitigate somewhat. > On Oct 22, 2019, at 8:26 PM, Konstantin Shalygin <k0...@k0ste.ru> wrote: > > On 10/22/19 7:52 PM, Thomas wrote: >> Node 1 >> 48x 1.6TB >> Node 2 >> 48x 1.6TB >> Node 3 >> 48x 1.6TB >> Node 4 >> 48x 1.6TB >> Node 5 >> 48x 7.2TB >> Node 6 >> 48x 7.2TB >> Node 7 >> 48x 7.2TB > > I suggest to balance disks in hosts, e.g. ~ 28x1.6TB + 20x7.2TB per host. > >> Why is the data distribution on the 1.6TB disks unequal? >> How can I correct this? > Balancer in upmap mode works with pools. I guess some of your 1.6TB OSD's not > serve some pools. > > > > k > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
