[ceph-users] Re: Forcibly move PGs from full to empty OSD

On 3/4/20 11:15 AM, Thomas Schneider wrote: > Hi, > > Ceph balancer is not working correctly; there's an open bug > report, too. > > Until this issue is not solved, I need a workaround because I get more > and more warnings about "nearfull osd(s)". > >

[ceph-users] Re: Forcibly move PGs from full to empty OSD

Hi Thomas To get the usage: ceph osd df | sort -nk8 #VAR is the ratio to avg util #WEIGHT is CRUSHMAP weight; typically the Disk capacity in TiB #REWEIGHT is temporary (until osd restart or ceph osd set noout) WEIGHT correction for manual rebalance You can use for temporary rew

[ceph-users] Re: Forcibly move PGs from full to empty OSD

Hi, I already use CRUSHMAP weight to manually control the OSD utilization. However this results in a situation where 5-10% of my 336 OSDs have a weight < 1.0, and this would impact/hinder ceph balancer to work. This means I would need to modify any OSD with weight < 1.0 first before ceph b

[ceph-users] Re: Forcibly move PGs from full to empty OSD

Hi Wido, can you please share some detailed instructions how to do this? And what do you mean with "respect your failure domain"? THX Am 04.03.2020 um 11:27 schrieb Wido den Hollander: > > On 3/4/20 11:15 AM, Thomas Schneider wrote: >> Hi, >> >> Ceph balancer is not working correctly; there's an

[ceph-users] Re: Forcibly move PGs from full to empty OSD

Hi Thomas, The two commands you're looking for are: ceph osd pg-upmap-items $pg $source_osd $dest_osd and to remove them ceph osd rm-pg-upmap-items $pg You need to pair this with finding which pg's are on your full osd's. I use ceph pg dump and grep for the pool number and osd. With respect to w

[ceph-users] Re: Forcibly move PGs from full to empty OSD

Hi Wido, can you please share some detailed instructions how to do this? And what do you mean with "respect your failure domain"? THX Am 04.03.2020 um 11:27 schrieb Wido den Hollander: > On 3/4/20 11:15 AM, Thomas Schneider wrote: >> Hi, >> >> Ceph balancer is not working correctly; there's an o

[ceph-users] Re: Forcibly move PGs from full to empty OSD

He means that if eg. you enforce 1 copy of a PG per rack, that any upmaps you enter don’t result in 2 or 3 in the same rack. If your CRUSH poilicy is one copy per *host* the danger is even higher that you could have data become unavailable or even lost in case of a failure. > On Mar 16, 2020,

[ceph-users] Re: Forcibly move PGs from full to empty OSD

Hi, thanks for this clarification. I'm running a 7-node-cluster and this risk should be managable. Am 16.03.2020 um 16:57 schrieb Anthony D'Atri: > He means that if eg. you enforce 1 copy of a PG per rack, that any upmaps you > enter don’t result in 2 or 3 in the same rack. If your CRUSH poil