Hi Dan, Thanks for your (very) prompt response.
pg_num 4096 pgp_num 2108 pgp_num_target 2248 Also I see this: #ceph balancer eval current cluster score 0.068634 (lower is better) #ceph balancer status { "last_optimize_duration": "0:00:00.025029", "plans": [], "mode": "upmap", "active": true, "optimize_result": "Too many objects (0.010762 > 0.010000) are misplaced; try again later", "last_optimize_started": "Tue Feb 15 09:05:32 2022" } Seems it is indeed limiting the data movement by the set 1% So it is safe to assume I can put the number to 4096 and the total amount of misplaced PG's keeps around 1%. Met vriendelijke groet, Kind Regards, Maarten van Ingen Specialist |SURF |maarten.vanin...@surf.nl <mailto:voornaam.achtern...@surf.nl>| T +31 30 88 787 3000 |M +31 6 19 03 90 19| SURF <http://www.surf.nl/> is the collaborative organisation for ICT in Dutch education and research Op 15-02-2022 09:01 heeft Dan van der Ster <daniel.vanders...@cern.ch> geschreven: Hi Maarten, With `ceph osd pool ls detail` does it have pgp_num_target set to 2248? If so, yes it's moving gradually to that number. Cheers, Dan > On 02/15/2022 8:55 AM Maarten van Ingen <maarten.vanin...@surf.nl> wrote: > > > Hi, > > After enabling the balancer (and set to upmap) on our environment it’s time to get the pgp_num on one of the pools on par with the pg_num. > This pool has pg_num set to 4096 and pgp_num to 2048 (by our mistake). > I just set the pgp_num to 2248 to keep data movement in check. > > Oddly enough I see it’s only increased to 2108, also it’s odd we now get this health warning: 1 pools have pg_num > pgp_num, which we haven’t seen before… > > > # ceph -s > cluster: > id: <id> > health: HEALTH_WARN > 1 pools have pg_num > pgp_num > > services: > mon: 5 daemons, quorum mon01,mon02,mon03,mon05,mon04 (age 3d) > mgr: mon01(active, since 3w), standbys: mon05, mon04, mon03, mon02 > mds: cephfs:1 {0=mon04=up:active} 4 up:standby > osd: 1278 osds: 1278 up (since 68m), 1278 in (since 22h); 74 remapped pgs > > data: > pools: 28 pools, 13824 pgs > objects: 441.41M objects, 1.5 PiB > usage: 4.5 PiB used, 6.9 PiB / 11 PiB avail > pgs: 15652608/1324221126 objects misplaced (1.182%) > 13693 active+clean > 74 active+remapped+backfilling > 56 active+clean+scrubbing+deep > 1 active+clean+scrubbing > > io: > client: 187 MiB/s rd, 2.2 GiB/s wr, 11.11k op/s rd, 5.63k op/s wr > recovery: 1.8 GiB/s, 533 objects/s > > > ceph osd pool get <pool> pgp_num > pgp_num: 2108 > > Is this default behaviour from ceph? > I get the feeling the balancer might have something to do here as well as we have set the balancer to only allow for 1% misplaced objects, to limit this as well. If that’s true, could I just set pgp_num to 4096 directly and CEPH limits the data movement by itself? > > We are running a fully updated Nautilus cluster. > > Met vriendelijke groet, > Kind Regards, > Maarten van Ingen > > Specialist |SURF |maarten.vanin...@surf.nl<mailto:voornaam.achtern...@surf.nl>| T +31 30 88 787 3000 |M +31 6 19 03 90 19| > SURF<http://www.surf.nl/> is the collaborative organisation for ICT in Dutch education and research > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io