Re: [ceph-users] Number of PGs: fix from start or change as we grow ?
Hello, On Wed, 3 Aug 2016 15:15:21 +0300 Maged Mokhtar wrote: > Hello, > > I would like to build a small cluster with 20 disks to start but in the > future would like to gradually increase it to maybe 200 disks. > Is it better to fix the number of PGs in the pool from the beginning or is it > better to start with a small number and then gradually change the number of > PGs as the system grows ? > Is the act of changing the number of PGs in a running cluster something that > can be done regularly ? > This is both something that's strongly hinted at in the documentation as well as discussed countless times on this ML (google is your friend), along with the means to minimize the impact of this action. Setting the "correct" PG value for a 200 OSD cluster (8192) at the start with 20 OSDs (recommended value 512) is not going to be pretty and will have your cluster in a warning state with about 1200 (!) PGs per OSD at the very least. Never mind CPU and RAM usage. Increasing PGs is an involved and costly operation, so it should be done as little as possible. However if your cluster is designed/configured well and not operating constantly at its breaking point, it's also an operation that should be doable w/o major impacts. I'd start with 1024 PGs on those 20 OSDs, at 50 OSDs go to 4096 PGs and at around 100 OSDs it is safe to go to 8192 PGs. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Number of PGs: fix from start or change as we grow ?
Changing the number of PGs is one of the most expensive operations you can run, and should be avoided as much as possible. Having said that you should try to avoid having way too many PGs with very few OSDs, but it's certainly preferable to splitting PGs... On Wed, Aug 3, 2016 at 1:15 PM, Maged Mokhtar wrote: > Hello, > > I would like to build a small cluster with 20 disks to start but in the > future would like to gradually increase it to maybe 200 disks. > Is it better to fix the number of PGs in the pool from the beginning or is > it better to start with a small number and then gradually change the number > of PGs as the system grows ? > Is the act of changing the number of PGs in a running cluster something that > can be done regularly ? > > Cheers > /Maged > > > > > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Number of PGs: fix from start or change as we grow ?
Hello, I would like to build a small cluster with 20 disks to start but in the future would like to gradually increase it to maybe 200 disks. Is it better to fix the number of PGs in the pool from the beginning or is it better to start with a small number and then gradually change the number of PGs as the system grows ? Is the act of changing the number of PGs in a running cluster something that can be done regularly ? Cheers /Maged ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com