Re: [ceph-users] a big cluster or several small

2018-05-16 Thread Jack
For what it worth, yahoo published their setup some years ago: https://yahooeng.tumblr.com/post/116391291701/yahoo-cloud-object-store-object-storage-at 54 nodes per cluster for 3.2PB of raw storage, I guess this leads to 16 * 4TB hdd per node, thus 896 per cluster (they may have used ssd as

Re: [ceph-users] a big cluster or several small

2018-05-16 Thread Matthew Vernon
Hi, On 14/05/18 17:49, Marc Boisis wrote: > Currently we have a 294 OSD (21 hosts/3 racks) cluster with RBD clients > only, 1 single pool (size=3). That's not a large cluster. > We want to divide this cluster into several to minimize the risk in case > of failure/crash. > For example, a

Re: [ceph-users] a big cluster or several small

2018-05-16 Thread Alexandre DERUMIER
dre - Mail original - De: "Piotr Dałek" <piotr.da...@corp.ovh.com> À: "ceph-users" <ceph-users@lists.ceph.com> Envoyé: Mardi 15 Mai 2018 09:14:53 Objet: Re: [ceph-users] a big cluster or several small On 18-05-14 06:49 PM, Marc Boisis wrote: > > Hi,

Re: [ceph-users] a big cluster or several small

2018-05-15 Thread Piotr Dałek
On 18-05-14 06:49 PM, Marc Boisis wrote: Hi, Hello, Currently we have a 294 OSD (21 hosts/3 racks) cluster with RBD clients only, 1 single pool (size=3). We want to divide this cluster into several to minimize the risk in case of failure/crash. For example, a cluster for the mail, another

Re: [ceph-users] a big cluster or several small

2018-05-14 Thread Paul Emmerich
Hi, don't do multiple clusters on the same server without containers; support for the cluster name stuff is deprecated and will probably be removed: https://github.com/ceph/ceph-deploy/pull/441 Also, I wouldn't split your cluster (yet?), ~300 OSDs is still quite small. But it depends on the

Re: [ceph-users] a big cluster or several small

2018-05-14 Thread João Paulo Sacchetto Ribeiro Bastos
t:* Monday, May 14, 2018 9:50 AM > *To:* ceph-users > *Subject:* [ceph-users] a big cluster or several small > > > > > Hi, > > > > Hello, > > Currently we have a 294 OSD (21 hosts/3 racks) cluster with RBD clients > only, 1 single pool (size=3). > > &g

Re: [ceph-users] a big cluster or several small

2018-05-14 Thread Michael Kuriger
] On Behalf Of Marc Boisis Sent: Monday, May 14, 2018 9:50 AM To: ceph-users Subject: [ceph-users] a big cluster or several small Hi, Hello, Currently we have a 294 OSD (21 hosts/3 racks) cluster with RBD clients only, 1 single pool (size=3). We want to divide this cluster into several

Re: [ceph-users] a big cluster or several small

2018-05-14 Thread Jack
Well I currently manage 27 nodes, over 9 clusters There is some burden that you should considers The easiest is : "what do we do when two smalls clusters, which grows slowly, need more space" With one cluster: buy a node, add it, done With two clusters: buy two nodes, add them, done This can be

[ceph-users] a big cluster or several small

2018-05-14 Thread Marc Boisis
Hi, Hello, Currently we have a 294 OSD (21 hosts/3 racks) cluster with RBD clients only, 1 single pool (size=3). We want to divide this cluster into several to minimize the risk in case of failure/crash. For example, a cluster for the mail, another for the file servers, a test cluster ... Do