[ceph-users] Small cluster for VMs hosting
Hi to all I've been far from ceph from a couple of years (CephFS was still unstable) I would like to test it again, some questions for a production cluster for VMs hosting: 1. Is CephFS stable? 2. Can I spin up a 3 nodes cluster with mons, MDS and osds on the same machine? 3. Hardware suggestions? 4. How can I understand the ceph health status output, in details? I've not seen any docs about this 5. How can I know if cluster is fully synced or if any background operation (scrubbing, replication, ...) Is running? 6. Is 10G Ethernet mandatory? Currently I only have 4 gigabit nic (2 for public traffic, 2 for cluster traffic) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Small cluster for VMs hosting
Hi Gandalf, El 07/11/17 a las 14:16, Gandalf Corvotempesta escribió: Hi to all I've been far from ceph from a couple of years (CephFS was still unstable) I would like to test it again, some questions for a production cluster for VMs hosting: 1. Is CephFS stable? Yes. 2. Can I spin up a 3 nodes cluster with mons, MDS and osds on the same machine? Yes. 3. Hardware suggestions? Depends on your load. :-) 4. How can I understand the ceph health status output, in details? I've not seen any docs about this I think it is quite self-explanatory when you know how works Ceph. Don't run Ceph if you don't understand it :) Hell don't plan a Ceph deployment before understanding it either (reading this list can help too, look at the archives). 5. How can I know if cluster is fully synced or if any background operation (scrubbing, replication, ...) Is running? Looking at the health status output. 6. Is 10G Ethernet mandatory? Currently I only have 4 gigabit nic (2 for public traffic, 2 for cluster traffic) It is not mandatory, I administer 4 3-node clusters and all have 1gbit NICs. Cheers Eneko -- Zuzendari Teknikoa / Director Técnico Binovo IT Human Project, S.L. Telf. 943569206 Astigarraga bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa) www.binovo.es ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Small cluster for VMs hosting
On 07/11/17 13:16, Gandalf Corvotempesta wrote:
> Hi to all
> I've been far from ceph from a couple of years (CephFS was still unstable)
>
> I would like to test it again, some questions for a production cluster for
> VMs hosting:
>
> 1. Is CephFS stable?
Yes, CephFS is stable and safe (though it can have performance issues relating
to creating/removing files if your layout requires very large numbers of files
in a single directory)
> 2. Can I spin up a 3 nodes cluster with mons, MDS and osds on the same
> machine?
Recommended practice is not to co-locate OSDs with other ceph daemons, but
realistically lots of people do this (me included) and it works fine. Just
don't overload your nodes. In recent versions (kraken, luminous) there's a new
ceph-mgr daemon to keep in mind too.
> 3. Hardware suggestions?
Depends quite a lot on your budget and what performance you need. Ceph's
relatively CPU-heavy as these storage solutions go so good CPUs is advised, I
understand that single-threaded performance is probably more important than
having lots of cores if you're dealing with very very fast OSDs (like on NVMe).
Default memory requirements are 1GB/HDD OSD and 3GB/SSD OSD when using the
bluestore backend, but add maybe 50% for overhead due to fragmentation etc.
plus the resource cost of your other daemons.
> 4. How can I understand the ceph health status output, in details? I've not
> seen any docs about this
Read up on http://docs.ceph.com/docs/master/rados/operations/monitoring-osd-pg/
and http://docs.ceph.com/docs/master/rados/operations/pg-states/ -
understanding what the different states that PGs and OSDs can be in mean should
be enough for you to grok ceph status output.
> 5. How can I know if cluster is fully synced or if any background operation
> (scrubbing, replication, ...) Is running?
"ceph status" ("ceph -s" for short) will give you a point in time report of
your cluster state including PG states. If things are scrubbing or whatever
that will represented in the PG states. "ceph -w" will give you status and then
a rolling output of status changes/reports if the cluster does anything
interesting. One of the functions available in the newer ceph-mgr daemon is an
http dashboard giving you a quick overview of cluster health.
> 6. Is 10G Ethernet mandatory? Currently I only have 4 gigabit nic (2 for
> public traffic, 2 for cluster traffic)
It's not mandatory, but the more bandwidth you can throw at ceph generally the
happier it is. If you expect relatively lightweight usage I wouldn't worry -
but if performance was an issue and nodes otherwise healthy, 1G links as
bottlenecks would be the first thing I checked.
You seem interested in cephfs but you mention you're looking at ceph as a
backend for VM hosting, is that coincidental or are you intending to use disk
images stored as files in cephfs? Using RBDs would be a much more sensible idea
if so.
--
Rich
signature.asc
Description: OpenPGP digital signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Small cluster for VMs hosting
2017-11-07 14:49 GMT+01:00 Richard Hesketh : > Read up on > http://docs.ceph.com/docs/master/rados/operations/monitoring-osd-pg/ and > http://docs.ceph.com/docs/master/rados/operations/pg-states/ - understanding > what the different states that PGs and OSDs can be in mean should be enough > for you to grok ceph status output. Thaks, I was looking exactly for this. > You seem interested in cephfs but you mention you're looking at ceph as a > backend for VM hosting, is that coincidental or are you intending to use disk > images stored as files in cephfs? Using RBDs would be a much more sensible > idea if so. I would like to use both, RBDs and FS (for different things, in example, cephFS for hosting some maildirs) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
