hi guys , your responses were very useful. Lets suppose I have my 3 nodes running and forming a cluster. Please confirm: a -Can I add the ceph storage at any time ? b- All nodes should be running the same pve version ? c- All nodes should have 1 or more non used storages with no hardware raid to be included in the ceph ? Those storages (c) should be exactly same in capacity , speed , and so ... ? What can goes wrong if dont have 10 but 1 gbps ports ? Regards. Leandro
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Libre de virus. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> El mié, 8 sept 2021 a las 19:21, ic (<[email protected]>) escribió: > Hi there, > > > On 8 Sep 2021, at 14:46, Leandro Roggerone <[email protected]> > wrote: > > > > I would like to know the goods that a ceph storage can bring to my > existing > > cluster. > > What is an easy / recommended way to implement it ? > > Wich hardware should I consider to use ? > > First, HW. > > Get two Cisco Nexus 3064PQ (they typically go for $600-700 for 48 10G > ports) and two Intel X520-DA2 per server. > > Hook up each port of the Intel cards to each of the Nexuses, getting a > full redundancy between network cards and switches. > > Add 4x40G DAC cables between the switches, setup 2 as VPC peer-links, 2 as > a simple L2 trunk (can provide more details as why if needed). > > Use ports 0 from both NICs for ceph, ports 1 for VM traffic. This way you > get 2x10 Gbps for Ceph only and 2x10 Gbps for everything else, and if you > loose one card or one switch, you still have 10 Gbps for each. > > The benefits? With default configuration, your data lives in 3 places. > Also, scale out. You know the expensive stuff, hyperconverged servers > (nutanix and such) ? You get that with this. > > The performance is wild, just moved my customers from a proxmox cluster > backed by a TrueNAS server (full flash, 4x10Gbps) to a 3 node cluster of > AMD EPYC nodes with Ceph on local SATA SSDs and the VMs started flying. > > Keep your old storage infrastructure, whatever that is, for backups with > PBS. > > YMMV > > Regards, ic > > _______________________________________________ > pve-user mailing list > [email protected] > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > _______________________________________________ pve-user mailing list [email protected] https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
