On 11/22/18 7:29 PM, Frank Thommen wrote: > Please excuse, if this is too basic, but after reading > https://pve.proxmox.com/wiki/Cluster_Manager I wondered, if the > cluster/corosync network could be built by directly connected network > interfaces. I.e not like this: > > +-------+ > | pve01 |----------+ > +-------+ | > | > +-------+ +----------------+ > | pve02 |-----| network switch | > +-------+ +----------------+ > | > +-------+ | > | pve03 |----------+ > +-------+ > > > but like this: > > +-------+ > | pve01 |---+ > +-------+ | > | | > +-------+ | > | pve02 | | > +-------+ | > | | > +-------+ | > | pve03 |---+ > +-------+ > > (all connections 1Gbit, there are currently not plans to extend over three > nodes) > > I can't see any drawback in that solution. It would remove one layer of > hardware dependency and potential spof (the switch). If we don't trust the > interfaces, we might be able to configure a second network with the three > remaining interfaces. > > Is such a "direct-connection" topology feasible? Recommended? Strictly not > recommended?
full mesh is certainly not bad. for cluster network (corosync) latency is the key, bandwidth isn't really much needed. So this surely not bad. We use also some 10g (or 40G, not sure) full mesh for a ceph cluster network - you safe a not to cheap switch and get full bandwidth and good latency. The limiting factor is that this gets quite complex for bigger clusters, but besides that it doesn't really has any drawbacks for inter cluster connects, AFAICT. For multicast you need to try, as Uwe said, I'm currently not sure, it could work as Linux can route multicast just fine (mrouter) but I don't remember exactly anymore - sorry. But if you try it it'd be great if you report back. Else unicast is in those cluster sizes always an option - you really shouldn't have a problem as long as you do not put storage traffic together with corosync (cluster) on the same net (corosync gets to much latency spikes then). > > I am currently just planning and thinking and there is no cluster (or even a > PROXMOX server) in place. > > Cheers > frank _______________________________________________ pve-user mailing list pve-user@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
