We have not reached the practical limit so far, and the theoretical limits are not practically realistic. The theoretical absolute limit on the number of nodes and the number of resources is somewhat more than 2 billion each. For DRBD resources, the maximum number of resources is limited by the size of the TCP/IP port number pool, since each DRBD resource is assigned a unique TCP/IP port number. Assuming that each resource has at least one volume, the maximum number of resources of each type is normally somewhat more than 1 million per resource type, due to operating system limitations on the number of devices per device driver type. For DRBD, the maximum number of volumes per resource is 65534, and the maximum number of volumes for all resources is ~ 1 million.
We have some partners who work on fairly large clusters, but I do not have exact numbers, so I would have to ask what the exact number of nodes and resources is. In our own test environment, we have run clusters with about 50 nodes, and we have run configurations with more than 2000 resource definitions and more than 4500 resources per cluster (all of those were replicated DRBD resources, not just local volumes). This was close to the practical limit of network traffic between DRBD resources on a single network interface (so that replication connection may become unreliable due to network traffic overload), but it works normally if that traffic is distributed using multiple network interfaces (or alternatively, a single faster one, of course). Anyhow, knowing the exact limits in LINSTOR only rules out LINSTOR's software logic as the source of a problem, but since most of the external environment that LINSTOR runs in - operating system, device drivers, utilities, filesystems, etc. - was not designed with determinism in mind. There is simply no culture of knowing the exact limitations of all the components and making sure that their implementations correctly and consistently implement them. As a result, the actual limits of the system as a whole are unknown. br, Robert On 6/17/19 4:48 PM, kvaps wrote: > Hello, > > Is there any known limits for the single linstor cluster? > How much nodes and resources can be created within the same cluster? > What is largest deployment do you have in your experience? > > Thank in advance! > > - kvaps > _______________________________________________ > drbd-user mailing list > [email protected] > http://lists.linbit.com/mailman/listinfo/drbd-user -- Robert ALTNOEDER - Software Developer +43-1-817-82-92 x72 <tel:+4318178292> [email protected] <mailto:[email protected]> LIN <http://www.linbit.com/en/>BIT <http://www.linbit.com/en/> | Keeping the Digital World Running DRBD HA - Disaster Recovery - Software-defined Storage t <https://twitter.com/linbit> / f <https://www.facebook.com/pg/linbitdrbd/posts/> / in <https://www.linkedin.com/company/linbit> / y <https://www.youtube.com/user/linbit> / g+ <https://plus.google.com/+Linbit/about> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. _______________________________________________ drbd-user mailing list [email protected] http://lists.linbit.com/mailman/listinfo/drbd-user
