Ganeti provides high availability by ensuring N+1 redundancy is maintained. In some situations, however, like planning larger maintenance events, it is desirable to have an estimate for how many nodes can be removes with the cluster remaining operational. Add a design for this concept.
Signed-off-by: Klaus Aehlig <[email protected]> --- Makefile.am | 1 + doc/design-draft.rst | 1 + doc/design-n-m-redundancy.rst | 71 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 73 insertions(+) create mode 100644 doc/design-n-m-redundancy.rst diff --git a/Makefile.am b/Makefile.am index b5a353d..5df7d2c 100644 --- a/Makefile.am +++ b/Makefile.am @@ -704,6 +704,7 @@ docinput = \ doc/design-multi-reloc.rst \ doc/design-multi-storage-htools.rst \ doc/design-multi-version-tests.rst \ + doc/design-n-m-redundancy.rst \ doc/design-network.rst \ doc/design-network2.rst \ doc/design-node-add.rst \ diff --git a/doc/design-draft.rst b/doc/design-draft.rst index 247eeca..49e7ab9 100644 --- a/doc/design-draft.rst +++ b/doc/design-draft.rst @@ -28,6 +28,7 @@ Design document drafts design-migration-speed-hbal.rst design-memory-over-commitment.rst design-macvtap.rst + design-n-m-redundancy.rst .. vim: set textwidth=72 : .. Local Variables: diff --git a/doc/design-n-m-redundancy.rst b/doc/design-n-m-redundancy.rst new file mode 100644 index 0000000..696bd5e --- /dev/null +++ b/doc/design-n-m-redundancy.rst @@ -0,0 +1,71 @@ +=========================== +Checking for N+M redundancy +=========================== + +.. contents:: :depth: 4 + +This document describes how the level of redundancy is estimated +in Ganeti. + + +Current state and shortcomings +============================== + +Ganeti keeps the cluster N+1 redundant, also taking into account +:doc:`design-shared-storage-redundancy`. However, e.g., for planning +maintenance, it is sometimes desirable to know from how many node +losses the cluster can recover from. This is also useful information, +when operating big clusters and expecting long times for hardware repair. + + +Proposed changes +================ + +Higher redundancy as a sequential concept +----------------------------------------- + +The intuitive meaning of an N+M redundant cluster is that M nodes can +fail without instances being lost. However, when DRBD is used, already +failure of 2 nodes can cause complete loss of an instance. Therefore, the +best we can hope for, is to be able to recover from M sequential failures. + +Definition of M+M redundancy +---------------------------- + +We keep the definition of :doc:`design-shared-storage-redundancy`. Moreover, +for M a non-negative integer, we define a cluster to be N+(M+2) redundant, +if after draining any node the standard rebalancing procedure (as, e.g., +provided by `hbal`) will fully evacuate that node and result in an N+(M+1) +redundant cluster. + +Independence of Groups +---------------------- + +Immediately from the definition, we see that the redundancy level, i.e., +the maximal M such that the cluster is N+M redundant, can be computed +in a group-by-group manner: the standard balancing algorithm will never +move instances between node groups. The redundancy level of the cluster +is then the minimum of the redundancy level of the independent groups. + +Estimation of the redundancy level +---------------------------------- + +The definition of N+M redundancy requires to consider M failures in +arbitrary order, thus considering super-exponentially many cases for +large M. As, however, balancing moves instances anyway, the redundancy +level mainly depends on the amount of node resources available to the +instances in a node group. So we can get a good approximation of the +redundancy level of a node group by only considering draining one largest +node in that group. This is how Ganeti will estimate the redundancy level. + +Modifications to existing tools +------------------------------- + +As redundancy levels higher than N+1 are mainly about planning capacity, +they level of redundancy only needs to be computed on demand. Hence, we +keep the tool changes minimal. + +- ``hcheck`` will report the level of redundancy for each node group as + a new output parameter + +The rest of Ganeti will not be changed. -- 2.6.0.rc2.230.g3dd15c0
