The branch, master has been updated via 3cc596d2b459d834f9785b3a98027e46431ff2b9 (commit) via 6a1d88a17321f7e1dc84b4823d5e7588516a6904 (commit) via 8adb255e62dbe60d1e983047acd7b9c941231d11 (commit) via f6ce18d011dd9043b04256690d826deb2640cd89 (commit) via c09aeaecad7d3232b1c07bab826b96818756f5e0 (commit) via 4dc08e37dec464c8785a2ddae15c7c69d3c81ac3 (commit) via 4c87e7cb3fa2cf2e034fa8454364e0a7fe0c8f81 (commit) via e06476e07197b7327b8bdac9c0b2e7281798ffec (commit) via bcd5f587aff3ba536cb0b5ef00d2d802352bae25 (commit) via edda58a45915494027785608126b5da7c98fee85 (commit) via 5c820b2398a42af0e94bc524854a1ad144a63f7b (commit) via 98bd58a98d34ecca89c9042417d7527a18a5ecf9 (commit) via d357d52dbd533444a4af6151d04ba119a1533068 (commit) via 20631f5f29859920844dd8f410e24917aabd3dfd (commit) via 06ad6b8a19f830472b0ed65cb52e7c3ea74ed1dc (commit) from 1a5410e8349cdb96fdc51aa5ecd4f5734f6798a5 (commit)
http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 3cc596d2b459d834f9785b3a98027e46431ff2b9 Author: Martin Schwenke <mar...@meltin.net> Date: Tue Jan 8 10:21:49 2013 +1100 tests/eventscripts: Ratchet down debug level for ctdb_takeover_tests The default IP allocation algorithm used by ctdb_takeover_tests changed from "non-deterministic IPs" to "LCP2". The latter generates a lot more debug output. ctdb_takeover_tests is used by the ctdb tool stub to calculate IP address changes for failovers. This resulted in unexpected debug output that caused tests to fail. Since eventscript tests don't care how IP allocations are arrived at, the best solution is to turn down the debug level. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 6a1d88a17321f7e1dc84b4823d5e7588516a6904 Author: Martin Schwenke <mar...@meltin.net> Date: Fri Dec 14 17:12:01 2012 +1100 recoverd: Separate each IP allocation algorithm into its own function This makes the code much more readable and maintainable. As a side effect, fix a memory leak in LCP2. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 8adb255e62dbe60d1e983047acd7b9c941231d11 Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 13 13:23:32 2012 +1100 recoverd: New function unassign_unsuitable_ips() Move the code into a new function so it can be called from a number of places. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit f6ce18d011dd9043b04256690d826deb2640cd89 Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 13 12:15:32 2012 +1100 recoverd: Move failback retry loop into basic_failback() and lcp2_failback() The retry loop is currently in ctdb_takeover_run_core(). Pushing it into each function will make it possible to put each algorithm into a separate top-level function. This will make the code much clearer and more maintainable. Also keep associated test code compatible. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit c09aeaecad7d3232b1c07bab826b96818756f5e0 Author: Martin Schwenke <mar...@meltin.net> Date: Tue Dec 11 15:49:17 2012 +1100 recoverd: Trying to failback more IPs no longer allocates unassigned IPs Neither basic_failback() nor lcp2_failback() unassign IPs anymore, so there's no point looping back that far. Also fix a unit test that now fails because looping back to handle unassigned IPs is no longer logged. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 4dc08e37dec464c8785a2ddae15c7c69d3c81ac3 Author: Martin Schwenke <mar...@meltin.net> Date: Tue Dec 11 15:43:36 2012 +1100 recoverd: basic_failback() can call find_takeover_node() directly Instead of unassigning, looping back and depending on basic_allocate_unassigned. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 4c87e7cb3fa2cf2e034fa8454364e0a7fe0c8f81 Author: Martin Schwenke <mar...@meltin.net> Date: Tue Dec 11 15:01:12 2012 +1100 recoverd: Don't do failback at all when deterministic IPs are in use This seems to be the right thing to do instead of calling into the failback code and continually skipping the release of an IP. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit e06476e07197b7327b8bdac9c0b2e7281798ffec Author: Martin Schwenke <mar...@meltin.net> Date: Fri Dec 14 17:10:41 2012 +1100 recoverd: Move the test for both 'DeterministicIPs' and 'NoIPFailback' set If this is done earlier then some other logic can be improved. Also, this should be a warning since no error condition is set. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit bcd5f587aff3ba536cb0b5ef00d2d802352bae25 Author: Martin Schwenke <mar...@meltin.net> Date: Fri Dec 14 17:10:05 2012 +1100 recoverd: Fix a memory leak in IP allocation Signed-off-by: Martin Schwenke <mar...@meltin.net> commit edda58a45915494027785608126b5da7c98fee85 Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 20 16:27:27 2012 +1100 tests/takeover: Add some LCP2 tests for case when no node are healthy 3 tests should assign IPs to all nodes. 3 tests set NoIPTakeoverOnDisabled=1 and should drop all IPs. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 5c820b2398a42af0e94bc524854a1ad144a63f7b Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 20 16:26:42 2012 +1100 tests/takeover: Initial tests for deterministic IPs Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 98bd58a98d34ecca89c9042417d7527a18a5ecf9 Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 20 16:25:53 2012 +1100 tests/takeover: Do output filtering for deterministic IPs algorithm too Signed-off-by: Martin Schwenke <mar...@meltin.net> commit d357d52dbd533444a4af6151d04ba119a1533068 Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 20 16:24:58 2012 +1100 tests/takeover: Support testing of NoIPTakeoverOnDisabled Via $CTDB_SET_NoIPTakeoverOnDisabled. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 20631f5f29859920844dd8f410e24917aabd3dfd Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 20 14:52:05 2012 +1100 tests/takeover: IP allocation now selected via $CTDB_IP_ALGORITHM Default to LCP2, like ctdbd. Also support "det" for deterministic IPs. Signed-off-by: Martin Schwenke <mar...@meltin.net> commit 06ad6b8a19f830472b0ed65cb52e7c3ea74ed1dc Author: Martin Schwenke <mar...@meltin.net> Date: Thu Dec 13 20:29:22 2012 +1100 tests/takeover: Support valgrinding the takeover code Signed-off-by: Martin Schwenke <mar...@meltin.net> ----------------------------------------------------------------------- Summary of changes: server/ctdb_takeover.c | 268 +++++++++++++++----------- tests/eventscripts/stubs/ctdb | 3 +- tests/src/ctdb_takeover_tests.c | 29 ++- tests/takeover/{nondet.001.sh => det.001.sh} | 1 + tests/takeover/{nondet.002.sh => det.002.sh} | 13 +- tests/takeover/{nondet.003.sh => det.003.sh} | 13 +- tests/takeover/lcp2.005.sh | 18 -- tests/takeover/{lcp2.008.sh => lcp2.014.sh} | 20 +- tests/takeover/lcp2.015.sh | 31 +++ tests/takeover/{lcp2.003.sh => lcp2.016.sh} | 34 ++-- tests/takeover/lcp2.017.sh | 32 +++ tests/takeover/{lcp2.007.sh => lcp2.018.sh} | 39 ++-- tests/takeover/{lcp2.007.sh => lcp2.019.sh} | 37 ++-- tests/takeover/scripts/local.sh | 22 +-- 14 files changed, 326 insertions(+), 234 deletions(-) copy tests/takeover/{nondet.001.sh => det.001.sh} (85%) copy tests/takeover/{nondet.002.sh => det.002.sh} (65%) copy tests/takeover/{nondet.003.sh => det.003.sh} (66%) copy tests/takeover/{lcp2.008.sh => lcp2.014.sh} (84%) create mode 100755 tests/takeover/lcp2.015.sh copy tests/takeover/{lcp2.003.sh => lcp2.016.sh} (52%) create mode 100755 tests/takeover/lcp2.017.sh copy tests/takeover/{lcp2.007.sh => lcp2.018.sh} (56%) copy tests/takeover/{lcp2.007.sh => lcp2.019.sh} (59%) Changeset truncated at 500 lines: diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c index df543fe..d601b17 100644 --- a/server/ctdb_takeover.c +++ b/server/ctdb_takeover.c @@ -1604,17 +1604,22 @@ static void basic_allocate_unassigned(struct ctdb_context *ctdb, /* Basic non-deterministic rebalancing algorithm. */ -static bool basic_failback(struct ctdb_context *ctdb, +static void basic_failback(struct ctdb_context *ctdb, struct ctdb_node_map *nodemap, uint32_t mask, struct ctdb_public_ip_list *all_ips, - int num_ips, - int *retries) + int num_ips) { int i; - int maxnode, maxnum=0, minnode, minnum=0, num; + int maxnode, maxnum, minnode, minnum, num, retries; struct ctdb_public_ip_list *tmp_ip; + retries = 0; + +try_again: + maxnum=0; + minnum=0; + /* for each ip address, loop over all nodes that can serve this ip and make sure that the difference between the node serving the most and the node serving the least ip's are @@ -1673,13 +1678,6 @@ static bool basic_failback(struct ctdb_context *ctdb, continue; } - /* If we want deterministic IPs then dont try to reallocate - them to spread out the load. - */ - if (1 == ctdb->tunable.deterministic_public_ips) { - continue; - } - /* if the spread between the smallest and largest coverage by a node is >=2 we steal one of the ips from the node with most coverage to even things out a bit. @@ -1687,23 +1685,19 @@ static bool basic_failback(struct ctdb_context *ctdb, want to spend too much time balancing the ip coverage. */ if ( (maxnum > minnum+1) - && (*retries < (num_ips + 5)) ){ + && (retries < (num_ips + 5)) ){ struct ctdb_public_ip_list *tmp; - /* mark one of maxnode's vnn's as unassigned and try - again - */ + /* Reassign one of maxnode's VNNs */ for (tmp=all_ips;tmp;tmp=tmp->next) { if (tmp->pnn == maxnode) { - tmp->pnn = -1; - (*retries)++; - return true; + (void)find_takeover_node(ctdb, nodemap, mask, tmp, all_ips); + retries++; + goto try_again;; } } } } - - return false; } struct ctdb_rebalancenodes { @@ -2002,7 +1996,7 @@ static int lcp2_cmp_imbalance_pnn(const void * a, const void * b) * node with the highest LCP2 imbalance, and then determines the best * IP/destination node combination to move from the source node. */ -static bool lcp2_failback(struct ctdb_context *ctdb, +static void lcp2_failback(struct ctdb_context *ctdb, struct ctdb_node_map *nodemap, uint32_t mask, struct ctdb_public_ip_list *all_ips, @@ -2011,7 +2005,9 @@ static bool lcp2_failback(struct ctdb_context *ctdb, { int i, num_newly_healthy; struct lcp2_imbalance_pnn * lips; - bool ret; + bool again; + +try_again: /* It is only worth continuing if we have suitable target * nodes to transfer IPs to. This check is much cheaper than @@ -2024,7 +2020,7 @@ static bool lcp2_failback(struct ctdb_context *ctdb, } } if (num_newly_healthy == 0) { - return false; + return; } /* Put the imbalances and nodes into an array, sort them and @@ -2039,7 +2035,7 @@ static bool lcp2_failback(struct ctdb_context *ctdb, qsort(lips, nodemap->num, sizeof(struct lcp2_imbalance_pnn), lcp2_cmp_imbalance_pnn); - ret = false; + again = false; for (i = 0; i < nodemap->num; i++) { /* This means that all nodes had 0 or 1 addresses, so * can't be imbalanced. @@ -2055,76 +2051,23 @@ static bool lcp2_failback(struct ctdb_context *ctdb, lips[i].imbalance, lcp2_imbalances, newly_healthy)) { - ret = true; + again = true; break; } } talloc_free(lips); - return ret; + if (again) { + goto try_again; + } } -/* The calculation part of the IP allocation algorithm. */ -static void ctdb_takeover_run_core(struct ctdb_context *ctdb, - struct ctdb_node_map *nodemap, - struct ctdb_public_ip_list **all_ips_p) +static void unassign_unsuitable_ips(struct ctdb_context *ctdb, + struct ctdb_node_map *nodemap, + struct ctdb_public_ip_list *all_ips, + uint32_t mask) { - int i, num_healthy, retries, num_ips; - uint32_t mask; - struct ctdb_public_ip_list *all_ips, *tmp_ip; - uint32_t *lcp2_imbalances; - bool *newly_healthy; - - TALLOC_CTX *tmp_ctx = talloc_new(ctdb); - - /* Count how many completely healthy nodes we have */ - num_healthy = 0; - for (i=0;i<nodemap->num;i++) { - if (!(nodemap->nodes[i].flags & (NODE_FLAGS_INACTIVE|NODE_FLAGS_DISABLED))) { - num_healthy++; - } - } - - /* If we have healthy nodes then we will only consider them - for serving public addresses - */ - mask = NODE_FLAGS_INACTIVE|NODE_FLAGS_DISABLED; - if ((num_healthy == 0) && - (ctdb->tunable.no_ip_takeover_on_disabled == 0)) { - /* We didnt have any completely healthy nodes so - use "disabled" nodes as a fallback - */ - mask = NODE_FLAGS_INACTIVE; - } - - /* since nodes only know about those public addresses that - can be served by that particular node, no single node has - a full list of all public addresses that exist in the cluster. - Walk over all node structures and create a merged list of - all public addresses that exist in the cluster. - - keep the tree of ips around as ctdb->ip_tree - */ - all_ips = create_merged_ip_list(ctdb); - *all_ips_p = all_ips; /* minimal code changes */ - - /* Count how many ips we have */ - num_ips = 0; - for (tmp_ip=all_ips;tmp_ip;tmp_ip=tmp_ip->next) { - num_ips++; - } - - /* If we want deterministic ip allocations, i.e. that the ip addresses - will always be allocated the same way for a specific set of - available/unavailable nodes. - */ - if (1 == ctdb->tunable.deterministic_public_ips) { - DEBUG(DEBUG_NOTICE,("Deterministic IPs enabled. Resetting all ip allocations\n")); - for (i=0,tmp_ip=all_ips;tmp_ip;tmp_ip=tmp_ip->next,i++) { - tmp_ip->pnn = i%nodemap->num; - } - } - + struct ctdb_public_ip_list *tmp_ip; /* mark all public addresses with a masked node as being served by node -1 @@ -2150,51 +2093,144 @@ static void ctdb_takeover_run_core(struct ctdb_context *ctdb, tmp_ip->pnn = -1; } } +} - if (1 == ctdb->tunable.lcp2_public_ip_assignment) { - lcp2_init(tmp_ctx, nodemap, mask, all_ips, &lcp2_imbalances, &newly_healthy); - } +static void ip_alloc_deterministic_ips(struct ctdb_context *ctdb, + struct ctdb_node_map *nodemap, + struct ctdb_public_ip_list *all_ips, + uint32_t mask) +{ + struct ctdb_public_ip_list *tmp_ip; + int i; - /* now we must redistribute all public addresses with takeover node - -1 among the nodes available + DEBUG(DEBUG_NOTICE,("Deterministic IPs enabled. Resetting all ip allocations\n")); + /* Allocate IPs to nodes in a modulo fashion so that IPs will + * always be allocated the same way for a specific set of + * available/unavailable nodes. */ - retries = 0; -try_again: - if (1 == ctdb->tunable.lcp2_public_ip_assignment) { - lcp2_allocate_unassigned(ctdb, nodemap, mask, all_ips, lcp2_imbalances); - } else { - basic_allocate_unassigned(ctdb, nodemap, mask, all_ips); + + for (i=0,tmp_ip=all_ips;tmp_ip;tmp_ip=tmp_ip->next,i++) { + tmp_ip->pnn = i%nodemap->num; } - /* If we dont want ips to fail back after a node becomes healthy - again, we wont even try to reallocat the ip addresses so that - they are evenly spread out. - This can NOT be used at the same time as DeterministicIPs ! + /* IP failback doesn't make sense with deterministic + * IPs, since the modulo step above implicitly fails + * back IPs to their "home" node. + */ + if (1 == ctdb->tunable.no_ip_failback) { + DEBUG(DEBUG_WARNING, ("WARNING: 'NoIPFailback' set but ignored - incompatible with 'DeterministicIPs\n")); + } + + unassign_unsuitable_ips(ctdb, nodemap, all_ips, mask); + + basic_allocate_unassigned(ctdb, nodemap, mask, all_ips); + + /* No failback here! */ +} + +static void ip_alloc_nondeterministic_ips(struct ctdb_context *ctdb, + struct ctdb_node_map *nodemap, + struct ctdb_public_ip_list *all_ips, + uint32_t mask) +{ + /* This should be pushed down into basic_failback. */ + struct ctdb_public_ip_list *tmp_ip; + int num_ips = 0; + for (tmp_ip=all_ips;tmp_ip;tmp_ip=tmp_ip->next) { + num_ips++; + } + + unassign_unsuitable_ips(ctdb, nodemap, all_ips, mask); + + basic_allocate_unassigned(ctdb, nodemap, mask, all_ips); + + /* If we don't want IPs to fail back then don't rebalance IPs. */ + if (1 == ctdb->tunable.no_ip_failback) { + return; + } + + /* Now, try to make sure the ip adresses are evenly distributed + across the nodes. */ + basic_failback(ctdb, nodemap, mask, all_ips, num_ips); +} + +static void ip_alloc_lcp2(struct ctdb_context *ctdb, + struct ctdb_node_map *nodemap, + struct ctdb_public_ip_list *all_ips, + uint32_t mask) +{ + uint32_t *lcp2_imbalances; + bool *newly_healthy; + + TALLOC_CTX *tmp_ctx = talloc_new(ctdb); + + unassign_unsuitable_ips(ctdb, nodemap, all_ips, mask); + + lcp2_init(tmp_ctx, nodemap, mask, all_ips, &lcp2_imbalances, &newly_healthy); + + lcp2_allocate_unassigned(ctdb, nodemap, mask, all_ips, lcp2_imbalances); + + /* If we don't want IPs to fail back then don't rebalance IPs. */ if (1 == ctdb->tunable.no_ip_failback) { - if (1 == ctdb->tunable.deterministic_public_ips) { - DEBUG(DEBUG_ERR, ("ERROR: You can not use 'DeterministicIPs' and 'NoIPFailback' at the same time\n")); - } goto finished; } - - /* now, try to make sure the ip adresses are evenly distributed - across the node. + /* Now, try to make sure the ip adresses are evenly distributed + across the nodes. */ - if (1 == ctdb->tunable.lcp2_public_ip_assignment) { - if (lcp2_failback(ctdb, nodemap, mask, all_ips, lcp2_imbalances, newly_healthy)) { - goto try_again; + lcp2_failback(ctdb, nodemap, mask, all_ips, lcp2_imbalances, newly_healthy); + +finished: + talloc_free(tmp_ctx); +} + +/* The calculation part of the IP allocation algorithm. */ +static void ctdb_takeover_run_core(struct ctdb_context *ctdb, + struct ctdb_node_map *nodemap, + struct ctdb_public_ip_list **all_ips_p) +{ + int i, num_healthy; + uint32_t mask; + + /* Count how many completely healthy nodes we have */ + num_healthy = 0; + for (i=0;i<nodemap->num;i++) { + if (!(nodemap->nodes[i].flags & (NODE_FLAGS_INACTIVE|NODE_FLAGS_DISABLED))) { + num_healthy++; } + } + + /* If we have healthy nodes then we will only consider them + for serving public addresses + */ + mask = NODE_FLAGS_INACTIVE|NODE_FLAGS_DISABLED; + if ((num_healthy == 0) && + (ctdb->tunable.no_ip_takeover_on_disabled == 0)) { + /* We didnt have any completely healthy nodes so + use "disabled" nodes as a fallback + */ + mask = NODE_FLAGS_INACTIVE; + } + + /* since nodes only know about those public addresses that + can be served by that particular node, no single node has + a full list of all public addresses that exist in the cluster. + Walk over all node structures and create a merged list of + all public addresses that exist in the cluster. + + keep the tree of ips around as ctdb->ip_tree + */ + *all_ips_p = create_merged_ip_list(ctdb); + + if (1 == ctdb->tunable.lcp2_public_ip_assignment) { + ip_alloc_lcp2(ctdb, nodemap, *all_ips_p, mask); + } else if (1 == ctdb->tunable.deterministic_public_ips) { + ip_alloc_deterministic_ips(ctdb, nodemap, *all_ips_p, mask); } else { - if (basic_failback(ctdb, nodemap, mask, all_ips, num_ips, &retries)) { - goto try_again; - } + ip_alloc_nondeterministic_ips(ctdb, nodemap, *all_ips_p, mask); } - /* finished distributing the public addresses, now just send the - info out to the nodes */ -finished: /* at this point ->pnn is the node which will own each IP or -1 if there is no node that can cover this ip */ diff --git a/tests/eventscripts/stubs/ctdb b/tests/eventscripts/stubs/ctdb index e06e199..3abdd16 100755 --- a/tests/eventscripts/stubs/ctdb +++ b/tests/eventscripts/stubs/ctdb @@ -126,7 +126,8 @@ ip_reallocate () fi _flags="${_flags}${_flags:+,}${_this}" done - "ctdb_takeover_tests" \ + CTDB_TEST_LOGLEVEL=2 \ + "ctdb_takeover_tests" \ "ctdb_takeover_run_core" "$_flags" <"$FAKE_CTDB_IP_LAYOUT" | sort >"$_t" mv "$_t" "$FAKE_CTDB_IP_LAYOUT" diff --git a/tests/src/ctdb_takeover_tests.c b/tests/src/ctdb_takeover_tests.c index 96cb9e5..837741e 100644 --- a/tests/src/ctdb_takeover_tests.c +++ b/tests/src/ctdb_takeover_tests.c @@ -346,18 +346,32 @@ void ctdb_test_init(const char nodestates[], /* Fake things up... */ (*ctdb)->num_nodes = numnodes; + /* Default to LCP2 */ + (*ctdb)->tunable.lcp2_public_ip_assignment = 1; (*ctdb)->tunable.deterministic_public_ips = 0; (*ctdb)->tunable.disable_ip_failover = 0; (*ctdb)->tunable.no_ip_failback = 0; - if (getenv("CTDB_LCP2")) { - if (strcmp(getenv("CTDB_LCP2"), "yes") == 0) { + if (getenv("CTDB_IP_ALGORITHM")) { + if (strcmp(getenv("CTDB_IP_ALGORITHM"), "lcp2") == 0) { (*ctdb)->tunable.lcp2_public_ip_assignment = 1; - } else { + } else if (strcmp(getenv("CTDB_IP_ALGORITHM"), "nondet") == 0) { + (*ctdb)->tunable.lcp2_public_ip_assignment = 0; + } else if (strcmp(getenv("CTDB_IP_ALGORITHM"), "det") == 0) { (*ctdb)->tunable.lcp2_public_ip_assignment = 0; + (*ctdb)->tunable.deterministic_public_ips = 1; + } else { + fprintf(stderr, "ERROR: unknown IP algorithm %s\n", + getenv("CTDB_IP_ALGORITHM")); + exit(1); } } + (*ctdb)->tunable.no_ip_takeover_on_disabled = 0; + if (getenv("CTDB_SET_NoIPTakeoverOnDisabled")) { + (*ctdb)->tunable.no_ip_takeover_on_disabled = (uint32_t) strtoul(getenv("CTDB_SET_NoIPTakeoverOnDisabled"), NULL, 0); + } + *nodemap = talloc_array(*ctdb, struct ctdb_node_map, numnodes); (*nodemap)->num = numnodes; @@ -444,12 +458,9 @@ void ctdb_test_lcp2_failback_loop(const char nodestates[]) NODE_FLAGS_INACTIVE|NODE_FLAGS_DISABLED, all_ips, &lcp2_imbalances, &newly_healthy); -try_again: - if (lcp2_failback(ctdb, nodemap, - NODE_FLAGS_INACTIVE|NODE_FLAGS_DISABLED, - all_ips, lcp2_imbalances, newly_healthy)) { - goto try_again; - } + lcp2_failback(ctdb, nodemap, + NODE_FLAGS_INACTIVE|NODE_FLAGS_DISABLED, + all_ips, lcp2_imbalances, newly_healthy); print_ctdb_public_ip_list(all_ips); diff --git a/tests/takeover/nondet.001.sh b/tests/takeover/det.001.sh similarity index 85% copy from tests/takeover/nondet.001.sh copy to tests/takeover/det.001.sh index 7150607..dec7010 100755 --- a/tests/takeover/nondet.001.sh +++ b/tests/takeover/det.001.sh @@ -5,6 +5,7 @@ define_test "3 nodes, 1 healthy" required_result <<EOF +DATE TIME [PID]: Deterministic IPs enabled. Resetting all ip allocations 192.168.21.254 2 192.168.21.253 2 192.168.21.252 2 diff --git a/tests/takeover/nondet.002.sh b/tests/takeover/det.002.sh similarity index 65% copy from tests/takeover/nondet.002.sh copy to tests/takeover/det.002.sh index 4ff801b..21c55bd 100755 --- a/tests/takeover/nondet.002.sh +++ b/tests/takeover/det.002.sh @@ -5,15 +5,16 @@ define_test "3 nodes, 2 healthy" required_result <<EOF -192.168.21.254 2 +DATE TIME [PID]: Deterministic IPs enabled. Resetting all ip allocations +192.168.21.254 0 192.168.21.253 0 -192.168.21.252 0 -192.168.20.254 2 +192.168.21.252 2 +192.168.20.254 0 192.168.20.253 2 -192.168.20.252 0 -192.168.20.251 2 +192.168.20.252 2 +192.168.20.251 0 192.168.20.250 0 -192.168.20.249 0 +192.168.20.249 2 EOF simple_test 0,2,0 <<EOF diff --git a/tests/takeover/nondet.003.sh b/tests/takeover/det.003.sh similarity index 66% copy from tests/takeover/nondet.003.sh copy to tests/takeover/det.003.sh index 2a9dfb4..3666047 100755 --- a/tests/takeover/nondet.003.sh +++ b/tests/takeover/det.003.sh @@ -5,15 +5,16 @@ define_test "3 nodes, 1 -> all healthy" required_result <<EOF +DATE TIME [PID]: Deterministic IPs enabled. Resetting all ip allocations -- CTDB repository