The branch, master has been updated
via 93423cb ctdb-logging: Add forward declaration of debug_level
via 3715432 ctdb-tests: Clean up some tests where IP movement is checked
via 81a8758 ctdb-tests: Remove dependency on log ringbuffer from
missing IP test
via e3089d7 ctdb-tests: Make all_ips_on_node() do what it should
via 81213af ctdb-tests: Factor out new function
get_test_ip_mask_and_iface()
via 4b8cfe4 ctdb-tests: Simplify and rename
wait_until_ips_are_on_nodeglob()
from 8a6445d WHATSNEW: some fixes
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 93423cb1f594244e916e4ac3cb1d32220c48c172
Author: Martin Schwenke <mar...@meltin.net>
Date: Tue Sep 23 06:07:47 2014 +1000
ctdb-logging: Add forward declaration of debug_level
Warnings are currently produced when compiling Samba and
ctdb_private.h is included. A forward enum declaration avoids the
warning.
This is a temporary measure. The log ringbuffer should be removed
soon.
Signed-off-by: Martin Schwenke <mar...@meltin.net>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
Autobuild-User(master): Amitay Isaacs <ami...@samba.org>
Autobuild-Date(master): Tue Sep 23 10:31:50 CEST 2014 on sn-devel-104
commit 371543207e8955d6665fcec03b261d80cde2401b
Author: Martin Schwenke <mar...@meltin.net>
Date: Fri Sep 12 13:20:43 2014 +1000
ctdb-tests: Clean up some tests where IP movement is checked
Some of this implements logic that exists in functions. Some of it is
overly complicated and potentially failure-prone.
Signed-off-by: Martin Schwenke <mar...@meltin.net>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
commit 81a8758b9b24bbb7200be4682016f989b904bda8
Author: Martin Schwenke <mar...@meltin.net>
Date: Fri Sep 12 15:21:49 2014 +1000
ctdb-tests: Remove dependency on log ringbuffer from missing IP test
The log ringbuffer will probably be removed. The test can be
implemented just as reliably by checking IP assignments using "ctdb
ip".
Update wait_until_ips_are_on_node() to print a more useful log
message.
Signed-off-by: Martin Schwenke <mar...@meltin.net>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
commit e3089d7da17e2f25d426005eca0fe55397f2689f
Author: Martin Schwenke <mar...@meltin.net>
Date: Wed Sep 17 20:34:39 2014 +1000
ctdb-tests: Make all_ips_on_node() do what it should
The "-n all" is wrong.
Simplify the implementation and tighten up some uses of this function.
_select_test_node_and_ips() can't use this function anymore.
Signed-off-by: Martin Schwenke <mar...@meltin.net>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
commit 81213af32ae17eaae33f9d27c9b7ce63c84ce5df
Author: Martin Schwenke <mar...@meltin.net>
Date: Fri Sep 12 13:40:01 2014 +1000
ctdb-tests: Factor out new function get_test_ip_mask_and_iface()
Signed-off-by: Martin Schwenke <mar...@meltin.net>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
commit 4b8cfe4847477e3cfdb3f4dd7070226a6604bc38
Author: Martin Schwenke <mar...@meltin.net>
Date: Fri Sep 12 13:34:51 2014 +1000
ctdb-tests: Simplify and rename wait_until_ips_are_on_nodeglob()
The glob functionality is unsed so simplify the code by removing it.
Rename this function to wait_until_ips_are_on_node(). Update all
calls.
Signed-off-by: Martin Schwenke <mar...@meltin.net>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
-----------------------------------------------------------------------
Summary of changes:
ctdb/include/ctdb_private.h | 1 +
ctdb/tests/complex/11_ctdb_delip_removes_ip.sh | 81 ++---------------
ctdb/tests/scripts/integration.bash | 78 +++++++++++++----
ctdb/tests/simple/16_ctdb_config_add_ip.sh | 111 +++---------------------
ctdb/tests/simple/17_ctdb_config_delete_ip.sh | 59 ++-----------
ctdb/tests/simple/23_ctdb_moveip.sh | 94 ++++++--------------
ctdb/tests/simple/31_ctdb_disable.sh | 35 ++------
ctdb/tests/simple/32_ctdb_enable.sh | 41 ++-------
ctdb/tests/simple/41_ctdb_stop.sh | 33 ++------
ctdb/tests/simple/42_ctdb_continue.sh | 37 ++-------
ctdb/tests/simple/60_recoverd_missing_ip.sh | 58 ++++---------
11 files changed, 158 insertions(+), 470 deletions(-)
Changeset truncated at 500 lines:
diff --git a/ctdb/include/ctdb_private.h b/ctdb/include/ctdb_private.h
index 43daf36..02602e1 100644
--- a/ctdb/include/ctdb_private.h
+++ b/ctdb/include/ctdb_private.h
@@ -1461,6 +1461,7 @@ struct ctdb_get_log_addr {
extern int log_ringbuf_size;
+enum debug_level;
TDB_DATA ctdb_log_ringbuffer_collect_log(TALLOC_CTX *mem_ctx,
enum debug_level max_level);
void ctdb_collect_log(struct ctdb_context *ctdb, struct ctdb_get_log_addr
*log_addr);
diff --git a/ctdb/tests/complex/11_ctdb_delip_removes_ip.sh
b/ctdb/tests/complex/11_ctdb_delip_removes_ip.sh
index 043c345..d67cb07 100755
--- a/ctdb/tests/complex/11_ctdb_delip_removes_ip.sh
+++ b/ctdb/tests/complex/11_ctdb_delip_removes_ip.sh
@@ -5,36 +5,7 @@ test_info()
cat <<EOF
Verify that a node's public IP address can be deleted using 'ctdb deleteip'.
-Check that the address is actually deleted from the interface.
-
-Prerequisites:
-
-* An active CTDB cluster with at least 2 active nodes.
-
-* Test must be run on a real or virtual cluster rather than against
- local daemons. There is nothing intrinsic to this test that forces
- this - it is because tests run against local daemons don't use the
- regular eventscripts. Local daemons put public addresses on
- loopback, so we can't reliably test when IPs have moved between
- nodes.
-
-Steps:
-
-1. Verify that the status on all of the ctdb nodes is 'OK'.
-2. Use 'ctdb ip' on one of the nodes to list the IP addresses being
- served.
-3. Select an IP address being served by the node and check that it
- actually appears on the interface it is supposed to be on.
-4. Delete the IP address using 'ctdb delip'.
-5. Verify that the deleted IP address is no longer listed using the
- all_ips_on_node helper function.
-6. Verify that the deleted IP address no longer appears on the
- interface it was on.
-
-Expected results:
-
-* 'ctdb delip' removes an IP address from the list of public IP
- addresses being served by a node and from the network interface.
+This is an extended version of simple/17_ctdb_config_delete_ip.sh
EOF
}
@@ -51,55 +22,23 @@ cluster_is_healthy
# Reset configuration
ctdb_restart_when_done
-echo "Getting list of public IPs..."
-all_ips_on_node -v 0
-
-# Select an IP/node to remove.
-num_ips=$(echo "$out" | wc -l)
-num_to_remove=$(($RANDOM % $num_ips))
-
-# Find the details in the list.
-i=0
-while [ $i -le $num_to_remove ] ; do
- read ip_to_remove test_node
- i=$(($i + 1))
-done <<<"$out"
-
-echo "Determining interface for ${ip_to_remove} on ${test_node}."
-try_command_on_node $test_node "ctdb ip -Y -v"
-iface=$(echo "$out" | awk -F: -v ip=${ip_to_remove} -v pnn=${test_node} '$2 ==
ip && $3 == pnn { print $4 }')
-echo "$iface"
-[ -n "$iface" ]
-
-echo "Checking that node ${test_node} hosts ${ip_to_remove} on interface
${iface}..."
-try_command_on_node $test_node "ip addr show dev $iface | grep -E
'inet[[:space:]]*${ip_to_remove}/'"
-
-echo "Attempting to remove ${ip_to_remove} from node ${test_node}."
-try_command_on_node $test_node $CTDB delip $ip_to_remove
-
-echo "Sleeping..."
-sleep_for 1
+select_test_node_and_ips
+get_test_ip_mask_and_iface
-test_node_ips=""
-while read ip pnn ; do
- [ "$pnn" = "$test_node" ] && \
- test_node_ips="${test_node_ips}${test_node_ips:+ }${ip}"
-done <<<"$out" # bashism to avoid problem setting variable in pipeline.
+echo "Checking that node ${test_node} hosts ${test_ip} on interface
${iface}..."
+try_command_on_node $test_node "ip addr show dev $iface | grep -E
'inet[[:space:]]*${test_ip}/'"
-if [ "${test_node_ips/${ip_to_remove}}" = "$test_node_ips" ] ; then
- echo "GOOD: That worked!"
-else
- echo "BAD: The remove IP address is still there!"
- testfailures=1
-fi
+echo "Attempting to remove ${test_ip} from node ${test_node}."
+try_command_on_node $test_node $CTDB delip $test_ip
+wait_until_ips_are_on_node '!' $test_node $test_ip
timeout=60
increment=5
count=0
-echo "Waiting for ${ip_to_remove} to disappear from ${iface}..."
+echo "Waiting for ${test_ip} to disappear from ${iface}..."
while : ; do
try_command_on_node -v $test_node "ip addr show dev $iface"
- if echo "$out" | grep -E 'inet[[:space:]]*${ip_to_remove}/'; then
+ if echo "$out" | grep -E 'inet[[:space:]]*${test_ip}/'; then
echo "Still there..."
if [ $(($count * $increment)) -ge $timeout ] ; then
echo "BAD: Timed out waiting..."
diff --git a/ctdb/tests/scripts/integration.bash
b/ctdb/tests/scripts/integration.bash
index dec60a2..1835949 100644
--- a/ctdb/tests/scripts/integration.bash
+++ b/ctdb/tests/scripts/integration.bash
@@ -152,7 +152,8 @@ sanity_check_ips ()
prev="$ipp"
done <<<"$ips"
- echo "BAD: a node was -1 or IPs are only assigned to one node"
+ echo "BAD: a node was -1 or IPs are only assigned to one node:"
+ echo "$ips"
echo "Are you running an old version of CTDB?"
return 1
}
@@ -160,13 +161,15 @@ sanity_check_ips ()
# This returns a list of "ip node" lines in $out
all_ips_on_node()
{
- local node=$@
- try_command_on_node $node "$CTDB ip -Y -n all | cut -d ':' -f1-3 | sed -e
'1d' -e 's@^:@@' -e 's@:@ @g'"
+ local node="$1"
+ try_command_on_node $node \
+ "$CTDB ip -Y | awk -F: 'NR > 1 { print \$2, \$3 }'"
}
_select_test_node_and_ips ()
{
- all_ips_on_node 0
+ try_command_on_node any \
+ "$CTDB ip -Y -n all | awk -F: 'NR > 1 { print \$2, \$3 }'"
test_node="" # this matches no PNN
test_node_ips=""
@@ -202,6 +205,25 @@ select_test_node_and_ips ()
return 0
}
+# Sets: mask, iface
+get_test_ip_mask_and_iface ()
+{
+ # Find the interface
+ try_command_on_node $test_node "$CTDB ip -v -Y | awk -F: -v ip=$test_ip
'\$2 == ip { print \$4 }'"
+ iface="$out"
+
+ if [ -z "$TEST_LOCAL_DAEMONS" ] ; then
+ # Find the netmask
+ try_command_on_node $test_node ip addr show to $test_ip
+ mask="${out##*/}"
+ mask="${mask%% *}"
+ else
+ mask="24"
+ fi
+
+ echo "$test_ip/$mask is on $iface"
+}
+
#######################################
# Wait until either timeout expires or command succeeds. The command
@@ -380,29 +402,31 @@ wait_until_node_has_status ()
}
# Useful for superficially testing IP failover.
-# IPs must be on nodes matching nodeglob.
-# If the first argument is '!' then the IPs must not be on nodes
-# matching nodeglob.
-ips_are_on_nodeglob ()
+# IPs must be on the given node.
+# If the first argument is '!' then the IPs must not be on the given node.
+ips_are_on_node ()
{
local negating=false
if [ "$1" = "!" ] ; then
negating=true ; shift
fi
- local nodeglob="$1" ; shift
+ local node="$1" ; shift
local ips="$*"
local out
- all_ips_on_node 1
+ all_ips_on_node $node
+ local check
for check in $ips ; do
+ local ip pnn
while read ip pnn ; do
if [ "$check" = "$ip" ] ; then
- case "$pnn" in
- ($nodeglob) if $negating ; then return 1 ; fi ;;
- (*) if ! $negating ; then return 1 ; fi ;;
- esac
+ if [ "$pnn" = "$node" ] ; then
+ if $negating ; then return 1 ; fi
+ else
+ if ! $negating ; then return 1 ; fi
+ fi
ips="${ips/${ip}}" # Remove from list
break
fi
@@ -418,11 +442,27 @@ ips_are_on_nodeglob ()
[ -z "$ips" ]
}
-wait_until_ips_are_on_nodeglob ()
+wait_until_ips_are_on_node ()
{
- echo "Waiting for IPs to fail over..."
+ # Go to some trouble to print a use description of what is happening
+ local not=""
+ if [ "$1" == "!" ] ; then
+ not="no longer "
+ fi
+ local node=""
+ local ips=""
+ local i
+ for i ; do
+ [ "$i" != "!" ] || continue
+ if [ -z "$node" ] ; then
+ node="$i"
+ continue
+ fi
+ ips="${ips}${ips:+, }${i}"
+ done
+ echo "Waiting for ${ips} to ${not}be assigned to node ${node}"
- wait_until 60 ips_are_on_nodeglob "$@"
+ wait_until 60 ips_are_on_node "$@"
}
node_has_some_ips ()
@@ -431,7 +471,7 @@ node_has_some_ips ()
local out
- all_ips_on_node 1
+ all_ips_on_node $node
while read ip pnn ; do
if [ "$node" = "$pnn" ] ; then
@@ -444,7 +484,7 @@ node_has_some_ips ()
wait_until_node_has_some_ips ()
{
- echo "Waiting for node to have some IPs..."
+ echo "Waiting for some IPs to be assigned to node ${test_node}"
wait_until 60 node_has_some_ips "$@"
}
diff --git a/ctdb/tests/simple/16_ctdb_config_add_ip.sh
b/ctdb/tests/simple/16_ctdb_config_add_ip.sh
index dc28130..b5d76ea 100755
--- a/ctdb/tests/simple/16_ctdb_config_add_ip.sh
+++ b/ctdb/tests/simple/16_ctdb_config_add_ip.sh
@@ -5,32 +5,8 @@ test_info()
cat <<EOF
Verify that an IP address can be added to a node using 'ctdb addip'.
-This test goes to some trouble to figure out which IP address to add
-but assumes a 24-bit subnet mask. It does not handle IPv6. It does
-not do any network level checks that the new IP address is reachable
-but simply trusts 'ctdb ip' that the address has been added. There is
-also an extra prerequisite that the node being added to already has
-public addresses - this is difficult to avoid if the extra address is
-to be sensibly chosen.
-
-Prerequisites:
-
-* An active CTDB cluster with at least 2 active nodes.
-
-Steps:
-
-1. Verify that the status on all of the ctdb nodes is 'OK'.
-2. Use 'ctdb ip' on one of the nodes to list the IP addresses being
- served.
-3. Add an additional public address to be served by the node, using
- 'ctdb addip'.
-4. Verify that this IP address has been added to the list of IP
- addresses being served by the node, using the 'ctdb ip' command.
-
-Expected results:
-
-* 'ctdb ip' adds an IP address to the list of public IP addresses
- being served by a node.
+This test does not do any network level checks to make sure IP
+addresses are actually on interfaces. It just consults "ctdb ip".
EOF
}
@@ -45,79 +21,16 @@ cluster_is_healthy
# Reset configuration
ctdb_restart_when_done
-echo "Getting list of public IPs..."
-all_ips_on_node 0
-
-# When selecting test_node we just want a node that has public IPs.
-# This will work and is economically semi-randomly. :-)
-read x test_node <<<"$out"
-
-test_node_ips=""
-all_ips=""
-while read ip pnn ; do
- all_ips="${all_ips}${all_ips:+ }${ip}"
- [ "$pnn" = "$test_node" ] && \
- test_node_ips="${test_node_ips}${test_node_ips:+ }${ip}"
-done <<<"$out"
-
-echo "Selected node ${test_node} with IPs: $test_node_ips"
-
-# Try to find a free IP adddress. This is inefficient but should
-# succeed quickly.
-if [ -z "$TEST_LOCAL_DAEMONS" ] ; then
- try_command_on_node $test_node "ip addr show"
- all_test_node_ips=$(echo "$out" | sed -rn -e
's@^[[:space:]]+inet[[:space:]]+([[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+/[[:digit:]]+).*[[:space:]]([^[:space:]]+)+$@\1:\2@p')
-else
- all_test_node_ips=""
-fi
-add_ip=""
-
-# Use an IP already on one of the nodes, remove the last octet and
-# loop through the possible IP addreses.
-for i in $test_node_ips ; do
- prefix="${i%.*}"
- for j in $(seq 101 199) ; do
- try="${prefix}.${j}"
- # Try to make sure it isn't used anywhere!
-
- # First, make sure it isn't an existing public address on the
- # cluster.
- for k in $all_ips ; do
- [ "$try" = "$k" ] && continue 2
- done
-
- # Also make sure it isn't some other address in use on the
- # node.
- for k in $all_test_node_ips ; do
- [ "$try" = "${k%/*}" ] && continue 2
- done
-
- # Get the interface details for $i, which our address is a
- # close relative of. This should never fail but it can't hurt
- # to be careful...
- try_command_on_node $test_node "ctdb ip -v -Y"
- while IFS=":" read x ip pnn iface x ; do
- if [ "$i" = "$ip" ]; then
- add_ip="$try/32:$iface"
- break 3
- fi
- done <<<"$out"
- done
-done
+select_test_node_and_ips
+get_test_ip_mask_and_iface
-if [ -z "$add_ip" ] ; then
- echo "BAD: Unable to find IP address to add."
- exit 1
-fi
+echo "Deleting IP $test_ip from all nodes"
+try_command_on_node $test_node $CTDB delip -n all $test_ip
+wait_until_ips_are_on_node '!' $test_node $test_ip
-echo "Adding IP: ${add_ip/:/ on interface }"
-try_command_on_node $test_node $CTDB addip ${add_ip/:/ }
+# Debugging...
+try_command_on_node -v all $CTDB ip
-echo "Waiting for IP to be added..."
-if wait_until 60 ips_are_on_nodeglob $test_node ${add_ip%/*} ; then
- echo "That worked!"
-else
- echo "BAD: IP didn't get added."
- try_command_on_node $test_node $CTDB ip -n all
- exit 1
-fi
+echo "Adding IP ${test_ip}/${mask} on ${iface}, node ${test_node}"
+try_command_on_node $test_node $CTDB addip ${test_ip}/${mask} $iface
+wait_until_ips_are_on_node $test_node $test_ip
diff --git a/ctdb/tests/simple/17_ctdb_config_delete_ip.sh
b/ctdb/tests/simple/17_ctdb_config_delete_ip.sh
index 1ad9f33..80d2699 100755
--- a/ctdb/tests/simple/17_ctdb_config_delete_ip.sh
+++ b/ctdb/tests/simple/17_ctdb_config_delete_ip.sh
@@ -5,28 +5,8 @@ test_info()
cat <<EOF
Verify that a node's public IP address can be deleted using 'ctdb deleteip'.
-This test does not do any network level checks that the IP address is
-no longer reachable but simply trusts 'ctdb ip' that the address has
-been deleted.
-
-Prerequisites:
-
-* An active CTDB cluster with at least 2 active nodes.
-
-Steps:
-
-1. Verify that the status on all of the ctdb nodes is 'OK'.
-2. Use 'ctdb ip' on one of the nodes to list the IP addresses being
- served.
-3. Delete one public IP address being be served by the node, using
- 'ctdb delip'.
-4. Verify that the delete IP address is no longer listed using the
- all_ips_on_node helper function.
-
-Expected results:
-
-* 'ctdb delip' removes an IP address from the list of public IP
- addresses being served by a node.
+This test does not do any network level checks to make sure IP
+addresses are actually on interfaces. It just consults "ctdb ip".
EOF
}
@@ -41,35 +21,8 @@ cluster_is_healthy
# Reset configuration
ctdb_restart_when_done
-echo "Getting list of public IPs..."
-all_ips_on_node -v 0
-
-# Select an IP/node to remove.
-num_ips=$(echo "$out" | wc -l)
-num_to_remove=$(($RANDOM % $num_ips))
-
-# Find the details in the list.
-i=0
-while [ $i -le $num_to_remove ] ; do
- read ip_to_remove test_node
- i=$(($i + 1))
-done <<<"$out"
-
-echo "Attempting to remove ${ip_to_remove} from node ${test_node}."
-try_command_on_node $test_node $CTDB delip $ip_to_remove
-
-echo "Sleeping..."
-sleep_for 1
-
-test_node_ips=""
-while read ip pnn ; do
- [ "$pnn" = "$test_node" ] && \
- test_node_ips="${test_node_ips}${test_node_ips:+ }${ip}"
-done <<<"$out" # bashism to avoid problem setting variable in pipeline.
+select_test_node_and_ips
-if [ "${test_node_ips/${ip_to_remove}}" = "$test_node_ips" ] ; then
- echo "GOOD: That worked!"
-else
- echo "BAD: The remove IP address is still there!"
- testfailures=1
-fi
+echo "Deleting IP ${test_ip} from node ${test_node}"
+try_command_on_node $test_node $CTDB delip $test_ip
+wait_until_ips_are_on_node '!' $test_node $test_ip
diff --git a/ctdb/tests/simple/23_ctdb_moveip.sh
b/ctdb/tests/simple/23_ctdb_moveip.sh
index 7c09e58..f6e9027 100755
--- a/ctdb/tests/simple/23_ctdb_moveip.sh
+++ b/ctdb/tests/simple/23_ctdb_moveip.sh
@@ -5,27 +5,10 @@ test_info()
cat <<EOF
Verify that 'ctdb moveip' allows movement of public IPs between cluster nodes.
-To work, this test unsets DeterministicIPs and sets NoIPFailback.
-
-This test does not do any network level checks that the IP address is
-no longer reachable but simply trusts 'ctdb ip' that the address has
-been deleted.
-
-Prerequisites:
-
-* An active CTDB cluster with at least 2 active nodes.
-
-Steps:
-
-1. Verify that the status on all of the ctdb nodes is 'OK'.
-2. Use 'ctdb ip' on one of the nodes to list the IP addresses being
- served.
-3. Use 'ctdb moveip' to move an address from one node to another.
-4. Verify that the IP is no longer being hosted by the first node and is now
being hosted by the second node.
+This test does not do any network level checks to make sure IP
+addresses are actually on interfaces. It just consults "ctdb ip".
--
Samba Shared Repository
