The branch, master has been updated via 8d9a050 ctdb-tests: Add 31.clamd eventscript unit tests via 1eec073 ctdb-tests: Enhance ss stub to check for listening Unix domain sockets via 1aff2f8 ctdb-scripts: Switch ctdb_check_unix_socket() to use ss via ff635f9 ctdb-scripts: Clean up ctdb_check_unix_socket() via bff8d41 ctdb-daemon: Don't release all IPs before "startup" event via 6e58891 ctdb-recoverd: Abort recovery/takeover if recmaster changes via f57d379 ctdb-daemon: GET_DB_SEQNUM should read database conditionally via 5d2f267 ctdb-daemon: Add a function to check if db access is allowed via 96aef23 ctdb-tests: Fix ctdb test binary name in path testing via ff75f08 ctdb-tests: Wait up to 30 seconds for process to be registered in ctdbd from 3ff1b83 s3: vfs: catia: compression get/set must act only on base file, and must cope with fsp==NULL.
https://git.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 8d9a050a844822a0415cf4488c9ee687d613dd87 Author: Martin Schwenke <mar...@meltin.net> Date: Sat Sep 2 20:59:32 2017 +1000 ctdb-tests: Add 31.clamd eventscript unit tests These test that ctdb_check_unix_socket() is working. Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> Autobuild-User(master): Martin Schwenke <mart...@samba.org> Autobuild-Date(master): Tue Sep 12 16:14:12 CEST 2017 on sn-devel-144 commit 1eec073e8a997ff43b710189461ccef28872b039 Author: Martin Schwenke <mar...@meltin.net> Date: Sat Sep 2 20:57:56 2017 +1000 ctdb-tests: Enhance ss stub to check for listening Unix domain sockets Generalise command-line parsing, taking hints from old netstat stub, and use FAKE_NETSTAT_UNIX_LISTEN to specify listening Unix domain sockets. Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> commit 1aff2f87469e11b30a45d3040b251335285e7bb3 Author: Martin Schwenke <mar...@meltin.net> Date: Sat Mar 18 21:55:04 2017 +1100 ctdb-scripts: Switch ctdb_check_unix_socket() to use ss Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> commit ff635f90edd8e012ac2838303cb0285ab09b8db6 Author: Martin Schwenke <mar...@meltin.net> Date: Sat Mar 18 21:53:06 2017 +1100 ctdb-scripts: Clean up ctdb_check_unix_socket() Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> commit bff8d410f93a0e50d16aff4896bea2d48e46f33a Author: Martin Schwenke <mar...@meltin.net> Date: Mon Sep 5 13:38:18 2016 +1000 ctdb-daemon: Don't release all IPs before "startup" event This doesn't belong in the monitoring/startup code and it is already done in the 10.interface "init" event. Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> commit 6e588913dd55f4b45bc7b60b46eed505c11b529c Author: Amitay Isaacs <ami...@gmail.com> Date: Fri Sep 8 11:24:27 2017 +1000 ctdb-recoverd: Abort recovery/takeover if recmaster changes Recovery and takeover are run via helper from recovery daemon. While the helpers are running, it's possible for the current node to lose election. If that happens, abort the currently running recovery/takeover helper. Signed-off-by: Amitay Isaacs <ami...@gmail.com> Reviewed-by: Martin Schwenke <mar...@meltin.net> commit f57d379446c551bca5906247c622e857c77089b0 Author: Amitay Isaacs <ami...@gmail.com> Date: Thu Sep 7 17:21:03 2017 +1000 ctdb-daemon: GET_DB_SEQNUM should read database conditionally BUG: https://bugzilla.samba.org/show_bug.cgi?id=13021 Once the recovery starts and databases are frozen, then all the record access is postponed till the recovery is complete except reading the database sequence number. Database access for reading sequence number is done via a control which does not check if the databases are frozen or not. If the database is frozen and if the freeze transaction is not started (this can happen when a node is inactive, or during recovery when the database is frozen but the transaction has not yet started), then trying to read sequence number will cause ctdb daemon to deadlock. Before reading the sequence number, check if the database access is allowed. Signed-off-by: Amitay Isaacs <ami...@gmail.com> Reviewed-by: Martin Schwenke <mar...@meltin.net> commit 5d2f2677de65a0fd6683bb759d80ebced604fa6b Author: Amitay Isaacs <ami...@gmail.com> Date: Thu Sep 7 17:18:18 2017 +1000 ctdb-daemon: Add a function to check if db access is allowed BUG: https://bugzilla.samba.org/show_bug.cgi?id=13021 Signed-off-by: Amitay Isaacs <ami...@gmail.com> Reviewed-by: Martin Schwenke <mar...@meltin.net> commit 96aef2371c6c1e0c6bd13874a71583eb9609959b Author: Amitay Isaacs <ami...@gmail.com> Date: Tue Sep 5 13:52:47 2017 +1000 ctdb-tests: Fix ctdb test binary name in path testing BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012 Signed-off-by: Amitay Isaacs <ami...@gmail.com> Reviewed-by: Martin Schwenke <mar...@meltin.net> commit ff75f0836aef56476ec45a3bc8f3ca22c118e3a4 Author: Martin Schwenke <mar...@meltin.net> Date: Tue Sep 12 11:51:19 2017 +1000 ctdb-tests: Wait up to 30 seconds for process to be registered in ctdbd BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012 This avoids a potential race where the client is not properly registered before "ctdb process-exists" is called. Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> ----------------------------------------------------------------------- Summary of changes: ctdb/config/functions | 22 +-- ctdb/include/ctdb_private.h | 1 + ctdb/server/ctdb_freeze.c | 18 +++ ctdb/server/ctdb_monitor.c | 3 - ctdb/server/ctdb_persistent.c | 5 + ctdb/server/ctdb_recoverd.c | 9 ++ ctdb/tests/eventscripts/31.clamd.monitor.001.sh | 13 ++ ctdb/tests/eventscripts/31.clamd.monitor.002.sh | 15 ++ ctdb/tests/eventscripts/31.clamd.monitor.003.sh | 15 ++ ctdb/tests/eventscripts/scripts/local.sh | 7 + ctdb/tests/eventscripts/stubs/ss | 177 +++++++++++++++++++----- ctdb/tests/scripts/test_wrap | 2 +- ctdb/tests/simple/07_ctdb_process_exists.sh | 4 +- 13 files changed, 241 insertions(+), 50 deletions(-) create mode 100755 ctdb/tests/eventscripts/31.clamd.monitor.001.sh create mode 100755 ctdb/tests/eventscripts/31.clamd.monitor.002.sh create mode 100755 ctdb/tests/eventscripts/31.clamd.monitor.003.sh Changeset truncated at 500 lines: diff --git a/ctdb/config/functions b/ctdb/config/functions index 1131229..3805d48 100755 --- a/ctdb/config/functions +++ b/ctdb/config/functions @@ -376,16 +376,22 @@ ctdb_check_tcp_ports() ###################################################### # check a unix socket -# usage: ctdb_check_unix_socket SERVICE_NAME <socket_path> +# usage: ctdb_check_unix_socket SOCKPATH ###################################################### -ctdb_check_unix_socket() { - socket_path="$1" - [ -z "$socket_path" ] && return +ctdb_check_unix_socket() +{ + _sockpath="$1" - if ! netstat --unix -a -n | grep -q "^unix.*LISTEN.*${socket_path}$"; then - echo "ERROR: $service_name socket $socket_path not found" - return 1 - fi + if [ -z "$_sockpath" ] ; then + echo "ERROR: ctdb_check_unix_socket() requires socket path" + return 1 + fi + + _out=$(ss -l -x "src ${_sockpath}" | tail -n +2) + if [ -z "$_out" ] ; then + echo "ERROR: ${service_name} not listening on ${_sockpath}" + return 1 + fi } ################################################ diff --git a/ctdb/include/ctdb_private.h b/ctdb/include/ctdb_private.h index 5b95b60..f5966cd 100644 --- a/ctdb/include/ctdb_private.h +++ b/ctdb/include/ctdb_private.h @@ -627,6 +627,7 @@ int32_t ctdb_control_wipe_database(struct ctdb_context *ctdb, TDB_DATA indata); bool ctdb_db_frozen(struct ctdb_db_context *ctdb_db); bool ctdb_db_all_frozen(struct ctdb_context *ctdb); +bool ctdb_db_allow_access(struct ctdb_db_context *ctdb_db); /* from server/ctdb_keepalive.c */ diff --git a/ctdb/server/ctdb_freeze.c b/ctdb/server/ctdb_freeze.c index d92f707..c41fc7d 100644 --- a/ctdb/server/ctdb_freeze.c +++ b/ctdb/server/ctdb_freeze.c @@ -874,3 +874,21 @@ bool ctdb_db_all_frozen(struct ctdb_context *ctdb) } return true; } + +bool ctdb_db_allow_access(struct ctdb_db_context *ctdb_db) +{ + if (ctdb_db->freeze_mode == CTDB_FREEZE_NONE) { + /* If database is not frozen, then allow access. */ + return true; + } else if (ctdb_db->freeze_transaction_started) { + /* If database is frozen, allow access only if the + * transaction is started. This is required during + * recovery. + * + * If a node is inactive, then transaction is not started. + */ + return true; + } + + return false; +} diff --git a/ctdb/server/ctdb_monitor.c b/ctdb/server/ctdb_monitor.c index 738acb1..223220c 100644 --- a/ctdb/server/ctdb_monitor.c +++ b/ctdb/server/ctdb_monitor.c @@ -270,9 +270,6 @@ static void ctdb_run_startup(struct tevent_context *ev, return; } - /* release any IPs we hold from previous runs of the daemon */ - ctdb_release_all_ips(ctdb); - DEBUG(DEBUG_NOTICE,("Running the \"startup\" event.\n")); ret = ctdb_event_script_callback(ctdb, ctdb->monitor->monitor_context, diff --git a/ctdb/server/ctdb_persistent.c b/ctdb/server/ctdb_persistent.c index 1811ae8..fc28655 100644 --- a/ctdb/server/ctdb_persistent.c +++ b/ctdb/server/ctdb_persistent.c @@ -344,6 +344,11 @@ static int32_t ctdb_get_db_seqnum(struct ctdb_context *ctdb, goto done; } + if (! ctdb_db_allow_access(ctdb_db)) { + ret = -1; + goto done; + } + key.dptr = (uint8_t *)discard_const(keyname); key.dsize = strlen(keyname) + 1; diff --git a/ctdb/server/ctdb_recoverd.c b/ctdb/server/ctdb_recoverd.c index 9488bc2..2b94fed 100644 --- a/ctdb/server/ctdb_recoverd.c +++ b/ctdb/server/ctdb_recoverd.c @@ -1026,6 +1026,7 @@ static int helper_run(struct ctdb_recoverd *rec, TALLOC_CTX *mem_ctx, struct tevent_fd *fde; const char **args; int nargs, ret; + uint32_t recmaster = rec->recmaster; state = talloc_zero(mem_ctx, struct helper_state); if (state == NULL) { @@ -1085,6 +1086,14 @@ static int helper_run(struct ctdb_recoverd *rec, TALLOC_CTX *mem_ctx, while (!state->done) { tevent_loop_once(rec->ctdb->ev); + + /* If recmaster changes, we have lost election */ + if (recmaster != rec->recmaster) { + D_ERR("Recmaster changed to %u, aborting %s\n", + rec->recmaster, type); + state->result = 1; + break; + } } close(state->fd[0]); diff --git a/ctdb/tests/eventscripts/31.clamd.monitor.001.sh b/ctdb/tests/eventscripts/31.clamd.monitor.001.sh new file mode 100755 index 0000000..bd97057 --- /dev/null +++ b/ctdb/tests/eventscripts/31.clamd.monitor.001.sh @@ -0,0 +1,13 @@ +#!/bin/sh + +. "${TEST_SCRIPTS_DIR}/unit.sh" + +define_test "Not managed, clamd not listening" + +export CTDB_MANAGES_CLAMD=no +export CTDB_CLAMD_SOCKET="/var/run/clamd.sock" + +setup_generic + +ok_null +simple_test diff --git a/ctdb/tests/eventscripts/31.clamd.monitor.002.sh b/ctdb/tests/eventscripts/31.clamd.monitor.002.sh new file mode 100755 index 0000000..bfd7634 --- /dev/null +++ b/ctdb/tests/eventscripts/31.clamd.monitor.002.sh @@ -0,0 +1,15 @@ +#!/bin/sh + +. "${TEST_SCRIPTS_DIR}/unit.sh" + +define_test "Managed, clamd not listening" + +export CTDB_MANAGES_CLAMD=yes +export CTDB_CLAMD_SOCKET="/var/run/clamd.sock" + +setup_generic + +required_result 1 <<EOF +ERROR: clamd not listening on $CTDB_CLAMD_SOCKET +EOF +simple_test diff --git a/ctdb/tests/eventscripts/31.clamd.monitor.003.sh b/ctdb/tests/eventscripts/31.clamd.monitor.003.sh new file mode 100755 index 0000000..e57f025 --- /dev/null +++ b/ctdb/tests/eventscripts/31.clamd.monitor.003.sh @@ -0,0 +1,15 @@ +#!/bin/sh + +. "${TEST_SCRIPTS_DIR}/unit.sh" + +define_test "Managed, clamd listening" + +export CTDB_MANAGES_CLAMD=yes +export CTDB_CLAMD_SOCKET="/var/run/clamd.sock" + +setup_generic + +unix_socket_listening "$CTDB_CLAMD_SOCKET" + +ok_null +simple_test diff --git a/ctdb/tests/eventscripts/scripts/local.sh b/ctdb/tests/eventscripts/scripts/local.sh index 1d51232..83877c3 100644 --- a/ctdb/tests/eventscripts/scripts/local.sh +++ b/ctdb/tests/eventscripts/scripts/local.sh @@ -135,6 +135,13 @@ tcp_port_down () done } +unix_socket_listening () +{ + _s="$1" + + FAKE_NETSTAT_UNIX_LISTEN="${FAKE_NETSTAT_UNIX_LISTEN} ${_s}" +} + _tcp_connections () { _count="$1" diff --git a/ctdb/tests/eventscripts/stubs/ss b/ctdb/tests/eventscripts/stubs/ss index 1a3db0d..bb291b2 100755 --- a/ctdb/tests/eventscripts/stubs/ss +++ b/ctdb/tests/eventscripts/stubs/ss @@ -4,23 +4,73 @@ prog="ss" usage () { - cat >&2 <<EOF -Usage: $prog -tn state established [ '(' ip-filter ')' ] [ '(' port-filter ')' ] + cat >&2 <<EOF +Usage: $prog { -t|--tcp | -x|--unix } [options] [ FILTER ] A fake ss stub that prints items depending on the variables -FAKE_NETSTAT_TCP_ESTABLISHED and FAKE_NETSTAT_TCP_ESTABLISHED_FILE. +FAKE_NETSTAT_TCP_ESTABLISHED, FAKE_TCP_LISTEN, +FAKE_NETSTAT_UNIX_LISTEN, depending on command-line options. -Note that "-tn state established" must be given. +Note that -n is ignored. EOF - exit 1 + exit 1 } -if [ "$1" != "-tn" -o "$2" != "state" -o "$3" != "established" ] ; then - usage -fi +not_supported () +{ + echo "Options not supported in stub: $*" >&2 + usage +} -shift 3 +############################################################ + +# +parse_filter () +{ + # Very limited implementation: + # We only expect to find || inside parentheses + # We don't expect to see && - it is implied by juxtaposition + # Operator for port comparison is ignored and assumed to be == + + # Build lists of source ports and source IP addresses where + # each entry is surrounded by '|' characters. These lists can + # be easily "searched" using the POSIX prefix and suffix + # removal operators. + in_parens=false + sports="|" + srcs="|" + + while [ -n "$1" ] ; do + case "$1" in + \() + in_parens=true + shift + ;; + \)) + in_parens=false + shift + ;; + \|\|) + if ! $in_parens ; then + not_supported "|| in parentheses" + fi + shift + ;; + sport) + p="${3#:}" ; sports="${sports}${p}|" + shift 3 + ;; + src) + ip="${2#\[}" ; ip="${ip%\]}" ; srcs="${srcs}${ip}|" + shift 2 + ;; + *) + usage + ;; + esac + done +} # Check if socket has matches in both ok_ips and ok_ports filter_socket () @@ -44,31 +94,11 @@ filter_socket () ss_tcp_established () { - echo "Recv-Q Send-Q Local Address:Port Peer Address:Port" - - # Very limited implementation: - # We only expect to find || inside parentheses - # We don't expect to see && - it is implied by juxtaposition - # Operator for port comparison is ignored and assumed to be == - - # Build lists of source ports and source IP addresses where each - # entry is surrounded by '|' characters. These lists can be - # easily "searched" using the POSIX prefix and suffix removal - # operators. - in_parens=false - sports="|" - srcs="|" - - while [ -n "$1" ] ; do - case "$1" in - \() in_parens=true ; shift ;; - \)) in_parens=false ; shift ;; - \|\|) if ! $in_parens ; then usage ; fi ; shift ;; - sport) p="${3#:}" ; sports="${sports}${p}|" ; shift 3 ;; - src) ip="${2#\[}" ; ip="${ip%\]}" ; srcs="${srcs}${ip}|" ; shift 2 ;; - *) usage ;; - esac - done + if $header ; then + echo "Recv-Q Send-Q Local Address:Port Peer Address:Port" + fi + + parse_filter $* for i in $FAKE_NETSTAT_TCP_ESTABLISHED ; do src="${i%|*}" @@ -88,5 +118,80 @@ ss_tcp_established () done <"$FAKE_NETSTAT_TCP_ESTABLISHED_FILE" } -# Yes, lose the quoting so we can do a hacky parsing job -ss_tcp_established $* +############################################################ + +unix_listen () +{ + if $header ; then + cat <<EOF +Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port" +EOF + fi + + parse_filter $* + + _n=12345 + for _s in $FAKE_NETSTAT_UNIX_LISTEN ; do + # ss matches Unix domain sockets as either src or + # sport. + if filter_socket "$srcs" "$sports" "${_s}:" || \ + filter_socket "$srcs" "$sports" ":${_s}" ; then + printf "u_str LISTEN 0 128 %s %d * 0\n" "$_s" "$_n" + _n=$((_n + 1)) + fi + done +} + +############################################################ + +# Defaults. +tcp=false +unix=false +all=false +listen=false +header=true + +orig="$*" +temp=$(getopt -n "$prog" -o "txnalHh" -l tcp -l unix -l help -- "$@") +[ $? -eq 0 ] || usage + +eval set -- "$temp" + +while true ; do + case "$1" in + --tcp|-t) tcp=true ; shift ;; + --unix|-x) unix=true ; shift ;; + -l) listen=true ; shift ;; + -a) all=true ; shift ;; + -H) header=false ; shift ;; + -n) shift ;; + --) shift ; break ;; + -h|--help|*) usage ;; + esac +done + +$tcp || $unix || not_supported "$*" + +if $tcp ; then + if [ "$1" != "state" -o "$2" != "established" ] || $listen ; then + usage + fi + + shift 2 + + # Yes, lose the quoting so we can do a hacky parsing job + ss_tcp_established $* + + exit +fi + +if $unix ; then + if ! $listen ; then + not_supported "$orig" + fi + + # Yes, lose the quoting so we can do a hacky parsing job + unix_listen $* + + exit +fi diff --git a/ctdb/tests/scripts/test_wrap b/ctdb/tests/scripts/test_wrap index 176310e..3db3180 100755 --- a/ctdb/tests/scripts/test_wrap +++ b/ctdb/tests/scripts/test_wrap @@ -10,7 +10,7 @@ TEST_SCRIPTS_DIR=$(dirname $0) # We need the test binaries (i.e. tests/bin/) to be in $PATH. If they # aren't already in $PATH then we know that tests/bin/ sits alongside # tests/scripts/. -f="ctdb_bench" +f="fetch_ring" if [ ! $(which $f >/dev/null 2>&1) ] ; then d=$(dirname "$TEST_SCRIPTS_DIR")/bin [ -x "$d/$f" ] && PATH="$d:$PATH" diff --git a/ctdb/tests/simple/07_ctdb_process_exists.sh b/ctdb/tests/simple/07_ctdb_process_exists.sh index c44924b..f24e93a 100755 --- a/ctdb/tests/simple/07_ctdb_process_exists.sh +++ b/ctdb/tests/simple/07_ctdb_process_exists.sh @@ -52,9 +52,9 @@ cleanup () ctdb_test_exit_hook_add cleanup -echo "Checking for PID $client_pid on node $test_node" +echo "Waiting until PID $client_pid is registered on node $test_node" status=0 -try_command_on_node $test_node \ +wait_until 30 try_command_on_node $test_node \ "$CTDB process-exists ${client_pid}" || status=$? echo "$out" -- Samba Shared Repository