The branch, v4-5-test has been updated via b8ae9cd vfs_default: unlock the right file in copy chunk via 4426d43 ctdb-tests: Add "13.per_ip_routing shutdown" test via 88ef026 ctdb-scripts: Fix regression when cleaning up routing table IDs via 5650e20 ctdb-scripts: Fix remaining uses of "ctdb gratiousarp" via 3d55b17 ctdb-tests: Add robust mutex test via 8fe2fc0 ctdb-locking: Explicitly unlock record/db in lock helper via 727f5d4 ctdb-locking: Remove support for locking multiple databases from 430cad1 VERSION: Bump version up to 4.5.5...
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-5-test - Log ----------------------------------------------------------------- commit b8ae9cd9b5401c4ca973ded5453801d438e6ff64 Author: Björn Jacke <b...@sernet.de> Date: Thu Jan 19 21:51:41 2017 +0100 vfs_default: unlock the right file in copy chunk Signed-off-by: Bjoern Jacke <b...@sernet.de> Reviewed-by: David Disseldorp <dd...@samba.org> Autobuild-User(master): Björn Jacke <b...@sernet.de> Autobuild-Date(master): Sat Jan 21 17:00:54 CET 2017 on sn-devel-144 BUG: https://bugzilla.samba.org/show_bug.cgi?id=12535 (cherry picked from commit 5059c8e2e3a6159bc2917ddd80d09fab35b39e66) Autobuild-User(v4-5-test): Karolin Seeger <ksee...@samba.org> Autobuild-Date(v4-5-test): Wed Jan 25 23:47:29 CET 2017 on sn-devel-144 commit 4426d4343710dded806e02b3d763614f3393dfb3 Author: Martin Schwenke <mar...@meltin.net> Date: Mon Jan 16 11:08:51 2017 +1100 ctdb-tests: Add "13.per_ip_routing shutdown" test Ensure that it doesn't mangle the rt_tables file. BUG: https://bugzilla.samba.org/show_bug.cgi?id=12516 Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> Autobuild-User(master): Amitay Isaacs <ami...@samba.org> Autobuild-Date(master): Tue Jan 17 06:02:23 CET 2017 on sn-devel-144 (cherry picked from commit eaa508b82650197a7d473a24b3362e9e9c329937) commit 88ef026ec67c06e2c0a36aaa128c7be54f0cc5f2 Author: Martin Schwenke <mar...@meltin.net> Date: Mon Jan 16 07:24:15 2017 +1100 ctdb-scripts: Fix regression when cleaning up routing table IDs Commit 0ca00267cd2620a14968961738bcd2a69b597e95 removed explicit continuations in strings for awk programs. In one case this causes a disconnect between condition and action, where an implicit continuation does not work. This results in duplicate lines in the rt_tables file. Move the opening brace for the action to make the implicit continuation work as expected. An alternative would be to revert the removal of the explicit continuations and add shellcheck tags. However, that doesn't mean that an author of future code will necessarily use explicit continuations, so the same mistake might still be make in the future. BUG: https://bugzilla.samba.org/show_bug.cgi?id=12516 Reported-by: Barry Evans <bev...@pixitmedia.com> Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> (cherry picked from commit f9368f8e129cb32ee30cb6501a6fe728db37e1d5) commit 5650e208de9177ac166e56ff2b7767baad8e014d Author: Martin Schwenke <mar...@meltin.net> Date: Mon Jan 16 13:38:50 2017 +1100 ctdb-scripts: Fix remaining uses of "ctdb gratiousarp" This changed to "ctdb gratarp" some time ago but the scripts were never updated. Fix the documentation for the ctdb tool too. BUG: https://bugzilla.samba.org/show_bug.cgi?id=12512 Reported-by: Ralph Böhme <s...@samba.org> Signed-off-by: Martin Schwenke <mar...@meltin.net> Reviewed-by: Amitay Isaacs <ami...@gmail.com> (cherry picked from commit 5e00a6b346325f52e35b9785eaffd72239aebcf5) commit 3d55b17d28724cb5ddb5a6fb7b6a7b5cd28bdc65 Author: Amitay Isaacs <ami...@gmail.com> Date: Fri Dec 2 15:11:20 2016 +1100 ctdb-tests: Add robust mutex test BUG: https://bugzilla.samba.org/show_bug.cgi?id=12469 This demonstrates robust mutex bug on linux/glibc system. Signed-off-by: Amitay Isaacs <ami...@gmail.com> Reviewed-by: Martin Schwenke <mar...@meltin.net> Autobuild-User(master): Amitay Isaacs <ami...@samba.org> Autobuild-Date(master): Thu Jan 12 07:59:34 CET 2017 on sn-devel-144 (cherry picked from commit 7794497bc909fa7b02da9d9ce1fc496a8fa2a9ae) commit 8fe2fc0b9e6be7166062f6802e0af1740fbb2f5a Author: Amitay Isaacs <ami...@gmail.com> Date: Tue Nov 29 17:20:45 2016 +1100 ctdb-locking: Explicitly unlock record/db in lock helper BUG: https://bugzilla.samba.org/show_bug.cgi?id=12469 Instead of killing lock helper processes with SIGKILL, send SIGTERM so lock helper processes can explicitly unlock record/db. Signed-off-by: Amitay Isaacs <ami...@gmail.com> Reviewed-by: Martin Schwenke <mar...@meltin.net> (cherry picked from commit 3a56a16b06cf6d1cce613ec29f5ea46630902072) commit 727f5d48512fe91d59fbdf7d0caa96d0f156e003 Author: Amitay Isaacs <ami...@gmail.com> Date: Tue Nov 29 17:13:41 2016 +1100 ctdb-locking: Remove support for locking multiple databases BUG: https://bugzilla.samba.org/show_bug.cgi?id=12469 The code to lock multiple databases has been dropped from ctdb_lock.c. Signed-off-by: Amitay Isaacs <ami...@gmail.com> Reviewed-by: Martin Schwenke <mar...@meltin.net> (cherry picked from commit 5b1076dc61f5e3f006c1b8cef98e7d2d3cc1bfba) ----------------------------------------------------------------------- Summary of changes: ctdb/config/events.d/10.interface | 4 +- ctdb/config/events.d/13.per_ip_routing | 8 +- ctdb/config/events.d/91.lvs | 2 +- ctdb/doc/ctdb.1.xml | 4 +- ctdb/server/ctdb_lock.c | 6 +- ctdb/server/ctdb_lock_helper.c | 201 ++++++++++++++--- ctdb/tests/eventscripts/13.per_ip_routing.024.sh | 31 +++ ctdb/tests/eventscripts/stubs/ctdb | 2 +- ctdb/tests/src/test_mutex_raw.c | 261 +++++++++++++++++++++++ ctdb/wscript | 8 +- source3/modules/vfs_default.c | 2 +- 11 files changed, 482 insertions(+), 47 deletions(-) create mode 100755 ctdb/tests/eventscripts/13.per_ip_routing.024.sh create mode 100644 ctdb/tests/src/test_mutex_raw.c Changeset truncated at 500 lines: diff --git a/ctdb/config/events.d/10.interface b/ctdb/config/events.d/10.interface index 51abc44..e41596d 100755 --- a/ctdb/config/events.d/10.interface +++ b/ctdb/config/events.d/10.interface @@ -215,7 +215,7 @@ updateip) # 2) remove the IP from the old interface (and new interface, to be sure) # 3) add the IP to the new interface # 4) remove the firewall rule - # 5) use ctdb gratiousarp to propagate the new mac address + # 5) use ctdb gratarp to propagate the new mac address # 6) use netstat -tn to find existing connections, and tickle them _oiface=$2 niface=$3 @@ -240,7 +240,7 @@ updateip) flush_route_cache # propagate the new mac address - $CTDB gratiousarp "$ip" "$niface" + $CTDB gratarp "$ip" "$niface" # tickle all existing connections, so that dropped packets # are retransmited and the tcp streams work diff --git a/ctdb/config/events.d/13.per_ip_routing b/ctdb/config/events.d/13.per_ip_routing index f0e4609..41724bb 100755 --- a/ctdb/config/events.d/13.per_ip_routing +++ b/ctdb/config/events.d/13.per_ip_routing @@ -184,8 +184,8 @@ clean_up_table_ids () -v pre="$table_id_prefix" \ '/^#/ || !(min <= $1 && $1 <= max) && - !(index($2, pre) == 1) - { print $0 }' "$rt_tables" >"$_tmp" + !(index($2, pre) == 1) { + print $0 }' "$rt_tables" >"$_tmp" mv "$_tmp" "$rt_tables" ) 9>"$rt_tables_lock" @@ -398,7 +398,7 @@ takeip) # flush our route cache set_proc sys/net/ipv4/route/flush 1 - $CTDB gratiousarp "$ip" "$iface" + $CTDB gratarp "$ip" "$iface" ;; updateip) @@ -416,7 +416,7 @@ updateip) # flush our route cache set_proc sys/net/ipv4/route/flush 1 - $CTDB gratiousarp "$ip" "$niface" + $CTDB gratarp "$ip" "$niface" tickle_tcp_connections "$ip" ;; diff --git a/ctdb/config/events.d/91.lvs b/ctdb/config/events.d/91.lvs index 052b509..9725ee8 100755 --- a/ctdb/config/events.d/91.lvs +++ b/ctdb/config/events.d/91.lvs @@ -110,7 +110,7 @@ ipreallocated) ipvsadm -a -t "$CTDB_LVS_PUBLIC_IP" -r 127.0.0.1 ipvsadm -a -u "$CTDB_LVS_PUBLIC_IP" -r 127.0.0.1 - $CTDB gratiousarp \ + $CTDB gratarp \ "$CTDB_LVS_PUBLIC_IP" "$CTDB_LVS_PUBLIC_IFACE" >/dev/null 2>&1 flush_route_cache diff --git a/ctdb/doc/ctdb.1.xml b/ctdb/doc/ctdb.1.xml index d295213..71af0a5 100644 --- a/ctdb/doc/ctdb.1.xml +++ b/ctdb/doc/ctdb.1.xml @@ -1353,9 +1353,9 @@ dbid:0xb775fff6 name:secrets.tdb path:/usr/local/var/lib/ctdb/persistent/secrets </refsect2> <refsect2> - <title>gratiousarp <parameter>IPADDR</parameter> <parameter>INTERFACE</parameter></title> + <title>gratarp <parameter>IPADDR</parameter> <parameter>INTERFACE</parameter></title> <para> - Send out a gratious ARP for the specified interface through + Send out a gratuitous ARP for the specified interface through the specified interface. This command is mainly used by the ctdb eventscripts. </para> diff --git a/ctdb/server/ctdb_lock.c b/ctdb/server/ctdb_lock.c index 405a29e..5ecf454e 100644 --- a/ctdb/server/ctdb_lock.c +++ b/ctdb/server/ctdb_lock.c @@ -192,7 +192,7 @@ static int ctdb_lock_context_destructor(struct lock_context *lock_ctx) lock_ctx->request->lctx = NULL; } if (lock_ctx->child > 0) { - ctdb_kill(lock_ctx->ctdb, lock_ctx->child, SIGKILL); + ctdb_kill(lock_ctx->ctdb, lock_ctx->child, SIGTERM); if (lock_ctx->type == LOCK_RECORD) { DLIST_REMOVE(lock_ctx->ctdb_db->lock_current, lock_ctx); } else { @@ -672,7 +672,7 @@ static void ctdb_lock_schedule(struct ctdb_context *ctdb) ctdb_lock_timeout_handler, (void *)lock_ctx); if (lock_ctx->ttimer == NULL) { - ctdb_kill(ctdb, lock_ctx->child, SIGKILL); + ctdb_kill(ctdb, lock_ctx->child, SIGTERM); lock_ctx->child = -1; close(lock_ctx->fd[0]); return; @@ -687,7 +687,7 @@ static void ctdb_lock_schedule(struct ctdb_context *ctdb) (void *)lock_ctx); if (lock_ctx->tfd == NULL) { TALLOC_FREE(lock_ctx->ttimer); - ctdb_kill(ctdb, lock_ctx->child, SIGKILL); + ctdb_kill(ctdb, lock_ctx->child, SIGTERM); lock_ctx->child = -1; close(lock_ctx->fd[0]); return; diff --git a/ctdb/server/ctdb_lock_helper.c b/ctdb/server/ctdb_lock_helper.c index 8aa0870..c4f628f 100644 --- a/ctdb/server/ctdb_lock_helper.c +++ b/ctdb/server/ctdb_lock_helper.c @@ -20,8 +20,12 @@ #include "replace.h" #include "system/filesys.h" #include "system/network.h" +#include "system/wait.h" #include <talloc.h> +#include <tevent.h> + +#include "lib/util/tevent_unix.h" #include "ctdb_private.h" @@ -30,6 +34,11 @@ static char *progname = NULL; static bool realtime = true; +struct lock_state { + struct tdb_context *tdb; + TDB_DATA key; +}; + static void set_priority(void) { const char *ptr; @@ -72,7 +81,7 @@ static void usage(void) fprintf(stderr, "\n"); fprintf(stderr, "Usage: %s <log-fd> <ctdbd-pid> <output-fd> RECORD <db-path> <db-flags> <db-key>\n", progname); - fprintf(stderr, " %s <log-fd> <ctdbd-pid> <output-fd> DB <db1-path> <db1-flags> [<db2-path> <db2-flags>...]\n", + fprintf(stderr, " %s <log-fd> <ctdbd-pid> <output-fd> DB <db1-path> <db1-flags>\n", progname); } @@ -93,10 +102,9 @@ static uint8_t *hex_decode_talloc(TALLOC_CTX *mem_ctx, return buffer; } -static int lock_record(const char *dbpath, const char *dbflags, const char *dbkey) +static int lock_record(const char *dbpath, const char *dbflags, + const char *dbkey, struct lock_state *state) { - TDB_DATA key; - struct tdb_context *tdb; int tdb_flags; /* No error checking since CTDB always passes sane values */ @@ -104,23 +112,25 @@ static int lock_record(const char *dbpath, const char *dbflags, const char *dbke /* Convert hex key to key */ if (strcmp(dbkey, "NULL") == 0) { - key.dptr = NULL; - key.dsize = 0; + state->key.dptr = NULL; + state->key.dsize = 0; } else { - key.dptr = hex_decode_talloc(NULL, dbkey, &key.dsize); + state->key.dptr = hex_decode_talloc(NULL, dbkey, + &state->key.dsize); } - tdb = tdb_open(dbpath, 0, tdb_flags, O_RDWR, 0600); - if (tdb == NULL) { - fprintf(stderr, "%s: Error opening database %s\n", progname, dbpath); + state->tdb = tdb_open(dbpath, 0, tdb_flags, O_RDWR, 0600); + if (state->tdb == NULL) { + fprintf(stderr, "%s: Error opening database %s\n", + progname, dbpath); return 1; } set_priority(); - if (tdb_chainlock(tdb, key) < 0) { + if (tdb_chainlock(state->tdb, state->key) < 0) { fprintf(stderr, "%s: Error getting record lock (%s)\n", - progname, tdb_errorstr(tdb)); + progname, tdb_errorstr(state->tdb)); return 1; } @@ -130,26 +140,26 @@ static int lock_record(const char *dbpath, const char *dbflags, const char *dbke } - -static int lock_db(const char *dbpath, const char *dbflags) +static int lock_db(const char *dbpath, const char *dbflags, + struct lock_state *state) { - struct tdb_context *tdb; int tdb_flags; /* No error checking since CTDB always passes sane values */ tdb_flags = strtol(dbflags, NULL, 0); - tdb = tdb_open(dbpath, 0, tdb_flags, O_RDWR, 0600); - if (tdb == NULL) { - fprintf(stderr, "%s: Error opening database %s\n", progname, dbpath); + state->tdb = tdb_open(dbpath, 0, tdb_flags, O_RDWR, 0600); + if (state->tdb == NULL) { + fprintf(stderr, "%s: Error opening database %s\n", + progname, dbpath); return 1; } set_priority(); - if (tdb_lockall(tdb) < 0) { + if (tdb_lockall(state->tdb) < 0) { fprintf(stderr, "%s: Error getting db lock (%s)\n", - progname, tdb_errorstr(tdb)); + progname, tdb_errorstr(state->tdb)); return 1; } @@ -158,13 +168,114 @@ static int lock_db(const char *dbpath, const char *dbflags) return 0; } +struct wait_for_parent_state { + struct tevent_context *ev; + pid_t ppid; +}; + +static void wait_for_parent_check(struct tevent_req *subreq); + +static struct tevent_req *wait_for_parent_send(TALLOC_CTX *mem_ctx, + struct tevent_context *ev, + pid_t ppid) +{ + struct tevent_req *req, *subreq; + struct wait_for_parent_state *state; + + req = tevent_req_create(mem_ctx, &state, struct wait_for_parent_state); + if (req == NULL) { + return NULL; + } + + state->ev = ev; + state->ppid = ppid; + + if (ppid == 1) { + tevent_req_done(req); + return tevent_req_post(req, ev); + } + + subreq = tevent_wakeup_send(state, ev, + tevent_timeval_current_ofs(5,0)); + if (tevent_req_nomem(subreq, req)) { + return tevent_req_post(req, ev); + } + tevent_req_set_callback(subreq, wait_for_parent_check, req); + + return req; +} + +static void wait_for_parent_check(struct tevent_req *subreq) +{ + struct tevent_req *req = tevent_req_callback_data( + subreq, struct tevent_req); + struct wait_for_parent_state *state = tevent_req_data( + req, struct wait_for_parent_state); + bool status; + + status = tevent_wakeup_recv(subreq); + TALLOC_FREE(subreq); + if (! status) { + /* Ignore error */ + fprintf(stderr, "locking: tevent_wakeup_recv() failed\n"); + } + + if (kill(state->ppid, 0) == -1 && errno == ESRCH) { + tevent_req_done(req); + return; + } + + subreq = tevent_wakeup_send(state, state->ev, + tevent_timeval_current_ofs(5,0)); + if (tevent_req_nomem(subreq, req)) { + return; + } + tevent_req_set_callback(subreq, wait_for_parent_check, req); +} + +static bool wait_for_parent_recv(struct tevent_req *req) +{ + if (tevent_req_is_unix_error(req, NULL)) { + return false; + } + + return true; +} + +static void cleanup(struct lock_state *state) +{ + if (state->tdb != NULL) { + if (state->key.dsize == 0) { + tdb_unlockall(state->tdb); + } else { + tdb_chainunlock(state->tdb, state->key); + } + tdb_close(state->tdb); + } +} + +static void signal_handler(struct tevent_context *ev, + struct tevent_signal *se, + int signum, int count, void *siginfo, + void *private_data) +{ + struct lock_state *state = (struct lock_state *)private_data; + + cleanup(state); + exit(0); +} int main(int argc, char *argv[]) { + struct tevent_context *ev; + struct tevent_signal *se; + struct tevent_req *req; + struct lock_state state = { 0 }; int write_fd, log_fd; char result = 0; int ppid; const char *lock_type; + bool status; reset_scheduler(); @@ -186,6 +297,20 @@ int main(int argc, char *argv[]) write_fd = atoi(argv[3]); lock_type = argv[4]; + ev = tevent_context_init(NULL); + if (ev == NULL) { + fprintf(stderr, "locking: tevent_context_init() failed\n"); + exit(1); + } + + se = tevent_add_signal(ev, ev, SIGTERM, 0, + signal_handler, &state); + if (se == NULL) { + fprintf(stderr, "locking: tevent_add_signal() failed\n"); + talloc_free(ev); + exit(1); + } + if (strcmp(lock_type, "RECORD") == 0) { if (argc != 8) { fprintf(stderr, "%s: Invalid number of arguments (%d)\n", @@ -193,20 +318,17 @@ int main(int argc, char *argv[]) usage(); exit(1); } - result = lock_record(argv[5], argv[6], argv[7]); + result = lock_record(argv[5], argv[6], argv[7], &state); } else if (strcmp(lock_type, "DB") == 0) { - int n; - - /* If there are no databases specified, no need for lock */ - if (argc > 5) { - for (n=5; n+1<argc; n+=2) { - result = lock_db(argv[n], argv[n+1]); - if (result != 0) { - break; - } - } + if (argc != 7) { + fprintf(stderr, + "locking: Invalid number of arguments (%d)\n", + argc); + usage(); + exit(1); } + result = lock_db(argv[5], argv[6], &state); } else { fprintf(stderr, "%s: Invalid lock-type '%s'\n", progname, lock_type); @@ -216,6 +338,21 @@ int main(int argc, char *argv[]) send_result(write_fd, result); - ctdb_wait_for_process_to_exit(ppid); + req = wait_for_parent_send(ev, ev, ppid); + if (req == NULL) { + fprintf(stderr, "locking: wait_for_parent_send() failed\n"); + cleanup(&state); + exit(1); + } + + tevent_req_poll(req, ev); + + status = wait_for_parent_recv(req); + if (! status) { + fprintf(stderr, "locking: wait_for_parent_recv() failed\n"); + } + + talloc_free(ev); + cleanup(&state); return 0; } diff --git a/ctdb/tests/eventscripts/13.per_ip_routing.024.sh b/ctdb/tests/eventscripts/13.per_ip_routing.024.sh new file mode 100755 index 0000000..7daacbb --- /dev/null +++ b/ctdb/tests/eventscripts/13.per_ip_routing.024.sh @@ -0,0 +1,31 @@ +#!/bin/sh + +. "${TEST_SCRIPTS_DIR}/unit.sh" + +define_test "Single IP, restores original rt_tables" + +setup_ctdb +setup_ctdb_policy_routing + +create_policy_routing_config 1 default + +_rt_tables="$CTDB_SYS_ETCDIR/iproute2/rt_tables" +_rt_orig=$(mktemp --tmpdir="$EVENTSCRIPTS_TESTS_VAR_DIR") +cp "$_rt_tables" "$_rt_orig" + +ctdb_get_1_public_address | { + read dev ip bits + + ok_null + simple_test_event "takeip" $dev $ip $bits + + ok <<EOF +Removing ip rule for public address ${ip} for routing table ctdb.${ip} +EOF + simple_test_event "shutdown" +} + +ok_null +simple_test_command diff -u "$_rt_orig" "$_rt_tables" + +check_routes 0 diff --git a/ctdb/tests/eventscripts/stubs/ctdb b/ctdb/tests/eventscripts/stubs/ctdb index a1cba70..28868d1 100755 --- a/ctdb/tests/eventscripts/stubs/ctdb +++ b/ctdb/tests/eventscripts/stubs/ctdb @@ -445,7 +445,7 @@ case "$1" in echo "|${2:-monitor}|${_b}|${_code}|${_status}|${_d1}|${_d2}|${_err_out}|" done ;; - gratiousarp) : ;; # Do nothing for now + gratarp) : ;; # Do nothing for now ip) ctdb_ip "$@" ;; pnn|xpnn) ctdb_pnn ;; enable) ctdb_enable "$@";; diff --git a/ctdb/tests/src/test_mutex_raw.c b/ctdb/tests/src/test_mutex_raw.c new file mode 100644 index 0000000..8e3cae3 --- /dev/null +++ b/ctdb/tests/src/test_mutex_raw.c @@ -0,0 +1,261 @@ +/* + Robust mutex test + + Copyright (C) Amitay Isaacs 2016 + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, see <http://www.gnu.org/licenses/>. +*/ + +/* + * Run this test as follows: -- Samba Shared Repository