The branch, master has been updated via 4d58d0f tdb: build and run unit tests in tdb/test/ via 205242e tdb/test: fix up tests for use in SAMBA tdb code. via 8fa345d tdb: wean CCAN-style unit tests off of tap. via 0802791 tdb: import unit tests from CCAN into tdb/test/ via 390b9a2 tdb: make tdb_private.h idempotent. from ad2a2c4 s4:torture: add another SMB2 rename test
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 4d58d0fa8f936e7efdc02e31c053d42a47b3e62a Author: Rusty Russell <ru...@rustcorp.com.au> Date: Tue Feb 14 14:45:29 2012 +1030 tdb: build and run unit tests in tdb/test/ Now we can build the test binaries: the CCAN style is to compile everything called "compile_ok*.c", compile and run everything called "run*.c", compile, link with the module, and run everything called "api*.c", and link any other C files (presumably test helpers) into all the tests. Unfortunately, actually passing that between the various parts of wscript is painful, so I open-coded the names. Also, the tests expect to be run in a (temporary) directory they can pollute, with the test directory found in test/ (to find the canned TDB files, for example). Signed-off-by: Rusty Russell <ru...@rustcorp.com.au> Autobuild-User: Rusty Russell <ru...@rustcorp.com.au> Autobuild-Date: Tue Feb 14 06:53:46 CET 2012 on sn-devel-104 commit 205242e1769f96e0e8fccd52378965d35dd02093 Author: Rusty Russell <ru...@rustcorp.com.au> Date: Tue Feb 14 14:45:21 2012 +1030 tdb/test: fix up tests for use in SAMBA tdb code. 1) Make sure we include "tdb_private.h" first, to get the right headers (esp. the correct setting of _FILE_OFFSET_BITS before unistd.h). 2) Fix 3G file test since expand logic has changed. 3) Fix nested transaction test, since default is to allow nesting. 4) Capture fdatasync, which was slowing down transaction expand. Signed-off-by: Rusty Russell <ru...@rustcorp.com.au> commit 8fa345d952328c5866f3a0f835f3599343c51b00 Author: Rusty Russell <ru...@rustcorp.com.au> Date: Tue Feb 14 14:45:19 2012 +1030 tdb: wean CCAN-style unit tests off of tap. We could use subunit, but that's overkill. Just print messages when we fail, and use exit status. Signed-off-by: Rusty Russell <ru...@rustcorp.com.au> commit 0802791081ba39298aa93f0e6860c3b62800df73 Author: Rusty Russell <ru...@rustcorp.com.au> Date: Tue Feb 14 04:05:43 2012 +1030 tdb: import unit tests from CCAN into tdb/test/ I pulled tdb into CCAN as an experiment a while ago; it doesn't belong there, but it has accumulated some important unit tests. These are copied from CCAN version init-1486-gc438ec1 with #include "../" changed to #include "../common/". Signed-off-by: Rusty Russell <ru...@rustcorp.com.au> commit 390b9a2dd8447ecd16e3957c02fa886781797733 Author: Rusty Russell <ru...@rustcorp.com.au> Date: Tue Feb 14 04:04:43 2012 +1030 tdb: make tdb_private.h idempotent. The most convenient way to write unit tests in C is to directly #include the C files (CCAN uses this, for example). That works quite well, but it means that tdb_private.h now needs to be protected against multiple inclusions. Signed-off-by: Rusty Russell <ru...@rustcorp.com.au> ----------------------------------------------------------------------- Summary of changes: lib/tdb/Makefile | 2 +- lib/tdb/common/tdb_private.h | 3 + lib/tdb/test/external-agent.c | 195 +++++++++++++++++ lib/tdb/test/external-agent.h | 41 ++++ .../test/jenkins-be-hash.tdb} | Bin 696 -> 696 bytes .../test/jenkins-le-hash.tdb} | Bin 696 -> 696 bytes lib/tdb/test/lock-tracking.c | 146 +++++++++++++ lib/{tdb2 => tdb}/test/lock-tracking.h | 0 lib/tdb/test/logging.c | 33 +++ lib/tdb/test/logging.h | 11 + .../test/old-nohash-be.tdb} | Bin 696 -> 696 bytes .../test/old-nohash-le.tdb} | Bin 696 -> 696 bytes lib/tdb/test/run-3G-file.c | 129 +++++++++++ lib/tdb/test/run-bad-tdb-header.c | 60 ++++++ lib/tdb/test/run-check.c | 66 ++++++ lib/tdb/test/run-corrupt.c | 129 +++++++++++ lib/tdb/test/run-die-during-transaction.c | 225 ++++++++++++++++++++ lib/tdb/test/run-endian.c | 65 ++++++ lib/tdb/test/run-incompatible.c | 187 ++++++++++++++++ lib/tdb/test/run-nested-transactions.c | 80 +++++++ lib/tdb/test/run-nested-traverse.c | 91 ++++++++ lib/tdb/test/run-no-lock-during-traverse.c | 116 ++++++++++ lib/tdb/test/run-oldhash.c | 51 +++++ lib/tdb/test/run-open-during-transaction.c | 184 ++++++++++++++++ lib/tdb/test/run-readonly-check.c | 54 +++++ lib/tdb/test/run-rwlock-check.c | 47 ++++ lib/tdb/test/run-summary.c | 65 ++++++ lib/tdb/test/run-transaction-expand.c | 110 ++++++++++ lib/tdb/test/run-traverse-in-transaction.c | 90 ++++++++ lib/tdb/test/run-wronghash-fail.c | 122 +++++++++++ lib/tdb/test/run-zero-append.c | 42 ++++ lib/tdb/test/run.c | 51 +++++ .../test/rwlock-be.tdb1 => tdb/test/rwlock-be.tdb} | Bin 696 -> 696 bytes .../test/rwlock-be.tdb1 => tdb/test/rwlock-le.tdb} | Bin 696 -> 696 bytes lib/tdb/test/tap-interface.h | 40 ++++ lib/tdb/test/tap-to-subunit.h | 155 ++++++++++++++ .../test/tdb1.corrupt => tdb/test/tdb.corrupt} | Bin 192512 -> 192512 bytes lib/tdb/wscript | 94 ++++++++- 38 files changed, 2676 insertions(+), 8 deletions(-) create mode 100644 lib/tdb/test/external-agent.c create mode 100644 lib/tdb/test/external-agent.h copy lib/{tdb2/test/jenkins-be-hash.tdb1 => tdb/test/jenkins-be-hash.tdb} (100%) copy lib/{tdb2/test/jenkins-le-hash.tdb1 => tdb/test/jenkins-le-hash.tdb} (100%) create mode 100644 lib/tdb/test/lock-tracking.c copy lib/{tdb2 => tdb}/test/lock-tracking.h (100%) create mode 100644 lib/tdb/test/logging.c create mode 100644 lib/tdb/test/logging.h copy lib/{tdb2/test/old-nohash-be.tdb1 => tdb/test/old-nohash-be.tdb} (100%) copy lib/{tdb2/test/old-nohash-le.tdb1 => tdb/test/old-nohash-le.tdb} (100%) create mode 100644 lib/tdb/test/run-3G-file.c create mode 100644 lib/tdb/test/run-bad-tdb-header.c create mode 100644 lib/tdb/test/run-check.c create mode 100644 lib/tdb/test/run-corrupt.c create mode 100644 lib/tdb/test/run-die-during-transaction.c create mode 100644 lib/tdb/test/run-endian.c create mode 100644 lib/tdb/test/run-incompatible.c create mode 100644 lib/tdb/test/run-nested-transactions.c create mode 100644 lib/tdb/test/run-nested-traverse.c create mode 100644 lib/tdb/test/run-no-lock-during-traverse.c create mode 100644 lib/tdb/test/run-oldhash.c create mode 100644 lib/tdb/test/run-open-during-transaction.c create mode 100644 lib/tdb/test/run-readonly-check.c create mode 100644 lib/tdb/test/run-rwlock-check.c create mode 100644 lib/tdb/test/run-summary.c create mode 100644 lib/tdb/test/run-transaction-expand.c create mode 100644 lib/tdb/test/run-traverse-in-transaction.c create mode 100644 lib/tdb/test/run-wronghash-fail.c create mode 100644 lib/tdb/test/run-zero-append.c create mode 100644 lib/tdb/test/run.c copy lib/{tdb2/test/rwlock-be.tdb1 => tdb/test/rwlock-be.tdb} (100%) copy lib/{tdb2/test/rwlock-be.tdb1 => tdb/test/rwlock-le.tdb} (100%) create mode 100644 lib/tdb/test/tap-interface.h create mode 100644 lib/tdb/test/tap-to-subunit.h copy lib/{tdb2/test/tdb1.corrupt => tdb/test/tdb.corrupt} (100%) Changeset truncated at 500 lines: diff --git a/lib/tdb/Makefile b/lib/tdb/Makefile index 4c28653..fe44ff6 100644 --- a/lib/tdb/Makefile +++ b/lib/tdb/Makefile @@ -11,7 +11,7 @@ install: uninstall: $(WAF) uninstall -test: +test: FORCE $(WAF) test $(TEST_OPTIONS) testenv: diff --git a/lib/tdb/common/tdb_private.h b/lib/tdb/common/tdb_private.h index 4fc7381..9913284 100644 --- a/lib/tdb/common/tdb_private.h +++ b/lib/tdb/common/tdb_private.h @@ -1,3 +1,5 @@ +#ifndef TDB_PRIVATE_H +#define TDB_PRIVATE_H /* Unix SMB/CIFS implementation. @@ -280,3 +282,4 @@ void tdb_header_hash(struct tdb_context *tdb, uint32_t *magic1_hash, uint32_t *magic2_hash); unsigned int tdb_old_hash(TDB_DATA *key); size_t tdb_dead_space(struct tdb_context *tdb, tdb_off_t off); +#endif /* TDB_PRIVATE_H */ diff --git a/lib/tdb/test/external-agent.c b/lib/tdb/test/external-agent.c new file mode 100644 index 0000000..d3fe891 --- /dev/null +++ b/lib/tdb/test/external-agent.c @@ -0,0 +1,195 @@ +#include "external-agent.h" +#include "lock-tracking.h" +#include "logging.h" +#include <sys/types.h> +#include <sys/wait.h> +#include <unistd.h> +#include <err.h> +#include <fcntl.h> +#include <stdlib.h> +#include <limits.h> +#include <string.h> +#include <errno.h> +#include "../common/tdb_private.h" +#include "tap-interface.h" +#include <stdio.h> +#include <stdarg.h> + +static struct tdb_context *tdb; + +static enum agent_return do_operation(enum operation op, const char *name) +{ + TDB_DATA k; + enum agent_return ret; + TDB_DATA data; + + if (op != OPEN && op != OPEN_WITH_CLEAR_IF_FIRST && !tdb) { + diag("external: No tdb open!"); + return OTHER_FAILURE; + } + + k.dptr = (void *)name; + k.dsize = strlen(name); + + locking_would_block = 0; + switch (op) { + case OPEN: + if (tdb) { + diag("Already have tdb %s open", tdb_name(tdb)); + return OTHER_FAILURE; + } + tdb = tdb_open_ex(name, 0, TDB_DEFAULT, O_RDWR, 0, + &taplogctx, NULL); + if (!tdb) { + if (!locking_would_block) + diag("Opening tdb gave %s", strerror(errno)); + ret = OTHER_FAILURE; + } else + ret = SUCCESS; + break; + case OPEN_WITH_CLEAR_IF_FIRST: + if (tdb) + return OTHER_FAILURE; + tdb = tdb_open_ex(name, 0, TDB_CLEAR_IF_FIRST, O_RDWR, 0, + &taplogctx, NULL); + ret = tdb ? SUCCESS : OTHER_FAILURE; + break; + case TRANSACTION_START: + ret = tdb_transaction_start(tdb) == 0 ? SUCCESS : OTHER_FAILURE; + break; + case FETCH: + data = tdb_fetch(tdb, k); + if (data.dptr == NULL) { + if (tdb_error(tdb) == TDB_ERR_NOEXIST) + ret = FAILED; + else + ret = OTHER_FAILURE; + } else if (data.dsize != k.dsize + || memcmp(data.dptr, k.dptr, k.dsize) != 0) { + ret = OTHER_FAILURE; + } else { + ret = SUCCESS; + } + free(data.dptr); + break; + case STORE: + ret = tdb_store(tdb, k, k, 0) == 0 ? SUCCESS : OTHER_FAILURE; + break; + case TRANSACTION_COMMIT: + ret = tdb_transaction_commit(tdb)==0 ? SUCCESS : OTHER_FAILURE; + break; + case CHECK: + ret = tdb_check(tdb, NULL, NULL) == 0 ? SUCCESS : OTHER_FAILURE; + break; + case NEEDS_RECOVERY: + ret = tdb_needs_recovery(tdb) ? SUCCESS : FAILED; + break; + case CLOSE: + ret = tdb_close(tdb) == 0 ? SUCCESS : OTHER_FAILURE; + tdb = NULL; + break; + default: + ret = OTHER_FAILURE; + } + + if (locking_would_block) + ret = WOULD_HAVE_BLOCKED; + + return ret; +} + +struct agent { + int cmdfd, responsefd; +}; + +/* Do this before doing any tdb stuff. Return handle, or NULL. */ +struct agent *prepare_external_agent(void) +{ + int pid, ret; + int command[2], response[2]; + char name[1+PATH_MAX]; + + if (pipe(command) != 0 || pipe(response) != 0) + return NULL; + + pid = fork(); + if (pid < 0) + return NULL; + + if (pid != 0) { + struct agent *agent = malloc(sizeof(*agent)); + + close(command[0]); + close(response[1]); + agent->cmdfd = command[1]; + agent->responsefd = response[0]; + return agent; + } + + close(command[1]); + close(response[0]); + + /* We want to fail, not block. */ + nonblocking_locks = true; + log_prefix = "external: "; + while ((ret = read(command[0], name, sizeof(name))) > 0) { + enum agent_return result; + + result = do_operation(name[0], name+1); + if (write(response[1], &result, sizeof(result)) + != sizeof(result)) + err(1, "Writing response"); + } + exit(0); +} + +/* Ask the external agent to try to do an operation. */ +enum agent_return external_agent_operation(struct agent *agent, + enum operation op, + const char *name) +{ + enum agent_return res; + unsigned int len; + char *string; + + if (!name) + name = ""; + len = 1 + strlen(name) + 1; + string = malloc(len); + + string[0] = op; + strcpy(string+1, name); + + if (write(agent->cmdfd, string, len) != len + || read(agent->responsefd, &res, sizeof(res)) != sizeof(res)) + res = AGENT_DIED; + + free(string); + return res; +} + +const char *agent_return_name(enum agent_return ret) +{ + return ret == SUCCESS ? "SUCCESS" + : ret == WOULD_HAVE_BLOCKED ? "WOULD_HAVE_BLOCKED" + : ret == AGENT_DIED ? "AGENT_DIED" + : ret == FAILED ? "FAILED" + : ret == OTHER_FAILURE ? "OTHER_FAILURE" + : "**INVALID**"; +} + +const char *operation_name(enum operation op) +{ + switch (op) { + case OPEN: return "OPEN"; + case OPEN_WITH_CLEAR_IF_FIRST: return "OPEN_WITH_CLEAR_IF_FIRST"; + case TRANSACTION_START: return "TRANSACTION_START"; + case FETCH: return "FETCH"; + case STORE: return "STORE"; + case TRANSACTION_COMMIT: return "TRANSACTION_COMMIT"; + case CHECK: return "CHECK"; + case NEEDS_RECOVERY: return "NEEDS_RECOVERY"; + case CLOSE: return "CLOSE"; + } + return "**INVALID**"; +} diff --git a/lib/tdb/test/external-agent.h b/lib/tdb/test/external-agent.h new file mode 100644 index 0000000..dffdca9 --- /dev/null +++ b/lib/tdb/test/external-agent.h @@ -0,0 +1,41 @@ +#ifndef TDB_TEST_EXTERNAL_AGENT_H +#define TDB_TEST_EXTERNAL_AGENT_H + +/* For locking tests, we need a different process to try things at + * various times. */ +enum operation { + OPEN, + OPEN_WITH_CLEAR_IF_FIRST, + TRANSACTION_START, + FETCH, + STORE, + TRANSACTION_COMMIT, + CHECK, + NEEDS_RECOVERY, + CLOSE, +}; + +/* Do this before doing any tdb stuff. Return handle, or -1. */ +struct agent *prepare_external_agent(void); + +enum agent_return { + SUCCESS, + WOULD_HAVE_BLOCKED, + AGENT_DIED, + FAILED, /* For fetch, or NEEDS_RECOVERY */ + OTHER_FAILURE, +}; + +/* Ask the external agent to try to do an operation. + * name == tdb name for OPEN/OPEN_WITH_CLEAR_IF_FIRST, + * record name for FETCH/STORE (store stores name as data too) + */ +enum agent_return external_agent_operation(struct agent *handle, + enum operation op, + const char *name); + +/* Mapping enum -> string. */ +const char *agent_return_name(enum agent_return ret); +const char *operation_name(enum operation op); + +#endif /* TDB_TEST_EXTERNAL_AGENT_H */ diff --git a/lib/tdb/test/lock-tracking.c b/lib/tdb/test/lock-tracking.c new file mode 100644 index 0000000..b6f1cc2 --- /dev/null +++ b/lib/tdb/test/lock-tracking.c @@ -0,0 +1,146 @@ +/* We save the locks so we can reaquire them. */ +#include "../common/tdb_private.h" +#include <unistd.h> +#include <fcntl.h> +#include <stdarg.h> +#include <stdlib.h> +#include "tap-interface.h" +#include "lock-tracking.h" + +struct lock { + struct lock *next; + unsigned int off; + unsigned int len; + int type; +}; +static struct lock *locks; +int locking_errors = 0; +bool suppress_lockcheck = false; +bool nonblocking_locks; +int locking_would_block = 0; +void (*unlock_callback)(int fd); + +int fcntl_with_lockcheck(int fd, int cmd, ... /* arg */ ) +{ + va_list ap; + int ret, arg3; + struct flock *fl; + bool may_block = false; + + if (cmd != F_SETLK && cmd != F_SETLKW) { + /* This may be totally bogus, but we don't know in general. */ + va_start(ap, cmd); + arg3 = va_arg(ap, int); + va_end(ap); + + return fcntl(fd, cmd, arg3); + } + + va_start(ap, cmd); + fl = va_arg(ap, struct flock *); + va_end(ap); + + if (cmd == F_SETLKW && nonblocking_locks) { + cmd = F_SETLK; + may_block = true; + } + ret = fcntl(fd, cmd, fl); + + /* Detect when we failed, but might have been OK if we waited. */ + if (may_block && ret == -1 && (errno == EAGAIN || errno == EACCES)) { + locking_would_block++; + } + + if (fl->l_type == F_UNLCK) { + struct lock **l; + struct lock *old = NULL; + + for (l = &locks; *l; l = &(*l)->next) { + if ((*l)->off == fl->l_start + && (*l)->len == fl->l_len) { + if (ret == 0) { + old = *l; + *l = (*l)->next; + free(old); + } + break; + } + } + if (!old && !suppress_lockcheck) { + diag("Unknown unlock %u@%u - %i", + (int)fl->l_len, (int)fl->l_start, ret); + locking_errors++; + } + } else { + struct lock *new, *i; + unsigned int fl_end = fl->l_start + fl->l_len; + if (fl->l_len == 0) + fl_end = (unsigned int)-1; + + /* Check for overlaps: we shouldn't do this. */ + for (i = locks; i; i = i->next) { + unsigned int i_end = i->off + i->len; + if (i->len == 0) + i_end = (unsigned int)-1; + + if (fl->l_start >= i->off && fl->l_start < i_end) + break; + if (fl_end >= i->off && fl_end < i_end) + break; + + /* tdb_allrecord_lock does this, handle adjacent: */ + if (fl->l_start == i_end && fl->l_type == i->type) { + if (ret == 0) { + i->len = fl->l_len + ? i->len + fl->l_len + : 0; + } + goto done; + } + } + if (i) { + /* Special case: upgrade of allrecord lock. */ + if (i->type == F_RDLCK && fl->l_type == F_WRLCK + && i->off == FREELIST_TOP + && fl->l_start == FREELIST_TOP + && i->len == 0 + && fl->l_len == 0) { + if (ret == 0) + i->type = F_WRLCK; + goto done; + } + if (!suppress_lockcheck) { + diag("%s lock %u@%u overlaps %u@%u", + fl->l_type == F_WRLCK ? "write" : "read", + (int)fl->l_len, (int)fl->l_start, + i->len, (int)i->off); + locking_errors++; + } + } + + if (ret == 0) { + new = malloc(sizeof *new); + new->off = fl->l_start; + new->len = fl->l_len; + new->type = fl->l_type; + new->next = locks; + locks = new; + } + } +done: + if (ret == 0 && fl->l_type == F_UNLCK && unlock_callback) + unlock_callback(fd); + return ret; +} + +unsigned int forget_locking(void) +{ + unsigned int num = 0; + while (locks) { + struct lock *next = locks->next; + free(locks); + locks = next; + num++; + } + return num; +} diff --git a/lib/tdb2/test/lock-tracking.h b/lib/tdb/test/lock-tracking.h similarity index 100% copy from lib/tdb2/test/lock-tracking.h copy to lib/tdb/test/lock-tracking.h diff --git a/lib/tdb/test/logging.c b/lib/tdb/test/logging.c new file mode 100644 index 0000000..dfab486 --- /dev/null +++ b/lib/tdb/test/logging.c @@ -0,0 +1,33 @@ +#include "logging.h" +#include "tap-interface.h" +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +bool suppress_logging = false; +const char *log_prefix = ""; + +/* Turn log messages into tap diag messages. */ +static void taplog(struct tdb_context *tdb, + enum tdb_debug_level level, + const char *fmt, ...) +{ + va_list ap; + char line[200]; + + if (suppress_logging) + return; + + va_start(ap, fmt); + vsprintf(line, fmt, ap); + va_end(ap); + + /* Strip trailing \n: diag adds it. */ + if (line[0] && line[strlen(line)-1] == '\n') + diag("%s%.*s", log_prefix, (unsigned)strlen(line)-1, line); + else + diag("%s%s", log_prefix, line); +} + +struct tdb_logging_context taplogctx = { taplog, NULL }; diff --git a/lib/tdb/test/logging.h b/lib/tdb/test/logging.h new file mode 100644 index 0000000..89e77b2 --- /dev/null +++ b/lib/tdb/test/logging.h @@ -0,0 +1,11 @@ +#ifndef TDB_TEST_LOGGING_H +#define TDB_TEST_LOGGING_H +#include "replace.h" +#include "../include/tdb.h" +#include <stdbool.h> + +extern bool suppress_logging; +extern const char *log_prefix; +extern struct tdb_logging_context taplogctx; + +#endif /* TDB_TEST_LOGGING_H */ diff --git a/lib/tdb/test/run-3G-file.c b/lib/tdb/test/run-3G-file.c new file mode 100644 index 0000000..5ea4fcf --- /dev/null +++ b/lib/tdb/test/run-3G-file.c @@ -0,0 +1,129 @@ +/* We need this otherwise fcntl locking fails. */ +#define _FILE_OFFSET_BITS 64 +#define _XOPEN_SOURCE 500 +#include "../common/tdb_private.h" +#include "../common/io.c" +#include "../common/tdb.c" -- Samba Shared Repository