The branch, master has been updated
via 7900f319db3 ctdb-server: Load optional tunables.d/ directory
via 598d9b4dfd4 ctdb-server: Whitespace fixes
via 6863799ce93 ctdb-tests: Add tests for tunables directory loading
via 4f1e3844b43 ctdb-common: Add tunable directory loading
via de069d6b950 ctdb-tests: Allow tunables unit test to process a
second file
via 4352a8da5ac ctdb-common: Log tunables filename instead of function
name
via cdb02782f95 ctdb-common: Log a message if the tunables file does
not exist
via 7d010715c59 ctdb-tests: Add logging support to the tunables unit
test
via 2ec739a039e ctdb-common: Require separate initialisation of tunable
defaults
via 96865764305 ctdb-tests: Reformat with "shfmt -w -p -i 0 -fn"
via c6f2664b489 ctdb-common: Fix includes to make file standalone
from b72cc7ce6ac libcli/smb: use talloc_asprintf_addbuf() in
smbXcli_session_dump_keys()
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 7900f319db3135000f71836dbe0279c616a6cf20
Author: Martin Schwenke <[email protected]>
Date: Wed Jun 25 22:18:16 2025 +1000
ctdb-server: Load optional tunables.d/ directory
Change the variable name to "path" so it makes sense to reuse it for
the directory.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
Autobuild-User(master): Martin Schwenke <[email protected]>
Autobuild-Date(master): Wed Jul 23 00:02:47 UTC 2025 on atb-devel-224
commit 598d9b4dfd47f1101f8d531b6f96d31c4194ff4b
Author: Martin Schwenke <[email protected]>
Date: Wed Jun 25 22:21:39 2025 +1000
ctdb-server: Whitespace fixes
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit 6863799ce93b655f2f412f304b1556b9f09dd1a9
Author: Martin Schwenke <[email protected]>
Date: Fri Jun 27 17:35:03 2025 +1000
ctdb-tests: Add tests for tunables directory loading
The missing 2nd file testcase becomes the missing directory testcase,
because you can't easily have both. See the comment in
tunable_test.c.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit 4f1e3844b432dc2b72e62bf17cb4dd36f28d6380
Author: Martin Schwenke <[email protected]>
Date: Fri Jun 27 15:11:59 2025 +1000
ctdb-common: Add tunable directory loading
This is simpler than include files and more general than just allowing
a 2nd file to be loaded.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit de069d6b950531b3c2ad4584ae661980bd25a982
Author: Martin Schwenke <[email protected]>
Date: Wed Jun 25 22:14:05 2025 +1000
ctdb-tests: Allow tunables unit test to process a second file
The second file is optional.
Make $tfile the default to avoid having to update all of the single
file testcases.
Add test cases for second file.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit 4352a8da5ac05b8745b6197fa80774d8b0fe5f4b
Author: Martin Schwenke <[email protected]>
Date: Fri Jun 27 15:05:04 2025 +1000
ctdb-common: Log tunables filename instead of function name
When loading multiple files, this is more informative.
Tweak a couple of error messages to ensure they all mention
"tunables".
Update the unit testcases to match.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit cdb02782f950bf3a7585a8390a0c40ed81cd3971
Author: Martin Schwenke <[email protected]>
Date: Sun Jun 29 10:16:20 2025 +1000
ctdb-common: Log a message if the tunables file does not exist
Switch the unit test script to INFO debug level to ensure this output
appears when expected.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit 7d010715c591fc05871078958c08eb7207729c65
Author: Martin Schwenke <[email protected]>
Date: Sat Jun 28 12:01:41 2025 +1000
ctdb-tests: Add logging support to the tunables unit test
Make tunable_test respect CTDB_DEBUGLEVEL. Using test_options.[ch]
would be overkill here. This means including logging.c - we can't
link to the subsystem containing logging.c because the file being
tested (tunable.c) is part of that subsystem.
Support logging in the test script. tunable_ok() builds the logging
output for the good path. Set the debug level to NOTICE and update
expected results for individual failure path tests.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit 2ec739a039e3a9af82e614afeae54359fb05b288
Author: Martin Schwenke <[email protected]>
Date: Wed Jun 25 20:46:26 2025 +1000
ctdb-common: Require separate initialisation of tunable defaults
Dropping this from ctdb_tunable_load_file() allows that function to be
called multiple times for different files. The caller sets the
defaults.
In the test script, factor out the handling of a single tunables file
in a similar way. Ignoring missing/unreadable files is OK because
this function will only be called for test successes (hence "ok" in
the name). There will never be existing, unreadable files. The code
being tested ignores missing files, so do that here too.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit 968657643052a90bab1deb5ffd4938a256e5ee14
Author: Martin Schwenke <[email protected]>
Date: Sat Jun 28 12:12:33 2025 +1000
ctdb-tests: Reformat with "shfmt -w -p -i 0 -fn"
Best reviewed with "git show -w".
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
commit c6f2664b489fd2eb0ce029fb3ab86fc77ca809f0
Author: Martin Schwenke <[email protected]>
Date: Wed Jun 25 20:44:52 2025 +1000
ctdb-common: Fix includes to make file standalone
Avoid lots of red underlines in my editor.
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
-----------------------------------------------------------------------
Summary of changes:
ctdb/common/tunable.c | 100 ++++++++++--
ctdb/common/tunable.h | 10 ++
ctdb/doc/ctdb-tunables.7.xml | 15 +-
ctdb/server/ctdb_tunables.c | 42 +++--
ctdb/tests/UNIT/cunit/tunable_test_001.sh | 245 +++++++++++++++++++++++++-----
ctdb/tests/src/tunable_test.c | 54 ++++++-
6 files changed, 401 insertions(+), 65 deletions(-)
Changeset truncated at 500 lines:
diff --git a/ctdb/common/tunable.c b/ctdb/common/tunable.c
index f366f231e53..e7525f586b6 100644
--- a/ctdb/common/tunable.c
+++ b/ctdb/common/tunable.c
@@ -18,7 +18,9 @@
*/
#include "replace.h"
+#include "system/dir.h"
#include "system/filesys.h"
+#include "system/glob.h"
#include "system/locale.h"
#include "system/network.h"
@@ -289,7 +291,7 @@ char *ctdb_tunable_names_to_string(TALLOC_CTX *mem_ctx)
struct tunable_load_state {
struct ctdb_tunable_list *tun_list;
bool status;
- const char *func;
+ const char *file;
};
static bool tunable_section(const char *section, void *private_data)
@@ -297,8 +299,9 @@ static bool tunable_section(const char *section, void
*private_data)
struct tunable_load_state *state =
(struct tunable_load_state *)private_data;
- D_ERR("%s: Invalid line for section [%s] - sections not supported \n",
- state->func,
+ D_ERR("%s: Invalid line for section [%s] - "
+ "tunables sections not supported \n",
+ state->file,
section);
state->status = false;
@@ -317,7 +320,9 @@ static bool tunable_option(const char *name,
int ret;
if (value[0] == '\0') {
- D_ERR("%s: Invalid line containing \"%s\"\n", state->func,
name);
+ D_ERR("%s: Invalid tunables line containing \"%s\"\n",
+ state->file,
+ name);
state->status = false;
return true;
}
@@ -325,7 +330,7 @@ static bool tunable_option(const char *name,
num = smb_strtoul(value, NULL, 0, &ret, SMB_STR_FULL_STR_CONV);
if (ret != 0) {
D_ERR("%s: Invalid value \"%s\" for tunable \"%s\"\n",
- state->func,
+ state->file,
value,
name);
state->status = false;
@@ -337,12 +342,12 @@ static bool tunable_option(const char *name,
(uint32_t)num,
&obsolete);
if (!ok) {
- D_ERR("%s: Unknown tunable \"%s\"\n", state->func, name);
+ D_ERR("%s: Unknown tunable \"%s\"\n", state->file, name);
state->status = false;
return true;
}
if (obsolete) {
- D_ERR("%s: Obsolete tunable \"%s\"\n", state->func, name);
+ D_ERR("%s: Obsolete tunable \"%s\"\n", state->file, name);
state->status = false;
return true;
}
@@ -356,18 +361,16 @@ bool ctdb_tunable_load_file(TALLOC_CTX *mem_ctx,
{
struct tunable_load_state state = {
.tun_list = tun_list,
+ .file = file,
.status = true,
- .func = __FUNCTION__,
};
FILE *fp;
bool status;
- ctdb_tunable_set_defaults(tun_list);
-
fp = fopen(file, "r");
if (fp == NULL) {
if (errno == ENOENT) {
- /* Doesn't need to exist */
+ D_INFO("Optional tunables file %s not found\n", file);
return true;
}
@@ -394,8 +397,81 @@ bool ctdb_tunable_load_file(TALLOC_CTX *mem_ctx,
fclose(fp);
if (!status) {
- DBG_ERR("Syntax error\n");
+ D_ERR("%s: Syntax error\n", file);
}
return status && state.status;
}
+
+static int tunables_filter(const struct dirent *de)
+{
+ int ret;
+
+ /* Match a script pattern */
+ ret = fnmatch("*.tunables", de->d_name, 0);
+ if (ret == 0) {
+ return 1;
+ }
+
+ return 0;
+}
+
+bool ctdb_tunable_load_directory(TALLOC_CTX *mem_ctx,
+ struct ctdb_tunable_list *tun_list,
+ const char *dir)
+{
+ struct dirent **namelist = NULL;
+ int count = 0;
+ bool status = true;
+ int i = 0;
+
+ count = scandir(dir, &namelist, tunables_filter, alphasort);
+ if (count == -1) {
+ switch (errno) {
+ case ENOENT:
+ D_INFO("Optional tunables directory %s not found\n",
+ dir);
+ break;
+ default:
+ DBG_ERR("Failed to open directory %s (err=%d)\n",
+ dir,
+ errno);
+ status = false;
+ }
+ goto done;
+ }
+
+ if (count == 0) {
+ goto done;
+ }
+
+ for (i = 0; i < count; i++) {
+ char *file = NULL;
+ bool file_status = false;
+
+ file = talloc_asprintf(mem_ctx,
+ "%s/%s",
+ dir,
+ namelist[i]->d_name);
+ if (file == NULL) {
+ DBG_ERR("Memory allocation error\n");
+ goto done;
+ }
+
+ file_status = ctdb_tunable_load_file(mem_ctx, tun_list, file);
+ if (!file_status) {
+ status = false;
+ }
+ TALLOC_FREE(file);
+ }
+
+done:
+ if (namelist != NULL && count != -1) {
+ for (i = 0; i < count; i++) {
+ free(namelist[i]);
+ }
+ free(namelist);
+ }
+
+ return status;
+}
diff --git a/ctdb/common/tunable.h b/ctdb/common/tunable.h
index 89f99f1644f..bd8ee45bf1a 100644
--- a/ctdb/common/tunable.h
+++ b/ctdb/common/tunable.h
@@ -20,6 +20,13 @@
#ifndef __CTDB_TUNABLE_H__
#define __CTDB_TUNABLE_H__
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <talloc.h>
+
+#include "protocol/protocol.h"
+
void ctdb_tunable_set_defaults(struct ctdb_tunable_list *tun_list);
bool ctdb_tunable_get_value(struct ctdb_tunable_list *tun_list,
const char *tunable_str, uint32_t *value);
@@ -31,5 +38,8 @@ char *ctdb_tunable_names_to_string(TALLOC_CTX *mem_ctx);
bool ctdb_tunable_load_file(TALLOC_CTX *mem_ctx,
struct ctdb_tunable_list *tun_list,
const char *file);
+bool ctdb_tunable_load_directory(TALLOC_CTX *mem_ctx,
+ struct ctdb_tunable_list *tun_list,
+ const char *dir);
#endif /* __CTDB_TUNABLE_H__ */
diff --git a/ctdb/doc/ctdb-tunables.7.xml b/ctdb/doc/ctdb-tunables.7.xml
index 766213e1423..56b1e055a84 100644
--- a/ctdb/doc/ctdb-tunables.7.xml
+++ b/ctdb/doc/ctdb-tunables.7.xml
@@ -38,9 +38,17 @@
</para>
<para>
- Tunables can be set at startup from the
- <filename>/usr/local/etc/ctdb/ctdb.tunables</filename>
- configuration file.
+ Tunables can be set at startup via optional configuration files.
+ First, <filename>/usr/local/etc/ctdb/ctdb.tunables</filename> is
+ loaded, if this file exists. After this, all files with names
+ matching <filename>*.tunables</filename> in directory
+ <filename>/usr/local/etc/ctdb/tunables.d/</filename> are loaded
+ in the current locale's collation order, if the directory
+ exists.
+ </para>
+
+ <para>
+ The format of lines in tunables files is:
<literallayout>
<replaceable>TUNABLE</replaceable>=<replaceable>VALUE</replaceable>
@@ -722,6 +730,7 @@ MonitorInterval=20
<simplelist>
<member><filename>/usr/local/etc/ctdb/ctdb.tunables</filename></member>
+
<member><filename>/usr/local/etc/ctdb/tunables.d/*.tunables</filename></member>
</simplelist>
</refsect1>
diff --git a/ctdb/server/ctdb_tunables.c b/ctdb/server/ctdb_tunables.c
index 0dce656a93c..66c4d2d645b 100644
--- a/ctdb/server/ctdb_tunables.c
+++ b/ctdb/server/ctdb_tunables.c
@@ -1,4 +1,4 @@
-/*
+/*
ctdb tunables code
Copyright (C) Andrew Tridgell 2007
@@ -7,12 +7,12 @@
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
-
+
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
-
+
You should have received a copy of the GNU General Public License
along with this program; if not, see <http://www.gnu.org/licenses/>.
*/
@@ -142,12 +142,10 @@ int32_t ctdb_control_list_tunables(struct ctdb_context
*ctdb, TDB_DATA *outdata)
bool ctdb_tunables_load(struct ctdb_context *ctdb)
{
- bool status;
+ bool status = false; /* fail by default */
+ bool dir_status = false;
TALLOC_CTX *tmp_ctx;
- char *file = NULL;
-
- /* Fail by default */
- status = false;
+ char *path = NULL;
tmp_ctx = talloc_new(ctdb);
if (tmp_ctx == NULL) {
@@ -155,15 +153,35 @@ bool ctdb_tunables_load(struct ctdb_context *ctdb)
goto done;
}
- file = path_etcdir_append(tmp_ctx, "ctdb.tunables");
- if (file == NULL) {
+ ctdb_tunable_set_defaults(&ctdb->tunable);
+
+ path = path_etcdir_append(tmp_ctx, "ctdb.tunables");
+ if (path == NULL) {
D_ERR("Failed to construct path for ctdb.tunables\n");
goto done;
}
- status = ctdb_tunable_load_file(tmp_ctx, &ctdb->tunable, file);
- /* No need to log error, already logged above */
+ status = ctdb_tunable_load_file(tmp_ctx, &ctdb->tunable, path);
+ /*
+ * Continue loading directory on failure to avoid forcing a
+ * typo-prone admin to play whack-a-mole. Final result is
+ * still failure.
+ */
+
+ /* Avoid talloc confusion in static analysers... */
+ talloc_free(path);
+
+ path = path_etcdir_append(tmp_ctx, "tunables.d");
+ if (path == NULL) {
+ D_ERR("Failed to construct path for tunables.d\n");
+ goto done;
+ }
+
+ dir_status = ctdb_tunable_load_directory(tmp_ctx, &ctdb->tunable, path);
+ if (!dir_status) {
+ status = false;
+ }
done:
talloc_free(tmp_ctx);
return status;
diff --git a/ctdb/tests/UNIT/cunit/tunable_test_001.sh
b/ctdb/tests/UNIT/cunit/tunable_test_001.sh
index c68cd69c64a..a639c8772cb 100755
--- a/ctdb/tests/UNIT/cunit/tunable_test_001.sh
+++ b/ctdb/tests/UNIT/cunit/tunable_test_001.sh
@@ -3,10 +3,14 @@
. "${TEST_SCRIPTS_DIR}/unit.sh"
tfile="${CTDB_TEST_TMP_DIR}/tunable.$$"
+tfile2="${CTDB_TEST_TMP_DIR}/tunable2.$$"
+tdir="${CTDB_TEST_TMP_DIR}/tunabled.$$"
-remove_files ()
+remove_files()
{
- rm -f "$tfile"
+ rm -f "$tfile" "$tfile2"
+ rm -f "${tdir}/"* 2>/dev/null || true
+ rmdir "$tdir" 2>/dev/null || true
}
test_cleanup remove_files
@@ -62,15 +66,61 @@ IPAllocAlgorithm=2
AllowMixedVersions=0
"
-ok_tunable_defaults ()
+ok_tunable_defaults()
{
ok "$defaults"
}
+tunable_log()
+{
+ _level="$1"
+ _msg="$2"
+
+ _all=":DEBUG:INFO:NOTICE:WARNING:ERR:"
+ # Keep the debug levels log at. This strips off the levels up
+ # to and including the current $CTDB_DEBUGLEVEL, but then puts
+ # back $CTDB_DEBUGLEVEL. Cheaper than a loop...
+ _want=":${CTDB_DEBUGLEVEL}:${_all#*":${CTDB_DEBUGLEVEL}:"}"
+
+ case "$_want" in
+ *":${_level}:"*)
+ log="${log}${_msg}
+" # Intentional newline
+ ;;
+ esac
+}
+
+# Update $_map with tunable settings from 1 file
+# values
+ok_tunable_1()
+{
+ _file="$1"
+
+ if [ ! -r "$_file" ]; then
+ tunable_log "INFO" "Optional tunables file ${_file} not found"
+ return
+ fi
+
+ tunable_log "NOTICE" "Loading tunables from ${_file}"
+
+ while IFS='= ' read -r _var _val; do
+ case "$_var" in
+ \#* | "") continue ;;
+ esac
+ _decval=$((_val))
+ _vl=$(echo "$_var" | tr '[:upper:]' '[:lower:]')
+ _map=$(echo "$_map" |
+ sed -e "s|^\\(${_vl}:.*=\\).*\$|\\1${_decval}|")
+ done <"$_file"
+}
+
# Set required output to a version of $defaults where values for
-# tunables specified in $tfile replace the default values
-ok_tunable ()
+# tunables specified in the given file(s) replace the default values
+ok_tunable()
{
+ _f1="${1:-"${tfile}"}"
+ _f2="${2:-""}"
+
# Construct a version of $defaults prepended with a lowercase
# version of the tunable variable, to allow case-insensitive
# matching. This would be easier with the GNU sed
@@ -78,23 +128,37 @@ ok_tunable ()
# condition in awk causes empty lines to be skipped, in case
# there are trailing empty lines in $defaults.
_map=$(echo "$defaults" |
- awk -F= '$0 { printf "%s:%s=%s\n", tolower($1), $1, $2 }')
-
- # Replace values for tunables set in $tfile
- while IFS='= ' read -r _var _val ; do
- case "$_var" in
- \#* | "") continue ;;
- esac
- _decval=$((_val))
- _vl=$(echo "$_var" | tr '[:upper:]' '[:lower:]')
- _map=$(echo "$_map" |
- sed -e "s|^\\(${_vl}:.*=\\).*\$|\\1${_decval}|")
- done <"$tfile"
+ awk -F= '$0 { printf "%s:%s=%s\n", tolower($1), $1, $2 }')
+
+ log=""
+
+ #
+ # Replace values for tunables that are set in each file
+ #
+
+ ok_tunable_1 "$_f1"
+
+ if [ -n "$_f2" ]; then
+ if [ -f "$_f2" ]; then
+ ok_tunable_1 "$_f2"
+ elif [ -d "$_f2" ]; then
+ for _t in "${_f2}/"*.tunables; do
+ if [ ! -e "$_t" ]; then
+ break
+ fi
+ ok_tunable_1 "$_t"
+ done
+ elif [ ! -e "$_f2" ]; then
+ tunable_log "INFO" "Optional tunables directory ${_f2}
not found"
+ fi
+ fi
# Set result, stripping off lowercase tunable prefix
- ok "$(echo "$_map" | awk -F: '{ print $2 }')"
+ ok "${log}$(echo "$_map" | awk -F: '{ print $2 }')"
}
+export CTDB_DEBUGLEVEL="INFO"
+
test_case "Unreadable file"
: >"$tfile"
chmod a-r "$tfile"
@@ -113,35 +177,40 @@ rm -f "$tfile"
test_case "Invalid file, contains 1 word"
echo "Hello" >"$tfile"
required_error EINVAL <<EOF
-ctdb_tunable_load_file: Invalid line containing "Hello"
+Loading tunables from ${tfile}
+${tfile}: Invalid tunables line containing "Hello"
EOF
unit_test tunable_test "$tfile"
test_case "Invalid file, contains multiple words"
echo "Hello world!" >"$tfile"
required_error EINVAL <<EOF
-ctdb_tunable_load_file: Invalid line containing "Hello world!"
+Loading tunables from ${tfile}
+${tfile}: Invalid tunables line containing "Hello world!"
EOF
unit_test tunable_test "$tfile"
test_case "Invalid file, missing value"
echo "EnableBans=" >"$tfile"
required_error EINVAL <<EOF
-ctdb_tunable_load_file: Invalid line containing "EnableBans"
+Loading tunables from ${tfile}
+${tfile}: Invalid tunables line containing "EnableBans"
EOF
unit_test tunable_test "$tfile"
test_case "Invalid file, invalid value (not a number)"
echo "EnableBans=value" >"$tfile"
required_error EINVAL <<EOF
-ctdb_tunable_load_file: Invalid value "value" for tunable "EnableBans"
+Loading tunables from ${tfile}
+${tfile}: Invalid value "value" for tunable "EnableBans"
EOF
unit_test tunable_test "$tfile"
test_case "Invalid file, missing key"
echo "=123" >"$tfile"
required_error EINVAL <<EOF
-ctdb_tunable_load_file: Syntax error
+Loading tunables from ${tfile}
+${tfile}: Syntax error
EOF
unit_test tunable_test "$tfile"
@@ -150,28 +219,32 @@ cat >"$tfile" <<EOF
=0
--
Samba Shared Repository
