The branch, v4-22-test has been updated
via baea7672858 CVE-2025-0620: smbd: smbd doesn't pick up group
membership changes when re-authenticating an expired SMB session
via 43698c834c3 smbd: use fsp->name_hash in check_parent_access_fsp()
via 0527b007031 smbd: remove parent_dirname_compatible_open()
via 4cc5ed62dd3 selftest: stop running smb2.streams against the
ad_dc_ntvfs
via 44811da2e66 selftest: stop running smb2.streams against "ad_dc"
environment
via 9b651c01be1 smbd: implement H-lease breaks on parent directory of
rename target
via 4874eb99e6e smbd: add access_mask to
delay_for_handle_lease_break_send()
via 2b8ac68790f smbd: add has_delete_access_opens()
via b171beb860c smbd: support breaking leases on an object where we
don't have an own internal open
via 4c62ed11087 smbd: expand logging in contend_dirleases()
via e23cddcc2e6 smbtorture: fix test smb2.notify-inotify.inotify-rename
via edd94c6d709 smbtorture: add test smb2.dirlease.rename_dst_parent
via 31f4023ce9f smbtorture: add support for closing a handle when
receiving a lease break
via 0939ddae86b smbtorture: make torture_lease_break_callback() static
via b58ffb3b149 smbtorture: remove unused torture_lease_ignore_handler()
via 0c47b4e7ca2 ctdb-daemon: Modernise some DEBUGs
via 64e92bc93d1 ctdb-daemon: Add configuration option shutdown extra
timeout
via 302af7a928e ctdb-daemon: Run "startipreallocate" event in SHUTDOWN
runstate
via 8b49433a41b ctdb-daemon: Add configuration option shutdown failover
timeout
via 9e4b88f800b ctdb-daemon: Add failover on shutdown
via ee3415e3c86 ctdb-protocol: Add CTDB server SRVID range
via b0a66c42704 ctdb-daemon: Avoid aborting during early shutdown
from 6f21f9527d6 s3:rpc_server: make sure we can bind to the same port
on all ip addresses
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-22-test
- Log -----------------------------------------------------------------
commit baea767285800bc96e2e965741ac4dc4377c9f32
Author: Ralph Boehme <[email protected]>
Date: Fri May 23 08:47:06 2025 +0200
CVE-2025-0620: smbd: smbd doesn't pick up group membership changes when
re-authenticating an expired SMB session
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15707
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Anoop C S <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
(cherry picked from commit 6f0ae60428a024b4aba92a8103a698c1eca2357c)
Autobuild-User(v4-22-test): Jule Anger <[email protected]>
Autobuild-Date(v4-22-test): Thu Jun 5 11:55:52 UTC 2025 on atb-devel-224
commit 43698c834c39c7ddf25d39cfe426c3d34c8d4048
Author: Ralph Boehme <[email protected]>
Date: Sat May 24 11:47:37 2025 +0200
smbd: use fsp->name_hash in check_parent_access_fsp()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
Autobuild-User(master): Ralph Böhme <[email protected]>
Autobuild-Date(master): Wed May 28 16:03:59 UTC 2025 on atb-devel-224
(backported from commit 9b9fc589e55d467c97fd4580c2d6d9aa8cb73b13)
[[email protected]: removed write_time arg to get_file_infos() in master]
commit 0527b0070319046f32a5594aea5c0a9c9947e883
Author: Ralph Boehme <[email protected]>
Date: Thu May 22 17:52:28 2025 +0200
smbd: remove parent_dirname_compatible_open()
This is now handled correctly smbd_smb2_setinfo_rename_dst_parent_check().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 17ed8c0f9a0ab8b4b7feb213b4e3a0765f8cc0cd)
commit 4cc5ed62dd340fc718843befcf7d0a1fccf1d8b5
Author: Ralph Boehme <[email protected]>
Date: Mon May 26 12:32:16 2025 +0200
selftest: stop running smb2.streams against the ad_dc_ntvfs
This will soon start failing with
UNEXPECTED(failure): samba4.smb2.streams.rename2(ad_dc_ntvfs)
REASON: Exception: Exception: ../../source4/torture/smb2/streams.c:1533:
status was NT_STATUS_OBJECT_NAME_COLLISION, expected NT_STATUS_OK: CHECK_STATUS
and I don't see the point in tracking this down for a dead product.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 69a7d5881bd0d91d270b4a8be4c75dcd94f19897)
commit 44811da2e66272412967886623762008036c4690
Author: Ralph Boehme <[email protected]>
Date: Mon May 26 11:18:57 2025 +0200
selftest: stop running smb2.streams against "ad_dc" environment
Drop running smb2.streams tests against the "ad_dc" environment, to simplify
test failure handling with the knownfail file.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 8c0888a94bbcd55b5746305ab82c9ce50095db32)
commit 9b651c01be1add9dbee06a6bc2ca9022247d4408
Author: Ralph Boehme <[email protected]>
Date: Thu May 22 11:42:13 2025 +0200
smbd: implement H-lease breaks on parent directory of rename target
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 91b7a5e7ac308040bd60a172280e4429ee25f3b0)
commit 4874eb99e6e62b4289eece67db157b16196a32cb
Author: Ralph Boehme <[email protected]>
Date: Fri May 23 17:06:50 2025 +0200
smbd: add access_mask to delay_for_handle_lease_break_send()
No change in behaviour, existing caller all pass SEC_RIGHTS_DIR_ALL.
Prepares
for selectively breaking only H-leases if the access_mask of the
corresponding
open contains DELETE_ACCESS.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 5d4565c1f974b75a1c080f4503613201ecaf7001)
commit 2b8ac68790ff3b8d2ce5bcdc509bf3417b6f9dcb
Author: Ralph Boehme <[email protected]>
Date: Wed May 21 19:17:54 2025 +0200
smbd: add has_delete_access_opens()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 1351b613679acb063d7ef8bc63d438e1ec973a9a)
commit b171beb860c6c70e009a6f0b402b34ca601935ae
Author: Ralph Boehme <[email protected]>
Date: Thu May 22 16:57:49 2025 +0200
smbd: support breaking leases on an object where we don't have an own
internal open
So far, when dealing with the rename destination, we had an internal open on
that which ensured get_existing_share_mode_lock() would always return a
lock.
Subsequently I'll want delay_for_handle_lease_break_send() to be callable
on an
object that doesn't have a full internal open including locking.tdb entry,
but
merely an open handle from filename_convert_dirfsp().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 03c46342e2a65e6d81bd581471310c0ec2cbbdfb)
commit 4c62ed110876382d252180068d1a83e7e6bb58b4
Author: Ralph Boehme <[email protected]>
Date: Fri May 23 16:35:35 2025 +0200
smbd: expand logging in contend_dirleases()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 509081e7ed064899701a9e53b1597c33bcf5b77d)
commit e23cddcc2e6333c7ebab5bcb3d1d12bb5b9a470c
Author: Ralph Boehme <[email protected]>
Date: Fri May 23 07:26:53 2025 +0200
smbtorture: fix test smb2.notify-inotify.inotify-rename
Need to remove SEC_STD_DELETE from the access mask, otherwise we can't move
files into this directory.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 8b346857b837259c017b47cb6a935ed54afc8c60)
commit edd94c6d70958e598bbda553cfc878b3663f6f3e
Author: Ralph Boehme <[email protected]>
Date: Thu May 22 16:56:45 2025 +0200
smbtorture: add test smb2.dirlease.rename_dst_parent
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit a2827f4d6d8b56de08adaee35a051022f255769e)
commit 31f4023ce9ffb9a682cdbec540417432221f12a9
Author: Ralph Boehme <[email protected]>
Date: Thu May 22 17:50:46 2025 +0200
smbtorture: add support for closing a handle when receiving a lease break
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit d88885b77cd9d86969eac2cd2010ed31e329106a)
commit 0939ddae86b7a66421c864ddb909a877bba9bc92
Author: Ralph Boehme <[email protected]>
Date: Thu May 22 17:50:09 2025 +0200
smbtorture: make torture_lease_break_callback() static
It's only used in this compilation unit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit 56fe5e8ef192f38b928eed9b454709242f02699e)
commit b58ffb3b149c95a283b8828c36f07a1bff671dfd
Author: Ralph Boehme <[email protected]>
Date: Thu May 22 17:49:31 2025 +0200
smbtorture: remove unused torture_lease_ignore_handler()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15861
Signed-off-by: Ralph Boehme <[email protected]>
Reviewed-by: Bjoern Jacke <[email protected]>
(cherry picked from commit e1a38cd3f9f5665c9f7dd202fec1c7ec72fa419c)
commit 0c47b4e7ca21ef47aef43d13566bde678dc2740f
Author: Martin Schwenke <[email protected]>
Date: Mon May 19 10:06:21 2025 +1000
ctdb-daemon: Modernise some DEBUGs
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15858
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
Autobuild-User(master): Martin Schwenke <[email protected]>
Autobuild-Date(master): Thu May 29 10:57:35 UTC 2025 on atb-devel-224
(cherry picked from commit 5a582bddd834fffe2b27cc8b2e9468fa84dfc6f2)
commit 64e92bc93d1b42f0aeb937c75abbbe133d7d3d84
Author: Martin Schwenke <[email protected]>
Date: Mon May 19 09:06:38 2025 +1000
ctdb-daemon: Add configuration option shutdown extra timeout
See documentation change for details.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15858
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
(cherry picked from commit 3a770c8d46934870f42059640b0aaa0c76a3f4fb)
commit 302af7a928e41ff8fbe73dbf076dd55c48e9890d
Author: Martin Schwenke <[email protected]>
Date: Thu May 15 14:01:16 2025 +1000
ctdb-daemon: Run "startipreallocate" event in SHUTDOWN runstate
Even though all nodes may be shutting down there is still a very small
window for a race when multiple nodes are shut down. For simplicity,
assume 2 nodes. Assume the shutdowns of nodes are staggered, which is
usual because they're usually initiated by a loop (e.g. onnode -p all
ctdb shutdown). Although commands can continue in parallel, some
commands are started later than others.
Consider this sequence:
1. Node 0 reaches ctdb_shutdown_takeover() in
ctdb_shutdown_sequence() and a takeover run starts
2. Node 1 has not yet set its runlevel to SHUTDOWN in
ctdb_shutdown_sequence()
3. The leader node asks node 1 which IPs it can host
4. Node 1 replies "all of them"
5. Node 1 now sets its runlevel to SHUTDOWN in
ctdb_shutdown_sequence()
6. The leader node continues with the takeover run, first asking all
nodes to run "startipreallocate"
7. Node 0 runs "startipreallocate", so its NFS server starts grace
8. Node 1 does not run "startipreallocate" because it is not in
RUNNING runstate, so its NFS server does not start grace
9. The leader node continues with the takeover run, first asking all
nodes to run "releaseip" for IPs they can no longer hold
10. Node 0 releases all IPs, since it is SHUTDOWN runstate (so can't
host IPs)
11. As part of this, the NFS server on node 0 releases locks held
against IPs it is releasing
12. A client connected to node 1, where the NFS server is not in
grace, takes ("steals") one of those locks
This client is then permitted to reclaim the lock when nodes are
restarted.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15858
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
(cherry picked from commit 4877541cfd8f782f516f6471edc52629720963fb)
commit 8b49433a41bcc3ecfbce562363d0e2a3b07ce4ec
Author: Martin Schwenke <[email protected]>
Date: Mon May 12 12:00:28 2025 +1000
ctdb-daemon: Add configuration option shutdown failover timeout
Allows the timeout for failover during shutdown to be modified.
Defaults to 10s.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15858
Signed-off-by: Martin Schwenke <[email protected]>
SQ
Reviewed-by: Amitay Isaacs <[email protected]>
(cherry picked from commit dd9b73119afd3a0c60c87c938b5aefc766ca78d2)
commit 9e4b88f800bd838dacbf769b4319554ac391a3a2
Author: Martin Schwenke <[email protected]>
Date: Mon May 12 11:33:19 2025 +1000
ctdb-daemon: Add failover on shutdown
Without this, NFS servers on other nodes will not go into grace before
this node releases locks. This should also support improved behaviour
for SMB durable file handles.
The timeout is currently a constant 10s. However, it will
subsequently be switched to an option.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15858
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
(cherry picked from commit b84fbd7b3fedc998633400981ce0c5dc963d052e)
commit ee3415e3c86067cfc5bfe40fc35da6f88684c9ba
Author: Martin Schwenke <[email protected]>
Date: Wed May 14 16:55:51 2025 +1000
ctdb-protocol: Add CTDB server SRVID range
Normally, communication from other components to ctdbd is done via
controls. However, there are contexts where receiving SRVID messages
in ctdbd makes sense, such as replies to outgoing SRVID messages.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15858
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
(cherry picked from commit 631d1d38ad10c73aa559561bea6b5ed45c2226c4)
commit b0a66c42704b51329bee68cf49d9f9ce7de3b1d3
Author: Martin Schwenke <[email protected]>
Date: Wed May 21 22:17:42 2025 +1000
ctdb-daemon: Avoid aborting during early shutdown
An early shutdown can put ctdbd into SHUTDOWN runstate before ctdbd
has completed all early initialisation. Some of the start-time
transitions then attempt to set the runstate to FIRST_RECOVERY or
RUNNING, which would make the runstate go backwards, so ctdbd aborts.
Upcoming changes cause ctdbd shutdown to take longer, so the problem
will become more likely. With those changes, this can be
unreliably (50% of the time?) triggered by:
ctdb/tests/INTEGRATION/simple/cluster.091.version_check.sh
since it does an early shutdown due to a version mismatch.
Avoid this by noticing when the runstate is SHUTDOWN and refusing to
continue with subsequent early initialisation steps, which aren't
needed when shutting down.
Earlier runstate transitions do not seems likely to cause an abort
during early shutdown. The following:
./tests/local_daemons.sh foo start 0; ./tests/local_daemons.sh foo stop 0
sees ctdbd already into FIRST_RECOVERY before the shutdown is
processed.
The change to ctdb_run_startup() probably isn't strictly necessary.
There will be no abort in this case. ctdb_shutdown_sequence() will
always run the "shutdown" event and then stop the event daemon, so it
doesn't seem possible that services could be left running. However,
we might as well avoid running the "startup" event when shutting down,
even if only to avoid confusing logs.
Ultimately, it seems like some redesign would be needed to avoid this
in a more predictable manner, rather than responding when an early
initialisation step inconveniently completes during shutdown. For
example, hanging a lot of the start-time event handling off a common
talloc context, could allow it to be cancelled with a single
TALLOC_FREE(). However, a change like that would involve a lot of
analysis to ensure that the talloc hierarchy is correct and there is
no change of free'd pointers being dereferenced. So, we're probably
better off just keeping this issue in mind during a broader redesign.
This workaround appears to be sufficient.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15858
Signed-off-by: Martin Schwenke <[email protected]>
Reviewed-by: Amitay Isaacs <[email protected]>
(cherry picked from commit c03e6b9d50cac67fe33dc6b120996d1915331be6)
-----------------------------------------------------------------------
Summary of changes:
ctdb/conf/ctdb_config.c | 8 +
ctdb/conf/ctdb_config.h | 2 +
ctdb/conf/failover_conf.c | 12 ++
ctdb/conf/failover_conf.h | 3 +
ctdb/doc/ctdb.conf.5.xml | 50 +++++++
ctdb/protocol/protocol.h | 7 +
ctdb/server/ctdb_daemon.c | 229 ++++++++++++++++++++++++++++-
ctdb/server/ctdb_monitor.c | 18 +++
ctdb/server/ctdb_takeover.c | 5 +-
ctdb/tests/UNIT/cunit/config_test_001.sh | 2 +
selftest/knownfail | 3 -
source3/selftest/tests.py | 1 -
source3/smbd/close.c | 41 ++++++
source3/smbd/conn.c | 4 +-
source3/smbd/open.c | 11 +-
source3/smbd/proto.h | 5 +
source3/smbd/smb2_close.c | 1 +
source3/smbd/smb2_oplock.c | 32 ++--
source3/smbd/smb2_reply.c | 48 ------
source3/smbd/smb2_setinfo.c | 180 +++++++++++++++++++++++
source3/smbd/smbXsrv_session.c | 1 +
source4/selftest/tests.py | 1 +
source4/torture/smb2/lease.c | 110 ++++++++++++++
source4/torture/smb2/lease_break_handler.c | 44 ++++--
source4/torture/smb2/lease_break_handler.h | 7 +-
source4/torture/smb2/notify.c | 6 +-
source4/torture/smb2/streams.c | 20 ++-
27 files changed, 742 insertions(+), 109 deletions(-)
Changeset truncated at 500 lines:
diff --git a/ctdb/conf/ctdb_config.c b/ctdb/conf/ctdb_config.c
index f75bf374a80..d9f6f3a5457 100644
--- a/ctdb/conf/ctdb_config.c
+++ b/ctdb/conf/ctdb_config.c
@@ -110,6 +110,14 @@ static void setup_config_pointers(struct conf_context
*conf)
FAILOVER_CONF_SECTION,
FAILOVER_CONF_DISABLED,
&ctdb_config.failover_disabled);
+ conf_assign_integer_pointer(conf,
+ FAILOVER_CONF_SECTION,
+ FAILOVER_CONF_SHUTDOWN_EXTRA_TIMEOUT,
+ &ctdb_config.shutdown_extra_timeout);
+ conf_assign_integer_pointer(conf,
+ FAILOVER_CONF_SECTION,
+ FAILOVER_CONF_SHUTDOWN_FAILOVER_TIMEOUT,
+ &ctdb_config.shutdown_failover_timeout);
/*
* Legacy
diff --git a/ctdb/conf/ctdb_config.h b/ctdb/conf/ctdb_config.h
index 575e3045fa4..01195a4c9f5 100644
--- a/ctdb/conf/ctdb_config.h
+++ b/ctdb/conf/ctdb_config.h
@@ -44,6 +44,8 @@ struct ctdb_config {
/* Failover */
bool failover_disabled;
+ int shutdown_extra_timeout;
+ int shutdown_failover_timeout;
/* Legacy */
bool realtime_scheduling;
diff --git a/ctdb/conf/failover_conf.c b/ctdb/conf/failover_conf.c
index 3f9f749fcae..424021b7a22 100644
--- a/ctdb/conf/failover_conf.c
+++ b/ctdb/conf/failover_conf.c
@@ -50,4 +50,16 @@ void failover_conf_init(struct conf_context *conf)
FAILOVER_CONF_DISABLED,
false,
check_static_boolean_change);
+
+ conf_define_integer(conf,
+ FAILOVER_CONF_SECTION,
+ FAILOVER_CONF_SHUTDOWN_EXTRA_TIMEOUT,
+ 0,
+ NULL);
+
+ conf_define_integer(conf,
+ FAILOVER_CONF_SECTION,
+ FAILOVER_CONF_SHUTDOWN_FAILOVER_TIMEOUT,
+ 10,
+ NULL);
}
diff --git a/ctdb/conf/failover_conf.h b/ctdb/conf/failover_conf.h
index d7ac0ac507d..08f5fb8939c 100644
--- a/ctdb/conf/failover_conf.h
+++ b/ctdb/conf/failover_conf.h
@@ -25,6 +25,9 @@
#define FAILOVER_CONF_SECTION "failover"
#define FAILOVER_CONF_DISABLED "disabled"
+#define FAILOVER_CONF_SHUTDOWN_EXTRA_TIMEOUT "shutdown extra timeout"
+#define FAILOVER_CONF_SHUTDOWN_FAILOVER_TIMEOUT "shutdown failover timeout"
+
void failover_conf_init(struct conf_context *conf);
diff --git a/ctdb/doc/ctdb.conf.5.xml b/ctdb/doc/ctdb.conf.5.xml
index 7bdbc038f7d..048e02196bb 100644
--- a/ctdb/doc/ctdb.conf.5.xml
+++ b/ctdb/doc/ctdb.conf.5.xml
@@ -496,6 +496,56 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>shutdown extra timeout = <parameter>TIMEOUT</parameter></term>
+ <listitem>
+ <para>
+ CTDB will wait for TIMEOUT seconds after failover
+ completes during shutdown. This can provide extra time
+ for SMB durable handles to be reclaimed. If set to 0 then
+ no extra timeout occurs.
+ </para>
+ <para>
+ This timeout only occurs if both of the following
+ conditions are true:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ shutdown failover timeout (below) is not 0
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Failover during shutdown completes and does not time out
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ Default: <literal>0</literal>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>shutdown failover timeout = <parameter>TIMEOUT</parameter></term>
+ <listitem>
+ <para>
+ CTDB will wait for TIMEOUT seconds for failover to
+ complete during shutdown. This allows NFS servers on
+ other nodes to go into grace during graceful shutdown of a
+ node. Failover during shutdown also helps with SMB
+ durable handle reclaim.
+ </para>
+ <para>
+ Set this to 0 to disable explicit failover on shutdown.
+ </para>
+ <para>
+ Default: <literal>10</literal>
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</refsect1>
diff --git a/ctdb/protocol/protocol.h b/ctdb/protocol/protocol.h
index c775c4bcc64..ecec0a45891 100644
--- a/ctdb/protocol/protocol.h
+++ b/ctdb/protocol/protocol.h
@@ -234,6 +234,13 @@ struct ctdb_call {
#define CTDB_SRVID_TEST_RANGE 0xAE00000000000000LL
+/* Range of ports reserved for CTDB server (top 8 bits)
+ * All ports matching the 8 top bits are reserved for exclusive use by
+ * the CTDB server
+ */
+#define CTDB_SRVID_SERVER_RANGE 0x9E00000000000000LL
+
+
enum ctdb_controls {CTDB_CONTROL_PROCESS_EXISTS = 0,
CTDB_CONTROL_STATISTICS = 1,
/* #2 removed */
diff --git a/ctdb/server/ctdb_daemon.c b/ctdb/server/ctdb_daemon.c
index 46bc324ae87..3bdf9fee6f5 100644
--- a/ctdb/server/ctdb_daemon.c
+++ b/ctdb/server/ctdb_daemon.c
@@ -23,6 +23,7 @@
#include "system/wait.h"
#include "system/time.h"
+#include <errno.h>
#include <talloc.h>
/* Allow use of deprecated function tevent_loop_allow_nesting() */
#define TEVENT_DEPRECATED
@@ -41,6 +42,7 @@
#include "ctdb_client.h"
#include "protocol/protocol.h"
+#include "protocol/protocol_basic.h"
#include "protocol/protocol_api.h"
#include "common/rb_tree.h"
@@ -50,7 +52,9 @@
#include "common/logging.h"
#include "common/pidfile.h"
#include "common/sock_io.h"
+#include "common/srvid.h"
+#include "conf/ctdb_config.h"
#include "conf/node.h"
struct ctdb_client_pid_list {
@@ -2219,15 +2223,234 @@ done:
return ret;
}
+/*
+ * Construct a SRVID for accepting replies to this ctdbd. The bottom
+ * 24 bits of the PNN are used in the top half. extra_mask is used in
+ * the bottom half.
+ */
+
+static uint64_t ctdb_srvid_id(struct ctdb_context *ctdb, uint32_t extra_mask)
+{
+ uint64_t pnn_mask = (uint64_t)(ctdb->pnn & 0xFFFFFF) << 32;
+
+ return CTDB_SRVID_SERVER_RANGE | pnn_mask | extra_mask;
+}
+
+/*
+ * Do a takeover run on shutdown
+ *
+ * This allows for a graceful transition of resources to another node.
+ * This ensures all nodes go into grace for NFS and, with an extra
+ * timeout, allows data transfer for SMB durable handles.
+ *
+ * Nodes need to be in CTDB_RUNSTATE_RUNNING to host public IP
+ * addresses. So, this node will release all IPs. The good news is
+ * that a node can remain leader when in CTDB_RUNSTATE_SHUTDOWN, so
+ * shutting down the cluster will not be adversely delayed by this.
+ * The only issue to guard against is delaying shutdown of this node
+ * if it is the only node and doesn't have CTDB_CAP_RECMASTER, in
+ * which case there is no node to do the takeover run. Hence, the
+ * timeout.
+ */
+
+struct shutdown_takeover_state {
+ bool takeover_done;
+ bool timed_out;
+ struct tevent_timer *te;
+ unsigned int leader_broadcast_count;
+};
+
+static void shutdown_takeover_handler(uint64_t srvid,
+ TDB_DATA data,
+ void *private_data)
+{
+ struct shutdown_takeover_state *state = private_data;
+ int32_t result = 0;
+ size_t count = 0;
+ int ret = 0;
+
+ ret = ctdb_int32_pull(data.dptr, data.dsize, &result, &count);
+ if (ret == EMSGSIZE) {
+ /*
+ * Can't happen unless there's bug somewhere else, so
+ * just ignore - ctdb_shutdown_takeover() will
+ * probably time out...
+ */
+ DBG_WARNING("Wrong size for result\n");
+ return;
+ }
+
+ if (result == -1) {
+ /*
+ * No early return - can't afford endless retries
+ * during shutdown...
+ */
+ DBG_WARNING("Takeover run failed\n");
+ } else {
+ DBG_NOTICE("Takeover run successful by node=%"PRIi32"\n",
+ result);
+ }
+
+ state->takeover_done = true;
+}
+
+static void shutdown_timeout_handler(struct tevent_context *ev,
+ struct tevent_timer *te,
+ struct timeval yt,
+ void *private_data)
+{
+ struct shutdown_takeover_state *state = private_data;
+
+ TALLOC_FREE(state->te);
+ state->timed_out = true;
+}
+
+static void shutdown_leader_handler(uint64_t srvid,
+ TDB_DATA data,
+ void *private_data)
+{
+ struct shutdown_takeover_state *state = private_data;
+ uint32_t pnn = 0;
+ size_t count = 0;
+ int ret = 0;
+
+ ret = ctdb_uint32_pull(data.dptr, data.dsize, &pnn, &count);
+ if (ret == EMSGSIZE) {
+ /*
+ * Can't happen unless there's bug somewhere else, so
+ * just ignore
+ */
+ DBG_WARNING("Wrong size for result\n");
+ return;
+ }
+
+ DBG_DEBUG("Leader broadcast received from node=%"PRIu32"\n", pnn);
+ state->leader_broadcast_count++;
+}
+
+static void ctdb_shutdown_takeover(struct ctdb_context *ctdb)
+{
+ struct shutdown_takeover_state state = {
+ .takeover_done = false,
+ .timed_out = false,
+ .te = NULL,
+ .leader_broadcast_count = 0,
+ };
+ /*
+ * This one is memcpy()ed onto the wire, so initialise below
+ * after ZERO_STRUCT(), to keep things valgrind clean
+ */
+ struct ctdb_srvid_message rd;
+ struct TDB_DATA rddata = {
+ .dptr = (uint8_t *)&rd,
+ .dsize = sizeof(rd),
+ };
+ int ret = 0;
+
+ if (ctdb_config.shutdown_failover_timeout <= 0) {
+ return;
+ }
+
+ ZERO_STRUCT(rd);
+ rd = (struct ctdb_srvid_message) {
+ .pnn = ctdb->pnn,
+ .srvid = ctdb_srvid_id(ctdb, 0),
+ };
+
+ ret = srvid_register(ctdb->srv,
+ ctdb->srv,
+ rd.srvid,
+ shutdown_takeover_handler,
+ &state);
+ if (ret != 0) {
+ DBG_WARNING("Failed to register takeover run handler\n");
+ return;
+ }
+
+ state.te = tevent_add_timer(
+ ctdb->ev,
+ ctdb->srv,
+ timeval_current_ofs(ctdb_config.shutdown_failover_timeout, 0),
+ shutdown_timeout_handler,
+ &state);
+ if (state.te == NULL) {
+ DBG_WARNING("Failed to set shutdown timeout\n");
+ goto done;
+ }
+
+ ret = srvid_register(ctdb->srv,
+ ctdb->srv,
+ CTDB_SRVID_LEADER,
+ shutdown_leader_handler,
+ &state);
+ if (ret != 0) {
+ /* Leader broadcasts provide extra information, so no
+ * problem if they can't be monitored...
+ */
+ DBG_WARNING("Failed to register leader handler\n");
+ }
+
+ ret = ctdb_daemon_send_message(ctdb,
+ CTDB_BROADCAST_CONNECTED,
+ CTDB_SRVID_TAKEOVER_RUN,
+ rddata);
+ if (ret != 0) {
+ DBG_WARNING("Failed to send IP takeover run request\n");
+ goto done;
+ }
+
+ while (!state.takeover_done && !state.timed_out) {
+ tevent_loop_once(ctdb->ev);
+ }
+
+ if (state.takeover_done) {
+ goto done;
+ }
+
+ if (state.timed_out) {
+ DBG_WARNING("Timed out waiting for takeover run "
+ "(%u leader broadcasts received)\n",
+ state.leader_broadcast_count);
+ }
+done:
+ srvid_deregister(ctdb->srv, CTDB_SRVID_TAKEOVER_RUN, &state);
+ srvid_deregister(ctdb->srv, CTDB_SRVID_LEADER, &state);
+ TALLOC_FREE(state.te);
+
+ if (!state.takeover_done || ctdb_config.shutdown_extra_timeout <= 0) {
+ return;
+ }
+
+ state.timed_out = false;
+ state.te = tevent_add_timer(
+ ctdb->ev,
+ ctdb->srv,
+ timeval_current_ofs(ctdb_config.shutdown_extra_timeout, 0),
+ shutdown_timeout_handler,
+ &state);
+ if (state.te == NULL) {
+ DBG_WARNING("Failed to set extra timeout\n");
+ return;
+ }
+
+ DBG_NOTICE("Waiting %ds for shutdown extra timeout\n",
+ ctdb_config.shutdown_extra_timeout);
+ while (!state.timed_out) {
+ tevent_loop_once(ctdb->ev);
+ }
+ DBG_INFO("shutdown extra timeout complete\n");
+}
+
void ctdb_shutdown_sequence(struct ctdb_context *ctdb, int exit_code)
{
if (ctdb->runstate == CTDB_RUNSTATE_SHUTDOWN) {
- DEBUG(DEBUG_NOTICE,("Already shutting down so will not
proceed.\n"));
+ D_NOTICE("Already shutting down so will not proceed.\n");
return;
}
- DEBUG(DEBUG_ERR,("Shutdown sequence commencing.\n"));
+ D_ERR("Shutdown sequence commencing.\n");
ctdb_set_runstate(ctdb, CTDB_RUNSTATE_SHUTDOWN);
+ ctdb_shutdown_takeover(ctdb);
ctdb_stop_recoverd(ctdb);
ctdb_stop_keepalive(ctdb);
ctdb_stop_monitoring(ctdb);
@@ -2237,7 +2460,7 @@ void ctdb_shutdown_sequence(struct ctdb_context *ctdb,
int exit_code)
ctdb->methods->shutdown(ctdb);
}
- DEBUG(DEBUG_ERR,("Shutdown sequence complete, exiting.\n"));
+ D_ERR("Shutdown sequence complete, exiting.\n");
exit(exit_code);
}
diff --git a/ctdb/server/ctdb_monitor.c b/ctdb/server/ctdb_monitor.c
index ab58ec485fe..869a589e6e5 100644
--- a/ctdb/server/ctdb_monitor.c
+++ b/ctdb/server/ctdb_monitor.c
@@ -217,6 +217,11 @@ static void ctdb_run_startup(struct tevent_context *ev,
*/
static void ctdb_startup_callback(struct ctdb_context *ctdb, int status, void
*p)
{
+ if (ctdb->runstate == CTDB_RUNSTATE_SHUTDOWN) {
+ DBG_WARNING("Detected early shutdown, not starting
monitoring\n");
+ return;
+ }
+
if (status != 0) {
DEBUG(DEBUG_ERR,("startup event failed\n"));
tevent_add_timer(ctdb->ev, ctdb->monitor->monitor_context,
@@ -249,6 +254,12 @@ static void ctdb_run_startup(struct tevent_context *ev,
struct ctdb_context);
int ret;
+ if (ctdb->runstate == CTDB_RUNSTATE_SHUTDOWN) {
+ DBG_WARNING(
+ "Detected early shutdown, not running startup event\n");
+ return;
+ }
+
/* This is necessary to avoid the "startup" event colliding
* with the "ipreallocated" event from the takeover run
* following the first recovery. We might as well serialise
@@ -432,6 +443,13 @@ void ctdb_stop_monitoring(struct ctdb_context *ctdb)
*/
void ctdb_wait_for_first_recovery(struct ctdb_context *ctdb)
{
+ if (ctdb->runstate == CTDB_RUNSTATE_SHUTDOWN) {
+ DBG_WARNING(
+ "Detected early shutdown, "
+ "not waiting for first recovery\n");
+ return;
+ }
+
ctdb_set_runstate(ctdb, CTDB_RUNSTATE_FIRST_RECOVERY);
ctdb->monitor = talloc(ctdb, struct ctdb_monitor_state);
diff --git a/ctdb/server/ctdb_takeover.c b/ctdb/server/ctdb_takeover.c
index dc4fcf395b7..60f60e29ffc 100644
--- a/ctdb/server/ctdb_takeover.c
+++ b/ctdb/server/ctdb_takeover.c
@@ -2519,8 +2519,9 @@ int32_t ctdb_control_start_ipreallocate(struct
ctdb_context *ctdb,
struct start_ipreallocate_callback_state *state;
/* Nodes that are not RUNNING can not host IPs */
- if (ctdb->runstate != CTDB_RUNSTATE_RUNNING) {
- DBG_INFO("Skipping \"startipreallocate\" event, not RUNNING\n");
+ if (ctdb->runstate < CTDB_RUNSTATE_RUNNING) {
+ DBG_INFO("Skipping \"startipreallocate\" event, "
+ "not RUNNING/SHUTDOWN\n");
return 0;
}
diff --git a/ctdb/tests/UNIT/cunit/config_test_001.sh
b/ctdb/tests/UNIT/cunit/config_test_001.sh
index 9e7ac462618..b4d73a993aa 100755
--- a/ctdb/tests/UNIT/cunit/config_test_001.sh
+++ b/ctdb/tests/UNIT/cunit/config_test_001.sh
@@ -49,6 +49,8 @@ ok <<EOF
# debug script =
[failover]
# disabled = false
+ # shutdown extra timeout = 0
+ # shutdown failover timeout = 10
[legacy]
# realtime scheduling = true
# lmaster capability = true
diff --git a/selftest/knownfail b/selftest/knownfail
index 51aae99d6b4..103a0bb1d76 100644
--- a/selftest/knownfail
+++ b/selftest/knownfail
@@ -204,11 +204,8 @@
^samba4.smb2.ioctl-on-stream.ioctl-on-stream\(ad_dc_ntvfs\)
--
Samba Shared Repository
