The branch, master has been updated via abf867d inotify: Properly handle cross-dir renames via d6e2db6 lib: Fix signed/unsigned comparisons via accb6b4 s4: torture: leases. Simple lease_v2 rename test "v2_rename". via f435f1b s3: leases: Make SMB2 setinfo SMB2_FILE_RENAME_INFORMATION_INTERNAL async. via bddd600 s3: leases: send_break_message() public. via cd6269c s3: leases: Make aio_add_req_to_fsp() public. via 4061b8d s3: leases : Cope with renaming leased open files. via 59cd638 s3: leases: Add leases_db_rename() to cope with renaming a leased file. from 959b9ea ctdb-recoverd: Process all the records for vacuum fetch in a loop
https://git.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit abf867da764b60e94c43ab9ec1d5338ac99ec18d Author: Volker Lendecke <v...@samba.org> Date: Fri Dec 5 15:38:45 2014 +0000 inotify: Properly handle cross-dir renames When watching two subdirectories with inotify and a file is moved between both, we get a IN_MOVED_FROM for the source watch and a IN_MOVED_TO for the destination watch. Without this patch we create a NOTIFY_ACTION_OLD_NAME for the old directory. We hold this back in notify_fsp, expecting the NEW_NAME immediately after it. In the cross-directory rename case this does not work, we'll not get the NEW_NAME, there is no NEW_NAME in that directory. This patch changes us to create NOTIFY_ACTION_REMOVED and NOTIFY_ACTION_ADDED in this case. Not sure this is right, but at least it is better than before: We get something at all. This is more likely to happen with the notifyd approach, as there we inotify-watch many subdirectories from one process. Without nootifyd you had to have two explorer windows open and do a nfs or local mv between those two directories to find this. Signed-off-by: Volker Lendecke <v...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> Autobuild-User(master): Jeremy Allison <j...@samba.org> Autobuild-Date(master): Fri Dec 5 21:01:54 CET 2014 on sn-devel-104 commit d6e2db6f52fa26ae3c044ff56ebf814313ca4eb8 Author: Volker Lendecke <v...@samba.org> Date: Wed Dec 3 15:54:19 2014 +0100 lib: Fix signed/unsigned comparisons Signed-off-by: Volker Lendecke <v...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> commit accb6b4fb8184ebe176dce76bda6b1f02584b99e Author: Jeremy Allison <j...@samba.org> Date: Thu Dec 4 21:29:47 2014 -0800 s4: torture: leases. Simple lease_v2 rename test "v2_rename". Proves that renaming files can break handle leases. With the previous patches we now pass this. Pair-Programmed-With: Stefan Metzmacher <me...@samba.org> Signed-off-by: Jeremy Allison <j...@samba.org> Signed-off-by: Stefan Metzmacher <me...@samba.org> commit f435f1b3acb75c065166e3077c01acbd88601f34 Author: Jeremy Allison <j...@samba.org> Date: Thu Dec 4 21:19:32 2014 -0800 s3: leases: Make SMB2 setinfo SMB2_FILE_RENAME_INFORMATION_INTERNAL async. If there are any RH leases we must break them to read and must wait for the client response before doing the rename. Pair-Programmed-With: Stefan Metzmacher <me...@samba.org> Signed-off-by: Jeremy Allison <j...@samba.org> Signed-off-by: Stefan Metzmacher <me...@samba.org> commit bddd6004ee400cf90d08d174e9fb867a129433e7 Author: Jeremy Allison <j...@samba.org> Date: Thu Dec 4 21:15:24 2014 -0800 s3: leases: send_break_message() public. We're going to need this to allow async SMB2 setinfo renames to send lease break messages as well as the open code. Signed-off-by: Jeremy Allison <j...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> commit cd6269c9ea601e89bf9cd8b6c73f79cf594b8c7f Author: Jeremy Allison <j...@samba.org> Date: Thu Dec 4 21:13:33 2014 -0800 s3: leases: Make aio_add_req_to_fsp() public. We're going to need this to stop handle closures with outstanding async SMB2 renames causing a crash. Signed-off-by: Jeremy Allison <j...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 4061b8db1bbdb53a0755277b1f0ceeccb6dc7eaa Author: Jeremy Allison <j...@samba.org> Date: Thu Dec 4 10:14:23 2014 -0800 s3: leases : Cope with renaming leased open files. Signed-off-by: Jeremy Allison <j...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 59cd638a1d241fae037d45e9ed1f029f65ac9080 Author: Jeremy Allison <j...@samba.org> Date: Thu Dec 4 10:13:47 2014 -0800 s3: leases: Add leases_db_rename() to cope with renaming a leased file. Signed-off-by: Jeremy Allison <j...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> ----------------------------------------------------------------------- Summary of changes: lib/util/data_blob.c | 4 +- lib/util/util.c | 2 +- source3/locking/leases_db.c | 22 +++++ source3/locking/leases_db.h | 6 +- source3/locking/locking.c | 25 ++++- source3/smbd/aio.c | 2 +- source3/smbd/notify_inotify.c | 10 +- source3/smbd/open.c | 2 +- source3/smbd/proto.h | 4 + source3/smbd/smb2_setinfo.c | 218 ++++++++++++++++++++++++++++++++++++++++++ source4/torture/smb2/lease.c | 134 ++++++++++++++++++++++++++ 11 files changed, 419 insertions(+), 10 deletions(-) Changeset truncated at 500 lines: diff --git a/lib/util/data_blob.c b/lib/util/data_blob.c index 1b0e6ab..4723669 100644 --- a/lib/util/data_blob.c +++ b/lib/util/data_blob.c @@ -135,7 +135,7 @@ print the data_blob as hex string **/ _PUBLIC_ char *data_blob_hex_string_lower(TALLOC_CTX *mem_ctx, const DATA_BLOB *blob) { - int i; + size_t i; char *hex_string; hex_string = talloc_array(mem_ctx, char, (blob->length*2)+1); @@ -155,7 +155,7 @@ _PUBLIC_ char *data_blob_hex_string_lower(TALLOC_CTX *mem_ctx, const DATA_BLOB * _PUBLIC_ char *data_blob_hex_string_upper(TALLOC_CTX *mem_ctx, const DATA_BLOB *blob) { - int i; + size_t i; char *hex_string; hex_string = talloc_array(mem_ctx, char, (blob->length*2)+1); diff --git a/lib/util/util.c b/lib/util/util.c index 157a4aa..562f7df 100644 --- a/lib/util/util.c +++ b/lib/util/util.c @@ -732,7 +732,7 @@ _PUBLIC_ void dump_data_pw(const char *msg, const uint8_t * data, size_t len) */ _PUBLIC_ bool all_zero(const uint8_t *ptr, size_t size) { - int i; + size_t i; if (!ptr) return true; for (i=0;i<size;i++) { if (ptr[i]) return false; diff --git a/source3/locking/leases_db.c b/source3/locking/leases_db.c index 67c93ff..7e000aa 100644 --- a/source3/locking/leases_db.c +++ b/source3/locking/leases_db.c @@ -385,3 +385,25 @@ NTSTATUS leases_db_parse(const struct GUID *client_guid, } return state.status; } + +NTSTATUS leases_db_rename(const struct GUID *client_guid, + const struct smb2_lease_key *lease_key, + const struct file_id *id, + const char *filename_new, + const char *stream_name_new) +{ + NTSTATUS status; + + status = leases_db_del(client_guid, + lease_key, + id); + if (!NT_STATUS_IS_OK(status)) { + return status; + } + + return leases_db_add(client_guid, + lease_key, + id, + filename_new, + stream_name_new); +} diff --git a/source3/locking/leases_db.h b/source3/locking/leases_db.h index f570356..906a99b 100644 --- a/source3/locking/leases_db.h +++ b/source3/locking/leases_db.h @@ -42,5 +42,9 @@ NTSTATUS leases_db_parse(const struct GUID *client_guid, const char *stream_name, void *private_data), void *private_data); - +NTSTATUS leases_db_rename(const struct GUID *client_guid, + const struct smb2_lease_key *lease_key, + const struct file_id *id, + const char *filename_new, + const char *stream_name_new); #endif /* _LEASES_DB_H_ */ diff --git a/source3/locking/locking.c b/source3/locking/locking.c index d144f5c..dd73f68 100644 --- a/source3/locking/locking.c +++ b/source3/locking/locking.c @@ -471,7 +471,7 @@ bool rename_share_filename(struct messaging_context *msg_ctx, size_t sn_len; size_t msg_len; char *frm = NULL; - int i; + uint32_t i; bool strip_two_chars = false; bool has_stream = smb_fname_dst->stream_name != NULL; struct server_id self_pid = messaging_server_id(msg_ctx); @@ -565,6 +565,29 @@ bool rename_share_filename(struct messaging_context *msg_ctx, (uint8 *)frm, msg_len); } + for (i=0; i<d->num_leases; i++) { + /* Update the filename in leases_db. */ + NTSTATUS status; + struct share_mode_lease *l; + + l = &d->leases[i]; + + status = leases_db_rename(&l->client_guid, + &l->lease_key, + &id, + d->base_name, + d->stream_name); + if (!NT_STATUS_IS_OK(status)) { + /* Any error recovery possible here ? */ + DEBUG(1,("Failed to rename lease key for " + "renamed file %s:%s. %s\n", + d->base_name, + d->stream_name, + nt_errstr(status))); + continue; + } + } + return True; } diff --git a/source3/smbd/aio.c b/source3/smbd/aio.c index 56b4cfc..e2306a9 100644 --- a/source3/smbd/aio.c +++ b/source3/smbd/aio.c @@ -115,7 +115,7 @@ static int aio_del_req_from_fsp(struct aio_req_fsp_link *lnk) return 0; } -static bool aio_add_req_to_fsp(files_struct *fsp, struct tevent_req *req) +bool aio_add_req_to_fsp(files_struct *fsp, struct tevent_req *req) { size_t array_len; struct aio_req_fsp_link *lnk; diff --git a/source3/smbd/notify_inotify.c b/source3/smbd/notify_inotify.c index efb659f..1fcd8ec 100644 --- a/source3/smbd/notify_inotify.c +++ b/source3/smbd/notify_inotify.c @@ -118,6 +118,7 @@ static bool filter_match(struct inotify_watch_context *w, */ static void inotify_dispatch(struct inotify_private *in, struct inotify_event *e, + int prev_wd, uint32_t prev_cookie, struct inotify_event *e2) { @@ -140,13 +141,14 @@ static void inotify_dispatch(struct inotify_private *in, } else if (e->mask & IN_DELETE) { ne.action = NOTIFY_ACTION_REMOVED; } else if (e->mask & IN_MOVED_FROM) { - if (e2 != NULL && e2->cookie == e->cookie) { + if (e2 != NULL && e2->cookie == e->cookie && + e2->wd == e->wd) { ne.action = NOTIFY_ACTION_OLD_NAME; } else { ne.action = NOTIFY_ACTION_REMOVED; } } else if (e->mask & IN_MOVED_TO) { - if (e->cookie == prev_cookie) { + if ((e->cookie == prev_cookie) && (e->wd == prev_wd)) { ne.action = NOTIFY_ACTION_NEW_NAME; } else { ne.action = NOTIFY_ACTION_ADDED; @@ -198,6 +200,7 @@ static void inotify_handler(struct tevent_context *ev, struct tevent_fd *fde, int bufsize = 0; struct inotify_event *e0, *e; uint32_t prev_cookie=0; + int prev_wd = -1; NTSTATUS status; /* @@ -234,7 +237,8 @@ static void inotify_handler(struct tevent_context *ev, struct tevent_fd *fde, if (bufsize >= sizeof(*e)) { e2 = (struct inotify_event *)(e->len + sizeof(*e) + (char *)e); } - inotify_dispatch(in, e, prev_cookie, e2); + inotify_dispatch(in, e, prev_wd, prev_cookie, e2); + prev_wd = e->wd; prev_cookie = e->cookie; e = e2; } diff --git a/source3/smbd/open.c b/source3/smbd/open.c index c6b67f4..8f19a36 100644 --- a/source3/smbd/open.c +++ b/source3/smbd/open.c @@ -1244,7 +1244,7 @@ static NTSTATUS open_mode_check(connection_struct *conn, * our client. */ -static NTSTATUS send_break_message(struct messaging_context *msg_ctx, +NTSTATUS send_break_message(struct messaging_context *msg_ctx, const struct share_mode_entry *exclusive, uint16_t break_to) { diff --git a/source3/smbd/proto.h b/source3/smbd/proto.h index e40a77b..9980d03 100644 --- a/source3/smbd/proto.h +++ b/source3/smbd/proto.h @@ -91,6 +91,7 @@ NTSTATUS schedule_aio_smb2_write(connection_struct *conn, DATA_BLOB in_data, bool write_through); bool cancel_smb2_aio(struct smb_request *smbreq); +bool aio_add_req_to_fsp(files_struct *fsp, struct tevent_req *req); /* The following definitions come from smbd/blocking.c */ @@ -615,6 +616,9 @@ NTSTATUS change_dir_owner_to_parent(connection_struct *conn, const char *fname, SMB_STRUCT_STAT *psbuf); bool is_stat_open(uint32 access_mask); +NTSTATUS send_break_message(struct messaging_context *msg_ctx, + const struct share_mode_entry *exclusive, + uint16_t break_to); struct deferred_open_record; bool is_deferred_open_async(const struct deferred_open_record *rec); NTSTATUS create_directory(connection_struct *conn, struct smb_request *req, diff --git a/source3/smbd/smb2_setinfo.c b/source3/smbd/smb2_setinfo.c index d95bd3d..e6981d1 100644 --- a/source3/smbd/smb2_setinfo.c +++ b/source3/smbd/smb2_setinfo.c @@ -25,6 +25,9 @@ #include "../libcli/smb/smb_common.h" #include "trans2.h" #include "../lib/util/tevent_ntstatus.h" +#include "../librpc/gen_ndr/open_files.h" +#include "source3/lib/dbwrap/dbwrap_watch.h" +#include "messages.h" static struct tevent_req *smbd_smb2_setinfo_send(TALLOC_CTX *mem_ctx, struct tevent_context *ev, @@ -156,6 +159,186 @@ static void smbd_smb2_request_setinfo_done(struct tevent_req *subreq) } } +struct defer_rename_state { + struct tevent_req *req; + struct smbd_smb2_request *smb2req; + struct tevent_context *ev; + struct files_struct *fsp; + char *data; + int data_size; +}; + +static void defer_rename_done(struct tevent_req *subreq); + +static struct tevent_req *delay_rename_for_lease_break(struct tevent_req *req, + struct smbd_smb2_request *smb2req, + struct tevent_context *ev, + struct files_struct *fsp, + struct share_mode_lock *lck, + char *data, + int data_size) + +{ + struct tevent_req *subreq; + uint32_t i; + struct share_mode_data *d = lck->data; + struct defer_rename_state *rename_state; + bool delay = false; + struct timeval timeout; + + if (fsp->oplock_type != LEASE_OPLOCK) { + return NULL; + } + + for (i=0; i<d->num_share_modes; i++) { + struct share_mode_entry *e = &d->share_modes[i]; + struct share_mode_lease *l = NULL; + uint32_t e_lease_type = get_lease_type(d, e); + uint32_t break_to; + + if (e->op_type != LEASE_OPLOCK) { + continue; + } + + l = &d->leases[e->lease_idx]; + + if (smb2_lease_equal(fsp_client_guid(fsp), + &fsp->lease->lease.lease_key, + &l->client_guid, + &l->lease_key)) { + continue; + } + + if (share_mode_stale_pid(d, i)) { + continue; + } + + if (!(e_lease_type & SMB2_LEASE_HANDLE)) { + continue; + } + + delay = true; + break_to = (e_lease_type & ~SMB2_LEASE_HANDLE); + + send_break_message(fsp->conn->sconn->msg_ctx, e, break_to); + } + + if (!delay) { + return NULL; + } + + /* Setup a watch on this record. */ + rename_state = talloc_zero(req, struct defer_rename_state); + if (rename_state == NULL) { + return NULL; + } + + rename_state->req = req; + rename_state->smb2req = smb2req; + rename_state->ev = ev; + rename_state->fsp = fsp; + rename_state->data = data; + rename_state->data_size = data_size; + + subreq = dbwrap_record_watch_send( + rename_state, + ev, + lck->data->record, + fsp->conn->sconn->msg_ctx); + + if (subreq == NULL) { + exit_server("Could not watch share mode record for rename\n"); + } + + tevent_req_set_callback(subreq, defer_rename_done, rename_state); + + timeout = timeval_set(OPLOCK_BREAK_TIMEOUT*2, 0); + if (!tevent_req_set_endtime(subreq, + ev, + timeval_sum(&smb2req->request_time, &timeout))) { + exit_server("Could not set rename timeout\n"); + } + + return subreq; +} + +static void defer_rename_done(struct tevent_req *subreq) +{ + struct defer_rename_state *state = tevent_req_callback_data( + subreq, struct defer_rename_state); + NTSTATUS status; + struct share_mode_lock *lck; + int ret_size = 0; + bool ok; + + status = dbwrap_record_watch_recv(subreq, state->req, NULL); + TALLOC_FREE(subreq); + if (!NT_STATUS_IS_OK(status)) { + DEBUG(5, ("dbwrap_record_watch_recv returned %s\n", + nt_errstr(status))); + tevent_req_nterror(state->req, status); + return; + } + + /* + * Make sure we run as the user again + */ + ok = change_to_user(state->smb2req->tcon->compat, + state->smb2req->session->compat->vuid); + if (!ok) { + tevent_req_nterror(state->req, NT_STATUS_ACCESS_DENIED); + return; + } + + /* should we pass FLAG_CASELESS_PATHNAMES here? */ + ok = set_current_service(state->smb2req->tcon->compat, 0, true); + if (!ok) { + tevent_req_nterror(state->req, NT_STATUS_ACCESS_DENIED); + return; + } + + /* Do we still need to wait ? */ + lck = get_existing_share_mode_lock(state->req, state->fsp->file_id); + if (lck == NULL) { + tevent_req_nterror(state->req, NT_STATUS_UNSUCCESSFUL); + return; + } + subreq = delay_rename_for_lease_break(state->req, + state->smb2req, + state->ev, + state->fsp, + lck, + state->data, + state->data_size); + if (subreq) { + /* Yep - keep waiting. */ + TALLOC_FREE(state); + TALLOC_FREE(lck); + return; + } + + /* Do the rename under the lock. */ + status = smbd_do_setfilepathinfo(state->fsp->conn, + state->smb2req->smb1req, + state, + SMB2_FILE_RENAME_INFORMATION_INTERNAL, + state->fsp, + state->fsp->fsp_name, + &state->data, + state->data_size, + &ret_size); + + TALLOC_FREE(lck); + SAFE_FREE(state->data); + + if (!NT_STATUS_IS_OK(status)) { + tevent_req_nterror(state->req, status); + return; + } + + tevent_req_done(state->req); +} + struct smbd_smb2_setinfo_state { struct smbd_smb2_request *smb2req; }; @@ -173,6 +356,7 @@ static struct tevent_req *smbd_smb2_setinfo_send(TALLOC_CTX *mem_ctx, struct smbd_smb2_setinfo_state *state = NULL; struct smb_request *smbreq = NULL; connection_struct *conn = smb2req->tcon->compat; + struct share_mode_lock *lck = NULL; NTSTATUS status; req = tevent_req_create(mem_ctx, &state, @@ -284,6 +468,39 @@ static struct tevent_req *smbd_smb2_setinfo_send(TALLOC_CTX *mem_ctx, memcpy(data, in_input_buffer.data, data_size); } + if (file_info_level == SMB2_FILE_RENAME_INFORMATION_INTERNAL) { + struct tevent_req *subreq; + + lck = get_existing_share_mode_lock(mem_ctx, + fsp->file_id); + if (lck == NULL) { + tevent_req_nterror(req, + NT_STATUS_UNSUCCESSFUL); + return tevent_req_post(req, ev); + } + + subreq = delay_rename_for_lease_break(req, + smb2req, + ev, + fsp, + lck, + data, + data_size); + if (subreq) { + /* Wait for lease break response. */ + + /* Ensure we can't be closed in flight. */ + if (!aio_add_req_to_fsp(fsp, req)) { + TALLOC_FREE(lck); + tevent_req_nterror(req, NT_STATUS_NO_MEMORY); + return tevent_req_post(req, ev); + } + + TALLOC_FREE(lck); + return req; + } + } + status = smbd_do_setfilepathinfo(conn, smbreq, state, file_info_level, fsp, @@ -291,6 +508,7 @@ static struct tevent_req *smbd_smb2_setinfo_send(TALLOC_CTX *mem_ctx, &data, data_size, &ret_size); + TALLOC_FREE(lck); SAFE_FREE(data); if (!NT_STATUS_IS_OK(status)) { if (NT_STATUS_EQUAL(status, NT_STATUS_INVALID_LEVEL)) { diff --git a/source4/torture/smb2/lease.c b/source4/torture/smb2/lease.c index 9d14aeb..c1b6420 100644 --- a/source4/torture/smb2/lease.c +++ b/source4/torture/smb2/lease.c @@ -3483,6 +3483,139 @@ static bool test_lease_timeout(struct torture_context *tctx, return ret; } +static bool test_lease_v2_rename(struct torture_context *tctx, + struct smb2_tree *tree) +{ + TALLOC_CTX *mem_ctx = talloc_new(tctx); + struct smb2_create io; + struct smb2_lease ls1; + struct smb2_lease ls2; + struct smb2_handle h, h1, h2; + union smb_setfileinfo sinfo; + const char *fname = "lease_v2_rename_src.dat"; + const char *fname_dst = "lease_v2_rename_dst.dat"; + bool ret = true; + NTSTATUS status; + uint32_t caps; + enum protocol_types protocol; + + caps = smb2cli_conn_server_capabilities(tree->session->transport->conn); -- Samba Shared Repository