The branch, master has been updated via 23988f19e7c s4:torture/smb2: add smb2.bench.echo via 8ee783c4803 s4:torture/smb2: teach smb2.bench.path-contention-shared about --option="torture:qdepth=4" via 72caffbe111 s4:param: add --option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4" support... via cd01f513469 s3:g_lock: use TDB_VOLATILE to avoid fcntl locks via a0a97d27f7a smbd: avoid calling SMB_VFS_FGET_NT_ACL() if do_not_check_mask already covers all via 8c7e8c5f80f s3:include: remove unused update_stat_ex_file_id() prototype from 1b470aaa67b s3:passdb: Consolidate error checking in fetch_ldap_pw()
https://git.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 23988f19e7cc2823d6c0c0f40af0195d0a3b81bf Author: Stefan Metzmacher <me...@samba.org> Date: Wed Aug 10 13:14:52 2022 +0000 s4:torture/smb2: add smb2.bench.echo This test calls SMB2_Echo in a loop per connection. For 4 connections with 2 parallel loops use this: time smbtorture //127.0.0.1/m -Uroot%test smb2.bench.echo \ --option="torture:timelimit=600" \ --option="torture:nprocs=1" \ --option="torture:qdepth=2" Sometimes the bottleneck is the smbtorture process. In order to bring the smbd process to 100% cpu, you can use '--option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4"' and run multiple instances of the test at the same time, which both talk to the same smbd process. This is a very useful test to show how many requests are possible at the raw SMB2 layer. Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> Autobuild-User(master): Jeremy Allison <j...@samba.org> Autobuild-Date(master): Thu Aug 11 19:23:37 UTC 2022 on sn-devel-184 commit 8ee783c4803d28cccc39144afa7b78c4b9e0cc2e Author: Stefan Metzmacher <me...@samba.org> Date: Wed Aug 10 11:43:20 2022 +0000 s4:torture/smb2: teach smb2.bench.path-contention-shared about --option="torture:qdepth=4" This can now test more than one open/close loop per connection. time smbtorture //127.0.0.1/m -Uroot%test \ smb2.create.bench-path-contention-shared \ --option='torture:bench_path=' \ --option="torture:timelimit=60" \ --option="torture:nprocs=1" \ --option="torture:qdepth=4" The default is still 1, but it's very useful for tests. Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> commit 72caffbe1115c57ad38270eaeb951f6b97bf62b3 Author: Stefan Metzmacher <me...@samba.org> Date: Wed Aug 10 13:15:45 2022 +0000 s4:param: add --option="libsmb:client_guid=6112f7d3-9528-4a2a-8861-0ca129aae6c4" support... We already handle this in the source3/libsmb code, but it's good to have this also for torture tests. Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> commit cd01f5134696f7789fbc2933629ac2606feb0b5e Author: Stefan Metzmacher <me...@samba.org> Date: Wed Aug 10 13:16:14 2022 +0000 s3:g_lock: use TDB_VOLATILE to avoid fcntl locks This improves 'time smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 50000000' from ~1.400.000 to ~3.400.000 operations per second any a testsystem. As we also use TDB_VOLATILE for locking.tdb, this is a much more realistic test now. Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> commit a0a97d27f7a60dbd86317b51bec0ece2476e8c8d Author: Stefan Metzmacher <me...@samba.org> Date: Tue Aug 9 14:07:12 2022 +0000 smbd: avoid calling SMB_VFS_FGET_NT_ACL() if do_not_check_mask already covers all This is inspired by 0d4cb5a641e1fea2d369bdc66470a580321366c2, which avoids SMB_VFS_FGET_NT_ACL() for the root user again. Opens with just FILE_READ_ATTRIBUTES are very common, so it's worth optimizing for it. Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> commit 8c7e8c5f80f1488456f9dd6225020d29f74458d2 Author: Stefan Metzmacher <me...@samba.org> Date: Wed Aug 10 16:45:26 2022 +0200 s3:include: remove unused update_stat_ex_file_id() prototype It was removed by commit 643da37fd139413651a6198fb0f6e550f7de6584 Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> ----------------------------------------------------------------------- Summary of changes: source3/include/proto.h | 1 - source3/lib/g_lock.c | 2 +- source3/smbd/open.c | 69 +++-- source4/param/loadparm.c | 11 +- source4/torture/smb2/create.c | 591 ++++++++++++++++++++++++++++++++++-------- 5 files changed, 538 insertions(+), 136 deletions(-) Changeset truncated at 500 lines: diff --git a/source3/include/proto.h b/source3/include/proto.h index aebbd7f9c47..6a6edc36dfb 100644 --- a/source3/include/proto.h +++ b/source3/include/proto.h @@ -171,7 +171,6 @@ int sys_fcntl_long(int fd, int cmd, long arg); int sys_fcntl_int(int fd, int cmd, int arg); void update_stat_ex_mtime(struct stat_ex *dst, struct timespec write_ts); void update_stat_ex_create_time(struct stat_ex *dst, struct timespec create_time); -void update_stat_ex_file_id(struct stat_ex *dst, uint64_t file_id); void update_stat_ex_from_saved_stat(struct stat_ex *dst, const struct stat_ex *src); int sys_stat(const char *fname, SMB_STRUCT_STAT *sbuf, diff --git a/source3/lib/g_lock.c b/source3/lib/g_lock.c index 52a1cf9fa46..d683c4ddee6 100644 --- a/source3/lib/g_lock.c +++ b/source3/lib/g_lock.c @@ -236,7 +236,7 @@ struct g_lock_ctx *g_lock_ctx_init(TALLOC_CTX *mem_ctx, mem_ctx, db_path, 0, - TDB_CLEAR_IF_FIRST|TDB_INCOMPATIBLE_HASH, + TDB_CLEAR_IF_FIRST|TDB_INCOMPATIBLE_HASH|TDB_VOLATILE, O_RDWR|O_CREAT, 0600, DBWRAP_LOCK_ORDER_3, diff --git a/source3/smbd/open.c b/source3/smbd/open.c index 6d3dee9ce18..fa3be0a6e3b 100644 --- a/source3/smbd/open.c +++ b/source3/smbd/open.c @@ -99,9 +99,11 @@ static NTSTATUS smbd_check_access_rights_fname( struct connection_struct *conn, const struct smb_filename *smb_fname, bool use_privs, - uint32_t access_mask) + uint32_t access_mask, + uint32_t do_not_check_mask) { uint32_t rejected_share_access; + uint32_t effective_access; rejected_share_access = access_mask & ~(conn->share_access); @@ -114,6 +116,14 @@ static NTSTATUS smbd_check_access_rights_fname( return NT_STATUS_ACCESS_DENIED; } + effective_access = access_mask & ~do_not_check_mask; + if (effective_access == 0) { + DBG_DEBUG("do_not_check_mask override on %s. Granting 0x%x for free.\n", + smb_fname_str_dbg(smb_fname), + (unsigned int)access_mask); + return NT_STATUS_OK; + } + if (!use_privs && get_current_uid(conn) == (uid_t)0) { /* I'm sorry sir, I didn't know you were root... */ DBG_DEBUG("root override on %s. Granting 0x%x\n", @@ -151,39 +161,16 @@ static NTSTATUS smbd_check_access_rights_sd( const struct smb_filename *smb_fname, struct security_descriptor *sd, bool use_privs, - uint32_t access_mask) + uint32_t access_mask, + uint32_t do_not_check_mask) { uint32_t rejected_mask = access_mask; - uint32_t do_not_check_mask = 0; NTSTATUS status; if (sd == NULL) { goto access_denied; } - /* - * If we can access the path to this file, by - * default we have FILE_READ_ATTRIBUTES from the - * containing directory. See the section: - * "Algorithm to Check Access to an Existing File" - * in MS-FSA.pdf. - * - * se_file_access_check() also takes care of - * owner WRITE_DAC and READ_CONTROL. - */ - do_not_check_mask = FILE_READ_ATTRIBUTES; - - /* - * Samba 3.6 and earlier granted execute access even - * if the ACL did not contain execute rights. - * Samba 4.0 is more correct and checks it. - * The compatibilty mode allows one to skip this check - * to smoothen upgrades. - */ - if (lp_acl_allow_execute_always(SNUM(conn))) { - do_not_check_mask |= FILE_EXECUTE; - } - status = se_file_access_check(sd, get_current_nttok(conn), use_privs, @@ -263,6 +250,7 @@ NTSTATUS smbd_check_access_rights_fsp(struct files_struct *dirfsp, uint32_t access_mask) { struct security_descriptor *sd = NULL; + uint32_t do_not_check_mask = 0; NTSTATUS status; /* Cope with fake/printer fsp's. */ @@ -288,10 +276,34 @@ NTSTATUS smbd_check_access_rights_fsp(struct files_struct *dirfsp, return NT_STATUS_OK; } + /* + * If we can access the path to this file, by + * default we have FILE_READ_ATTRIBUTES from the + * containing directory. See the section: + * "Algorithm to Check Access to an Existing File" + * in MS-FSA.pdf. + * + * se_file_access_check() also takes care of + * owner WRITE_DAC and READ_CONTROL. + */ + do_not_check_mask = FILE_READ_ATTRIBUTES; + + /* + * Samba 3.6 and earlier granted execute access even + * if the ACL did not contain execute rights. + * Samba 4.0 is more correct and checks it. + * The compatibilty mode allows one to skip this check + * to smoothen upgrades. + */ + if (lp_acl_allow_execute_always(SNUM(fsp->conn))) { + do_not_check_mask |= FILE_EXECUTE; + } + status = smbd_check_access_rights_fname(fsp->conn, fsp->fsp_name, use_privs, - access_mask); + access_mask, + do_not_check_mask); if (!NT_STATUS_EQUAL(status, NT_STATUS_MORE_PROCESSING_REQUIRED)) { return status; } @@ -314,7 +326,8 @@ NTSTATUS smbd_check_access_rights_fsp(struct files_struct *dirfsp, fsp->fsp_name, sd, use_privs, - access_mask); + access_mask, + do_not_check_mask); } /* diff --git a/source4/param/loadparm.c b/source4/param/loadparm.c index 81892d540ef..7d3891dc0b9 100644 --- a/source4/param/loadparm.c +++ b/source4/param/loadparm.c @@ -35,6 +35,15 @@ void lpcfg_smbcli_options(struct loadparm_context *lp_ctx, struct smbcli_options *options) { + struct GUID client_guid; + const char *str = NULL; + + str = lpcfg_parm_string(lp_ctx, NULL, "libsmb", "client_guid"); + if (str != NULL) { + GUID_from_string(str, &client_guid); + } else { + client_guid = GUID_random(); + } *options = (struct smbcli_options) { .max_xmit = lpcfg_max_xmit(lp_ctx), .max_mux = lpcfg_max_mux(lp_ctx), @@ -48,7 +57,7 @@ void lpcfg_smbcli_options(struct loadparm_context *lp_ctx, .use_oplocks = true, .use_level2_oplocks = true, .smb2_capabilities = SMB2_CAP_ALL, - .client_guid = GUID_random(), + .client_guid = client_guid, .max_credits = WINDOWS_CLIENT_PURE_SMB2_NEGPROT_INITIAL_CREDIT_ASK, .smb3_capabilities = smb311_capabilities_parse("client", lpcfg_client_smb3_signing_algorithms(lp_ctx), diff --git a/source4/torture/smb2/create.c b/source4/torture/smb2/create.c index 8f1e94b5773..9277488e9d7 100644 --- a/source4/torture/smb2/create.c +++ b/source4/torture/smb2/create.c @@ -22,6 +22,7 @@ #include "includes.h" #include "libcli/smb2/smb2.h" #include "libcli/smb2/smb2_calls.h" +#include "libcli/smb/smbXcli_base.h" #include "torture/torture.h" #include "torture/util.h" #include "torture/smb2/proto.h" @@ -126,6 +127,333 @@ __location__, (unsigned int)(sattrib), fname); \ }} while (0) +/* + stress testing keepalive iops + */ + +struct test_smb2_bench_echo_conn; +struct test_smb2_bench_echo_loop; + +struct test_smb2_bench_echo_state { + struct torture_context *tctx; + size_t num_conns; + struct test_smb2_bench_echo_conn *conns; + size_t num_loops; + struct test_smb2_bench_echo_loop *loops; + struct timeval starttime; + int timecount; + int timelimit; + uint64_t num_finished; + double total_latency; + double min_latency; + double max_latency; + bool ok; + bool stop; +}; + +struct test_smb2_bench_echo_conn { + struct test_smb2_bench_echo_state *state; + int idx; + struct smb2_tree *tree; +}; + +struct test_smb2_bench_echo_loop { + struct test_smb2_bench_echo_state *state; + struct test_smb2_bench_echo_conn *conn; + int idx; + struct tevent_immediate *im; + struct tevent_req *req; + struct timeval starttime; + uint64_t num_started; + uint64_t num_finished; + double total_latency; + double min_latency; + double max_latency; + NTSTATUS error; +}; + +static void test_smb2_bench_echo_loop_do( + struct test_smb2_bench_echo_loop *loop); + +static void test_smb2_bench_echo_loop_start(struct tevent_context *ctx, + struct tevent_immediate *im, + void *private_data) +{ + struct test_smb2_bench_echo_loop *loop = + (struct test_smb2_bench_echo_loop *) + private_data; + + test_smb2_bench_echo_loop_do(loop); +} + +static void test_smb2_bench_echo_loop_done(struct tevent_req *req); + +static void test_smb2_bench_echo_loop_do( + struct test_smb2_bench_echo_loop *loop) +{ + struct test_smb2_bench_echo_state *state = loop->state; + + loop->num_started += 1; + loop->starttime = timeval_current(); + loop->req = smb2cli_echo_send(state->loops, + state->tctx->ev, + loop->conn->tree->session->transport->conn, + 1000); + torture_assert_goto(state->tctx, loop->req != NULL, + state->ok, asserted, "smb2_create_send"); + + tevent_req_set_callback(loop->req, + test_smb2_bench_echo_loop_done, + loop); + return; +asserted: + state->stop = true; +} + +static void test_smb2_bench_echo_loop_done(struct tevent_req *req) +{ + struct test_smb2_bench_echo_loop *loop = + (struct test_smb2_bench_echo_loop *) + _tevent_req_callback_data(req); + struct test_smb2_bench_echo_state *state = loop->state; + double latency = timeval_elapsed(&loop->starttime); + TALLOC_CTX *frame = talloc_stackframe(); + + torture_assert_goto(state->tctx, loop->req == req, + state->ok, asserted, __location__); + loop->error = smb2cli_echo_recv(req); + torture_assert_ntstatus_ok_goto(state->tctx, loop->error, + state->ok, asserted, __location__); + SMB_ASSERT(latency >= 0.000001); + + if (loop->num_finished == 0) { + /* first round */ + loop->min_latency = latency; + loop->max_latency = latency; + } + + loop->num_finished += 1; + loop->total_latency += latency; + + if (latency < loop->min_latency) { + loop->min_latency = latency; + } + + if (latency > loop->max_latency) { + loop->max_latency = latency; + } + + TALLOC_FREE(frame); + test_smb2_bench_echo_loop_do(loop); + return; +asserted: + state->stop = true; + TALLOC_FREE(frame); +} + +static void test_smb2_bench_echo_progress(struct tevent_context *ev, + struct tevent_timer *te, + struct timeval current_time, + void *private_data) +{ + struct test_smb2_bench_echo_state *state = + (struct test_smb2_bench_echo_state *)private_data; + uint64_t num_echos = 0; + double total_echo_latency = 0; + double min_echo_latency = 0; + double max_echo_latency = 0; + double avs_echo_latency = 0; + size_t i; + + state->timecount += 1; + + for (i=0;i<state->num_loops;i++) { + struct test_smb2_bench_echo_loop *loop = + &state->loops[i]; + + num_echos += loop->num_finished; + total_echo_latency += loop->total_latency; + if (min_echo_latency == 0.0 && loop->min_latency != 0.0) { + min_echo_latency = loop->min_latency; + } + if (loop->min_latency < min_echo_latency) { + min_echo_latency = loop->min_latency; + } + if (max_echo_latency == 0.0) { + max_echo_latency = loop->max_latency; + } + if (loop->max_latency > max_echo_latency) { + max_echo_latency = loop->max_latency; + } + loop->num_finished = 0; + loop->total_latency = 0.0; + } + + state->num_finished += num_echos; + state->total_latency += total_echo_latency; + if (state->min_latency == 0.0 && min_echo_latency != 0.0) { + state->min_latency = min_echo_latency; + } + if (min_echo_latency < state->min_latency) { + state->min_latency = min_echo_latency; + } + if (state->max_latency == 0.0) { + state->max_latency = max_echo_latency; + } + if (max_echo_latency > state->max_latency) { + state->max_latency = max_echo_latency; + } + + if (state->timecount < state->timelimit) { + te = tevent_add_timer(state->tctx->ev, + state, + timeval_current_ofs(1, 0), + test_smb2_bench_echo_progress, + state); + torture_assert_goto(state->tctx, te != NULL, + state->ok, asserted, "tevent_add_timer"); + + if (!torture_setting_bool(state->tctx, "progress", true)) { + return; + } + + avs_echo_latency = total_echo_latency / num_echos; + + torture_comment(state->tctx, + "%.2f second: " + "echo[num/s=%llu,avslat=%.6f,minlat=%.6f,maxlat=%.6f] \r", + timeval_elapsed(&state->starttime), + (unsigned long long)num_echos, + avs_echo_latency, + min_echo_latency, + max_echo_latency); + return; + } + + avs_echo_latency = state->total_latency / state->num_finished; + num_echos = state->num_finished / state->timelimit; + + torture_comment(state->tctx, + "%.2f second: " + "echo[num/s=%llu,avslat=%.6f,minlat=%.6f,maxlat=%.6f]\n", + timeval_elapsed(&state->starttime), + (unsigned long long)num_echos, + avs_echo_latency, + state->min_latency, + state->max_latency); + +asserted: + state->stop = true; +} + +static bool test_smb2_bench_echo(struct torture_context *tctx, + struct smb2_tree *tree) +{ + struct test_smb2_bench_echo_state *state = NULL; + bool ret = true; + int torture_nprocs = torture_setting_int(tctx, "nprocs", 4); + int torture_qdepth = torture_setting_int(tctx, "qdepth", 1); + size_t i; + size_t li = 0; + int timelimit = torture_setting_int(tctx, "timelimit", 10); + struct tevent_timer *te = NULL; + uint32_t timeout_msec; + + state = talloc_zero(tctx, struct test_smb2_bench_echo_state); + torture_assert(tctx, state != NULL, __location__); + state->tctx = tctx; + state->num_conns = torture_nprocs; + state->conns = talloc_zero_array(state, + struct test_smb2_bench_echo_conn, + state->num_conns); + torture_assert(tctx, state->conns != NULL, __location__); + state->num_loops = torture_nprocs * torture_qdepth; + state->loops = talloc_zero_array(state, + struct test_smb2_bench_echo_loop, + state->num_loops); + torture_assert(tctx, state->loops != NULL, __location__); + state->ok = true; + state->timelimit = MAX(timelimit, 1); + + timeout_msec = tree->session->transport->options.request_timeout * 1000; + + torture_comment(tctx, "Opening %zu connections\n", state->num_conns); + + for (i=0;i<state->num_conns;i++) { + struct smb2_tree *ct = NULL; + DATA_BLOB out_input_buffer = data_blob_null; + DATA_BLOB out_output_buffer = data_blob_null; + size_t pcli; + + state->conns[i].state = state; + state->conns[i].idx = i; + + if (!torture_smb2_connection(tctx, &ct)) { + torture_comment(tctx, "Failed opening %zu/%zu connections\n", i, state->num_conns); + return false; + } + state->conns[i].tree = talloc_steal(state->conns, ct); + + smb2cli_conn_set_max_credits(ct->session->transport->conn, 8192); + smb2cli_ioctl(ct->session->transport->conn, + timeout_msec, + ct->session->smbXcli, + ct->smbXcli, + UINT64_MAX, /* in_fid_persistent */ + UINT64_MAX, /* in_fid_volatile */ + UINT32_MAX, + 0, /* in_max_input_length */ + NULL, /* in_input_buffer */ + 1, /* in_max_output_length */ + NULL, /* in_output_buffer */ + SMB2_IOCTL_FLAG_IS_FSCTL, + ct, + &out_input_buffer, + &out_output_buffer); + torture_assert(tctx, + smbXcli_conn_is_connected(ct->session->transport->conn), + "smbXcli_conn_is_connected"); + + for (pcli = 0; pcli < torture_qdepth; pcli++) { + struct test_smb2_bench_echo_loop *loop = &state->loops[li]; + + loop->idx = li++; + loop->state = state; + loop->conn = &state->conns[i]; + loop->im = tevent_create_immediate(state->loops); + torture_assert(tctx, loop->im != NULL, __location__); + + tevent_schedule_immediate(loop->im, + tctx->ev, + test_smb2_bench_echo_loop_start, + loop); + } + } -- Samba Shared Repository