The branch, master has been updated via 1cff50febec dsdb: sort DSDB_EXTENDED defines by OID via 1fd4cdfafaa drepl: schema repl race condition fix via f460bb52ed4 Conditionally disable macOS incompatible tests via 440ddf8470b Send status to systemd on daemon start via 97c5a698aa2 manpage: Advise vfs_fruit:veto_appledouble=yes can break rsync via 0a449c2b743 s4:scripting:bin: rm rpcclient via ce33dd1196e dsdb: Remove readOnlySchema concept from Samba from 8a313bbdd5c lib:tsocket: Check for DOXYGEN as a #define
https://git.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 1cff50febecfb3c0443a17da29a047eea2e09e61 Author: Andrew Bartlett <abart...@samba.org> Date: Fri Dec 14 14:49:03 2018 +1300 dsdb: sort DSDB_EXTENDED defines by OID This helps avoid duplicate values and clearly indicates what value to select next. Signed-off-by: Andrew Bartlett <abart...@samba.org> Reviewed-by: Gary Lockyer <g...@catalyst.net.nz> Autobuild-User(master): Andrew Bartlett <abart...@samba.org> Autobuild-Date(master): Mon Dec 17 04:30:39 CET 2018 on sn-devel-144 commit 1fd4cdfafaa6a41c824d1b3d76635bf3e446de0f Author: Aaron Haslett <aaronhasl...@catalyst.net.nz> Date: Wed Nov 21 13:55:53 2018 +1300 drepl: schema repl race condition fix Adds final schema consistency check before committing changes. Aborts if corruption found. BUG: https://bugzilla.samba.org/show_bug.cgi?id=12889 Signed-off-by: Aaron Haslett <aaronhasl...@catalyst.net.nz> Reviewed-by: Gary Lockyer <g...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit f460bb52ed4393722d58f9e6d30bd0c7e6a51f1d Author: Will <wil...@gmail.com> Date: Tue Nov 7 21:52:34 2017 -0600 Conditionally disable macOS incompatible tests Symbols _getgrent_r and _getpwent_r in source4/torture/local/nss_tests.c are undefined in macOS. It seems that checking HAVE_GETGRENT_R and HAVE_GETPWENT_R and conditionally disabling those tests as suggested by hirochachacha in the referenced bug allows samba on both `master` and `samba-4.7.1` to build properly on macOS/darwin. BUG: https://bugzilla.samba.org/show_bug.cgi?id=11984 Signed-off-by: Will Haley <wil...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Ralph Boehme <s...@samba.org> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 440ddf8470b11a46066d282bf8945201d547c192 Author: Marcos Mello <marcos...@gmail.com> Date: Fri Nov 30 09:25:07 2018 -0200 Send status to systemd on daemon start systemd service files run in no-forking mode (--foreground) since 8b6f58194da7e849cdb9d20712dff49b17a93a77. Rearrange sd_notify() call in become_daemon() to only send status to systemd in this mode (Type=notify is not designed to monitor forking). Drop READY=0 (it does nothing) and MAINPID= (unnecessary because the process spawned by systemd is already the main PID). Also remove STATUS= prefix from debug messages. Signed-off-by: Marcos Mello <marcos...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Alexander Bokovoy <a...@samba.org> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 97c5a698aa2ba99bc07c7db8bc12bfcb7afa662e Author: Adam Nielsen <malvine...@shikadi.net> Date: Sun Oct 15 11:56:32 2017 +1000 manpage: Advise vfs_fruit:veto_appledouble=yes can break rsync Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Uri Simchoni <u...@samba.org> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 0a449c2b7438ad6012e16d7764edf12cc7277474 Author: Guo Qiao <guoq...@gmail.com> Date: Fri Dec 14 11:23:46 2018 +1300 s4:scripting:bin: rm rpcclient This file only works in py2, and no one use it. Just delete it. Signed-off-by: Joe Guo <j...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> Reviewed-by: Gary Lockyer <g...@catalyst.net.nz> commit ce33dd1196e4be5350675aa6ba8acbccb12f703e Author: Andrew Bartlett <abart...@samba.org> Date: Mon Dec 17 10:17:08 2018 +1300 dsdb: Remove readOnlySchema concept from Samba This is a hold-over from the LDAP backend project, which has not yet been revived. There will be bigger issues than what to do if the schema changes if this ever comes back and our schema code is way to complex at the moment. Signed-off-by: Andrew Bartlett <abart...@samba.org> Reviewed-by: Gary Lockyer <g...@catalyst.net.nz> ----------------------------------------------------------------------- Summary of changes: docs-xml/manpages/vfs_fruit.8.xml | 10 +- lib/util/become_daemon.c | 18 +- source4/dsdb/repl/replicated_objects.c | 35 ++- source4/dsdb/samdb/ldb_modules/samba_dsdb.c | 5 - source4/dsdb/samdb/ldb_modules/schema_load.c | 72 +++---- source4/dsdb/samdb/samdb.h | 72 ++++--- source4/scripting/bin/rpcclient | 305 --------------------------- source4/torture/local/nss_tests.c | 4 + 8 files changed, 117 insertions(+), 404 deletions(-) delete mode 100755 source4/scripting/bin/rpcclient Changeset truncated at 500 lines: diff --git a/docs-xml/manpages/vfs_fruit.8.xml b/docs-xml/manpages/vfs_fruit.8.xml index e4ca7bd1828..c5bd593a139 100644 --- a/docs-xml/manpages/vfs_fruit.8.xml +++ b/docs-xml/manpages/vfs_fruit.8.xml @@ -331,9 +331,13 @@ <parameter>file</parameter>, vfs_fruit may create ._ AppleDouble files. This options controls whether these ._ AppleDouble files are vetoed which prevents the client from accessing them.</para> - <para>Vetoing ._ files may break some applications, eg - extracting Mac ZIP archives from Mac clients failes, - because they contain ._ files. Setting this option to + <para>Vetoing ._ files may break some applications, e.g. + extracting Mac ZIP archives from Mac clients fails, + because they contain ._ files. <command>rsync</command> will + also be unable to sync files beginning with underscores, as + the temporary files it uses for these will start with ._ and + so cannot be created.</para> + <para>Setting this option to false will fix this, but the abstraction leak of exposing the internally created ._ files may have other unknown side effects.</para> diff --git a/lib/util/become_daemon.c b/lib/util/become_daemon.c index 232eda69b64..89991b7981c 100644 --- a/lib/util/become_daemon.c +++ b/lib/util/become_daemon.c @@ -73,14 +73,12 @@ void become_daemon(bool do_fork, bool no_session, bool log_stdout) exit_daemon("Fork failed", errno); } if (newpid) { -#if defined(HAVE_LIBSYSTEMD_DAEMON) || defined(HAVE_LIBSYSTEMD) - sd_notifyf(0, - "READY=0\nSTATUS=Starting process...\n" - "MAINPID=%lu", - (unsigned long) newpid); -#endif /* HAVE_LIBSYSTEMD_DAEMON */ _exit(0); } +#if defined(HAVE_LIBSYSTEMD_DAEMON) || defined(HAVE_LIBSYSTEMD) + } else { + sd_notify(0, "STATUS=Starting process..."); +#endif } /* detach from the terminal */ @@ -120,7 +118,7 @@ void exit_daemon(const char *msg, int error) msg, error); #endif - DBG_ERR("STATUS=daemon failed to start: %s, error code %d\n", + DBG_ERR("daemon failed to start: %s, error code %d\n", msg, error); exit(1); } @@ -134,7 +132,7 @@ void daemon_ready(const char *daemon) sd_notifyf(0, "READY=1\nSTATUS=%s: ready to serve connections...", daemon); #endif - DBG_ERR("STATUS=daemon '%s' finished starting up and ready to serve " + DBG_ERR("daemon '%s' finished starting up and ready to serve " "connections\n", daemon); } @@ -144,7 +142,7 @@ void daemon_status(const char *daemon, const char *msg) daemon = "Samba"; } #if defined(HAVE_LIBSYSTEMD_DAEMON) || defined(HAVE_LIBSYSTEMD) - sd_notifyf(0, "\nSTATUS=%s: %s", daemon, msg); + sd_notifyf(0, "STATUS=%s: %s", daemon, msg); #endif - DBG_ERR("STATUS=daemon '%s' : %s\n", daemon, msg); + DBG_ERR("daemon '%s' : %s\n", daemon, msg); } diff --git a/source4/dsdb/repl/replicated_objects.c b/source4/dsdb/repl/replicated_objects.c index fd567e9395c..372fb2d6928 100644 --- a/source4/dsdb/repl/replicated_objects.c +++ b/source4/dsdb/repl/replicated_objects.c @@ -914,8 +914,10 @@ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, } talloc_free(ext_res); - /* Save our updated prefixMap */ + /* Save our updated prefixMap and check the schema is good. */ if (working_schema) { + struct ldb_result *ext_res_2; + werr = dsdb_write_prefixes_from_schema_to_ldb(working_schema, ldb, working_schema); @@ -924,7 +926,9 @@ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, if (used_global_schema) { dsdb_set_global_schema(ldb); } else if (cur_schema ) { - dsdb_reference_schema(ldb, cur_schema, SCHEMA_MEMORY_ONLY); + dsdb_reference_schema(ldb, + cur_schema, + SCHEMA_MEMORY_ONLY); } DEBUG(0,("Failed to save updated prefixMap: %s\n", win_errstr(werr))); @@ -932,6 +936,33 @@ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, TALLOC_FREE(tmp_ctx); return werr; } + + /* + * Use dsdb_schema_from_db through dsdb extended to check we + * can load the schema currently sitting in the transaction. + * We need this check because someone might have written to + * the schema or prefixMap before we started the transaction, + * which may have caused corruption. + */ + ret = ldb_extended(ldb, DSDB_EXTENDED_SCHEMA_LOAD, + NULL, &ext_res_2); + + if (ret != LDB_SUCCESS) { + if (used_global_schema) { + dsdb_set_global_schema(ldb); + } else if (cur_schema) { + dsdb_reference_schema(ldb, cur_schema, SCHEMA_MEMORY_ONLY); + } + DEBUG(0,("Corrupt schema write attempt detected, " + "aborting schema modification operation.\n" + "This probably happened due to bad timing of " + "another schema edit: %s (%s)\n", + ldb_errstring(ldb), + ldb_strerror(ret))); + ldb_transaction_cancel(ldb); + TALLOC_FREE(tmp_ctx); + return WERR_FOOBAR; + } } ret = ldb_transaction_prepare_commit(ldb); diff --git a/source4/dsdb/samdb/ldb_modules/samba_dsdb.c b/source4/dsdb/samdb/ldb_modules/samba_dsdb.c index fa58f19db29..3356999f342 100644 --- a/source4/dsdb/samdb/ldb_modules/samba_dsdb.c +++ b/source4/dsdb/samdb/ldb_modules/samba_dsdb.c @@ -541,11 +541,6 @@ static int samba_dsdb_init(struct ldb_module *module) } else { return ldb_error(ldb, LDB_ERR_OPERATIONS_ERROR, "invalid backend type"); } - ret = ldb_set_opaque(ldb, "readOnlySchema", (void*)1); - if (ret != LDB_SUCCESS) { - ldb_set_errstring(ldb, "Failed to set readOnlySchema opaque"); - } - cred = ldb_get_opaque(ldb, "credentials"); if (!cred || !cli_credentials_authentication_requested(cred)) { ret = set_ldap_credentials(ldb, use_sasl_external); diff --git a/source4/dsdb/samdb/ldb_modules/schema_load.c b/source4/dsdb/samdb/ldb_modules/schema_load.c index 6a3001d05e4..473a2e0a1f7 100644 --- a/source4/dsdb/samdb/ldb_modules/schema_load.c +++ b/source4/dsdb/samdb/ldb_modules/schema_load.c @@ -386,7 +386,6 @@ static int schema_load(struct ldb_context *ldb, bool *need_write) { struct dsdb_schema *schema; - void *readOnlySchema; int ret, metadata_ret; TALLOC_CTX *frame = talloc_stackframe(); @@ -413,33 +412,7 @@ static int schema_load(struct ldb_context *ldb, return LDB_SUCCESS; } - readOnlySchema = ldb_get_opaque(ldb, "readOnlySchema"); - - /* If we have the readOnlySchema opaque, then don't check for - * runtime schema updates, as they are not permitted (we would - * have to update the backend server schema too */ - if (readOnlySchema != NULL) { - struct dsdb_schema *new_schema; - ret = dsdb_schema_from_db(module, frame, 0, &new_schema); - if (ret != LDB_SUCCESS) { - ldb_debug_set(ldb, LDB_DEBUG_FATAL, - "schema_load_init: dsdb_schema_from_db() failed: %d:%s: %s", - ret, ldb_strerror(ret), ldb_errstring(ldb)); - TALLOC_FREE(frame); - return ret; - } - - /* "dsdb_set_schema()" steals schema into the ldb_context */ - ret = dsdb_set_schema(ldb, new_schema, SCHEMA_MEMORY_ONLY); - if (ret != LDB_SUCCESS) { - ldb_debug_set(ldb, LDB_DEBUG_FATAL, - "schema_load_init: dsdb_set_schema() failed: %d:%s: %s", - ret, ldb_strerror(ret), ldb_errstring(ldb)); - TALLOC_FREE(frame); - return ret; - } - - } else if (metadata_ret == LDB_SUCCESS) { + if (metadata_ret == LDB_SUCCESS) { ret = dsdb_set_schema_refresh_function(ldb, dsdb_schema_refresh, module); if (ret != LDB_SUCCESS) { @@ -579,26 +552,35 @@ static int schema_load_extended(struct ldb_module *module, struct ldb_request *r struct dsdb_schema *schema; int ret; - if (strcmp(req->op.extended.oid, DSDB_EXTENDED_SCHEMA_UPDATE_NOW_OID) != 0) { - return ldb_next_request(module, req); - } - /* Force a refresh */ - schema = dsdb_get_schema(ldb, NULL); - - ret = dsdb_schema_set_indices_and_attributes(ldb, - schema, - SCHEMA_WRITE); + if (strcmp(req->op.extended.oid, DSDB_EXTENDED_SCHEMA_LOAD) == 0) { - if (ret != LDB_SUCCESS) { - ldb_asprintf_errstring(ldb, "Failed to write new " - "@INDEXLIST and @ATTRIBUTES " - "records for updated schema: %s", - ldb_errstring(ldb)); + ret = dsdb_schema_from_db(module, req, 0, &schema); + if (ret == LDB_SUCCESS) { + return ldb_module_done(req, NULL, NULL, LDB_SUCCESS); + } return ret; + + } else if (strcmp(req->op.extended.oid, DSDB_EXTENDED_SCHEMA_UPDATE_NOW_OID) == 0) { + /* Force a refresh */ + schema = dsdb_get_schema(ldb, NULL); + + ret = dsdb_schema_set_indices_and_attributes(ldb, + schema, + SCHEMA_WRITE); + + if (ret != LDB_SUCCESS) { + ldb_asprintf_errstring(ldb, "Failed to write new " + "@INDEXLIST and @ATTRIBUTES " + "records for updated schema: %s", + ldb_errstring(ldb)); + return ret; + } + + return ldb_next_request(module, req); + } else { + /* Pass to next module, the partition one should finish the chain */ + return ldb_next_request(module, req); } - - /* Pass to next module, the partition one should finish the chain */ - return ldb_next_request(module, req); } static int schema_read_lock(struct ldb_module *module) diff --git a/source4/dsdb/samdb/samdb.h b/source4/dsdb/samdb/samdb.h index e1b0e4aa4e3..fd8d4e4497e 100644 --- a/source4/dsdb/samdb/samdb.h +++ b/source4/dsdb/samdb/samdb.h @@ -234,6 +234,12 @@ struct dsdb_extended_replicated_object { struct ldb_dn *local_parent_dn; }; +/* + * the schema_dn is passed as struct ldb_dn in + * req->op.extended.data + */ +#define DSDB_EXTENDED_SCHEMA_UPDATE_NOW_OID "1.3.6.1.4.1.7165.4.4.2" + struct dsdb_extended_replicated_objects { /* * this is the version of the dsdb_extended_replicated_objects @@ -261,18 +267,46 @@ struct dsdb_extended_replicated_objects { bool originating_updates; }; +/* In ldb.h: LDB_EXTENDED_SEQUENCE_NUMBER 1.3.6.1.4.1.7165.4.4.3 */ + #define DSDB_EXTENDED_CREATE_PARTITION_OID "1.3.6.1.4.1.7165.4.4.4" struct dsdb_create_partition_exop { struct ldb_dn *new_dn; }; +/* this takes a struct dsdb_fsmo_extended_op */ +#define DSDB_EXTENDED_ALLOCATE_RID_POOL "1.3.6.1.4.1.7165.4.4.5" + +struct dsdb_fsmo_extended_op { + uint64_t fsmo_info; + struct GUID destination_dsa_guid; +}; + +#define DSDB_EXTENDED_SCHEMA_UPGRADE_IN_PROGRESS_OID "1.3.6.1.4.1.7165.4.4.6" + /* - * the schema_dn is passed as struct ldb_dn in - * req->op.extended.data + * passed from the descriptor module in order to + * store the recalucated nTSecurityDescriptor without + * modifying the replPropertyMetaData. */ -#define DSDB_EXTENDED_SCHEMA_UPDATE_NOW_OID "1.3.6.1.4.1.7165.4.4.2" +#define DSDB_EXTENDED_SEC_DESC_PROPAGATION_OID "1.3.6.1.4.1.7165.4.4.7" +struct dsdb_extended_sec_desc_propagation_op { + struct ldb_dn *nc_root; + struct ldb_dn *dn; + bool include_self; +}; -#define DSDB_EXTENDED_SCHEMA_UPGRADE_IN_PROGRESS_OID "1.3.6.1.4.1.7165.4.4.6" +/* this takes no data */ +#define DSDB_EXTENDED_CREATE_OWN_RID_SET "1.3.6.1.4.1.7165.4.4.8" + +/* this takes a struct dsdb_extended_allocate_rid */ +#define DSDB_EXTENDED_ALLOCATE_RID "1.3.6.1.4.1.7165.4.4.9" + +struct dsdb_extended_allocate_rid { + uint32_t rid; +}; + +#define DSDB_EXTENDED_SCHEMA_LOAD "1.3.6.1.4.1.7165.4.4.10" #define DSDB_OPENLDAP_DEREFERENCE_CONTROL "1.3.6.1.4.1.4203.666.5.16" @@ -312,36 +346,6 @@ struct dsdb_extended_dn_store_format { #define DSDB_OPAQUE_PARTITION_MODULE_MSG_OPAQUE_NAME "DSDB_OPAQUE_PARTITION_MODULE_MSG" -/* this takes a struct dsdb_fsmo_extended_op */ -#define DSDB_EXTENDED_ALLOCATE_RID_POOL "1.3.6.1.4.1.7165.4.4.5" - -struct dsdb_fsmo_extended_op { - uint64_t fsmo_info; - struct GUID destination_dsa_guid; -}; - -/* this takes no data */ -#define DSDB_EXTENDED_CREATE_OWN_RID_SET "1.3.6.1.4.1.7165.4.4.8" - -/* this takes a struct dsdb_extended_allocate_rid */ -#define DSDB_EXTENDED_ALLOCATE_RID "1.3.6.1.4.1.7165.4.4.9" - -struct dsdb_extended_allocate_rid { - uint32_t rid; -}; - -/* - * passed from the descriptor module in order to - * store the recalucated nTSecurityDescriptor without - * modifying the replPropertyMetaData. - */ -#define DSDB_EXTENDED_SEC_DESC_PROPAGATION_OID "1.3.6.1.4.1.7165.4.4.7" -struct dsdb_extended_sec_desc_propagation_op { - struct ldb_dn *nc_root; - struct ldb_dn *dn; - bool include_self; -}; - #define DSDB_ACL_CHECKS_DIRSYNC_FLAG 0x1 #define DSDB_SAMDB_MINIMUM_ALLOWED_RID 1000 diff --git a/source4/scripting/bin/rpcclient b/source4/scripting/bin/rpcclient deleted file mode 100755 index 4660db3fde6..00000000000 --- a/source4/scripting/bin/rpcclient +++ /dev/null @@ -1,305 +0,0 @@ -#!/usr/bin/env python3 - -import sys, os, string - -# Find right directory when running from source tree -sys.path.insert(0, "bin/python") - -from cmd import Cmd -from optparse import OptionParser -from pprint import pprint - -import dcerpc, samr - -def swig2dict(obj): - """Convert a swig object to a dictionary.""" - - result = {} - - for attr in filter(lambda x: type(x) == str, dir(obj)): - - if attr[:2] == '__' and attr[-2:] == '__': - continue - - if attr == 'this' or attr == 'thisown': - continue - - result[attr] = getattr(obj, attr) - - return result - -class rpcclient(Cmd): - - prompt = 'rpcclient$ ' - - def __init__(self, server, cred): - Cmd.__init__(self) - self.server = server - self.cred = cred - - def emptyline(self): - - # Default for empty line is to repeat last command - yuck - - pass - - def onecmd(self, line): - - # Override the onecmd() method so we can trap error returns - - try: - Cmd.onecmd(self, line) - except dcerpc.NTSTATUS as arg: - print 'The command returned an error: %s' % arg[1] - - # Command handlers - - def do_help(self, line): - """Displays on-line help for rpcclient commands.""" - Cmd.do_help(self, line) - - def do_shell(self, line): - - status = os.system(line) - - if os.WIFEXITED(status): - if os.WEXITSTATUS(status) != 0: - print 'Command exited with code %d' % os.WEXITSTATUS(status) - else: - print 'Command exited with signal %d' % os.WTERMSIG(status) - - def do_EOF(self, line): - """Exits rpcclient.""" - print - sys.exit(0) - - # SAMR pipe commands - - def do_SamrEnumDomains(self, line): - """Enumerate domain names.""" - - usage = 'usage: SamrEnumDomains' - - if line != '': - print usage - return - - pipe = dcerpc.pipe_connect( - 'ncacn_np:%s' % self.server, - dcerpc.DCERPC_SAMR_UUID, int(dcerpc.DCERPC_SAMR_VERSION), - self.cred) - - connect_handle = samr.Connect(pipe) - - for i in connect_handle.EnumDomains(): - print i - - def do_SamrLookupDomain(self, line): - """Return the SID for a domain.""" - - usage = 'SamrLookupDomain DOMAIN' - - parser = OptionParser(usage) - options, args = parser.parse_args(string.split(line)) - - if len(args) != 1: - print 'usage:', usage - return - - pipe = dcerpc.pipe_connect( - 'ncacn_np:%s' % self.server, - dcerpc.DCERPC_SAMR_UUID, int(dcerpc.DCERPC_SAMR_VERSION), - self.cred) - - connect_handle = samr.Connect(pipe) - - print connect_handle.LookupDomain(args[0]) - - def do_SamrQueryDomInfo(self, line): - """Return information about a domain designated by its SID.""" - - usage = 'SamrQueryDomInfo DOMAIN_SID [info_level]' - - parser = OptionParser(usage) - options, args = parser.parse_args(string.split(line)) - - if (len(args) == 0) or (len(args) > 2): - print 'usage:', usage - return - - pipe = dcerpc.pipe_connect( - 'ncacn_np:%s' % self.server, - dcerpc.DCERPC_SAMR_UUID, int(dcerpc.DCERPC_SAMR_VERSION), - self.cred) - - connect_handle = samr.Connect(pipe) - domain_handle = connect_handle.OpenDomain(args[0]) - - if (len(args) == 2): - result = domain_handle.QueryDomainInfo(int(args[1])) - else: -- Samba Shared Repository