The branch, master has been updated via 3bc1cf9 s4-test/repl_schema: remove unused and commented out code via 9c78bdf s4-test/repl_schema: Make sure every object is with unique name via dda73b8 s4-dsdb_schema: Handle remote ATTIDs based on msDs-IntId value via 52f7e38 s4-schema_syntax: Log error message when _dsdb_syntax_OID_oid_drsuapi_to_ldb() fails via 0a9f780 s4-drepl: We won't need a working schema for empty replicas sent. via 2b74838 s4-dsdb_schema: We need base_dn in Schema's shallow copy too via 3a8fa09 s4-schema_syntax: We should use make_ATTID function when converting remote-ATTID to local one via 18744a9 s4-drepl: User working schema for commiting objects when replicating Schema NC via c94e5d4 s4-repl: Allow dsdb_replicated_objects_commit() to use different schema while committing objects via afee8a2 s4-schema_syntax: Use remote prefixMap to map remote ATTID to local one via efcc3c0 s4-test/repl_schema: use 'top' as default base class for our test classSchema objects via 1379075 s4-test/repl_schema: New test to test a classSchema with custom attribute via 79e1a71 s4-dsdb_schema: Seize using global_schema when referencing new schema for an LDB from 4bcedda s3-waf: libwbclient does not depend on talloc anymore.
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 3bc1cf94e3a0febe8d3d53229aa49c3d8eb8c1b6 Author: Kamen Mazdrashki <kame...@samba.org> Date: Wed Dec 15 00:24:23 2010 +0200 s4-test/repl_schema: remove unused and commented out code Autobuild-User: Kamen Mazdrashki <kame...@samba.org> Autobuild-Date: Wed Dec 15 01:37:10 CET 2010 on sn-devel-104 commit 9c78bdfd478ab6229f48de1aa907818151840da4 Author: Kamen Mazdrashki <kame...@samba.org> Date: Mon Dec 13 20:15:26 2010 +0200 s4-test/repl_schema: Make sure every object is with unique name This way, test writer don't have to be careful to choose unique objects suffix commit dda73b85e6e7102a2de6a020f1271140bf8f3aaf Author: Kamen Mazdrashki <kame...@samba.org> Date: Sat Dec 11 01:59:05 2010 +0200 s4-dsdb_schema: Handle remote ATTIDs based on msDs-IntId value If we get such an msDs-IntId value, then we should just use it, there is no mapping available for such values commit 52f7e38d6a350cfd645371c82a8c2b189cf7531f Author: Kamen Mazdrashki <kame...@samba.org> Date: Fri Dec 10 04:22:58 2010 +0200 s4-schema_syntax: Log error message when _dsdb_syntax_OID_oid_drsuapi_to_ldb() fails I haven't found a way to test this function during replication so far, but when I do, it will be useful to notice this error in the log file commit 0a9f780d26d8d193081915f4f3aff7b7f8335a28 Author: Kamen Mazdrashki <kame...@samba.org> Date: Fri Dec 10 04:17:09 2010 +0200 s4-drepl: We won't need a working schema for empty replicas sent. Without this check, receiving empty replica leads to a situation where we left with a working_schema attached to the ldb. The problem here is that working_schema is not fully functional schema cache and keeping it attached to the ldb may lead to modules failing to accomplish their jobs commit 2b74838c14f4ee77236634c7b10b8ac26eff40fa Author: Kamen Mazdrashki <kame...@samba.org> Date: Fri Dec 10 04:08:58 2010 +0200 s4-dsdb_schema: We need base_dn in Schema's shallow copy too commit 3a8fa09c4f1691346dadf975bc456e696b09ef65 Author: Kamen Mazdrashki <kame...@samba.org> Date: Fri Dec 10 04:03:00 2010 +0200 s4-schema_syntax: We should use make_ATTID function when converting remote-ATTID to local one We may have no prefix for the remote ATTID (remote OID strictly speaking) So this is the place for us to update our local prefixMap adding a prefix for the numeric OID we've recived commit 18744a95312666cad2c646c2bb550da4277968c1 Author: Kamen Mazdrashki <kame...@samba.org> Date: Fri Dec 10 03:55:24 2010 +0200 s4-drepl: User working schema for commiting objects when replicating Schema NC commit c94e5d44385d23172a8776e4c12d71e30c9c6616 Author: Kamen Mazdrashki <kame...@samba.org> Date: Fri Dec 10 02:55:30 2010 +0200 s4-repl: Allow dsdb_replicated_objects_commit() to use different schema while committing objects working_schema is to be used while committing a Schema replica. When we replicate Schema, then we most probably won't be able to convert all replicated objects using the current Schema cache (as we don't know anything about those new objects). Thus, during Schema replication, we make a temporary working_schema that contains both our current Schema + all objects we get on the wire. When we commit those new objects, we should use our working_schema (by setting it to the ldb), and after all changes are commited, we can refresh the schema cache so we have a brand new, full-featured Schema cache commit afee8a28c408efd3323b9064929b00be6eb89815 Author: Kamen Mazdrashki <kame...@samba.org> Date: Thu Dec 9 04:45:17 2010 +0200 s4-schema_syntax: Use remote prefixMap to map remote ATTID to local one in dsdb_attribute_drsuapi_to_ldb() function. drsuapi_DsReplicaAttribute *in parameter come from remote DC so we can't rely on in->attid to map it directly to an dsdb_attribute in our local schema cache commit efcc3c0d796d66c5eddeabf5f8fe00769a17b72b Author: Kamen Mazdrashki <kame...@samba.org> Date: Thu Dec 9 04:57:08 2010 +0200 s4-test/repl_schema: use 'top' as default base class for our test classSchema objects Otherwise we will end up passing whole inheritance chain every time we create some new fancy classSchema object (as the 'cls-A' and 'cls-B' ones in test_classWithCustomAttribute test) commit 13790757620827eed73a81462386d269bdfa0fa7 Author: Kamen Mazdrashki <kame...@samba.org> Date: Thu Dec 9 04:42:13 2010 +0200 s4-test/repl_schema: New test to test a classSchema with custom attribute Create new Attribute and a Class, that has value for newly created attribute. This should check code path that searches for AttributeID_id in Schema cacheThis test. It also tests how we replicate a leaf classSchema that inherits from a new classSchema with attribute added - tests both dsdb_attribute_drsuapi_to_ldb() and _dsdb_syntax_OID_obj_drsuapi_to_ldb() syntax handler commit 79e1a71e5673de5d2e5bf78c7c2adbcd59bbc004 Author: Kamen Mazdrashki <kame...@samba.org> Date: Thu Dec 9 04:31:14 2010 +0200 s4-dsdb_schema: Seize using global_schema when referencing new schema for an LDB Without this change, when a schema is set to ldb, the effect is that dsdb_get_schema() returns global_schema preferably. Thus we end up with two schemas in effect: - global one, which is the old one and it is still used everywhere - new one, which is just cached in ldb, but can't be used, as there is no way to access it ----------------------------------------------------------------------- Summary of changes: source4/dsdb/repl/drepl_out_helpers.c | 15 +++++--- source4/dsdb/repl/replicated_objects.c | 55 +++++++++++++++++++++++++++++ source4/dsdb/schema/schema_init.c | 8 ++++- source4/dsdb/schema/schema_set.c | 6 +++ source4/dsdb/schema/schema_syntax.c | 39 +++++++++++++++++---- source4/libnet/libnet_vampire.c | 4 +- source4/torture/drs/python/repl_schema.py | 30 ++++++++++++++-- 7 files changed, 138 insertions(+), 19 deletions(-) Changeset truncated at 500 lines: diff --git a/source4/dsdb/repl/drepl_out_helpers.c b/source4/dsdb/repl/drepl_out_helpers.c index f02fae9..6bb2516 100644 --- a/source4/dsdb/repl/drepl_out_helpers.c +++ b/source4/dsdb/repl/drepl_out_helpers.c @@ -542,7 +542,7 @@ static void dreplsrv_op_pull_source_apply_changes_trigger(struct tevent_req *req struct dreplsrv_partition *partition = state->op->source_dsa->partition; struct dreplsrv_drsuapi_connection *drsuapi = state->op->source_dsa->conn->drsuapi; struct dsdb_schema *schema; - struct dsdb_schema *working_schema; + struct dsdb_schema *working_schema = NULL; const struct drsuapi_DsReplicaOIDMapping_Ctr *mapping_ctr; uint32_t object_count; struct drsuapi_DsReplicaObjectListItemEx *first_object; @@ -588,8 +588,11 @@ static void dreplsrv_op_pull_source_apply_changes_trigger(struct tevent_req *req return; } - /* Decide what working schema to use for object conversion */ - if (ldb_dn_compare(partition->dn, ldb_get_schema_basedn(service->samdb)) == 0) { + /* + * Decide what working schema to use for object conversion. + * We won't need a working schema for empty replicas sent. + */ + if (first_object && ldb_dn_compare(partition->dn, schema->base_dn) == 0) { /* create working schema to convert objects with */ status = dsdb_repl_make_working_schema(service->samdb, schema, @@ -604,12 +607,10 @@ static void dreplsrv_op_pull_source_apply_changes_trigger(struct tevent_req *req tevent_req_nterror(req, NT_STATUS_INTERNAL_ERROR); return; } - } else { - working_schema = schema; } status = dsdb_replicated_objects_convert(service->samdb, - working_schema, + working_schema ? working_schema : schema, partition->nc.dn, mapping_ctr, object_count, @@ -629,6 +630,7 @@ static void dreplsrv_op_pull_source_apply_changes_trigger(struct tevent_req *req } status = dsdb_replicated_objects_commit(service->samdb, + working_schema, objects, &state->op->source_dsa->notify_uSN); talloc_free(objects); @@ -639,6 +641,7 @@ static void dreplsrv_op_pull_source_apply_changes_trigger(struct tevent_req *req tevent_req_nterror(req, nt_status); return; } + if (state->op->extended_op == DRSUAPI_EXOP_NONE) { /* if it applied fine, we need to update the highwatermark */ *state->op->source_dsa->repsFrom1 = rf1; diff --git a/source4/dsdb/repl/replicated_objects.c b/source4/dsdb/repl/replicated_objects.c index 1ea1640..f3b6356 100644 --- a/source4/dsdb/repl/replicated_objects.c +++ b/source4/dsdb/repl/replicated_objects.c @@ -433,11 +433,20 @@ WERROR dsdb_replicated_objects_convert(struct ldb_context *ldb, return WERR_OK; } +/** + * Commits a list of replicated objects. + * + * @param working_schema dsdb_schema to be used for resolving + * Classes/Attributes during Schema replication. If not NULL, + * it will be set on ldb and used while committing replicated objects + */ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, + struct dsdb_schema *working_schema, struct dsdb_extended_replicated_objects *objects, uint64_t *notify_uSN) { struct ldb_result *ext_res; + struct dsdb_schema *cur_schema = NULL; int ret; uint64_t seq_num1, seq_num2; @@ -459,8 +468,33 @@ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, return WERR_FOOBAR; } + /* + * Set working_schema for ldb in case we are replicating from Schema NC. + * Schema won't be reloaded during Replicated Objects commit, as it is + * done in a transaction. So we need some way to search for newly + * added Classes and Attributes + */ + if (working_schema) { + /* store current schema so we can fall back in case of failure */ + cur_schema = dsdb_get_schema(ldb, objects); + + ret = dsdb_reference_schema(ldb, working_schema, false); + if (ret != LDB_SUCCESS) { + DEBUG(0,(__location__ "Failed to reference working schema - %s\n", + ldb_strerror(ret))); + /* TODO: Map LDB Error to NTSTATUS? */ + ldb_transaction_cancel(ldb); + return WERR_INTERNAL_ERROR; + } + } + ret = ldb_extended(ldb, DSDB_EXTENDED_REPLICATED_OBJECTS_OID, objects, &ext_res); if (ret != LDB_SUCCESS) { + /* restore previous schema */ + if (cur_schema ) { + dsdb_reference_schema(ldb, cur_schema, false); + } + DEBUG(0,("Failed to apply records: %s: %s\n", ldb_errstring(ldb), ldb_strerror(ret))); ldb_transaction_cancel(ldb); @@ -470,6 +504,10 @@ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, ret = ldb_transaction_prepare_commit(ldb); if (ret != LDB_SUCCESS) { + /* restore previous schema */ + if (cur_schema ) { + dsdb_reference_schema(ldb, cur_schema, false); + } DEBUG(0,(__location__ " Failed to prepare commit of transaction: %s\n", ldb_errstring(ldb))); return WERR_FOOBAR; @@ -477,6 +515,10 @@ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, ret = dsdb_load_partition_usn(ldb, objects->partition_dn, &seq_num2, NULL); if (ret != LDB_SUCCESS) { + /* restore previous schema */ + if (cur_schema ) { + dsdb_reference_schema(ldb, cur_schema, false); + } DEBUG(0,(__location__ " Failed to load partition uSN\n")); ldb_transaction_cancel(ldb); return WERR_FOOBAR; @@ -491,10 +533,23 @@ WERROR dsdb_replicated_objects_commit(struct ldb_context *ldb, ret = ldb_transaction_commit(ldb); if (ret != LDB_SUCCESS) { + /* restore previous schema */ + if (cur_schema ) { + dsdb_reference_schema(ldb, cur_schema, false); + } DEBUG(0,(__location__ " Failed to commit transaction\n")); return WERR_FOOBAR; } + /* + * Reset the Schema used by ldb. This will lead to + * a schema cache being refreshed from database. + */ + if (working_schema) { + cur_schema = dsdb_get_schema(ldb, NULL); + /* TODO: What we do in case dsdb_get_schema() fail? + * We can't fallback at this point anymore */ + } DEBUG(2,("Replicated %u objects (%u linked attributes) for %s\n", objects->num_objects, objects->linked_attributes_count, diff --git a/source4/dsdb/schema/schema_init.c b/source4/dsdb/schema/schema_init.c index 977941e..00170a2 100644 --- a/source4/dsdb/schema/schema_init.c +++ b/source4/dsdb/schema/schema_init.c @@ -57,11 +57,17 @@ struct dsdb_schema *dsdb_schema_copy_shallow(TALLOC_CTX *mem_ctx, return NULL; } + /* schema base_dn */ + schema_copy->base_dn = ldb_dn_copy(schema_copy, schema->base_dn); + if (!schema_copy->base_dn) { + goto failed; + } + /* copy prexiMap & schemaInfo */ schema_copy->prefixmap = dsdb_schema_pfm_copy_shallow(schema_copy, schema->prefixmap); if (!schema_copy->prefixmap) { - return NULL; + goto failed; } schema_copy->schema_info = talloc_strdup(schema_copy, schema->schema_info); diff --git a/source4/dsdb/schema/schema_set.c b/source4/dsdb/schema/schema_set.c index 2134115..004d7eb 100644 --- a/source4/dsdb/schema/schema_set.c +++ b/source4/dsdb/schema/schema_set.c @@ -452,6 +452,12 @@ int dsdb_reference_schema(struct ldb_context *ldb, struct dsdb_schema *schema, return ldb_oom(ldb); } + /* Make this ldb use local schema preferably */ + ret = ldb_set_opaque(ldb, "dsdb_use_global_schema", NULL); + if (ret != LDB_SUCCESS) { + return ret; + } + ret = dsdb_schema_set_attributes(ldb, schema, write_attributes); if (ret != LDB_SUCCESS) { return ret; diff --git a/source4/dsdb/schema/schema_syntax.c b/source4/dsdb/schema/schema_syntax.c index 2f8f9a5..1bb9a4f 100644 --- a/source4/dsdb/schema/schema_syntax.c +++ b/source4/dsdb/schema/schema_syntax.c @@ -97,7 +97,7 @@ static bool dsdb_syntax_attid_from_remote_attid(const struct dsdb_syntax_ctx *ct return false; } - werr = dsdb_schema_pfm_attid_from_oid(ctx->schema->prefixmap, oid, id_local); + werr = dsdb_schema_pfm_make_attid(ctx->schema->prefixmap, oid, id_local); if (!W_ERROR_IS_OK(werr)) { DEBUG(0,("OID->ATTID failed (%s) for: %s\n", win_errstr(werr), oid)); return false; @@ -1094,7 +1094,11 @@ static WERROR _dsdb_syntax_OID_oid_drsuapi_to_ldb(const struct dsdb_syntax_ctx * status = dsdb_schema_pfm_oid_from_attid(ctx->pfm_remote, attid, out->values, &oid); - W_ERROR_NOT_OK_RETURN(status); + if (!W_ERROR_IS_OK(status)) { + DEBUG(0,(__location__ ": Error: Unknown ATTID 0x%08X\n", + attid)); + return status; + } out->values[i] = data_blob_string_const(oid); } @@ -2642,17 +2646,38 @@ WERROR dsdb_attribute_drsuapi_to_ldb(struct ldb_context *ldb, { const struct dsdb_attribute *sa; struct dsdb_syntax_ctx syntax_ctx; + uint32_t attid_local; + + /* use default syntax conversion context */ + dsdb_syntax_ctx_init(&syntax_ctx, ldb, schema); + syntax_ctx.pfm_remote = pfm_remote; - sa = dsdb_attribute_by_attributeID_id(schema, in->attid); + switch (dsdb_pfm_get_attid_type(in->attid)) { + case DSDB_ATTID_TYPE_PFM: + /* map remote ATTID to local ATTID */ + if (!dsdb_syntax_attid_from_remote_attid(&syntax_ctx, mem_ctx, in->attid, &attid_local)) { + DEBUG(0,(__location__ ": Can't find local ATTID for 0x%08X\n", + in->attid)); + return WERR_FOOBAR; + } + break; + case DSDB_ATTID_TYPE_INTID: + /* use IntId value directly */ + attid_local = in->attid; + break; + default: + /* we should never get here */ + DEBUG(0,(__location__ ": Invalid ATTID type passed for conversion - 0x%08X\n", + in->attid)); + return WERR_INVALID_PARAMETER; + } + + sa = dsdb_attribute_by_attributeID_id(schema, attid_local); if (!sa) { DEBUG(1,(__location__ ": Unknown attributeID_id 0x%08X\n", in->attid)); return WERR_FOOBAR; } - /* use default syntax conversion context */ - dsdb_syntax_ctx_init(&syntax_ctx, ldb, schema); - syntax_ctx.pfm_remote = pfm_remote; - return sa->syntax->drsuapi_to_ldb(&syntax_ctx, sa, in, mem_ctx, out); } diff --git a/source4/libnet/libnet_vampire.c b/source4/libnet/libnet_vampire.c index 1d7d726..40cf01e 100644 --- a/source4/libnet/libnet_vampire.c +++ b/source4/libnet/libnet_vampire.c @@ -446,7 +446,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s } } - status = dsdb_replicated_objects_commit(s->ldb, schema_objs, &seq_num); + status = dsdb_replicated_objects_commit(s->ldb, NULL, schema_objs, &seq_num); if (!W_ERROR_IS_OK(status)) { DEBUG(0,("Failed to commit objects: %s\n", win_errstr(status))); return werror_to_ntstatus(status); @@ -720,7 +720,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data, NDR_PRINT_DEBUG(replPropertyMetaDataBlob, objs->objects[i].meta_data); } } - status = dsdb_replicated_objects_commit(s->ldb, objs, &seq_num); + status = dsdb_replicated_objects_commit(s->ldb, NULL, objs, &seq_num); if (!W_ERROR_IS_OK(status)) { DEBUG(0,("Failed to commit objects: %s\n", win_errstr(status))); return werror_to_ntstatus(status); diff --git a/source4/torture/drs/python/repl_schema.py b/source4/torture/drs/python/repl_schema.py index 9bb12a2..ea44626 100644 --- a/source4/torture/drs/python/repl_schema.py +++ b/source4/torture/drs/python/repl_schema.py @@ -53,6 +53,8 @@ class DrsReplSchemaTestCase(samba.tests.TestCase): # prefix for all objects created obj_prefix = None + # current Class or Attribute object id + obj_id = 0 def setUp(self): super(DrsReplSchemaTestCase, self).setUp() @@ -68,7 +70,7 @@ class DrsReplSchemaTestCase(samba.tests.TestCase): # initialize objects prefix if not done yet if self.obj_prefix is None: t = time.strftime("%s", time.gmtime()) - DrsReplSchemaTestCase.obj_prefix = "DrsReplSchema-%s-" % t + DrsReplSchemaTestCase.obj_prefix = "DrsReplSchema-%s" % t # cache some of RootDSE props self.schema_dn = self.info_dc1["schemaNamingContext"][0] @@ -111,7 +113,8 @@ class DrsReplSchemaTestCase(samba.tests.TestCase): def _make_obj_names(self, base_name): '''Try to create a unique name for an object that is to be added to schema''' - obj_name = self.obj_prefix + base_name + self.obj_id += 1 + obj_name = "%s-%d-%s" % (self.obj_prefix, self.obj_id, base_name) obj_ldn = obj_name.replace("-", "") obj_dn = "CN=%s,%s" % (obj_name, self.schema_dn) return (obj_dn, obj_name, obj_ldn) @@ -125,7 +128,7 @@ class DrsReplSchemaTestCase(samba.tests.TestCase): "governsId": "1.2.840." + str(random.randint(1,100000)) + ".1.5.13", "instanceType": "4", "objectClassCategory": "1", - "subClassOf": "organizationalPerson", + "subClassOf": "top", "systemOnly": "FALSE"} # allow overriding/adding attributes if not attrs is None: @@ -207,6 +210,27 @@ class DrsReplSchemaTestCase(samba.tests.TestCase): for c_dn in c_dn_list: self._check_object(c_dn) + def test_classWithCustomAttribute(self): + """Create new Attribute and a Class, + that has value for newly created attribute. + This should check code path that searches for + AttributeID_id in Schema cache""" + # add new attributeSchema object + (a_ldn, a_dn) = self._schema_new_attr(self.ldb_dc1, "attr-A") + # add a base classSchema class so we can use our new + # attribute in class definition in a sibling class + (c_ldn, c_dn) = self._schema_new_class(self.ldb_dc1, "cls-A", + {"systemMayContain": a_ldn}) + # add new classSchema object with value for a_ldb attribute + (c_ldn, c_dn) = self._schema_new_class(self.ldb_dc1, "cls-B", + {"objectClass": ["top", "classSchema", c_ldn], + a_ldn: "test_classWithCustomAttribute"}) + # force replication from DC1 to DC2 + self._net_drs_replicate(DC=self.dnsname_dc2, fromDC=self.dnsname_dc1, nc_dn=self.schema_dn) + # check objects are replicated + self._check_object(c_dn) + self._check_object(a_dn) + def test_attribute(self): """Simple test for attributeSchema replication""" # add new attributeSchema object -- Samba Shared Repository