Quanah Gibson-Mount wrote: > --On Wednesday, April 7, 2021 8:02 PM +0000 paul.jc(a)yahoo.com wrote: > > > Hello, > > I am setting up n-way multi-provider replication. > > > > Should we expect a full sync when we setup sync for the first time on a > > new provider? Can we load a backup data.mdb file first and have deltasync > > run from there to prevent a full sync (i.e. to save time)? What is the > > expected behavior/best practice? > Given the known issues in standard syncrepl it is ill advised to do a full > sync. Additionaly it's a waste of time. ;) > > The best practice is to either pre-load the DB via slapadd or provide it a > binary copy of the db from mdb_copy -c. > > > Also, I have some questions around mdb_copy. > > Is there any concern using the -c flag for compacting the data.mdb file > > to be used for restore (or new n-way provider if that works)? I > > notice the compacted file is smaller on disk than the original data.mdb > > (due to the removed freed pages). I assume the data sets are still the > > same? Are there tools to compare the two? > You should use mdb_copy -c. All it does is de-fragment the database as > much as possible. > > > And last question, I am using mdb_copy to create a backup but > > standard > > linux cp to put the new file into /var/lib/ldap/data.mdb. Will this cause > > any issues or is it recommended to use mdb_copy to also move the data.mdb > > file into place? > You can just use cp after you've made the initial copy. > > --Quanah > > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> Quanah Gibson-Mount wrote: > --On Wednesday, April 7, 2021 8:02 PM +0000 paul.jc(a)yahoo.com wrote: > > > Hello, > > I am setting up n-way multi-provider replication. > > > > Should we expect a full sync when we setup sync for the first time on a > > new provider? Can we load a backup data.mdb file first and have deltasync > > run from there to prevent a full sync (i.e. to save time)? What is the > > expected behavior/best practice? > Given the known issues in standard syncrepl it is ill advised to do a full > sync. Additionaly it's a waste of time. ;) > > The best practice is to either pre-load the DB via slapadd or provide it a > binary copy of the db from mdb_copy -c. > > > Also, I have some questions around mdb_copy. > > Is there any concern using the -c flag for compacting the data.mdb file > > to be used for restore (or new n-way provider if that works)? I > > notice the compacted file is smaller on disk than the original data.mdb > > (due to the removed freed pages). I assume the data sets are still the > > same? Are there tools to compare the two? > You should use mdb_copy -c. All it does is de-fragment the database as > much as possible. > > > And last question, I am using mdb_copy to create a backup but > > standard > > linux cp to put the new file into /var/lib/ldap/data.mdb. Will this cause > > any issues or is it recommended to use mdb_copy to also move the data.mdb > > file into place? > You can just use cp after you've made the initial copy. > > --Quanah > > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> Quanah Gibson-Mount wrote: > --On Wednesday, April 7, 2021 8:02 PM +0000 paul.jc(a)yahoo.com wrote: > > > Hello, > > I am setting up n-way multi-provider replication. > > > > Should we expect a full sync when we setup sync for the first time on a > > new provider? Can we load a backup data.mdb file first and have deltasync > > run from there to prevent a full sync (i.e. to save time)? What is the > > expected behavior/best practice? > Given the known issues in standard syncrepl it is ill advised to do a full > sync. Additionaly it's a waste of time. ;) > > The best practice is to either pre-load the DB via slapadd or provide it a > binary copy of the db from mdb_copy -c. > > > Also, I have some questions around mdb_copy. > > Is there any concern using the -c flag for compacting the data.mdb file > > to be used for restore (or new n-way provider if that works)? I > > notice the compacted file is smaller on disk than the original data.mdb > > (due to the removed freed pages). I assume the data sets are still the > > same? Are there tools to compare the two? > You should use mdb_copy -c. All it does is de-fragment the database as > much as possible. > > > And last question, I am using mdb_copy to create a backup but > > standard > > linux cp to put the new file into /var/lib/ldap/data.mdb. Will this cause > > any issues or is it recommended to use mdb_copy to also move the data.mdb > > file into place? > You can just use cp after you've made the initial copy. > > --Quanah > > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> Quanah Gibson-Mount wrote: > --On Wednesday, April 7, 2021 8:02 PM +0000 paul.jc(a)yahoo.com wrote: > > > Hello, > > I am setting up n-way multi-provider replication. > > > > Should we expect a full sync when we setup sync for the first time on a > > new provider? Can we load a backup data.mdb file first and have deltasync > > run from there to prevent a full sync (i.e. to save time)? What is the > > expected behavior/best practice? > Given the known issues in standard syncrepl it is ill advised to do a full > sync. Additionaly it's a waste of time. ;) > > The best practice is to either pre-load the DB via slapadd or provide it a > binary copy of the db from mdb_copy -c. > > > Also, I have some questions around mdb_copy. > > Is there any concern using the -c flag for compacting the data.mdb file > > to be used for restore (or new n-way provider if that works)? I > > notice the compacted file is smaller on disk than the original data.mdb > > (due to the removed freed pages). I assume the data sets are still the > > same? Are there tools to compare the two? > You should use mdb_copy -c. All it does is de-fragment the database as > much as possible. > > > And last question, I am using mdb_copy to create a backup but > > standard > > linux cp to put the new file into /var/lib/ldap/data.mdb. Will this cause > > any issues or is it recommended to use mdb_copy to also move the data.mdb > > file into place? > You can just use cp after you've made the initial copy. > > --Quanah > > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> Quanah Gibson-Mount wrote: > --On Wednesday, April 7, 2021 8:02 PM +0000 paul.jc(a)yahoo.com wrote: > > > Hello, > > I am setting up n-way multi-provider replication. > > > > Should we expect a full sync when we setup sync for the first time on a > > new provider? Can we load a backup data.mdb file first and have deltasync > > run from there to prevent a full sync (i.e. to save time)? What is the > > expected behavior/best practice? > Given the known issues in standard syncrepl it is ill advised to do a full > sync. Additionaly it's a waste of time. ;) > > The best practice is to either pre-load the DB via slapadd or provide it a > binary copy of the db from mdb_copy -c. > > > Also, I have some questions around mdb_copy. > > Is there any concern using the -c flag for compacting the data.mdb file > > to be used for restore (or new n-way provider if that works)? I > > notice the compacted file is smaller on disk than the original data.mdb > > (due to the removed freed pages). I assume the data sets are still the > > same? Are there tools to compare the two? > You should use mdb_copy -c. All it does is de-fragment the database as > much as possible. > > > And last question, I am using mdb_copy to create a backup but > > standard > > linux cp to put the new file into /var/lib/ldap/data.mdb. Will this cause > > any issues or is it recommended to use mdb_copy to also move the data.mdb > > file into place? > You can just use cp after you've made the initial copy. > > --Quanah > > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> Quanah Gibson-Mount wrote: > --On Wednesday, April 7, 2021 8:02 PM +0000 paul.jc(a)yahoo.com wrote: > > > Hello, > > I am setting up n-way multi-provider replication. > > > > Should we expect a full sync when we setup sync for the first time on a > > new provider? Can we load a backup data.mdb file first and have deltasync > > run from there to prevent a full sync (i.e. to save time)? What is the > > expected behavior/best practice? > Given the known issues in standard syncrepl it is ill advised to do a full > sync. Additionaly it's a waste of time. ;) > > The best practice is to either pre-load the DB via slapadd or provide it a > binary copy of the db from mdb_copy -c. > > > Also, I have some questions around mdb_copy. > > Is there any concern using the -c flag for compacting the data.mdb file > > to be used for restore (or new n-way provider if that works)? I > > notice the compacted file is smaller on disk than the original data.mdb > > (due to the removed freed pages). I assume the data sets are still the > > same? Are there tools to compare the two? > You should use mdb_copy -c. All it does is de-fragment the database as > much as possible. > > > And last question, I am using mdb_copy to create a backup but > > standard > > linux cp to put the new file into /var/lib/ldap/data.mdb. Will this cause > > any issues or is it recommended to use mdb_copy to also move the data.mdb > > file into place? > You can just use cp after you've made the initial copy. > > --Quanah > > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> Quanah Gibson-Mount wrote: > --On Wednesday, April 7, 2021 8:02 PM +0000 paul.jc(a)yahoo.com wrote: > > > Hello, > > I am setting up n-way multi-provider replication. > > > > Should we expect a full sync when we setup sync for the first time on a > > new provider? Can we load a backup data.mdb file first and have deltasync > > run from there to prevent a full sync (i.e. to save time)? What is the > > expected behavior/best practice? > Given the known issues in standard syncrepl it is ill advised to do a full > sync. Additionaly it's a waste of time. ;) > > The best practice is to either pre-load the DB via slapadd or provide it a > binary copy of the db from mdb_copy -c. > > > Also, I have some questions around mdb_copy. > > Is there any concern using the -c flag for compacting the data.mdb file > > to be used for restore (or new n-way provider if that works)? I > > notice the compacted file is smaller on disk than the original data.mdb > > (due to the removed freed pages). I assume the data sets are still the > > same? Are there tools to compare the two? > You should use mdb_copy -c. All it does is de-fragment the database as > much as possible. > > > And last question, I am using mdb_copy to create a backup but > > standard > > linux cp to put the new file into /var/lib/ldap/data.mdb. Will this cause > > any issues or is it recommended to use mdb_copy to also move the data.mdb > > file into place? > You can just use cp after you've made the initial copy. > > --Quanah > > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> Hi Quanah & Paul
I am finding myself with similar questions. I have a single provider (provider1) setup with consumers replicating via delta sync. I want to get to an n-way setup as described in the open ldap documentation where I end up with provider1, provider2, and provider3. 1. Do providers need to use delta-sync between themselves in n-way if the consumers are? Do we need to use delta sync for both the config replication and the data replication? So I need two access logs now for both config db and data db? 2. Is copying the mdb binary the best way to provision provider2 and provider3? Why does slapadd support a -S and a -w flag which presumably sets things up for replication properly (serverIDs, CSN calculation), yet we can just copy the mdb binary to the new provider2 and provider3? Why does no serverID or contextCSN need to be considered when copying mdb_copy mdb binaries between providers? 3. I've taken a stab at provisioning new providers for n-way without delta-sync between providers. In some instances I see the new providers undergo a full data sync, in other instances they sometimes seem to realize they are already caught up. Would using delta sync solve this phenomenon? Further, my consumers start panicking and losing 'delta sync' and say they go back to a full sync, however things never 'catch up'. Simply restarting slapd seemed to resolve that. Do you guys find that you often need to restart slapd? 4. Section 18.2.1 in 2.4 Admin Guide explains "Note: since the database state is stored in both the changelog DB and the main DB on the provider, it is important to backup/restore both the changelog DB and the main DB using slapcat/slapadd when restoring a DB or copying it to another machine." Section 18.3.2.2 in 2.4 Admin Guide explains "Note: An accesslog database is unique to a given provider. It should never be replicated." If I just add an mdb binary to my new providers, should I include the access log from the binary it was copied from? I am uncertain when I need to include an access log or not. Seems like the documents say you need it and other resources online think you can start with an empty access log. Appreciate your consideration. Thomas
