Re: Slapd with mysql and unixODBC
Hi, Ace [EMAIL PROTECTED] writes: Hi All, Can anybody tell me which all packages one will need to configure LDAP with mysql? I have, unixODBC unixODBC-devel Connector/ODBC 3.51 MySQL 5.0 and LDAP - 2.3.32 Still I am facing error while starting slapd - ./slapd: error while loading shared libraries: libiodbc.so.2: cannot open shared object file: No such file or directory I have successfully build slapd with unixODBC-devel and use MySQL-4.1.10. -Dieter -- Dieter Klünter | Systemberatung http://www.dkluenter.de GPG Key ID:8EF7B6C6
Re: duplicate attributetype: 2.5.4.2
Quanah Gibson-Mount wrote: --On Thursday, May 24, 2007 8:03 PM -0400 Pedro Espinoza [EMAIL PROTECTED] wrote: Gurus: I am newbie to this ldap thing. I followed garytt's installation guide. When I tried to start slapd, I got the following error. I searched on net, but in vain. Can you shed some light on it, and how to rectify it. /usr/local/etc/openldap/schema/core.schema: line 77: Duplicate attributeType: 2.5.4.2 Ignore this warning, it is harmless. Actually it most likely indicates that you're using an older core.schema file with a newer slapd, and it's not just a warning, it's a fatal error. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LDAP and MySQL - No database selected
Yes. On same directory with structural *.sql you have data *.sql scripts. You can import this sql files. Cheers! On 5/25/07, Ace [EMAIL PROTECTED] wrote: I could get LDAP started with mysql. One query - === [EMAIL PROTECTED] libexec]# ldapsearch -x -b '' -s base '(objectclass=*)' namingContexts* # extended LDIF # # LDAPv3 # base with scope baseObject # filter: (objectclass=*) # requesting: namingContexts # # dn: namingContexts: o=sql,c=RU # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 [EMAIL PROTECTED] libexec]# ldapsearch -LLL -s sub -b o=sql,c=RU (objectClass=*) *SASL/DIGEST-MD5 authentication started Please enter your password: ldap_sasl_interactive_bind_s: Invalid credentials (49) additional info: SASL(-13): user not found: no secret in database == Why is my second LDAP search not working? Anyone has samples on inserting/searching records from LDAP with MySQL backend? On 5/24/07, Ace [EMAIL PROTECTED] wrote: I got it working now after I restrted my LinuxIts weird...but its true! Thanks for you help! On 5/23/07, Ace [EMAIL PROTECTED] wrote: The funny part here, my odbc test result - == 1: SQLDriverConnect = [MySQL][ODBC 3.51 Driver]Could not determine the driver name; could not lookup setup library. DSN=(test) (0) SQLSTATE=HY000 1: ODBC_Connect = [MySQL][ODBC 3.51 Driver]Could not determine the driver name; could not lookup setup library. DSN=(test) (0) SQLSTATE=HY000 == Where as LDAP is able to connect to MySQL. I am using test database from MySQL and DSN name is test. One more thing, why was slapd looking for /tmp/mysql.sock? I had specified Socket = /var/lib/mysql/mysql.sock Here is my odbc.ini - ; begin odbc.ini [ODBC Data Sources] test = Connector/ODBC 3.51 Driver DSN [test] Driver = /usr/lib/libmyodbc3.so Description = Connector/ODBC 3.51 Driver DSN Server = localhost DSN = test Port = 3306 Socket = /var/lib/mysql/mysql.sock User = root Password = mysql Database = test ServerType = MySQL Option = TraceFile = /var/log/mysql_test_trace.log Trace = 0 [Default] Driver = /usr/lib/libmyodbc3.so Description = Connector/ODBC 3.51 Driver DSN Server = localhost DSN = test Port = 3306 Socket = /var/lib/mysql/mysql.sock User = root Password = mysql Database = test ServerType = MySQL Option = TraceFile = /var/log/mysql_test_trace.log Trace = 0 ;end odbc.ini On 5/23/07, Gabriel Stein [EMAIL PROTECTED] wrote: What´s your result for odbctest ou iodbctest? This command shows your ODBC connection? And more: you set your slapd.conf with your ODBC Connection Name, not your MySQL dbname. For example, if you have a database with name ldap and setup a ODBC connection with name test, you need set test on your slapd.conf An ODBC Connection for example, in your /etc/odbc.ini [ODBC Data Sources] test = MySQL ODBC 2.50 Driver DSN [test] Driver = /usr/local/lib/libmyodbc.so Description = MySQL ODBC 2.50 Driver DSN DSN = test Server = localhost PORT = 3306 SOCKET = /var/lib/mysql/mysql.sock User = *valid_user* Password = *valid_passwd* Database = ldap ReadOnly = no ServerType = MySQL FetchBufferSize = 99 ServerOptions = ConnectOptions = OPTION = 3 TraceFile = /var/log/mysql_test_trace.log Trace = 0 Cheers, On 5/23/07, Ace [EMAIL PROTECTED] wrote: Hi Friends, I am trying to configure LDAP with MySQL 5.0 as backend. I am using Connector/ODBC 3.51 and LDAP 2.3.32 built from sources. While starting slapd I get following error - === backsql_load_schema_map(): oc_query SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings backsql_load_schema_map(): error executing oc_query: Return code: -1 nativeErrCode=1046 SQLengineState=3D000 msg=[MySQL][ODBC 3.51Driver][ mysqld-5.0.27-standard]No database selected ==backsql_free_db_conn() === My odbc.ini file has all the details and I can see in slapd log that its getting dbname before connective to MySQL server. slapd.conf has dbname = test. Thanks in advance! -- Cheers, Rajan -- /\ Gabriel Stein [EMAIL PROTECTED] MSN: [EMAIL PROTECTED] Administrador de Redes - Network Administrador Linux User #223750 51-92796310 Porto Alegre - RS - Brasil -- I may be miles away... but I am just a mail away so keep mailing...!!! Cheers, Rajan -- /\ Gabriel Stein [EMAIL PROTECTED] MSN: [EMAIL
Re: Help with Syncrepl
My Provider settings are ### #Replication Stuff# ### #updateref the ldap server to which clients submit update requests updateref ldap://pi.sunrocket.com #Provider (master) must be implemeneted as an overlay overlay syncprov #syncprov-checkpoint ops minutes syncprov-checkpoint 100 10 #syncprov-sessionlog size syncprov-sessionlog 100 On May 24, 2007, at 7:23 PM, Gavin Henry wrote: What are your provider settings? On 24/05/07, Steven Bambling [EMAIL PROTECTED] wrote: All, I am in the process of setting up replication between 2 ldap servers...I am moving from the older slupd to syncrepl. Wh I try to start the ldap server after adding in the necessary config into slapd.conf file I get this error. syncrepl: database already shadowed Below is the parameters that I am using for syncrepl any help or a point in the correct direction would be much appreciated. ### #Replication Stuff# ### #updated=the DN allowed to make changes to the replica (masteer) updatedncn=copycat,dc=srtest,dc=com #updateref the ldap server to which clients submit update requests updateref ldap://pi.sunrocket.com #syncrepl rid=replica ID syncreplrid=420 #Address of the provider (master) ldap server provider=ldap://pi.sunrocket.com:389 #tyep=refreshOnly|refreshAndPersist type=refreshAndPersist #interval=dd:hh:mm:ss interval=01:00:00:00 #retry=retry interval # of retries retry=60 10 300 3 #searchbase=base DN searchbase=dc=srtest,dc=com #scope=sub|one|base scope=sub #filter=filter string filter=(objectClass=*) #attrs=attr list attrs=*,+ #schemachecking=on|off schemachecking=off Thanks, STEVE
Re: duplicate attributetype: 2.5.4.2
--On Friday, May 25, 2007 2:31 AM -0700 Howard Chu [EMAIL PROTECTED] wrote: Quanah Gibson-Mount wrote: --On Thursday, May 24, 2007 8:03 PM -0400 Pedro Espinoza [EMAIL PROTECTED] wrote: Gurus: I am newbie to this ldap thing. I followed garytt's installation guide. When I tried to start slapd, I got the following error. I searched on net, but in vain. Can you shed some light on it, and how to rectify it. /usr/local/etc/openldap/schema/core.schema: line 77: Duplicate attributeType: 2.5.4.2 Ignore this warning, it is harmless. Actually it most likely indicates that you're using an older core.schema file with a newer slapd, and it's not just a warning, it's a fatal error. Oh, I was thinking of those annoying schema warning messages that have been going on in 2.3 for a while. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: duplicate attributetype: 2.5.4.2
On 5/25/07, Quanah Gibson-Mount [EMAIL PROTECTED] wrote: --On Friday, May 25, 2007 2:31 AM -0700 Howard Chu [EMAIL PROTECTED] wrote: Quanah Gibson-Mount wrote: --On Thursday, May 24, 2007 8:03 PM -0400 Pedro Espinoza [EMAIL PROTECTED] wrote: Gurus: I am newbie to this ldap thing. I followed garytt's installation guide. When I tried to start slapd, I got the following error. I searched on net, but in vain. Can you shed some light on it, and how to rectify it. /usr/local/etc/openldap/schema/core.schema: line 77: Duplicate attributeType: 2.5.4.2 Ignore this warning, it is harmless. Actually it most likely indicates that you're using an older core.schema file with a newer slapd, and it's not just a warning, it's a fatal error. Oh, I was thinking of those annoying schema warning messages that have been going on in 2.3 for a while. I have figured out: my slapd.conf contains two entries for core.schema. I removed the other one, it is working, Thanks, Pedro.
Re: LDAP and MySQL - No database selected
I could get LDAP started with mysql. One query - === [EMAIL PROTECTED] libexec]# ldapsearch -x -b '' -s base '(objectclass=*)' namingContexts* # extended LDIF # # LDAPv3 # base with scope baseObject # filter: (objectclass=*) # requesting: namingContexts # # dn: namingContexts: o=sql,c=RU # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 [EMAIL PROTECTED] libexec]# ldapsearch -LLL -s sub -b o=sql,c=RU (objectClass=*) *SASL/DIGEST-MD5 authentication started Please enter your password: ldap_sasl_interactive_bind_s: Invalid credentials (49) additional info: SASL(-13): user not found: no secret in database == Why is my second LDAP search not working? Anyone has samples on inserting/searching records from LDAP with MySQL backend? On 5/24/07, Ace [EMAIL PROTECTED] wrote: I got it working now after I restrted my LinuxIts weird...but its true! Thanks for you help! On 5/23/07, Ace [EMAIL PROTECTED] wrote: The funny part here, my odbc test result - == 1: SQLDriverConnect = [MySQL][ODBC 3.51 Driver]Could not determine the driver name; could not lookup setup library. DSN=(test) (0) SQLSTATE=HY000 1: ODBC_Connect = [MySQL][ODBC 3.51 Driver]Could not determine the driver name; could not lookup setup library. DSN=(test) (0) SQLSTATE=HY000 == Where as LDAP is able to connect to MySQL. I am using test database from MySQL and DSN name is test. One more thing, why was slapd looking for /tmp/mysql.sock? I had specified Socket = /var/lib/mysql/mysql.sock Here is my odbc.ini - ; begin odbc.ini [ODBC Data Sources] test = Connector/ODBC 3.51 Driver DSN [test] Driver = /usr/lib/libmyodbc3.so Description = Connector/ODBC 3.51 Driver DSN Server = localhost DSN = test Port = 3306 Socket = /var/lib/mysql/mysql.sock User = root Password = mysql Database = test ServerType = MySQL Option = TraceFile = /var/log/mysql_test_trace.log Trace = 0 [Default] Driver = /usr/lib/libmyodbc3.so Description = Connector/ODBC 3.51 Driver DSN Server = localhost DSN = test Port = 3306 Socket = /var/lib/mysql/mysql.sock User = root Password = mysql Database = test ServerType = MySQL Option = TraceFile = /var/log/mysql_test_trace.log Trace = 0 ;end odbc.ini On 5/23/07, Gabriel Stein [EMAIL PROTECTED] wrote: What´s your result for odbctest ou iodbctest? This command shows your ODBC connection? And more: you set your slapd.conf with your ODBC Connection Name, not your MySQL dbname. For example, if you have a database with name ldap and setup a ODBC connection with name test, you need set test on your slapd.conf An ODBC Connection for example, in your /etc/odbc.ini [ODBC Data Sources] test = MySQL ODBC 2.50 Driver DSN [test] Driver = /usr/local/lib/libmyodbc.so Description = MySQL ODBC 2.50 Driver DSN DSN = test Server = localhost PORT = 3306 SOCKET = /var/lib/mysql/mysql.sock User = *valid_user* Password = *valid_passwd* Database = ldap ReadOnly = no ServerType = MySQL FetchBufferSize = 99 ServerOptions = ConnectOptions = OPTION = 3 TraceFile = /var/log/mysql_test_trace.log Trace = 0 Cheers, On 5/23/07, Ace [EMAIL PROTECTED] wrote: Hi Friends, I am trying to configure LDAP with MySQL 5.0 as backend. I am using Connector/ODBC 3.51 and LDAP 2.3.32 built from sources. While starting slapd I get following error - === backsql_load_schema_map(): oc_query SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings backsql_load_schema_map(): error executing oc_query: Return code: -1 nativeErrCode=1046 SQLengineState=3D000 msg=[MySQL][ODBC 3.51Driver][ mysqld-5.0.27-standard]No database selected ==backsql_free_db_conn() === My odbc.ini file has all the details and I can see in slapd log that its getting dbname before connective to MySQL server. slapd.conf has dbname = test. Thanks in advance! -- Cheers, Rajan -- /\ Gabriel Stein [EMAIL PROTECTED] MSN: [EMAIL PROTECTED] Administrador de Redes - Network Administrador Linux User #223750 51-92796310 Porto Alegre - RS - Brasil -- I may be miles away... but I am just a mail away so keep mailing...!!! Cheers, Rajan
Re: how to create db files
--On May 25, 2007 11:32:48 AM -0700 Craig [EMAIL PROTECTED] wrote: Uh... it won't start without the id2entry file: Checking configuration files for : bdb_db_open: db_open(/var/lib/ldap/id2entry.bdb) failed: No such file or directory (2) Any other ideas? I said, load the server with your initial LDIF file (i.e., use slapadd). That will create the id2entry file. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: how to create db files
--On May 25, 2007 11:54:30 AM -0700 Craig [EMAIL PROTECTED] wrote: Sorry, I had only used ldapadd up till now. Thanx so much! BTW, I accidently used an older version of slapadd at first (2.2.13) and I get the following error: # slapadd -f /etc/openldap/slapd.conf -l base.ldif /etc/openldap/schema/core.schema: line 366: AttributeType not found: description slapadd: bad configuration file! Does anyone know what changed? I looked at the CHANGES file for 2.3.35 and saw a few changes related to schema handling, but am not sure which is directly related to the above. (BTW, I am just curious... The add works just fine when I use slapadd from the 2.3.35 release.) Again, it all works now. Thanx again! OpenLDAP 2.2 and OpenLDAP 2.3 use different schemes for storing the data in the BDB database, so db's created with a 2.2 slapadd are not compatible with 2.3. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: how to create db files
Quanah Gibson-Mount wrote: --On May 25, 2007 11:32:48 AM -0700 Craig [EMAIL PROTECTED] wrote: Uh... it won't start without the id2entry file: Checking configuration files for : bdb_db_open: db_open(/var/lib/ldap/id2entry.bdb) failed: No such file or directory (2) This is just coming from a slaptest invocation in your init script. Comment out the slaptest, or start slapd by hand. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: how to create db files
Sorry, I had only used ldapadd up till now. Thanx so much! BTW, I accidently used an older version of slapadd at first (2.2.13) and I get the following error: # slapadd -f /etc/openldap/slapd.conf -l base.ldif /etc/openldap/schema/core.schema: line 366: AttributeType not found: description slapadd: bad configuration file! Does anyone know what changed? I looked at the CHANGES file for 2.3.35 and saw a few changes related to schema handling, but am not sure which is directly related to the above. (BTW, I am just curious... The add works just fine when I use slapadd from the 2.3.35 release.) Again, it all works now. Thanx again! Craig Quanah Gibson-Mount wrote: --On May 25, 2007 11:32:48 AM -0700 Craig [EMAIL PROTECTED] wrote: Uh... it won't start without the id2entry file: Checking configuration files for : bdb_db_open: db_open(/var/lib/ldap/id2entry.bdb) failed: No such file or directory (2) Any other ideas? I said, load the server with your initial LDIF file (i.e., use slapadd). That will create the id2entry file. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
2.3 slapd_db_recover
I had been using openldap 2.2.13 installed via yum. I installed 2.3.35 via src. But, I don't see slapd_db_recover (or any of the related apps; slapd_db_checkpoint, slapd_db_dump, etc.) I also did a search thru the 2.3.35 src files and I don't see any special configure/make flags that builds them. Am I missing something? Is there is tools package I need to install? TIA!
server startup overhead
I continue to have trouble with getting a freshly started server to be responsive. One problem in particular is one that I thought had been resolved some time ago but is apparently biting me right now... With the hdb backend (at least in OL 2.3.34 and OL 2.3.35) if you perform a search with a search base deeper than the root suffix, the search takes a very long time to complete if the cache hasn't been established. In my case the difference is less than a second versus several hours. I'm not sure yet which bit of cache needs to be primed. I can switch back and forth searching with the same filter in the root and then a child search base with the same results. Is this a bug recursion or something that I just hadn't been noticing? What would be the best search to perform to prepare whatever cache is getting hit to make searches outside of the root DN faster? -- Eric Irrgang - UT Austin ITS Unix Systems - (512)475-9342
Re: 2.3 slapd_db_recover
--On May 25, 2007 12:09:58 PM -0700 Craig [EMAIL PROTECTED] wrote: I had been using openldap 2.2.13 installed via yum. I installed 2.3.35 via src. But, I don't see slapd_db_recover (or any of the related apps; slapd_db_checkpoint, slapd_db_dump, etc.) I also did a search thru the 2.3.35 src files and I don't see any special configure/make flags that builds them. Am I missing something? Is there is tools package I need to install? None of those are commands supplied by OpenLDAP. They look like BDB DB commands prefixed with slapd_. Must be something weird your distro does. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: 2.3 slapd_db_recover
--On May 25, 2007 1:53:04 PM -0700 Craig [EMAIL PROTECTED] wrote: Hmm... it seems related: $ slapd_db_recover blah usage: db_recover [-ceVv] [-h home] [-P password] [-t [[CC]YY]MMDDhhmm[.SS]] (I am assuming it is related because the useage line refers to db_recover and not slapd_db_recover.) However, they are not identical: $ ll /usr/bin/db_recover -r-xr-xr-x 1 root root 8704 Feb 21 2005 /usr/bin/db_recover $ ll /usr/sbin/slapd_db_recover -rwxr-xr-x 1 root root 8932 Aug 12 2006 /usr/sbin/slapd_db_recover Obviously, the difference may be superficial. But, I'd still like to know where the slapd_db_* apps come from. BTW, I run CentOS 4.4 and the rpm came from the centos repo: http://isoredirect.centos.org/centos/4.4/os/i386/CentOS/RPMS/openldap-ser vers-2.2.13-6.4E.i386.rpm My guess would be it is the version of db_recover that matches the version of BDB that OpenLDAP was compiled with. For example, you cannot use the BDB 4.3 db_recover command to recover a BDB 4.2 database, etc. So they are simply trying to protect the user by having them use slapd_* named BDB commands so that things are known to match up. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: 2.3 slapd_db_recover
Hmm... it seems related: $ slapd_db_recover blah usage: db_recover [-ceVv] [-h home] [-P password] [-t [[CC]YY]MMDDhhmm[.SS]] (I am assuming it is related because the useage line refers to db_recover and not slapd_db_recover.) However, they are not identical: $ ll /usr/bin/db_recover -r-xr-xr-x 1 root root 8704 Feb 21 2005 /usr/bin/db_recover $ ll /usr/sbin/slapd_db_recover -rwxr-xr-x 1 root root 8932 Aug 12 2006 /usr/sbin/slapd_db_recover Obviously, the difference may be superficial. But, I'd still like to know where the slapd_db_* apps come from. BTW, I run CentOS 4.4 and the rpm came from the centos repo: http://isoredirect.centos.org/centos/4.4/os/i386/CentOS/RPMS/openldap-servers-2.2.13-6.4E.i386.rpm $ rpm -qlp openldap-servers-2.2.13-6.4E.i386.rpm |grep slapd_db /usr/lib/libslapd_db-4.2.so /usr/lib/tls/i486/libslapd_db-4.2.so /usr/lib/tls/i586/libslapd_db-4.2.so /usr/lib/tls/i686/libslapd_db-4.2.so /usr/sbin/slapd_db_archive /usr/sbin/slapd_db_checkpoint /usr/sbin/slapd_db_deadlock /usr/sbin/slapd_db_dump /usr/sbin/slapd_db_load /usr/sbin/slapd_db_printlog /usr/sbin/slapd_db_recover /usr/sbin/slapd_db_stat /usr/sbin/slapd_db_upgrade /usr/sbin/slapd_db_verify Quanah Gibson-Mount wrote: --On May 25, 2007 12:09:58 PM -0700 Craig [EMAIL PROTECTED] wrote: I had been using openldap 2.2.13 installed via yum. I installed 2.3.35 via src. But, I don't see slapd_db_recover (or any of the related apps; slapd_db_checkpoint, slapd_db_dump, etc.) I also did a search thru the 2.3.35 src files and I don't see any special configure/make flags that builds them. Am I missing something? Is there is tools package I need to install? None of those are commands supplied by OpenLDAP. They look like BDB DB commands prefixed with slapd_. Must be something weird your distro does. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: how to create db files
Howard Chu wrote: Quanah Gibson-Mount wrote: --On May 25, 2007 11:32:48 AM -0700 Craig [EMAIL PROTECTED] wrote: Uh... it won't start without the id2entry file: Checking configuration files for : bdb_db_open: db_open(/var/lib/ldap/id2entry.bdb) failed: No such file or directory (2) This is just coming from a slaptest invocation in your init script. Comment out the slaptest, or start slapd by hand. Adding the data via slapadd worked perfectly. (I had only used ldapadd before, which, obviously, requires slapd to be running.)
referential integrity
Hello I have been reading the docs, looking for a way to acheive referential integrity as it is done in RDBMS. For instance, if I want to constraint the value of atribute title to a set of value, so that any modification to a value not in the list would be denied. Let's say the set of legal values could be stored in the directory, ad objects of a dedicated class; How can it be done? slapo-refint is not about denying changes, slapo-constraint can only enforce regex matching.. Is there anything else I missed? -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz [EMAIL PROTECTED]
Re: server startup overhead
I continue to have trouble with getting a freshly started server to be responsive. One problem in particular is one that I thought had been resolved some time ago but is apparently biting me right now... With the hdb backend (at least in OL 2.3.34 and OL 2.3.35) if you perform a search with a search base deeper than the root suffix, the search takes a very long time to complete if the cache hasn't been established. In my case the difference is less than a second versus several hours. I'm not sure yet which bit of cache needs to be primed. I can switch back and forth searching with the same filter in the root and then a child search base with the same results. Have you set DB_CONFIG to reflect reasonable settings given the size of your database? Is this a bug recursion or something that I just hadn't been noticing? What would be the best search to perform to prepare whatever cache is getting hit to make searches outside of the root DN faster?
Re: server startup overhead
Eric Irrgang wrote: I continue to have trouble with getting a freshly started server to be responsive. One problem in particular is one that I thought had been resolved some time ago but is apparently biting me right now... With the hdb backend (at least in OL 2.3.34 and OL 2.3.35) if you perform a search with a search base deeper than the root suffix, the search takes a very long time to complete if the cache hasn't been established. In my case the difference is less than a second versus several hours. I'm not sure yet which bit of cache needs to be primed. I can switch back and forth searching with the same filter in the root and then a child search base with the same results. If it takes several hours, then most likely your BDB cache is too small. As for which cache, it's either the DN cache (aka EntryInfo in the code) or the IDL cache. (Currently the DN cache size is not configurable, will probably add a keyword for that in 2.4.) Is this a bug recursion or something that I just hadn't been noticing? What would be the best search to perform to prepare whatever cache is getting hit to make searches outside of the root DN faster? Priming the caches will only help if you actually have sufficient RAM available. If the DB is too large, then there's not much you can do about it. If you have sufficient RAM, then doing a subtree search from the root, on an unindexed attribute, looking for a value that doesn't exist, will hit every entry in the DB and fully prime the DN cache (and the DN-related info in the IDL cache). It will cycle the full contents of the dn2id and id2entry DBs through the BDB cache as well. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: server startup overhead
Eric Irrgang wrote: Is there a way (with or without attaching a debugger) to find out what my IDL cache and DN cache is doing? Using a debugger, set a breakpoint inside any of the backend functions. When the bdb pointer is set up, print out its data. The bdb-bi_cache structure records all the info about the entry cache. The bdb-bi_idl_* fields records the info about the IDL cache. In 2.4 some of these counters are exposed via back-monitor. We can add more to the monitor entry as needed. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: referential integrity
Emmanuel Dreyfus wrote: Hello I have been reading the docs, looking for a way to acheive referential integrity as it is done in RDBMS. For instance, if I want to constraint the value of atribute title to a set of value, so that any modification to a value not in the list would be denied. Let's say the set of legal values could be stored in the directory, ad objects of a dedicated class; How can it be done? slapo-refint is not about denying changes, slapo-constraint can only enforce regex matching.. Is there anything else I missed? Write a new overlay to do what you want... -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: referential integrity
Howard Chu [EMAIL PROTECTED] wrote: Write a new overlay to do what you want... Right, I just wanted confirmation that 1) nothing exists yet to do it 2) the idea is not heretic I'll have a look at it. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz [EMAIL PROTECTED]
Re: server startup overhead
Well, once an (objectclass=*) search finishes the ou=people,dc=basedn searches run fast again. Unfortunately it takes over half an hour to run the first time and I have to make sure that during that time no one has access to cause extra threads to start searching. On Fri, 25 May 2007, Eric Irrgang wrote: I continue to have trouble with getting a freshly started server to be responsive. One problem in particular is one that I thought had been resolved some time ago but is apparently biting me right now... With the hdb backend (at least in OL 2.3.34 and OL 2.3.35) if you perform a search with a search base deeper than the root suffix, the search takes a very long time to complete if the cache hasn't been established. In my case the difference is less than a second versus several hours. I'm not sure yet which bit of cache needs to be primed. I can switch back and forth searching with the same filter in the root and then a child search base with the same results. Is this a bug recursion or something that I just hadn't been noticing? What would be the best search to perform to prepare whatever cache is getting hit to make searches outside of the root DN faster? -- Eric Irrgang - UT Austin ITS Unix Systems - (512)475-9342
Re: server startup overhead
Searching for objectclass=* and only asking for the entryDN attribute is almost an order of magnitude faster than searching for everything. I like the idea of searching for a non-existent value of an unindexed attribute. It is 40% faster than the objectclass=* search (down to 16 minutes) but does not fix the searches made outside of the directory's root search base. At least not quite. The test search I tried took four minutes (about an order of magnitude better than no priming search) and the following test searches were under a second. Any thoughts on what extra magic might be coming out of the objectclass=* into the caches? My DB cache is just barely sufficient but it seems to be large enough. Examining the db_stat output, iostat, and vmstat, all indications are that by the time I've got right about the time my DB cache approaches full my DB cache hits rapidly approach 100% and I'm neither paging in or out. I can't really go any bigger. cachesize 10 idlcachesize 30 dbconfig set_cachesize 10 0 1 shm_key 7 dbconfig set_shm_key 7 A freshly restarted server quickly ends up with 11 or 12 Gigs resident with a VM size of 13+ gigs and within a week or so is up to 16 Gig VM size and 13 or 14 Gigs resident. On a 16Gig machine, an inch beyond that and the OS starts to run out of room and I risk swap madness. I suppose I ought to track down where the CPU cycles are going, whether it is ACL processing or just the overhead of getting all of the attributes from DB and building LDIF. On Fri, 25 May 2007, Howard Chu wrote: Eric Irrgang wrote: I continue to have trouble with getting a freshly started server to be responsive. One problem in particular is one that I thought had been resolved some time ago but is apparently biting me right now... With the hdb backend (at least in OL 2.3.34 and OL 2.3.35) if you perform a search with a search base deeper than the root suffix, the search takes a very long time to complete if the cache hasn't been established. In my case the difference is less than a second versus several hours. I'm not sure yet which bit of cache needs to be primed. I can switch back and forth searching with the same filter in the root and then a child search base with the same results. If it takes several hours, then most likely your BDB cache is too small. As for which cache, it's either the DN cache (aka EntryInfo in the code) or the IDL cache. (Currently the DN cache size is not configurable, will probably add a keyword for that in 2.4.) Is this a bug recursion or something that I just hadn't been noticing? What would be the best search to perform to prepare whatever cache is getting hit to make searches outside of the root DN faster? Priming the caches will only help if you actually have sufficient RAM available. If the DB is too large, then there's not much you can do about it. If you have sufficient RAM, then doing a subtree search from the root, on an unindexed attribute, looking for a value that doesn't exist, will hit every entry in the DB and fully prime the DN cache (and the DN-related info in the IDL cache). It will cycle the full contents of the dn2id and id2entry DBs through the BDB cache as well. -- Eric Irrgang - UT Austin ITS Unix Systems - (512)475-9342
Re: server startup overhead
Is there a way (with or without attaching a debugger) to find out what my IDL cache and DN cache is doing? -- Eric Irrgang - UT Austin ITS Unix Systems - (512)475-9342
