Re: Rebuild openldap so all modules/overlays are separate
Chris Jacobs wrote: Main question: how to build openldap with all overlays as dynamically loaded modules (ie: optional). Why? These problems have been covered before, recently even. I'm trying to get Solaris 9/10 clients talking to openldap server. I can do getent passwduser but getent passwd or getent group or even ldaplist all fail. I haven't had any success with Benjamin's acl fix/work around, or configuring the overlay for sssvlv in slapd.conf. So, I thought I'd recompile openldap - without sssvlv support, but then I thought why have to do this again in the future if I find another incompatibility? I'd prefer to simply have all overlays/modules as separate loadable modules. When I compiled OpenLDAP, I configured it with: ./configure --with-tls=openssl \ --enable-crypt \ --enable-dynamic \ --enable-ldap \ --enable-lmpasswd \ --enable-modules \ --enable-overlays \ --enable-spasswd \ --sysconfdir=/etc The enable-overlays option builds in a number of overlays I use and am interested in using in the future - however, it also enables sss/vlv. I supposed I could try to build it again with that option followed by --disable-sssvlv but what I'd really prefer is a way to build openldap with the overlays as separate modules that I can load/unload when/as I need them. I've searched the documentation, but I can't figure out how. It's quite likely I haven't looked in the right spot. I'm currently trying to make my way through the configure script... Blech. Thanks for any pointers, configure --help -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP on T2000
Emmanuel Lecharny wrote: On 12/2/10 2:36 PM, Nick Folino wrote: I'm evaluating OpenLDAP on various platforms and am having trouble getting the tools (slapadd, slapindex, etc) to perform well on Sun's T2000 servers. For example slapadd is running at about 50k/s. On an x86 server importing the same ldif yields over 1M/s. All my experience with OpenLDAP so far shows x86 servers far outperforming Sun T2000s, given comparable memory and disk. The individual Sun cores are just too slow. 1 *million* addition per second ? Either you have an issue with the unit you use, or there is something wrong going on but you didn't noticed the problem... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapadd: corrupted double-linked list
Marc Patermann wrote: Hi, Marc Patermann schrieb am 06.12.2010 16:40 Uhr: what could the following possibly be? This looks like something valgrind ought to be able to diagnose. When I do a # slapadd -q -v -c -l dump.ldif to a SLES 11 SP1 (openldap 2.3.20) I get this after a few 1000 entries: I have a fresh install of SLES 11 SP1 on a new machine. I installed 2.4.23. When I did the config and tried to put initial data (8(!) objects) into the server, I got glibc errors too. # slapadd -q -c -v -l init.ldif added: ... ... _ 100.00% eta none elapsednone fast! Closing DB... *** glibc detected *** slapadd: free(): invalid pointer: 0x7f89c0273210 *** === Backtrace: = /lib64/libc.so.6(+0x75018)[0x7f89bd242018] /lib64/libc.so.6(cfree+0x6c)[0x7f89bd246f6c] /usr/lib64/libdb-4.5.so(__ham_db_close+0x19)[0x7f89bf0d9139] /usr/lib64/libdb-4.5.so(__db_refresh+0x340)[0x7f89bf120a20] /usr/lib64/libdb-4.5.so(__db_close+0x51)[0x7f89bf120f91] /usr/lib64/libdb-4.5.so(__db_close_pp+0xd5)[0x7f89bf135445] slapadd(+0xc15eb)[0x7f89bfd065eb] ... As I found out, when I deactivate the four index lines, it works. Each of these four lines causes an glibc error. #index objectclass,uid,maileq #index sn,cn,givenName sub,eq #index dhcpHWAddress,dhcpClassData,dhcpOption eq #index entryCSN,entryUUID,zoneName eq Hm? :( Marc -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: x500UniqueIdentifier
Juan Gonzalez wrote: Hi, I’m trying to insert userCertificate values containing x500UniqueIdentifiers. When the value appears at the SubjectNames, it inserts correctly. By this I assume there is a validation for the field formatting. When I have a x500UniqueIdentifier at the IssuerNames it fails to insert. Is there a specific place where valid attributes and syntaxes for IssuerNames should be declared? Certificate name validation just uses whatever schema is already loaded into slapd, and x500UniqueIdentifier is in the core schema file so it should already be present in your configuration. However, subjectNames aren't fully parsed during certificate validation, while issuerNames are. So e.g. a syntax error in subjectName will not be detected at insert time. Can you post an example certificate? I have a feeling that our DN validator here may not be handling the syntax for x500UniqueIdentifier but would like to double check. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: x500UniqueIdentifier
Howard Chu wrote: Juan Gonzalez wrote: Hi, I’m trying to insert userCertificate values containing x500UniqueIdentifiers. When the value appears at the SubjectNames, it inserts correctly. By this I assume there is a validation for the field formatting. When I have a x500UniqueIdentifier at the IssuerNames it fails to insert. Is there a specific place where valid attributes and syntaxes for IssuerNames should be declared? Certificate name validation just uses whatever schema is already loaded into slapd, and x500UniqueIdentifier is in the core schema file so it should already be present in your configuration. However, subjectNames aren't fully parsed during certificate validation, while issuerNames are. So e.g. a syntax error in subjectName will not be detected at insert time. Can you post an example certificate? I have a feeling that our DN validator here may not be handling the syntax for x500UniqueIdentifier but would like to double check. I should also warn you, x500UniqueIdentifier has a bitstring syntax, and this syntax is extremely clumsy in LDAP. Judging from the certificate you sent me, you're trying to use this attribute as a regular octetstring. The two are quite different. E.g. if you have a value in the certificate x500UniqueIdentifier=ABC in LDAP this is represented as x500UniqueIdentifier='010101100111'B I would strongly encourage you to use some other attribute other than x500UniqueIdentifier, and to avoid any attributes that use bitstring syntax as this one does. The bitstring syntax is intended for data items that are actually an arbitrary string of bits, in particular, a string that may not be aligned on octet boundaries. If the only data you're storing is actually sequences of octets, this is a very inefficient choice of syntax. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: openldap and kerberos integration
Thierry Lacoste wrote: Hello, I'm experimenting with integrating Kerberos and OpenLDAP following roughly http://wiki.mandriva.com/en/Projects/OpenLDAP_DIT I'm using CentOS and Buchan Milne's repository (http://staff.telkomsa.net/packages/rhel5/ ) both for OpenLDAP and Heimdal. I've almost succeeded except for password integration. It seems that the smbk5pwd module provided by openldap2.4- servers-2.4.22-1.el5 in /usr/lib/openldap2.4/smbpwd.so is built without kerberos support. With smbk5pwd-enable krb5 I have the following error: /etc/openldap2.4/slapd.conf: line 154: smbk5pwd:smbk5pwd-enable module smbk5pwd-enable only allowed when compiled with -DDO_KRB5. What is the easiest option to get a kerberos supporting smbk5pwd? I have no comment on other people's builds. BTW I'd appreciate any recommandations about providing kerberos and LDAP authentication (with the same password) in a production setting. Should I use Heimdal or MIT kerberos ? If Heimdal, is it better to use OpenLDAP as a backend for Kerberos or let Kerberos use its native backend? If OpenLDAP as a backend, is it better to use {K5KEY} as the userPassword or let smbk5pwd synchronize everything? Read the smbk5pwd README. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: openldap and kerberos integration
Thierry Lacoste wrote: On 9 déc. 10, at 22:03, Howard Chu wrote: Thierry Lacoste wrote: Hello, I'm experimenting with integrating Kerberos and OpenLDAP following roughly http://wiki.mandriva.com/en/Projects/OpenLDAP_DIT I'm using CentOS and Buchan Milne's repository (http://staff.telkomsa.net/packages/rhel5/ ) both for OpenLDAP and Heimdal. I've almost succeeded except for password integration. It seems that the smbk5pwd module provided by openldap2.4- servers-2.4.22-1.el5 in /usr/lib/openldap2.4/smbpwd.so is built without kerberos support. With smbk5pwd-enable krb5 I have the following error: /etc/openldap2.4/slapd.conf: line 154: smbk5pwd:smbk5pwd-enable module smbk5pwd-enable only allowed when compiled with -DDO_KRB5. What is the easiest option to get a kerberos supporting smbk5pwd? I have no comment on other people's builds. Sorry for the noise. I found it here http://staff.telkomsa.net/packages/rhel5/openldap2.4-smbk5pwd-2.4.21-4.el5.i386.rpm BTW I'd appreciate any recommandations about providing kerberos and LDAP authentication (with the same password) in a production setting. Should I use Heimdal or MIT kerberos ? If Heimdal, is it better to use OpenLDAP as a backend for Kerberos or let Kerberos use its native backend? If OpenLDAP as a backend, is it better to use {K5KEY} as the userPassword or let smbk5pwd synchronize everything? Read the smbk5pwd README. I'v read it. Your answer seems to imply that I should use Heimdal and then OpenLDAP as it's backend. Am I right? It's more than just implied. The README says the code was written for Heimdal. If you want to use smbk5pwd at all, then you must use Heimdal. Do you recommend using {K5KEY} as the userPassword? If you want LDAP Simple Binds to use the same password as Kerberos, then yes. If not, then no. There are security implications to using it; without strong protections on the LDAP sessions you're exposing the Kerberos password on the network and it ordinarily never should be exposed. (Also, just because you think you have strong protections in place, it's still less secure than keeping it completely off the wire. Particularly when somebody screws up (e.g. Debian) and their TLS software is doing something stupid like generating predictable sequences of random keys, etc.) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: openldap and kerberos integration
Thierry Lacoste wrote: On 10 déc. 10, at 20:57, Howard Chu wrote: Thierry Lacoste wrote: BTW I'd appreciate any recommandations about providing kerberos and LDAP authentication (with the same password) in a production setting. Should I use Heimdal or MIT kerberos ? If Heimdal, is it better to use OpenLDAP as a backend for Kerberos or let Kerberos use its native backend? If OpenLDAP as a backend, is it better to use {K5KEY} as the userPassword or let smbk5pwd synchronize everything? Read the smbk5pwd README. I'v read it. Your answer seems to imply that I should use Heimdal and then OpenLDAP as it's backend. Am I right? It's more than just implied. The README says the code was written for Heimdal. If you want to use smbk5pwd at all, then you must use Heimdal. Sorry my question was not very clear. I wan't LDAP Simple Binds and Kerberos with the same password. I find smbk5pwd and OpenLDAP as a Heimdal backend very appealing but maybe there are good reasons to use another Kerberos implementation and/or store passwords in the Kerberos native backend (adding e.g. SASL in the mix to make LDAP Simple Binds use pass-through authentication), obviously ruling out smbk5pwd. If all of your clients can use SASL Binds, then that is the best choice, and you can ignore smbk5pwd. Unfortunately I have no control over the clients so I need something transparent for them. What I had in mind was setting {sasl}lo...@my.realm in the userPassword. The realm would be on a KDC hosted on the same box as the LDAP server to minimize network traffic. It seems much less elegant than the Heimdal/OpenLDAP integration with smbk5pwd but (maybe) has some benefits: I can use another KDC (e.g. MIT) and use the native kerberos backend. My first tests are OK but as of now I have no idea about replication. Any recommandation for the best way to have LDAP Simple Binds and Kerberos with the same password are much appreciated. Do you recommend using {K5KEY} as the userPassword? If you want LDAP Simple Binds to use the same password as Kerberos, then yes. If not, then no. AFAICS with smbk5pwd I have two ways to have LDAP Simple Binds and Kerberos with the same password. 1) force use of ldappasswd to make smbk5pwd synchronize all passwords; 2) assign {K5KEY} to the userPassword and use kpasswd to change a password. If I understood correctly, the second method makes the passwords identical by construction while the first allow passwords to desynchronize if changed without ldappasswd. The point of smbk5pwd is to fully synchronize, that means it works in both directions. When configured correctly and {K5KEY} is used, it doesn't matter whether kpasswd or ldappasswd is used, everything stays in sync. I noticed some differences. In particular ldappasswd updates sambaLMPassword while kpasswd does not. I suppose we can delete sambaLMPassword support by now, certainly no one should be using it any more. With ldappasswd, here is an excerpt of my auditlog: replace: userPassword replace: sambaPwdLastSet replace: sambaLMPassword replace: sambaNTPassword replace: krb5KeyVersionNumber replace: krb5Key With kpasswd, I have : replace: krb5KeyVersionNumber replace: krb5PasswordEnd replace: sambaPwdMustChange delete: krb5Key add: krb5Key replace: sambaNTPassword There is no either-or as you outline with your two choices above. You must use {K5KEY} if you want any of the synchronization to work. Here I'm confused. I tested smbk5pwd with SSHA password hashes. ldappasswd correctly updates passwords and kerberos keys such that LDAP simple binds and kerberos authentication both work with the same password. In that case there's both an SSHA hashed copy and a KDC hashed copy of the password. With {K5KEY} there is only the KDC hash. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: openldap and kerberos integration
Hugo Monteiro wrote: On 12/15/2010 07:19 PM, Howard Chu wrote: Thierry Lacoste wrote: I noticed some differences. In particular ldappasswd updates sambaLMPassword while kpasswd does not. I suppose we can delete sambaLMPassword support by now, certainly no one should be using it any more. I'm sorry but did i understand correctly that sambaLMPassword will no longer be updated while using the smbk5pwd overlay? Also, i would like to know why do you state that no one should be using it any more. Besides Samba itself, it can (as is) used by freeradius while using PEAP and MsCHAPv2 for wireless clients authentication. That's interesting, especially since the KDC itself doesn't maintain sambaLMPassword. The LANMAN hash mechanism has been obsolete for years, it is intrinsically weak and is not a good security mechanism. I think you're mistaken, anyway; according to RFC2759 which defines MSCHAPv2, it uses an NT hash, not a LANMAN hash. The LANMAN hash was used for MSCHAPv1 which is also obsolete. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Filesystem backend options for embedded openldap
Bruce Edge wrote: On Sat, Dec 18, 2010 at 12:26 PM, Peter Lambrechtsen plambrecht...@gmail.com wrote: Or perhaps TinyLdap? http://www.fefe.de/tinyldap/ Also FreeRadius (if your app's support Radius and LDAP) supports a myriad of backend databases. Hmm, very interesting. I was not aware of this project. My concern is that there's been very little activity on the project in recent years. Is there no simple, reliable, backend config for openldap? I'm not concerned with speed, just reliability and data integrity. I'd settle for a 10x performance penalty for data integrity. You could give back-ldif a try. It certainly will not perform well, but it's so simple that data corruption wouldn't be an issue. -Bruce On Sun, Dec 19, 2010 at 5:59 AM, Bruce Edgebruce.e...@gmail.com wrote: On Sat, Dec 18, 2010 at 12:19 AM, Peter Lambrechtsen plambrecht...@gmail.com wrote: Why don't you use SQLite instead??? It's pretty rock solid backend database. Unless your client side only wants to talk LDAP. Hi, Thanks the the response. One of the reasons for ldap is that it handles all the authentication for a lot of the packages we're using out of the box so it was a natural progression to extend it to handle the other data we need as well. The only probems, it appears, is how to make it more failsafe on the back end. -Bruce On Sat, Dec 18, 2010 at 6:48 AM, Bruce Edgebruce.e...@gmail.com wrote: Perhaps a bit more detail... During testing our developers frequently hang the target machines. This usually results in a corrupted ldap database even though no write activity was present on the box since long before the crash. What ldap config tuning options are required to get slapd to sync the backend to a state where power loss / kernel crashes do not corrupt the data? Thanks -Bruce On Wed, Dec 15, 2010 at 10:25 AM, Bruce Edgebruce.e...@gmail.com wrote: I'm working on an embedded system for which I would like to use openldap as the means of config storage. I've spent a lot of time RTFM'ing and still feel that there is a lot that is escaping me as far as the optimal configuration. If the primary goal is data safety and zero human intervention, what would be the optimal combination of file system / backend storage / and config options? I would like to never have to manually recover a database and have it gracefully recover from power failures. Speed is not an issue as it's very low traffic. Integritiy is everything. It's target storage is a USB flash device. Are there any special considerations WRT flash storage and ldap? Thanks in advance. -Bruce -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Granting write to ou
Dieter Kluenter wrote: Am Wed, 22 Dec 2010 14:46:50 +0100 schrieb Thomas D. Dahlmanndomi...@domingo.dk: Hi I'm trying to add ordinary users write access to a specific ou. I've googled a lot and haven't really found any useful regarding to openldap 2.4 (slapd.d format). What would be the correct syntax for a ldapmodify command to accomplish this to the dn: ou=addressbook,dc=example,dc=net ? something like: ldapmodify -D cn=config -W -H ldap://some.host dn:olcDatabase={1}hdb,cn=config changetype: modify replace: olcAccess olcAccess: {1} to dn.subtree=ou=addressbook,dc=example,dc=net by users write by * read - The numbers {1} are fictious, replace with your real values. Do not supply the {x} numbers at all when you're just doing a replace. slapd generates them itself, so there's no need. You only need to provide the {x} numbers when you actually want to reference a specific value in a multivalued attribute. Even then, they're just a convenience, not absolutely essential. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Kerberos/GSSAPI issues
Brian Candler wrote: Hello, I'm setting up an openldap server for Kerberos (GSSAPI) authentication only. I'm using slapd-2.4.21 from Ubuntu 10.04.1. It's basically working, and I had to do very little other than change export KRB5_KTNAME in /etc/default/slapd to point to the service keytab. However, there are a couple of strange things which I wonder if someone could help me with. (1) According to the documentation at http://www.openldap.org/doc/admin24/sasl.html#GSSAPI then the authentication DN should be uid=primary[/instance],cn=realm,cn=gssapi,cn=auth However, running slapd in debug mode I see the cn=realm is missing. That's normal. The SASL library doesn't provide the realm name when it is equal to the default realm. This has been true of Cyrus SASL for probably the past dozen years. Read the Cyrus SASL documentation. (2) I would like to be able to do ldapsearch without specifying -Y GSSAPI explicitly. However if I omit it, the client picks DIGEST-MD5 instead (which isn't much use, since I have no passwords in the database) Configure a sasl/slapd.conf with the options you want. Read the Cyrus SASL documentation. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Kerberos/GSSAPI issues
Dieter Kluenter wrote: Am Wed, 29 Dec 2010 16:50:17 + schrieb Brian Candlerb.cand...@pobox.com: On Wed, Dec 29, 2010 at 07:57:43AM +0100, Dieter Kluenter wrote: The default ssf of ldapi is 71, but you may change localSSF in slapd.conf(5). [...] Thank you, that is very clear. Having changed that, I can use EXTERNAL with minssf=112, but not GSSAPI. I find that if I set minssf=56 it's fine, but at minssf=57 it isn't. It looks like this is a fundamental limitation of the GSSAPI: http://lists.andrew.cmu.edu/pipermail/cyrus-sasl/2006-September/000628.html http://lists.andrew.cmu.edu/pipermail/cyrus-sasl/2006-September/000635.html FYI, here's what I see with minssf=57 (the 'No such attribute' error is somewhat confusing) r...@noc:~# ldapsearch ldap_sasl_interactive_bind_s: No such attribute (16) r...@noc:~# ldapsearch -Y GSSAPI SASL/GSSAPI authentication started ldap_sasl_interactive_bind_s: Inappropriate authentication (48) additional info: SASL(-15): mechanism too weak for this user: mech GSSAPI is too weak That is because Kerberos DES, und thus GSSAPI, only has a security strength factor of 56. But this value is pure fiction, it's an arbitrary value hardcoded into the SASL gssapi plugin. Generally Kerberos is using triple-DES today, or AES. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP 2.4.24?
Michael Ströder wrote: Marco Pizzoli wrote: I would like to know if is there a plan to make a new release of OL in the following months. http://www.openldap.org/lists/openldap-devel/201012/msg1.html There were some commits to CVS branch OPENLDAP_REL_ENG_2_4 since then. So help testing it: Not yet. Wait for the release engineer to announce a call for testing. The tree is still in flux. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: viewing cn=config
E.S. Rosenberg wrote: Hi, sorry that it took me a while to reply. So I do have the config backend, since debian moved me over automatically. So far the slapcat method has been working very nicely for me. Another question I ran into is how do you edit cn=config, by default I have not been able to figure out. I have so far seen two methods that work for me but they both feel a bit 'hacky' so I was wondering if there is another 'official' method. The two things I have done: 1. edit the files in slapd.d (after stopping the openldap server to be on the safe side). Never do that. 2. using ldapmodify, but this was only possible after I added a olcRootPW attribute to the olcDatabase object of the config backend by editing the file as described here: http://www.zarafa.com/wiki/index.php/OpenLdap:_Switch_to_dynamic_config_backend_%28cn%3Dconfig%29#Add_or_Change_password_of_RootDN It confused me at first since the configdb entry has the following acl: olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break I didn't understand how I could get access, it looked like I had to be root but even as root ldapmodify would refuse duty. I guess after seeing these mails I should have been using something like -Y EXTERNAL? Yes, from your description, you should have used ldapmodify -Y EXTERNAL -H ldapi:/// while running as root. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ACLs to allow users to traverse DIT down to their own entry, lock down everything else
Andreas Ntaflos wrote: So what do I need to do so users can specify dc=example,dc=com as search base and traverse the directory tree down to their own DN entry? Read the Operation Requirements section of slapd.access(5). -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Index for objectclass does not work...
Steeg Carson wrote: 2011/1/4 Quanah Gibson-Mountqua...@zimbra.com: --On Tuesday, January 04, 2011 1:43 AM +0100 Steeg Carson steeg.car...@googlemail.com wrote: I simulate this on my database just right now: I suggest you read: http://www.openldap.org/lists/openldap-technical/201011/msg00146.html to understand how indices and their slots work. As I now understand, the entire index for one attribute (e.g. objectClass) is split in several indexes. They holds for each path/node (resp. DN, but not leaf) an separate index for this attribute with all hits for his subtree (and for onelevel too). No. Only the dn2id table maintains any notion of nodes and subtrees. All indices are global to the database and have no notion of scope. If I do an ldapsearch with -b cn=ownPath,ou=root the slapd takes the index which is bound on this node/DN? In my DIT are 470812 entires. The objectClass=subEngine exists 104384 times in the entire directory (ou=root). The objectClass=subEngine exists only 1 time under cn=ownPath,ou=root, By default an index slot can only maintain 65535 records before it overflows and loses precision. Once it loses precision, you tend to get results like this. If you need to accomodate larger indices you can tweak a constant in back-bdb/back-bdb.h and recompile. You'll probably also need to increase LDAP_PVT_THREAD_STACK_SIZE. Another workaround, without recompiling, would be to sort your entries such that all of the entries of the subEngine class are loaded in contiguous order. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Index for objectclass does not work...
Steeg Carson wrote: 2011/1/6 Howard Chuh...@symas.com: Steeg Carson wrote: 2011/1/4 Quanah Gibson-Mountqua...@zimbra.com: --On Tuesday, January 04, 2011 1:43 AM +0100 Steeg Carson steeg.car...@googlemail.comwrote: I simulate this on my database just right now: I suggest you read: http://www.openldap.org/lists/openldap-technical/201011/msg00146.html to understand how indices and their slots work. As I now understand, the entire index for one attribute (e.g. objectClass) is split in several indexes. They holds for each path/node (resp. DN, but not leaf) an separate index for this attribute with all hits for his subtree (and for onelevel too). No. Only the dn2id table maintains any notion of nodes and subtrees. All indices are global to the database and have no notion of scope. But what does mean (from http://www.openldap.org/lists/openldap-technical/201011/msg00146.html): Ordinarily at each level of the tree we keep an index tallying all of the children beneath that point. In back-bdb this index is used for subtree searches and for onelevel searches. So if I do a search, I'll get every time ALL results (ID's) from the index for the searched value. If my search uses additionally a searchbase the slapd takes all ID's and lookup in id2entry.bdb to get the DN for the ID and compare? The search scope provides a set of candidates, consisting of every entry within that scope of the tree. An index lookup provides a set of candidates, consisting of every entry in the DB that matches the index. The intersection of these sets forms the set of candidate entries that must be examined in depth. Can you recommend a good book, where I can read all such things and understand, how openldap really works? This are all very important things for design and operation. There is no book on the internals of back-bdb or hdb. The source code is there for anyone to read. If you want to learn more, there are also extensive discussions on the design approaches and tradeoffs in the archives of the openldap-devel mailing list. For the most part, these discussions have only ever been of interest or relevance to other OpenLDAP developers. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: users, groups, etc. for posix authentication?
Christ Schlacta wrote: is there any reason that a posix usernames, groups, passwords, etc. must be stored in distinct locations in a directory ? I realize this mostly applies to the padl pam/nis and the libnsspam-ldapd module specific. can they be stored in other structures effectively and usefully? can they be stored on a department by department basis, or in any other organizational scheme? (ou=arbitrary1,dc=... having groups and users, while ou=arb2,ou=arb3,dc=... also has users and groups?) if a scheme like the above is used, will all users and groups be available on a system? must they be free of naming conflicts, or will group=users,ou=arbitrary1,... be different from group=users,ou=arb2,ou=arb3,... ? if they're different, how would this be indicated by the systems? A POSIX system considers usernames to be a flat namespace. If you store them in separate branches of a directory, you create the possibility of having duplicate names in separate branches, and the base OS will not be able to handle that. This question has nothing to do with LDAP and has no place on this forum. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Index for objectclass does not work...
Steeg Carson wrote: Hello Howard, By default an index slot can only maintain 65535 records before it overflows and loses precision. Once it loses precision, you tend to get results like this. If you need to accomodate larger indices you can tweak a constant in back-bdb/back-bdb.h and recompile. You'll probably also need to increase LDAP_PVT_THREAD_STACK_SIZE. can you please guide to do this? Which constant must be changed, which values are suitable for this constant and for LDAP_PVT_THREAD_STACK_SIZE? If I change this values, which other impact does it have of used memory and the slapd? The constant is BDB_IDL_LOGN in back-bdb/idl.h. Incrementing it by 1 will double the range of an index slot before it loses precision. It will also double the amount of memory used by all of the indexing functions. I think you can safely double the current value without overrunning the default thread stack size. But if you go even higher you'll probably need to increase it. The current value for LDAP_PVT_THREAD_STACK_SIZE is (1 * 1024 * 1024 * sizeof(void *)) (4MB on a 32 bit machine, 8MB on 64 bit machine). If you need to raise it I would suggest adding e.g. XDEF=-DLDAP_PVT_THREAD_STACK_SIZE=16777216 to your make invocation and recompiling libldap_r with this new value. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Handling slapd.d in OpenLDAP and Kerberos
Quanah Gibson-Mount wrote: --On Friday, January 07, 2011 9:22 PM +0530 sarathkrishn...@gmail.com wrote: We think adding this in slapd.conf should serve the purpose. Any suggestions/help is highly appreciated. Use slaptest to convert your slapd.conf file to the slapd-config format, and read what it does for the authz regexp. Then you'll clearly see how to handle what you did in slapd.conf in the new format. Or just read the slapd-config(5) manpage and the Admin Guide... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX
Chris Jacobs wrote: That appears to be the point. See: http://www.openldap.org/software/man.cgi?query=ppolicyapropos=0sektion=0manpath=OpenLDAP+2.3-Releaseformat=html ... No results. Sounds like the search index is out of date. Still, all you have to do is go here http://www.openldap.org/software/man.cgi?query=(5)sektion=apropos=1manpath=OpenLDAP+2.3-Releasetitle=Section and the manpage is there: http://www.openldap.org/software/man.cgi?query=slapo-ppolicysektion=5apropos=0manpath=OpenLDAP+2.3-Release But better yet, just type man 5 slapo-ppolicy on your machine. Why people waste time searching the web when everything is on their local machine still boggles my mind. Also look for the ppolicy in: http://www.openldap.org/doc/admin23/schema.html#Distributed%20Schema%20Files ... It's not there. The Admin Guide was never intended to be an exhaustive reference - it is after all only a guide. Every software component is documented in manpages. The manpages should always be the first place you look, not the Guide, and not the web. Where did you get the schema and the libraries necessary? FWIW: the password policy and MUCH more reliable syncing between servers is why we upgraded in my shop (turned off the old 2.3 master last week after finally overcoming last hurdles: solaris and use by other custom systems.) - chris Chris Jacobs, Systems Administrator Apollo Group | Apollo Marketing | Aptimus 2001 6th Ave Ste 3200 | Seattle, WA 98121 phone: 206.839-8245 | cell: 206.601.3256 | Fax: 208.441.9661 email: chris.jac...@apollogrp.edu - Original Message - From: openldap-technical-boun...@openldap.orgopenldap-technical-boun...@openldap.org To: Quanah Gibson-Mountqua...@zimbra.com Cc: openldap-technical@openldap.orgopenldap-technical@openldap.org Sent: Wed Jan 12 23:38:54 2011 Subject: Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX 13.01.2011 11:55, Quanah Gibson-Mount пишет: --On January 13, 2011 11:42:29 AM +0600 Konstantin Boyandin temmo...@gmail.com wrote: Hello, OpenLDAP version: 2.3.43-12 (CentOS 5.5), 64-bit. In order to enable ppolicy overlay, I am trying to create the relevant entries, as specified in http://www.openldap.org/doc/admin24/overlays.html#Password%20Policies I would suggest you compare the version you are running (2.3) with the version that the document you are reading uses (2.4). There is an obvious difference there, and it is a major one. I suggest you run a current supported release of OpenLDAP that matches the documentation you are using. Thanks. I opened the 2.3 admin link instead: http://www.openldap.org/doc/admin23/ and it has no overlays section at all. That's weird, since I am using replication feature and there's a directive overlay syncprov in /etc/openldap/slapd.conf How can I find the reasons for 'Invalid syntax' error in such a situation? Thanks. This message is private and confidential. If you have received it in error, please notify the sender and remove it from your system. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LDAP and PAM: account is expired, but pam_ldap allows authentification
Konstantin Boyandin wrote: Hello, Could someone direct me to the source of wisdom to solve this: I have set correctly the fields (attributes) shadowExpire shadowLastChange shadowMin shadowMax to make the account expired (OpenLDAP used to run NT domain), but when I ssh to a server using pam_ldap authentication, it is still allowed to login. How pam_ldap should be instructed to take the expiration attributes ito account? Ask on a pam_ldap mailing list. pam_ldap is not a piece of OpenLDAP software, your question is off topic here. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX
Konstantin Boyandin wrote: Hello, OpenLDAP version: 2.3.43-12 (CentOS 5.5), 64-bit. In order to enable ppolicy overlay, I am trying to create the relevant entries, as specified in http://www.openldap.org/doc/admin24/overlays.html#Password%20Policies I import two LDIFs, first: dn: ou=Policies,dc=example,dc=com objectClass: organizationalUnit objectClass: top ou: Policies and second dn: cn=default,ou=Policies,dc=example,dc=com cn: default objectClass: top objectClass: pwdPolicy objectClass: person pwdAllowUserChange: TRUE pwdAttribute: userPassword pwdCheckQuality: 2 pwdExpireWarning: 600 pwdFailureCountInterval: 30 pwdGraceAuthNLimit: 2 pwdInHistory: 5 pwdLockout: TRUE pwdLockoutDuration: 0 pwdMaxAge: 7776000 pwdMaxFailure: 5 pwdMinAge: 0 pwdMinLength: 5 pwdMustChange: FALSE pwdSafeModify: FALSE sn: dummy value The first loads OK. When I try to import the second, I receive this diagnostics: Could not add object cn=default,ou=Policies,dc=itelsib,dc=com Message: Invalid syntax Error code: 0x15 (LDAP_INVALID_SYNTAX) Error description: An invalid attribute value was specified. Could someone suggest what's wrong with the attribute name? OpenLDAP never produces the text you provided above. It seems you're using some other LDAP tool to do this import, and it is not showing you the actual error message sent from the server. OpenLDAP slapd will always identify the actual attribute and value that causes an error. I suggest you try importing this entry with OpenLDAP's ldapadd and examine the error message from there. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LDAP and PAM: account is expired, but pam_ldap allows authentification
Indexer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/2011, at 17:45, Konstantin Boyandin wrote: Hello, Could someone direct me to the source of wisdom to solve this: I have set correctly the fields (attributes) shadowExpire shadowLastChange shadowMin shadowMax to make the account expired (OpenLDAP used to run NT domain), but when I ssh to a server using pam_ldap authentication, it is still allowed to login. How pam_ldap should be instructed to take the expiration attributes ito account? Isnt this handled via nsswitch? Can you show us your /etc/nsswitch.conf, and your /etc/ldap.conf (not your /etc/openldap/ldap.conf As a reminder - the OpenLDAP-technical list is for the discussion of actual OpenLDAP software, as well as how to make other software interoperate with it. Questions that are purely about how to use 3rd party software foo work at all do not belong on this list. There is no evidence that the original poster is having any trouble using OpenLDAP. His question is entirely about making 3rd party software work, and those questions belong on the support forums for those 3rd party software packages. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LDAP and PAM: account is expired, but pam_ldap allows authentification
Indexer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 shadowExpire shadowLastChange shadowMin shadowMax to make the account expired (OpenLDAP used to run NT domain), but when I ssh to a server using pam_ldap authentication, it is still allowed to login. This look to be a question where the user does not know what is responsible for the issue he is seeing, but does relate to his attempt to use OpenLDAP. He is correct in asking here, and helpfully pointing him in the correct direction is the right course of action, rather than saying you are wrong to ask this here. This problem may have been to him related to missing elements from his user objects (which would have been openldap) or it was anything else. Pointing him to pam_ldap was the correct action. Also you said As a reminder - the OpenLDAP-technical list is for the discussion of actual OpenLDAP software, as well as how to make other software interoperate with it. Questions that are purely about how to use 3rd party software foo work at all do not belong on this list. This counts as other software interoperate with it. from where I am sitting. I have seen many questions like this, and I think it should be something we answer and point people in the correct direction of rather than saying you'll get no help here So instead of going to a doctor to be referred to a specialist, you will go straight to a specialist without knowing what your problem is? makes complete sense. It was obvious that he was not asking why doesn't my pam_ldap talk to my OpenLDAP server. Missing elements from the user objects is a *data* problem, it is not an interoperability problem. He would have the same issue whether the server was OpenLDAP, Oracle, or M$AD. It has nothing to do with OpenLDAP, and a careful reader would have known all of this. If you're not reading carefully, you should not be responding to the posts. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX
Konstantin Boyandin wrote: 13.01.2011 13:39, Howard Chu writes: Konstantin Boyandin wrote: Hello, OpenLDAP version: 2.3.43-12 (CentOS 5.5), 64-bit. In order to enable ppolicy overlay, I am trying to create the relevant entries, as specified in http://www.openldap.org/doc/admin24/overlays.html#Password%20Policies I import two LDIFs, first: dn: ou=Policies,dc=example,dc=com objectClass: organizationalUnit objectClass: top ou: Policies and second dn: cn=default,ou=Policies,dc=example,dc=com cn: default objectClass: top objectClass: pwdPolicy objectClass: person pwdAllowUserChange: TRUE pwdAttribute: userPassword pwdCheckQuality: 2 pwdExpireWarning: 600 pwdFailureCountInterval: 30 pwdGraceAuthNLimit: 2 pwdInHistory: 5 pwdLockout: TRUE pwdLockoutDuration: 0 pwdMaxAge: 7776000 pwdMaxFailure: 5 pwdMinAge: 0 pwdMinLength: 5 pwdMustChange: FALSE pwdSafeModify: FALSE sn: dummy value The first loads OK. When I try to import the second, I receive this diagnostics: Could not add object cn=default,ou=Policies,dc=itelsib,dc=com Message: Invalid syntax Error code: 0x15 (LDAP_INVALID_SYNTAX) Error description: An invalid attribute value was specified. Could someone suggest what's wrong with the attribute name? OpenLDAP never produces the text you provided above. It seems you're using some other LDAP tool to do this import, and it is not showing you the actual error message sent from the server. OpenLDAP slapd will always identify the actual attribute and value that causes an error. I suggest you try importing this entry with OpenLDAP's ldapadd and examine the error message from there. I tried importing with slapadd. The output: str2entry: invalid value for attributeType pwdAttribute #0 (syntax 1.3.6.1.4.1.1466.115.121.1.38) slapadd: could not parse entry (line=22) The error above refers to the allowed value of pwdAttribute, which can only be userPassword now. The problem is the value for this attribute in LDIF *is* userPassword, as in the cited sample. I checked the LDIF - no 'invisible' characters around the value. Sounds like you don't actually have the ppolicy overlay configured on the database you're loading into. The pwdAttribute syntax handler is part of the ppolicy overlay and will only get installed if you configure the overlay on the target database. JFYI, I checked the values for the attributes using man page. This, and other references provided with packages is where I look first prior to asking on the Net. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX
Pierangelo Masarati wrote: Chris Jacobs wrote: Perhaps try man slapo_ppolicy The man page name is slapo-ppolicy(5). - it should hopefully provide the limits and acceptable values and compare with your ldif to find the cause of Error description: An invalid attribute value was specified. Alternative: reduce the number of attributes (divide and conquer) to find the culprit. Perhaps also checking the schema file for the limits or acceptable values. Or check the archives, e.g. http://www.openldap.org/lists/openldap-software/200802/msg00337.html: for some time, in OpenLDAP 2.3, the pwdAttribute could only contain OIDs. That issue was fixed long before 2.3.43, which he says he is running. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX
Pierangelo Masarati wrote: Konstantin Boyandin wrote: Or check the archives, e.g. http://www.openldap.org/lists/openldap-software/200802/msg00337.html: for some time, in OpenLDAP 2.3, the pwdAttribute could only contain OIDs. Thank you very much! After I changed the string to pwdAttribute: 2.5.4.35 the import was a success. This problem's solved. So reading Web *can* be of more use than reading man pages only. The archives of the OpenLDAP project are indeed part of the web. In order to get valuable information you need to be able to dig it out from tons of s**t, though. Indeed. In fact Dieter's answer was already 3 years out of date when he posted it. The issue in question is ITS#4025 which was fixed in September 2005 and released in OpenLDAP 2.3.8. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Best and most efficient way to upgrade openLDAP
Ghyslain Tissot wrote: Hello, We are looking for input on how to upgrade openLDAP from 2.2 to 2.3. We have looked and it seems the only documented way to upgrade is by unloading and reloading the data which is very time consuming (approx. 8-10 hours). Since this is a production system it is not adequate. What options are available? You should realize that you're moving from one unsupported release to another unsupported release, both have been obsoleted by OpenLDAP 2.4 for a number of years already. To give more more background: - there are currently approximately 7M entries in the LDAP; - we are migrating to a new server with a new version of openLDAP (dependant on RHEL version - previously RHEL 4 now RHEL 5); - we tried the option creating a staging server with 2.2 and importing on the new system (v2.3) the extracted data on the new system. Then we activated replication (SLURPD) between the two and tried updating the staging server with the transaction logs of production hoping the data would then replicate to the new system. But that does not work. Despite slurpd's many flaws, this approach should have worked fine. Since you provide no details, there's no way to tell what went wrong. But in the meantime, slurpd is also obsolete and no longer supported. Any ideas on how it can be done with minimal impact to the current LDAP and reducing the outage period? The outage period should only last as long as it takes to run slapcat on your current server. I would go one step further: edit your 2.2 slapd.conf and turn on read-only mode - then your outage for servicing reads is only as long as it takes to restart the server. Once the server is in read-only mode you can run slapcat while slapd is running so there's no further service impact. Loading on the new server, read the new slapadd(8) manpage, look at using the -q option. Do yourself a favor and use OpenLDAP 2.4. Questions about 2.3 or older will be ignored by the Project. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Failover Failure Advice
Anton Chu wrote: I've setup a master and slave ldap service for failover; however, I'd like some advice on how to keep the ldap clients cached with the ldap creds if ever the master and slave ldap server goes. I've tried to extend the time of the caching on nscd - name server caching daemon - but it doesnt work when I add ldap users to certain groups. I've also tried pam caching credentials but doesn't work that well either. Finally, I also tried sssd but couldn't get it to work on my Ubuntu 10.10 clients. Anyone have simple solution that works when slave and master ldap servers get out of commission? I've thought about getent passwd /etc/passwd cron job, etc. Setup OpenLDAP nssov on all clients, use proxycache overlay and/or syncrepl to continue operating when servers and/or networks fail. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: nss_override_attribute_value
Hugo Monteiro wrote: Hello, I was wondering if there are any plans to make nss_override/default_attribute_value allow the use of env vars to perform substitutions. Something along the line of nss_override_attribute_valuehomeDirectory/home/users/${USER} This would allow great flexibility, specially when combined with pam_env and such. You're on the wrong list. There is no piece of OpenLDAP software that uses nss_override_attribute_value so discussing enhancements to it is pretty far off base. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How to enable 'pwdPolicySubentry' in ppolicy.schema
Alexey Shalin wrote: Hello, How to enable 'pwdPolicySubentry' in ppolicy.schema, I added this into ppolicy.schema Never modify the schema files distributed with OpenLDAP. attributetype ( 1.3.6.1.4.1.42.2.27.8.1.23 NAME 'pwdPolicySubentry' DESC 'The pwdPolicy subentry in effect for this object' EQUALITY distinguishedNameMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 SINGLE-VALUE USAGE directoryOperation ) But after that my slapd do not started. Of course. Schema files are only for defining user attributes. Operational attributes must be implemented in code and cannot be defined from a schema config file. This particular attribute is already implemented in the ppolicy overlay so there is no need to define it again anyway. should I upgrade openldap to the last ver ? That would make no difference here, but it's always best to stay up to date. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Exported group can't be imported again: Invalid syntax (21) additional info: objectClass: value #0 invalid per syntax
Thomas Schweikle wrote: Am 21.01.2011 13:19, schrieb Brian Candler: On Thu, Jan 20, 2011 at 08:04:00PM +0100, Thomas Schweikle wrote: The group I want to add: dn: cn=somegroup,ou=Groups,dc=example,dc=org objectClass: posixGroups objectClass: top gidNumer: 3000 cn: somegroup memberUid: someuser Are you sure you mean posixGroups, not posixGroup ? And gidNumer instead of gidNumber ? This was the point: the export was with posixGroups, but the newer schema only allowed posixGroup. Must have been changed sometime ... :( This schema has not changed in a dozen years. Your previous LDAP server simply didn't do schema validation and allowed you to store whatever garbage you gave it without checking. Have a look at nis.schema (or nis.ldif) to see what attributes are required or permitted for posixGroup. I corrected all posixGroups to posixGroup and import worked! -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapd : segmentation violation with Mingw/MSYS
Jean-François Berne wrote: Hello * I need help from experienced users concerning openLDAP under Mingw : I get a SIGSEGV upon slapd invocation. Please take a look at what I did and tell what I do wrong. Perhaps someone already had this problem. The SIGSEGV is at the end. 2. result: you can spot SIGSEGV right at the beginning. GNU gdb (GDB) 7.2 [...] This GDB was configured as mingw32. [...] Reading symbols from C:\MinGW\openldap-2.4.23\libexec/slapd.exe...(no debugging symbols found)...done. (gdb) start Function main not defined. Make breakpoint pending on future shared library load? (y or [n]) n Starting program: C:\MinGW\openldap-2.4.23\libexec/slapd.exe [New Thread 3572.0xbbc] Program received signal SIGSEGV, Segmentation fault. 0x6086ec49 in msys-1!getuid () from C:\MinGW\msys\1.0\bin\msys-1.0.dll (gdb) quit Looks like you built an MSYS binary, since you're crashing inside the msys DLL. That's not a good idea, you should be configuring gcc to build a pure Win32 binary instead. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MemberOf attribute not being returned
Mark Cairney wrote: Hi, I'm sure this was working in the past on this server but Im now not getting anything returned when I request the memberOf attribute. I compiled OpenLDAP 2.4.23 with the following flags: ./configure --prefix=/usr/local/authz --enable-meta --enable-ldap --enable-bdb --enable-monitor --enable-syncprov --enable-translucent --enable-memberof --enable-dyngroup --enable-dynlist --with-threads --with-tls --with-cyrus-sasl --enable-syslog --enable-spasswd cd make depend make make test make install I'm using slapd.d and I have the following in /usr/local/authz/etc/openldap/slapd.d/cn=config/olcDatabase={1}bdb olcOverlay={0}dynlist.ldif olcOverlay={1}memberof.ldif olcOverlay={2}syncprov.ldif The contents of olcOverlay\=\{1\}memberof.ldif are: You should not be poking or peeking at the files inside slapd.d. You should be using slapcat -n0 or ldapsearch -b cn=config to show the contents of the config database. As with other slapd databases, its structure and format are subject to change without notice at any time. The only thing guaranteed to remain compatible is the LDAP interfaces to the database. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How to get the socket to listen for replies from referral connections?
Ian Puleston wrote: Hi, Some time back I implemented an OpenLDAP-based LDAP client in a network appliance that uses code based on the tools for LDAP searches etc. I used the tool code because it gave me a nice easy LDIF-based API, but the downside was that each search operation was totally synchronous, so I'm now working on updating it to support asynchronous searches. I have a daemon task that listens for replies from the server after a request is set, and that then calls ldap_result, ldap_parse_result, etc. - basically all the things that ldap_do_search does synchronously. The daemon task, being ouside of OpenLDAP, gets the socket to listen on for each request via ldap_get_option with LDAP_OPT_SOCKBUF. This is working well. The problem is when new connections are opened when a referral/reference is followed. In this case the daemon needs to listen for replies on these too, but there is no way that I can see to get at its socket from outside the OpenLDAP core. LDAP_OPT_SOCKBUF gets the socket for the primary connection and there is no equivalent mechanism to get the sockets for the referral connections. What I've had to do is to add a new LDAP_OPT_REF_SOCKBUFS in my local OpenLDAP code to return a list of the socket buffers for additional connections opened for referrals. I am using an oldish version - 2.3.32 so: - am I missing any alternate way to get these sockets to listen on? Yes. Turn off automatic referral chasing and take care of referrals in your own code, then you always have access to every socket. - has anything been added in more recent versions that will help with this? Yes, look at LDAP_OPT_CONNECT_CB in 2.4. You can set a callback that is invoked right after a connection is opened and right after it's closed. Your callback function gets the sockbuf as one of the parameters. - if not would this new option LDAP_OPT_REF_SOCKBUFS be a useful addition? If so I could submit it via the bug reporting. No, I don't think it's needed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Authentication for on the fly configuration updates in OpenLDAP 2.4
Dan White wrote: I have discovered this myself, and I personally just rebuild from my own slapd.conf. I just took a look at the debian/slapd.conf template file in squeeze, which presumably is what the package installation uses to ultimately generate the slapd.d config backend. I've copied it here: http://web.olp.net/dwhite/openldap/slapd-squeeze-default.conf This config is missing two pretty important items in my opinion: authz-regexp gidNumber=0\\\+uidNumber=0,cn=peercred,cn=external,cn=auth cn=admin,@SUFFIX@ and databaseconfig rootdn cn=admin,@SUFFIX@ See: http://www.openldap.org/lists/openldap-technical/201101/msg00047.html Your recommendation assumes that a typical slapd installation has only one main database, and the local host sysadmin is also the LDAP DB admin. In other scenarios where there are multiple databases, it's more appropriate to leave the cn=config rootdn at its default and separate the role of slapd administrator from regular database admin. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: AccessLog behaviour
Marco Pizzoli wrote: Hi all, I would like to know the internal behaviour of slapo-accesslog. Read the slapo-accesslog(5) manpage. Re-read the logpurge description and don't skip any of it. I would like to have accesslog with a data retention of 10 days. In my ldap usage scenario this retention produce an accesslog db of quite 10GB of data. Things are working quite fine, but when the log db contains data older than the retention I defined after restart slapd has a thread allocated at 100% of cpu of one processor. I logged at loglevel 992 = 512 + 256 + 128 + 64 + 32, blocked with iptables every external ldap access and saw these lines in the log: [cut] Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to reqStart=20110202042854.08Z,cn=log03,dc=mycorp.it http://mycorp.it reqStart requested Feb 3 09:37:08 ldap03 slapd[16289]: = root access granted Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted by manage(=mwrscxd) Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter 6 Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter Feb 3 09:37:08 ldap03 slapd[16289]: LE Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to reqStart=20110202042854.10Z,cn=log03,dc=mycorp.it reqStart requested Feb 3 09:37:08 ldap03 slapd[16289]: = root access granted Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted by manage(=mwrscxd) Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter 6 Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter Feb 3 09:37:08 ldap03 slapd[16289]: LE Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to reqStart=20110202042854.12Z,cn=log03,dc=mycorp.it reqStart requested Feb 3 09:37:08 ldap03 slapd[16289]: = root access granted Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted by manage(=mwrscxd) Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter 6 Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter Feb 3 09:37:08 ldap03 slapd[16289]: LE Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to reqStart=20110202042854.14Z,cn=log03,dc=mycorp.it reqStart requested Feb 3 09:37:08 ldap03 slapd[16289]: = root access granted Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted by manage(=mwrscxd) Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter 6 Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter Feb 3 09:37:08 ldap03 slapd[16289]: LE Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to reqStart=20110202042855.00Z,cn=log03,dc=mycorp.it reqStart requested Feb 3 09:37:08 ldap03 slapd[16289]: = root access granted Feb 3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted by manage(=mwrscxd) Feb 3 09:37:08 ldap03 slapd[16289]: = test_filter 6 Feb 3 09:37:09 ldap03 slapd[16289]: = test_filter Feb 3 09:37:09 ldap03 slapd[16289]: LE Feb 3 09:37:09 ldap03 slapd[16289]: = access_allowed: search access to reqStart=20110202042855.02Z,cn=log03,dc=mycorp.it reqStart requested Feb 3 09:37:09 ldap03 slapd[16289]: = root access granted Feb 3 09:37:09 ldap03 slapd[16289]: = access_allowed: search access granted by manage(=mwrscxd) Feb 3 09:37:09 ldap03 slapd[16289]: = test_filter 6 Feb 3 09:37:09 ldap03 slapd[16289]: = test_filter Feb 3 09:37:09 ldap03 slapd[16289]: LE Feb 3 09:37:09 ldap03 slapd[16289]: = access_allowed: search access to reqStart=20110202042855.04Z,cn=log03,dc=mycorp.it reqStart requested [cut] I could assume that slapo-accesslog is querying all entries older than the retention and passes to delete them. What if this step to end the job will take more time than the interval time of re-starting? Augmenting the frequency of this job could help in this meaning? Every best practice to follow to use accesllog effectively will be very appreciated! :-) Thanks Marco Pizzoli -- _ Non è forte chi non cade, ma chi cadendo ha la forza di rialzarsi. Jim Morrison -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How to use LDAP_OPT_CONNECT_ASYNC?
Ian Puleston wrote: Hi, I'm trying to use the new LDAP_OPT_CONNECT_ASYNC with OpenLDAP 2.4.23 client and I can't see how it can be done. After creating the connection structure via ldap_initialize() I then call ldap_sasl_bind() to do the bind. That calls down to ldap_new_connection() (ldap_sasl_bind - ldap_send_initial_request - ldap_open_defconn - ldap_new_connection) and that implements the asynchronous connect, setting the lconn_status to LDAP_CONNST_CONNECTING and not calling ldap_int_poll() in ldap_pvt_connect(). So far so good. But ldap_send_initial_request() then gets a good return code from ldap_open_defconn(), and so it goes on to call ldap_send_server_request() to send the bind, and that then calls ldap_int_poll() which waits for the connect to complete, making the whole operation synchronous. It seems to me that for this to work asynchronously, either: 1. ldap_send_initial_request() should check for lconn_status to LDAP_CONNST_CONNECTING on return from ldap_open_defconn(), and if it is then return, probably with error LDAP_X_CONNECTING. It should then be possible to re-call ldap_sasl_bind() when the connect has completed. 2. Establish the connection via ldap_new_connection() before calling ldap_sasl_bind(). But the functions to do that don't appear to be available on the API. ldap_open() could have probably done it, but it is now deprecated. So this would appear to require a new API function that would simply call ldap_open_defconn() and return LDAP_X_CONNECTING. Or is there some other way to achieve this? Since ldap_open_defconn() is not named ldap_int_xxx or ldap_pvt_xxx I'd guess you should just use that. Another alternative is to simply use your own code to create a connection, and call ldap_init_fd() when you're ready. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: 'Inheriting' an attribute from a group.
ian+openl...@comtek.co.uk wrote: Is there a recommended, 'production' way to inherit certain attributes from a group? Perhaps some kind of overlay? For example, if a user is in the 'users' group and the users group has the attribute userQuota=100 then the user will appear to have userQuota=100. If, on the other hand, userQuota=200 is set on the user then that would override the current value. I don't think there's anything in the current code that does this, but you could modify the collect (collective attributes) overlay to do it. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Compile contrib modules?
Marco Pizzoli wrote: Hi list, Today I downloaded the HEAD repository and tried to compile also the contrib module lastbind. With the configure script I'm not able to have that module built in. I have to do something particular? The configure script doesn't do anything with contrib. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapo-lastbind
Marco Pizzoli wrote: On 7 Feb 2011 22:31, Marco Pizzoli marco.pizz...@gmail.com mailto:marco.pizz...@gmail.com wrote: Hi Jonathan, I understand the locality of the attribute, but my question is motivated by the ppolicy_forward_updates directive that addresses this identical problem. Do you think it would be possible to have a similar feature with this overlay? Thanks Marco On 7 Feb 2011 20:49, Jonathan CLARKE jonathan.cla...@normation.com mailto:jonathan.cla...@normation.com wrote: Note that authTimestamp is a DSA-specific attribute. (Look at its schema definition, it is dsaOperation.) Such attributes are, by design, not replicated, since their content is only meant for and valid on the particular DSA. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapd.d syntax help for ldap proxy server
Anton Chu wrote: Does anyone have a working ldap proxy configuration script? Some attributes such as olcURI are not welcomed with slapd on ubuntu 10.10. My goals is to make a standalone proxy. Make sure you're actually using a valid schema. ldapsearch -Y EXTERNAL -H ldapi:/// -b cn=schema,cn=config -s base Read the definition of the olcLDAPConfig objectclass. If you still can't figure that out, then just write a regular slapd.conf and convert it to cn=config format using slaptest. TIA, Anton On Fri, Feb 4, 2011 at 12:46 PM, Dieter Kluenter die...@dkluenter.de mailto:die...@dkluenter.de wrote: Am Fri, 4 Feb 2011 11:45:36 -0800 schrieb Anton Chu anton@telecommand.com mailto:anton@telecommand.com: I'm trying to setup a ldap proxy server for push based replication. I'm in need of help with providing the correct syntax on installing a ldap proxy using slapd.d instead of slapd.conf.The items in bold are the questionable syntax that can crossover to slapd.d. Here's my slapd.d configuration: Standalone LDAP Proxy: # load the schemas ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/cosine.ldif ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/inetorgperson.ldif ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/nis.ldif ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/misc.ldif ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/ldapns.ldif ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/openldap.ldif # Load dynamic backend modules dn: cn=module,cn=config objectClass: olcModuleList cn: module olcModulepath: /usr/lib/ldap olcModuleload: back_hdb olcModuleload: syncprov # Database settings dn: olcDatabase=hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {1}hdb This should be a ldap database, not a hdb database databaseldap # ignore conflicts with other databases, as we need to push out to same suffix hidden on suffix dc=suretecsystems,dc=com rootdn cn=slapd-ldap uri ldap://localhost:9012/ lastmod on # We don't need any access to this DSA restrictall acl-bindbindmethod=simple binddn=cn=replicator,dc=suretecsystems,dc=com credentials=testing syncreplrid=001 provider=ldap://localhost:9011/ binddn=cn=replicator,dc=suretecsystems,dc=com bindmethod=simple credentials=testing searchbase=dc=suretecsystems,dc=com type=refreshAndPersist retry=5 5 300 5 overlay syncprov -Dieter -- Dieter Klünter | Systemberatung http://dkluenter.de GPG Key ID:DA147B05 53°37'09,95N 10°08'02,42E -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Multimaster replication of cn=config working too well?!
Mark Cairney wrote: Hi, Has anyone experienced problems with the cn=config directory syncing the same change across all nodes and breaking the configuration? In the past I've had problems where the syncrepl parameters are synced across all nodes, including that node itself, which results in the servers then trying to sync with themselves. The most recent problem however was that the olcSaslHost value was synced to be one particular server on all 3 nodes. As I was using GSSAPI to do the replication this broke it completely and it meant that I could only bind via GSSAPI to one of the 3 servers. In the meantime I've set the servers not to syncrepl cn=config but this isn't ideal. Anyone else on the list had similar problems? The potential for royally screwing things up has been discussed on the -devel list from time to time. The use of serverIDs and explicit URL matching was introduced to prevent consumers pointing at themselves, that's already documented. For other settings, it's probably best to add some excluded attributes to your consumer configs. And of course, turning off multimaster replication of cn=config is always a good possibility. Nobody said you *had* to do it. We only document it to show that you *can* do it if you need it, that doesn't mean it's a recommended scenario. Kind regards, Mark /* Mark Cairney ITI UNIX Section Information Services University of Edinburgh Tel: 0131 650 6565 Email: mark.cair...@ed.ac.uk */ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Handling slapd.d in OpenLDAP and Kerberos
Jaap Winius wrote: Quoting sarathkrishn...@gmail.comsarathkrishn...@gmail.com: For authenticating via OpenLDAP, the principles needs to be rewritten (using authz-policy and authz-regexp). We know how to do that in older version of OpenLDAP which had (slapd.conf) but don't know how to do the same in new OpenLDAP which has slapd.d directory instead. The manuals also doesn't say anything on this issue. The switch from slapd.conf to cn=config takes a little getting used to, plus the migration script may not work for you, but in the end I produced a set of procedures that should tell you most of what you want to know: * Integrated Kerberos-OpenLDAP provider on Debian squeeze http://www.rjsystems.nl/en/2100-d6-kerberos-openldap-provider.php * Integrated Kerberos-OpenLDAP consumer on Debian squeeze http://www.rjsystems.nl/en/2100-d6-kerberos-openldap-consumer.php True, I didn't use Ubuntu in these examples, but I would not be surprised if the procedures were almost identical, certainly with cn=config. If you read http://highlandsun.com/hyc/drafts/draft-chu-ldap-xordered-xx.html You could simplify your ACL changes in 7.1.1.x. dn: olcDatabase={1}hdb,cn=config changetype: modify delete: olcAccess olcAccess: {2} olcAccess: {1} olcAccess: {0} - Similarly in 7.1.2.x you don't need to specify the prefixes when you're adding rules in order. Cheers, Jaap -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapd.conf for proxy to AD
Buchan Milne wrote: On Wednesday, 9 February 2011 01:13:38 masar...@aero.polimi.it wrote: As far as I recall, what you need is not possible. You can: - have authenticated users proxied with their identity asserted, or - all users, including unauthenticated ones auth'd as a fixed identity but not both. Please note that you're asking OpenLDAP's slapd to bridge the gap between two broken pieces of code: clients that cannot be configured to bind, Because the OP wants to have anonymous access for some clients of the proxy does not necessarily mean the software is broken, there could be organisational reasons (e.g. AD administrator not prepared to create multiple proxy accounts for different applications). and AD that cannot be configured to accept anonymous requests (AFAIK). Again, AD can be configured to allow anonymous binds, but AFAIK as of Windows 2003, it is disabled by default, and many AD security standards (used by e.g. financial auditing companies in their IT auditing) mandate that it not be enabled. Feel free to suggest an enhancement that allows to handle this scenario, though. IMHO, it would make sense to support this mode of operation. Sorry but that just doesn't compute. If you have organizational security standards that are being audited and they forbid anonymous access, then allowing anonymous access to an OpenLDAP proxy that connects to AD is going to be equally forbidden. Come back with an argument that actually makes sense from a security perspective first. Otherwise this is just creating back doors to do end-runs around your sysadmins and company policies. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: port level security for auth and anon
Christopher Louis Jackson wrote: I am looking for help with setup of security with my openldap config. I currently have RHEL 6 with ldap:// and ldaps:// working for both auth binds and anon binds. What I want to do is allow anon binds on ldap:// and require authentication over an encrypted stream on ldaps:// my current access is set to: access to attrs=userPassword by anonymous auth by self read by * none access to * by * read I do not have a security statement in my slapd.conf. I have tried a few things such as changing the userpassword access to: access to userPassword by anonymous auth sasl_ssf=128 break by anonymous auth tls=128 by self read but the syntax is not correct and the config will not load with above. Any help would be great. As the slapd.access(5) manpage clearly states, the syntax is access to what [ by who [ access ] [ control ] ]+ sasl_ssf=128 is a who specifier but you have it after the access specifier. We don't just write things randomly. Read and follow what's actually written in front of you. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Mac OS X OpenLDAP allows anonymous access to all fields
RAT wrote: The ongoing saga... While trying to add the ACL for to hide userPassword, we get the following error: ldap_modify: Insufficient access (50) I'm beginning to think Apple has this locked down... You should ask Apple. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How to use LDAP_OPT_CONNECT_ASYNC?
Ian Puleston wrote: -Original Message- From: masar...@aero.polimi.it [mailto:masar...@aero.polimi.it] Sent: Thursday, February 03, 2011 9:35 PM ldap_int_poll() will return -2 if timeout is 0 and the filedes was not active. So ldap_send_server_request() will return LDAP_X_CONNECTING if async. I've hit a new problem with this which I'm pretty sure is a bug (or else an unsupported case) which is that LDAP_OPT_CONNECT_ASYNC doesn't work with TLS connections. It is a timing issue so it may work fine in some implementations, and in fact it works fine for me every time if I have any debug output turned on. The problem is that ldap_int_tls_connect fails, and I'm pretty sure that it fails because it tries to start the TLS handshake before the connect has completed. Adding just one printf at the start of ldap_int_tls_connect delays it enough that it then works OK every time, presumably because the connect then completes first. I'm working on a fix now, and I think what is needed is: 1. A call to ldap_int_poll in ldap_int_tls_start if async. Then it should abort without calling ldap_int_tls_connect if not ready (probably return LDAP_X_CONNECTING). 2. In ldap_send_initial_request don't call ldap_send_server_request if lconn_status is LDAP_CONNST_CONNECTING and using TLS (since TLS needs to be started first). Instead return LDAP_X_CONNECTING. 3. Probably something similar to #1 for the case where ldap_start_tls_s is called from ldap_new_connection. If this all works I'll put in a bug report with a patch. Sounds to me like you should just file the bug report first. The patch can come later. Also I don't believe the proper fix should touch so many places. All you should need is to fix the caller of ldap_int_tls_connect(), and as that's a static function there is only one place to fix. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How to use LDAP_OPT_CONNECT_ASYNC?
Ian Puleston wrote: Hi Howard, -Original Message- From: Howard Chu [mailto:h...@symas.com] Ian Puleston wrote: I'm working on a fix now, and I think what is needed is: 1. A call to ldap_int_poll in ldap_int_tls_start if async. Then it should abort without calling ldap_int_tls_connect if not ready (probably return LDAP_X_CONNECTING). 2. In ldap_send_initial_request don't call ldap_send_server_request if lconn_status is LDAP_CONNST_CONNECTING and using TLS (since TLS needs to be started first). Instead return LDAP_X_CONNECTING. 3. Probably something similar to #1 for the case where ldap_start_tls_s is called from ldap_new_connection. If this all works I'll put in a bug report with a patch. Sounds to me like you should just file the bug report first. The patch can come later. I need to get this fixed in the port I am using so I will do that and attach the patch to the bug report. The relevant openldap developers can then choose whether to use that patch as-is or work out something different, and if the latter I can replace the fix in my port with it when it gets released. You seem to be making the bizarre assumption that only you and other OpenLDAP developers will care. Filing the bug report even before you have the patch allows other people in the community to be alerted and get involved. That is the purpose of an open source community, and the purpose of a public bug tracker. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Ppolicy does not seem to work
Jan Kohnert wrote: So there comes the next question: Is there a way to lock out specific users permanently (other than creating a cronjob setting the lockout time new after 900s) or do I need to set pwdLockoutDuration to inf and so are forced to manually reset users whose accounts were tried to be cracked? Read the slapo-ppolicy manpage again. This is explicitly documented. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Slapd Security based on port
Andrew Findlay wrote: On Mon, Feb 14, 2011 at 07:49:10PM +, Chris Jackson wrote: I know: Anonymous bind can be disabled by disallow bind_anon and Unauthenticated bind mechanism is disabled by default. But if I use disallow bind_anon it stops in on both ports. I want to stop it just on ldaps://. Maybe you should stop thinking about ports and start thinking about *where* the LDAP clients are. You can then permit anon access to clients within your own network (by IP range) and permit access by any authenticated user, before denying all other cases. Remember to allow enough access for the external users to connect and bind in the first place! Note that it is almost impossible to hide the *existance* of an entry, so if DNs are guessable it is possible that a determined outsider could work out who is in your directory. See the disclose ACL privilege - you can hide the existence if you really want to. slapd's security mechanisms will support just about any conceivable security policy. If some of the data is very sensitive you may prefer to set up an 'outside' server and replicate just a subset of the data to it. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldap auth does not works after openldap upgrade
Andrew Findlay wrote: On Tue, Feb 15, 2011 at 05:08:43PM -0200, Leonardo Carneiro wrote: fileserver:/etc/ldap# /usr/sbin/slapd -h ldapi:/// ldap:/// -g openldap -u openldap -F /etc/ldap/slapd.d -d 128 Aha! Your server is using LDAP-based config so it is ignoring the config file entirely. Does these changes that we are making into slapd.conf really being processed? Normally, i see just the -F /etc/ldap/slapd.d flag and never the -f /etc/ldap/slapd.conf. I suspect the config file was converted to a config dir during the Debian upgrade process, so the file is now being ignored. I also suspect that there may not be a valid password set on the cn=config suffix, so you will not be able to manage the server through LDAP either. Since it's starting on ldapi:/// he should just do a SASL EXTERNAL bind on ldapi:// using Unix root. Pretty sure Debian packages it with the appropriate authz-regexp already configured. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldap auth does not works after openldap upgrade
Andrew Findlay wrote: On Wed, Feb 16, 2011 at 02:51:19AM -0800, Howard Chu wrote: I also suspect that there may not be a valid password set on the cn=config suffix, so you will not be able to manage the server through LDAP either. Since it's starting on ldapi:/// he should just do a SASL EXTERNAL bind on ldapi:// using Unix root. Pretty sure Debian packages it with the appropriate authz-regexp already configured. I don't have a Debian Squeeze server at present so I cannot check that. Where is this documented? I am having great trouble finding any clear description of how to actually access cn=config in the bootstrap case. I don't know where Debian documents their bootstrap config, you'll have to ask them. Similarly I cannot find anything that clearly describes the use of SASL EXTERNAL with ldapi. http://tools.ietf.org/html/draft-chu-ldap-ldapi-00 If you can point me at some authoritative statements I will propose a patch for the Admin Guide. Andrew -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldap auth does not works after openldap upgrade
Leonardo Carneiro wrote: On Wed, Feb 16, 2011 at 8:51 AM, Howard Chu h...@symas.com mailto:h...@symas.com wrote: Andrew Findlay wrote: On Tue, Feb 15, 2011 at 05:08:43PM -0200, Leonardo Carneiro wrote: fileserver:/etc/ldap# /usr/sbin/slapd -h ldapi:/// ldap:/// -g openldap -u openldap -F /etc/ldap/slapd.d -d 128 Aha! Your server is using LDAP-based config so it is ignoring the config file entirely. Does these changes that we are making into slapd.conf really being processed? Normally, i see just the -F /etc/ldap/slapd.d flag and never the -f /etc/ldap/slapd.conf. I suspect the config file was converted to a config dir during the Debian upgrade process, so the file is now being ignored. I also suspect that there may not be a valid password set on the cn=config suffix, so you will not be able to manage the server through LDAP either. Since it's starting on ldapi:/// he should just do a SASL EXTERNAL bind on ldapi:// using Unix root. Pretty sure Debian packages it with the appropriate authz-regexp already configured. Tks for the tips guys. I'll try all this stuff at launch time here in Brazil (about 2 hours from now). There are to many users using Samba right now, and every time I have to restart the OpenLDAP something on samba crashes. The entire point of using cn=config for configuration is that config changes don't require slapd restarts. As far as i'm concerned, i didn't have the need to use SASL, and it seems that all this SASL mechanism was auto-introduced in my setup after the upgrade. Is it safe to edit /etc/defaults/slapd and remove the ldapi:/// parameter in SLAPD_SERVICES line or i can break something very hard doing this? Removing that would be dumb. It's there to give you an easy means to administer the server. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldapi:
Andrew Findlay wrote: On Wed, Feb 16, 2011 at 08:37:01AM -0800, Quanah Gibson-Mount wrote: I don't get where you're getting this idea from. I've never been required to use ldapi for setting up cn=config. The requirement of using ldapi for cn=config management is something Debian/Ubuntu is doing. Which is why, again, I say this specifically is not something to be documented by the OpenLDAP project as a cn=config requirement. OK - forget the link to cn=config - that is just what got me interested in ldapi: to start with. The ldapi: transport seems to be almost undocumented. There is a brief (but useful) mention in ldap.conf(5) but very little elsewhere. I have found enough hints in the archives to get started, so I will file some suggestions. Everything worth noting about ldapi: was already published as an IETF Informational Draft, and I've already posted the link to that document several times. Beyond that it's no different from any other LDAP URL. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldap design
Hendrik van der Ploeg wrote: Hello, Thank you for answering. Yes, every customer has it's own ldap server which should be master. */Q: You do not want it the other way around and have 4 (multi-)master in the datacenter and 1200+ replicas outside, do you? /* That would be the best design indeed, but the problem is that the customers need write access to their own ldapserver. Now I hear you thinking; Customers can have write access in the datacentre and it then will be replicated to the customers own ldapserver. The problem is that a lot of customers have quite a bad vpn connection to the datacentre and when they add a user for example it must be available immediatley. And with a failing vpn connection nothing happens. they MUST be able to write in their own local ldapserver Your reasoning is sound. It's just a question of system resources; 1200 consumer configs in a single slapd process will probably occupy a lot of RAM. An active consumer requires a slapd thread when processing incoming changes. If you want instantaneous processing of all incoming changes that means you must be able to handle 1200 concurrent threads, worst case. If your real world load will be lower than that, adjust downward accordingly. Regards Hendrik Noordwijkerhout Holland Hendrik van der Ploeg schrieb am 15.02.2011 08:47 Uhr: I'm in doubt what design I need to use for openldap This is the situation; We have 1200+ customers using LDAP. We want to replicate all these ldap server to 1 big ldapserver in a datacentre with a multi-master config. This means each customer has its own ldap server and you will have 1200+ ldap servers? So all the customers are a master-ldap who replicate to the datacentre. all the customers (=ldap servers) _are_ master-ldap? This would mean you have 1200+ provider/master!? My idea was to build in the datacentre a ldapcluster of about 4 server What for then? My question is: Will this be stable, because there will be 1200+ ldapservers replicating to 4 ldapserver in the datacentre. You do not want it the other way around and have 4 (multi-)master in the datacenter and 1200+ replicas outside, do you? I know this depends on the number of write actions at the customers. All I can say is that write actions at the customers isn't THAT much. I really hope somebody can give me an answer or maybe there's somebody else with the same config We have one provider and 160 consumers - and this is IMHO called a lot here, if I'm right ... Marc -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Conflict resolution on Syncrepl
Diego Lima wrote: Hello all, This is mostly a theoretical question. I currently have some OpenLDAP multimaster clusters (using syncrepl and mirrormode) and although it never happened before, what would happen if two (or more) masters received a write request exactly at the same time? The entry would have the same entryCSN on both servers and the contextCSN would be updated at the same time? Would this lead to a situation where the servers were actually out of sync but they would think they had the same entries? No. On a properly configured setup it is impossible for two servers to produce the same entryCSN (regardless of timestamp issues) since they each have unique serverIDs. Conflict resolution will always resolve consistently and deterministically in syncrepl. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Error: CSN too old and replication fails
Howard Chu wrote: Jérémy Wagner wrote: Hello, I'm facing some issues with syncrepl... The simplest situation in which I was able to reproduce the problem consists of 1 provider and 1 consumer. I have configured syncrepl to do a partial replication : olcSyncrepl: {0}rid=105 provider=ldaps://:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter=(mail=*) searchbase=ou=groups,o=mydir scope=sub schemachecking=off attrs=cn description mail member + type=refreshAndPersist retry=60 + tls_cert= tls_cacert= tls_key= olcSyncrepl: {1}rid=107 provider=ldaps://:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter=(objectclass=*) searchbase=ou=operators,o=mydir scope=sub schemachecking=off type=refreshAndPersist retry=60 + tls_cert= tls_cacert= tls_key= olcSyncrepl: {2}rid=104 provider=ldaps://:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter=(uid=mytestuser) searchbase=ou=people,o=mydir scope=sub schemachecking=off type=refreshAndPersist retry=60 + tls_cert= tls_cacert= tls_key= The initial sync (i.e. from an empty base) is fine, but then when I modify an entry (uid=mytestuser), it is not updated on the consumer, and I get the following messages: Feb 16 15:42:32 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144232.222152Z#00#000#00 Feb 16 15:42:32 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144232.222152Z#00#000#00 Feb 16 15:42:32 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144232.222152Z#00#000#00 Feb 16 15:42:32 do_syncrep2: rid=104 CSN too old, ignoring 20110216144232.222152Z#00#000#00 Feb 16 15:43:05 do_syncrep2: rid=105 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=105 NEW_COOKIE: rid=105,csn=20110216144305.545784Z#00#000#00 Feb 16 15:43:05 slap_queue_csn: queing 0x7fb58c020b50 20110216144305.545784Z#00#000#00 Feb 16 15:43:05 do_syncrep2: rid=107 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144305.545784Z#00#000#00 Feb 16 15:43:05 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144305.545784Z#00#000#00 Feb 16 15:43:05 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144305.545784Z#00#000#00 Feb 16 15:43:05 do_syncrep2: rid=104 CSN too old, ignoring 20110216144305.545784Z#00#000#00 (both servers are perfectly in sync with an NTP daemon) It looks like ITS #6619 is quite similar, and unresolved. What do these messages mean ? Is there a way to force the update ? Your current configuration is unsupported. Ordinarily when you configure multiple consumers in a DB it is assumed they are all pointing at different providers, and each provider will have a different serverID. In your case, since it is all a single provider with a single serverID, all 3 consumers will share a single contextCSN. Any update in any branch will update the contextCSN to that newest state. Any updates in other branches that hadn't been received yet (due to random timing between threads) will be received as too old because the contextCSN already has a newer state. A workaround for this would be to split each consumer into its own subordinate database, then each will have a private copy of its own contextCSN to work with. Another potential workaround, if all of the consumer configs were identical except for their baseDN, would be to simply consolidate them all into a single consumer, with a more complex search filter. searchbase=o=mydir filter=(| ((entryDN:dnSubtreematch:=ou=groups,o=mydir)(mail=*)) (entryDN:dnSubtreematch:=ou=operators,o=mydir) ((entryDN:dnSubtreematch:=ou=people,o=mydir)(uid=mytestuser))) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Conflict resolution on Syncrepl
Diego Lima wrote: No. On a properly configured setup it is impossible for two servers to produce the same entryCSN (regardless of timestamp issues) since they each have unique serverIDs. Conflict resolution will always resolve consistently and deterministically in syncrepl. Thanks a lot for your explanation! So, should one assume that the server with the highest serverID will have priority when it comes to conflict resolution? Yes. Also, when and how is the serverID determined? Is this manually configurable? Yes. Read the slapd config manpage. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: undefined reference to `ldap_int_tls_impl' (tls2.c) ?
Elle Y Suzuki wrote: make is exiting with several undefined reference to 'ldap_int_tls_impl' error messages: Pretty interesting. This basically means that configure ran without selecting any TLS implementation, but still left TLS enabled. Can you upload your config.log file somewhere for us to examine? ./.libs/libldap.a(tls2.o): In function `ldap_pvt_tls_ctx_free': /home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:79: undefined reference to `ldap_int_tls_impl' ./.libs/libldap.a(tls2.o): In function `tls_init': /home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:163: undefined reference to `ldap_int_tls_impl' /home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:163: undefined reference to `ldap_int_tls_impl' /home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:168: undefined reference to `ldap_int_tls_impl' ./.libs/libldap.a(tls2.o): In function `ldap_pvt_tls_check_hostname': /home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:494: undefined reference to `ldap_int_tls_impl' ./.libs/libldap.a(tls2.o):/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:860: more undefined references to `ldap_int_tls_impl' follow collect2: ld returned 1 exit status make[2]: *** [apitest] Error 1 make[2]: Leaving directory `/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap' make[1]: *** [all-common] Error 1 make[1]: Leaving directory `/home/vmplanet/Downloads/openldap-2.4.24/libraries' make: *** [all-common] Error 1 what is the proper way to fix this so that i might successfully make? BACKGROUND i am attempting to set up and install openldap 2.4.24 on a virtual kubuntu on windows xp. 'configure' and 'make depend' appear to run successfully. i execute the following steps: export CPPFLAGS=-I/usr/include/nss -I/usr/include/nspr -I/usr/local/BerkeleyDB.5.0/include export LDFLAGS=-L/usr/lib/nss -L/usr/local/BerkeleyDB.5.0/lib export LD_LIBRARY_PATH=/home/vmplanet/Downloads/db-5.0.32/build_unix/.libs configure --enable-shell=yes --without-threads make depend (in case you're wondering why back-shell, i've included details in a previous thread, http://tinyurl.com/4flbgh4. note that in that thread, i successfully installed openldap directly on windows, but had questions about successfully testing back-shell -- and still do.) going back to the errors above, i note in which files this 'ldap_int_tls_impl' appears with a grep: ./libraries/libldap_r/tls2.c:40:static tls_impl *tls_imp = ldap_int_tls_impl; ./libraries/libldap_r/tls_g.c:1095:tls_impl ldap_int_tls_impl = { ./libraries/libldap_r/tls_o.c:1253:tls_impl ldap_int_tls_impl = { ./libraries/libldap_r/tls_m.c:3040:tls_impl ldap_int_tls_impl = { ./libraries/libldap/tls2.c:40:static tls_impl *tls_imp =ldap_int_tls_impl; ./libraries/libldap/tls_g.c:1095:tls_impl ldap_int_tls_impl = { ./libraries/libldap/ldap-tls.h:75:extern tls_impl ldap_int_tls_impl; Binary file ./libraries/libldap/.libs/libldap-2.4.so.2.6.0 matches Binary file ./libraries/libldap/.libs/libldap.so matches Binary file ./libraries/libldap/.libs/tls2.o matches Binary file ./libraries/libldap/.libs/libldap-2.4.so.2 matches Binary file ./libraries/libldap/.libs/libldap.a matches Binary file ./libraries/libldap/tls2.o matches ./libraries/libldap/tls_o.c:1253:tls_impl ldap_int_tls_impl = { ./libraries/libldap/tls_m.c:3040:tls_impl ldap_int_tls_impl = { ldap_int_tls_impl appears (defined) in the fellow tls* files above. is this then an indication that tls2.c is not communicating or 'linked' to its fellow tls* files or ? the output from 'make depend' appears to show that mkdep traversed the libldap and libldap_r subdirectories and successfully constructed the necessary sets of include file dependencies. admittedly, i am not altogether that familiar with (the intricacies of) the compile/make process. please advise; thanks in advance. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: threads and concurrency
Omer Faruk SEN wrote: Hi , I have and ldap server (openldap-2.4.23 ) in slapd.conf I set concurrency 8192 threads 256 but when I do top -b -H -d 2 -p 25560 I always see 8 threads ( show 9 slapd instances) not more even if I instruct to create 256 threads. Is there anything that I miss? Additionally what is the releation of threads and concurrency. Is there any calculation method that I must obey? concurrency generally does nothing on most platforms. I believe it might have had an effect on old Solaris systems with their lwp library. You didn't mention what OS you're using so it's impossible to tell you if this directive has any relevance for you. Most likely not. The threads setting tells the maximum number of threads slapd will spawn. It creates them on demand, not all at once. If you want to see 256 threads then you have to submit enough requests to make the server busy enough to actually use them. Keep in mind that each thread uses a thread stack of 4MB on 32 bit systems, 8MB on 64 bit systems. So you're looking at 1-2GB of RAM dedicated to the threads with a setting like this. Make sure you don't configure more threads than you have RAM for... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: merging local and remote attributes
Hugo Monteiro wrote: Hello list, I have been trying to use translucent overlay to merge attributes between a remote and a local server (both 2.4.23). From the slapo-translucent man page i read: Attributes may be specified as both local and remote if desired. and In any case, both the local and remote entries corresponding to a search result will be merged before being returned to the client. The thing is that if i specify an attribute (objectclass) to be both local and remote, i can only get/search for the local entries. Choosing either separately will work as advertised though. Again, i ask if this is a bug, a subtlety i have missed or this is not supposed to work with objectClass attribute? Re-read the manpage. 1st paragraph. Entries retrieved from a remote LDAP server may have some or all attributes overridden, or new attributes added, by entries in the local database before being presented to the client. This overlay does not merge attributes, it replaces them. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Poor performance on Solaris
Quanah Gibson-Mount wrote: --On Thursday, February 24, 2011 3:53 PM +0100 juergen.spren...@swisscom.com wrote: Hi, we had some performance issues on our ldap servers running Solaris 10 sparc. I did some tests using slamd http://www.slamd.com/ and got disturbing results: ldap-service: OpenLDAP 2.4.23, setup identical on both boxes, threads=64, identical content. box1: hardware: Sun Microsystems sun4v SPARC Enterprise T5120 memory:32 GB RAM os: Solaris 10 s10s_u9wos_14a searches (avg/second): 1521 Slowaris is always tedious with OpenLDAP. At a previous job, I was able to replace 9 slowaris boxes with 4 linux boxes, and even then, just one of the Linux boxes could handle the complete load it took the 9 slowaris boxes to run. I will note that if you are going to use slowaris, I highly advise you set a memory key rather than using on disk cache for BDB if your DB is any size over about 4 GB. Other than that, you'll generally just have to deal with the fact it will be significantly slower than Linux. Actually, you should always use a shared memory key on Solaris. Using mmap'd files is just too slow on that OS. Some time ago, I did benchmarks of Linux 2.6 vs Slowaris-x86 on identical hardware, and it still was noticeably slower, although not as bad as when using the sparc architecture. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Poor performance on Solaris
Brett @Google wrote: On Fri, Feb 25, 2011 at 10:25 AM, Howard Chu h...@symas.com mailto:h...@symas.com wrote: I will note that if you are going to use slowaris, I highly advise you set a memory key rather than using on disk cache for BDB if your DB is any size over about 4 GB. Other than that, you'll generally just have to deal with the fact it will be significantly slower than Linux. Actually, you should always use a shared memory key on Solaris. Using mmap'd files is just too slow on that OS. Incidentlly, What is the uniqueness of a shared memory key, is it unique amoung all instances on a single server ? The significance of a shared memory key depends entirely on your operating system. But by its nature, shared memory is global to a machine, so you can assume that all keys must be unique on a particular machine. I have personally only ever needed a shared memory key on one of a group of ldap servers. But i noticed that the bdb logs are not written to disk when shared memory is configured, is this a good or bad thing ? The use of shared memory has no bearing on how the BDB library writes its transaction logs. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problem unexpected failing slapd
Ruud Baart wrote: Sorry, I think I found a way to start slapd with gdb. The main mistake I made was using uid openldap. It should be run as root. # gdb -q -x /root/gdb.init --args /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf -l local5 gdb.init: handle all nostop run quit This way I can modify the start-stop script. With a restart there is no need for user intervention. Now it runs and functions. I assume this is the way Howard Chu suggests. Let's wait and see what happens. No. I did not say to start slapd using gdb. I said to attach gdb to the running slapd, which means slapd should already be started, using whatever method you normally use to start it. What you've done will accomplish nothing. Op 28-2-2011 12:10, Ruud Baart schreef: Op 28-2-2011 3:00, Howard Chu schreef: Ruud Baart wrote: Problem: For a customer we use LDAP for many years. Last year suddenly the slapd service just stopped without any traces in the logfiles. After a restart of slapd everything works fine again. But the problem was there: it was not an incident, now and then slapd just stops and always without any traces in the logfiles. Sometime three times a day, sometime a week without a failure. I can't find a pattern or any relation to any other service on the linux server. Attach to the running slapd with gdb, type handle all nostop continue and let it run. If there's a crash you'll see what happened in gdb. I tried it but I don't get this working. I have no experience with gdb. I assume I do something wrong. Normally slapd daemon runs like this on a test machine /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf -l local5 If I try this running attached to gdb: su openldap -s /bin/bash gdb --args /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf -l local5 GNU gdb (GDB) 7.0.1-debian This GDB was configured as i486-linux-gnu. .. Reading symbols from /usr/sbin/slapd...(no debugging symbols found)...done. (gdb) handle all nostop SignalStop Print Pass to program Description SIGHUPNoYes Yes Hangup SIGQUIT NoYes Yes Quit ... EXC_EMULATION NoYes Yes Emulation instruction EXC_SOFTWARE NoYes Yes Software generated exception EXC_BREAKPOINT No Yes Yes Breakpoint (gdb) (gdb) continue The program is not being run. (gdb) run Starting program: /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf -l local5 [Thread debugging using libthread_db enabled] Program exited with code 01. At this point I have no idea what to do. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problem unexpected failing slapd
Ruud Baart wrote: Thank you all for the fast answers. I now attached gdb to slapd and get some results: (gdb) handle all nostop (gdb) continue Continuing. Program received signal SIGPIPE, Broken pipe. Program received signal SIGPIPE, Broken pipe. [New Thread 0xb01f6b70 (LWP 1548)] Program received signal SIGPIPE, Broken pipe. .. more of the same .. Program received signal SIGPIPE, Broken pipe. Program received signal SIGPIPE, Broken pipe. ---Typereturn to continue, or qreturn to quit--- Program received signal SIGPIPE, Broken pipe. Program received signal SIGTERM, Terminated. This is not a crash or any error in slapd; some external command was used to kill the slapd process. [New Thread 0xaf4f3b70 (LWP 1968)] [New Thread 0xaf0f2b70 (LWP 1969)] [New Thread 0xaecf1b70 (LWP 1970)] [New Thread 0xae8f0b70 (LWP 1971)] [Thread 0xaf4f3b70 (LWP 1968) exited] [Thread 0xb1bfcb70 (LWP 1474) exited] [Thread 0xb5268b70 (LWP 1462) exited] [Thread 0xaf0f2b70 (LWP 1969) exited] [Thread 0xb4565b70 (LWP 1463) exited] [Thread 0xb01f6b70 (LWP 1548) exited] [Thread 0xb5669b70 (LWP 1461) exited] [Thread 0xb17fbb70 (LWP 1475) exited] [Thread 0xae8f0b70 (LWP 1971) exited] [Thread 0xb28ffb70 (LWP 1464) exited] [Thread 0xaecf1b70 (LWP 1970) exited] Program exited normally. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: memberof/accesslog overlays together
Michael Ströder wrote: HI! Unfortunately for privacy reasons I can't provide example data. I'm using slapo-memberof, slapo-refint and slapo-access in exactly this order. Now I'm analysing some strange things where modifications to group entries and the subsequent modifications by slapo-memberof are not correctly written to the accesslog DB. Questions: 1. Is this overlay order a correct setup? Should this work? Or should slapo-memberof be invoked after slapo-access? Try changing the order and see for yourself. 2. CHANGES of release 2.4.24 lists a bunch of fixes to slapo-memberof. Any changes which possibly affect writing to accesslog DB? (I'm not sure whether we had problems like this with 2.4.23 or not though.) Doesn't sound like it. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Allowing users to add, but not delete, entries?
Gervase Markham wrote: Hi, Summary: is it possible to configure access control such that users to can add, but not delete, entries? This is difficult, because as far as I can see the write permission does not distinguish between adding and deleting. Can someone tell me if this is possible? Sounds to me like you haven't read the slapd.access(5) manpage. The answer is yes. Read the section on THE ACCESS FIELD in the manpage. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Allowing users to add, but not delete, entries?
Gervase Markham wrote: On 03/03/11 15:33, Howard Chu wrote: Note that a Guide is not a reference manual; it is not intended to be complete or exhaustive. I'm not sure this use of terminology is universal; here is a counter-example: http://www.bugzilla.org/docs/tip/en/html/ It's certainly been the norm for Unix at least. E.g., compare the Guides vs Reference Manuals in the Solaris documentation. http://download.oracle.com/docs/cd/E19253-01/index.html Or the 4BSD system documentation: http://www.freebsd.org/doc/en/books/faq/bibliography.html Material in the 4BSD User's Supplementary Documents and Programmer's Supplementary Documents was a bit more friendly in prose, but only explained the approach to how to use the system. The details of each command or function are only found in the Reference Manuals. It's only intended to provide an overview; the detailed/exhaustive docs are always the manpages. I would suggest that neither: http://www.openldap.org/doc/ nor http://www.openldap.org/doc/admin24/preface.html nor the individual pages in the Admin Guide give the impression either that the Admin Guide is not complete, or that the manual pages are an essential read or the canonical reference. Perhaps it would help OpenLDAP users if this were made a little more clear? The preface.html which you linked above clearly states: Scope of this Document This document provides a guide for installing OpenLDAP Software 2.4 (http://www.openldap.org/software/) on UNIX (and UNIX-like) systems. The document is aimed at experienced system administrators with basic understanding of LDAP-based directory services. This document is meant to be used in conjunction with other OpenLDAP information resources provided with the software package and on the project's site (http://www.OpenLDAP.org/) on the World Wide Web. The site makes available a number of resources. ... And then goes on to list various other resources. The Manual Pages, of course are the most obvious information resource provided directly with the software, and in Unix they are traditionally the authoritative reference. Given the heritage of this project it seems obvious to me that this is how the docs would be structured. If the phrase meant to be used in conjunction isn't a strong enough hint that the Guide is not a standalone document, please feel free to suggest an alternate or stronger wording. Note that the preface also invites you to do the same. Thank you for your help and support :-) Thank you for your feedback. We're always listening. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Replication monitoring
Torsten Schlabach (Tascel eG) wrote: Hi Andreas! Your servers should be recognized as openldap. * Check the Environment page if it says OpenLDAP. * Click on one of the servers to open the server view page and you should now see two more buttons. Yes, that works now. But I still didn't see any replication agreements. So I digged a bit into the code. I first thought the problem may be that my monitoring user cannot read cn=config, but it can. I originally used version 2.0.4, but I also checked the current SVN version; it still has the same (and possibly other) problems. In your function openldap_setBackEndDatabases you are querying for objectclass=olcBdbConfig. That means you will be missing several databases, because you are only asking for BDB databases. In our example, we are using HDB databases. So changing this to objectclass=olcHdbConfig made our replication agreements visible all of a sudden. But I guess this is not how to do it as there may be many other backend types. You probably should be searching for olcDatabaseConfig instead, which is the common superclass of all DB types. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: syncrepl missing entries in openldap 2.4.23
james_whitea...@mcafee.com wrote: So does OpenLDAP have another mechanism to do this? Combine two ldap providers into a single consumer that is? I've already answered that. You need to read more carefully. Jim On Mar 4, 2011, at 4:31 PM, Howard Chu wrote: james_whitea...@mcafee.com wrote: I am trying to set up syncrepl to have multiple providers to a single consumer. Basically allowing me to combine two ldap's into a single ldap. I know this is probably not a standard configuration but seems like it should work. This seems to work for a while but then all of the records from one of the providers is deleted. And even though the consumer still is polling both providers the records will will not get added back. No, this setup will always fail in the manner you describe. The way a syncrepl refresh works by default is that the provider tells the consumer about every entry it knows about within the search context. The consumer then deletes everything on its side that the provider didn't enumerate. Since both of your consumers are using the identical search base, every time one of them refreshes it will always delete everything the other one retrieved. (This is the normal operation of a syncrepl refresh Present phase. Read RFC4533 for the detailed explanation.) It's possible to get this working, somewhat, using delta-syncrepl, which usually does not use a Present phase. However, if the consumer ever lags behind the provider's log (i.e., the consumer's state is older than the oldest entry in the provider's log) then delta-syncrepl falls back to normal syncrepl, and you'll hit the refresh Present phase again. So in general, what you're trying to do is unsupported. Here is my consumer syncrepl configuration. The providers are a standard provider configuration. Any help would be appreciated. Jim serverID 064 databasebdb suffix o=dogcatfish rootdn cn=admin,o=dogcatfish limits dn.exact=cn=admin,o=dogcatfish size=unlimited time=unlimited # Cleartext passwords, especially for the rootdn, should # be avoid. See slappasswd(8) and slapd.conf(5) for details. # Use of strong authentication encouraged. rootpw admin # syncrepl configuration syncrepl rid=64 provider=ldap://provider1 type=refreshOnly interval=00:00:01:00 retry=60 10 300 + searchbase=o=dogcatfish filter=(objectClass=*) scope=sub attrs=*,+ schemachecking=off bindmethod=simple binddn=cn=admin,o=dogcatfish credentials=admin # syncrepl configuration syncrepl rid=68 provider=ldap://provider2 type=refreshOnly interval=00:00:01:00 retry=60 10 300 + searchbase=o=dogcatfish filter=(objectClass=*) scope=sub attrs=*,+ schemachecking=off bindmethod=simple binddn=cn=admin,o=dogcatfish credentials=admin # Indices to maintain index contextCSN,entryCSN,entryUUID,objectClass,cn,dc,mail eq checkpoint 1024 5 mirrormode TRUE -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: [Solved] Poor performance on Solaris
juergen.spren...@swisscom.com wrote: Hi, first I wish to thank all those who supplied helpful hints to solve the problem, especially Quanah Gibson-Mount and Howard Chu. My performance issue was solved by switching from memory mapped keys to shared memory keys for hdb as suggested by Quanah and Howard. Putting 'shm_key 10' into slapd.conf and restart of slapd solved the problem. Performance improvement was about factor 10 on the Solaris box. As the problem was gone connection logging was switched off, which additionally doubled throughput as Solaris sylog can't do asynchronous logging. During further tests slapd stopped responding when many concurrent connections were active. This behaviour was caused by default settings in /etc/system. Adding two lines to /etc/system and reboot solved the problem: set rlim_fd_max = 16368 set rlim_fd_cur = 8192 Should be enough for about 8000 connections. Performance before: box1: hardware: Sun Microsystems sun4v SPARC Enterprise T5120 memory:32 GB RAM os: Solaris 10 s10s_u9wos_14a searches (avg/second): 1521 Performance after: box1: hardware: Sun Microsystems sun4v SPARC Enterprise T5120 memory:32 GB RAM os: Solaris 10 s10s_u9wos_14a searches (avg/second): 37000 What is the CPU utilization that you see at this point? The T5120 supposedly has 64 hardware threads, but in my tests on this architecture, we never got better than 50% utilization (a couple years ago). I'm curious if you can make use of the multiple listener feature added in 2.4.24. I no longer have access to any machines of this architecture to test for myself. Also are you using the on-chip ethernet interface? My next step will be tuning network parameters. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LDAP browsers and cn=config
Gervase Markham wrote: On 07/03/11 17:49, Gervase Markham wrote: oldRootDN: cn=admin,cn=config ^ And that would be the problem :-| Thank you for your help.shuffles feet in an embarrassed fashion cn=config is an LDAP database, it is not a collection of files for you to edit by hand. You are supposed to use ldapmodify on it, for reasons of this very nature. I.e., ldapmodify gets syntax-checked and stupid typos of this sort get caught. If you had used ldapmodify -H ldapi:/// -Y EXTERNAL to add the desired attributes you wouldn't have these silly problems. If your LDAP browsers don't support ldapi:/// that's their deficiency... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Another thought on backends
Michael Smith wrote: Anybody interested in collaborating on a back-python analogous to back-perl? After a cursory glance at the code for the latter, it looks like it wouldn't be hard to adapt for the Python case. Go ahead if you like. Though it may be simpler to just use back-sock. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LDAP browsers and cn=config
Gervase Markham wrote: Can you tell me which LDAP browsers do support this scheme? After all, the other part of my message was asking for advice on which was best. Ironically, I used to use the Netscape browser as my preferred LDAP browser, many years ago. Don't recall when they axed that feature, it was quite handy, and getting the results pretty-printed in HTML was really nice. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: [Solved] Poor performance on Solaris
juergen.spren...@swisscom.com wrote: Hi Howard, On-chip ethernet interfaces are currently not used for slapd, They are currently used for management access and netbackup. Zones running slapd use nxge type ethernet interfaces. Did some quick tests. Increasing threads to 64 caused higher cpu utilization but also a dramatic decrease in performance on OpenLDAP releases 2.4.2[34]. That could not be compensated by increasing listener_threads. Have tested values default, 4 and 8 for listener threads. Thanks for taking the time to try this out. Yes, increasing the number of threads beyond a certain threshold typically causes a performance decrease due to inefficiencies in the system's scheduler. This is why we use a relatively low number of threads by default. When I set up tests for maximum performance, I usually have to bind individual slapd threads to specific CPU cores to avoid this problem. (Look into the pbind or psrset commands on Solaris.) Staying with 32 threads and setting listener_threads to 4 Increased searches/s from 37000 to 48000. More listener_threads had no significant impact. 30% gain, not bad. CPU utilization by slapd was at about 65%. If you needed to, I believe you can coax a fair bit more out of this using pbind. You generally can't get slapd above 90% utilization because the network device driver will consume the rest. (Which is why I was curious about whether the on-chip ethernet is more efficient.) I.e., on an optimally tuned setup, the network will be the performance limiter, not slapd. Juergen Sprenger -Original Message- From: Howard Chu [mailto:h...@symas.com] Sent: Monday, March 07, 2011 10:25 PM To: Sprenger Jürgen, ITS-SDL-SO-WXS-USO-BE1 Cc: openldap-technical@openldap.org Subject: Re: [Solved] Poor performance on Solaris juergen.spren...@swisscom.com wrote: Hi, first I wish to thank all those who supplied helpful hints to solve the problem, especially Quanah Gibson-Mount and Howard Chu. My performance issue was solved by switching from memory mapped keys to shared memory keys for hdb as suggested by Quanah and Howard. Putting 'shm_key 10' into slapd.conf and restart of slapd solved the problem. Performance improvement was about factor 10 on the Solaris box. As the problem was gone connection logging was switched off, which additionally doubled throughput as Solaris sylog can't do asynchronous logging. During further tests slapd stopped responding when many concurrent connections were active. This behaviour was caused by default settings in /etc/system. Adding two lines to /etc/system and reboot solved the problem: set rlim_fd_max = 16368 set rlim_fd_cur = 8192 Should be enough for about 8000 connections. Performance before: box1: hardware: Sun Microsystems sun4v SPARC Enterprise T5120 memory:32 GB RAM os: Solaris 10 s10s_u9wos_14a searches (avg/second): 1521 Performance after: box1: hardware: Sun Microsystems sun4v SPARC Enterprise T5120 memory:32 GB RAM os: Solaris 10 s10s_u9wos_14a searches (avg/second): 37000 What is the CPU utilization that you see at this point? The T5120 supposedly has 64 hardware threads, but in my tests on this architecture, we never got better than 50% utilization (a couple years ago). I'm curious if you can make use of the multiple listener feature added in 2.4.24. I no longer have access to any machines of this architecture to test for myself. Also are you using the on-chip ethernet interface? My next step will be tuning network parameters. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: syncrepl with with multiple subordinate databases
Dieter Kluenter wrote: Hi, I am facing a severe problem with the replication of subordinate databases and a log database. The initial replication of the subordinate databases is successful but afterwards slapd crashes, A backtrace and a momory map can be found here A crash should be submitted as a bug report to the ITS, not to a discussion mailing list. http://pastebin.de/15919 Your trace says there was an invalid free(); running under valgrind would probably give more information. the provider slapd.conf can be found here http://pastebin.de/15920 You should not be using syncprov-reloadhint on anything except for the actual cn=log database. the consumer slapd.conf can be found here http://pastebin.de/15922 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Null Search Base
Andrew Findlay wrote: On Wed, Mar 09, 2011 at 04:34:16PM -0700, l...@mm.st wrote: A security scanner was run against our ldap severs and came back with a warning stating The remote LDAP server supports search requests with a null, or empty, base object. This allows information to be retrieved without any prior knowledge of the directory structure. Coupled with a NULL BIND, an anonymous user may be able to query your LDAP server using a tool . . . I assume the warning is due to the namingContext attribute and if desired an acl could be setup to stop the retrival on the information. That seems very likely, and as you say an ACL could be used to prevent it. In this context the 'empty base object' refers to the Root DSE, and it contains information that some LDAP client programs depend on. Blocking access to it would almost certainly cause trouble for those clients. It is very unlikely that the list of naming contexts and supported LDAP extensions is in any sense secret, so don't let some auditor bully you into breaking your system just to fit some tick-box notion of security. The important stuff comes further down in the DIT, and you need a tool specific to your organisational policy to point out exposures there. When the tool doesn't even call the object by its proper name (Root DSE) it's a sure sign the tool authors have no idea what they're talking about. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldapsearch and extended server controls?
Brian Reichert wrote: I have a case where I'm trying to replicate some other application's search against Active Directory. A deconstruction of a packet capture of that app's conversation shows it's making use of a control that I don't know how to specify using any combination of ldapsearch(1) or ldap.conf(5). Specifically, I see this coming over the wire as a control on the search: http://msdn.microsoft.com/en-us/library/aa366989%28v=vs.85%29.aspx 1.2.840.113556.1.4.417 LDAP_SERVER_SHOW_DELETED_OID Research shows there lots these sorts of extended control; yay, I learned something new! Is there a way of utilizing these sorts of controls via ldeapsearch? Yes. Use -e 1.2.840.113556.1.4.417 Thanks for any advice you may have... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Dynamic modules unloading
Коновалов Андрей Александрович wrote: Hello! I have OpenLDAP 2.4.24 built with -enable-dynamic and --enable-modules options So, dynamic modules support is on and my question is: Why can i load module (for example: dn: cn=module{0},cn=config changeType: modify add: olcModuleLoad olcModuleLoad: translucent.la ) But when i'm trying to unload it (because it's not used at all), i cant do this anywhere? dn: cn=module{0},cn=config changeType: modify delete: olcModuleLoad olcModuleLoad: {6}translucent.la # the same with simply olcModuleLoad: translucent.la modifying entry cn=module{0},cn=config ldap_modify: Other (e.g., implementation specific) error (80) additional info: cannot delete olcModuleLoad If it is not realised yet, you may consume my question as a feature request :) Many reasons. First because there simply hasn't been a need. Second because many platforms that OpenLDAP supports don't actually support unloading of dynamic modules. (E.g., on AIX 3, unloading a module is supported but doesn't in fact free up any of the memory, that part of the address space is gone. Unloading and reloading over and over eventually eats up all the address space.) There are some isolated cases where it might be useful, but they're relatively rare. E.g., unloading a module so that a one-line patch can be applied, and then reloading it. In the more common case, since the slapd APIs are not frozen, you can't rely on unloading and mixing versions of modules. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: delta-sync - ContextCSN on proivder older than consumers
Dieter Kluenter wrote: Am Sun, 13 Mar 2011 17:39:17 -0700 schrieb Yuri Bankyurib...@gmail.com: After doing more testing I have noticed that it is the 'Group member modify entryCSNs' that seem to get ignored by the Provider, but picked up by the Consumers. All other changes, adding or removing users seems to update the ContextCSN on the Provider correctly. So a work around would be to make some kind of random change to an entry in my DIT ( after making changes to group membership), so that the Provider has the correct ContextCSN. A simple change like modifying the description field for a user would accomplish this. I would like to get to the bottom of this though, without such a work around. Could this have anything to do with the memberOf overlay, which I am using? It is more likely that the contextCSN of accesslog db is older than the last contextCSN provided by the provider. It's too unclear to make such an assumption. Best bet still is to switch to 2.4.24 and see if the problem remains. And yes, it's most likely related to the memberOf overlay. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: delta-sync - ContextCSN on proivder older than consumers
Dieter Kluenter wrote: Am Mon, 14 Mar 2011 02:43:53 -0700 schrieb Howard Chuh...@symas.com: Dieter Kluenter wrote: Am Sun, 13 Mar 2011 17:39:17 -0700 schrieb Yuri Bankyurib...@gmail.com: After doing more testing I have noticed that it is the 'Group member modify entryCSNs' that seem to get ignored by the Provider, but picked up by the Consumers. All other changes, adding or removing users seems to update the ContextCSN on the Provider correctly. So a work around would be to make some kind of random change to an entry in my DIT ( after making changes to group membership), so that the Provider has the correct ContextCSN. A simple change like modifying the description field for a user would accomplish this. I would like to get to the bottom of this though, without such a work around. Could this have anything to do with the memberOf overlay, which I am using? It is more likely that the contextCSN of accesslog db is older than the last contextCSN provided by the provider. It's too unclear to make such an assumption. It's based on experience, my test environment shows the same results. A modify to one of the subordinate databases results in error 53 trying to sync the other subordinate database. Comparing contextCSN of all four databases showed that the client presents a CSN that is younger than that of access db, but I am still investigating The original poster has no subordinate databases. Whatever behavior you're experiencing is totally unrelated, and you're making unfounded assumptions that there is any relevance between your situation and his. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Postgres perhaps - was Re: Another thought on backends
Richard Troy wrote: On Mon, 14 Mar 2011, Hallvard B Furuseth wrote: Michael Smith writes: Anybody interested in collaborating on a back-python analogous to back-perl? After a cursory glance at the code for the latter, it looks like it wouldn't be hard to adapt for the Python case. I started on that once, but quit because Python and slapd both want their conflicting autoconf-generated files #included: OpenLDAP's portable.h vs. Python's pyconfig.h. back-perl escapes that: Perl uses metaconfig instead of autoconf. I have no interest with what you are proposing but I am very interested in porting OpenLDAP to Postgres. The compelling use case is that now, with the advent of binary replication in Postgres 9, one can replicate your Posgres installation - which you may be doing for any number of other reasons - and have that replication carry with it the OpenLDAP dataset. So, it reduces the systems management / maintenance overhead while increasing security (one DB system instead of two). Also, the Postgres replication is _very_ efficient and has no inherent limit on the number of replicants, each of which can be used for query-only, if desired. It's my understanding that someone has already done this - ported OpenLDAP to Postgres, and there's a web page describing it out there somewhere. I have been looking for some free time (yeah, like _that's_ going to happen any time soon!) to do the port. And since I haven't quite gotten around to it, I'm wondering if there's general interest? Anyone? Are you saying there's a low level Postgres-specific API that can be leveraged? If you're just going to use SQL or something euqivalent, I don't see any advantage to that. Regards, Richard -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How to turn on LDAP_DEVEL flag?
Коновалов Андрей Александрович wrote: I want to test component matching support in OpenLDAP 2.4.24, but it is masked by LDAP_DEVEL flag, which is turned on only when LDAP_VENDOR_VERSION=0. So... could you tell me please, how to turn on LDAP_DEVEL directly in a proper way? It is masked for a reason - it is work in progress and not suitable for use. Since you are asking this question, you are apparently not a qualified developer, therefore you should not touch it. It won't work. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Undefined reference to ber_* with 2.4.24
frank.offerma...@caseris.de wrote: Hello, I am able to compile OpenLDAP 2.4.23 with Berkeley DB 4.8.30 with MSYS and windows with configure params: configure --prefix=/mingw --enable-acceslog --with-tls But if I try to compile OpenLDAP 2.4.24 I get several undefined references to ber_* functions when liblutil is build. Did I miss a change in the way to build OpenLDAP 2.4.24? Seems to be a bug in the Makefile introduced with the new libldif. Please file an ITS for this. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP reports busy and hangs
Frank Budde wrote: Hi, with OpenLDAP 2.4.22 on Solaris 10 we discovered following situation: from time to time the slapd log file contains message daemon: select: listen=7 busy while ldap continues to operate normally. Sounds similar to this http://www.openldap.org/its/index.cgi/Incoming?id=6833 After 7 days of operation suddenly ldap does not respond any more and keeps printing daemon: select: listen=7 busy. Log file shows: daemon: select: listen=7 busy daemon: listen=7, new connection on 30 daemon: activity on 1 descriptor daemon: waked daemon: added 30r (active) listener=0 conn=5656 fd=30 ACCEPT from IP=192.168.200.131:40993 (IP=192.168.200.16:9929) daemon: select: listen=7 active_threads=1 tvp=zero daemon: activity on 2 descriptors daemon: waked daemon: select: listen=7 active_threads=1 tvp=zero daemon: activity on 1 descriptor daemon: activity on: 30r daemon: read activity on 30 daemon: select: listen=7 active_threads=1 tvp=zero daemon: activity on 1 descriptor daemon: waked conn=5656 op=0 BIND dn=cn=ldapadmin,dc=abc,dc=com method=128 daemon: select: listen=7 active_threads=1 tvp=zero conn=5656 op=0 BIND dn=cn=ldapadmin,dc=abc,dc=com mech=SIMPLE ssf=0 conn=5656 op=0 RESULT tag=97 err=0 text= daemon: activity on 1 descriptor daemon: activity on: 30r daemon: read activity on 30 daemon: select: listen=7 active_threads=1 tvp=zero daemon: activity on 1 descriptor daemon: waked daemon: select: listen=7 active_threads=1 tvp=zero conn=5656 op=1 SRCH base=ou=users,dc=abc,dc=com scope=1 deref=3 filter=(uid=abcservice-prd) daemon: activity on 1 descriptor daemon: select: listen=7 busy daemon: select: listen=7 busy daemon: select: listen=7 busy daemon: select: listen=7 busy slapd.conf contains gentlehup on idletimeout 15 conn_max_pending 10 conn_max_pending_auth 30 My question: How can the problem be solved or reproduced? The problem comes up in production environment and I need to reproduce it in test environment. How can this be done? (If the problem can be reproduced with 2.4.22 I will try it with ldap 2.4.24.) Thanks for any help, /Frank -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How to turn on LDAP_DEVEL flag?
Dieter Kluenter wrote: Am Tue, 15 Mar 2011 15:22:55 +0300 schrieb Коновалов Андрей Александровичmudr...@list.ru: I want to test component matching support in OpenLDAP 2.4.24, but it is masked by LDAP_DEVEL flag, which is turned on only when LDAP_VENDOR_VERSION=0. So... could you tell me please, how to turn on LDAP_DEVEL directly in a proper way? When compiling openldap use CFLAGS=-DLDAP_DEVEL -DLDAP_COMP_MATCH No. This code is guaranteed to crash. It is irresponsible of you to give this advice. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP Memory Usage
Diego Lima wrote: Hello all, I'm experiencing some problems with some OpenLDAP servers: the slapd process seems to always use more memory, eventually reaching a point where it has consumed all the available server memory and is killed by the OOM killer. The servers have 32gb of memory plus 32gb of swap space and are running Debian Lenny (with kernel 2.6.26-2-amd64), and we have compiled OpenLDAP 2.4.23 from source, and we're using Berkeley DB 4.6. The servers are dedicated to running OpenLDAP, so they don't have other processes that use a significant amount of memory. This is a relatively high-volume environment, with 4 servers running with mirrormode to enable multi-master replication. The current database size is about 900mb. Is there any setting that could limit this memory usage? I don't feel this is normal considering our DB size and cache sizes. Very likely ITS#6660, you need to update to 2.4.24. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: DSEE to OpenLDAP
Bill MacAllister wrote: I'm not sure that replication between the two is an option (although would love it if it was :), so I have looked into exporting the current DSEE environment to a LDIF, and attempted to then import it into openldap (using slapadd), but ran into a few issues... The issue I'm currently stuck on is getting the data in the LDIF into a format that can be imported using slapadd. Currently, I have issues with automounts, pwdReset attribs etc etc... Why can't you just use ldapsearch on the existing directory and use that as input to slapadd? slapadd is meant for well-formed LDIF input, mainly from slapcat. DSEE is known to not support schema checking of any kind, therefore any LDIF you obtain from it will most likely be full of garbage and not well-formed. As such, you should only use ldapadd to import it, so you get full error checking on the import. Of course, that only gets you the data. I would bet that the real issue will be getting the access controls correct. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP Memory Usage
Tim Mooney wrote: In regard to: Re: OpenLDAP Memory Usage, Quanah Gibson-Mount said (at...: I suggest reading over: http://www.openldap.org/pub/hyc/scale2007.pdf Thanks Quanah, that was very helpful. Slides 23 and 33-35 (comparing the various memory allocators) were very useful, especially when coupled with the openldap-devel threads starting on August 24 2006. I also found slide 26, about the problem with trying to benchmark OpenLDAP using slow bench mark tools, interesting. What tools are recommended these days, for benchmarking an OpenLDAP installation? The most promising ones I could find were SLAMD (which was panned in slide 26) and cbe4. Are there others that are noteworthy? SLAMD is still the most convenient to use, in terms of overall features. I just make sure that I have several times as much horsepower in the clients as in the target server, to generate sufficient load on the server. (This can make benching very powerful servers rather difficult. Sometimes I will also resort to simple timing of the OpenLDAP test clients when the client machines are otherwise too underpowered for SLAMD. I try to avoid that because then I'm stuck having to tabulate the results manually.) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: syncrepl and mass deletes
LALOT Dominique wrote: Hello, I am testing the replication feature in a multimaster environment replicating into a single database. As stated before, I added serverid to my providers. I just have two providers for test purpose. I tested mass updates on a provider, stopped my replica during updates, then start again and it's OK, it updates the entries If I do the same for mass deletes. I deleted 4 entries while stopping the consumer. My consumer is still with 3 undeleted entries. I left the consumer for hours, restarting it twice. It seems there is no regular compare between consumer or provider in such situation. I'll simplify to test in a single provider setup, to see if it works. All servers are 2.4.23 Please try your test with 2.4.24 instead. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: syncrepl and mass deletes
LALOT Dominique wrote: Howard, I was obliged to remove slapd package on consumer. Then compile in 2.4.24 and restart. doing the same tests, there was nothing diffferent. provider is still 2.4.23 My test: delete 3 entries, stop consumer when deleting; start again consumer when it's finished. Consumer is then out of sync. You haven't posted your provider config. It appears you're using syncprov-sessionlog. Obviously both ends of this system are vital to solving the puzzle. Mar 23 17:27:50 ldaprelay slapd[4358]: slapd stopped. Mar 23 17:34:45 ldaprelay slapd[4577]: @(#) $OpenLDAP: slapd 2.4.24 (Mar 23 2011 16:52:04) $#012#011r...@ldaprelay.univ-amu.fr:/usr/local/src/openldap-2.4.24/servers/slapd Mar 23 17:34:45 ldaprelay slapd[4579]: slapd starting Mar 23 17:34:45 ldaprelay slapd[4579]: do_syncrep2: rid=040 LDAP_RES_INTERMEDIATE - REFRESH_DELETE Mar 23 17:34:45 ldaprelay slapd[4579]: do_syncrep2: rid=020 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Mar 23 17:34:48 ldaprelay slapd[4579]: last message repeated 175 times Mar 23 17:34:48 ldaprelay slapd[4579]: do_syncrep2: rid=020 LDAP_RES_INTERMEDIATE - REFRESH_PRESENT Mar 23 17:34:48 ldaprelay slapd[4579]: do_syncrep2: rid=020 cookie=rid=020,sid=020,csn=20110323163416.105518Z#00#020#00 Mar 23 17:34:48 ldaprelay slapd[4579]: slap_queue_csn: queing 0x1a35560 20110323163416.105518Z#00#020#00 Mar 23 17:34:48 ldaprelay slapd[4579]: slap_graduate_commit_csn: removing 0x1a35620 20110323163416.105518Z#00#020#00 I changed the provider to 2.4.24 that makes deletes. Hopefully this one was built on tar.gz That makes no difference, after restarting the consumer, it does not delete extra entries Dom 2011/3/23 LALOT Dominique dom.la...@gmail.com mailto:dom.la...@gmail.com Hi Howard, We were told to migrate to 2.4.23 sometimes ago, and we did some work to update our production servers. Can I try 2.4.24 only on the consumer side? It would be a pain to migrate all servers to 2.4.24 without package. is this related to the last fixes? Fixed slapd syncrepl reuse of presence list (ITS#6707) Fixed slapd syncrepl uninitialized return code (ITS#6719) Fixed slapd syncrepl variable initialization (ITS#6739) Fixed slapd syncrepl refresh to use complete cookie (ITS#6807) Thanks Dom 2011/3/23 Howard Chu h...@symas.com mailto:h...@symas.com LALOT Dominique wrote: Hello, I am testing the replication feature in a multimaster environment replicating into a single database. As stated before, I added serverid to my providers. I just have two providers for test purpose. I tested mass updates on a provider, stopped my replica during updates, then start again and it's OK, it updates the entries If I do the same for mass deletes. I deleted 4 entries while stopping the consumer. My consumer is still with 3 undeleted entries. I left the consumer for hours, restarting it twice. It seems there is no regular compare between consumer or provider in such situation. I'll simplify to test in a single provider setup, to see if it works. All servers are 2.4.23 Please try your test with 2.4.24 instead. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: 8 principal limitation in openldap
Srivatsav M wrote: Hi, We are using OpenLDAP for authenticating users registered in a LDAP server (Open LDAP, Active Directory). After adding 8 principals (/etc/ldap.conf), none of the users registered in the /etc/ldap.conf file are able to login. nss_base_passwd OU=engg,DC=mycompany,DC=region,DC=someplace,DC=myarea,DC=compname,DC=parentcompname nss_base_shadow OU=engg,DC=mycompany,DC=region,DC=someplace,DC=myarea,DC=compname,DC=parentcompname nss_base_group OU=engg,DC=mycompany,DC=region,DC=someplace,DC=myarea,DC=compname,DC=parentcompname Can you please share the reason for this 7 limitation in the open ldap library. or how I can fix this issue. I am looking i for the header file in the source files whhich has this constant or limitation defined. I think you mean RDN not principal. And there is no such limitation in any OpenLDAP code. You're talking about nss-ldap; you should ask your question on their mailing list. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Large dynamic groups and performance
Quanah Gibson-Mount wrote: --On Monday, April 11, 2011 1:55 PM -0400 Alafrunn...@gmail.com wrote: I added those as well, no noticeable change in performance: cachesize 50 idlcachesize 50 checkpoint 1 15 In a heavy write environment, I've found that smaller more frequent checkpoints are better. Yours is set rather high. Also, you are definitely not using dynamic groups in the OpenLDAP sense of the word, although they would probably perform significantly better for you. Yes, I understand that - thats why I put it in quotes. I looked into using OpenLDAP dynamic lists, but I think I'm limited by the fact that some of our systems requiring these groups need to do searchs off of it based on the dynamic membership (and from what I can tell, its not possible to use it that way), ie they need to search for (uniquemember=cn=,cn=users,...) on my group section of the tree. I'll openly admit some of the values I have been picking for caching and checkpoint are somewhat arbitrary. I've been trying many different values and have yet to settle on any that work well. I'll gladly try any recommendations. Make sure you read over the dynlist overlay, I think you can do what you want. A group with 50,000+ members is probably going to take a long time to update. So OpenLDAP dynamic groups may well do better for you. The dynlist overlay does not support searches of the form (member=foo). While updates of such large groups will involve a lot of I/O (and thus will always be slow) you can still improve things a little using the sortvals config option. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: clarifications on cachesize, preferred db, et. al. from admin guide
Quanah Gibson-Mount wrote: --On Tuesday, April 12, 2011 10:42 AM -0500 Tim Mooney tim.moo...@ndsu.edu wrote: Check against OpenLDAP 2.4.25. I believe nearly all backends and overlays support slapd-config now. I'm using 2.4.25, but short of trying every one of the official and contrib overlays one by one, I don't know of any way to find out which ones don't support slapd-config. That's why I was asking: I was hoping someone knew what the holdouts were, or knew where it was documented. I meant more, use the source. From what I can tell, overlays with no cn=config support are: deref.c seqmod.c They also don't have manual pages, so probably aren't exactly supported anyway. They have no config settings at all, therefore they work regardless of static or dynamic config. At this point it's more useful to ask which don't work with dynamic config than which support dynamic config. There are several overlays that need no configuration. Between that and the -devel note about backends, I think you can create a complete list. - Admin Guide, chapter 21. The tuning chapter doesn't mention the potential benefits of using sysv shared memory vs. mmap'ed files on some platforms. Should it? Same offer for documentation patch applies, though I expect this one will need more feedback from the experts. Do you mean the shared memory keys, or something else? The shared memory keys. Definitely worth noting. It is a requirement for Solaris if you want any sort of decent performance. Apparently it's become more important on recent Linux kernels too. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: cn=config replication to consumer / slave servers
Quanah Gibson-Mount wrote: --On Thursday, April 14, 2011 5:28 PM -0700 Christopher Strider Cook cc...@pandora.com wrote: Alternately, I tried to setup a separate database cn=config_slave and have that snycrepl to the slave into cn=config... but that creates a naming missmatch. Is there an approved practice to achieve this, or some other pointers on avenues to explore? Use slapo-rwm to rewrite the cn=config_slave database to be cn=config on the replicas. No, that never worked well. Use suffixmassage in the syncrepl config statement. (Added in 2.4.24) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: cn=config replication to consumer / slave servers
Christopher Strider Cook wrote: On 4/15/11 11:46 AM, Howard Chu wrote: Quanah Gibson-Mount wrote: --On Thursday, April 14, 2011 5:28 PM -0700 Christopher Strider Cook cc...@pandora.com wrote: Alternately, I tried to setup a separate database cn=config_slave and have that snycrepl to the slave into cn=config... but that creates a naming missmatch. Is there an approved practice to achieve this, or some other pointers on avenues to explore? Use slapo-rwm to rewrite the cn=config_slave database to be cn=config on the replicas. No, that never worked well. Use suffixmassage in the syncrepl config statement. (Added in 2.4.24) Yes that looks like the right path.. I've just recompiled to gain this superpower... But: So I've got a 'stub' cn=config setup with enough (I think) to get it booted and connecting to the master to syncrepl over the rest of the config. olcDatabase\=\{0\}config.ldif has I think a proper syncrepl line: olcSyncrepl: {0}rid=001 provider=ldaps://hostxxx.com bindmethod=simp le binddn=cn=admin,cn=config_slave credentials=pass searchbase= cn=config_slave suffixmassage=cn=config schemachecking=on type=refreshAndPersist retry=60 + Please read the test059 script in the test suite, it already does all the necessary steps in the correct order. Something is still not quite right. In the logs I see a few troubling messages and the local cn=config doesn't update. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: cn=config replication to consumer / slave servers
Christopher Strider Cook wrote: So, the pointer to test059 was exactly what this issue needed and following it has lead me to an very good working setup with one puzzling final step. The problem I now face is that the initial cn=config entries used to do the first sync do not get overwritten by the data from the master. So the install password doesn't get replaced nor do the updated retry timeouts for olcSyncRepl, because, I'm assuming, the 'stub' entries have newer timestamps than those on the master. How can this be overcome from the perspective of the slave server. Updating the entries on the master triggers the update as you would expect. Is there a way to put the stub entries onto the slave with a timestamp in the past so that they get overwritten during the first sync? Or is there another way to trigger them to be updated? Use slapd -c. Read the slapd(8) manpage. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/