Re: trouble when switching from bdb to mdb database
Aleksander Dzierżanowski wrote: Hi. I have properly runnig setup of three multimaster OpenLDAP servers (version 2.4.36 from ltb project) with bdb database backend. Everything was working flawless so I decided to try out ‚new shiny' mdb database with the same configuration - the only thing I changed was removing ‚cache’ settings and adding ‚maxsize’. What I’m doing and observing: - clear all config and database on all masters. Generate new configuration from slapd.conf using ‚slaptest’ tool. - on master1 I add three base organizations let’s say o=test1 + o=test2 + o=test3 using slapadd [without -w switch] - on master1 I add some entries using ldapadd command so all organizations have now contextCSN attribute. - starting master1 - everything OK - starting master2 - everything OK, including succesfull replication from master1 - starting master3 - everything OK and including replication, but… some or all other master are dying unexpectedly. strace of dying process show: --- write(2, slapd: id2entry.c:509: mdb_opinfo_get: Assertion `!rc' failed.\n, 63) = 63 This is ITS#7662. It was fixed a while ago but the fix was accidentally omitted from releases. Fix will be in 2.4.38. — debug log last lines: — = mdb_entry_get: ndn: „o=test1” = mdb_entry_get: oc: (null), at: „contextCSN — But when I do ‚slapcat’ I can clearly see contextCSN for all o=test[123] databases... Is it bug or some possible replication configuration issue? — Olo -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: RE24 testing call (OpenLDAP 2.4.37)
Sergio NNX wrote: Why? The current testing call is for 2.4.38. Sorry, I'm always behind! Anyway, could you have a look at my feedback? In the meantime, I'll start building and testing OpenLDAP 2.4.38! For what it's worth, I've seen similar issues with current MSYS. Unfortunately it seems a number of their tools are broken compared to older MSYS releases. I haven't had the time to isolate all of the problems yet, may take a closer look after LDAPCon next week. (Reminder - LDAPCon in Paris next week: http://ldapcon.org/ ) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Error while updating olcDbConfig
Daniel Jung wrote:
Hi wonderful ppl at openldap-
Running 2.4.37 - on centos 6.4
using following ldif to update olcDbConfig
dn: olcDatabase={2}hdb,cn=config
changetype: modify
replace: olcDbConfig
olcDbConfig: set_lg_max 10485760
olcDbConfig: set_lg_regionmax 262144
olcDbConfig: set_lg_bsize 2097152
olcDbConfig: set_lg_dir /var/lib/ldap/
olcDbConfig: set_lk_max_locks 8192
olcDbConfig: set_lk_max_objects 8192
olcDbConfig: set_flags DB_LOG_AUTOREMOVE
olcDbConfig: set_tas_spins 1
-
ldapmodifyx -d 1 -h localhost -D 'cn=Manager,cn=config' -W -f /var/tmp/ldif
Getting the following error
res_errno: 80, res_error: failed to reopen database, rc=22, res_matched:
ldap_free_request (origid 2, msgid 2)
ldap_parse_result
ber_scanf fmt ({iAA) ber:
ber_scanf fmt (}) ber:
ldap_msgfree
ldap_err2string
ldap_modify: Other (e.g., implementation specific) error (80)
additional info: failed to reopen database, rc=22
Going through the list lead me to believe that changes will be applied after
the restart? but didnt say it causes an error. Also does olcDbNoSync play a
role here ?
The change is applied immediately, there should not be any restart required.
Unfortunately, it seems like the changes you applied were invalid for the
version of BerkeleyDB you're using. Your LDIF worked perfectly fine for me,
using BDB 5.3.21.
Another possibility is that the directory you specified for set_lg_dir was
invalid, didn't exist, or you didn't have permissions to write it. One further
possibility is that you were changing this value - in that case, you need to
make sure to manually move all of the BDB logfiles from the old location to
the new location first, otherwise BDB won't be able to find them and it will
think the BDB environment is corrupted.
Using the -d debug flag on the client doesn't tell you anything useful though,
you need to use -d on slapd to see what the actual error was.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Openldap for proxy AD
Clément OUDOT wrote: 2013/11/19 Jason Brandt jbra...@fsmail.bradley.edu: You are trying to authenticate through the credentials stored in your active directory servers, not the passwords stored in LDAP, correct? If that is the case, then the easiest means to accomplish that are to use SASL for authentication. Or he could just read up on slapo-pbind. You can check this how-to: http://ltb-project.org/wiki/documentation/general/sasl_delegation Clément. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: password hashes and simple binds
Dieter Klünter wrote:
Am Sat, 23 Nov 2013 13:24:56 +0100
schrieb Michael Ströder mich...@stroeder.com:
Dieter Klünter wrote:
Hi,
I have a ldap server (2.4.36) with various password hashes
{CLEARTEXT} {KERBEROS} {SSHA} for different users, there is no
pasword-hash declaration in slapd.conf. Now i face a strange
behaviour with {CLEARTEXT} hash. that is:
userPassword: {CLEARTEXT} secret
^^^
I'd try to remove this extra space. Not sure though.
Just to demonstrate the various hash scheme {CLEARTEXT} results:
http://pastebin.de/37485
-Dieter
CLEARTEXT is not an actual hash scheme. It's only handled by the SASL code.
For Simple Binds, a cleartext password must not have any scheme specifier at all.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Password Issues between provider and consumer
Choure, Sidd wrote: Here is the version info - slapd -V @(#) $OpenLDAP: slapd 2.4.23 (Apr 22 2013 05:03:41) $ mockbu...@x86-007.build.bos.redhat.com:/builddir/build/BUILD/openldap-2.4. 23/openldap-2.4.23/build-servers/servers/slapd. There are few practical implementation documents for the new cn=config version of openldap and the ones available all use the bdb database. Examples use whatever was convenient for the document author at the time. They're obviously not the only choice, nor are they necessarily the best or recommended choice. Learn to think for yourself instead of blindly copy/pasting other peoples' examples. The back-bdb code is not particularly *old*, contrary to what Quanah implied. But BerkeleyDB is definitely *obsolete*, and with the license change in 6.0.20, it is no longer legal for use in open source LDAP servers. (But given its obsolescence, the license issue is somewhat irrelevant.) What can I do about this password issue? Am I missing come ACL or option that needs to be added? Siddharth Choure Senior Systems Engineer -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Password Issues between provider and consumer
Choure, Sidd wrote: So you don¹t have a solution? The Project only supports current releases, the current release is 2.4.38. Since you're running Red Hat's old build of 2.4.23, which was released over 3 years ago, you should contact them for support. Siddharth Choure Senior Systems Engineer On 11/25/13, 8:52 AM, Howard Chu h...@symas.com wrote: Choure, Sidd wrote: Here is the version info - slapd -V @(#) $OpenLDAP: slapd 2.4.23 (Apr 22 2013 05:03:41) $ mockbu...@x86-007.build.bos.redhat.com:/builddir/build/BUILD/openldap-2. 4. 23/openldap-2.4.23/build-servers/servers/slapd. There are few practical implementation documents for the new cn=config version of openldap and the ones available all use the bdb database. Examples use whatever was convenient for the document author at the time. They're obviously not the only choice, nor are they necessarily the best or recommended choice. Learn to think for yourself instead of blindly copy/pasting other peoples' examples. The back-bdb code is not particularly *old*, contrary to what Quanah implied. But BerkeleyDB is definitely *obsolete*, and with the license change in 6.0.20, it is no longer legal for use in open source LDAP servers. (But given its obsolescence, the license issue is somewhat irrelevant.) What can I do about this password issue? Am I missing come ACL or option that needs to be added? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP with ppolicy and SSSD configuration question.
Viviano, Brad wrote: Hello, I've searched the archives of this list, the web as best I can, and have this same question asked to the sssd-devel mailing list and can not seem to find an answer this my question. I have a RHEL 6.4 server with OpenLDAP 2.4.23-32.el6_4.1 and sssd 1.9.2-129.el6, both installed as standard RPM's from Redhat. I have ppolicy configured in slapd and on another RHEL6.4 system have sssd setup as a client. Everything works fine with password expires, grace periods, etc and sssd, if the user has to enter their password. But, if the user is using an SSH public key, setting the account as locked or the password is expired still allows them to log in. I can't seem to find a good solution that forces the user to change their password before they can login. Why would you expect anything to validate their password if they are using an SSH public key? pam only gets the ppolicy info if it performs an LDAP Bind with the user's password. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP with ppolicy and SSSD configuration question.
Viviano, Brad wrote: Howard, I'm not expecting it to validate their password, I am expecting it to check if their account is locked for some reason. If their account is locked in LDAP, it shouldn't let them login under any circumstances. For technical reasons we need ssh public keys to operate (IBM GPFS), but I don't want the user to be able to circumvent LDAP authority. If I lock their account in LDAP they shouldn't be able to login to any system, and I shouldn't have to go to every one of my systems and disable their SSH keys manually. You're missing the point. ppolicy is an abbreviation of *Password Policy* - if the user didn't supply a password, then the policy is irrelevant and cannot be applied. pwdAccountLockedTime is set when a user has too many failed login attempts using their password. It is not a generic account is disabled flag. If you want that, you need to define your own attribute for the purpose because there is no generic *Account Policy* spec for LDAP. (This is in fact one of the outstanding objections to the last ppolicy draft, which prevented it from moving forward as a standard RFC.) The ideal case would be that ppolicy has an attribute that lists if the account is locked or not. This would also be useful when using pwdLockoutDuration. If I'm using pwdLockoutDuration and pwdAccountLockedTime is set, I don't really know if the account is locked because I then have to do the math and take the pwdAccountLockedTime and add the value of pwdLockoutDuration for the policy applied to that user and see if their account is in fact locked. If ppolicy just provided a true/false in addtion to the LockedTime, that would be much more useful. Does anyone have a suggestions of a overlay that could create a derived attribute based on pwdAccountLockedTime so I could get a True/False value. Thanks, -Brad Viviano === Brad Viviano High Performance Computing Scientific Visualization Lockheed Martin, Supporting the EPA Research Triangle Park, NC 919-541-2696 HSCSS Task Order Lead - Ravi Nair 919-541-5467 - nair.r...@epa.gov High Performance Computing Subtask Lead - Durward Jones 919-541-5043 - jones.durw...@epa.gov Environmental Modeling and Visualization Lead - Heidi Paulsen 919-541-1834 - paulsen.he...@epa.gov From: Howard Chu h...@symas.com Sent: Monday, November 25, 2013 1:07 PM To: Viviano, Brad; openldap-technical@openldap.org Subject: Re: OpenLDAP with ppolicy and SSSD configuration question. Viviano, Brad wrote: Hello, I've searched the archives of this list, the web as best I can, and have this same question asked to the sssd-devel mailing list and can not seem to find an answer this my question. I have a RHEL 6.4 server with OpenLDAP 2.4.23-32.el6_4.1 and sssd 1.9.2-129.el6, both installed as standard RPM's from Redhat. I have ppolicy configured in slapd and on another RHEL6.4 system have sssd setup as a client. Everything works fine with password expires, grace periods, etc and sssd, if the user has to enter their password. But, if the user is using an SSH public key, setting the account as locked or the password is expired still allows them to log in. I can't seem to find a good solution that forces the user to change their password before they can login. Why would you expect anything to validate their password if they are using an SSH public key? pam only gets the ppolicy info if it performs an LDAP Bind with the user's password. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Different approaches to setup ldap client
Christian Manal wrote: On 26.11.2013 16:27, slacker lnx wrote: Hello, I have a few ldap clients which were set up by my previous sys-admin. 1. In some of the servers I see that the configuration is done in /etc/pam_ldap.conf, /etc/nslcd.conf and there is a nslcd process running on the clients. 2. On other servers I find that there is only an /etc/ldap.conf and there are no nslcd process running. The configuration values in both the approaches are the same. Are there two different ways to setup an ldap client. I would like to understand both these approaches. In the second approach is there some other process which does the ldap lookup? Can someone share links to docs related to both the installation steps. Thanks Hi, in the cases with /etc/pam_ldap.conf, /etc/libnss_ldap.conf, /etc/ldap.conf and the like, with no daemon running, the system is very likely using PADL's nss_ldap [1] and pam_ldap [2] libs. In case of /etc/nslcd.conf and a running daemon, it's using nss-pam-ldapd [3] which, as you'll read on the website, started out as a fork of nss_ldap. In your first case, they probably switched from pam_/nss_ldap to nslcd and didn't clean up the old config. Possibly because pam_/nss_ldap made problems [4]. Another option would be sssd [5]. Dunno if there are more for Linux. You've forgotten OpenLDAP nssov, which is where the PAM in nss-pam-ldapd came from. Regards, Chrisitan Manal [1] http://www.padl.com/OSS/nss_ldap.html [2] http://www.padl.com/OSS/pam_ldap.html [3] http://arthurdejong.org/nss-pam-ldapd/ [4] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=579647 [5] https://fedorahosted.org/sssd/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: Q: monitoring attributes
Ulrich Windl wrote: BTW: I noticed that there is no schema for entryCSN neither... See doc/drafts/draft-chu-ldap-csn-xx.txt -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: Q: monitoring attributes
Ulrich Windl wrote: Dieter Klünterdie...@dkluenter.de schrieb am 27.11.2013 um 11:15 in In fact spaces in a commonName attribute value are in accordance with X.520 practice. Hi! I believe that, but when printed two spaces look very similar to one space, and as I guess every space is significant, you can probably have Joe OneSpace and Joe TwoSpaces as RDN. Read RFC4518, don't guess. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP with ppolicy and SSSD configuration question.
Viviano, Brad wrote: Adjusting ACL's seems like overkill for this situation and I have to work within the bounds of what sssd offers. sssd doesn't have a native check for pwdAccountLockedTime when it does ppolicy based checking, the code just isn't there. sssd for LDAP auth does support a True/False check for account locked, which is how Redhat DS, 389ds and IPA do it, from what I've read. I've added a True/False as a schema extension, tested it and it works. If I manually set accountLocked to TRUE on a DN, the user can't login at all, it logs in the messages file the account it locked. Works perfect. You're still missing the point that if the user didn't provide a password for LDAP Bind, then LDAP authentication *didn't happen*. If SSSD is still doing an LDAP authorization check here, that's a *bug*, because without a successful authentication, it has no way to verify that the provided username matches any particular LDAP entry. Therefore it cannot assert that any particular LDAP attribute has any relevance to the current login attempt. My question is, is there a better way to set that True/False attribute value based on pwdAccountLockedTime. What I am looking for is, if pwdAccountLockedTime is set for DN=x, then also set accountLocked=true for DN=x. Sure, I can do that with an external script, but is there a way to do it from within slapd. Of course you can write a slapd overlay to do this, but it would be a mistake because your security mechanism is broken. Anyone with permission to write to the directory could spoof their username attribute and lockout any user of their choice. SSSD has a lot of issues. This is why we still recommend nssov. Basically can I create a virtual attribute so when a user queries for accountLocked, it actually does a check for something else (pwdAccountLockedTime) and based on that value returns True or False. I'm thinking in terms of a stored procedure offered on many SQL servers. Thanks, -Brad Viviano -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: OpenLDAP with ppolicy and SSSD configuration question.
Viviano, Brad wrote: Howard, I don't see your point. Clearly. I'm not debating a user providing a password or not. I'm discussing how to inform the client that an account is locked. Slapd already knows the account for DN=x is locked because the user provided an invalid password too many times according the the policy and it set pwdAccountLockedTime. The issue is, sssd, which is the standard for RHEL6 and what I have to deal with doesn't understand that value. It wants a True/False, not a timestamp. So what I'm asking about is, translating a timestamp to a True/False. slapd may indeed know that an entry DN=x has an accountLocked attribute set to true, if you ask it. But *nobody* knows that DN=x corresponds to user y until an LDAP Bind operation has been performed successfully for DN=x with a credential that user y provides. Until you perform that verification step, you cannot assert that any attribute in DN=x has any relevance to user y's login attempt. In your original post you talked about two scenarios: one, where a user logs in with a password, in which case an LDAP Bind is performed and a ppolicy response control can be returned giving the account lock status. two, where a user logs in with only an ssh public key. In the first case, everything works and there's nothing to worry about. The second case is what I've been explaining to you here. If you are forced to rely on sssd then you are unfortunately being forced to rely on an insecure system. I don't understand your second point. ANYONE can lock out a user with ppolicy and that has nothing to so with sssd. I could do an ldapsearch and use any users DN with an invalid password and lock them out if I hit the policy settings that trigger the lock. Heck I could write a perl script that requested every user with posixAccount objectClass and then proceed to bind with invalid passwords to lock out the entire directory as a DoS. Any such attempts would be obvious from syslog activity. What I was talking about would be a simple LDAP Modify request that would never raise suspicion or trip any alarms. But the obvious and more serious consequence is that the Modify request could be used for the reverse - spoofing to re-enable a locked account. At any rate, as long as you insist on talking about how RHEL works I suggest you contact RedHat for further support on this issue. It's their broken software designs you're dealing with. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Memory leak question
Markus Moeller wrote: Apologies it is 2.4.33 and It is part of OpenSuse 12.3. Looks like a legitimate leak to me, please submit the info to the ITS. Thanks. Markus -Original Message- From: Michael Ströder Sent: Tuesday, December 03, 2013 7:51 AM To: Markus Moeller ; openldap-technical@openldap.org Subject: Re: Memory leak question Markus Moeller wrote: I am using openldap 2.2.4.33 with cyrus-sasl 2.1.25 Are you using 2.4.33? Did you compile it yourself? If yes, why not try with the latest release 2.4.38 first? Which OS? Are you using packages shipped with a distribution? Ciao, Michael. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Q: empty groups (groupOfNames, member)
Ulrich Windl wrote: Hi! I had a problem with empty groups: You and everyone else in the world. A quick search would turn up hundreds of posts on this topic. object class groupOfNames has a MUST member attribute, so you cannot create an empty group. I consider this to be a bug in the object class definition, specifically as groupOfNames is structural, and not auxillary. So in SLES empty (POSIX) groups are created with a namedObject structural class. Unfortunately because of structural object class modification from 'namedObject' to 'groupOfNames' not allowed, the entry has to be recreated whenever the first member is added or the last member is removed to/from a group. While examining the problem,. I found out that the namedObject (rfc2307bis.schema) has ist cn attribute optional: ## namedObject is needed for groups without members objectclass ( 1.3.6.1.4.1.5322.13.1.1 NAME 'namedObject' SUP top STRUCTURAL MAY cn ) I'd consider this workaround as a bug also. This is why we wrote a new version of rfc2307bis. http://tools.ietf.org/html/draft-howard-rfc2307bis-02 Two questions remaining: 1) is there a technical reason against empty groups? I'd consider them as valid as empty arrays. The groupOfNames definition comes from X.500. Ask the ITU what they were thinking. 2) Is it an LDAP requirement to forbid structural changes in object classes, or is it an implementation restriction? In my experience the ID of an entry is (if not the entry's UUID) more the value of DN rather than the structural objectClass... It is an X.500 requirement. Read the specs instead of asking what LDAP requires. Insights? Regards, Ulrich -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Max length allowed for a password
Rob Tanner wrote: Hi, We are looking at extending the allowed length of passwords we allow people to use (the theory being that a short phrase is easier to remember than a shorter, but arbitrary string of characters). But since we use our ldap server for authentication to a whole host of online tools, including several portals, I need to know the max length of the source password when doing a bind. There are no maximum lengths in LDAP. Limits imposed by other applications depend on the particular application. Thanks, Rob *Rob Tanner* UNIX Services Manager Linfield College, McMinnville Oregon *ITS will never ask you for your password. Please don’t share yours with anyone! * -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Fwd: Linux kernel performance regressions
Original Message Subject: Linux kernel performance regressions Date: Wed, 11 Dec 2013 04:33:06 -0800 From: Howard Chu h...@symas.com To: openldap-de...@openldap.org openldap-de...@openldap.org We upgraded from kernel 3.5 to 3.12.3 to update some of our benchmark numbers and hit some major performance regressions, mainly because the kernel is throttling processes that use too much CPU. This is definitely a kernel bug, as the throttle mechanism belongs to the realtime scheduler and none of the processes being affected had realtime priority when the throttle kicked in. I've posted a query to the linux kernel mailing list but haven't gotten any satisfactory answers yet. The same throttling behavior also occurs with 3.11.10, but there are no corresponding messages in the kernel log. The email thread is here http://lkml.indiana.edu/hypermail/linux/kernel/1312.1/02313.html A patch that may be related is described here http://lkml.org/lkml/2013/5/29/640 which explains part of the observed behavior, but not all of it (and indeed may be a red herring, unless it has some interaction with the realtime scheduler). There appear to be other serious networking related regressions in 3.12 as well. http://lkml.indiana.edu/hypermail/linux/kernel/1312.1/02588.html I recommend staying on 3.10 for production servers until this is sorted out. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MDB_BAD_RSLOT while executing slapacl
Igor Zinovik wrote:
Hello.
Today I run slapacl to test access rights and saw this:
ldap3# sudo slapacl -F /etc/openldap/slapd.d/ ...
...
52a8ae51 mdb_opinfo_get: err MDB_BAD_RSLOT: Invalid reuse of reader
locktable slot(-30783)
read access to o: ALLOWED
Sounds like ITS#7662.
At
http://symas.com/mdb/doc/group__errors.html#ga1b6cbb28da30e28c48c9df66dd398bf0
I read explanation that this error means: Txn has too many dirty pages.
You need to read more carefully.
slapd 2.4.36 with Linux kernel 3.7.10 x86_64.
In my slapd I have single database with mdb backend:
dn: olcDatabase={1}mdb,config
...
olcDbNoSync: FALSE
olcDbMaxSize: 1073741824
olcDbMode: 0600
olcDbSearchStack: 16
Two questions:
0. What might cause this error?
1. Should I worry about this?
You should upgrade to get the fix for #7662.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MDB_BAD_RSLOT while executing slapacl
Igor Zinovik wrote: 2013/12/12 Howard Chu h...@symas.com mailto:h...@symas.com You should upgrade to get the fix for #7662. I upgraded my slapd to 2.4.38, but I still see error message when I execute slapacl. I also removed data.mdb and lock.mdb, imported data back to ldap using backup copy and I still see error message. Post your config, sample data, and the exact slapacl command you used. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Running slapcat and slapd at the same time
Chris Card wrote: Hi all, I am running openldap 2.4.36 with Berkeley DB 4.6.21 under centos 6.3 64 bit on a Dell 710 with 96 Gb RAM. My slapd configuration sets the BDB cachesize to use a shared memory segment of 47 Gb (id2entry.bdb is about 40 Gb currently). I use slapcat to take regular backups using the same configuration as slapd, running in parallel with slapd, and the backup works successfully, but uses a lot of memory when it is running. Nonsense. slapcat itself only uses as much memory as required to load one single entry at a time. It will attach to the BDB cache, but as you just stated up above, that's *shared* memory. It has no impact on system RAM if slapd was already using it anyway. I would like to be able to run slapcat with a smaller memory requirement, and to this end I have experimented with running slapcat with a different configuration as follows: steps to efficiently corrupt a BDB environment deleted Is there a way to run slapcat with a different memory configuration to slapd? No. Nor is there any reason to try. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldapsearch limit of 500 entries
Jason Brandt wrote:
Note that this will replace any existing limits you have set.
Please: Do NOT post answers without posting links to the relevant
documentation. Our goal here is to teach people to understand how to find
answers for themselves, not to give them mindless copy/paste answers.
On Thu, Dec 12, 2013 at 1:24 PM, Jason Brandt jbra...@fsmail.bradley.edu
mailto:jbra...@fsmail.bradley.edu wrote:
Global size limit modification ldif file (using cn=config):
dn: cn=config
changetype: modify
replace: olcSizeLimit
olcSizeLimit: size.soft=100 size.hard=500
Per user size limit changes:
dn: olcDatabase={1}hdb,cn=config
changetype: modify
replace: olcLimits
olcLimits: dn.exact=uid=user,ou=people,dc=example,dc=com size=unlimited
On Thu, Dec 12, 2013 at 1:16 PM, Clint Petty cpe...@luthresearch.com
mailto:cpe...@luthresearch.com wrote:
My ldapsearch command is only returning a max of 500 entries, while I
know I have over 9,000 entries in the database. If I do not have a
slapd.conf file, how can I increase the sizelimit, to display all my
entries?
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldapsearch limit of 500 entries
Clint Petty wrote:
Thanks Jason,
I resolved this issue by adding:
olcSizeLimit: -1
to the etc/ldap/slapd.d/cn=config.ldif file.
You are not supposed to manually edit the config database files. You should
have fed your change in to the running slapd using ldapmodify.
cn=config is a slapd database. It will very likely migrate to an LMDB backend
in the future. Don't get the notion of manually editing it into your head,
because it won't be possible.
and then restarting slapd.
There is no need to restart slapd to make configuration changes, if you do
them correctly - i.e., using ldapmodify.
Now works!
*From:*Jason Brandt [mailto:jbra...@fsmail.bradley.edu]
*Sent:* Thursday, December 12, 2013 11:25 AM
*To:* Clint Petty
*Cc:* openldap-technical@openldap.org
*Subject:* Re: ldapsearch limit of 500 entries
Note that this will replace any existing limits you have set.
On Thu, Dec 12, 2013 at 1:24 PM, Jason Brandt jbra...@fsmail.bradley.edu
mailto:jbra...@fsmail.bradley.edu wrote:
Global size limit modification ldif file (using cn=config):
dn: cn=config
changetype: modify
replace: olcSizeLimit
olcSizeLimit: size.soft=100 size.hard=500
Per user size limit changes:
dn: olcDatabase={1}hdb,cn=config
changetype: modify
replace: olcLimits
olcLimits: dn.exact=uid=user,ou=people,dc=example,dc=com size=unlimited
On Thu, Dec 12, 2013 at 1:16 PM, Clint Petty cpe...@luthresearch.com
mailto:cpe...@luthresearch.com wrote:
My ldapsearch command is only returning a max of 500 entries, while I know I
have over 9,000 entries in the database. If I do not have a slapd.conf file,
how can I increase the sizelimit, to display all my entries?
--
Jason K. Brandt
Systems Administrator
Bradley University
(309) 677-2958 tel:%28309%29%20677-2958
--
Jason K. Brandt
Systems Administrator
Bradley University
(309) 677-2958
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: Max length allowed for a password
Ulrich Windl wrote: Howard Chu h...@symas.com schrieb am 09.12.2013 um 21:46 in Nachricht 52a62c26.8080...@symas.com: Rob Tanner wrote: Hi, We are looking at extending the allowed length of passwords we allow people to use (the theory being that a short phrase is easier to remember than a shorter, but arbitrary string of characters). But since we use our ldap server for authentication to a whole host of online tools, including several portals, I need to know the max length of the source password when doing a bind. There are no maximum lengths in LDAP. Limits imposed by other applications depend on the particular application. Right, but what about typical input buffer lengths in the openLDAP tools (like ldapsearch)? Part of the point of the open source movement is to allow end users to read the source and see what the code they use actually does. This is a cornerstone of free software, and a key distinction between it and proprietary software. Learn to exercise this freedom. Since LDAP imposes no limits on the maximum length of a password, it would be moronic for OpenLDAP software to impose limits of its own; that would prevent the tools from working with all possible LDAP servers. You can either trust that we are not morons, or distrust that and use something else instead, or read the source and see for yourself. Whatever choice, your question was a waste of time. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: Max length allowed for a password
Hallvard Breien Furuseth wrote: On 2013-12-13 08:57, Hallvard Breien Furuseth wrote: On 2013-12-13 08:17, Ulrich Windl wrote: Howard Chu h...@symas.com schrieb am 09.12.2013 There are no maximum lengths in LDAP. Limits imposed by other applications depend on the particular application. Right, but what about typical input buffer lengths in the openLDAP tools (like ldapsearch)? Right... libldap has #define LDIF_MAXLINE 4096, you must wrap longer lines (start each continuation line with a space). That doesn't impose a max length of the attribute value though. More to the point, ldapsearch() co use getpassphrase() if available, and a Solaris manpage says it limits input to 257 chars. The fallback implementation in OpenLDAP liblutil allows 512 including the final \0. This is not conclusive though. There is no limit on passwords passed on the commandline, nor on passwords read from a file. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldapsearch limit of 500 entries
Christian Kratzer wrote: Hi, On Thu, 12 Dec 2013, Howard Chu wrote: Clint Petty wrote: Thanks Jason, I resolved this issue by adding: olcSizeLimit: -1 to the etc/ldap/slapd.d/cn=config.ldif file. You are not supposed to manually edit the config database files. You should have fed your change in to the running slapd using ldapmodify. cn=config is a slapd database. It will very likely migrate to an LMDB backend in the future. Don't get the notion of manually editing it into your head, because it won't be possible. might be a good time to start enforcing the checksum in the back-ldif files as the inline comment does not seem to help One step at a time. Expect this to be enforced in 2.5. head -2 ~ldap/slapd.d/cn\=config.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 99e8a9c7 Looking at the back-ldif source code the checksum is only enforced when LDAP_DEBUG is defined. Greetings Christian -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Linux kernel performance regressions
Aris Setyawan wrote: Is this mean, that we can't use all of CPU's power with 3.12 kernel? It looks like the bug may have been fixed in 3.12.4 or 3.12.5. At least, I haven't seen it happen again after upgrading. On 12/11/13, Howard Chu h...@symas.com wrote: Original Message Subject: Linux kernel performance regressions Date: Wed, 11 Dec 2013 04:33:06 -0800 From: Howard Chu h...@symas.com To: openldap-de...@openldap.org openldap-de...@openldap.org We upgraded from kernel 3.5 to 3.12.3 to update some of our benchmark numbers and hit some major performance regressions, mainly because the kernel is throttling processes that use too much CPU. This is definitely a kernel bug, as the throttle mechanism belongs to the realtime scheduler and none of the processes being affected had realtime priority when the throttle kicked in. I've posted a query to the linux kernel mailing list but haven't gotten any satisfactory answers yet. The same throttling behavior also occurs with 3.11.10, but there are no corresponding messages in the kernel log. The email thread is here http://lkml.indiana.edu/hypermail/linux/kernel/1312.1/02313.html A patch that may be related is described here http://lkml.org/lkml/2013/5/29/640 which explains part of the observed behavior, but not all of it (and indeed may be a red herring, unless it has some interaction with the realtime scheduler). There appear to be other serious networking related regressions in 3.12 as well. http://lkml.indiana.edu/hypermail/linux/kernel/1312.1/02588.html I recommend staying on 3.10 for production servers until this is sorted out. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Storing graph data structure with MDB_FIXEDMAP
Aris Setyawan wrote: If you want to create a record which has fields referencing the addresses of other fields within the same record, that is the goal for FIXEDMAP but this feature requires the relocation functions, which currently are not implemented. Is in the future, this feature will be removed? Do you have a plan about it? No definite plans. It's something of a wishlist item for now. On 11/30/13, Howard Chu h...@symas.com wrote: Aris Setyawan wrote: Hi, I'm new in this list. I plan to store graph data structure (vertex and edge) in LMDB. The edge data value of a record will contain a pointer which is pointing to the address of vertex data value. So, I must make it sure that data value address is not changing during operation. According to the doc, I can use MDB_FIXEDMAP. And from the doc: ... the memory map will always reside at the same virtual address and pointers used to reference data items in the database will be constant across multiple invocations. ... Spesifically: ... pointers used to reference data items in the database will be constant ... Is this mean that I must allocate memory to write data using mdb_put with MDB_RESERVE flag? Yes, but from what you've described, this will not do what you want. The address of a record is not guaranteed to remain constant, so you cannot safely reference the address of one record from another record. The only safe way to reference one record from another is using its key. If you want to create a record which has fields referencing the addresses of other fields within the same record, that is the goal for FIXEDMAP but this feature requires the relocation functions, which currently are not implemented. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Replicate cn=config between masters and slaves?
Christian Kratzer wrote: Hi, On Mon, 16 Dec 2013, Howard Chu wrote: Christian Kratzer wrote: Hi, On Mon, 16 Dec 2013, Karsten Heymann wrote: Hi, when setting up a cluster with 1+x masters and 1+x slaves, is it possible to use a single cn=config replicated between all of these? Or do masters and slaves need to have different settings anyhow. I'd like not to have to manage schema information and acls in two places. as you generally tend to have slightly different settings on masters and slaves there is currently no other way byt to have separate sets of cn=config for masters and slaves. Use syncrepl suffixmassage. Read this thread. http://www.openldap.org/lists/openldap-technical/201104/msg00206.html interesting. I never noticed that. One still has to keep full parallel copies of different cn=config variations. I had been thinking about a schmeme of having additional attributes under cn=config that would specify to which servers they apply using their respective server URI. Something like the following: dn: olcDatabase=mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: mdb olcDbDirectory: /var/lib/ldap olcSuffix: dc=example,dc=org dn: cn=master-subconfig,olcDatabase=mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig objectClass: xxxServerConfig olcDatabase: mdb xxxServerConfigURI: ldap://ldap-master-1 xxxServerConfigURI: ldap://ldap-master-2 olcSyncrepl: id=1 provider=ldap://ldap-master-1 ... olcSyncrepl: id=2 provider=ldap://ldap-master-2 ... olcMirrorMode: TRUE dn: cn=slave-subconfig,olcDatabase=mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig objectClass: xxxServerConfig olcDatabase: mdb xxxServerConfigURI: ldap://ldap-slave1 xxxServerConfigURI: ldap://ldap-slave2 olcSyncrepl: id=1 provider=ldap://ldap-master-1 ... olcSyncrepl: id=2 provider=ldap://ldap-master-2 ... olcMirrorMode: FALSE that would allow one put site wide shared defaults into the parent entry and group servers with differences under subentries. One of the changes that has been waiting in the wings for a long time is to move syncrepl config out of the olcDatabase entry and into its own olcOverlay entry. At that point a lot of config needs will make a lot more sense, and there'll be no need for gyrations of shared/merged config. 3. How to best handle backward compatibility. We've done this transition once before, changing backglue/subordinate from a database keyword into its own overlay. I don't think it will present any special problems. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Question about search performance
Ferne Quinlan wrote: Hi, Is there someone use openldap on windows?I found the query performance on windows is too bad, Any solution or relevant official documents? This has always been true. In our tests, while OpenLDAP on Windows is usually over 10x faster than e.g. ActiveDirectory, it is still 2x slower than OpenLDAP on Linux. See the benchmark results presented last month at LDAPCon. http://lanyrd.com/2013/ldapcon/sckyhr/ The last time we tested, in 2007, OL on Windows was 3x slower than on Linux, so our Windows support has improved significantly from then till now, but the fact remains that Windows is an inferior OS. Inefficient network stack, inefficient device driver stack, inefficient kernel interface for mutex primitives, etc. etc. etc. If you're using Windows in a serious server deployment you've wasted at least 50% of your budget, in terms of performance/$$. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MDB_BAD_RSLOT while executing slapacl
I started with empty config and empty database with slapd-2.4.38: # sudo slapadd -F /etc/openldap/slapd.d -n0 -l config.ldif _ 100.00% eta none elapsednone fast! Closing DB... I import single object into my catalog: # cat initial-import.ldif dn: dc=example,dc=org dc: example objectClass: organization objectClass: dcObject o: Example # sudo slapadd -F /etc/openldap/slapd.d -b dc=example,dc=org -l config.ldif _ 100.00% eta none elapsednone fast! Closing DB... Trying to check access: # sudo slapacl -F /etc/openldap/slapd.d -D uid=zinovik,ou=people,dc=example,dc=org \ -b dc=example,dc=org o/read authcDN: uid=zinovik,ou=people,dc=example,dc=org 52abd7bc mdb_opinfo_get: err MDB_BAD_RSLOT: Invalid reuse of reader locktable slot(-30783) read access to o: ALLOWED I used your config and got no such behavior here. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MDB_PAGE_FULL
Pieter Martin wrote:
Hi,
I am using lmdb to implement a graph db via java JNI.
Initial results are amazing.
Anyhow I am having trouble with deletions.
I am getting MDB_PAGE_FULL when trying to delete.
Sounds like ITS#7756, you should try with git rev
b77689f97572980d5de729e2fe6d8a5e6e9d6218
A simplified version of the code is as follows.
//set the key
...
while ((rc = mdb_cursor_get(vertexCursor, key, data,
MDB_SET_RANGE)) == 0) {
rc = mdb_cursor_del(vertexCursor, 0);
if (rc != 0) {
printf(removeVertex 4 %i\n, rc);
goto fail;
}
//set the key
...
}
The mdb_cursor_del returns a error code -30786 (MDB_PAGE_FULL) on about
the 38th loop.
if there are very few entries then the exception does not happen.
Is there something obvious that I am doing wrong to cause a
MDB_PAGE_FULL exception?
Thanks
Pieter
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Oracle OpenLDAP PPolicy ppolicy and the hierarchy
Michael Ströder wrote: Michael Ströder wrote: Arthur de Jong wrote: Since you cannot do joins in LDAP, every group with member attributes such as cn=Joe,ou=People,dc=... will require another lookup per member to find the username (uid attribute). This very much depends on the implementation of the NSS provider. AFAIK sssd simply searches all posixAccount and posixGroup entries and resolves group membership internally from the local sssd cache database. If a NSS provider does not support something similar it should be expanded to do so or one should not use it at all. Furthermore there's slapo-deref which seems to work. The client control can be used to retrieve all the 'uid' values in member entries. The NSS provider has to extract the 'uid' values from the response control value. See https://tools.ietf.org/html/draft-masarati-ldap-deref Was going to reply but Michael beat me to it. Reiterating all the points Michael made. There is no good reason to use memberUid or uniqueMember in LDAP, both of these schema elements are deeply flawed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Oracle OpenLDAP PPolicy ppolicy and the hierarchy
Arthur de Jong wrote: On Wed, 2013-12-25 at 16:44 +0100, Michael Ströder wrote: Furthermore there's slapo-deref which seems to work. The client control can be used to retrieve all the 'uid' values in member entries. The NSS provider has to extract the 'uid' values from the response control value. See https://tools.ietf.org/html/draft-masarati-ldap-deref Sadly, the Internet Draft expired without turning into an RFC. I also can't find any documentation on slapo-deref, do you have any pointers? Also, do you have any idea whether this is implemented by a significant part of the LDAP servers out there (is it worth the effort to implement this client-side)? This was developed at the request of the Samba team, and some of those developers also worked on SSSD, so it has already been implemented in significant volumes. There is also a memberof overlay that populates memberOf attributes in users. Would it be difficult to make a memberuid overlay that populates memberUid attributes in the group? That would be counterproductive. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Ldap password policy not throwing different errors
Idan Fridman wrote:
Hi,
So how will you distinct between the cases? How user or admin will be able to
know if that user is blocked?
Reread the slapo-ppolicy(5) manpage. It states quite clearly that
ppolicy_use_lockout only affects the ppolicy response control. Your client
must Bind using the ppolicy request control in order to generate this result
code, and it must properly parse the ppolicy response control to see the
actual code.
Thanks,
Idan.
- Reply message -
From: Dieter Klünter die...@dkluenter.de
To: openldap-technical@openldap.org openldap-technical@openldap.org
Subject: Ldap password policy not throwing different errors
Date: Sun, Jan 5, 2014 21:33
Am Sun, 5 Jan 2014 15:13:51 +
schrieb Idan Fridman id...@cellebrite.com:
Hi,
I use ppolicy overlay and enabled ppolicy_use_lockout to separate
between invalid password and locked accounts.
databasebdb
suffix dc=openiam,dc=com
rootdn cn=Manager,dc=openiam,dc=com
rootpw {SSHA}2ttRoo/t5HuMT2nPxtI6goVUML5R2H9h
# PPolicy Configuration
overlay ppolicy
ppolicy_default cn=default,ou=policies,dc=openiam,dc=com
ppolicy_use_lockout
ppolicy_hash_cleartext
I tried to lock user account by entering wrong password couple of
times (pwdMaxFailure)
The user is being locked but when I try to login again I still get
the same error:
Invalid credentials (49)
Any idea why i am not getting diffrent error to disticnt between the
cases?
1. there is no appropriate result message for password policy. RFC 4511
Section 4.1.9 defines all result messages and Appendix A provides in
brief a general description.
2. In your particular case result 49 is a substitution in order to
prevent an unauthorized disclosure.
-Dieter
--
Dieter Klünter | Systemberatung
http://dkluenter.de
GPG Key ID:DA147B05
53°37'09,95N
10°08'02,42E
This e-mail and the information it contains may be privileged and/or
confidential. It is intended solely for the use of the named recipient(s). If
you are not the intended recipient you may not disclose, copy, distribute or
retain any part of this message or attachments. If you have received this
e-mail in error please notify the sender immediately [by clicking 'Reply'] and
delete this e-mail.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: DBIS - new IETF drafts to replace NIS and RFC2307
Mark R Bannister wrote: In August this year, I submitted some new IETF drafts with the intent that they would replace NIS and RFC2307. It introduces Directory Based Information Services (DBIS). Michael Ströder suggested I post a link to this mailing list. The following article (and further articles on my blog) discuss some of the aspects of DBIS, and link to all of the relevant internet drafts: http://technicalprose.blogspot.co.uk/2013/08/introducing-dbis.html I am currently working on a reference implementation here: http://sourceforge.net/projects/dbis/ This may be coming out of the blue for many of you, and I appreciate you may need some time to read and digest my drafts, and then ask me lots of questions. But my first question to you is, where is the best place for discussion related to these drafts? Is this the mailing list to use? The ietf-ldapext mailing list would be most appropriate. I have several comments regarding this proposal but I'll post them to the ldapext list. Best regards, Mark Bannister. e: d...@proseconsulting.co.uk -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Problem with back-mdb
Grégory T wrote:
Hello,
before starting, we wish a happy new year to the community !
Now, we start with presentation.
We have compiled the Openldap build version 2.4.38 on a ubuntu 12.04.
This openldap server is a dual processor Intel Xeon with 4 Gb Memory. This
machine is virtualized (vmware).
Openldap is configured with back-mdb, overlays memberof and syncprov, and
envflags writemap and nosync.
We encounter a problem with a callback procedure in a perl script.
This script have to remove a couple of attributes for each entry (~120 000).
In order to save memory consumption, we use a callback function to treat each
entry one after one.
Everything works with back-bdb but ... there are mistakes with back-mdb.
In ldap.log, this kind of message appears many times :
ldapserver slapd[7264]: connection_input: conn=1014 deferring operation:
awaiting write
ldapserver last message repeated 55 times
This is an interesting problem. The above message indicates that a slapd
thread on that connection has been blocked writing a result to the client.
I.e., back-mdb is so much faster than back-bdb that it's returning results
faster than your client can read them. Once it gets blocked like this, it
defers any new requests on that connection until it gets unblocked. (I.e., it
waits until the client catches up on reading.)
And suddenly,
ldapserver slapd[7264]: conn=1014 fd=14 closed (connection lost)
connection lost actually means the client closed the connection without
performing an Unbind.
Our script result in connection reset by peer message.
The script stop around the 1040 entry.
Although the connection was closed by server, it still continue to run.
However, no explicit message in ldap.log.
If we run the script once again, the next ~1040 entries will be treated and
the script stop.
We have try with envflags writemap and nosync deactivated ... but unsuccessful.
This is the ldapsearch request :
my
$result=$ldapMaster-search(base=$baseDN,scope='sub',filter=$filter,typesonly='1',callback=\callback);
And this is the callback function :
sub callback{
my ($search,$ent)=@_;
return unless defined $ent;
my $entry = $search-shift_entry();
my @tabAttribToDel=();
if (defined $entry){
my $dn=$entry-dn();
$countDN++;
foreach my $attrib($entry-attributes){
if ($attrib ~~ @tabOldAttrib){
$countAttrib++;
push (@tabAttribToDel,$attrib);
}
}
if (($SUPPR==1) (@tabAttribToDel)) {
$countTrace++;
my $result = $ldapMaster-modify($dn,delete = \@tabAttribToDel);
warn (nbEntry:$countTrace, nbAttr:$countAttrib) Error with $dn:
.$result-error.\n if $result-code;
}
}
}
Why is this script working well with back-bdb but not with back-mdb ?
Working with back-bdb because it's slow enough that your client never falls
behind. With back-mdb your perl script is too slow to keep up.
One workaround, the avoid the deferring - use a 2nd connection for your modify
requests.
Thanks,
--
Grégory Trucy
Ingénieur système et réseau
Aix-Marseille Université
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: mdb searchstack parameter
Paul B. Henson wrote: From reading the documentation, it sounds like when configuring this parameter you need to strike a balance between potential performance impact if it is too low, and wasted memory if it is too high. While running, is there any way to tell whether or not a search exceeded your configured value and required additional allocation? Whether via the monitor backend, or a log entry, or? Ideally, it would be nice to be able to tell if your configured value was too low and adjust as necessary. Since you mention that you're migrating from hdb, you can most likely ignore this parameter. It has the identical meaning in hdb after all, and if you never had to change it under hdb there's no reason to change it for mdb. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: configuring mdb maxsize
Paul B. Henson wrote: From: Quanah Gibson-Mount [mailto:qua...@zimbra.com] Sent: Wednesday, January 08, 2014 8:20 AM I change the maxsize of my db all the time, while slapd is running, without issue, as long as the maxsize is not attempted to be set to less than value of the actual size of the DB. Cool; thanks for the data point. I rather thought the manpage was self-explanatory. Put another way - in the middle of a heavy write load when your server is running at 100% utilization is the wrong time to discover that you need to increase the maxsize. A smart admin picks a very large number at init time (e.g. some percentage of the disk partition size) and never has to worry about it again. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: mdb expected growth
Paul B. Henson wrote: So after updating our dev ldap environment to use mdb, and slapadd'ing a fresh copy from our production environment into it, the database was using 828M: Environment Info Map address: (nil) Map size: 2147483648 Page size: 4096 Max pages: 524288 Number of pages used: 211612 Last transaction ID: 1333 Max readers: 126 Number of readers used: 6 Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 29 I then proceeded to run a test load on it (basically, I had a script I put together when I added the memberof overlay that ripped through all of our groups, removing and then re-adding all members). After this test run, the use jumped to 1.9G: Environment Info Map address: (nil) Map size: 2147483648 Page size: 4096 Max pages: 524288 Number of pages used: 479536 Last transaction ID: 1075380 Max readers: 126 Number of readers used: 11 Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 29 There is no new data in the database, all that happened was existing data was removed and then re-added. Is this drastic increase in space utilization expected in such a scenario? From the sound of your quite vague test description, sure. As it states in the LMDB doc, long-lived reader transactions prevent reuse of freed pages. http://symas.com/mdb/doc/ You have a long search operation running, and are issuing writes while the search is in progress. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MDB_BAD_RSLOT while executing slapacl
Francesco Malvezzi wrote: This is a follow-up of: https://www.openldap.org/lists/openldap-technical/201312/msg00119.html I have observed in the wild the MDB_BAD_RSLOT while executing slapacl error on two different servers: [...] 52d3c079 = check a_group_pat: cn=agentsgroup,ou=groups,dc=example,dc=org 52d3c079 mdb_opinfo_get: err MDB_BAD_RSLOT: Invalid reuse of reader locktable slot(-30783) 52d3c079 = check a_group_pat: cn=admins,ou=groups,dc=example,dc=org 52d3c079 mdb_opinfo_get: err MDB_BAD_RSLOT: Invalid reuse of reader locktable slot(-30783) [...] On the other hand, I am not able to reproduce the error with the example data given by Igor. The slapacl returns 'ALLOW'. Might be of interest my config yielding the error? If you can reproduce this reliably, yes, please send your config, sample data, and exact slapacl command. thank you, Francesco /usr/local/openldap-2.4/libexec/slapd -V @(#) $OpenLDAP: slapd 2.4.38 (Dec 4 2013 11:26:00) $ francesco@c1:/usr/local/src/openldap-2.4.38/servers/slapd -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: mdb expected growth
Paul B. Henson wrote: From: Howard Chu [mailto:h...@symas.com] Sent: Sunday, January 12, 2014 1:41 PM From the sound of your quite vague test description, sure. As it states in the LMDB doc, long-lived reader transactions prevent reuse of freed pages. http://symas.com/mdb/doc/ You have a long search operation running, and are issuing writes while the search is in progress. Hmm, I don't think that is the case. The way the script works is that it first generates a list of all of our groups (which doesn't even come from LDAP, it gets pulled out of a database that is another component of our idm infrastructure). Then, for each group, it does a search to get all the members, and then removes/adds them 500 at a time until it has processed all of them. So there is one search, then repeated modify remove/modify add, then another completely separate search, etc, through all of them. The connection to the LDAP server is persistent throughout the script, but there is no single operation whose duration persists. Does it have anything to do with being a syncrepl provider? The documentation sometimes describes syncrepl as being a persistent search, and the size increase only occurred on the primary master, not the secondary master or read-only slaves. The syncprov persistent search implementation doesn't result in a long-lived read transaction. You can use mdb_stat -r to see what's going on as far as read transactions in the DB. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: BDB fine grained lock manager
Chris Card wrote: Hi All, We are running openldap 2.4.36 and Berkeley DB 4.6.21 (+ 4 patches) on centos 6.3 64 bit. Does anyone know if building Berkeley DB with --enable-fine_grained_lock_manager is likely to make any difference to slapd performance? Try it and see... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: java API for SAML integration
Jignesh wrote: Any help on this topic? There's not much interest in java here. Try asking the ApacheDS folks. On Jan 2, 2014, at 10:23 PM, Jignesh Patel wrote: We have been using SimpleSAMLphp http://simplesamlphp.org/ with openldap 2.4.33. Is there any Java API available which can provide similar functionality? Jignesh Patel Chief Architect * -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: lmdb memory usage when writing lots of data with memorymap option
Luc Vlaming wrote: Hi, Currently I am creating support for using LMDB as a new storage backend for one of our products. At the moment I am testing import bulk data into lmdb using transactions that span a single record of 10MB. The total db size afterwards is 5GB. I also tested with records of 1MB. I noticed a very odd thing: when using the MDB_WRITEMAP option, memory usage grows very quickly and linear with the amount of data stored into the database. (memory usage ends up a bit higher than 5GB). when not using MDB_WRITEMAP, however, memory usage stays very low. Does anyone have a suggestion what might be wrong and what causes such different behaviour with and without using the memorymap option? There is nothing wrong. It is simply writing to the shared memory map. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: lmdb memory usage when writing lots of data with memorymap option
Luc Vlaming wrote: Hi, If it was simply writing to the memory map, shouldn't memory usage decrease as soon as everything is written? Why? Sounds to me like you don't understand how demand paging works in a virtual memory system. The key rule here - if there is no demand, then there is no paging. The memory usage continuous to be high for as long as the database is open, even if the program just waits afterwards. Is that to be expected as well? Because that would mean that the process would simply run out of memory if more data is writting than the machine has as ram. That's not how virtual memory works. Go read up on it. Regards, Luc Met vriendelijke groeten, Luc Vlaming KXA Software Innovations voorheen Dysi Software Innovations bezoekadres: Hoendiep Noordzijde 21 9843TG, Grijpskerk Luc Vlaming tel: 06 16 353 426 email: vlam...@softwareinnovations.nl mailto:vlam...@softwareinnovations.nl url: www.softwareinnovations.nl http://www.softwareinnovations.nl/ On Wed, Jan 15, 2014 at 11:10 PM, Howard Chu h...@symas.com mailto:h...@symas.com wrote: Luc Vlaming wrote: Hi, Currently I am creating support for using LMDB as a new storage backend for one of our products. At the moment I am testing import bulk data into lmdb using transactions that span a single record of 10MB. The total db size afterwards is 5GB. I also tested with records of 1MB. I noticed a very odd thing: when using the MDB_WRITEMAP option, memory usage grows very quickly and linear with the amount of data stored into the database. (memory usage ends up a bit higher than 5GB). when not using MDB_WRITEMAP, however, memory usage stays very low. Does anyone have a suggestion what might be wrong and what causes such different behaviour with and without using the memorymap option? There is nothing wrong. It is simply writing to the shared memory map. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/__project/ http://www.openldap.org/project/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: lmdb memory usage when writing lots of data with memorymap option
Ulrich Windl wrote: Off-topic: I can remember a statement of the late 80ies where a programmer claimed the 32-bit address space is so large that one does not have to care about garbage collection in virtual address space; just use new addresses. I think even with 64 bit one should always try not to waste address space. Wow, good thing you're here to remind us of these little details, I would never have thought of that! What on earth will we do now? Oh wait, right in the LMDB design paper, there's a discussion of garbage collection. Someone must have thought of it already, after all. You're the worst kind of fool, one who believes he knows more than he actually does, in a world where it's trivially easy to acquire the actual facts. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Access of entryUUID in an overlay module
Alexander Kläser wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Dear all,
I am new to OpenLDAP. In order to process transactions in a separate
piece of software, I am parsing the output of a slightly modified
version of the auditlog overlay module. As I need to know the
entryUUID of a modified LDAP object, I tried to query it and write it
along with the other information into the auditlog file.
My current patch seems to navigate slapd into a deadlock situation. I
do not understand the reason for that and would be very happy if
someone may help to point me into the correct direction for
implementing this.
Simple: after you fetch the entry, you also must release it.
Your patch is also broken in that you've inserted your code into the generic
code path, so it will execute for Add operations as well. Obviously for an Add
operation, there will not be an entry in the DB to fetch, all of the entry
data is part of the Operation.
Here is my patch that I am working with, so far:
- --- ./servers/slapd/overlays/auditlog.c.orig2014-01-09
12:26:22.06000 -0500
+++ ./servers/slapd/overlays/auditlog.c 2014-01-17 11:13:34.62400
- -0500
@@ -24,6 +24,7 @@
#ifdef SLAPD_OVER_AUDITLOG
#include stdio.h
+#include time.h
#include ac/string.h
#include ac/ctype.h
@@ -74,8 +75,13 @@
Modifications *m;
struct berval *b, *who = NULL, peername;
char *what, *whatm, *suffix;
- - time_t stamp;
+ struct timeval stamp;
int i;
+ int rc;
+ Entry *e = NULL;
+ Attribute *a_entryUUID;
+ char *entryUUID = NULL;
+ BackendInfo *bi = op-o_bd-bd_info;
if ( rs-sr_err != LDAP_SUCCESS ) return SLAP_CB_CONTINUE;
@@ -119,6 +125,16 @@
if ( !who )
who = op-o_dn;
+ /* get the entryUUID */
+ //rc = overlay_entry_get_ov( op, op-o_req_ndn, NULL, NULL, 0,
e, on );
+ op-o_bd-bd_info = (BackendInfo *)on-on_info;
+ rc = be_entry_get_rw( op, op-o_req_ndn, NULL, NULL, 0, e );
+ op-o_bd-bd_info = bi;
+ if ( rc == LDAP_SUCCESS ) {
+ a_entryUUID = attr_find( e-e_attrs,
slap_schema.si_ad_entryUUID );
+ entryUUID = a_entryUUID-a_vals[0].bv_val;
+ }
+
peername = op-o_conn-c_peer_name;
ldap_pvt_thread_mutex_lock(ad-ad_mutex);
if((f = fopen(ad-ad_logfile, a)) == NULL) {
@@ -126,9 +142,11 @@
return SLAP_CB_CONTINUE;
}
- - stamp = slap_get_time();
- - fprintf(f, # %s %ld %s%s%s %s conn=%ld\n,
- - what, (long)stamp, suffix, who ? : , who ? who-bv_val : ,
+ gettimeofday(stamp, NULL);
+ fprintf(f, # %s %ld.%06ld %s %s%s%s %s conn=%ld\n,
+ what, stamp.tv_sec, stamp.tv_usec,
+ entryUUID,
+ suffix, who ? : , who ? who-bv_val : ,
peername.bv_val ? peername.bv_val: , op-o_conn-c_connid);
if ( !BER_BVISEMPTY( op-o_conn-c_dn )
@@ -176,7 +194,7 @@
break;
}
- - fprintf(f, # end %s %ld\n\n, what, (long)stamp);
+ fprintf(f, # end %s %ld.%06ld\n\n, what, stamp.tv_sec,
stamp.tv_usec);
fclose(f);
ldap_pvt_thread_mutex_unlock(ad-ad_mutex);
thanks so much in advance
Alex
- --
Dr. Alexander Kläser
Open Source Software Engineer
Univention GmbH
be open.
Mary-Somerville-Str.1
28359 Bremen
Tel. : +49 421 22232-59
Fax : +49 421 22232-99
klae...@univention.de
http://www.univention.de
Geschäftsführer: Peter H. Ganten
HRB 20755 Amtsgericht Bremen
Steuer-Nr.: 71-597-02876
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlLZWZkACgkQgX5Q1Nb/qB0VSACfanVbcP79n3gEnH0zivWSThGA
dxUAnj+2NaKhLweCFjFMtGoHPdHzByQZ
=DHYq
-END PGP SIGNATURE-
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: providing a default value for an attribute (w/ translucent proxy)
Dan Pritts wrote: Hi all - I've got a familiar story, I'm sure. I have a campus-wide AD infrastructure that I'd like to leverage. Unfortunately, my LDAP client (an Isilon NAS) requires some attributes that AD doesn't populate. e.g., homeDirectory. As it turns out, the silly thing doesn't even use these values, at least not for our use case. But the entire lookup fails if it doesn't get something back for those attributes. I presumably can use a translucent proxy to set these attributes locally. I tested this successfully. Ideally, though, instead of creating DNs for each person on the translucent proxy server, I'd like to just have slapd fill in a default value for these attributes when they are empty. I would also be happy with a method that just overwrites the existing attribute, regardless of whether it exists. Again, for ALL DNs... I don't want to create each account individually in both places. I thought maybe the rewrite overlay would allow this but it seems like I was incorrect (please advise if it can be done). I have poked around and found no other way to do this. Am I missing something? If the actual value doesn't matter, try using a collective attribute instead. thanks -- Dan Pritts ICPSR Computing Network Services University of Michigan +1 (734)615-7362 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LMDB database size
Markus Doppelbauer wrote: Hello, We are using BerkeleyDB since a decade - but we are searching for something more lightweight. LMDB is what we are looking for ... ^^ The small code and the benchmark-results are very impressive! A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has about 20 GB.This particular database is a kind of archive - hardly read and hardly written (maybe once per minute). Is it possible to mmap() a 20 GB database into the RAM? Yes. If yes, will the server start swapping? No. Thanks a lot Markus -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: LMDB database size
Ulrich Windl wrote: Howard Chu h...@symas.com schrieb am 25.01.2014 um 19:07 in Nachricht 52e3fd6e.9070...@symas.com: Markus Doppelbauer wrote: Hello, We are using BerkeleyDB since a decade - but we are searching for something more lightweight. LMDB is what we are looking for ... ^^ The small code and the benchmark-results are very impressive! A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has about 20 GB.This particular database is a kind of archive - hardly read and hardly written (maybe once per minute). Is it possible to mmap() a 20 GB database into the RAM? Yes. If yes, will the server start swapping? No. This is only true (AFAIK) You've already demonstrated multiple times that as far as you know is not far at all. if the applications uses huge pages or memory locking. In other cases it depends on the OS and the other applications. In general mmap files swap onto themselves, and you would not call it swapping in the usual sense... Thus, the answer is No, as I already said, and you have added no value to the discussion. Ulrich -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MDB_BAD_VALSIZE error
Chris Card wrote: Hi All, I'm running openldap 2.4.36 on centos 6.3, 64 bit. I'm finally trying out MDB as a replacement for BDB in our openldap setup, and I've noticed that I'm getting MDB_BAD_VALSIZE errors when adding some DNs, which could be added without error when using BDB as the backend: 52e9170b = mdb_dn2id_add 0x5da55e: -30781 52e9170b mdb_add: dn2id_add failed: MDB_BAD_VALSIZE: Too big key/data, key is empty, or wrong DUPFIXED size (-30781) Looking at the code, it seems that this error happens when MDB_MAXKEYSIZE is exceeded and MDB_MAXKEYSIZE is set to 511 as far as I can see. The DNs which are giving this error are between 232 and 255 characters long. Does the MDB backend have an implicit limit on the length of valid DNs? slapd has a limit of 8192 bytes on DNs. RDNs in back-mdb are limited by MDB_MAXKEYSIZE. Can this be increased (by changing MDB_MAXKEYSIZE) and if so, what is the longest DN which can be supported by the MDB backend? The longest DN supported is 8192 bytes. The longest RDN would be something close to one half of MDB_MAXKEYSIZE. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: MDB_BAD_VALSIZE error
Chris Card wrote: Howard Chu wrote: Subject: Re: MDB_BAD_VALSIZE error Chris Card wrote: Hi All, I'm running openldap 2.4.36 on centos 6.3, 64 bit. I'm finally trying out MDB as a replacement for BDB in our openldap setup, and I've noticed that I'm getting MDB_BAD_VALSIZE errors when adding some DNs, which could be added without error when using BDB as the backend: 52e9170b = mdb_dn2id_add 0x5da55e: -30781 52e9170b mdb_add: dn2id_add failed: MDB_BAD_VALSIZE: Too big key/data, key is empty, or wrong DUPFIXED size (-30781) Looking at the code, it seems that this error happens when MDB_MAXKEYSIZE is exceeded and MDB_MAXKEYSIZE is set to 511 as far as I can see. The DNs which are giving this error are between 232 and 255 characters long. Does the MDB backend have an implicit limit on the length of valid DNs? slapd has a limit of 8192 bytes on DNs. RDNs in back-mdb are limited by MDB_MAXKEYSIZE. Can this be increased (by changing MDB_MAXKEYSIZE) and if so, what is the longest DN which can be supported by the MDB backend? The longest DN supported is 8192 bytes. The longest RDN would be something close to one half of MDB_MAXKEYSIZE. Thanks, I understand now. The longest RDN appears to be 245 characters on my system. Can MDB_MAXKEYSIZE be increased? If so, what is the maximum safe value? Read the comments in lmdb.h and/or mdb.c. The limits are documented there. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Have you seen this FUD - IT pros suffer OpenLDAP configuration headaches ?
Gavin Henry wrote: http://searchdatacenter.techtarget.com/feature/IT-pros-suffer-OpenLDAP-configuration-headaches Any one been in touch with them? I saw some of this on twitter before, ignored it since none of the parties involved have any clue what they're talking about. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Have you seen this FUD - IT pros suffer OpenLDAP configuration headaches ?
Turbo Fredriksson wrote: On Jan 30, 2014, at 5:35 PM, Howard Chu wrote: I saw some of this on twitter before, ignored it since none of the parties involved have any clue what they're talking about. Personally, I think it's spot on. It IS hard to configure an LDAP server, and even harder to understand how it works (the object based part). Took me three months first time, and I'm not an idiot. The object based part is *LDAP*, so that complaint is not specific to OpenLDAP. The part about RedHat seems fairly accurate to me, it *is* true that they have their own commercial LDAP server to sell, and they have no great interest in OpenLDAP working well on their platforms. Even today, I need to consult either my own book or the howto (or seriously skim through the man pages) to setup a new server. And I still need to read the docs when configuring an Apache HTTP server. That's why we have manpages, there's nothing wrong about that. And even worse if when you want to optimize the backend... There's a lot of magic there The LMDB backend has no tuning/optimization. That's one of the reasons it exists today. And with the new config backend!? I haven't even had the time or energy to go that far yet! I think you (and everyone else) are blowing this way out of proportion. Compare the example from here http://www.openldap.org/doc/admin24/slapdconf2.html#Configuration%20Example to the slapd.conf example http://www.openldap.org/doc/admin24/slapdconfig.html#Configuration%20File%20Example They aren't that different, and anyone familiar with slapd.conf and LDIF files should have no trouble mapping concepts from one to the other. And if you aren't familiar with slapd.conf *and* LDIF then you don't know enough to be an OpenLDAP administrator in the first place, you need to do more homework. That's just life. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Have you seen this FUD - IT pros suffer OpenLDAP configuration headaches ?
Turbo Fredriksson wrote: [Sorry Howard for sending it to you personally. It was meant for the list. I sent a copy to the list as well. I hope you don't mind if I send this reply to the list. I've included every word, so not to take something out of context.] No problem, after I saw your reply on the list I figured out that was your original intent. And even worse if when you want to optimize the backend... There's a lot of magic there The LMDB backend has no tuning/optimization. That's one of the reasons it exists today. Yeah, but isn't it quite slow with lmdb? I haven't tested that in years, so I don't know. One wouldn't run it in production though? You are (unfortunately) confusing the very new back-mdb with the very old, long obsolete, years-ago-deleted, back-ldbm. OpenLDAP back-mdb is quite simply the fastest LDAP engine in the world, by a huge margin over all other directory software in existence. And with the new config backend!? I haven't even had the time or energy to go that far yet! I think you (and everyone else) are blowing this way out of proportion. Compare the example from here I know how it works and I don't really have that much problem with it, it's just so much more difficult to setup (initially) and then maintain than a simple text file. It's way better, but it IS also more complicated (than just fire up an editor, modify the part you want and then issue a service restart - can't be much simpler than that)... I don't even fire up an editor, I just issue an ldapmodify - no service restart needed either, no interruption of service to clients. There's nothing smoother and more transparent than that. And if you aren't familiar with slapd.conf *and* LDIF then you don't know enough to be an LDAP administrator in the first place, you need to do more homework. That's just life. I couldn't agree more! I've taken over more than my fair share of badly setup and maintained OpenLDAP servers to get really pissed at all the ones not having a clue what they're doing. It's not just making a config file/backend to allow the server to start, it's more planning on how the database should look like (where to put what and what object classes to use and allow), setting up access control etc, etc. The actually planing of the database (the content) is the most important part, and it require quite a lot of reading and testing before it's understood properly to be able to be used to any extent. But then there's the integration to the rest of the system (pam login and what not), Kerberos, SASL, etc, etc... My point wasn't to argue about the validity of how the OpenLDAP server and it's config file/backend work etc. I fully agree and have no problems with it. My point was that the website isn't WRONG - it IS hard! Maybe it SHOULD be hard? The whole concept of an LDAP server is a difficult subject, and shouldn't be taken lightly. Perhaps. Vendors like RedHat make it worse than necessary though, forcing the use of Mozilla NSS which is still not ready for primetime, despite having existed for 20+ years. People made enough mistakes configuring TLS support when all you needed was to drop a few PEM files into place. With RedHat requiring the use of MozNSS they make the setup even more opaque and crash-prone. That's not OpenLDAP's fault, that's all RedHat. Unfortunately, it seems that way to many beginners that have been installing a distribution at home is starting to work as a Linux tech/admin thinking that just because the've run it at their workstation at home for a couple of months makes them good enough to work in a professional environment. I see that in a lot of OpenSource project I'm part of. Complete noobs want to use something complicated that require quite a lot of homework. And then comes complaining when things go south! Or even worse, they bad mouth the project or the technology! (Open)LDAP is one of those many things that require a lot more from the admin than say ... installing a mail server locally... On Debian GNU/Linux that's practically automatic. Just answer a couple of questions, and it works... It's sad that the website in question (and from what one could take from this - that people 'out there') actually thinks that this should be easy. But it's not (technically) wrong... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Have you seen this FUD - IT pros suffer OpenLDAP configuration headaches ?
Wiebe Cazemier wrote: - Original Message - (snip) (long) (snip) I am by no means an LDAP expert, but as an experienced Linux sysadmin I do have to say that I have had some very tricky issues with OpenLDAP. One of them involved fiddling for days with difficulty changing the root password, after finally finding out that the Ubuntu docs were wrong [1]; they had cause me to create two admin users, with the passwords in plain text no less. The other involved getting 'TLS required' on the TCP connection, which seems to be undocumented. Nonsense. The security directive is documented in slapd.conf(5) and slapd-config(5) manpages. My question on Serverfault about it [2] is getting to be quite popular. Forcing encryption would have been a lot easier if a different port for SSL wasn't deprecated. As usual when you go to unofficial support channels, all you get is garbage from unqualified self-proclaimed experts. The highest ranked answer on your question is flat wrong, and refers to Zytrax documentation, which is poorly plagiarized from outdated copies of the OpenLDAP Admin Guide and mixed with a generous helping of misinformation from their own addled brains. [1] https://bugs.launchpad.net/serverguide/+bug/1094842 [2] https://serverfault.com/questions/459718/configure-openldap-with-tls-required Regardless of what you may think about the tone of postings on this list (which is ludicrous to begin with since emails by their nature are horrible at conveying tone or emotion), actual subject matter experts monitor this list and make sure that correct answers get posted and that BS is censured. That is the purpose of this forum. If you want to be coddled, feel free to look elsewhere. If you want real answers, this is the place. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Have you seen this FUD - IT pros suffer OpenLDAP configuration headaches ?
Turbo Fredriksson wrote: And my opinion (and many, many others!) have been that it is. And that there's something huge lacking in the OpenLDAP documentation. But every time this is brought up, all the maintainers get very hostile. Let me be clear about this - there is nothing hostile in this message: The Project is run by volunteers. If you want something to improve, then contribute your time to making it happen. There is no chain of command where any person gives orders and someone else carries them out. Areas get worked on when interested people step up and work on them. The areas of code that I've worked on were the areas that interested me personally. Areas that don't interest me get ignored. (Some things, like asserts and other annoying crashes, may get my attention even though the code doesn't interest me, but these are somewhat rare.) The topic of documentation does raise some ire these days, particularly because we see companies like Zytrax slurping up our Admin Guide and regurgitating it with a large dose of their own misunderstanding and misinformation. If you actually want to help make things better for the Project and the community, then do so - submit documentation patches back to us. Don't splinter off and write your own fantasy novel of how you think things work. Write patches to the Guide to expand on the areas you think need to be explained better. Fragmenting the knowledge base only increases confusion, it doesn't make things better. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapd.d and back-sql
Robson, Alan wrote: Hi all, I’m trying to set up back-sql on Ubuntu using openldap version 2.4.39 that uses slapd.d as its configuration store. All of the config examples I can find online use slapd.conf. A 2011 email from this mailing list explains that it simply is not possible to use the slapd.d config method to specify a dbname. Is that still the case ? Read the schema. ldapsearch -b cn=schema,cn=config -s base [and whatever authentication you need] | grep -i sql I am trying to add an entry for my mysql database using ldapadd and the following ldif… # order value not present so entry will be appended # to current children dn: olcDatabase=sql,cn=config objectClass: olcDatabaseConfig olcDatabase: mysql dbName: ldap olcSuffix: dc=nodomain The response I get is… adding new entry olcDatabase=sql,cn=config ldap_add: Undefined attribute type (17) additional info: dbName: attribute type undefined I tried dbname (no capital N) as well, but I get the same result. As you may be able to tell, I have a very loose grip on what to do here ! I’d appreciate some guidance. I am confident that I have odbc and the database/user set up as it describes in the manual section on back-sql. My database connection is called “ldap” in odbc.ini. /etc/odbcinst.ini: [mysql] Description = ODBC for MySQL Driver = /usr/lib/x86_64-linux-gnu/odbc/libmyodbc.so Setup = /usr/lib/x86_64-linux-gnu/odbc/libodbcmyS.so FileUsage = 1 /etc/odbc.ini: [ldap] Description = MySQL connection to 'test' database Driver= mysql Database = ldap UserName = ldap Password = ldap Server= localhost Port = 3306 Socket= /var/run/mysqld/mysqld.sock If I need to go back to using slapd.conf, how can I generate one from my slapd.d setup ? Many thanks Alan -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapd.d and back-sql
Robson, Alan wrote: That's a great suggestion, but I don't see anything that looks like a likely replacement for dbname and no mention of the term sql anywhere (for that matter there's not much mention of hdb or bdb either). The output is attached, if the list permits attachments that is... You're not looking carefully enough. It certainly has olcObjectClasses: ... olcHdbConfig 'HDB backend configuration' It certainly does not have BDB or SQL, meaning neither of those backends are present in your slapd. Perhaps they are in dynamic modules that you haven't loaded. Otherwise, you simply don't have them available on that install. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Deprecation of back-bdb
Clément OUDOT wrote: 2014-02-03 Quanah Gibson-Mount qua...@zimbra.com mailto:qua...@zimbra.com: --On Monday, February 03, 2014 4:48 PM +0100 Clément OUDOT clem.ou...@gmail.com mailto:clem.ou...@gmail.com wrote: Hi, as Howard said in his conference at FOSDEM, back-bdb is deprecated, and is disabled by default. Should I fill an ITS? No. It is deprecated in 2.5 and later. If I remember well, BDB should be deprecated for 2.4 and HDB deprecated for 2.5. See slide 37 of this presentation: http://fr.slideshare.net/ldapcon/whats-new-in-openldap Indeed. We disabled back-bdb in HEAD September 2008. commit 6dfa7b760e14a051ee15ea6fac5cc1784ef0883b Apparently that was not propagated to the RE24 configure.in. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Slow to add 1 million items
Brent Bice wrote: On 02/05/2014 11:45 AM, Quanah Gibson-Mount wrote: Ok. I had some wierd perf issues with Ubuntu10 and Ubuntu12. I was able to resolve them on Ubuntu12 by upgrading to a newer 3.x kernel (currently 3.10.11-031011-generic on my Ubuntu12 box). Also, it isn't necessary to use ext4 (I use ext2 for the ldap data with mdb). ext4 in the 2.x kernel has some serious perf problems as well (when using writemap). Yeah, what he said. :-) It's also worth compiling from source rather than just using whatever version of whatever binaries some distro provides. It's not hard and that way you have control over exactly what version of OpenLDAP you run regardless of distro. BTW, I was using xfs for the OpenLDAP instance I sorta optimized for writes that I benchmarked recently. I think Howard and/or Quanah did a bunch of benchmarking of the MDB back-end on a whole bunch of filesystems and ext2 won. I use XFS simply cuz a modern version of XFS is decently fast, yields more usable space than ext4, and I really wanted a journaled filesystem. OpenLDAP on XFS was good 'nuff for my purposes, but it's prob'ly significantly faster on ext2 (from what I read). Journaling may be good for other uses, but it's completely redundant for LMDB. If you have an entire partition to dedicate to the purpose, you're still better off just using ext2. xfs and some of these other filesystems can be faster when allocating new space (extent-based allocation) but if you manually preallocate the space for the DB file that's also moot. With an ext2 filesystem, mounted with noatime, and preallocated space, LMDB can run with zero metadata update overhead. That makes all writes faster and makes it immune to filesystem corruption, thus rendering fsck irrelevant. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Recommended version of BDB package
Mark Cairney wrote: On 07/02/2014 17:38, Quanah Gibson-Mount wrote: --On Friday, February 07, 2014 5:17 PM + Mark Cairney mark.cair...@ed.ac.uk wrote: Hi, Having recently experienced some issues with our Live server (2.4.38, Scientific Linux 6.4 64-bit) I was looking into possible causes and found the following Debian bug report which suggests that BDB 5.X is problematic with OpenLDAP. Well, there are lots of 5.x releases, you'd need to be more specific than that. I've used BDB 5.2.36 for years w/o issue. Apologies, I thought I'd stated it in my first email but on reading it back I haven't. It's BDB 5.3.21 I've tested all the way up to 6.0.20. Didn't notice anything good or bad about any of them, and all are slower than LMDB. Your ITS#7657 has gotten no attention, due to lack of interest. Aliases are a rather stupid part of LDAP; most other directory vendors don't even bother to implement them. You might be able to revive interest in that ITS by providing a complete test case (config+data) that demonstrates the issue. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapd-meta exclude syntax; larger problem with rwm, collective attributes.
Dan Pritts wrote: As I posted last week I am trying to put a proxy in front of Active Directory. AD has most of the required attributes for my application, but I need to fill in a couple that are missing. Translucent proxy makes sense, combined with the collect overlay. Unfortunately, slapd crashes when it encounters a DN from AD that has one of the collect attributes (its 7797). Not just a lookup failure, a hard crash. :( That's expected, the collective attribute is not allowed to already exist. Crash reports without stack traces are also less than helpful... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Have you seen this FUD - IT pros suffer OpenLDAP configuration headaches ?
harry.j...@arcor.de wrote:
#!/bin/sed -rf
# Author: Harry Jede
# produce human readable but still machine parseable
# olcAccess lines and removes the ordering numbers in {}
# because humans don't need them, really.
Nice formatting, but just a note - the ordering prefixes are there to allow
you to insert new ACLs in the precise spot where they belong. So in fact,
human or machine, they are necessary. We wouldn't throw things in there for no
reason.
Otherwise, to insert one rule in front of existing rules, you would need to
delete and reinsert all of the rules.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: SASL DIGEST-MD5 works but PLAIN/LOGIN fails
Michael Ströder wrote: The trace shows for PLAIN or LOGIN (running slapd -d config,stats,stats2,acl,args,trace): SASL [conn=1002] Error: unknown password verifier(s) slapd My /usr/lib64/sasl.conf contains: snip pwcheck_method: slapd mech_list: plain login digest-md5 external snip I've checked that this is the right file by setting pwcheck_method: foobar which appears in the logs then. There is no such pwcheck_method. The SASL error message is correct. Read the Cyrus SASL docs for pwcheck_method. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: SASL DIGEST-MD5 works but PLAIN/LOGIN fails
Philip Guenther wrote: On Sun, 9 Feb 2014, Dieter Klünter wrote: Am Sat, 08 Feb 2014 23:18:22 +0100 schrieb Michael Ströder mich...@stroeder.com: Dieter Klünter wrote: Am Sat, 08 Feb 2014 11:45:52 +0100 schrieb Michael Ströder mich...@stroeder.com: [...] Yes, got that working in the meantime. But unfortunately this only works with clear-text userPassword values. That's definitely not what I want. Than you have to rely on an external password store, i.e. kerberos. But that all is security by obscurity if you don't harden the network and services. At least at one time, cyrus-sasl supported the ability to request from the auxprop mechanism alternate types of data that would be sufficient for doing the server-side of the SASL exchange. For example, with CRAM-MD5 it could store the HMAC-MD5 context keyed with the user password (c.f. RFC2195), which is sufficient for calculating the expected digest. The cyrus-sasl code would request both that form and the cleartext and then use whichever it got. Of course, for both cram-md5 and digest-md5, the prehash form was sufficent to authenticate using those as the client, so the security gain was negligible: yeah, you didn't have the 'real' cleartext form, but you could still login as the user. Add on the complexity of keeping that data up to date and the benefit of this seems complexity seems hard to justify. I believe that that property of the server authenticator being sufficent to authenticate as the client is *not* true of some newer mechs: iirc, SRP uses some form of zero-knowledge proof such that the server doesn't need to store a password-equivalent. That might also be the case with the SCRAM-* family. Still, there are at least two problems with that as a solution: 1) can you really require the SASL clients in your setup to use SRP? (if it's that easy to require new mechanisms, I would expect more use of Kerberos!) 2) does the openldap 'slapd' auxprop support lookup of those alternate data types as other attributes? If so, what/where's the schema for them? The slapd auxprop simply looks up whatever attribute names SASL passed in. If there's no schema definition for these attributes, naturally those lookups will fail but the failure is ignored, the auxprop just returns as much as it knows about. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: TLS authentication broken in Ubuntu 12.04
Ali Gholami wrote: Thanks Quanah, I could resolve the error but the error message was not helpful. I stopped the apparmor service and used strace to debug. I realized the server certificate path was not defined correctly to be loaded. I think p11-kit: couldn't list directory: /etc/pkcs11/modules: Permission denied is not really the correct error message. It should be something like certificate not found etc. Send a bug report to Ubuntu then, this error message comes from their GnuTLS library, not from OpenLDAP. Ali On 02/10/2014 10:09 PM, Quanah Gibson-Mount wrote: --On Sunday, February 09, 2014 11:49 PM +0100 Ali Gholami ghol...@kth.se wrote: I used the debug mode: --- slapd -d 2 52f80527 @(#) $OpenLDAP: slapd (Sep 19 2013 22:39:38) $ buildd@panlong:/build/buildd/openldap-2.4.28/debian/build/servers/slapd p11-kit: couldn't list directory: /etc/pkcs11/modules: Permission denied 52f80527 main: TLS init def ctx failed: -1 52f80527 slapd stopped. 52f80527 connections_destroy: nothing to destroy. --- Does anyone know why TLS ctx fails to initialize? Because it gets permission denied when trying to access /etc/pkcs11/modules, exactly as it states. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapo-accesslog
Michael wrote: On Feb 12, 2014, at 3:07 AM, Dieter Klünter die...@dkluenter.de wrote: Am Tue, 11 Feb 2014 23:56:23 -0500 schrieb Michael mlstarlin...@hotmail.com: I'm trying to configure a means to be able to get the lastlogin time for users in my environment. However, since I'm stuck using the RHEL version of OpenLDAP I can't take advantage of the lastbind overlay Well that jut put a damper on my day. Do you have any other suggestions on how to capture all BINDS in a particular environment? Ditch the lobotomized build that RedHat provides and compile it yourself. It will work better than the RedHat build anyway, since they've crippled theirs so much. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: Use the current date/timestamp in filters
Ulrich Windl wrote: Hi! ... more useless blather. Michael's response was correct and sufficient. When you have no idea what you're talking about, you should listen not speak. Michael Strödermich...@stroeder.com schrieb am 12.02.2014 um 22:12 in Nachricht 52fbe3b0.7040...@stroeder.com: Laurent Meunier wrote: I'm trying to build a ldap query with the current date or the current timestamp (something like myAttributeDate = now()). All solutions I've found on Internet require to compute the current date in a script and then build the ldap query with the computed date. Unfortunately, I can't do this because the ldap query will be in a static file: ldap_filter in /etc/saslauthd.conf. Is it possible with OpenLDAP to use the current date or timestamp in a ldap query without compute it in a script? I vaguely remember that Daniel Pluta contributed patches for his I-D. http://tools.ietf.org/html/draft-pluta-ldap-srv-side-current-time-match Maybe you should dig in the ITS. Ciao, Michael. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LMDB test assertion failures on Linux/MIPS
Martin Lucina wrote: Hi, I've recently started using LMDB in a new project targeted to MIPS (and later also ARM) architectures. While developing my storage code I found that test cases which execute perfectly on x86 were failing with assertions inside LMDB on mips and mipsel devices. Investigating further, I've found that the mtest test program included with LMDB also fails. Specifically, the failures I'm seeing are any of the following: mdb.c:2635: Assertion 'pglast = env-me_pglast' failed in mdb_freelist_save() or mdb.c:5100: Assertion 'IS_BRANCH(mc-mc_pg[mc-mc_top])' failed in mdb_cursor_sibling() or mdb.c:5176: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() or mdb.c:1713: Assertion 'rc == 0' failed in mdb_page_dirty() etc. The failures are intermittent in that there's about a 50% chance mtest will complete successfully. The line numbers are against LMDB 0.9.11 source. Target devices and toolchain combinations I have tried (all fail): - mips-sf-linux-musl cross compilers from musl.codu.org, musl libc 0.9.15. Atheros AR9344 cpu, OpenWRT trunk (Linux 3.10.28). - mipsel-sf-linux-musl cross compilers from musl.codu.org, musl libc 0.9.15, Broadcom BCM4706, Tomato firmware by shibby (Linux 2.6.22.x based on ASUS SDK). In all cases the code fails both with and without optimization, and is compiled as a static executable. Consulting with a friend who is also developing on MIPS devices he said he ran into similar problems when trying LMDB. He suggested that I check for problems with unaligned memory accesses; indeed compiling LMDB with -Wcast-align produces many warnings regarding such accesses. My knowledge of LMDB and MIPS internals is not up to debugging this but it seems like unaligned accesses may(?) be the underlying cause. That seems pretty unlikely, considering that the code works fine on SPARC which also has quite stringent alignment requirements. I presume you're compiling a 32 bit binary, not 64 bit. Perhaps shorts are bigger than 16 bits on your platform? What is the status of LMDB on MIPS? Has anyone tried it? Any help would be much appreciated. Martin -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: ldap/pwd complexity and PAM?
Doug OLeary wrote: Hi; In my free time, I've been studying openldap and the ppolicy overlay. I started working on password complexity today. While searching for information on implementing complexity, I ran across the link immediately following which seems to indicate that openldap honors the settings in /etc/pam.d/password-auth. http://ubuntuforums.org/showthread.php?t=2172393 No, that's not what that thread says at all. I tried configuring that on a test kvm and can't even get it working with local accounts so obviously I borked something in the password-auth file - like maybe not even the right pam.d file; however, before I spend a whole lot of time troubleshooting this, is my understanding accurate? Will openldap honor the settings in pam.d? No, OpenLDAP doesn't know anything about PAM settings. All that that thread is saying is that you must configure PAM correctly if you want PAM to enforce password quality *when you change passwords using PAM*. If you change LDAP passwords via LDAP, PAM is nowhere in the picture. It seems that'd be a whole lot cleaner and more supportable than compiling a specialized password checking module. Any info greatly appreciated. Thanks for your time. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LMDB test assertion failures on Linux/MIPS
Martin Lucina wrote: rich...@nbcs.rutgers.edu said: Rebuilt with the Sun compilers to get proper debug info, the faulting instruction is the same as the one in GCC: signal SEGV (no mapping at the fault address) in mdb_env_pick_meta at line 3349 in file mdb.c 3349 return (env-me_metas[0]-mm_txnid env-me_metas[1]-mm_txnid); Martin I don't have MIPS specsheets memorized/in front of me, but what's the pagesize? http://www.openldap.org/lists/openldap-devel/201310/msg5.html (and ITS#7713) 4k on the MIPS boards I'm using. The SPARC I tried on is 8k, but LMDB 0.9.11 from git uses sysconf(_SC_PAGE_SIZE) to get the page size at run time so that shouldn't be a problem. The SPARC and MIPS problems may or may not be related - I just tried SPARC since Howard mentioned it worked. Do you know a known-good version on SPARC? If so I can work from that and see if I can find the commits that introduced the problem. I just did a fresh build of 32 bit SPARC Solaris 10 with gcc 4.4.0 and mtest works fine. I get a number of warnings if I use -Wcast-align but in this case they're irrelevant. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: strategy for getting groupOfNames (AD) and posixAccount (Unix) to coexist?
Philip Colmer wrote: This was an area where I also got stuck when researching this last year. My conclusions were: 1. UNIX needs group membership to be UIDs and not DNs, so attempts to use a class that defines members with DNs are likely to fail. Nonsense. nss_ldap, nss-pam-ldapd, and nssov all support RFC2307bis. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: dynlist groups not usable in ACLs?
DRVTiny wrote: OpenLDAP 2.4.39, amd64, debian 7 When i use the group with only static members in by group/groupOfNames/member clause - all works perfectly But when i'm trying to use in ACL definition dynamic members in 1:1 identicaly group - it doesnt work at all and in slapd debug output i see: --- 530b1a22 dnMatch -40 dc=ru uid=konovalov-aa,ou=people,dc=svc,dc=ot,dc=ru --- where dc=ru is one static member of this group (all others is dynamic members and it is not compared to uid=konovalov-aa,ou=people,dc=svc,dc=ot,dc=ru at all). It is very strange behavior, because official documentation says that: --- Dynamic Groups are also supported in Access Control. Please see slapo-dynlist(5) and the Dynamic Lists overlay section. --- Any comments? Can i use dynlist'ed groups in OpenLDAP ACL? Yes, you can. But you cannot use group/groupOfNames for a dynamic group. This is already documented in the manpage. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LMDB - growing the database
Nat! wrote: Am 24.02.2014 um 04:21 schrieb b...@bitrate.net: generally speaking, i’d discourage you from looking at that limit from the perspective of “how large will my data be?”. instead, consider it a safeguard, for the os/environment. evaluate your particular environment, and use values amongst your various instances such that, were something unexpected to happen, the entire disk/partition/etc is not consumed to the point of choking out the os [or perhaps other more important processes, etc]. -ben I think this is valid, if you're thinking in terms of this is my database and this is my server where it runs on. I am more trying to use lmdb as a persistable hashtable, that I could put into a variety of my applications that me and other people would use. I have no idea beforehand, what the use is going to be on other peoples devices and most probably the other people wouldn't know either. Currently I am making a clone of the environment and then create a new bigger environment and copy from small into big. This seems to work so far, but it's just doesn't feel right to me. Certainly haven't seen the behavior you describe, but I seldom test on MacOS or HFS+. I would use FFS, since it supports sparse files. On Windows, Linux, and FreeBSD, there's no problem increasing the mapsize and preserving the existing data. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Replication from OpenLDAP to Fedora 389 DS
Italo Valcy wrote: Hi Dieter, Thanks for the tip! Actually the problem here is not which attributes are get by ryncrepl from the provider. The problem is that the backend seems to insert the attributes again and then send to the consumer (lastmod on, which is mandatory to syncrepl), do you see? For example, I've already tried attrs=* and exattrs=structuralObjectClass,entryUUID,entryCSN,creatorsName,createTimestamp,modifiersName,modifyTimestamp, however both fails to syncronize with the consumer because the backend seems to insert these attributes again (IMO because of the mandatory lastmod on). I'm using the same ideia from OpenLDAP replication document (proxy mode). Syncrepl requires entryCSN and entryUUID, it cannot work without them. You must add them to your 389DS schema if you want to attempt this. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: memberOf values case
Matteo Perego wrote: Hi, I know that openldap usage behaviour is case insensitive. False. OpenLDAP conforms to the schema. If an attribute is defined to be case-insensitive in the schema, then it will be treated as such. If it is defined to be case-sensitive, that will also be done. In my case a client use ldap attribute values in a case sensitive way. Your client should use attributes that are actually defined as case sensitive then. Fix your schema or fix your clients. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: LMDB get value
Markus Doppelbauer wrote: Hello, Test case (modified sample-mdb.c): http://doppelbauer.name/bug.c.txt If I run this proram the first time, LMDB finds the key. The second time, mdb_cursor_get() returns NOTFOUND. How to fix this issue? You're using an invalid dbi the second time, you didn't call mdb_dbi_open(). Thanks a lot Markus -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Fwd: Re: Aw: Re: LMDB get value
Original Message Subject: Re: Aw: Re: LMDB get value Date: Sat, 01 Mar 2014 13:39:47 -0800 From: Howard Chu h...@symas.com To: Markus Doppelbauer doppelba...@gmx.net CC: Markus Doppelbauer wrote: You are right. Thanks a lot! An other question. Is there a define-option to compile LMDB without a dependency to libpthread? (We will replace Daniel Bernsteins CDB for performance reasons with LMDB - so locking is not required). No, nor is such a flag needed. LMDB readers do no locking anyway. If you're replacing CDB, then none of this matters. Thanks a lot Markus *Gesendet:* Samstag, 01. März 2014 um 17:04 Uhr *Von:* Howard Chu h...@symas.com *An:* Markus Doppelbauer doppelba...@gmx.net, openldap-technical@openldap.org *Betreff:* Re: LMDB get value Markus Doppelbauer wrote: Hello, Test case (modified sample-mdb.c): http://doppelbauer.name/bug.c.txt If I run this proram the first time, LMDB finds the key. The second time, mdb_cursor_get() returns NOTFOUND. How to fix this issue? You're using an invalid dbi the second time, you didn't call mdb_dbi_open(). Thanks a lot Markus -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: NEW LDAP PROJECT
Borresen, John - 0442 - MITLL wrote: All, There is a new project; a group that I support will be using OpenAM to manage single sign-on (SSO). The environment is outward facing, where multiple entities (outside users) will log on to a web portal via openam/opensso. The OpenAM will query the OpenLDAP (currently it is an embedded OpenDJ implementation) for user information. Based on their user credentials they will, hopefully, be forwarded to either a Production, Development or a Demonstration environment. The group wants to migrate to OpenLDAP as OpenDJ is 1) not friendly 2) very slow. I've been looking around at the OpenAM/OpenDJ configuration and most of the schemas are specifically legacy Sun Microsystems Java specific. Basically, the OpenLDAP will be the User-store for OpenAM. 1) Should I migrate those schemas and everything else currently in OpenDJ over to OpenLDAP and import them? 2) What is the recommended methodology that I should follow to best implement the above scenario? Page 200 of the Packt book on OpenAM shows the steps needed to work with OpenLDAP. http://www.packtpub.com/openam/book Any pointers are much appreciated. Thanks, John D. Borresen (Dave) Linux/Unix Systems Administrator MIT Lincoln Laboratory Surveillance Systems Group 244 Wood St Lexington, MA 02420 Email: john.borre...@ll.mit.edu -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Critical GnuTLS bug ...
http://arstechnica.com/security/2014/03/critical-crypto-bug-leaves-linux-hundreds-of-apps-open-to-eavesdropping/ Perhaps folks will take us more seriously the next time we say don't use GnuTLS ... http://www.openldap.org/lists/openldap-devel/200802/msg00072.html -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Restricting access based on IP Address
kevin sullivan wrote: Hi, I am running an OpenLDAP server version 2.4.23 and I would like to restrict a user from connecting unless they are connecting via an ldapi connection or localhost. Specifically, I would like to only let the rootdn manage things from localhost or from an ldapi connection, which ensures that they are on localhost. I do not want to prevent other users from connecting to my LDAP server via an ldaps connection from anywhere on the network. rootdn ignores all access controls. Is this possible? I have read a good bit about access control directives, but I haven't seen what I am looking for. I am guessing that what I am looking for probably deals with 'sockname' or 'sockurl', but I don't know how to use those statements to properly configure slapd. Thanks, Kevin -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Critical GnuTLS bug ...
Michael Ströder wrote: Howard Chu wrote: http://arstechnica.com/security/2014/03/critical-crypto-bug-leaves-linux-hundreds-of-apps-open-to-eavesdropping/ Perhaps folks will take us more seriously the next time we say don't use GnuTLS ... http://www.openldap.org/lists/openldap-devel/200802/msg00072.html While I personally also prefer OpenSSL over GnUTLS it's not fair to blame developers if they publish a security issue themselves. This issue was found by a RedHat audit, not by the GnuTLS developers. The same underlying problem remains - the GnuTLS developers didn't know the first thing about X.509 certificates. They pointedly ignored (or were simply too inexperienced to even understand) the issues that were identified. And apparently, they still haven't learned, after all this time. One never knows which issues are in other preferred software packages which the developers are not honest enough to talk about. Ciao, Michael. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Slapd TLS issue
Eric Falbe wrote: Yes, the openldap rpm was just updated, but it did not take effect until the slapd deamon was restarted. I have not explicitly tried to use the Mozilla NSS database, I did not use the TLSCADIR(?) attribute and instead used: olcTLSCertificateFile , olcTLSCertificateKeyFile, and olcTLSCACertificateFile. I will look into that bug and the documentation you pointed me at. For the record, RedHat uses Mozilla NSS, not GnuTLS. But regardless, neither is recommended. Quoting from the bug report linked below: https://bugzilla.redhat.com/show_bug.cgi?id=707599#c56 Finally, I have a solution, there were too many bugs which were complicating this: The referenced bugs were eventually fixed, but myriad problems remain and MozNSS itself is fundamentally broken by design; or rather, it was designed for single-user web browsers and was never meant to be used as a system library that multi-user services depend on. If you enjoy pounding square pegs into round holes, you can keep trying to use OpenLDAP as built by RedHat, but most sensible people will use something that's actually fit for the purpose. Thanks Eric Falbe On Thu, Mar 6, 2014 at 5:29 PM, Terje Trane ter...@funcom.com mailto:ter...@funcom.com wrote: On 05.03.2014 22:27, Eric Falbe wrote: I have attempted to rebuild the database backend (with slapcat and slapadd), but am still getting this same error. I have my ssl (self-signed) certificates located in /etc/pki/tls/certs/ldap.__cassens.com.pem /etc/pki/tls/tls/certa/ca.pem /etc/pki/tls/private/ldap.__cassens.comKey.pem These certificates worked fine up untill today, does anyone have any insight on where to look to being troubleshooting this issue? Just a guess, but was the openldap rpm just updated? (or the service just restarted for the first time after a previous update). Could this be related to RedHat/CentOS rpms deciding to start using GnuTLS instead of OpenSSL? Try searching in their bug databases. E.g.: https://bugzilla.redhat.com/__show_bug.cgi?id=707599 https://bugzilla.redhat.com/show_bug.cgi?id=707599 --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: User-Managed Groups
Tim Gustafson wrote:
Hi,
I've had OpenLDAP set up for a while now such that users can create
groups and manage the groups that they've created. I've achieved this
by creating a new objectClass (called managedGroup) which adds the
manager attribute, and then I've set up ACLs like this:
olcAccess: {14}to dn.base=ou=Groups,dc=whatever
attrs=children
by users add
by * break
olcAccess: {15}to dn.subtree=ou=Groups,dc=whatever
filter=((objectClass=posixGroup)(objectClass=managedGroup)(gidNumber=1000))
attrs=entry
by users add
by * break
olcAccess: {16}to dn.subtree=ou=Groups,dc=whatever
attrs=cn,manager,memberUid,description
by set.exact=this/manager user write
by * break
I also have the unique overlay installed to prevent multiple groups
from having the same cn or gidNumber.
I've got a request from users to be able to re-name their groups now
too. I tried changing by users add to by users write in clause
14, and added the entry attribute to attrs= in clause 16, but the
server is still not letting users re-name their groups. The output of
the log file looks like this:
slapd[44745]: = acl_get: [16] attr entry
slapd[44745]: = acl_mask: access to entry
cn=test-1234,ou=Groups,dc=whatever, attr entry requested
slapd[44745]: = acl_mask: to all values by
uid=g-guest,ou=people,dc=whatever, (=0)
slapd[44745]: = check a_dn_pat: users
slapd[44745]: = acl_mask: [1] applying add(=arscxd) (stop)
slapd[44745]: = acl_mask: [1] mask: add(=arscxd)
slapd[44745]: = slap_access_allowed: write access denied by add(=arscxd)
slapd[44745]: = access_allowed: no more rules
What am I missing?
Read the Operation Requirements section of slapd.access(5).
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: How get more detailed error information than the provided by ldap_err2string?
Werner - Google wrote:
Hi,
I've the sample code bellow, and when i intentionally put the wrong
credentials, i get from ldap_err2string( rc ) the error message:
Invalid credentials.
Read the ldap_get_option(3), ldap_bind(3), and ldap_result(3) manpages.
But monitoring network traffic with wireshark, i can see that on the
bindResponse packet that returns from the server, i also get a more detailed
message. In my/this test case,
errorMessage: 80090308: LdapErr: DSID-0C0903AA, comment:
AcceptSecurityContext error, data 525, v1772
attached is also an image of the wireshark showing what i mean.
Question is, is there a way i could retrieve this more detailed message?
Thanks in advance for any help
Regards.
---
sample code:
if ( (ld = (LDAP *)ldap_init( pHostName, iPortNum )) == NULL ) {
perror( ldap_init failed. Reason?: );
exit ( 1 );
}
if ( (rc=ldap_set_option( ld, LDAP_OPT_PROTOCOL_VERSION, version)) !=
LDAP_SUCCESS ){
fprintf( stderr, ldap_set_option(LDAP_OPT_PROTOCOL_VERSION): %s\n,
ldap_err2string( rc ) );
exit( 1 );
}
if ( (rc=ldap_set_option(ld, LDAP_OPT_REFERRALS, LDAP_OPT_OFF)) !=
LDAP_SUCCESS){
fprintf( stderr, ldap_set_option(LDAP_OPT_REFERRALS): %s\n, ldap_err2string(
rc ));
exit( 1 );
}
rc = ldap_simple_bind_s( ld, auth_dn, auth_pw );
if ( rc != LDAP_SUCCESS ) {
fprintf( stderr, ldap_simple_bind_s() Failed: %s [%d]\n,
ldap_err2string(rc), rc);
ldap_unbind_s(ld); /* try unbind the failed connection anyway */
exit ( 1 );
}
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Fwd: Works with ldapsearch, but can't get it to work with ldap_search_ext_s()
Werner - Google wrote: Hi, After doing some more research and lot's of tcpdumping, i got some more info, but still don't know how to solve my problem. As i mentioned, i'm doing the search agains an Active Directory service. I do the serch with the code showed above, and with the tcpdump's i seems to show that when i do it with ldap_search_ext_s(), i get after the found item, a list of referals, like: dap://ForestDnsZones.example.dc/DC=ForestDnsZones,DC=example,DC=dc ldap://DomainDnsZones.example.dc/DC=DomainDnsZones,DC=example,DC=dc ldap://example.dc/CN=Configuration,DC=example,DC=dc and ldap_search_ext_s() tries to follow those referrals. On this attempt it tries the bind without the credentials, and than i get rejected by the server of sometimes i'm even unable to reach the mentioned server. I've tried adding to my test code, int referals = LDAP_OPT_OFF; /* before the init */ ldap_set_option( NULL, LDAP_OPT_REFERRALS, referals); as it seems to have no effect, i still get my test code trying to 'follow' the referals, i did try also put it as: /* after the ldap_init , using the returned LDAP* */ ldap_set_option( ld, LDAP_OPT_REFERRALS, referals); but still no effect. Sounds like you're not using OpenLDAP's libldap. What version of LDAP library are you actually using? And doing the same tcpdumping and running ldapsearch -d ... , it appears that ldapsearch, using the same search parameters as my test code, does NOT try to follow the referrals, even getting them back from the server the same as my test code. As additional info, the wireshark summary of the search return packet is something like: No. Time SourceDestination Protocol Length Info 97 4.8103690009.9.9.9 5.5.5.5 LDAP 405 searchResEntry(2) CN=Alonso.Vega,OU=Users,OU=Country,DC=example,DC=dc | searchResDone(2) Unknown result(9) (Referral: dap://ForestDnsZones.example.dc/DC=ForestDnsZones,DC=example,DC=dc ldap://DomainDnsZones.example.dc/DC=DomainDnsZones,DC=example,DC=dc ldap://example.dc/CN=Configuration,DC=example,DC=dc) [1 result] Any suggestion/help very appreciated on how i could avoid that the search tries to follow the referrals? Copy the code that the ldapsearch tool uses. You're using obsolete APIs in your code. thx -werner -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: user can't login via LDAP
accounts. [me@home:~] #ssh -qt spen...@ldap01.example.com mailto:spen...@ldap01.example.com spen...@ldap01.example.com mailto:spen...@ldap01.example.com's password: Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.2.0-54-virtual x86_64) [me@home~] #ssh -qt l...@ldap01.example.com mailto:l...@ldap01.example.com l...@ldap01.example.com mailto:l...@ldap01.example.com's password: Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.2.0-54-virtual x86_64) Again I can verify that neither account is local to the ldap server: [root@ldap01:~] #egrep (spencer|leo) /etc/passwd [root@ldap01:~] # Here's what my nsswitch looks like on the monitoring host (where spencer can't login but leo can): [root@monitor:~] #grep -v # /etc/nsswitch.conf passwd: files sss shadow: files sss group: files sss hosts: files dns I'm just wondering if there might be a problem in the config or what I can possibly do to nail down the source of the problem. Thanks Tim -- GPG me!! gpg --keyserver pool.sks-keyservers.net http://pool.sks-keyservers.net --recv-keys F186197B -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: open(ldap|ssh) interaction
Doug OLeary wrote: Hey; When using local accounts, ssh honors password expiration even if using public key authentication. This is the case at least on HPUX, Solaris, and various flavors of Linux. This is a good thing. I won't go through all the security reasons why passwords should periodically change. Suffice to say that they should and most companies have policies regarding password expiration. When using openldap, however, if a user is configured to use public key authentication, he is allowed access to the account regardless of the password aging and/or pwdReset parameter. Is there a way to force openssh to honor these settings like it does for local accounts? If you want to know how to control OpenSSH settings, it seems to me you should ask on an OpenSSH mailing list. Test environment is centos6.5 running on a kvm tying into an openldap server ver 2.4.23. My test environment is certainly following the symptoms of my client's unboundid server supporting a variety of linux platforms - all rhel based - from ver 4 through 6. Any help greatly appreciated. Doug O'Leary Senior UNIX/Security Admin CISSP, CISA, RHCSA, CEH O'Leary Computers Inc dkole...@olearycomputers.com (w) 630-904-6098 (c) 630-248-2749 linkedin: http://www.linkedin.com/in/dkoleary resume: http://www.olearycomputers.com/resume.html -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: LMDB test assertion failures on Linux/MIPS
Martin Lucina wrote: ulrich.wi...@rz.uni-regensburg.de said: Hi! I think a problem with your test program is that you don't wait for the write() thread to finish before you try to read the mmap(). See how locking on a producer-consumer (or reader-writer) relationship is usually implemented (If you don't have it ready, I could send you the algorithms). That shoudln't matter. More to the point, you don't wait for the write() thread to *start* - there's no guarantee that it will actually start running as soon as the barrier is released. A valid test has to know that the write() thread actually got scheduled and ran. The write thread opens the file descriptor with O_DSYNC, and all the test program is trying to verify is that the mmap eventually becomes consistent. You can ignore the pthread_barrier stuff, that just tries to eliminate thread creation time from the equation. Now that I think about the output, there is a fairly obvious explanation for the numbers - 1 means the writer thread got scheduled first. However I'm still not sure why the # of iterations on MIPS is so high - scheduling resolution on the box maybe? Martin -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: Regarding LDAP structure
that the model is flexible and adaptable over time, meaning that you don't have to get it right from the start and should be able to evolve your DIT over time, provided of course that your toolset is adequate. Web-based tools such as LAM for example are almost hard-wired into a People, Computer, Group paradigm whereas tools like PHPLDAPAdmin are more flexible but less intuitive. The latter provides a template mechanism which allows for easy customization to a particular implementation, but I think both (as almost all popular LDAP browsers/admin tools) are dumb in terms of moddn and modrdn so you need to hack them to work correctly with more complex implementations. Anyway, the point is that your entries should be organized anyway you want. I have done implementations where we can actually traverse the DIT in a hierarchical manner (e.g. by units and departments with people at different levels of the tree) but that can ALSO be queried by means of a common attribute(s). So you can actually have it both ways. I always prefer to model the DIT to reality and then group the entries by attributes to simplify queries. This gives you the best of both worlds as you can query at any level/branch and also allows to implement a DDS more easily. Actually I encourage mixing the X.500 with the Domain-based We have a very well documented reference implementation for an educational institution and we would happily share in a Wiki somewhere. Perhaps we can find a place where people can contribute reference implementations for different implementations and that allows for discussions, etc. Any idea where to post these?? Best, Alejandro Imass Yabarana Corporation -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: Regarding LDAP structure
Ulrich Windl wrote: Howard Chu h...@symas.com schrieb am 14.03.2014 um 10:36 in Nachricht 5322cd84.4080...@symas.com: Ulrich Windl wrote: Hi! I have a question on entryUUID: Most (comonly used) group-like structures use DNs for members. Are there any examples how to use entryUUID for group-like structures? There are no standard schema that do this. I'd note that your question and Alejandro's recommendation are contrary to the design of LDAP (and the X.500 data model) - LDAP is meant to be a read-optimized hierarchical data store. If you simply use entryUUIDs for all references then you might as well use a flat or relational database instead of a hierarchical one. Up to here I see no difference. Then you don't understand the difference between hierarchical namespaces and flat namespaces. I suppose you're not using any Unix filesystems very much either. In particular, listing memberships by DN gives you immediate knowledge of an entry's location in the hierarchy, and clients can use DN's for direct access to any entry of interest. Using entryUUID requires you to do a search, instead of a direct lookup. I agree here. There's of course a maintenance cost for using DNs as references - when DNs are changed, you might also need to change every entry that references them, which makes updates more expensive. But again, that's part of the LDAP design: writes can be more expensive, because reads must be as fast as possible. I tend to disagree: I think the DIT designers mixed up names and IDs right from the beginning. I guess that's why every entry has a DN, and not a DID (Distinquished ID). To me it seems that did not foresee that a DN might change. Maybe it was due to UUIDs not being used at that time. Today you can learn for the web trackers how to manage IDs correctly ;-) Maybe they new the DIT schema would be less attractive if you had non-speaking DIDs instead of DNs rich of semantics. But that virtual attractiveness seems to be a major problem: What happens if dn: cn=Jane Smith, ou=people, o=example.org gets married or divorced? Your entire question is negated by the fact that X.500 provided a ModifyDN operation from the very beginning. The point you're failing to realize is that nobody said that cn must be the distinguished naming attribute of an entry. This is also a distinguishing characteristic of M$ AD that differentiates it from true LDAP implementations - in AD, references are stored internally as GUIDs, and the GUID must be mapped to a name on every read operation. Thus they avoid the expense of referential integrity updates when DNs change, but as a consequence, read operations in AD are slower than writes. It's not a tradeoff that makes sense for most LDAP uses, but M$ AD is not a shining example of good design anyway. ease of use, not ease of implementation they say. Nonsense. Read optimization vs write optimization. AD chose to favor writes over reads, LDAP/X.500 is intended to favor reads over writes. M$ AD chose the wrong option. (It's all my opinion) Regards, Ulrich Regards, Ulrich Alejandro Imass aim...@yabarana.com schrieb am 13.03.2014 um 20:10 in Nachricht CAHieY7TFQne1gyGe9-fWH3oYzZy-oBOQSha=klkwowyzv3k...@mail.gmail.com: On Thu, Mar 13, 2014 at 12:18 PM, Joshua Riffle jrif...@apu.edu wrote: I'm aware this may not be the best mailing list to discuss something as generalized as best practices for LDAP structuring within OpenLDAP, but would anyone be able to direct me to a mailing list that would be better suited for this kind of conversation? I think it's an excellent discussion and I don't see why this list cannot accommodate it. After all, OpenLDAP is currently a reference model in the OSS world for LDAP so it could very well house discussion around reference models for DITs. I'm looking for any or all of these kinds of communications within a mailing list: Designing a person, account, group LDAP tree directory that would be scalable and flexible enough to grow to large sizes (millions) and still have a grip on best practices for identity management on an enterprise level. Usually you should aim towards a DDS (Distributed Directory Service) and all nodes sharing some sort of agreement in the DIT structure although it's not alway necessary. Specifically for an educational institution if I can share the aches and pains of other directory owners with similar problems. I also am trying to prove / disprove the use of having a person directory object with multiple child account objects as good or bad architecture and understand why. I've never seen this discussed in practice. Most LDAP implementations are quite poor and revolve around Posix and/or Windows AD management instead of using more elaborate DIT modelling , aliasing, and the entryUUID operational attribute (RFC 4530). The DIT model is unique to every application but I do agree with you that we should have some reference models that break the traditional People, Computer
Re: Planning migration to mdb
Nick Milas wrote:
Hi,
We have a running openldap installation (2.4.39) - a single master -
with cn=config and hdb backend.
So, config has the branches:
I know we must slapcat our data and slapadd it in mdb afterwards.
The question is: what changes should be done in the config DIT (and how)
so that the config changes from hdb backend to mdb backend?
Is it enough to slapcat config, then change:
olcDatabase from {1}hdb to {1}mdb
and:
objectClass from olcHdbConfig to olcMdbConfig
...?
That's a start.
Anything else?
Can we safely delete all values of the olcDbConfig attribute which
originate from converting from DB_CONFIG?
Not only *can you* - you *must*.
What attributes can we safely remove?
Read the schema definition of the olcHdbConfig objectclass, and compare it to
the definition of the olcMdbConfig objectclass. Delete anything that isn't
present in the olcMdbConfig objectclass.
The schema has everything you need to know. Use it.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: slapd shuts down for no reason
Hallvard Breien Furuseth wrote: On Sat, 2014-03-15 at 17:09 +0100, Christian Kratzer wrote: The message: Mar 15 07:06:25 ldap1 slapd[2798]: daemon: shutdown requested and initiated. you are sseing is from slapd receiving a HUP signal, so something must be sending it. HUP, INT, TERM or TRAP. TRAP is for debugging, not sure why that's included. Or SIGBREAK if it is defined and HAVE_NT_SERVICE_MANAGER is not. Looks like some Windows thing. Maybe it should only be included if defined(HAVE_NT_SERVICE_MANAGER). I think you've misread something, SIGBREAK is already #if defined(HAVE_NT_SERVICE_MANAGER). -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: mdb and bdb
Ulrich Windl wrote: Howard Chu h...@symas.com schrieb am 19.03.2014 um 18:29 in Nachricht 5329d3dc.3060...@symas.com: Quanah Gibson-Mount wrote: --On Wednesday, March 19, 2014 9:16 AM -0300 Friedrich Locke friedrich.lo...@gmail.com wrote: Hi folks! I wonder if, with the increased interest on mdb, the support for bdb will be removed in a near future. Thanks a lot. See the slides from LDAPCon 2013. http://lanyrd.com/2013/ldapcon/sckyhk/ Since Oracle has changed the license of BDB 6 to AGPLv3, most network services that used BDB will be required to migrate away from it. (This also affects Cyrus SASL and Heimdal Kerberos, but we already wrote LMDB drivers for them 3 years ago.) And since BDB is now technologically obsolete anyway, there's no reason to continue to support it. Hi! Why is BDB technologically obsolete? Can you elabporate? I've been doing so for at least the past 3 years. Read http://symas.com/mdb/ Regards, Ulrich Hi Friedrich, bdb and hdb will still exist, but be disabled by default in configure, with OpenLDAP 2.5. Removal would be slated for OpenLDAP 2.6. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: mdb and bdb
Ulrich Windl wrote: Howard Chu h...@symas.com schrieb am 20.03.2014 um 08:14 in Nachricht 532a9562.6070...@symas.com: Ulrich Windl wrote: Howard Chu h...@symas.com schrieb am 19.03.2014 um 18:29 in Nachricht 5329d3dc.3060...@symas.com: Quanah Gibson-Mount wrote: --On Wednesday, March 19, 2014 9:16 AM -0300 Friedrich Locke friedrich.lo...@gmail.com wrote: Hi folks! I wonder if, with the increased interest on mdb, the support for bdb will be removed in a near future. Thanks a lot. See the slides from LDAPCon 2013. http://lanyrd.com/2013/ldapcon/sckyhk/ Since Oracle has changed the license of BDB 6 to AGPLv3, most network services that used BDB will be required to migrate away from it. (This also affects Cyrus SASL and Heimdal Kerberos, but we already wrote LMDB drivers for them 3 years ago.) And since BDB is now technologically obsolete anyway, there's no reason to continue to support it. Hi! Why is BDB technologically obsolete? Can you elabporate? I've been doing so for at least the past 3 years. Read http://symas.com/mdb/ Hi! When reading, you just say that MDB has some features BDB does not have. Does that make BDB obsolete technology? I think it depends on the user's demands. Your reading skills need work. BDB is big, slow, prone to deadlock and corruption, and complex to configure and tune. It has only gotten moreso in each subsequent release. Considering that the purpose of a database is to safely, reliably, and efficiently store and retrieve data, BDB is by every measure obsolete. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Antw: Re: mdb and bdb
Florian Weimer wrote: * Ulrich Windl: When reading, you just say that MDB has some features BDB does not have. Does that make BDB obsolete technology? I think it depends on the user's demands. Berkeley DB supports arbitrary key sizes. For some use cases, this is quite handy, but it's apparently not relevant to OpenLDAP. True, not relevant. BDB supports keys of up to 2^32 bytes, as well as values up to 2^32 bytes. It costs quite a great deal of complexity to support such large keys, and there are very few cases that will ever use keys of such size. For any DB library to perform well, keys must be small enough to fit entirely in RAM and be compared quickly. Keys only need to be large enough to uniquely distinguish one value from another. For any data set that can be calculated by today's technology, this means keys need no more than 64 unique bits. Very large keys implies a bad data model design. Multiple concurrent writers are nice on paper, but probably are not worth the complexity for an in-process database. On the other hand, there's not much complexity to support this, and it saves the hassle of users complaining that their DBs corrupted when they accidentally opened the same DB from multiple programs at once. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Build mdb tools
POISSON Frédéric wrote: Hello, I'm trying to build mdb tools (mdb_stat and mdb_copy inside libraries/liblmdb directory) on a Solaris 10 SPARC operating system with OpenLDAP 2.4.39. Is there some requirements for building the tools on this OS because i have a error during build ? Here is my result (while i do not set any specific gcc flags) : # make ... gcc -pthreads -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast mdb_stat.o liblmdb.a -o mdb_stat Undefined first referenced symbol in file fdatasync liblmdb.a(mdb.o) ld: fatal: Symbol referencing errors. No output written to mdb_stat collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `mdb_stat' # Read the Solaris manpage for fdatasync. The requirement for building on any given OS is that you know your own OS. Notice that i use a gcc 3.4.3 and gcc option -pthread do not exist for this gcc release, i had to replace it by -pthreads. I try to read mdb.c for any specific options but i don't find anything... Notice also that i succeed to build OpenLDAP 2.4.39 with my Solaris 10 environment. Thanks, -- *Frederic Poisson* -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: Build mdb tools
Michael Ströder wrote: Christian Kratzer wrote: On Thu, 20 Mar 2014, Howard Chu wrote: POISSON Frédéric wrote: Hello, I'm trying to build mdb tools (mdb_stat and mdb_copy inside libraries/liblmdb directory) on a Solaris 10 SPARC operating system with OpenLDAP 2.4.39. Is there some requirements for building the tools on this OS because i have a error during build ? can we have these connected to the default build please. It's open source. Learn to edit the Makefile yourself if this matters to you. Or learn to type make twice. It's not that hard. In the meantime, it saves me time as a developer for things to work the way they currently do. If you're getting software from a distro they've already done this. +1 Ciao, Michael. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
