Re: Is there any directory server in public use?
On April 20, 2019 2:56:15 AM GMT+08:00, Howard Chu wrote: >This appears to be a >combination of colleges only teaching SQL (if they teach anything about >databases at all) and again, HR-oriented software only interacting with >SQL >(if they interact with external systems at all). Would be happy to see >otherwise. To be fair learning sql is easier. I didn't learned it from school, but after installing postgres on my computer and fired up psql with a good reference book on hand, I could conjure up simple database schemas and simple queries in a matter of hours. What openldap needs to gain mind share is an easy way to play with it. Something like psql, a shell that let you do DDL, DML and query by trial and error. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: Forbidden account password reuse of the last 5 password
On February 15, 2019 10:50:36 PM GMT+08:00, Howard Chu wrote: >slapd does not store plaintext passwords either. > sorry for spreading mis infomation based on my imagination. With ppolicy, can a user change his password after his password expired? I'd think no, because you have to bind before you modify the userpassword field, and if the password expired I'd think bind will fail. OTOH, kerberos does allow user to change password after expiration. this save me a lot of work, because my users always forgot to change pw in time. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: Forbidden account password reuse of the last 5 password
Michael Ströder writes: > On 2/14/19 8:19 AM, Derek Zhou wrote: >> Better use kerberos for advanced password policy requirements. You can >> use SASL to bridge LDAP's userPassword checking to a kerberos backend so >> everything still work and much safer. > > By which definition of "safe" is adding more complexity safer? > > Especially you don't know how the original poster does password changes. > Maybe he wants to use ppolicy response controls etc. > Yeah, adding kerberos is a complexity and you cannot change password via ldap anymore; has to go through the kerberos route. My notion of "safe" is only referring to the fact that the password text is not stored anywhere and the rogue admin cannot read user's passwords. I haven't found a good and up to date howto with step to step instrutctions on ppolicy with cn=config. I'd appreciate if someone here give my a pointer. Derek
Re: Forbidden account password reuse of the last 5 password
Tian Zhiying writes: > Hi > > Is there a feature that OpenLDAP password policy can forbidden user password > reuse of the last 5 password? > Better use kerberos for advanced password policy requirements. You can use SASL to bridge LDAP's userPassword checking to a kerberos backend so everything still work and much safer. Derek
Re: openldap proxy to kerberos
On Mon, Jan 07, 2019 at 04:18:36PM -0500, vad...@gmail.com wrote: > I am using openldap proxy today with ldap backend. > > Any suggestions on how to use kerberos as the backend? > Kerberos only has infomation necessary for authentication; like principals and policies. LDAP stores much more, such as group memberships, numerical uids, home directories, etc. So normally people use both LDAP and Kerberos, not Kerberos alone. There are 3 ways that Kerberos and LDAP can work together: 1, LDAP can use Kerberos to authenicate (bind) access 2, LDAP can forward authentication request to kerberos via SASL 3, Kerberos can use LDAP as a database backend In my organization we are using 1 and 2, but not 3. I think Microsoft AD also does something similiar under the hood.
storing large blob in LDAP
Hi, Is is possible to store large blob outside the main database, and only keep file names in the LDAP database? The blobs I want to stored are not as often used as other attributes and not indexed, so it seems to me that bloating up the main database is not economical in terms of fast disk space and overall database efficiency. Then I suppose I need a middle ware or something to combine the data from file system and data from LDAP server. Has anyone try that before? Or I should just store the blobs inside, and live with the bloated database? Thanks a lot Derek
Re: ldap user authentication, PAM and chsh (change shell): how to make it work?
On Saturday, December 15, 2018 06:18:49 PM Ryan Tandy wrote: > On Fri, Dec 14, 2018 at 03:24:17PM -0500, Jean-Francois Malouin wrote: > >I'm using libnss-ldap along with pam-ldap on Ubuntu and Debian clients. > > I have not tried this myself, but recent versions of nss-pam-ldapd > appear to include a 'chsh.ldap' command in the nslcd-utils package. > However it looks like that would require you to be using libnss-ldapd > and libpam-ldapd with nslcd, rather than the old libnss-ldap and > libpam-ldap. > It is probably not a good idea to do chsh in a LDAP controlled site in the first place. What if the user chsh into something not installed on every host, then realize she cannot login anymore? local chsh at least is protected by the local /etc/shells. It is probably simpler and safer to have a line of "exec zsh --login" in their .profile file -- Derek Zhou
Re: fast delete a lot of entries
On Thursday, December 13, 2018 05:24:53 PM Derek Zhou wrote: > Howard and others: > > Again on deleting lots of entries. I have 2 experiments: > > 1, in a fresh db insert 10 million entries. let's call this state A. then > delete 9 million entries over night. let's call this state B > 2, in a fresh db insert 1 million entries I call this state B' > > In B and B' even the 1 million entries are the same; so from user's > perspective B and B' are indistinguishable. However, deleting entries from B > is much slower than deleting entries from B', like 10x slower. It seems like > deleting speed depends on the peak db size, and how full the db currently is. > > My question is: is this wide a deletion performance gap expected? > Further testing shows that this deletion speed drop only happens in writemap mode. default mode is much better. See the following plots: 1, adding 10 million entries, default vs writemap As you can see, writemap is marginally faster, with curious period performance drop. 2, deleting all 10 million entries previously added, default vs writemap As you can see, in writemap mode, after ~10% entries are deleted, the performance went down hard. The test was not finished, would probably take hours. Whereas in default mode, delete speed was better in the beginning, there is also a performance drop after ~10% entries are deleted but much less severe. The test finished in ~4800 seconds. -- Derek Zhou
Re: fast delete a lot of entries
Howard and others: Again on deleting lots of entries. I have 2 experiments: 1, in a fresh db insert 10 million entries. let's call this state A. then delete 9 million entries over night. let's call this state B 2, in a fresh db insert 1 million entries I call this state B' In B and B' even the 1 million entries are the same; so from user's perspective B and B' are indistinguishable. However, deleting entries from B is much slower than deleting entries from B', like 10x slower. It seems like deleting speed depends on the peak db size, and how full the db currently is. My question is: is this wide a deletion performance gap expected? By the way, querying and adding speed are also different among A, B and B', but the gap is much smaller. also deletion speed gap between A and B' is not large. restarting daemon at state B does not improve deletion speed (so the difference between B and B' are persistent). In the meantime I will do another experiment: add back 9 million entries to state B and call it A'. then compare performance between A and A'. -- Derek Zhou Shannon Systems http://www.shannon-sys.com
speed up parallel add/delete
all I have a patch that can speed up parallel ldap add/delete a lot; in my testing about 2.5X. The idea is described here: the mdb backend does not really support parallelism in rw transactions, so add/delete are serialized by a mutex inside the mdb backend as small transactions. In my patch, all add/delete are put in a queue and executed in a dedicated worker thread. It may seem pointless at first, but by doing them all in the same thread, now I can merge concurrent ops from different clients into larger transactions, and reduce the number of expensive txn_commit calls (fsyncs). my test results are ( your mileage may vary) 10 thread parallel add: 1, unpatched slapd + default config, ~ 2800 op/s (performance stable over time) 2, unpatched slapd + dbnosync ~9000 op/s (lots of fluctuation due to background dirty page flush) 3, unpatched slapd + dpnosync + checkpoint every minute ~8000 op/s, with reduced fluctuation 4, patched slapd ~7300 op/s (fluctuation nearly gone) 2 is not a recommended config, because crashing or power lost will cause massive data lost. 3 is better, but still up to 1 minutes of data can be lost at crash. With this patch, I can achieve >90% of the performance of 3, with data durability as good as 1; or 2.5X performance of 1 with same data durability guaranty. The patch still need some tidy up; is this email list the place to send patches? -- Derek Zhou Shannon Systems http://www.shannon-sys.com
Re: On removing a duplicated ppolicy overlay
On Monday, December 03, 2018 02:11:27 PM Quanah Gibson-Mount wrote: > --On Monday, December 03, 2018 1:57 PM -0800 Daniel Howard > wrote: > > > > A potentially minor improvement along these lines could be a very nice > > feature enhancement for OpenLDAP. Thank you for your consideration. > > You're talking about two things: a) slapmodify, which is slated for > release with OpenLDAP 2.5, which allows offline modifications of cn=config, > and b) delete support in cn=config, also slated for release with OpenLDAP > 2.5. slapmodify is nice, however modifying schema is still hard. The old schema file format is still much better than the ldif format for viewing and editing. It would be a nice to have a tool that can parse and diff schemas in the old format and generate ldif modifications for slapmodify or ldapmodify to handle. -- Derek Zhou Shannon Systems http://www.shannon-sys.com
Re: fast delete a lot of entries
On Tuesday, November 27, 2018 5:13:41 PM CST Howard Chu wrote: > Derek Zhou wrote: > > Karsten > > > > Unfortunately, down time is even worse than slow. > > > > I wish OpenLDAP has an integrated search and delete operation, like delete > > entries based on a search filter, all executed in the same transaction in > > the server. That should be more efficient than doing search and delete > > from the client side because the B+ trees are just traversed once. Also > > the server can impose size limit on how many entries can be deleted in > > one go, just like the search, to avoid someone hogging the server. > > > > For the LDAP gurus here, the above idea should be technically possible, > > right? > In normal operation, as opposed to simple experimentation, why do you > actually need to delete a large number of entries quickly? If you just need > to make them unavailable to a bunch of clients, you can first do a subtree > rename to move them out of the way, and then delete incrementally at > whatever speed. If the database is going to occupy significant amount of diskspace, the sys admin in me will feel safer if I can delete fast. A file system can do that, a SQL database can do that. Imagine a user application is running wild and injecting a lot of junk data, being able to delete fast is crucial to keep the system up til the culprit is identified and killed. This scenario has happened to me several times, just not with a LDAP database. Derek
Re: fast delete a lot of entries
Karsten Unfortunately, down time is even worse than slow. I wish OpenLDAP has an integrated search and delete operation, like delete entries based on a search filter, all executed in the same transaction in the server. That should be more efficient than doing search and delete from the client side because the B+ trees are just traversed once. Also the server can impose size limit on how many entries can be deleted in one go, just like the search, to avoid someone hogging the server. For the LDAP gurus here, the above idea should be technically possible, right? Derek On Monday, November 26, 2018 03:36:16 PM Karsten Heymann wrote: > Hi Derek, > > no idea about the deletion speed, but if a short downtime is an option > and the entries to be deleted can be matched by a search filter, you > could do a > > slapcat -a '(!())' > > and the slapadd the resulting file into an empty database. Depending > on the amount of to entries be deleted in comparison to the total > number of entries in the database, this could be quicker than live > deletes. > > best regards > Karsten > Am Sa., 24. Nov. 2018 um 18:56 Uhr schrieb Derek Zhou > : > > > > dear list, > > > > I was experimenting with slapd by adding a lot of entries then deleting > > them. What I found is adding speed is not bad but deletion speed is > > lacking. With or without dbsync, delete speed is around half of that of > > adding. Naively, I thought deleting should be easier than adding, because > > in adding you actually need to pass and write whole entry of data, while in > > delete you can just mark the database page as free? The DB file did not > > shrink after massive deletion; which kind of suggest that deleting is only > > marking page as free, not really return them to the OS. I am using the > > latest git tree and the mdb backend. > > > > Another related idea is delete a whole branch from the DIT. LDAP is already > > hierarchical, to delete all entries under a branch, one would assume that > > there must be an better way than deleting entries with a client side > > script, like what I am doing? With SQL you can drop a table. With LDAP, can > > I delete a whole branch? > > > > -- > > Derek Zhou > > Shannon Systems > > http://www.shannon-sys.com > > -- Derek Zhou Shannon Systems http://www.shannon-sys.com
fast delete a lot of entries
dear list, I was experimenting with slapd by adding a lot of entries then deleting them. What I found is adding speed is not bad but deletion speed is lacking. With or without dbsync, delete speed is around half of that of adding. Naively, I thought deleting should be easier than adding, because in adding you actually need to pass and write whole entry of data, while in delete you can just mark the database page as free? The DB file did not shrink after massive deletion; which kind of suggest that deleting is only marking page as free, not really return them to the OS. I am using the latest git tree and the mdb backend. Another related idea is delete a whole branch from the DIT. LDAP is already hierarchical, to delete all entries under a branch, one would assume that there must be an better way than deleting entries with a client side script, like what I am doing? With SQL you can drop a table. With LDAP, can I delete a whole branch? -- Derek Zhou Shannon Systems http://www.shannon-sys.com
Re: delete a user schema from cn=config
On Monday, November 19, 2018 01:49:14 AM Howard Chu wrote: > Derek Zhou wrote: > > On Sun, Nov 18, 2018 at 09:29:10PM +, Howard Chu wrote: > >> Derek Zhou wrote: > >>> > >>> Understood. How do I find out what can be changed on the fly, what cannot > >>> as of the current git head? Document, pointer to source code all ok. > >> > >> Everything can be modified. Deleting entries is unsupported in 2.4. > >> > > Very cool. Let me clarify one more thing, if a schema is being modified, > > and if the current data > > is imcompatible with the modified schema, will it reject the modification > > or accept it, and the > > schema is only enforced for newer data? > > It's your responsibility to ensure that no existing data uses the schema > you're modifying. > The modification will simply be accepted. If you have data in your database > using the old > definition, and it uses a syntax with different normalization rules than the > new version, > your database will most likely be corrupted. > I see. So, is adding an optional (MAY) attribute to an objectClass ok? -- Derek Zhou Shannon Systems http://www.shannon-sys.com
Re: delete a user schema from cn=config
On Sun, Nov 18, 2018 at 09:29:10PM +, Howard Chu wrote: > Derek Zhou wrote: > > > > Understood. How do I find out what can be changed on the fly, what cannot > > as of the current git head? Document, pointer to source code all ok. > > Everything can be modified. Deleting entries is unsupported in 2.4. > Very cool. Let me clarify one more thing, if a schema is being modified, and if the current data is imcompatible with the modified schema, will it reject the modification or accept it, and the schema is only enforced for newer data? Derek
Re: delete a user schema from cn=config
On November 18, 2018 6:06:16 AM GMT+08:00, Quanah Gibson-Mount wrote: >Delete support for cn=config in many cases is part of the OpenLDAP 2.5 >release vs OpenLDAP 2.4. It may be simpler to use ldapmodify to adjust >the >incorrect schema elements. Failing that, you'll want to slapcat the >cn=config database, fix it by hand, and re-import via slapadd. > > Understood. How do I find out what can be changed on the fly, what cannot as of the current git head? Document, pointer to source code all ok. Thanks -- Derek
delete a user schema from cn=config
Hi list, I was experimenting locally written schema with a recent build of slapd. I added the schema ldif with no problem using ldapadd -QY EXTERNAL -H ldapi:/// ... But then I realized I made a mistake, so I try to delete it with ldapdelete -QY EXTERNAL -H ldapi:/// it says: ldap_delete: Server is unwilling to perform (53) what does it mean? The schema is not used by any data on the server. I can use slapmodify to remove it after server shutdown, but that kind of suck and not much different, or even worse than editing conf file? Derek
Re: cn=config default access control
On November 17, 2018 7:37:40 AM GMT+08:00, Quanah Gibson-Mount wrote: >--On Friday, November 16, 2018 10:11 AM +0800 Derek Zhou > wrote: > >> My argument is why isn't it the default? >A couple of immediate answers come to mind, there are probably more: > >a) OpenLDAP is used on numerous operating systems. Not all of those >operating systems support UNIX sockets. > >b) Not everyone configures slapd for use with ldapi > I see. But is it the most recommended way to review and edit cn=config on a unix like platform? If so, that should earn itself a spot on the quick start guide. If not, and simple auth is the way, that should be mentioned instead. Been able to edit config on a live system is a great feature, it is a shame that people only read the quick start guide dont know about it. >Once cn=config is the only way to configure OpenLDAP, such >documentation >will be removed. However, that won't be occurring in OpenLDAP 2.5, >which >is the next major release, so it is valid for this documentation to >remain >in OpenLDAP master for the time being. > you guys are really stingy on version numbers. just an observation. derek
cn=config default access control
Hi list,
I've been using openldap for a few years but yesterday I compiled slapd from
git head
for the first time. To my supprise that:
root@my-machine:/root#: ldapsearch -LLLQY EXTERNAL -H ldapi:/// -b cn=config
does not work. It took me a while to find out that by default the cn=config
database
has 'olcAccess: {0}to * by * none' and 'olcRootDN: cn=config' with no
olcRootPW, so
all access from ldap is denied. Once I know I used slapmodify to change
olcRootDN to
'gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth' and everything works
as
expected afterward.
My argument is why isn't it the default? I think debian packages already did
that.
cn=config is stored as plain text on the local file system so local root can
read
and change anyway. Changing cn=config is the first thing to do for any admin,
and
I am not exactly a newbie yet I still stumbled on it.
By the way, if we really want people to use cn=config exclusively, I suggest to
remove all mentioning of slapd.conf from the latest documentation. Old admins
appreciate cn=config more and there will be less distraction for newbies.
Derek
