Re: Rebuild openldap so all modules/overlays are separate

2010-12-01 Thread Howard Chu

Chris Jacobs wrote:

Main question:
how to build openldap with all overlays as dynamically loaded modules (ie: 
optional).

Why?
These problems have been covered before, recently even.  I'm trying to get 
Solaris 9/10 clients talking to openldap server.
I can do getent passwduser  but getent passwd or getent group or even 
ldaplist all fail.

I haven't had any success with Benjamin's acl fix/work around, or configuring 
the overlay for sssvlv in slapd.conf.

So, I thought I'd recompile openldap - without sssvlv support, but then I 
thought why have to do this again in the future if I find another 
incompatibility?  I'd prefer to simply have all overlays/modules as separate 
loadable modules.

When I compiled OpenLDAP, I configured it with:
./configure --with-tls=openssl \
 --enable-crypt \
 --enable-dynamic \
 --enable-ldap \
 --enable-lmpasswd \
 --enable-modules \
 --enable-overlays \
 --enable-spasswd \
 --sysconfdir=/etc

The enable-overlays option builds in a number of overlays I use and am 
interested in using in the future - however, it also enables sss/vlv.

I supposed I could try to build it again with that option followed by 
--disable-sssvlv but what I'd really prefer is a way to build openldap with the 
overlays as separate modules that I can load/unload when/as I need them.

I've searched the documentation, but I can't figure out how.  It's quite likely 
I haven't looked in the right spot.  I'm currently trying to make my way 
through the configure script...  Blech.

Thanks for any pointers,


configure --help

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: OpenLDAP on T2000

2010-12-02 Thread Howard Chu

Emmanuel Lecharny wrote:

On 12/2/10 2:36 PM, Nick Folino wrote:

I'm evaluating OpenLDAP on various platforms and am having trouble getting
the tools (slapadd, slapindex, etc) to perform well on Sun's T2000 servers.
For example slapadd is running at about 50k/s.  On an x86 server importing
the same ldif yields over 1M/s.


All my experience with OpenLDAP so far shows x86 servers far outperforming Sun 
T2000s, given comparable memory and disk. The individual Sun cores are just 
too slow.



1 *million* addition per second ? Either you have an issue with the unit
you use, or there is something wrong going on but you didn't noticed the
problem...


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: slapadd: corrupted double-linked list

2010-12-08 Thread Howard Chu

Marc Patermann wrote:

Hi,

Marc Patermann schrieb am 06.12.2010 16:40 Uhr:


what could the following possibly be?


This looks like something valgrind ought to be able to diagnose.



When I do a
# slapadd -q -v -c -l dump.ldif
to a SLES 11 SP1 (openldap 2.3.20)
I get this after a few 1000 entries:

I have a fresh install of SLES 11 SP1 on a new machine. I installed 2.4.23.
When I did the config and tried to put initial data (8(!) objects) into
the server, I got glibc errors too.

# slapadd -q -c -v -l init.ldif
added: ...
...
_ 100.00% eta   none elapsednone fast!
Closing DB...
*** glibc detected *** slapadd: free(): invalid pointer:
0x7f89c0273210 ***
=== Backtrace: =
/lib64/libc.so.6(+0x75018)[0x7f89bd242018]
/lib64/libc.so.6(cfree+0x6c)[0x7f89bd246f6c]
/usr/lib64/libdb-4.5.so(__ham_db_close+0x19)[0x7f89bf0d9139]
/usr/lib64/libdb-4.5.so(__db_refresh+0x340)[0x7f89bf120a20]
/usr/lib64/libdb-4.5.so(__db_close+0x51)[0x7f89bf120f91]
/usr/lib64/libdb-4.5.so(__db_close_pp+0xd5)[0x7f89bf135445]
slapadd(+0xc15eb)[0x7f89bfd065eb]
...

As I found out, when I deactivate the four index lines, it works. Each
of these four lines causes an glibc error.

#index   objectclass,uid,maileq
#index   sn,cn,givenName sub,eq

#index   dhcpHWAddress,dhcpClassData,dhcpOption  eq
#index   entryCSN,entryUUID,zoneName eq


Hm? :(

Marc




--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: x500UniqueIdentifier

2010-12-08 Thread Howard Chu

Juan Gonzalez wrote:

Hi, I’m trying to insert userCertificate values containing
x500UniqueIdentifiers. When the value appears at the SubjectNames, it inserts
correctly.

By this I assume there is a validation for the field formatting.

When I have a x500UniqueIdentifier at the IssuerNames it fails to insert.

Is there a specific place where valid attributes and syntaxes for IssuerNames
should be declared?


Certificate name validation just uses whatever schema is already loaded into 
slapd, and x500UniqueIdentifier is in the core schema file so it should 
already be present in your configuration.


However, subjectNames aren't fully parsed during certificate validation, while 
issuerNames are. So e.g. a syntax error in subjectName will not be detected at 
insert time.


Can you post an example certificate? I have a feeling that our DN validator 
here may not be handling the syntax for x500UniqueIdentifier but would like to 
double check.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: x500UniqueIdentifier

2010-12-08 Thread Howard Chu

Howard Chu wrote:

Juan Gonzalez wrote:

Hi, I’m trying to insert userCertificate values containing
x500UniqueIdentifiers. When the value appears at the SubjectNames, it inserts
correctly.

By this I assume there is a validation for the field formatting.

When I have a x500UniqueIdentifier at the IssuerNames it fails to insert.

Is there a specific place where valid attributes and syntaxes for IssuerNames
should be declared?


Certificate name validation just uses whatever schema is already loaded into
slapd, and x500UniqueIdentifier is in the core schema file so it should
already be present in your configuration.

However, subjectNames aren't fully parsed during certificate validation, while
issuerNames are. So e.g. a syntax error in subjectName will not be detected at
insert time.

Can you post an example certificate? I have a feeling that our DN validator
here may not be handling the syntax for x500UniqueIdentifier but would like to
double check.


I should also warn you, x500UniqueIdentifier has a bitstring syntax, and this 
syntax is extremely clumsy in LDAP. Judging from the certificate you sent me, 
you're trying to use this attribute as a regular octetstring. The two are 
quite different. E.g. if you have a value in the certificate 
x500UniqueIdentifier=ABC in LDAP this is represented as

x500UniqueIdentifier='010101100111'B

I would strongly encourage you to use some other attribute other than 
x500UniqueIdentifier, and to avoid any attributes that use bitstring syntax as 
this one does.


The bitstring syntax is intended for data items that are actually an arbitrary 
string of bits, in particular, a string that may not be aligned on octet 
boundaries. If the only data you're storing is actually sequences of octets, 
this is a very inefficient choice of syntax.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: openldap and kerberos integration

2010-12-09 Thread Howard Chu

Thierry Lacoste wrote:

Hello,

I'm experimenting with integrating Kerberos and OpenLDAP
following roughly http://wiki.mandriva.com/en/Projects/OpenLDAP_DIT

I'm using CentOS and Buchan Milne's repository 
(http://staff.telkomsa.net/packages/rhel5/
)
both for OpenLDAP and Heimdal.

I've almost succeeded except for password integration.
It seems that the smbk5pwd module provided by openldap2.4-
servers-2.4.22-1.el5
in /usr/lib/openldap2.4/smbpwd.so is built without kerberos support.

With smbk5pwd-enable krb5 I have the following error:
/etc/openldap2.4/slapd.conf: line 154: smbk5pwd:smbk5pwd-enable
module smbk5pwd-enable only allowed when compiled with -DDO_KRB5.

What is the easiest option to get a kerberos supporting smbk5pwd?


I have no comment on other people's builds.


BTW I'd appreciate any recommandations about providing kerberos and
LDAP authentication (with the same password) in a production setting.
Should I use Heimdal or MIT kerberos ?
If Heimdal, is it better to use OpenLDAP as a backend for Kerberos or
let Kerberos use its native backend?
If OpenLDAP as a backend, is it better to use {K5KEY} as the
userPassword or let smbk5pwd synchronize everything?


Read the smbk5pwd README.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: openldap and kerberos integration

2010-12-10 Thread Howard Chu

Thierry Lacoste wrote:


On 9 déc. 10, at 22:03, Howard Chu wrote:


Thierry Lacoste wrote:

Hello,

I'm experimenting with integrating Kerberos and OpenLDAP
following roughly http://wiki.mandriva.com/en/Projects/OpenLDAP_DIT

I'm using CentOS and Buchan Milne's repository 
(http://staff.telkomsa.net/packages/rhel5/
)
both for OpenLDAP and Heimdal.

I've almost succeeded except for password integration.
It seems that the smbk5pwd module provided by openldap2.4-
servers-2.4.22-1.el5
in /usr/lib/openldap2.4/smbpwd.so is built without kerberos support.

With smbk5pwd-enable krb5 I have the following error:
/etc/openldap2.4/slapd.conf: line 154: smbk5pwd:smbk5pwd-enable
module smbk5pwd-enable only allowed when compiled with -DDO_KRB5.

What is the easiest option to get a kerberos supporting smbk5pwd?


I have no comment on other people's builds.

Sorry for the noise. I found it here
http://staff.telkomsa.net/packages/rhel5/openldap2.4-smbk5pwd-2.4.21-4.el5.i386.rpm



BTW I'd appreciate any recommandations about providing kerberos and
LDAP authentication (with the same password) in a production setting.
Should I use Heimdal or MIT kerberos ?
If Heimdal, is it better to use OpenLDAP as a backend for Kerberos or
let Kerberos use its native backend?
If OpenLDAP as a backend, is it better to use {K5KEY} as the
userPassword or let smbk5pwd synchronize everything?


Read the smbk5pwd README.

I'v read it. Your answer seems to imply that I should use Heimdal and
then OpenLDAP as it's backend.
Am I right?


It's more than just implied. The README says the code was written for Heimdal. 
If you want to use smbk5pwd at all, then you must use Heimdal.



Do you recommend using {K5KEY} as the userPassword?


If you want LDAP Simple Binds to use the same password as Kerberos, then yes. 
If not, then no.


There are security implications to using it; without strong protections on the 
LDAP sessions you're exposing the Kerberos password on the network and it 
ordinarily never should be exposed. (Also, just because you think you have 
strong protections in place, it's still less secure than keeping it completely 
off the wire. Particularly when somebody screws up (e.g. Debian) and their TLS 
software is doing something stupid like generating predictable sequences of 
random keys, etc.)

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: openldap and kerberos integration

2010-12-15 Thread Howard Chu

Thierry Lacoste wrote:


On 10 déc. 10, at 20:57, Howard Chu wrote:


Thierry Lacoste wrote:


BTW I'd appreciate any recommandations about providing kerberos
and
LDAP authentication (with the same password) in a production
setting.
Should I use Heimdal or MIT kerberos ?
If Heimdal, is it better to use OpenLDAP as a backend for
Kerberos or
let Kerberos use its native backend?
If OpenLDAP as a backend, is it better to use {K5KEY} as the
userPassword or let smbk5pwd synchronize everything?


Read the smbk5pwd README.

I'v read it. Your answer seems to imply that I should use Heimdal
and
then OpenLDAP as it's backend.
Am I right?


It's more than just implied. The README says the code was written
for Heimdal. If you want to use smbk5pwd at all, then you must use
Heimdal.

Sorry my question was not very clear.
I wan't LDAP Simple Binds and Kerberos with the same password.
I find smbk5pwd and OpenLDAP as a Heimdal backend very appealing
but maybe there are good reasons to use another Kerberos
implementation
and/or store passwords in the Kerberos native backend (adding e.g.
SASL in the mix
to make LDAP Simple Binds use pass-through authentication), obviously
ruling out smbk5pwd.


If all of your clients can use SASL Binds, then that is the best
choice, and you can ignore smbk5pwd.

Unfortunately I have no control over the clients so I need something
transparent for them.
What I had in mind was setting {sasl}lo...@my.realm in the userPassword.
The realm would be on a KDC hosted on the same box as the LDAP server
to minimize network traffic.
It seems much less elegant than the Heimdal/OpenLDAP integration with
smbk5pwd
but (maybe) has some benefits: I can use another KDC (e.g. MIT) and
use the native kerberos backend.
My first tests are OK but as of now I have no idea about replication.

Any recommandation for the best way to have LDAP Simple Binds and
Kerberos with the same password
are much appreciated.




Do you recommend using {K5KEY} as the userPassword?


If you want LDAP Simple Binds to use the same password as Kerberos,
then yes. If not, then no.

AFAICS with smbk5pwd I have two ways to have LDAP Simple Binds and
Kerberos with the same password.
1) force use of ldappasswd to make smbk5pwd synchronize all
passwords;
2) assign {K5KEY} to the userPassword and use kpasswd to change a
password.

If I understood correctly, the second method makes the passwords
identical by construction
while the first allow passwords to desynchronize if changed without
ldappasswd.


The point of smbk5pwd is to fully synchronize, that means it works
in both directions. When configured correctly and {K5KEY} is used,
it doesn't matter whether kpasswd or ldappasswd is used, everything
stays in sync.


I noticed some differences. In particular ldappasswd updates
sambaLMPassword while kpasswd does not.


I suppose we can delete sambaLMPassword support by now, certainly no one 
should be using it any more.



With ldappasswd, here is an excerpt of my auditlog:

replace: userPassword
replace: sambaPwdLastSet
replace: sambaLMPassword
replace: sambaNTPassword
replace: krb5KeyVersionNumber
replace: krb5Key

With kpasswd, I have :

replace: krb5KeyVersionNumber
replace: krb5PasswordEnd
replace: sambaPwdMustChange
delete: krb5Key
add: krb5Key
replace: sambaNTPassword



There is no either-or as you outline with your two choices above.
You must use {K5KEY} if you want any of the synchronization to work.


Here I'm confused. I tested smbk5pwd with SSHA password hashes.
ldappasswd correctly updates passwords and kerberos keys such that
LDAP simple binds and kerberos authentication both work with the same
password.


In that case there's both an SSHA hashed copy and a KDC hashed copy of the 
password. With {K5KEY} there is only the KDC hash.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: openldap and kerberos integration

2010-12-15 Thread Howard Chu

Hugo Monteiro wrote:

On 12/15/2010 07:19 PM, Howard Chu wrote:

Thierry Lacoste wrote:

I noticed some differences. In particular ldappasswd updates
sambaLMPassword while kpasswd does not.


I suppose we can delete sambaLMPassword support by now, certainly no
one should be using it any more.



I'm sorry but did i understand correctly that sambaLMPassword will no
longer be updated while using the smbk5pwd overlay?
Also, i would like to know why do you state that no one should be using
it any more. Besides Samba itself, it can (as is) used by freeradius
while using PEAP and MsCHAPv2 for wireless clients authentication.


That's interesting, especially since the KDC itself doesn't maintain 
sambaLMPassword. The LANMAN hash mechanism has been obsolete for years, it is 
intrinsically weak and is not a good security mechanism.


I think you're mistaken, anyway; according to RFC2759 which defines MSCHAPv2, 
it uses an NT hash, not a LANMAN hash. The LANMAN hash was used for MSCHAPv1 
which is also obsolete.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Filesystem backend options for embedded openldap

2010-12-18 Thread Howard Chu

Bruce Edge wrote:

On Sat, Dec 18, 2010 at 12:26 PM, Peter Lambrechtsen
plambrecht...@gmail.com  wrote:

Or perhaps TinyLdap? http://www.fefe.de/tinyldap/

Also FreeRadius (if your app's support Radius and LDAP) supports a myriad of
backend databases.


Hmm, very interesting. I was not aware of this project.
My concern is that there's been very little activity on the project in
recent years.

Is there no simple, reliable, backend config for openldap?

I'm not concerned with speed, just reliability and data integrity.

I'd settle for a 10x performance penalty for data integrity.


You could give back-ldif a try. It certainly will not perform well, but it's 
so simple that data corruption wouldn't be an issue.


-Bruce



On Sun, Dec 19, 2010 at 5:59 AM, Bruce Edgebruce.e...@gmail.com  wrote:


On Sat, Dec 18, 2010 at 12:19 AM, Peter Lambrechtsen
plambrecht...@gmail.com  wrote:

Why don't you use SQLite instead??? It's pretty rock solid backend
database.

Unless your client side only wants to talk LDAP.


Hi,
Thanks the the response. One of the reasons for ldap is that it
handles all the authentication for a lot of the packages we're using
out of the box so it was a natural progression to extend it to handle
the other data we need as well.
The only probems, it appears, is how to make it more failsafe on the back
end.

-Bruce



On Sat, Dec 18, 2010 at 6:48 AM, Bruce Edgebruce.e...@gmail.com
wrote:


Perhaps a bit more detail...
During testing our developers frequently hang the target machines.
This usually results in a corrupted ldap database even though no write
activity was present on the box since long before the crash.

What ldap config tuning options are required to get slapd to sync the
backend to a state where power loss / kernel crashes do not corrupt
the data?

Thanks

-Bruce

On Wed, Dec 15, 2010 at 10:25 AM, Bruce Edgebruce.e...@gmail.com
wrote:

I'm working on an embedded system for which I would like to use
openldap as the means of config storage.
I've spent a lot of time RTFM'ing and still feel that there is a lot
that is escaping me as far as the optimal configuration.

If the primary goal is data safety and zero human intervention, what
would be the optimal combination of file system / backend storage /
and config options?

I would like to never have to manually recover a database and have it
gracefully recover from power failures. Speed is not an issue as it's
very low traffic. Integritiy is everything.
It's target storage is a USB flash device. Are there any special
considerations WRT flash storage and ldap?

Thanks in advance.

-Bruce












--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Granting write to ou

2010-12-22 Thread Howard Chu

Dieter Kluenter wrote:

Am Wed, 22 Dec 2010 14:46:50 +0100
schrieb Thomas D. Dahlmanndomi...@domingo.dk:


Hi

I'm trying to add ordinary users write access to a specific ou.

I've googled a lot and haven't really found any useful regarding to
openldap 2.4 (slapd.d format).

What would be the correct syntax for a ldapmodify command to
accomplish this to the dn: ou=addressbook,dc=example,dc=net ?


something like:
ldapmodify -D cn=config -W -H ldap://some.host
dn:olcDatabase={1}hdb,cn=config
changetype: modify
replace: olcAccess
olcAccess: {1} to dn.subtree=ou=addressbook,dc=example,dc=net by
  users write by * read
-

The numbers {1} are fictious, replace with your real values.


Do not supply the {x} numbers at all when you're just doing a replace. slapd 
generates them itself, so there's no need.


You only need to provide the {x} numbers when you actually want to reference a 
specific value in a multivalued attribute. Even then, they're just a 
convenience, not absolutely essential.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Kerberos/GSSAPI issues

2010-12-28 Thread Howard Chu

Brian Candler wrote:

Hello,

I'm setting up an openldap server for Kerberos (GSSAPI) authentication only.
I'm using slapd-2.4.21 from Ubuntu 10.04.1.

It's basically working, and I had to do very little other than change
export KRB5_KTNAME in /etc/default/slapd to point to the service keytab.

However, there are a couple of strange things which I wonder if someone
could help me with.

(1) According to the documentation at
http://www.openldap.org/doc/admin24/sasl.html#GSSAPI
then the authentication DN should be
uid=primary[/instance],cn=realm,cn=gssapi,cn=auth

However, running slapd in debug mode I see the cn=realm  is missing.


That's normal. The SASL library doesn't provide the realm name when it is 
equal to the default realm. This has been true of Cyrus SASL for probably the 
past dozen years. Read the Cyrus SASL documentation.



(2) I would like to be able to do ldapsearch without specifying -Y GSSAPI
explicitly. However if I omit it, the client picks DIGEST-MD5 instead
(which isn't much use, since I have no passwords in the database)


Configure a sasl/slapd.conf with the options you want.
Read the Cyrus SASL documentation.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Kerberos/GSSAPI issues

2010-12-29 Thread Howard Chu

Dieter Kluenter wrote:

Am Wed, 29 Dec 2010 16:50:17 +
schrieb Brian Candlerb.cand...@pobox.com:


On Wed, Dec 29, 2010 at 07:57:43AM +0100, Dieter Kluenter wrote:

The default ssf of ldapi is 71, but you may change localSSF in
slapd.conf(5).
[...]


Thank you, that is very clear.

Having changed that, I can use EXTERNAL with minssf=112, but not
GSSAPI.  I find that if I set minssf=56 it's fine, but at minssf=57
it isn't.

It looks like this is a fundamental limitation of the GSSAPI:
http://lists.andrew.cmu.edu/pipermail/cyrus-sasl/2006-September/000628.html
http://lists.andrew.cmu.edu/pipermail/cyrus-sasl/2006-September/000635.html

FYI, here's what I see with minssf=57 (the 'No such attribute' error
is somewhat confusing)

r...@noc:~# ldapsearch
ldap_sasl_interactive_bind_s: No such attribute (16)
r...@noc:~# ldapsearch -Y GSSAPI
SASL/GSSAPI authentication started
ldap_sasl_interactive_bind_s: Inappropriate authentication (48)
additional info: SASL(-15): mechanism too weak for this user:
mech GSSAPI is too weak


That is because Kerberos DES, und thus GSSAPI, only has a security
strength factor of 56.


But this value is pure fiction, it's an arbitrary value hardcoded into the 
SASL gssapi plugin. Generally Kerberos is using triple-DES today, or AES.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: OpenLDAP 2.4.24?

2011-01-03 Thread Howard Chu

Michael Ströder wrote:

Marco Pizzoli wrote:

I would like to know if is there a plan to make a new release of OL in
the following months.


http://www.openldap.org/lists/openldap-devel/201012/msg1.html

There were some commits to CVS branch OPENLDAP_REL_ENG_2_4 since then.
So help testing it:


Not yet. Wait for the release engineer to announce a call for testing. The 
tree is still in flux.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: viewing cn=config

2011-01-05 Thread Howard Chu

E.S. Rosenberg wrote:

Hi, sorry that it took me a while to reply.

So I do have the config backend, since debian moved me over automatically.
So far the slapcat method has been working very nicely for me.

Another question I ran into is how do you edit cn=config, by default I have
not been able to figure out.
I have so far seen two methods that work for me but they both feel a bit
'hacky' so I was wondering if there is another 'official' method.

The two things I have done:
1. edit the files in slapd.d (after stopping the openldap server to be on the
safe side).


Never do that.


2. using ldapmodify, but this was only possible after I added a olcRootPW
attribute to the olcDatabase object of the config backend by editing the file
as described here:
http://www.zarafa.com/wiki/index.php/OpenLdap:_Switch_to_dynamic_config_backend_%28cn%3Dconfig%29#Add_or_Change_password_of_RootDN

It confused me at first since the configdb entry has the following acl:
olcAccess: {0}to * by
dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * 
break
I didn't understand how I could get access, it looked like I had to be root
but even as root ldapmodify would refuse duty.

I guess after seeing these mails I should have been using something like -Y
EXTERNAL?


Yes, from your description, you should have used ldapmodify -Y EXTERNAL -H 
ldapi:/// while running as root.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: ACLs to allow users to traverse DIT down to their own entry, lock down everything else

2011-01-05 Thread Howard Chu

Andreas Ntaflos wrote:

So what do I need to do so users can specify dc=example,dc=com as
search base and traverse the directory tree down to their own DN entry?


Read the Operation Requirements section of slapd.access(5).

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Index for objectclass does not work...

2011-01-05 Thread Howard Chu

Steeg Carson wrote:

2011/1/4 Quanah Gibson-Mountqua...@zimbra.com:

--On Tuesday, January 04, 2011 1:43 AM +0100 Steeg Carson
steeg.car...@googlemail.com  wrote:


I simulate this on my database just right now:


I suggest you read:

http://www.openldap.org/lists/openldap-technical/201011/msg00146.html

to understand how indices and their slots work.



As I now understand, the entire index for one attribute (e.g.
objectClass) is split in several indexes. They holds for each
path/node (resp. DN, but not leaf) an separate index for this
attribute with all hits for his subtree (and for onelevel too).


No. Only the dn2id table maintains any notion of nodes and subtrees. All 
indices are global to the database and have no notion of scope.



If I do an ldapsearch with -b cn=ownPath,ou=root the slapd takes the
index which is bound on this node/DN?

In my DIT are 470812 entires.

The objectClass=subEngine exists 104384 times in the entire directory (ou=root).
The objectClass=subEngine exists only 1 time under  cn=ownPath,ou=root,


By default an index slot can only maintain 65535 records before it overflows 
and loses precision. Once it loses precision, you tend to get results like 
this. If you need to accomodate larger indices you can tweak a constant in 
back-bdb/back-bdb.h and recompile. You'll probably also need to increase 
LDAP_PVT_THREAD_STACK_SIZE.


Another workaround, without recompiling, would be to sort your entries such 
that all of the entries of the subEngine class are loaded in contiguous order.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Index for objectclass does not work...

2011-01-05 Thread Howard Chu

Steeg Carson wrote:

2011/1/6 Howard Chuh...@symas.com:

Steeg Carson wrote:


2011/1/4 Quanah Gibson-Mountqua...@zimbra.com:


--On Tuesday, January 04, 2011 1:43 AM +0100 Steeg Carson
steeg.car...@googlemail.comwrote:


I simulate this on my database just right now:


I suggest you read:

http://www.openldap.org/lists/openldap-technical/201011/msg00146.html

to understand how indices and their slots work.



As I now understand, the entire index for one attribute (e.g.
objectClass) is split in several indexes. They holds for each
path/node (resp. DN, but not leaf) an separate index for this
attribute with all hits for his subtree (and for onelevel too).


No. Only the dn2id table maintains any notion of nodes and subtrees. All
indices are global to the database and have no notion of scope.


But what does mean (from
http://www.openldap.org/lists/openldap-technical/201011/msg00146.html):

Ordinarily at each level of the tree we keep an index tallying all of
the children beneath that point. In back-bdb this index is used for
subtree searches and for onelevel searches.

So if I do a search, I'll get every time ALL results (ID's) from the
index for the searched value.  If my search uses additionally a
searchbase the slapd takes all ID's and lookup in id2entry.bdb to get
the DN for the ID and compare?


The search scope provides a set of candidates, consisting of every entry 
within that scope of the tree. An index lookup provides a set of candidates, 
consisting of every entry in the DB that matches the index. The intersection 
of these sets forms the set of candidate entries that must be examined in depth.



Can you recommend a good book, where I can read all such things and
understand, how openldap really works? This are all very important
things for design and operation.


There is no book on the internals of back-bdb or hdb. The source code is there 
for anyone to read. If you want to learn more, there are also extensive 
discussions on the design approaches and tradeoffs in the archives of the 
openldap-devel mailing list. For the most part, these discussions have only 
ever been of interest or relevance to other OpenLDAP developers.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: users, groups, etc. for posix authentication?

2011-01-05 Thread Howard Chu

Christ Schlacta wrote:

is there any reason that a posix usernames, groups, passwords, etc. must
be stored in distinct locations in a directory ?  I realize this mostly
applies to the padl pam/nis and the libnsspam-ldapd module specific.

can they be stored in other structures effectively and usefully?  can
they be stored on a department by department basis, or in any other
organizational scheme?  (ou=arbitrary1,dc=... having groups and users,
while ou=arb2,ou=arb3,dc=... also has users and groups?)  if a scheme
like the above is used, will all users and groups be available on a
system?  must they be free of naming conflicts, or will
group=users,ou=arbitrary1,... be different from
group=users,ou=arb2,ou=arb3,... ?  if they're different, how would this
be indicated by the systems?

A POSIX system considers usernames to be a flat namespace. If you store them 
in separate branches of a directory, you create the possibility of having 
duplicate names in separate branches, and the base OS will not be able to 
handle that.


This question has nothing to do with LDAP and has no place on this forum.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Index for objectclass does not work...

2011-01-06 Thread Howard Chu

Steeg Carson wrote:

Hello Howard,



By default an index slot can only maintain 65535 records before it overflows
and loses precision. Once it loses precision, you tend to get results like
this. If you need to accomodate larger indices you can tweak a constant in
back-bdb/back-bdb.h and recompile. You'll probably also need to increase
LDAP_PVT_THREAD_STACK_SIZE.


can you please guide to do this? Which constant must be changed, which
values are
suitable for this constant and for LDAP_PVT_THREAD_STACK_SIZE?



If I change this values, which other impact does it have of used
memory and the slapd?


The constant is BDB_IDL_LOGN in back-bdb/idl.h. Incrementing it by 1 will 
double the range of an index slot before it loses precision. It will also 
double the amount of memory used by all of the indexing functions. I think you 
can safely double the current value without overrunning the default thread 
stack size. But if you go even higher you'll probably need to increase it.


The current value for LDAP_PVT_THREAD_STACK_SIZE is (1 * 1024 * 1024 * 
sizeof(void *))


(4MB on a 32 bit machine, 8MB on 64 bit machine).

If you need to raise it I would suggest adding e.g.
  XDEF=-DLDAP_PVT_THREAD_STACK_SIZE=16777216
to your make invocation and recompiling libldap_r with this new value.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Handling slapd.d in OpenLDAP and Kerberos

2011-01-07 Thread Howard Chu

Quanah Gibson-Mount wrote:

--On Friday, January 07, 2011 9:22 PM +0530 sarathkrishn...@gmail.com wrote:


We think adding this in slapd.conf should serve the purpose. Any
suggestions/help is highly appreciated.


Use slaptest to convert your slapd.conf file to the slapd-config format,
and read what it does for the authz regexp.  Then you'll clearly see how to
handle what you did in slapd.conf in the new format.


Or just read the slapd-config(5) manpage and the Admin Guide...

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX

2011-01-12 Thread Howard Chu

Chris Jacobs wrote:

That appears to be the point.

See: 
http://www.openldap.org/software/man.cgi?query=ppolicyapropos=0sektion=0manpath=OpenLDAP+2.3-Releaseformat=html
... No results.


Sounds like the search index is out of date. Still, all you have to do is go 
here

http://www.openldap.org/software/man.cgi?query=(5)sektion=apropos=1manpath=OpenLDAP+2.3-Releasetitle=Section 



and the manpage is there:

http://www.openldap.org/software/man.cgi?query=slapo-ppolicysektion=5apropos=0manpath=OpenLDAP+2.3-Release

But better yet, just type man 5 slapo-ppolicy on your machine. Why people 
waste time searching the web when everything is on their local machine still 
boggles my mind.



Also look for the ppolicy in:
http://www.openldap.org/doc/admin23/schema.html#Distributed%20Schema%20Files
... It's not there.


The Admin Guide was never intended to be an exhaustive reference - it is after 
all only a guide. Every software component is documented in manpages. The 
manpages should always be the first place you look, not the Guide, and not the 
web.



Where did you get the schema and the libraries necessary?

FWIW: the password policy and MUCH more reliable syncing between servers is why 
we upgraded in my shop (turned off the old 2.3 master last week after finally 
overcoming last hurdles: solaris and use by other custom systems.)

- chris

Chris Jacobs, Systems Administrator
Apollo Group  |  Apollo Marketing | Aptimus
2001 6th Ave Ste 3200 | Seattle, WA 98121
phone: 206.839-8245 | cell: 206.601.3256 | Fax: 208.441.9661
email:  chris.jac...@apollogrp.edu

- Original Message -
From: 
openldap-technical-boun...@openldap.orgopenldap-technical-boun...@openldap.org
To: Quanah Gibson-Mountqua...@zimbra.com
Cc: openldap-technical@openldap.orgopenldap-technical@openldap.org
Sent: Wed Jan 12 23:38:54 2011
Subject: Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX

13.01.2011 11:55, Quanah Gibson-Mount пишет:



--On January 13, 2011 11:42:29 AM +0600 Konstantin Boyandin
temmo...@gmail.com  wrote:


Hello,

OpenLDAP version: 2.3.43-12 (CentOS 5.5), 64-bit.

In order to enable ppolicy overlay, I am trying to create the relevant
entries, as specified in

http://www.openldap.org/doc/admin24/overlays.html#Password%20Policies


I would suggest you compare the version you are running (2.3) with the
version that the document you are reading uses (2.4).  There is an
obvious difference there, and it is a major one.  I suggest you run a
current supported release of OpenLDAP that matches the documentation you
are using.


Thanks. I opened the 2.3 admin link instead:
http://www.openldap.org/doc/admin23/
and it has no overlays section at all. That's weird, since I am using
replication feature and there's a directive

overlay syncprov

in /etc/openldap/slapd.conf

How can I find the reasons for 'Invalid syntax' error in such a situation?
Thanks.


This message is private and confidential. If you have received it in error, 
please notify the sender and remove it from your system.





--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: LDAP and PAM: account is expired, but pam_ldap allows authentification

2011-01-12 Thread Howard Chu

Konstantin Boyandin wrote:

Hello,

Could someone direct me to the source of wisdom to solve this: I have
set correctly the fields (attributes)

shadowExpire
shadowLastChange
shadowMin
shadowMax

to make the account expired (OpenLDAP used to run NT domain), but when I
ssh to a server using pam_ldap authentication, it is still allowed to login.

How pam_ldap should be instructed to take the expiration attributes ito
account?


Ask on a pam_ldap mailing list. pam_ldap is not a piece of OpenLDAP software, 
your question is off topic here.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX

2011-01-12 Thread Howard Chu

Konstantin Boyandin wrote:

Hello,

OpenLDAP version: 2.3.43-12 (CentOS 5.5), 64-bit.

In order to enable ppolicy overlay, I am trying to create the relevant
entries, as specified in

http://www.openldap.org/doc/admin24/overlays.html#Password%20Policies

I import two LDIFs, first:

dn: ou=Policies,dc=example,dc=com
objectClass: organizationalUnit
objectClass: top
ou: Policies

and second

dn: cn=default,ou=Policies,dc=example,dc=com
cn: default
objectClass: top
objectClass: pwdPolicy
objectClass: person
pwdAllowUserChange: TRUE
pwdAttribute: userPassword
pwdCheckQuality: 2
pwdExpireWarning: 600
pwdFailureCountInterval: 30
pwdGraceAuthNLimit: 2
pwdInHistory: 5
pwdLockout: TRUE
pwdLockoutDuration: 0
pwdMaxAge: 7776000
pwdMaxFailure: 5
pwdMinAge: 0
pwdMinLength: 5
pwdMustChange: FALSE
pwdSafeModify: FALSE
sn: dummy value

The first loads OK.
When I try to import the second, I receive this diagnostics:

Could not add object cn=default,ou=Policies,dc=itelsib,dc=com
Message: Invalid syntax
Error code: 0x15 (LDAP_INVALID_SYNTAX)
Error description: An invalid attribute value was specified.

Could someone suggest what's wrong with the attribute name?


OpenLDAP never produces the text you provided above. It seems you're using 
some other LDAP tool to do this import, and it is not showing you the actual 
error message sent from the server. OpenLDAP slapd will always identify the 
actual attribute and value that causes an error. I suggest you try importing 
this entry with OpenLDAP's ldapadd and examine the error message from there.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: LDAP and PAM: account is expired, but pam_ldap allows authentification

2011-01-12 Thread Howard Chu

Indexer wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


On 13/01/2011, at 17:45, Konstantin Boyandin wrote:


Hello,

Could someone direct me to the source of wisdom to solve this: I have
set correctly the fields (attributes)

shadowExpire
shadowLastChange
shadowMin
shadowMax

to make the account expired (OpenLDAP used to run NT domain), but when I
ssh to a server using pam_ldap authentication, it is still allowed to login.

How pam_ldap should be instructed to take the expiration attributes ito
account?


Isnt this handled via nsswitch? Can you show us your /etc/nsswitch.conf, and 
your /etc/ldap.conf (not your /etc/openldap/ldap.conf


As a reminder - the OpenLDAP-technical list is for the discussion of actual 
OpenLDAP software, as well as how to make other software interoperate with it. 
Questions that are purely about how to use 3rd party software foo work at 
all do not belong on this list.


There is no evidence that the original poster is having any trouble using 
OpenLDAP. His question is entirely about making 3rd party software work, and 
those questions belong on the support forums for those 3rd party software 
packages.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: LDAP and PAM: account is expired, but pam_ldap allows authentification

2011-01-13 Thread Howard Chu

Indexer wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1




shadowExpire
shadowLastChange
shadowMin
shadowMax

to make the account expired (OpenLDAP used to run NT domain), but when I
ssh to a server using pam_ldap authentication, it is still allowed to login.


This look to be a question where the user does not know what is
responsible

for the issue he is seeing, but does relate to his attempt to use OpenLDAP. He
is correct in asking here, and helpfully pointing him in the correct direction
is the right course of action, rather than saying you are wrong to ask this
here. This problem may have been to him related to missing elements from his
user objects (which would have been openldap) or it was anything else.

Pointing him to pam_ldap was the correct action.


Also you said



As a reminder - the OpenLDAP-technical list is for the discussion of
actual OpenLDAP software, as well as how to make other software
interoperate with it. Questions that are purely about how to use 3rd
party software foo work at all do not belong on this list.



This counts as other software interoperate with it. from where I am
sitting. I have seen many questions like this, and I think it should be
something we answer and point people in the correct direction of rather
than saying you'll get no help here



So instead of going to a doctor to be referred to a specialist, you will go
straight to a specialist without knowing what your problem is? makes
complete sense.


It was obvious that he was not asking why doesn't my pam_ldap talk to my 
OpenLDAP server.


Missing elements from the user objects is a *data* problem, it is not an 
interoperability problem. He would have the same issue whether the server was 
OpenLDAP, Oracle, or M$AD. It has nothing to do with OpenLDAP, and a careful 
reader would have known all of this. If you're not reading carefully, you 
should not be responding to the posts.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX

2011-01-13 Thread Howard Chu

Konstantin Boyandin wrote:

13.01.2011 13:39, Howard Chu writes:

Konstantin Boyandin wrote:

Hello,

OpenLDAP version: 2.3.43-12 (CentOS 5.5), 64-bit.

In order to enable ppolicy overlay, I am trying to create the relevant
entries, as specified in

http://www.openldap.org/doc/admin24/overlays.html#Password%20Policies

I import two LDIFs, first:

dn: ou=Policies,dc=example,dc=com
objectClass: organizationalUnit
objectClass: top
ou: Policies

and second

dn: cn=default,ou=Policies,dc=example,dc=com
cn: default
objectClass: top
objectClass: pwdPolicy
objectClass: person
pwdAllowUserChange: TRUE
pwdAttribute: userPassword
pwdCheckQuality: 2
pwdExpireWarning: 600
pwdFailureCountInterval: 30
pwdGraceAuthNLimit: 2
pwdInHistory: 5
pwdLockout: TRUE
pwdLockoutDuration: 0
pwdMaxAge: 7776000
pwdMaxFailure: 5
pwdMinAge: 0
pwdMinLength: 5
pwdMustChange: FALSE
pwdSafeModify: FALSE
sn: dummy value

The first loads OK.
When I try to import the second, I receive this diagnostics:

Could not add object cn=default,ou=Policies,dc=itelsib,dc=com
Message: Invalid syntax
Error code: 0x15 (LDAP_INVALID_SYNTAX)
Error description: An invalid attribute value was specified.

Could someone suggest what's wrong with the attribute name?


OpenLDAP never produces the text you provided above. It seems you're
using some other LDAP tool to do this import, and it is not showing you
the actual error message sent from the server. OpenLDAP slapd will
always identify the actual attribute and value that causes an error. I
suggest you try importing this entry with OpenLDAP's ldapadd and examine
the error message from there.


I tried importing with slapadd. The output:

str2entry: invalid value for attributeType pwdAttribute #0 (syntax
1.3.6.1.4.1.1466.115.121.1.38)
slapadd: could not parse entry (line=22)

The error above refers to the allowed value of pwdAttribute, which can
only be userPassword now.

The problem is the value for this attribute in LDIF *is* userPassword,
as in the cited sample. I checked the LDIF - no 'invisible' characters
around the value.


Sounds like you don't actually have the ppolicy overlay configured on the 
database you're loading into. The pwdAttribute syntax handler is part of the 
ppolicy overlay and will only get installed if you configure the overlay on 
the target database.


JFYI, I checked the values for the attributes using man page. This, and
other references provided with packages is where I look first prior to
asking on the Net.




--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX

2011-01-13 Thread Howard Chu

Pierangelo Masarati wrote:

Chris Jacobs wrote:

Perhaps try man slapo_ppolicy


The man page name is slapo-ppolicy(5).


- it should hopefully provide the limits and acceptable values and compare with your ldif 
to find the cause of Error description: An invalid attribute value was 
specified.

Alternative: reduce the number of attributes (divide and conquer) to find the 
culprit.

Perhaps also checking the schema file for the limits or acceptable values.


Or check the archives, e.g.
http://www.openldap.org/lists/openldap-software/200802/msg00337.html:
for some time, in OpenLDAP 2.3, the pwdAttribute could only contain OIDs.


That issue was fixed long before 2.3.43, which he says he is running.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Problems importing ppolicy LDIF: LDAP_INVALID_SYNTAX

2011-01-13 Thread Howard Chu

Pierangelo Masarati wrote:

Konstantin Boyandin wrote:


Or check the archives, e.g.
http://www.openldap.org/lists/openldap-software/200802/msg00337.html:
for some time, in OpenLDAP 2.3, the pwdAttribute could only contain OIDs.


Thank you very much!
After I changed the string to

pwdAttribute: 2.5.4.35

the import was a success. This problem's solved. So reading Web *can* be
of more use than reading man pages only.


The archives of the OpenLDAP project are indeed part of the web.  In
order to get valuable information you need to be able to dig it out from
tons of s**t, though.


Indeed. In fact Dieter's answer was already 3 years out of date when he posted 
it. The issue in question is ITS#4025 which was fixed in September 2005 and 
released in OpenLDAP 2.3.8.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Best and most efficient way to upgrade openLDAP

2011-01-18 Thread Howard Chu

Ghyslain Tissot wrote:

Hello,
   We are looking for input on how to upgrade openLDAP from 2.2 to 2.3. We
have looked and it seems the only documented way to upgrade is by unloading
and reloading the data which is very time consuming (approx. 8-10 hours).
Since this is a production system it is not adequate. What options are 
available?


You should realize that you're moving from one unsupported release to another 
unsupported release, both have been obsoleted by OpenLDAP 2.4 for a number of 
years already.



To give more more background:

- there are currently approximately 7M entries in the LDAP;
- we are migrating to a new server with a new version of openLDAP (dependant
on RHEL version - previously RHEL 4 now RHEL 5);
- we tried the option creating a staging server with 2.2 and importing on the
new system (v2.3) the extracted data on the new system. Then we activated
replication (SLURPD) between the two and tried updating the staging server
with the transaction logs of production hoping the data would then replicate
to the new system. But that does not work.


Despite slurpd's many flaws, this approach should have worked fine. Since you 
provide no details, there's no way to tell what went wrong. But in the 
meantime, slurpd is also obsolete and no longer supported.



Any ideas on how it can be done with minimal impact to the current LDAP and
reducing the outage period?


The outage period should only last as long as it takes to run slapcat on your 
current server. I would go one step further: edit your 2.2 slapd.conf and turn 
on read-only mode - then your outage for servicing reads is only as long as it 
takes to restart the server. Once the server is in read-only mode you can run 
slapcat while slapd is running so there's no further service impact.


Loading on the new server, read the new slapadd(8) manpage, look at using the 
-q option.


Do yourself a favor and use OpenLDAP 2.4. Questions about 2.3 or older will be 
ignored by the Project.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Failover Failure Advice

2011-01-18 Thread Howard Chu

Anton Chu wrote:

I've setup a master and slave ldap service for failover; however, I'd like
some advice on how to keep the ldap clients cached with the ldap creds if ever
the master and slave ldap server goes.  I've tried to extend the time of the
caching on nscd - name server caching daemon - but it doesnt work when I add
ldap users to certain groups.  I've also tried pam caching credentials but
doesn't work that well either.  Finally, I also tried sssd but couldn't get it
to work on my Ubuntu 10.10 clients.  Anyone have simple solution that works
when slave and master ldap servers get out of commission?  I've thought about
getent passwd  /etc/passwd cron job, etc.


Setup OpenLDAP nssov on all clients, use proxycache overlay and/or syncrepl to 
continue operating when servers and/or networks fail.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: nss_override_attribute_value

2011-01-19 Thread Howard Chu

Hugo Monteiro wrote:

Hello,

I was wondering if there are any plans to make
nss_override/default_attribute_value allow the use of env vars to
perform substitutions.

Something along the line of

nss_override_attribute_valuehomeDirectory/home/users/${USER}

This would allow great flexibility, specially when combined with pam_env
and such.


You're on the wrong list. There is no piece of OpenLDAP software that uses 
nss_override_attribute_value so discussing enhancements to it is pretty far 
off base.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: How to enable 'pwdPolicySubentry' in ppolicy.schema

2011-01-20 Thread Howard Chu

Alexey Shalin wrote:

Hello,

How to enable  'pwdPolicySubentry' in ppolicy.schema, I added this into
ppolicy.schema


Never modify the schema files distributed with OpenLDAP.


attributetype ( 1.3.6.1.4.1.42.2.27.8.1.23

   NAME 'pwdPolicySubentry'

   DESC 'The pwdPolicy subentry in effect for this object'

   EQUALITY distinguishedNameMatch

   SYNTAX 1.3.6.1.4.1.1466.115.121.1.12

   SINGLE-VALUE

   USAGE directoryOperation )

But after that my slapd do not started.


Of course.

Schema files are only for defining user attributes. Operational attributes 
must be implemented in code and cannot be defined from a schema config file.


This particular attribute is already implemented in the ppolicy overlay so 
there is no need to define it again anyway.



should I upgrade openldap to the last ver ?


That would make no difference here, but it's always best to stay up to date.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Exported group can't be imported again: Invalid syntax (21) additional info: objectClass: value #0 invalid per syntax

2011-01-21 Thread Howard Chu

Thomas Schweikle wrote:

Am 21.01.2011 13:19, schrieb Brian Candler:

On Thu, Jan 20, 2011 at 08:04:00PM +0100, Thomas Schweikle wrote:

The group I want to add:

dn: cn=somegroup,ou=Groups,dc=example,dc=org
objectClass: posixGroups
objectClass: top
gidNumer: 3000
cn: somegroup
memberUid: someuser


Are you sure you mean posixGroups, not posixGroup ?
And gidNumer instead of gidNumber ?

This was the point: the export was with posixGroups, but the newer
schema only allowed posixGroup. Must have been changed sometime ... :(


This schema has not changed in a dozen years. Your previous LDAP server simply 
didn't do schema validation and allowed you to store whatever garbage you gave 
it without checking.



Have a look at nis.schema (or nis.ldif) to see what attributes are required
or permitted for posixGroup.

I corrected all posixGroups to posixGroup and import worked!


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: slapd : segmentation violation with Mingw/MSYS

2011-01-25 Thread Howard Chu

Jean-François Berne wrote:

   Hello *
I need help from experienced users concerning openLDAP under Mingw : I
get a SIGSEGV upon slapd invocation.
Please take a look at what I did and tell what I do wrong. Perhaps
someone already had this problem.
The SIGSEGV is at the end.



2. result: you can spot SIGSEGV right at the beginning.

GNU gdb (GDB) 7.2
[...] This GDB was configured as mingw32.
[...] Reading symbols from
C:\MinGW\openldap-2.4.23\libexec/slapd.exe...(no debugging symbols
found)...done.

(gdb) start
Function main not defined.
Make breakpoint pending on future shared library load? (y or [n]) n
Starting program: C:\MinGW\openldap-2.4.23\libexec/slapd.exe
[New Thread 3572.0xbbc]

Program received signal SIGSEGV, Segmentation fault.
0x6086ec49 in msys-1!getuid () from C:\MinGW\msys\1.0\bin\msys-1.0.dll
(gdb) quit


Looks like you built an MSYS binary, since you're crashing inside the msys 
DLL. That's not a good idea, you should be configuring gcc to build a pure 
Win32 binary instead.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: MemberOf attribute not being returned

2011-01-27 Thread Howard Chu

Mark Cairney wrote:

Hi,

I'm sure this was working in the past on this server but Im now not getting 
anything returned when I request the memberOf attribute.

I compiled OpenLDAP 2.4.23 with the following flags:

./configure --prefix=/usr/local/authz --enable-meta --enable-ldap --enable-bdb 
--enable-monitor --enable-syncprov --enable-translucent --enable-memberof 
--enable-dyngroup --enable-dynlist --with-threads --with-tls --with-cyrus-sasl 
--enable-syslog --enable-spasswd cd  make depend make make test make install

I'm using slapd.d and I have the following in 
/usr/local/authz/etc/openldap/slapd.d/cn=config/olcDatabase={1}bdb

olcOverlay={0}dynlist.ldif
olcOverlay={1}memberof.ldif
olcOverlay={2}syncprov.ldif

The contents of olcOverlay\=\{1\}memberof.ldif are:


You should not be poking or peeking at the files inside slapd.d. You should be 
using slapcat -n0 or ldapsearch -b cn=config to show the contents of the 
config database. As with other slapd databases, its structure and format are 
subject to change without notice at any time. The only thing guaranteed to 
remain compatible is the LDAP interfaces to the database.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: How to get the socket to listen for replies from referral connections?

2011-01-28 Thread Howard Chu

Ian Puleston wrote:

Hi,

Some time back I implemented an OpenLDAP-based LDAP client in a network

appliance that uses code based on the tools for LDAP searches etc. I used the
tool code because it gave me a nice easy LDIF-based API, but the downside was
that each search operation was totally synchronous, so I'm now working on
updating it to support asynchronous searches.


I have a daemon task that listens for replies from the server after a

request is set, and that then calls ldap_result, ldap_parse_result, etc. -
basically all the things that ldap_do_search does synchronously. The daemon
task, being ouside of OpenLDAP, gets the socket to listen on for each request
via ldap_get_option with LDAP_OPT_SOCKBUF. This is working well.


The problem is when new connections are opened when a referral/reference
is

followed. In this case the daemon needs to listen for replies on these too,
but there is no way that I can see to get at its socket from outside the
OpenLDAP core.

LDAP_OPT_SOCKBUF gets the socket for the primary connection and there is
no

equivalent mechanism to get the sockets for the referral connections.


What I've had to do is to add a new LDAP_OPT_REF_SOCKBUFS in my local

OpenLDAP code to return a list of the socket buffers for additional
connections opened for referrals.


I am using an oldish version - 2.3.32 so:

- am I missing any alternate way to get these sockets to listen on?


Yes. Turn off automatic referral chasing and take care of referrals in your 
own code, then you always have access to every socket.



- has anything been added in more recent versions that will help with this?


Yes, look at LDAP_OPT_CONNECT_CB in 2.4. You can set a callback that is 
invoked right after a connection is opened and right after it's closed. Your 
callback function gets the sockbuf as one of the parameters.



- if not would this new option LDAP_OPT_REF_SOCKBUFS be a useful addition? If 
so I could submit it via the bug reporting.


No, I don't think it's needed.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Authentication for on the fly configuration updates in OpenLDAP 2.4

2011-01-28 Thread Howard Chu

Dan White wrote:

I have discovered this myself, and I personally just rebuild from my own
slapd.conf. I just took a look at the debian/slapd.conf template file in
squeeze, which presumably is what the package installation uses to
ultimately generate the slapd.d config backend. I've copied it here:

http://web.olp.net/dwhite/openldap/slapd-squeeze-default.conf

This config is missing two pretty important items in my opinion:

authz-regexp
gidNumber=0\\\+uidNumber=0,cn=peercred,cn=external,cn=auth
cn=admin,@SUFFIX@

and

databaseconfig
rootdn  cn=admin,@SUFFIX@

See:

http://www.openldap.org/lists/openldap-technical/201101/msg00047.html


Your recommendation assumes that a typical slapd installation has only one 
main database, and the local host sysadmin is also the LDAP DB admin. In other 
scenarios where there are multiple databases, it's more appropriate to leave 
the cn=config rootdn at its default and separate the role of slapd 
administrator from regular database admin.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: AccessLog behaviour

2011-02-03 Thread Howard Chu

Marco Pizzoli wrote:

Hi all,
I would like to know the internal behaviour of slapo-accesslog.


Read the slapo-accesslog(5) manpage. Re-read the logpurge description and 
don't skip any of it.



I would like to have accesslog with a data retention of 10 days. In my ldap
usage scenario this retention produce an accesslog db of quite 10GB of data.

Things are working quite fine, but when the log db contains data older than
the retention I defined after restart slapd has a thread allocated at 100% of
cpu of one processor.
I logged at loglevel 992 = 512 + 256 + 128 + 64 + 32, blocked with iptables
every external ldap access and saw these lines in the log:

[cut]
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to
reqStart=20110202042854.08Z,cn=log03,dc=mycorp.it http://mycorp.it
reqStart requested
Feb  3 09:37:08 ldap03 slapd[16289]: = root access granted
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted
by manage(=mwrscxd)
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter 6
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter
Feb  3 09:37:08 ldap03 slapd[16289]: LE
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to
reqStart=20110202042854.10Z,cn=log03,dc=mycorp.it reqStart requested
Feb  3 09:37:08 ldap03 slapd[16289]: = root access granted
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted
by manage(=mwrscxd)
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter 6
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter
Feb  3 09:37:08 ldap03 slapd[16289]: LE
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to
reqStart=20110202042854.12Z,cn=log03,dc=mycorp.it reqStart requested
Feb  3 09:37:08 ldap03 slapd[16289]: = root access granted
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted
by manage(=mwrscxd)
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter 6
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter
Feb  3 09:37:08 ldap03 slapd[16289]: LE
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to
reqStart=20110202042854.14Z,cn=log03,dc=mycorp.it reqStart requested
Feb  3 09:37:08 ldap03 slapd[16289]: = root access granted
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted
by manage(=mwrscxd)
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter 6
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter
Feb  3 09:37:08 ldap03 slapd[16289]: LE
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access to
reqStart=20110202042855.00Z,cn=log03,dc=mycorp.it reqStart requested
Feb  3 09:37:08 ldap03 slapd[16289]: = root access granted
Feb  3 09:37:08 ldap03 slapd[16289]: = access_allowed: search access granted
by manage(=mwrscxd)
Feb  3 09:37:08 ldap03 slapd[16289]: = test_filter 6
Feb  3 09:37:09 ldap03 slapd[16289]: = test_filter
Feb  3 09:37:09 ldap03 slapd[16289]: LE
Feb  3 09:37:09 ldap03 slapd[16289]: = access_allowed: search access to
reqStart=20110202042855.02Z,cn=log03,dc=mycorp.it reqStart requested
Feb  3 09:37:09 ldap03 slapd[16289]: = root access granted
Feb  3 09:37:09 ldap03 slapd[16289]: = access_allowed: search access granted
by manage(=mwrscxd)
Feb  3 09:37:09 ldap03 slapd[16289]: = test_filter 6
Feb  3 09:37:09 ldap03 slapd[16289]: = test_filter
Feb  3 09:37:09 ldap03 slapd[16289]: LE
Feb  3 09:37:09 ldap03 slapd[16289]: = access_allowed: search access to
reqStart=20110202042855.04Z,cn=log03,dc=mycorp.it reqStart requested
[cut]

I could assume that slapo-accesslog is querying all entries older than the
retention and passes to delete them.

What if this step to end the job will take more time than the interval time of
re-starting?
Augmenting the frequency of this job could help in this meaning?

Every best practice to follow to use accesllog effectively will be very
appreciated! :-)

Thanks
   Marco Pizzoli


--
_
Non è forte chi non cade, ma chi cadendo ha la forza di rialzarsi.
 Jim Morrison



--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: How to use LDAP_OPT_CONNECT_ASYNC?

2011-02-03 Thread Howard Chu

Ian Puleston wrote:

Hi,

I'm trying to use the new LDAP_OPT_CONNECT_ASYNC with OpenLDAP 2.4.23 client 
and I can't see how it can be done.

After creating the connection structure via ldap_initialize() I then call 
ldap_sasl_bind() to do the bind. That calls down to ldap_new_connection() 
(ldap_sasl_bind -  ldap_send_initial_request -  ldap_open_defconn -  
ldap_new_connection) and that implements the asynchronous connect, setting the 
lconn_status to LDAP_CONNST_CONNECTING and not calling ldap_int_poll() in 
ldap_pvt_connect().

So far so good. But ldap_send_initial_request() then gets a good return code 
from ldap_open_defconn(), and so it goes on to call ldap_send_server_request() 
to send the bind, and that then calls ldap_int_poll() which waits for the 
connect to complete, making the whole operation synchronous.

It seems to me that for this to work asynchronously, either:

1. ldap_send_initial_request() should check for lconn_status to 
LDAP_CONNST_CONNECTING on return from ldap_open_defconn(), and if it is then 
return, probably with error LDAP_X_CONNECTING. It should then be possible to 
re-call ldap_sasl_bind() when the connect has completed.

2. Establish the connection via ldap_new_connection() before calling 
ldap_sasl_bind(). But the functions to do that don't appear to be available on 
the API. ldap_open() could have probably done it, but it is now deprecated. So 
this would appear to require a new API function that would simply call 
ldap_open_defconn() and return LDAP_X_CONNECTING.

Or is there some other way to achieve this?


Since ldap_open_defconn() is not named ldap_int_xxx or ldap_pvt_xxx I'd guess 
you should just use that. Another alternative is to simply use your own code 
to create a connection, and call ldap_init_fd() when you're ready.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: 'Inheriting' an attribute from a group.

2011-02-04 Thread Howard Chu

ian+openl...@comtek.co.uk wrote:

Is there a recommended, 'production' way to inherit certain attributes
from a group? Perhaps some kind of overlay?

For example, if a user is in the 'users' group and the users group has
the attribute userQuota=100 then the user will appear to have userQuota=100.

If, on the other hand, userQuota=200 is set on the user then that would
override the current value.


I don't think there's anything in the current code that does this, but you 
could modify the collect (collective attributes) overlay to do it.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Compile contrib modules?

2011-02-07 Thread Howard Chu

Marco Pizzoli wrote:

Hi list,
Today I downloaded the HEAD repository and tried to compile also the contrib
module lastbind.
With the configure script I'm not able to have that module built in.

I have to do something particular?


The configure script doesn't do anything with contrib.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: slapo-lastbind

2011-02-07 Thread Howard Chu

Marco Pizzoli wrote:

On 7 Feb 2011 22:31, Marco Pizzoli marco.pizz...@gmail.com
mailto:marco.pizz...@gmail.com wrote:
  Hi Jonathan,
  I understand the locality of the attribute, but my question is motivated
  by the ppolicy_forward_updates directive that addresses this identical
  problem.
  Do you think it would be possible to have a similar feature with this
  overlay?
 
  Thanks
  Marco
  On 7 Feb 2011 20:49, Jonathan CLARKE jonathan.cla...@normation.com
mailto:jonathan.cla...@normation.com
  wrote:


Note that authTimestamp is a DSA-specific attribute. (Look at its schema 
definition, it is dsaOperation.) Such attributes are, by design, not 
replicated, since their content is only meant for and valid on the particular DSA.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: slapd.d syntax help for ldap proxy server

2011-02-07 Thread Howard Chu

Anton Chu wrote:

Does anyone have a working ldap proxy configuration script?  Some attributes
such as olcURI are not welcomed with slapd on ubuntu 10.10.  My goals is to
make a standalone proxy.


Make sure you're actually using a valid schema.

ldapsearch -Y EXTERNAL -H ldapi:/// -b cn=schema,cn=config -s base

Read the definition of the olcLDAPConfig objectclass.

If you still can't figure that out, then just write a regular slapd.conf and 
convert it to cn=config format using slaptest.



TIA,
Anton

On Fri, Feb 4, 2011 at 12:46 PM, Dieter Kluenter die...@dkluenter.de
mailto:die...@dkluenter.de wrote:

Am Fri, 4 Feb 2011 11:45:36 -0800
schrieb Anton Chu anton@telecommand.com
mailto:anton@telecommand.com:

  I'm trying to setup a ldap proxy server for push based replication.
  I'm in need of help with providing the correct syntax on installing a
  ldap proxy using slapd.d instead of slapd.conf.The items in bold
  are the questionable syntax that can crossover to slapd.d.  Here's my
  slapd.d configuration:
 
 
  Standalone LDAP Proxy:
  
   # load the schemas
   ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/cosine.ldif
  
   ldapadd -Y EXTERNAL -H ldapi:///
   -f /etc/ldap/schema/inetorgperson.ldif
  
   ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/nis.ldif
  
   ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/misc.ldif
  
   ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/ldapns.ldif
  
   ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/openldap.ldif
  
  
# Load dynamic backend modules
  
   dn: cn=module,cn=config
  
   objectClass: olcModuleList
  
   cn: module
  
   olcModulepath: /usr/lib/ldap
  
   olcModuleload: back_hdb
  
   olcModuleload: syncprov
  
  
# Database settings
  
   dn: olcDatabase=hdb,cn=config
  
   objectClass: olcDatabaseConfig
  
   objectClass: olcHdbConfig
  
   olcDatabase: {1}hdb

This should be a ldap database, not a hdb database
  
   databaseldap
   # ignore conflicts with other databases, as we need to push
   out to same suffix hidden  on
   suffix dc=suretecsystems,dc=com
   rootdn cn=slapd-ldap
   uri ldap://localhost:9012/
  
   lastmod on
  
   # We don't need any access to this DSA
   restrictall
  
   acl-bindbindmethod=simple
   binddn=cn=replicator,dc=suretecsystems,dc=com
   credentials=testing
  
   syncreplrid=001
   provider=ldap://localhost:9011/
   binddn=cn=replicator,dc=suretecsystems,dc=com
   bindmethod=simple
   credentials=testing
   searchbase=dc=suretecsystems,dc=com
   type=refreshAndPersist
   retry=5 5 300 5
  
   overlay syncprov

-Dieter

--
Dieter Klünter | Systemberatung
http://dkluenter.de
GPG Key ID:DA147B05
53°37'09,95N
10°08'02,42E





--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Multimaster replication of cn=config working too well?!

2011-02-08 Thread Howard Chu

Mark Cairney wrote:

Hi,

Has anyone experienced problems with the cn=config directory syncing the same 
change across all nodes and breaking the configuration?

In the past I've had problems where the syncrepl parameters are synced

across all nodes, including that node itself, which results in the servers
then trying to sync with themselves.


The most recent problem however was that the olcSaslHost value was synced
to

be one particular server on all 3 nodes. As I was using GSSAPI to do the
replication this broke it completely and it meant that I could only bind via
GSSAPI to one of the 3 servers.


In the meantime I've set the servers not to syncrepl cn=config but this isn't 
ideal.

Anyone else on the list had similar problems?


The potential for royally screwing things up has been discussed on the -devel 
list from time to time. The use of serverIDs and explicit URL matching was 
introduced to prevent consumers pointing at themselves, that's already documented.


For other settings, it's probably best to add some excluded attributes to your 
consumer configs.


And of course, turning off multimaster replication of cn=config is always a 
good possibility. Nobody said you *had* to do it. We only document it to show 
that you *can* do it if you need it, that doesn't mean it's a recommended 
scenario.




Kind regards,

Mark

/*
Mark Cairney
ITI UNIX Section
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: mark.cair...@ed.ac.uk

*/





--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Handling slapd.d in OpenLDAP and Kerberos

2011-02-08 Thread Howard Chu

Jaap Winius wrote:

Quoting sarathkrishn...@gmail.comsarathkrishn...@gmail.com:


For authenticating via OpenLDAP, the principles needs to be rewritten (using
authz-policy and authz-regexp). We know how to do
that in older version of OpenLDAP which had (slapd.conf) but don't know how
to do the same in new OpenLDAP which has slapd.d directory instead.
The manuals also doesn't say anything on this issue.


The switch from slapd.conf to cn=config takes a little getting used
to, plus the migration script may not work for you, but in the end I
produced a set of procedures that should tell you most of what you
want to know:

* Integrated Kerberos-OpenLDAP provider on Debian squeeze
  http://www.rjsystems.nl/en/2100-d6-kerberos-openldap-provider.php

* Integrated Kerberos-OpenLDAP consumer on Debian squeeze
  http://www.rjsystems.nl/en/2100-d6-kerberos-openldap-consumer.php



True, I didn't use Ubuntu in these examples, but I would not be
surprised if the procedures were almost identical, certainly with
cn=config.


If you read
  http://highlandsun.com/hyc/drafts/draft-chu-ldap-xordered-xx.html

You could simplify your ACL changes in 7.1.1.x.

dn: olcDatabase={1}hdb,cn=config
changetype: modify
delete: olcAccess
olcAccess: {2}
olcAccess: {1}
olcAccess: {0}
-

Similarly in 7.1.2.x you don't need to specify the prefixes when you're adding 
rules in order.




Cheers,

Jaap




--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: slapd.conf for proxy to AD

2011-02-09 Thread Howard Chu

Buchan Milne wrote:

On Wednesday, 9 February 2011 01:13:38 masar...@aero.polimi.it wrote:



As far as I recall, what you need is not possible.  You can:

- have authenticated users proxied with their identity asserted, or

- all users, including unauthenticated ones auth'd as a fixed identity

but not both.  Please note that you're asking OpenLDAP's slapd to bridge
the gap between two broken pieces of code: clients that cannot be
configured to bind,


Because the OP wants to have anonymous access for some clients of the proxy
does not necessarily mean the software is broken, there could be
organisational reasons (e.g. AD administrator not prepared to create multiple
proxy accounts for different applications).


and AD that cannot be configured to accept anonymous
requests (AFAIK).


Again, AD can be configured to allow anonymous binds, but AFAIK as of Windows
2003, it is disabled by default, and many AD security standards (used by e.g.
financial auditing companies in their IT auditing) mandate that it not be
enabled.


Feel free to suggest an enhancement that allows to
handle this scenario, though.


IMHO, it would make sense to support this mode of operation.


Sorry but that just doesn't compute. If you have organizational security 
standards that are being audited and they forbid anonymous access, then 
allowing anonymous access to an OpenLDAP proxy that connects to AD is going to 
be equally forbidden.


Come back with an argument that actually makes sense from a security 
perspective first. Otherwise this is just creating back doors to do end-runs 
around your sysadmins and company policies.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: port level security for auth and anon

2011-02-11 Thread Howard Chu

Christopher Louis Jackson wrote:

I am looking for help with setup of security with my openldap config.
I currently have RHEL 6 with ldap:// and ldaps:// working for both auth binds
and anon binds.
What I want to do is allow anon binds on ldap:// and require authentication
over an encrypted stream on ldaps://
my current access is set to:
access to attrs=userPassword
by anonymous auth
by self read
by * none
access to *
by * read
I do not have a security statement in my slapd.conf.
I have tried a few things such as changing the userpassword access to:
access to userPassword
  by anonymous auth sasl_ssf=128 break
  by anonymous auth tls=128
  by self read
but the syntax is not correct and the config will not load with above.
Any help would be great.


As the slapd.access(5) manpage clearly states, the syntax is
 access to what [ by who [ access ] [ control ] ]+

sasl_ssf=128 is a who specifier but you have it after the access 
specifier.

We don't just write things randomly. Read and follow what's actually written 
in front of you.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Mac OS X OpenLDAP allows anonymous access to all fields

2011-02-11 Thread Howard Chu

RAT wrote:

The ongoing saga...

While trying to add the ACL for to hide userPassword, we get the following 
error:

ldap_modify: Insufficient access (50)

I'm beginning to think Apple has this locked down...


You should ask Apple.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: How to use LDAP_OPT_CONNECT_ASYNC?

2011-02-11 Thread Howard Chu

Ian Puleston wrote:

-Original Message-
From: masar...@aero.polimi.it [mailto:masar...@aero.polimi.it]
Sent: Thursday, February 03, 2011 9:35 PM

ldap_int_poll() will return -2 if timeout is 0 and the filedes was not
active.  So ldap_send_server_request() will return LDAP_X_CONNECTING if
async.


I've hit a new problem with this which I'm pretty sure is a bug (or else an 
unsupported case) which is that LDAP_OPT_CONNECT_ASYNC doesn't work with TLS 
connections. It is a timing issue so it may work fine in some implementations, 
and in fact it works fine for me every time if I have any debug output turned 
on.

The problem is that ldap_int_tls_connect fails, and I'm pretty sure that it 
fails because it tries to start the TLS handshake before the connect has 
completed. Adding just one printf at the start of ldap_int_tls_connect delays 
it enough that it then works OK every time, presumably because the connect then 
completes first.

I'm working on a fix now, and I think what is needed is:

1. A call to ldap_int_poll in ldap_int_tls_start if async. Then it should abort 
without calling ldap_int_tls_connect if not ready (probably return 
LDAP_X_CONNECTING).

2. In ldap_send_initial_request don't call ldap_send_server_request if 
lconn_status is LDAP_CONNST_CONNECTING and using TLS (since TLS needs to be 
started first). Instead return LDAP_X_CONNECTING.

3. Probably something similar to #1 for the case where ldap_start_tls_s is 
called from ldap_new_connection.

If this all works I'll put in a bug report with a patch.


Sounds to me like you should just file the bug report first. The patch can 
come later. Also I don't believe the proper fix should touch so many places. 
All you should need is to fix the caller of ldap_int_tls_connect(), and as 
that's a static function there is only one place to fix.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: How to use LDAP_OPT_CONNECT_ASYNC?

2011-02-12 Thread Howard Chu

Ian Puleston wrote:

Hi Howard,


-Original Message-
From: Howard Chu [mailto:h...@symas.com]

Ian Puleston wrote:

I'm working on a fix now, and I think what is needed is:

1. A call to ldap_int_poll in ldap_int_tls_start if async. Then it

should abort without calling ldap_int_tls_connect if not ready
(probably return LDAP_X_CONNECTING).


2. In ldap_send_initial_request don't call ldap_send_server_request

if lconn_status is LDAP_CONNST_CONNECTING and using TLS (since TLS
needs to be started first). Instead return LDAP_X_CONNECTING.


3. Probably something similar to #1 for the case where

ldap_start_tls_s is called from ldap_new_connection.


If this all works I'll put in a bug report with a patch.



Sounds to me like you should just file the bug report first. The patch
can come later.


I need to get this fixed in the port I am using so I will do that and
attach the patch to the bug report. The relevant openldap developers can
then choose whether to use that patch as-is or work out something
different, and if the latter I can replace the fix in my port with it when
it gets released.


You seem to be making the bizarre assumption that only you and other OpenLDAP 
developers will care. Filing the bug report even before you have the patch 
allows other people in the community to be alerted and get involved. That is 
the purpose of an open source community, and the purpose of a public bug tracker.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Ppolicy does not seem to work

2011-02-14 Thread Howard Chu

Jan Kohnert wrote:

So there comes the next question: Is there a way to lock out specific users
permanently (other than creating a cronjob setting the lockout time new after
900s) or do I need to set pwdLockoutDuration to inf and so are forced to
manually reset users whose accounts were tried to be cracked?


Read the slapo-ppolicy manpage again. This is explicitly documented.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Slapd Security based on port

2011-02-15 Thread Howard Chu

Andrew Findlay wrote:

On Mon, Feb 14, 2011 at 07:49:10PM +, Chris Jackson wrote:


I know:
Anonymous bind can be disabled by disallow bind_anon and
Unauthenticated

bind mechanism is disabled by default. But if I use disallow bind_anon it
stops in on both ports. I want to stop it just on ldaps://.


Maybe you should stop thinking about ports and start thinking about
*where* the LDAP clients are. You can then permit anon access to clients
within your own network (by IP range) and permit access by any
authenticated user, before denying all other cases. Remember to allow
enough access for the external users to connect and bind in the first
place!

Note that it is almost impossible to hide the *existance* of an entry,
so if DNs are guessable it is possible that a determined outsider could
work out who is in your directory.


See the disclose ACL privilege - you can hide the existence if you really 
want to.


slapd's security mechanisms will support just about any conceivable security 
policy.



If some of the data is very sensitive you may prefer to set up an
'outside' server and replicate just a subset of the data to it.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: ldap auth does not works after openldap upgrade

2011-02-16 Thread Howard Chu

Andrew Findlay wrote:

On Tue, Feb 15, 2011 at 05:08:43PM -0200, Leonardo Carneiro wrote:


fileserver:/etc/ldap# /usr/sbin/slapd -h ldapi:/// ldap:/// -g openldap -u
openldap -F /etc/ldap/slapd.d -d 128


Aha! Your server is using LDAP-based config so it is ignoring the config
file entirely.


Does these changes that we are making into slapd.conf really being
processed? Normally, i see just the -F /etc/ldap/slapd.d flag and never
the -f /etc/ldap/slapd.conf.


I suspect the config file was converted to a config dir during the
Debian upgrade process, so the file is now being ignored.

I also suspect that there may not be a valid password set on the
cn=config suffix, so you will not be able to manage the server through
LDAP either.


Since it's starting on ldapi:/// he should just do a SASL EXTERNAL bind on 
ldapi:// using Unix root. Pretty sure Debian packages it with the appropriate 
authz-regexp already configured.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: ldap auth does not works after openldap upgrade

2011-02-16 Thread Howard Chu

Andrew Findlay wrote:

On Wed, Feb 16, 2011 at 02:51:19AM -0800, Howard Chu wrote:


I also suspect that there may not be a valid password set on the
cn=config suffix, so you will not be able to manage the server through
LDAP either.


Since it's starting on ldapi:/// he should just do a SASL EXTERNAL
bind on ldapi:// using Unix root. Pretty sure Debian packages it
with the appropriate authz-regexp already configured.


I don't have a Debian Squeeze server at present so I cannot
check that.

Where is this documented? I am having great trouble finding
any clear description of how to actually access cn=config in
the bootstrap case.


I don't know where Debian documents their bootstrap config, you'll have to ask 
them.



Similarly I cannot find anything that
clearly describes the use of SASL EXTERNAL with ldapi.


http://tools.ietf.org/html/draft-chu-ldap-ldapi-00


If you can point me at some authoritative statements I will
propose a patch for the Admin Guide.

Andrew


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: ldap auth does not works after openldap upgrade

2011-02-16 Thread Howard Chu

Leonardo Carneiro wrote:

On Wed, Feb 16, 2011 at 8:51 AM, Howard Chu h...@symas.com
mailto:h...@symas.com wrote:

Andrew Findlay wrote:

On Tue, Feb 15, 2011 at 05:08:43PM -0200, Leonardo Carneiro wrote:

fileserver:/etc/ldap# /usr/sbin/slapd -h ldapi:/// ldap:/// -g
openldap -u
openldap -F /etc/ldap/slapd.d -d 128


Aha! Your server is using LDAP-based config so it is ignoring the config
file entirely.

Does these changes that we are making into slapd.conf really being
processed? Normally, i see just the -F /etc/ldap/slapd.d flag
and never
the -f /etc/ldap/slapd.conf.


I suspect the config file was converted to a config dir during the
Debian upgrade process, so the file is now being ignored.

I also suspect that there may not be a valid password set on the
cn=config suffix, so you will not be able to manage the server through
LDAP either.


Since it's starting on ldapi:/// he should just do a SASL EXTERNAL bind on
ldapi:// using Unix root. Pretty sure Debian packages it with the
appropriate authz-regexp already configured.

Tks for the tips guys. I'll try all this stuff at launch time here in Brazil
(about 2 hours from now). There are to many users using Samba right now, and
every time I have to restart the OpenLDAP something on samba crashes.


The entire point of using cn=config for configuration is that config changes 
don't require slapd restarts.



As far as i'm concerned, i didn't have the need to use SASL, and it seems that
all this SASL mechanism was auto-introduced in my setup after the upgrade. Is
it safe to edit /etc/defaults/slapd and remove the ldapi:/// parameter in
SLAPD_SERVICES line or i can break something very hard doing this?


Removing that would be dumb. It's there to give you an easy means to 
administer the server.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: ldapi:

2011-02-16 Thread Howard Chu

Andrew Findlay wrote:

On Wed, Feb 16, 2011 at 08:37:01AM -0800, Quanah Gibson-Mount wrote:


I don't get where you're getting this idea from.  I've never been
required to use ldapi for setting up cn=config.  The requirement of
using ldapi for cn=config management is something Debian/Ubuntu is
doing.  Which is why, again, I say this specifically is not
something to be documented by the OpenLDAP project as a cn=config
requirement.


OK - forget the link to cn=config - that is just what got me interested
in ldapi: to start with.

The ldapi: transport seems to be almost undocumented. There is a brief
(but useful) mention in ldap.conf(5) but very little elsewhere.

I have found enough hints in the archives to get started, so I will file
some suggestions.


Everything worth noting about ldapi: was already published as an IETF 
Informational Draft, and I've already posted the link to that document several 
times. Beyond that it's no different from any other LDAP URL.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: ldap design

2011-02-16 Thread Howard Chu

Hendrik van der Ploeg wrote:

Hello,

Thank you for answering.
Yes, every customer has it's own ldap server which should be master.

*/Q: You do not want it the other way around and have 4 (multi-)master in the
datacenter and 1200+ replicas outside, do you? /*
That would be the best design indeed, but the problem is that the customers
need write access to their own ldapserver.

Now I hear you thinking; Customers can have write access in the datacentre and
it then will be replicated to the customers own ldapserver.
The problem is that a lot of customers have quite a bad vpn connection to the
datacentre and when they add a user for example it must be available 
immediatley.
And with a failing vpn connection nothing happens. they MUST be able to write
in their own local ldapserver


Your reasoning is sound. It's just a question of system resources; 1200 
consumer configs in a single slapd process will probably occupy a lot of RAM. 
An active consumer requires a slapd thread when processing incoming changes. 
If you want instantaneous processing of all incoming changes that means you 
must be able to handle 1200 concurrent threads, worst case. If your real world 
load will be lower than that, adjust downward accordingly.


Regards

Hendrik
Noordwijkerhout
Holland




Hendrik van der Ploeg schrieb am 15.02.2011 08:47 Uhr:


I'm in doubt what design I need to use for openldap
This is the situation;

We have 1200+ customers using LDAP. We want to replicate all these ldap
server to 1 big ldapserver in a datacentre with a multi-master config.

This means each customer has its own ldap server and you will have 1200+
ldap servers?


So all the customers are a master-ldap who replicate to the datacentre.

all the customers (=ldap servers) _are_ master-ldap?
This would mean you have 1200+ provider/master!?


My idea was to build in the datacentre a ldapcluster of about 4 server

What for then?


My question is: Will this be stable, because there will be 1200+
ldapservers replicating
to 4 ldapserver in the datacentre.

You do not want it the other way around and have 4 (multi-)master in the
datacenter and 1200+ replicas outside, do you?


I know this depends on the number of write actions at the customers. All I
can say is that write actions at the customers isn't THAT much.

I really hope somebody can give me an answer or maybe there's somebody
else with the same config

We have one provider and 160 consumers - and this is IMHO called a lot
here, if I'm right ...


Marc






--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Conflict resolution on Syncrepl

2011-02-16 Thread Howard Chu

Diego Lima wrote:

Hello all,

This is mostly a theoretical question. I currently have some OpenLDAP
multimaster clusters (using syncrepl and mirrormode) and although it
never happened before, what would happen if two (or more) masters
received a write request exactly at the same time? The entry would
have the same entryCSN on both servers and the contextCSN would be
updated at the same time? Would this lead to a situation where the
servers were actually out of sync but they would think they had the
same entries?


No. On a properly configured setup it is impossible for two servers to produce 
the same entryCSN (regardless of timestamp issues) since they each have unique 
serverIDs. Conflict resolution will always resolve consistently and 
deterministically in syncrepl.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Error: CSN too old and replication fails

2011-02-16 Thread Howard Chu

Howard Chu wrote:

Jérémy Wagner wrote:

Hello,

I'm facing some issues with syncrepl...

The simplest situation in which I was able to reproduce the problem
consists of 1 provider and 1 consumer.

I have configured syncrepl to do a partial replication :

olcSyncrepl:
   {0}rid=105
   provider=ldaps://:636
   bindmethod=simple
   timeout=0
   network-timeout=0
   starttls=no
   filter=(mail=*)
   searchbase=ou=groups,o=mydir
   scope=sub
   schemachecking=off
   attrs=cn description mail member +
   type=refreshAndPersist
   retry=60 +
   tls_cert= tls_cacert= tls_key=

olcSyncrepl:
   {1}rid=107
   provider=ldaps://:636
   bindmethod=simple
   timeout=0
   network-timeout=0
   starttls=no
   filter=(objectclass=*)
   searchbase=ou=operators,o=mydir
   scope=sub
   schemachecking=off
   type=refreshAndPersist
   retry=60 +
   tls_cert= tls_cacert= tls_key=

olcSyncrepl:
   {2}rid=104
   provider=ldaps://:636
   bindmethod=simple
   timeout=0
   network-timeout=0
   starttls=no
   filter=(uid=mytestuser)
   searchbase=ou=people,o=mydir
   scope=sub
   schemachecking=off
   type=refreshAndPersist
   retry=60 +
   tls_cert= tls_cacert= tls_key=

The initial sync (i.e. from an empty base) is fine, but then when I
modify an entry (uid=mytestuser), it is not updated on the consumer, and
I get the
following messages:

Feb 16 15:42:32 do_syncrep2: rid=107 NEW_COOKIE:
rid=107,csn=20110216144232.222152Z#00#000#00
Feb 16 15:42:32 do_syncrep2: rid=104
cookie=rid=104,csn=20110216144232.222152Z#00#000#00
Feb 16 15:42:32 slap_graduate_commit_csn: removing 0x7fb58c0227d0
20110216144232.222152Z#00#000#00
Feb 16 15:42:32 do_syncrep2: rid=104 CSN too old, ignoring
20110216144232.222152Z#00#000#00
Feb 16 15:43:05 do_syncrep2: rid=105 LDAP_RES_INTERMEDIATE - NEW_COOKIE
Feb 16 15:43:05 do_syncrep2: rid=105 NEW_COOKIE:
rid=105,csn=20110216144305.545784Z#00#000#00
Feb 16 15:43:05 slap_queue_csn: queing 0x7fb58c020b50
20110216144305.545784Z#00#000#00
Feb 16 15:43:05 do_syncrep2: rid=107 LDAP_RES_INTERMEDIATE - NEW_COOKIE
Feb 16 15:43:05 do_syncrep2: rid=107 NEW_COOKIE:
rid=107,csn=20110216144305.545784Z#00#000#00
Feb 16 15:43:05 do_syncrep2: rid=104
cookie=rid=104,csn=20110216144305.545784Z#00#000#00
Feb 16 15:43:05 slap_graduate_commit_csn: removing 0x7fb58c0227d0
20110216144305.545784Z#00#000#00
Feb 16 15:43:05 do_syncrep2: rid=104 CSN too old, ignoring
20110216144305.545784Z#00#000#00

(both servers are perfectly in sync with an NTP daemon)

It looks like ITS #6619 is quite similar, and unresolved.

What do these messages mean ? Is there a way to force the update ?


Your current configuration is unsupported. Ordinarily when you configure
multiple consumers in a DB it is assumed they are all pointing at different
providers, and each provider will have a different serverID. In your case,
since it is all a single provider with a single serverID, all 3 consumers will
share a single contextCSN. Any update in any branch will update the contextCSN
to that newest state. Any updates in other branches that hadn't been received
yet (due to random timing between threads) will be received as too old
because the contextCSN already has a newer state.

A workaround for this would be to split each consumer into its own subordinate
database, then each will have a private copy of its own contextCSN to work with.


Another potential workaround, if all of the consumer configs were identical 
except for their baseDN, would be to simply consolidate them all into a single 
consumer, with a more complex search filter.


searchbase=o=mydir
filter=(|
 ((entryDN:dnSubtreematch:=ou=groups,o=mydir)(mail=*))
 (entryDN:dnSubtreematch:=ou=operators,o=mydir)
 ((entryDN:dnSubtreematch:=ou=people,o=mydir)(uid=mytestuser)))

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Conflict resolution on Syncrepl

2011-02-16 Thread Howard Chu

Diego Lima wrote:

No. On a properly configured setup it is impossible for two servers to
produce the same entryCSN (regardless of timestamp issues) since they each
have unique serverIDs. Conflict resolution will always resolve consistently
and deterministically in syncrepl.


Thanks a lot for your explanation! So, should one assume that the
server with the highest serverID will have priority when it comes to
conflict resolution?


Yes.


Also, when and how is the serverID determined? Is
this manually configurable?


Yes. Read the slapd config manpage.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: undefined reference to `ldap_int_tls_impl' (tls2.c) ?

2011-02-18 Thread Howard Chu

Elle Y Suzuki wrote:

make is exiting with several undefined reference to
'ldap_int_tls_impl' error messages:


Pretty interesting. This basically means that configure ran without selecting 
any TLS implementation, but still left TLS enabled. Can you upload your 
config.log file somewhere for us to examine?


./.libs/libldap.a(tls2.o): In function `ldap_pvt_tls_ctx_free':
/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:79:
undefined reference to `ldap_int_tls_impl'
./.libs/libldap.a(tls2.o): In function `tls_init':
/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:163:
undefined reference to `ldap_int_tls_impl'
/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:163:
undefined reference to `ldap_int_tls_impl'
/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:168:
undefined reference to `ldap_int_tls_impl'
./.libs/libldap.a(tls2.o): In function `ldap_pvt_tls_check_hostname':
/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:494:
undefined reference to `ldap_int_tls_impl'
./.libs/libldap.a(tls2.o):/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap/tls2.c:860:
more undefined references to `ldap_int_tls_impl' follow
collect2: ld returned 1 exit status
make[2]: *** [apitest] Error 1
make[2]: Leaving directory
`/home/vmplanet/Downloads/openldap-2.4.24/libraries/libldap'
make[1]: *** [all-common] Error 1
make[1]: Leaving directory
`/home/vmplanet/Downloads/openldap-2.4.24/libraries'
make: *** [all-common] Error 1

what is the proper way to fix this so that i might successfully make?


BACKGROUND
i am attempting to set up and install openldap 2.4.24 on a virtual
kubuntu on windows xp.

'configure' and 'make depend' appear to run successfully.
i execute the following steps:

export CPPFLAGS=-I/usr/include/nss -I/usr/include/nspr
-I/usr/local/BerkeleyDB.5.0/include
export LDFLAGS=-L/usr/lib/nss -L/usr/local/BerkeleyDB.5.0/lib
export LD_LIBRARY_PATH=/home/vmplanet/Downloads/db-5.0.32/build_unix/.libs

configure --enable-shell=yes --without-threads
make depend

(in case you're wondering why back-shell, i've included details in a
previous thread, http://tinyurl.com/4flbgh4.  note that in that thread,
i successfully installed openldap directly on windows, but had questions
about successfully testing back-shell -- and still do.)


going back to the errors above, i note in which files this
'ldap_int_tls_impl' appears with a grep:

./libraries/libldap_r/tls2.c:40:static tls_impl *tls_imp =
ldap_int_tls_impl;
./libraries/libldap_r/tls_g.c:1095:tls_impl ldap_int_tls_impl = {
./libraries/libldap_r/tls_o.c:1253:tls_impl ldap_int_tls_impl = {
./libraries/libldap_r/tls_m.c:3040:tls_impl ldap_int_tls_impl = {
./libraries/libldap/tls2.c:40:static tls_impl *tls_imp =ldap_int_tls_impl;
./libraries/libldap/tls_g.c:1095:tls_impl ldap_int_tls_impl = {
./libraries/libldap/ldap-tls.h:75:extern tls_impl ldap_int_tls_impl;
Binary file ./libraries/libldap/.libs/libldap-2.4.so.2.6.0 matches
Binary file ./libraries/libldap/.libs/libldap.so matches
Binary file ./libraries/libldap/.libs/tls2.o matches
Binary file ./libraries/libldap/.libs/libldap-2.4.so.2 matches
Binary file ./libraries/libldap/.libs/libldap.a matches
Binary file ./libraries/libldap/tls2.o matches
./libraries/libldap/tls_o.c:1253:tls_impl ldap_int_tls_impl = {
./libraries/libldap/tls_m.c:3040:tls_impl ldap_int_tls_impl = {

ldap_int_tls_impl appears (defined) in the fellow tls* files above.  is
this then an indication that tls2.c is not communicating or 'linked' to
its fellow tls* files or ?

the output from 'make depend' appears to show that mkdep traversed the
libldap and libldap_r subdirectories and successfully constructed the
necessary sets of include file dependencies.

admittedly, i am not altogether that familiar with (the intricacies of)
the compile/make process.

please advise; thanks in advance.




--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: threads and concurrency

2011-02-19 Thread Howard Chu

Omer Faruk SEN wrote:

Hi ,

I have and ldap server (openldap-2.4.23 ) in slapd.conf I set

concurrency  8192
threads 256

but when I do top -b  -H -d 2  -p 25560  I always see 8 threads ( show
9 slapd instances)  not more even if I instruct to create 256 threads.
  Is there anything that I miss?

Additionally what is the releation of threads and concurrency. Is
there any calculation method that I must obey?


concurrency generally does nothing on most platforms. I believe it might have 
had an effect on old Solaris systems with their lwp library. You didn't 
mention what OS you're using so it's impossible to tell you if this directive 
has any relevance for you. Most likely not.


The threads setting tells the maximum number of threads slapd will spawn. It 
creates them on demand, not all at once. If you want to see 256 threads then 
you have to submit enough requests to make the server busy enough to actually 
use them.


Keep in mind that each thread uses a thread stack of 4MB on 32 bit systems, 
8MB on 64 bit systems. So you're looking at 1-2GB of RAM dedicated to the 
threads with a setting like this. Make sure you don't configure more threads 
than you have RAM for...

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: merging local and remote attributes

2011-02-21 Thread Howard Chu

Hugo Monteiro wrote:

Hello list,

I have been trying to use translucent overlay to merge attributes
between a remote and a local server (both 2.4.23).

  From the slapo-translucent man page i read:

Attributes may be specified as both local and remote if desired.

and

In any case, both the local and remote entries corresponding to a
search result will be merged before being returned to the client.

The thing is that if i specify an attribute (objectclass) to be both
local and remote, i can only get/search for the local entries. Choosing
either separately will work as advertised though.

Again, i ask if this is a bug, a subtlety i have missed or this is not
supposed to work with objectClass attribute?


Re-read the manpage. 1st paragraph.

Entries  retrieved
   from  a  remote LDAP server may have some or all attributes overridden,
   or new attributes added, by entries in the local database before  being
   presented to the client.

This overlay does not merge attributes, it replaces them.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Poor performance on Solaris

2011-02-24 Thread Howard Chu

Quanah Gibson-Mount wrote:

--On Thursday, February 24, 2011 3:53 PM +0100
juergen.spren...@swisscom.com wrote:


Hi,

we had some performance issues on our ldap servers running Solaris 10
sparc.

I did some tests using slamd http://www.slamd.com/ and got disturbing
results:

ldap-service: OpenLDAP 2.4.23, setup identical on both boxes, threads=64,
identical content.

box1:
hardware: Sun Microsystems  sun4v SPARC Enterprise T5120
memory:32 GB RAM
os: Solaris 10 s10s_u9wos_14a
searches (avg/second): 1521


Slowaris is always tedious with OpenLDAP.  At a previous job, I was able to
replace 9 slowaris boxes with 4 linux boxes, and even then, just one of the
Linux boxes could handle the complete load it took the 9 slowaris boxes to
run.

I will note that if you are going to use slowaris, I highly advise you set
a memory key rather than using on disk cache for BDB if your DB is any size
over about 4 GB.  Other than that, you'll generally just have to deal with
the fact it will be significantly slower than Linux.


Actually, you should always use a shared memory key on Solaris. Using mmap'd 
files is just too slow on that OS.



Some time ago, I did benchmarks of Linux 2.6 vs Slowaris-x86 on identical
hardware, and it still was noticeably slower, although not as bad as when
using the sparc architecture.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Poor performance on Solaris

2011-02-24 Thread Howard Chu

Brett @Google wrote:

On Fri, Feb 25, 2011 at 10:25 AM, Howard Chu h...@symas.com
mailto:h...@symas.com wrote:


I will note that if you are going to use slowaris, I highly advise you 
set
a memory key rather than using on disk cache for BDB if your DB is any
size
over about 4 GB.  Other than that, you'll generally just have to deal 
with
the fact it will be significantly slower than Linux.



Actually, you should always use a shared memory key on Solaris. Using
mmap'd files is just too slow on that OS.


Incidentlly, What is the uniqueness of a shared memory key, is it unique
amoung all instances on a single server ?


The significance of a shared memory key depends entirely on your operating 
system. But by its nature, shared memory is global to a machine, so you can 
assume that all keys must be unique on a particular machine.



I have personally only ever needed a shared memory key on one of a group of
ldap servers.

But i noticed that the bdb logs are not written to disk when shared memory is
configured, is this a good or bad thing ?


The use of shared memory has no bearing on how the BDB library writes its 
transaction logs.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Problem unexpected failing slapd

2011-02-28 Thread Howard Chu

Ruud Baart wrote:

Sorry, I think I found a way to start slapd with gdb. The main mistake I
made was using uid openldap. It should be run as root.
  # gdb -q -x /root/gdb.init --args /usr/sbin/slapd -h ldap:///
ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf -l local5

gdb.init:
  handle all nostop
  run
  quit

This way I can modify the start-stop script. With a restart there is no
need for user intervention.

Now it runs and functions. I assume this is the way Howard Chu
suggests. Let's wait and see what happens.


No. I did not say to start slapd using gdb. I said to attach gdb to the 
running slapd, which means slapd should already be started, using whatever 
method you normally use to start it.


What you've done will accomplish nothing.


Op 28-2-2011 12:10, Ruud Baart schreef:


Op 28-2-2011 3:00, Howard Chu schreef:

Ruud Baart wrote:

Problem:
For a customer we use LDAP for many years. Last year suddenly the slapd
service just stopped without any traces in the logfiles. After a
restart
of slapd everything works fine again. But the problem was there: it was
not an incident, now and then slapd just stops and always without any
traces in the logfiles. Sometime three times a day, sometime a week
without a failure. I can't find a pattern or any relation to any other
service on the linux server.


Attach to the running slapd with gdb, type
 handle all nostop
 continue
and let it run. If there's a crash you'll see what happened in gdb.

I tried it but I don't get this working. I have no experience with
gdb. I assume I do something wrong.

Normally slapd daemon runs like this on a test machine
  /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -f
/etc/ldap/slapd.conf -l local5

If I try this running attached to gdb:
su openldap -s /bin/bash
gdb --args  /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u
openldap -f /etc/ldap/slapd.conf -l local5
GNU gdb (GDB) 7.0.1-debian

This GDB was configured as i486-linux-gnu.
..
Reading symbols from /usr/sbin/slapd...(no debugging symbols
found)...done.
(gdb) handle all nostop
SignalStop  Print   Pass to program Description
SIGHUPNoYes Yes Hangup
SIGQUIT   NoYes Yes Quit
...
EXC_EMULATION NoYes Yes Emulation instruction
EXC_SOFTWARE  NoYes Yes Software generated
exception
EXC_BREAKPOINT No   Yes Yes Breakpoint
(gdb)
(gdb) continue
The program is not being run.
(gdb) run
Starting program: /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u
openldap -f /etc/ldap/slapd.conf -l local5
[Thread debugging using libthread_db enabled]

Program exited with code 01.

At this point I have no idea what to do.






--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Problem unexpected failing slapd

2011-02-28 Thread Howard Chu

Ruud Baart wrote:

Thank you all for the fast answers.

I now attached gdb to slapd and get some results:

(gdb) handle all nostop
(gdb) continue
Continuing.

Program received signal SIGPIPE, Broken pipe.

Program received signal SIGPIPE, Broken pipe.
[New Thread 0xb01f6b70 (LWP 1548)]

Program received signal SIGPIPE, Broken pipe.
.. more of the same ..
Program received signal SIGPIPE, Broken pipe.

Program received signal SIGPIPE, Broken pipe.

---Typereturn  to continue, or qreturn  to quit---
Program received signal SIGPIPE, Broken pipe.

Program received signal SIGTERM, Terminated.


This is not a crash or any error in slapd; some external command was used to 
kill the slapd process.



[New Thread 0xaf4f3b70 (LWP 1968)]
[New Thread 0xaf0f2b70 (LWP 1969)]
[New Thread 0xaecf1b70 (LWP 1970)]
[New Thread 0xae8f0b70 (LWP 1971)]
[Thread 0xaf4f3b70 (LWP 1968) exited]
[Thread 0xb1bfcb70 (LWP 1474) exited]
[Thread 0xb5268b70 (LWP 1462) exited]
[Thread 0xaf0f2b70 (LWP 1969) exited]
[Thread 0xb4565b70 (LWP 1463) exited]
[Thread 0xb01f6b70 (LWP 1548) exited]
[Thread 0xb5669b70 (LWP 1461) exited]
[Thread 0xb17fbb70 (LWP 1475) exited]
[Thread 0xae8f0b70 (LWP 1971) exited]
[Thread 0xb28ffb70 (LWP 1464) exited]
[Thread 0xaecf1b70 (LWP 1970) exited]

Program exited normally.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: memberof/accesslog overlays together

2011-03-02 Thread Howard Chu

Michael Ströder wrote:

HI!

Unfortunately for privacy reasons I can't provide example data.

I'm using slapo-memberof, slapo-refint and slapo-access in exactly this order.

Now I'm analysing some strange things where modifications to group entries and
the subsequent modifications by slapo-memberof are not correctly written to
the accesslog DB.

Questions:

1. Is this overlay order a correct setup? Should this work? Or should
slapo-memberof be invoked after slapo-access?


Try changing the order and see for yourself.


2. CHANGES of release 2.4.24 lists a bunch of fixes to slapo-memberof. Any
changes which possibly affect writing to accesslog DB? (I'm not sure whether
we had problems like this with 2.4.23 or not though.)


Doesn't sound like it.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Allowing users to add, but not delete, entries?

2011-03-03 Thread Howard Chu

Gervase Markham wrote:

Hi,

Summary: is it possible to configure access control such that users to
can add, but not delete, entries?



This is difficult, because as far as I can see the write permission
does not distinguish between adding and deleting.

Can someone tell me if this is possible?


Sounds to me like you haven't read the slapd.access(5) manpage.

The answer is yes. Read the section on THE ACCESS FIELD in the manpage.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Allowing users to add, but not delete, entries?

2011-03-03 Thread Howard Chu

Gervase Markham wrote:

On 03/03/11 15:33, Howard Chu wrote:

Note that a Guide is not a reference manual; it is not intended to be
complete or exhaustive.


I'm not sure this use of terminology is universal; here is a
counter-example:
http://www.bugzilla.org/docs/tip/en/html/


It's certainly been the norm for Unix at least. E.g., compare the Guides vs 
Reference Manuals in the Solaris documentation.


http://download.oracle.com/docs/cd/E19253-01/index.html

Or the 4BSD system documentation:

http://www.freebsd.org/doc/en/books/faq/bibliography.html

Material in the 4BSD User's Supplementary Documents and Programmer's 
Supplementary Documents was a bit more friendly in prose, but only explained 
the approach to how to use the system. The details of each command or function 
are only found in the Reference Manuals.



It's only intended to provide an overview; the
detailed/exhaustive docs are always the manpages.


I would suggest that neither:
http://www.openldap.org/doc/
nor
http://www.openldap.org/doc/admin24/preface.html
nor the individual pages in the Admin Guide give the impression either
that the Admin Guide is not complete, or that the manual pages are an
essential read or the canonical reference.

Perhaps it would help OpenLDAP users if this were made a little more clear?


The preface.html which you linked above clearly states:


Scope of this Document

This document provides a guide for installing OpenLDAP Software 2.4 
(http://www.openldap.org/software/) on UNIX (and UNIX-like) systems. The 
document is aimed at experienced system administrators with basic 
understanding of LDAP-based directory services.


This document is meant to be used in conjunction with other OpenLDAP 
information resources provided with the software package and on the project's 
site (http://www.OpenLDAP.org/) on the World Wide Web. The site makes 
available a number of resources.


...


And then goes on to list various other resources. The Manual Pages, of course 
are the most obvious information resource provided directly with the software, 
and in Unix they are traditionally the authoritative reference. Given the 
heritage of this project it seems obvious to me that this is how the docs 
would be structured. If the phrase meant to be used in conjunction isn't a 
strong enough hint that the Guide is not a standalone document, please feel 
free to suggest an alternate or stronger wording. Note that the preface also 
invites you to do the same.



Thank you for your help and support :-)


Thank you for your feedback. We're always listening.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Replication monitoring

2011-03-04 Thread Howard Chu

Torsten Schlabach (Tascel eG) wrote:

Hi Andreas!


Your servers should be recognized as openldap.
* Check the Environment page if it says OpenLDAP.
* Click on one of the servers to open the server view page and you

should

now see two more buttons.


Yes, that works now. But I still didn't see any replication agreements.

So I digged a bit into the code. I first thought the problem may be that
my monitoring user cannot read cn=config, but it can.

I originally used version 2.0.4, but I also checked the current SVN
version; it still has the same (and possibly other) problems.

In your function openldap_setBackEndDatabases you are querying for
objectclass=olcBdbConfig. That means you will be missing several databases,
because you are only asking for BDB databases. In our example, we are using
HDB databases. So changing this to objectclass=olcHdbConfig made our
replication agreements visible all of a sudden.

But I guess this is not how to do it as there may be many other backend
types.


You probably should be searching for olcDatabaseConfig instead, which is the 
common superclass of all DB types.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: syncrepl missing entries in openldap 2.4.23

2011-03-04 Thread Howard Chu

james_whitea...@mcafee.com wrote:

So does OpenLDAP have another mechanism to do this? Combine two ldap providers 
into a single consumer that is?


I've already answered that. You need to read more carefully.



Jim


On Mar 4, 2011, at 4:31 PM, Howard Chu wrote:


james_whitea...@mcafee.com wrote:

I am trying to set up syncrepl to have multiple providers to a single

consumer. Basically allowing me to combine two ldap's into a single ldap. I
know this is probably not a standard configuration but seems like it should 
work.


This seems to work for a while but then all of the records from one of the

providers is deleted. And even though the consumer still is polling both
providers the records will will not get added back.

No, this setup will always fail in the manner you describe. The way a syncrepl
refresh works by default is that the provider tells the consumer about every
entry it knows about within the search context. The consumer then deletes
everything on its side that the provider didn't enumerate. Since both of your
consumers are using the identical search base, every time one of them
refreshes it will always delete everything the other one retrieved. (This is
the normal operation of a syncrepl refresh Present phase. Read RFC4533 for the
detailed explanation.)

It's possible to get this working, somewhat, using delta-syncrepl, which
usually does not use a Present phase. However, if the consumer ever lags
behind the provider's log (i.e., the consumer's state is older than the oldest
entry in the provider's log) then delta-syncrepl falls back to normal
syncrepl, and you'll hit the refresh Present phase again. So in general, what
you're trying to do is unsupported.



Here is my consumer syncrepl configuration. The providers are a standard 
provider configuration.

Any help would be appreciated.

Jim


serverID 064

databasebdb
suffix  o=dogcatfish
rootdn  cn=admin,o=dogcatfish

limits dn.exact=cn=admin,o=dogcatfish size=unlimited time=unlimited

# Cleartext passwords, especially for the rootdn, should
# be avoid.  See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
rootpw  admin

# syncrepl configuration
syncrepl rid=64
  provider=ldap://provider1
  type=refreshOnly
  interval=00:00:01:00
  retry=60 10 300 +
  searchbase=o=dogcatfish
  filter=(objectClass=*)
  scope=sub
  attrs=*,+
  schemachecking=off
  bindmethod=simple
  binddn=cn=admin,o=dogcatfish
  credentials=admin

# syncrepl configuration
syncrepl rid=68
  provider=ldap://provider2
  type=refreshOnly
  interval=00:00:01:00
  retry=60 10 300 +
  searchbase=o=dogcatfish
  filter=(objectClass=*)
  scope=sub
  attrs=*,+
  schemachecking=off
  bindmethod=simple
  binddn=cn=admin,o=dogcatfish
  credentials=admin

# Indices to maintain
index   contextCSN,entryCSN,entryUUID,objectClass,cn,dc,mail   eq
checkpoint 1024 5

mirrormode TRUE





--
   -- Howard Chu
   CTO, Symas Corp.   http://www.symas.com
   Director, Highland Sun http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/






--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: [Solved] Poor performance on Solaris

2011-03-07 Thread Howard Chu

juergen.spren...@swisscom.com wrote:

Hi,

first I wish to thank all those who supplied helpful hints to solve the 
problem, especially Quanah Gibson-Mount and Howard Chu.

My performance issue was solved by switching from memory mapped keys to shared 
memory keys for hdb as suggested by Quanah and Howard.
Putting 'shm_key 10' into slapd.conf and restart of slapd solved the problem.
Performance improvement was about factor 10 on the Solaris box.

As the problem was gone connection logging was switched off, which additionally 
doubled throughput as Solaris sylog can't do asynchronous logging.

During further tests slapd stopped responding when many concurrent connections 
were active.

This behaviour was caused by default settings in /etc/system.
Adding two lines to /etc/system and reboot solved the problem:
set rlim_fd_max = 16368
set rlim_fd_cur = 8192

Should be enough for about 8000 connections.

Performance before:
box1:
hardware: Sun Microsystems  sun4v SPARC Enterprise T5120
memory:32 GB RAM
os: Solaris 10 s10s_u9wos_14a
searches (avg/second): 1521

Performance after:
box1:
hardware: Sun Microsystems  sun4v SPARC Enterprise T5120
memory:32 GB RAM
os: Solaris 10 s10s_u9wos_14a
searches (avg/second): 37000


What is the CPU utilization that you see at this point? The T5120 supposedly 
has 64 hardware threads, but in my tests on this architecture, we never got 
better than 50% utilization (a couple years ago). I'm curious if you can make 
use of the multiple listener feature added in 2.4.24. I no longer have access 
to any machines of this architecture to test for myself. Also are you using 
the on-chip ethernet interface?


My next step will be tuning network parameters.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: LDAP browsers and cn=config

2011-03-07 Thread Howard Chu

Gervase Markham wrote:

On 07/03/11 17:49, Gervase Markham wrote:

oldRootDN: cn=admin,cn=config

^

And that would be the problem :-|

Thank you for your help.shuffles feet in an embarrassed fashion


cn=config is an LDAP database, it is not a collection of files for you to edit 
by hand. You are supposed to use ldapmodify on it, for reasons of this very 
nature. I.e., ldapmodify gets syntax-checked and stupid typos of this sort get 
caught.


If you had used ldapmodify -H ldapi:/// -Y EXTERNAL to add the desired 
attributes you wouldn't have these silly problems.


If your LDAP browsers don't support ldapi:/// that's their deficiency...
--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Another thought on backends

2011-03-07 Thread Howard Chu

Michael Smith wrote:


Anybody interested in collaborating on a back-python
analogous to back-perl? After a cursory glance at the
code for the latter, it looks like it wouldn't be hard to
adapt for the Python case.


Go ahead if you like. Though it may be simpler to just use back-sock.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: LDAP browsers and cn=config

2011-03-08 Thread Howard Chu

Gervase Markham wrote:

Can you tell me which LDAP browsers do support this scheme? After all,
the other part of my message was asking for advice on which was best.


Ironically, I used to use the Netscape browser as my preferred LDAP browser, 
many years ago. Don't recall when they axed that feature, it was quite handy, 
and getting the results pretty-printed in HTML was really nice.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: [Solved] Poor performance on Solaris

2011-03-09 Thread Howard Chu

juergen.spren...@swisscom.com wrote:

Hi Howard,

On-chip ethernet interfaces are currently not used for slapd,
They are currently used for management access and netbackup.
Zones running slapd use nxge type ethernet interfaces.

Did some quick tests. Increasing threads to 64 caused
higher cpu utilization but also a dramatic decrease
in performance on OpenLDAP releases 2.4.2[34].



That could not be compensated by increasing listener_threads.
Have tested values default, 4 and 8 for listener threads.


Thanks for taking the time to try this out. Yes, increasing the number of 
threads beyond a certain threshold typically causes a performance decrease due 
to inefficiencies in the system's scheduler. This is why we use a relatively 
low number of threads by default.


When I set up tests for maximum performance, I usually have to bind individual 
slapd threads to specific CPU cores to avoid this problem. (Look into the 
pbind or psrset commands on Solaris.)



Staying with 32 threads and setting listener_threads to 4
Increased searches/s from 37000 to 48000.
More listener_threads had no significant impact.


30% gain, not bad.


CPU utilization by slapd was at about 65%.


If you needed to, I believe you can coax a fair bit more out of this using 
pbind. You generally can't get slapd above 90% utilization because the network 
device driver will consume the rest. (Which is why I was curious about whether 
the on-chip ethernet is more efficient.) I.e., on an optimally tuned setup, 
the network will be the performance limiter, not slapd.


Juergen Sprenger

-Original Message-
From: Howard Chu [mailto:h...@symas.com]
Sent: Monday, March 07, 2011 10:25 PM
To: Sprenger Jürgen, ITS-SDL-SO-WXS-USO-BE1
Cc: openldap-technical@openldap.org
Subject: Re: [Solved] Poor performance on Solaris

juergen.spren...@swisscom.com wrote:

Hi,

first I wish to thank all those who supplied helpful hints to solve the 
problem, especially Quanah Gibson-Mount and Howard Chu.

My performance issue was solved by switching from memory mapped keys to shared 
memory keys for hdb as suggested by Quanah and Howard.
Putting 'shm_key 10' into slapd.conf and restart of slapd solved the problem.
Performance improvement was about factor 10 on the Solaris box.

As the problem was gone connection logging was switched off, which additionally 
doubled throughput as Solaris sylog can't do asynchronous logging.

During further tests slapd stopped responding when many concurrent connections 
were active.

This behaviour was caused by default settings in /etc/system.
Adding two lines to /etc/system and reboot solved the problem:
set rlim_fd_max = 16368
set rlim_fd_cur = 8192

Should be enough for about 8000 connections.

Performance before:
box1:
hardware: Sun Microsystems  sun4v SPARC Enterprise T5120
memory:32 GB RAM
os: Solaris 10 s10s_u9wos_14a
searches (avg/second): 1521

Performance after:
box1:
hardware: Sun Microsystems  sun4v SPARC Enterprise T5120
memory:32 GB RAM
os: Solaris 10 s10s_u9wos_14a
searches (avg/second): 37000


What is the CPU utilization that you see at this point? The T5120 supposedly
has 64 hardware threads, but in my tests on this architecture, we never got
better than 50% utilization (a couple years ago). I'm curious if you can make
use of the multiple listener feature added in 2.4.24. I no longer have access
to any machines of this architecture to test for myself. Also are you using
the on-chip ethernet interface?


My next step will be tuning network parameters.





--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: syncrepl with with multiple subordinate databases

2011-03-09 Thread Howard Chu

Dieter Kluenter wrote:

Hi,
I am facing a severe problem with the replication of subordinate
databases and a log database. The initial replication of the
subordinate databases is successful but afterwards slapd crashes, A
backtrace and a momory map can be found here


A crash should be submitted as a bug report to the ITS, not to a discussion 
mailing list.



http://pastebin.de/15919


Your trace says there was an invalid free(); running under valgrind would 
probably give more information.



the provider slapd.conf can be found here
http://pastebin.de/15920


You should not be using syncprov-reloadhint on anything except for the actual 
cn=log database.



the consumer slapd.conf can be found here
http://pastebin.de/15922


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Null Search Base

2011-03-10 Thread Howard Chu

Andrew Findlay wrote:

On Wed, Mar 09, 2011 at 04:34:16PM -0700, l...@mm.st wrote:


A security scanner was run against our ldap severs and came back with a
warning stating The remote LDAP server supports search requests with a
null, or empty, base object.  This allows information to be retrieved
without any prior knowledge of the directory structure.  Coupled with a
NULL BIND, an anonymous user may be able to query your LDAP server using
a tool . . .



I assume the warning is due to the namingContext attribute and
if desired an acl could be setup to stop the retrival on the
information.


That seems very likely, and as you say an ACL could be used to prevent
it. In this context the 'empty base object' refers to the Root DSE, and
it contains information that some LDAP client programs depend on.
Blocking access to it would almost certainly cause trouble for those
clients.

It is very unlikely that the list of naming contexts and supported LDAP
extensions is in any sense secret, so don't let some auditor bully you
into breaking your system just to fit some tick-box notion of security.
The important stuff comes further down in the DIT, and you need a tool
specific to your organisational policy to point out exposures there.


When the tool doesn't even call the object by its proper name (Root DSE) 
it's a sure sign the tool authors have no idea what they're talking about.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: ldapsearch and extended server controls?

2011-03-11 Thread Howard Chu

Brian Reichert wrote:

I have a case where I'm trying to replicate some other application's
search against Active Directory.

A deconstruction of a packet capture of that app's conversation
shows it's making use of a control that I don't know how to specify
using any combination of ldapsearch(1) or ldap.conf(5).

Specifically, I see this coming over the wire as a control on the
search:

   http://msdn.microsoft.com/en-us/library/aa366989%28v=vs.85%29.aspx

   1.2.840.113556.1.4.417 LDAP_SERVER_SHOW_DELETED_OID

Research shows there lots these sorts of extended control; yay, I
learned something new!

Is there a way of utilizing these sorts of controls via ldeapsearch?


Yes. Use -e 1.2.840.113556.1.4.417


Thanks for any advice you may have...



--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Dynamic modules unloading

2011-03-13 Thread Howard Chu

Коновалов Андрей Александрович wrote:

Hello!
I have OpenLDAP 2.4.24 built with -enable-dynamic and --enable-modules options

So, dynamic modules support is on and my question is:
Why can i load module

(for example:

dn: cn=module{0},cn=config
changeType: modify
add: olcModuleLoad
olcModuleLoad: translucent.la
)

But when i'm trying to unload it (because it's not used at all), i cant do this 
anywhere?

dn: cn=module{0},cn=config
changeType: modify
delete: olcModuleLoad
olcModuleLoad: {6}translucent.la
# the same with simply olcModuleLoad: translucent.la
modifying entry cn=module{0},cn=config
ldap_modify: Other (e.g., implementation specific) error (80)
additional info: cannot delete olcModuleLoad

If it is not realised yet, you may consume my question as a feature request :)


Many reasons. First because there simply hasn't been a need. Second because 
many platforms that OpenLDAP supports don't actually support unloading of 
dynamic modules. (E.g., on AIX 3, unloading a module is supported but 
doesn't in fact free up any of the memory, that part of the address space is 
gone. Unloading and reloading over and over eventually eats up all the address 
space.)


There are some isolated cases where it might be useful, but they're relatively 
rare. E.g., unloading a module so that a one-line patch can be applied, and 
then reloading it. In the more common case, since the slapd APIs are not 
frozen, you can't rely on unloading and mixing versions of modules.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: delta-sync - ContextCSN on proivder older than consumers

2011-03-14 Thread Howard Chu

Dieter Kluenter wrote:

Am Sun, 13 Mar 2011 17:39:17 -0700
schrieb Yuri Bankyurib...@gmail.com:


After doing more testing I have noticed that it is the 'Group member
modify entryCSNs' that seem to get ignored by the Provider, but
picked up by the Consumers. All other changes, adding or removing
users seems to update the ContextCSN on the Provider correctly.

So a work around would be to make some kind of random change to an
entry in my DIT ( after making changes to group membership), so that
the Provider has the correct ContextCSN. A simple change like
modifying the description field for a user would accomplish this. I
would like to get to the bottom of this though, without such a work
around.

Could this have anything to do with the memberOf overlay, which I am
using?


It is more likely that the contextCSN of accesslog db is older than
the last contextCSN provided by the provider.


It's too unclear to make such an assumption.

Best bet still is to switch to 2.4.24 and see if the problem remains.

And yes, it's most likely related to the memberOf overlay.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: delta-sync - ContextCSN on proivder older than consumers

2011-03-14 Thread Howard Chu

Dieter Kluenter wrote:

Am Mon, 14 Mar 2011 02:43:53 -0700
schrieb Howard Chuh...@symas.com:


Dieter Kluenter wrote:

Am Sun, 13 Mar 2011 17:39:17 -0700
schrieb Yuri Bankyurib...@gmail.com:


After doing more testing I have noticed that it is the 'Group
member modify entryCSNs' that seem to get ignored by the Provider,
but picked up by the Consumers. All other changes, adding or
removing users seems to update the ContextCSN on the Provider
correctly.

So a work around would be to make some kind of random change to an
entry in my DIT ( after making changes to group membership), so
that the Provider has the correct ContextCSN. A simple change like
modifying the description field for a user would accomplish this. I
would like to get to the bottom of this though, without such a work
around.

Could this have anything to do with the memberOf overlay, which I
am using?


It is more likely that the contextCSN of accesslog db is older than
the last contextCSN provided by the provider.


It's too unclear to make such an assumption.


It's based on experience, my test environment shows the same results. A
modify to one of the subordinate databases results in error 53 trying
to sync the other subordinate database. Comparing contextCSN of all
four databases showed that the client presents a CSN that is younger
than that of access db, but I am still investigating


The original poster has no subordinate databases. Whatever behavior you're 
experiencing is totally unrelated, and you're making unfounded assumptions 
that there is any relevance between your situation and his.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Postgres perhaps - was Re: Another thought on backends

2011-03-14 Thread Howard Chu

Richard Troy wrote:


On Mon, 14 Mar 2011, Hallvard B Furuseth wrote:


Michael Smith writes:

Anybody interested in collaborating on a back-python
analogous to back-perl? After a cursory glance at the
code for the latter, it looks like it wouldn't be hard to
adapt for the Python case.


I started on that once, but quit because Python and slapd both
want their conflicting autoconf-generated files #included:
OpenLDAP's portable.h vs. Python's pyconfig.h.
back-perl escapes that: Perl uses metaconfig instead of autoconf.



I have no interest with what you are proposing but I am very interested in
porting OpenLDAP to Postgres. The compelling use case is that now, with
the advent of binary replication in Postgres 9, one can replicate your
Posgres installation - which you may be doing for any number of other
reasons - and have that replication carry with it the OpenLDAP dataset.
So, it reduces the systems management / maintenance overhead while
increasing security (one DB system instead of two). Also, the Postgres
replication is _very_ efficient and has no inherent limit on the number of
replicants, each of which can be used for query-only, if desired.

It's my understanding that someone has already done this - ported OpenLDAP
to Postgres, and there's a web page describing it out there somewhere. I
have been looking for some free time (yeah, like _that's_ going to happen
any time soon!) to do the port. And since I haven't quite gotten around to
it, I'm wondering if there's general interest? Anyone?


Are you saying there's a low level Postgres-specific API that can be 
leveraged? If you're just going to use SQL or something euqivalent, I don't 
see any advantage to that.


Regards,
Richard





--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: How to turn on LDAP_DEVEL flag?

2011-03-17 Thread Howard Chu

Коновалов Андрей Александрович wrote:

I want to test component matching support in OpenLDAP 2.4.24, but it is
masked by LDAP_DEVEL flag, which is turned on only when
LDAP_VENDOR_VERSION=0. So... could you tell me please, how to turn on
LDAP_DEVEL directly in a proper way?


It is masked for a reason - it is work in progress and not suitable for use. 
Since you are asking this question, you are apparently not a qualified 
developer, therefore you should not touch it. It won't work.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Undefined reference to ber_* with 2.4.24

2011-03-17 Thread Howard Chu

frank.offerma...@caseris.de wrote:

Hello,

I am able to compile OpenLDAP 2.4.23 with Berkeley DB 4.8.30 with MSYS and
windows with configure params:
configure --prefix=/mingw --enable-acceslog --with-tls

But if I try to compile OpenLDAP 2.4.24 I get several undefined references
to ber_* functions when liblutil is build.
Did I miss a change in the way to build OpenLDAP 2.4.24?


Seems to be a bug in the Makefile introduced with the new libldif. Please file 
an ITS for this.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: OpenLDAP reports busy and hangs

2011-03-17 Thread Howard Chu

Frank Budde wrote:

Hi,
with OpenLDAP 2.4.22 on Solaris 10 we discovered following situation:
from time to time the slapd log file contains message
daemon: select: listen=7 busy
while ldap continues to operate normally.


Sounds similar to this
http://www.openldap.org/its/index.cgi/Incoming?id=6833


After 7 days of operation suddenly ldap does not respond any more and keeps printing 
daemon: select: listen=7 busy.
Log file shows:

daemon: select: listen=7 busy
daemon: listen=7, new connection on 30
daemon: activity on 1 descriptor
daemon: waked
daemon: added 30r (active) listener=0
conn=5656 fd=30 ACCEPT from IP=192.168.200.131:40993 (IP=192.168.200.16:9929)
daemon: select: listen=7 active_threads=1 tvp=zero
daemon: activity on 2 descriptors
daemon: waked
daemon: select: listen=7 active_threads=1 tvp=zero
daemon: activity on 1 descriptor
daemon: activity on: 30r
daemon: read activity on 30
daemon: select: listen=7 active_threads=1 tvp=zero
daemon: activity on 1 descriptor
daemon: waked
conn=5656 op=0 BIND dn=cn=ldapadmin,dc=abc,dc=com method=128
daemon: select: listen=7 active_threads=1 tvp=zero
conn=5656 op=0 BIND dn=cn=ldapadmin,dc=abc,dc=com mech=SIMPLE ssf=0
conn=5656 op=0 RESULT tag=97 err=0 text=
daemon: activity on 1 descriptor
daemon: activity on: 30r
daemon: read activity on 30
daemon: select: listen=7 active_threads=1 tvp=zero
daemon: activity on 1 descriptor
daemon: waked
daemon: select: listen=7 active_threads=1 tvp=zero
conn=5656 op=1 SRCH base=ou=users,dc=abc,dc=com scope=1 deref=3 
filter=(uid=abcservice-prd)
daemon: activity on 1 descriptor
daemon: select: listen=7 busy
daemon: select: listen=7 busy
daemon: select: listen=7 busy
daemon: select: listen=7 busy

slapd.conf contains
gentlehup on
idletimeout 15
conn_max_pending 10
conn_max_pending_auth 30

My question: How can the problem be solved or reproduced?
The problem comes up in production environment and I need to reproduce it in 
test environment. How can this be done?
(If the problem can be reproduced with 2.4.22 I will try it with ldap 2.4.24.)

Thanks for any help,
/Frank



--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: How to turn on LDAP_DEVEL flag?

2011-03-17 Thread Howard Chu

Dieter Kluenter wrote:

Am Tue, 15 Mar 2011 15:22:55 +0300
schrieb Коновалов Андрей Александровичmudr...@list.ru:


I want to test component matching support in OpenLDAP 2.4.24, but it
is masked by LDAP_DEVEL flag, which is turned on only when
LDAP_VENDOR_VERSION=0. So... could you tell me please, how to turn on
LDAP_DEVEL directly in a proper way?


When compiling openldap use CFLAGS=-DLDAP_DEVEL -DLDAP_COMP_MATCH


No. This code is guaranteed to crash. It is irresponsible of you to give this 
advice.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: OpenLDAP Memory Usage

2011-03-17 Thread Howard Chu

Diego Lima wrote:

Hello all,

I'm experiencing some problems with some OpenLDAP servers: the slapd
process seems to always use more memory, eventually reaching a point
where it has consumed all the available server memory and is killed by
the OOM killer. The servers have 32gb of memory plus 32gb of swap
space and are running Debian Lenny (with kernel 2.6.26-2-amd64), and
we have compiled OpenLDAP 2.4.23 from source, and we're using Berkeley
DB 4.6. The servers are dedicated to running OpenLDAP, so they don't
have other processes that use a significant amount of memory.

This is a relatively high-volume environment, with 4 servers running
with mirrormode to enable multi-master replication. The current
database size is about 900mb. Is there any setting that could limit
this memory usage? I don't feel this is normal considering our DB
size and cache sizes.


Very likely ITS#6660, you need to update to 2.4.24.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: DSEE to OpenLDAP

2011-03-21 Thread Howard Chu

Bill MacAllister wrote:

I'm not sure that replication between the two is an option (although would
love it if it was :), so I have looked into exporting the current DSEE
environment to a LDIF, and attempted to then import it into openldap
(using slapadd), but ran into a few issues...

The issue I'm currently stuck on is getting the data in the LDIF into a
format that can be imported using slapadd. Currently, I have issues with
automounts, pwdReset attribs etc etc...


Why can't you just use ldapsearch on the existing directory and use that
as input to slapadd?


slapadd is meant for well-formed LDIF input, mainly from slapcat. DSEE is 
known to not support schema checking of any kind, therefore any LDIF you 
obtain from it will most likely be full of garbage and not well-formed. As 
such, you should only use ldapadd to import it, so you get full error checking 
on the import.



Of course, that only gets you the data.  I would bet that the real issue
will be getting the access controls correct.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: OpenLDAP Memory Usage

2011-03-22 Thread Howard Chu

Tim Mooney wrote:

In regard to: Re: OpenLDAP Memory Usage, Quanah Gibson-Mount said (at...:

I suggest reading over:

http://www.openldap.org/pub/hyc/scale2007.pdf


Thanks Quanah, that was very helpful.  Slides 23 and 33-35 (comparing
the various memory allocators) were very useful, especially when coupled
with the openldap-devel threads starting on August 24 2006.

I also found slide 26, about the problem with trying to benchmark OpenLDAP
using slow bench mark tools, interesting.  What tools are recommended
these days, for benchmarking an OpenLDAP installation?  The most promising
ones I could find were SLAMD (which was panned in slide 26) and cbe4.
Are there others that are noteworthy?


SLAMD is still the most convenient to use, in terms of overall features. I 
just make sure that I have several times as much horsepower in the clients as 
in the target server, to generate sufficient load on the server. (This can 
make benching very powerful servers rather difficult. Sometimes I will also 
resort to simple timing of the OpenLDAP test clients when the client machines 
are otherwise too underpowered for SLAMD. I try to avoid that because then I'm 
stuck having to tabulate the results manually.)


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: syncrepl and mass deletes

2011-03-23 Thread Howard Chu

LALOT Dominique wrote:

Hello,

I am testing the replication feature in a multimaster environment replicating
into a single database. As stated before, I added serverid to my providers. I
just have two providers for test purpose.
I tested mass updates on a provider, stopped my replica during updates, then
start again and it's OK, it updates the entries
If I do the same for mass deletes. I deleted 4 entries while stopping the
consumer. My consumer is still with 3 undeleted entries. I left the
consumer for hours, restarting it twice.
It seems there is no regular compare between consumer or provider in such
situation. I'll simplify to test in a single provider setup, to see if it works.

All servers are 2.4.23


Please try your test with 2.4.24 instead.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: syncrepl and mass deletes

2011-03-23 Thread Howard Chu

LALOT Dominique wrote:

Howard,

I was obliged to remove slapd package on consumer. Then compile in 2.4.24 and
restart. doing the same tests, there was nothing diffferent. provider is still
2.4.23
My test: delete 3 entries, stop consumer when deleting; start again
consumer when it's finished. Consumer is then out of sync.


You haven't posted your provider config. It appears you're using 
syncprov-sessionlog. Obviously both ends of this system are vital to solving 
the puzzle.



Mar 23 17:27:50 ldaprelay slapd[4358]: slapd stopped.
Mar 23 17:34:45 ldaprelay slapd[4577]: @(#) $OpenLDAP: slapd 2.4.24 (Mar 23
2011 16:52:04)
$#012#011r...@ldaprelay.univ-amu.fr:/usr/local/src/openldap-2.4.24/servers/slapd
Mar 23 17:34:45 ldaprelay slapd[4579]: slapd starting
Mar 23 17:34:45 ldaprelay slapd[4579]: do_syncrep2: rid=040
LDAP_RES_INTERMEDIATE - REFRESH_DELETE
Mar 23 17:34:45 ldaprelay slapd[4579]: do_syncrep2: rid=020
LDAP_RES_INTERMEDIATE - SYNC_ID_SET
Mar 23 17:34:48 ldaprelay slapd[4579]: last message repeated 175 times
Mar 23 17:34:48 ldaprelay slapd[4579]: do_syncrep2: rid=020
LDAP_RES_INTERMEDIATE - REFRESH_PRESENT
Mar 23 17:34:48 ldaprelay slapd[4579]: do_syncrep2: rid=020
cookie=rid=020,sid=020,csn=20110323163416.105518Z#00#020#00
Mar 23 17:34:48 ldaprelay slapd[4579]: slap_queue_csn: queing 0x1a35560
20110323163416.105518Z#00#020#00
Mar 23 17:34:48 ldaprelay slapd[4579]: slap_graduate_commit_csn: removing
0x1a35620 20110323163416.105518Z#00#020#00

I changed the provider to 2.4.24 that makes deletes. Hopefully this one was
built on tar.gz
That makes no difference, after restarting the consumer, it does not delete
extra entries

Dom

2011/3/23 LALOT Dominique dom.la...@gmail.com mailto:dom.la...@gmail.com

Hi Howard,

We were told to migrate to 2.4.23 sometimes ago, and we did some work to
update our production servers. Can I try 2.4.24 only on the consumer side?
It would be a pain to migrate all servers to 2.4.24 without package.

is this related to the last fixes?

Fixed slapd syncrepl reuse of presence list (ITS#6707)
Fixed slapd syncrepl uninitialized return code (ITS#6719)
Fixed slapd syncrepl variable initialization (ITS#6739)


Fixed slapd syncrepl refresh to use complete cookie (ITS#6807)

Thanks

Dom


2011/3/23 Howard Chu h...@symas.com mailto:h...@symas.com

LALOT Dominique wrote:

Hello,

I am testing the replication feature in a multimaster environment
replicating
into a single database. As stated before, I added serverid to my
providers. I
just have two providers for test purpose.
I tested mass updates on a provider, stopped my replica during
updates, then
start again and it's OK, it updates the entries
If I do the same for mass deletes. I deleted 4 entries while
stopping the
consumer. My consumer is still with 3 undeleted entries. I
left the
consumer for hours, restarting it twice.
It seems there is no regular compare between consumer or provider
in such
situation. I'll simplify to test in a single provider setup, to
see if it works.

All servers are 2.4.23


Please try your test with 2.4.24 instead.



--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: 8 principal limitation in openldap

2011-03-25 Thread Howard Chu

Srivatsav M wrote:

Hi,

We are using OpenLDAP for authenticating users registered in a LDAP server
(Open LDAP, Active Directory). After adding 8 principals (/etc/ldap.conf),
none of the users registered in the /etc/ldap.conf file are able to login.

nss_base_passwd
OU=engg,DC=mycompany,DC=region,DC=someplace,DC=myarea,DC=compname,DC=parentcompname
nss_base_shadow
OU=engg,DC=mycompany,DC=region,DC=someplace,DC=myarea,DC=compname,DC=parentcompname
nss_base_group
OU=engg,DC=mycompany,DC=region,DC=someplace,DC=myarea,DC=compname,DC=parentcompname


Can you please share the reason for this 7 limitation in the open ldap
library. or how I can fix this issue. I am looking i for the header file in
the source files whhich has this constant or limitation defined.


I think you mean RDN not principal. And there is no such limitation in any 
OpenLDAP code. You're talking about nss-ldap; you should ask your question on 
their mailing list.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/


Re: Large dynamic groups and performance

2011-04-11 Thread Howard Chu

Quanah Gibson-Mount wrote:

--On Monday, April 11, 2011 1:55 PM -0400 Alafrunn...@gmail.com  wrote:



I added those as well, no noticeable change in performance:

cachesize 50
idlcachesize 50
checkpoint 1 15


In a heavy write environment, I've found that smaller more frequent
checkpoints are better.  Yours is set rather high.


Also, you are definitely not using dynamic groups in the OpenLDAP
sense of the word, although they would probably perform significantly
better for you.


Yes, I understand that - thats why I put it in quotes.  I looked into
using OpenLDAP dynamic lists, but I think I'm limited by the fact that
some of our systems requiring these groups need to do searchs off of
it based on the dynamic membership (and from what I can tell, its not
possible to use it that way), ie they need to search for
(uniquemember=cn=,cn=users,...) on my group section of the tree.

I'll openly admit some of the values I have been picking for caching
and checkpoint are somewhat arbitrary.  I've been trying many
different values and have yet to settle on any that work well.  I'll
gladly try any recommendations.


Make sure you read over the dynlist overlay, I think you can do what you
want.  A group with 50,000+ members is probably going to take a long time
to update.  So OpenLDAP dynamic groups may well do better for you.


The dynlist overlay does not support searches of the form (member=foo). While 
updates of such large groups will involve a lot of I/O (and thus will always 
be slow) you can still improve things a little using the sortvals config option.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/



Re: clarifications on cachesize, preferred db, et. al. from admin guide

2011-04-12 Thread Howard Chu

Quanah Gibson-Mount wrote:

--On Tuesday, April 12, 2011 10:42 AM -0500 Tim Mooney
tim.moo...@ndsu.edu  wrote:



Check against OpenLDAP 2.4.25.  I believe nearly all backends and
overlays  support slapd-config now.


I'm using 2.4.25, but short of trying every one of the official and
contrib overlays one by one, I don't know of any way to find out which
ones don't support slapd-config.  That's why I was asking: I was hoping
someone knew what the holdouts were, or knew where it was documented.


I meant more, use the source.  From what I can tell, overlays with no
cn=config support are:

deref.c
seqmod.c

They also don't have manual pages, so probably aren't exactly supported
anyway.


They have no config settings at all, therefore they work regardless of static 
or dynamic config. At this point it's more useful to ask which don't work 
with dynamic config than which support dynamic config. There are several 
overlays that need no configuration.



Between that and the -devel note about backends, I think you can create a
complete list.


- Admin Guide, chapter 21.  The tuning chapter doesn't mention the
potential benefits of using sysv shared memory vs. mmap'ed files on
some platforms.  Should it?  Same offer for documentation patch applies,
though I expect this one will need more feedback from the experts.


Do you mean the shared memory keys, or something else?


The shared memory keys.


Definitely worth noting.  It is a requirement for Solaris if you want any
sort of decent performance.


Apparently it's become more important on recent Linux kernels too.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/



Re: cn=config replication to consumer / slave servers

2011-04-15 Thread Howard Chu

Quanah Gibson-Mount wrote:

--On Thursday, April 14, 2011 5:28 PM -0700 Christopher Strider Cook
cc...@pandora.com  wrote:


Alternately, I tried to setup a separate database cn=config_slave and
have that snycrepl to the slave into cn=config... but that creates a
naming missmatch.

Is there an approved practice to achieve this, or some other pointers on
avenues to explore?


Use slapo-rwm to rewrite the cn=config_slave database to be cn=config on
the replicas.


No, that never worked well. Use suffixmassage in the syncrepl config 
statement. (Added in 2.4.24)


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/



Re: cn=config replication to consumer / slave servers

2011-04-15 Thread Howard Chu

Christopher Strider Cook wrote:


On 4/15/11 11:46 AM, Howard Chu wrote:

Quanah Gibson-Mount wrote:

--On Thursday, April 14, 2011 5:28 PM -0700 Christopher Strider Cook
cc...@pandora.com   wrote:


Alternately, I tried to setup a separate database cn=config_slave and
have that snycrepl to the slave into cn=config... but that creates a
naming missmatch.

Is there an approved practice to achieve this, or some other
pointers on
avenues to explore?


Use slapo-rwm to rewrite the cn=config_slave database to be cn=config on
the replicas.


No, that never worked well. Use suffixmassage in the syncrepl config
statement. (Added in 2.4.24)


Yes that looks like the right path.. I've just recompiled to gain this
superpower... But:

So I've got a 'stub' cn=config setup with enough (I think) to get it
booted and connecting to the master to syncrepl over the rest of the config.
olcDatabase\=\{0\}config.ldif  has I think a proper syncrepl line:
olcSyncrepl: {0}rid=001 provider=ldaps://hostxxx.com bindmethod=simp
   le binddn=cn=admin,cn=config_slave credentials=pass searchbase=
   cn=config_slave suffixmassage=cn=config schemachecking=on
type=refreshAndPersist retry=60 +


Please read the test059 script in the test suite, it already does all the 
necessary steps in the correct order.


Something is still not quite right. In the logs I see a few troubling
messages and the local cn=config doesn't update.


--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/



Re: cn=config replication to consumer / slave servers

2011-04-19 Thread Howard Chu

Christopher Strider Cook wrote:

So, the pointer to test059 was exactly what this issue needed and
following it has lead me to an very good working setup with one puzzling
final step.



The problem I now face is that the initial cn=config entries used to do
the first sync do not get overwritten by the data from the master. So
the install password doesn't get replaced nor do the updated retry
timeouts for olcSyncRepl, because, I'm assuming, the 'stub' entries have
newer timestamps than those on the master.

How can this be overcome from the perspective of the slave server.
Updating the entries on the master triggers the update as you would
expect. Is there a way to put the stub entries onto the slave with a
timestamp in the past so that they get overwritten during the first
sync? Or is there another way to trigger them to be updated?


Use slapd -c. Read the slapd(8) manpage.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/



<    1   2   3   4   5   6   7   8   9   10   >