Re: Openldap2.4.16 performance issue
William E Jojo w.j...@hvcc.edu writes: [...] Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -Yes, OS installed on VM -- what type ---Don’t know There is nothing suspicious, except for the virtual machine. Your really should carefully check layout and configuration of this VM, do not use virtual disks. Hi Dieter, Could you kindly explain what this means? I've been all over the inter-webs and I'm not finding anything concrete about OpenLDAP and VM's or Databases and VM's in general. The closest I came was about some database latency studies and some VMware propaganda. My comments are based on my own experience. I am not against virutal machines in principle, but only against full virtualisation, that is virtualised disks and cpus, in a production environment. A para virtualisation might be acceptable, if the hypervisor provides sufficient resources and exclusive read/write access to a disk and client connections are moderate. But in all cases you have to expect performance loss. We are about to launch a master and two replicas (utilizing delta-syncrepl) running in Ubuntu 10.04 on a VMware VSphere ESXi 4.0 cluster with four IBM x3650-M2's (2 Quad Nahalem and 64GB memory each) with virtual disks carved from NFS mounts to all of the VSphere servers in the cluster to facilitate HA and FT - by the VMware book. (BTW, I took one of Howard's old posts to heart and we are following the Ubuntu playbook and we have purchased Canonical 24x7 support/maintenance. :-) ) We have had no problems with this environment in development and get better results than on bare metal with or without RAID. I know that it is recommended that the logs live on another disk from the database and RAID is frowned upon, but I have difficulty with a few points: 1) Separate, unprotected disks seems illogical. The last log and the BDB files are necessary to start BDB in slapd, correct? So, if you lose either disk, you're in trouble. Backups are ok, but daily seems too long a time. Seeing as we process new user accounts every 15 minutes, this would not be ideal for us. Yes, slapd requires the last transaction logs in order to recover a corrupted database, but the risks can be minimized by setting appropriate checkpoints. Writing new entries every 15 minutes wouldn't cause much trouble if you checkpoint every 4K and every 3 minutes. 2) RAID-1 I can understand having an issue on writes. But what about LUNs in FC from a NetApp 3140? Virtual disks on NFS in VMware? Both of these are in the best practices of both VMware and NetApp documentation. Just a scenario based on my own experience: a organisation wide addressbook with some 100,000 user entries, each user may have arbitrary private addressbooks underneath its entry node. Mail clients use this addressbook for address completion, mail clients are badly configured to use the root entry as search base. Can you imagine the number of parallel read operations? In addition to this read operations consecutive write operations of adding new entries, writing to index databases writing transaction logs. If you do not allow the transaction logs to reside on a different disk, you sure will experience db deadlocks in search opeerations. 3) 13-disk 7200RPM SATA RAID-DP (NetApp) is far faster than a single disk, dual diak or RAID-1, so why wouldn't you use SAN/NAS storage? I seriously want to understand the VM concern as it pertains to OpenLDAP. I think more and more people are doing this very thing and will benefit from this discussion. The main concern is performance. If you only expect a few connections per second, performance is not a questions, but if you expect a few thousend connections per second, performance is an issue. [...] -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
Re: Openldap2.4.16 performance issue
- Original Message - From: Dieter Kluenter die...@dkluenter.de To: openldap-technical@openldap.org Sent: Saturday, August 21, 2010 3:07:04 AM Subject: Re: Openldap2.4.16 performance issue Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -Yes, OS installed on VM -- what type ---Don’t know There is nothing suspicious, except for the virtual machine. Your really should carefully check layout and configuration of this VM, do not use virtual disks. Hi Dieter, Could you kindly explain what this means? I've been all over the inter-webs and I'm not finding anything concrete about OpenLDAP and VM's or Databases and VM's in general. The closest I came was about some database latency studies and some VMware propaganda. We are about to launch a master and two replicas (utilizing delta-syncrepl) running in Ubuntu 10.04 on a VMware VSphere ESXi 4.0 cluster with four IBM x3650-M2's (2 Quad Nahalem and 64GB memory each) with virtual disks carved from NFS mounts to all of the VSphere servers in the cluster to facilitate HA and FT - by the VMware book. (BTW, I took one of Howard's old posts to heart and we are following the Ubuntu playbook and we have purchased Canonical 24x7 support/maintenance. :-) ) We have had no problems with this environment in development and get better results than on bare metal with or without RAID. I know that it is recommended that the logs live on another disk from the database and RAID is frowned upon, but I have difficulty with a few points: 1) Separate, unprotected disks seems illogical. The last log and the BDB files are necessary to start BDB in slapd, correct? So, if you lose either disk, you're in trouble. Backups are ok, but daily seems too long a time. Seeing as we process new user accounts every 15 minutes, this would not be ideal for us. 2) RAID-1 I can understand having an issue on writes. But what about LUNs in FC from a NetApp 3140? Virtual disks on NFS in VMware? Both of these are in the best practices of both VMware and NetApp documentation. 3) 13-disk 7200RPM SATA RAID-DP (NetApp) is far faster than a single disk, dual diak or RAID-1, so why wouldn't you use SAN/NAS storage? I seriously want to understand the VM concern as it pertains to OpenLDAP. I think more and more people are doing this very thing and will benefit from this discussion. Our database is 86K+ DN's averaging about 40 attributes each. We've tuned the HDB cache to 768MB in a shared memory segment and the pertinent master slapd.conf file shows: shm_key 100 cachesize 20 idlcachesize60 dncachesize 40 checkpoint 1024 15 . . . # main database . . . index objectClass eq index cn eq,sub index sn eq,sub index gn eq,sub index maileq,sub index uid eq,sub index displayname sub,eq index memberUid eq,sub index uidNumber eq index gidNumber eq index sambaSIDeq index sambaSIDlisteq index sambaDomainName eq index sambaPrimaryGroupSIDeq index sambaGroupType eq index entryCSN eq index entryUUID eq index default sub,eq Replicas have identical indexes and shared memory usage. Basically, just running database population tests with full checking turned on, I get the following results: Ubuntu 10.04.1 on all with OpenLDAP 2.4.21/BDB 4.7.25 (all generate 200-10MB log files): IBM x3550 2-quad 5450 16GB, RAID-1 15K 73GB 3.0Gb/s SAS, 86K DN's - 30 minutes IBM x3550 2-quad 5450 16GB, Single 15K 73GB 3.0Gb/s SAS, 86K DN's - 28 minutes IBM x3550 2-quad 5450 16GB, 2-single 15K 73GB (db and logs on separate disks) 3.0Gb/s SAS, 86K DN's - 28 minutes VMware guest 2-vCPU, 3GB memory, 100GB virtual disk on VMware NFS mount of 13-1TB 7200RPM SATA disks in NetApp 3140 - 4 minutes. We can replicate to another VM in 9 minutes and two VMs simultaneously to the same 13-disk aggregate in 13 minutes. Aside from VM clock skew problems, I don't see the benefit of Bare Metal and I'm feeling pretty dumb at the moment. Any insight from you, Quanah and/or Howard is humbly accepted and appreciated - I am here to learn. :-) Thank you, Bill -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
Re: Openldap2.4.16 performance issue
FWIW, our entire ldap infrastructure is on VMs - with one exception: on an overloaded VM cluster, we had tons of random auth failures - put in a pizza box machine to replace one of the VMs (and changed load balancer config to use the VM ldap box in that location only as a fail over backup). Our mirrored masters are VMs as well. It's all about the load - there's nothing inherently bad about OpenLDAP on VMs - many people simply overload them. - chris Chris Jacobs, Systems Administrator Apollo Group | Apollo Marketing | Aptimus 2001 6th Ave Ste 3200 | Seattle, WA 98121 phone: 206.839-8245 | cell: 206.601.3256 | Fax: 208.441.9661 email: chris.jac...@apollogrp.edu - Original Message - From: openldap-technical-boun...@openldap.org openldap-technical-boun...@openldap.org To: Dieter Kluenter die...@dkluenter.de Cc: openldap-technical@openldap.org openldap-technical@openldap.org Sent: Mon Aug 23 11:13:43 2010 Subject: Re: Openldap2.4.16 performance issue - Original Message - From: Dieter Kluenter die...@dkluenter.de To: openldap-technical@openldap.org Sent: Saturday, August 21, 2010 3:07:04 AM Subject: Re: Openldap2.4.16 performance issue Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -Yes, OS installed on VM -- what type ---Don’t know There is nothing suspicious, except for the virtual machine. Your really should carefully check layout and configuration of this VM, do not use virtual disks. Hi Dieter, Could you kindly explain what this means? I've been all over the inter-webs and I'm not finding anything concrete about OpenLDAP and VM's or Databases and VM's in general. The closest I came was about some database latency studies and some VMware propaganda. We are about to launch a master and two replicas (utilizing delta-syncrepl) running in Ubuntu 10.04 on a VMware VSphere ESXi 4.0 cluster with four IBM x3650-M2's (2 Quad Nahalem and 64GB memory each) with virtual disks carved from NFS mounts to all of the VSphere servers in the cluster to facilitate HA and FT - by the VMware book. (BTW, I took one of Howard's old posts to heart and we are following the Ubuntu playbook and we have purchased Canonical 24x7 support/maintenance. :-) ) We have had no problems with this environment in development and get better results than on bare metal with or without RAID. I know that it is recommended that the logs live on another disk from the database and RAID is frowned upon, but I have difficulty with a few points: 1) Separate, unprotected disks seems illogical. The last log and the BDB files are necessary to start BDB in slapd, correct? So, if you lose either disk, you're in trouble. Backups are ok, but daily seems too long a time. Seeing as we process new user accounts every 15 minutes, this would not be ideal for us. 2) RAID-1 I can understand having an issue on writes. But what about LUNs in FC from a NetApp 3140? Virtual disks on NFS in VMware? Both of these are in the best practices of both VMware and NetApp documentation. 3) 13-disk 7200RPM SATA RAID-DP (NetApp) is far faster than a single disk, dual diak or RAID-1, so why wouldn't you use SAN/NAS storage? I seriously want to understand the VM concern as it pertains to OpenLDAP. I think more and more people are doing this very thing and will benefit from this discussion. Our database is 86K+ DN's averaging about 40 attributes each. We've tuned the HDB cache to 768MB in a shared memory segment and the pertinent master slapd.conf file shows: shm_key 100 cachesize 20 idlcachesize60 dncachesize 40 checkpoint 1024 15 . . . # main database . . . index objectClass eq index cn eq,sub index sn eq,sub index gn eq,sub index maileq,sub index uid eq,sub index displayname sub,eq index memberUid eq,sub index uidNumber eq index gidNumber eq index sambaSIDeq index sambaSIDlisteq index sambaDomainName eq index sambaPrimaryGroupSIDeq index sambaGroupType eq index entryCSN eq index entryUUID eq index default sub,eq Replicas have identical indexes and shared memory usage. Basically, just running database population tests with full checking turned on, I get the following results: Ubuntu 10.04.1 on all with OpenLDAP 2.4.21/BDB 4.7.25 (all generate 200-10MB log files): IBM x3550 2-quad 5450 16GB, RAID-1 15K 73GB 3.0Gb/s SAS, 86K DN's - 30 minutes IBM x3550 2-quad 5450 16GB, Single 15K 73GB 3.0Gb/s SAS, 86K DN's - 28 minutes IBM x3550 2-quad 5450 16GB, 2-single 15K 73GB (db and logs on separate disks
Re: Openldap2.4.16 performance issue
- Original Message - From: Dan White dwh...@olp.net To: William E Jojo w.j...@hvcc.edu Cc: Dieter Kluenter die...@dkluenter.de, openldap-technical@openldap.org Sent: Monday, August 23, 2010 2:29:25 PM Subject: Re: Openldap2.4.16 performance issue On 23/08/10 14:13 -0400, William E Jojo wrote: Hi Dieter, Could you kindly explain what this means? I've been all over the inter-webs and I'm not finding anything concrete about OpenLDAP and VM's or Databases and VM's in general. The closest I came was about some database latency studies and some VMware propaganda. We are about to launch a master and two replicas (utilizing delta-syncrepl) running in Ubuntu 10.04 on a VMware VSphere ESXi 4.0 cluster with four IBM x3650-M2's (2 Quad Nahalem and 64GB memory each) with virtual disks carved from NFS mounts to all of the VSphere servers in the cluster to facilitate HA and FT - by the VMware book. I think NFS is a red flag here. To what extent to you use NFS in your setup? Do you store any berkeley files on an NFS mount? Nicely spotted, but no. :-) The are no databases on an NFS mount. The NFS buck stops at VMware. The Datastore is an NFS mount and the virtual disks are carved by VMware from this storage. The guest VM never sees anything but the virtualized hardware. Thank you for you input, it is much appreciated! Bill If so, you should carefully research the mixing of the two. The Berkeley documentation discusses it: http://tinyurl.com/28ehu3s -- Dan White
Re: Openldap2.4.16 performance issue
- Original Message - From: Chris Jacobs chris.jac...@apollogrp.edu To: w.j...@hvcc.edu w.j...@hvcc.edu, die...@dkluenter.de die...@dkluenter.de Cc: openldap-technical@openldap.org openldap-technical@openldap.org Sent: Monday, August 23, 2010 2:26:59 PM Subject: Re: Openldap2.4.16 performance issue FWIW, our entire ldap infrastructure is on VMs - with one exception: on an overloaded VM cluster, we had tons of random auth failures - put in a pizza box machine to replace one of the VMs (and changed load balancer config to use the VM ldap box in that location only as a fail over backup). Our mirrored masters are VMs as well. It's all about the load - there's nothing inherently bad about OpenLDAP on VMs - many people simply overload them. Ok, good to know. Load is always a concern for me as well. Thank you for the feedback! Bill - chris Chris Jacobs, Systems Administrator Apollo Group | Apollo Marketing | Aptimus 2001 6th Ave Ste 3200 | Seattle, WA 98121 phone: 206.839-8245 | cell: 206.601.3256 | Fax: 208.441.9661 email: chris.jac...@apollogrp.edu - Original Message - From: openldap-technical-boun...@openldap.org openldap-technical-boun...@openldap.org To: Dieter Kluenter die...@dkluenter.de Cc: openldap-technical@openldap.org openldap-technical@openldap.org Sent: Mon Aug 23 11:13:43 2010 Subject: Re: Openldap2.4.16 performance issue - Original Message - From: Dieter Kluenter die...@dkluenter.de To: openldap-technical@openldap.org Sent: Saturday, August 21, 2010 3:07:04 AM Subject: Re: Openldap2.4.16 performance issue Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -Yes, OS installed on VM -- what type ---Don’t know There is nothing suspicious, except for the virtual machine. Your really should carefully check layout and configuration of this VM, do not use virtual disks. Hi Dieter, Could you kindly explain what this means? I've been all over the inter-webs and I'm not finding anything concrete about OpenLDAP and VM's or Databases and VM's in general. The closest I came was about some database latency studies and some VMware propaganda. We are about to launch a master and two replicas (utilizing delta-syncrepl) running in Ubuntu 10.04 on a VMware VSphere ESXi 4.0 cluster with four IBM x3650-M2's (2 Quad Nahalem and 64GB memory each) with virtual disks carved from NFS mounts to all of the VSphere servers in the cluster to facilitate HA and FT - by the VMware book. (BTW, I took one of Howard's old posts to heart and we are following the Ubuntu playbook and we have purchased Canonical 24x7 support/maintenance. :-) ) We have had no problems with this environment in development and get better results than on bare metal with or without RAID. I know that it is recommended that the logs live on another disk from the database and RAID is frowned upon, but I have difficulty with a few points: 1) Separate, unprotected disks seems illogical. The last log and the BDB files are necessary to start BDB in slapd, correct? So, if you lose either disk, you're in trouble. Backups are ok, but daily seems too long a time. Seeing as we process new user accounts every 15 minutes, this would not be ideal for us. 2) RAID-1 I can understand having an issue on writes. But what about LUNs in FC from a NetApp 3140? Virtual disks on NFS in VMware? Both of these are in the best practices of both VMware and NetApp documentation. 3) 13-disk 7200RPM SATA RAID-DP (NetApp) is far faster than a single disk, dual diak or RAID-1, so why wouldn't you use SAN/NAS storage? I seriously want to understand the VM concern as it pertains to OpenLDAP. I think more and more people are doing this very thing and will benefit from this discussion. Our database is 86K+ DN's averaging about 40 attributes each. We've tuned the HDB cache to 768MB in a shared memory segment and the pertinent master slapd.conf file shows: shm_key 100 cachesize 20 idlcachesize 60 dncachesize 40 checkpoint 1024 15 . . . # main database . . . index objectClass eq index cn eq,sub index sn eq,sub index gn eq,sub index mail eq,sub index uid eq,sub index displayname sub,eq index memberUid eq,sub index uidNumber eq index gidNumber eq index sambaSID eq index sambaSIDlist eq index sambaDomainName eq index sambaPrimaryGroupSID eq index sambaGroupType eq index entryCSN eq index entryUUID eq index default sub,eq Replicas have identical indexes and shared memory usage. Basically, just running
Re: Openldap2.4.16 performance issue
Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -Yes, OS installed on VM -- what type ---Don’t know There is nothing suspicious, except for the virtual machine. Your really should carefully check layout and configuration of this VM, do not use virtual disks. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
RE: Openldap2.4.16 performance issue
Hi, On Fri, 20 Aug 2010, Singh, Devender (GE Capital, consultant) wrote: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm Apart from what all others have already said ext3 with a heavy write load is going to be a Problem with recent linux kernels. How is the rest of you guys coping with ext3 and the fsync issue ? Greetings Christian -- Christian Kratzer CK Software GmbH Email: c...@cksoft.de Schwarzwaldstr. 31 Phone: +49 7452 889 135 D-71131 Jettingen Fax: +49 7452 889 136 HRB 245288, Amtsgericht Stuttgart Web: http://www.cksoft.de/ Geschaeftsfuehrer: Christian Kratzer
Re: Openldap2.4.16 performance issue
Hello, this message exchange is really funny, since someone who calls himself an senior unix administrator is not able to answer simple questions about his own systems and wants uberfast results from an application point of view he doesn't pay any support for. Dear Devender, please review your questions and the answers you gave on this mailinglist and how you wrote and may think of the impression other people get from you. Thank for reading. bye. :) On Fri, Aug 20, 2010 at 12:18, Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -Yes, OS installed on VM -- what type ---Don’t know Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Dieter Kluenter Sent: Thursday, August 19, 2010 11:40 PM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: I agree with you. Please suggest me what to do for resolution of this issue. Frankly, this is simple unix system administration. A few questions: 1. hardware related - type of storage - raid level, if any - file system of disk(s) - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -- what type -Dieter Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Please find the below answers: [r...@abc openldap-data-ge_cw]# du -sh *.bdb 3.6M br.bdb 72K cn.bdb 32K displayName.bdb 234M dn2id.bdb 104K gr.bdb 419M id2entry.bdb 56K mail.bdb 1.4M objectClass.bdb 2.9M pf.bdb 212K pr.bdb 72K sn.bdb 72K uid.bdb I have seen the problems you describe before. Although a configured cache size of 250M and a database size of some 660M is not sufficient, it still is not such a bottleneck. To my experience a heavy cpu load is most likely based on heavy disk operations. If moving the transaction logs onto a separate disk didn't solve it, look for other concurrent read/write operations. Check whether the logs report constantly deadlocks. In some cases a journaling file system reduced performance. I experienced rather bad results with xfs. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6 -- To be or not to be -- Shakespeare | To do is to be -- Nietzsche | To be is to do -- Sartre | Do be do be do -- Sinatra
Re: Openldap2.4.16 performance issue
and I saw this issue with Linux KVM, RAID5 and xfs a few times. Well, when it comes to LDAP, there are three things wrong with that sentence. -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmad...@ivytech.edu
Re: Openldap2.4.16 performance issue
I think I see the problem here: echo $sender | grep -i consultant; echo $? Seriously though, I agree with you Benjamin. Devender: Please read ALL emails sent to you much more carefully as you have missed/ignored several of the free attempts to help you. If you want free help from a voluntary community, treat them with more respect, acknowledge all advice given (definitely do not ignore it), and try to help everyone by simply responding with extra information as and when it is asked of you! Just out of interest, what kind of Senior Unix Administrator are you if you can not see a problem with this: --snip-- - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID --snip-- if virtual machine, -Yes --snip-- and finally: --snip-- Databse1: Number of users : 83 Number of dns: 83 Database2: Number of users: 2000 Number of dns: 80 --snip-- In Application2(90% dependent on openldap) every user have 1000+ sub leafs entries --snip-- !! To top it off, you're running a potentially very active LDAP database in a VM!! How would you ever expect that level of quotation_fingershardware/quotation_fingers to cope with such potential volumes of network traffic anyway?? My advice before you post back: 1. Go back through the email trail and read ALL advice and act on it appropriately. 2. Upgrade OpenLDAP to a more (if not the most) recent version. 3. Get it on bare metal (preferably something with higher spec for improved IO (and maybe an off-loaded network card). My 2pence Russell Knighton On Fri, 2010-08-20 at 13:15 +0100, Benjamin Griese wrote: Hello, this message exchange is really funny, since someone who calls himself an senior unix administrator is not able to answer simple questions about his own systems and wants uberfast results from an application point of view he doesn't pay any support for. Dear Devender, please review your questions and the answers you gave on this mailinglist and how you wrote and may think of the impression other people get from you. Thank for reading. bye. :) On Fri, Aug 20, 2010 at 12:18, Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: Hi Dieter, Please find the below details: 1. hardware related - type of storage - Simply SATA had disk attached. - raid level, if any- No RAID - file system of disk(s)- ext3 on LVm - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -Yes, OS installed on VM -- what type ---Don’t know Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Dieter Kluenter Sent: Thursday, August 19, 2010 11:40 PM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: I agree with you. Please suggest me what to do for resolution of this issue. Frankly, this is simple unix system administration. A few questions: 1. hardware related - type of storage - raid level, if any - file system of disk(s) - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -- what type -Dieter Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Please find the below answers: [r...@abc openldap-data-ge_cw]# du -sh *.bdb 3.6Mbr.bdb 72K cn.bdb 32K displayName.bdb 234Mdn2id.bdb 104Kgr.bdb 419Mid2entry.bdb 56K mail.bdb 1.4MobjectClass.bdb 2.9Mpf.bdb 212Kpr.bdb 72K sn.bdb 72K uid.bdb I have seen the problems you describe before. Although a configured cache size of 250M and a database size of some 660M is not sufficient, it still is not such a bottleneck. To my experience a heavy cpu load is most likely based on heavy disk operations. If moving the transaction logs onto a separate disk didn't solve it, look for other concurrent read/write operations. Check whether the logs report constantly deadlocks. In some cases a journaling file system reduced performance. I experienced rather bad results with xfs. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6 -- To be or not to be -- Shakespeare | To do is to be -- Nietzsche | To be is to do -- Sartre | Do be do be do -- Sinatra -- - Russell Knighton - Motion Picture Solutions Ltd Richmond Cottage, 7b North End Road, London, W14 8ST Mob: +44 (0) 7758 210 744
Re: Openldap2.4.16 performance issue
Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Please find the below answers: [r...@abc openldap-data-ge_cw]# du -sh *.bdb 3.6Mbr.bdb 72K cn.bdb 32K displayName.bdb 234Mdn2id.bdb 104Kgr.bdb 419Mid2entry.bdb 56K mail.bdb 1.4MobjectClass.bdb 2.9Mpf.bdb 212Kpr.bdb 72K sn.bdb 72K uid.bdb I have seen the problems you describe before. Although a configured cache size of 250M and a database size of some 660M is not sufficient, it still is not such a bottleneck. To my experience a heavy cpu load is most likely based on heavy disk operations. If moving the transaction logs onto a separate disk didn't solve it, look for other concurrent read/write operations. Check whether the logs report constantly deadlocks. In some cases a journaling file system reduced performance. I experienced rather bad results with xfs. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
RE: Openldap2.4.16 performance issue
I agree with you. Please suggest me what to do for resolution of this issue. Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Dieter Kluenter Sent: Thursday, August 19, 2010 2:06 PM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Please find the below answers: [r...@abc openldap-data-ge_cw]# du -sh *.bdb 3.6Mbr.bdb 72K cn.bdb 32K displayName.bdb 234Mdn2id.bdb 104Kgr.bdb 419Mid2entry.bdb 56K mail.bdb 1.4MobjectClass.bdb 2.9Mpf.bdb 212Kpr.bdb 72K sn.bdb 72K uid.bdb I have seen the problems you describe before. Although a configured cache size of 250M and a database size of some 660M is not sufficient, it still is not such a bottleneck. To my experience a heavy cpu load is most likely based on heavy disk operations. If moving the transaction logs onto a separate disk didn't solve it, look for other concurrent read/write operations. Check whether the logs report constantly deadlocks. In some cases a journaling file system reduced performance. I experienced rather bad results with xfs. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
RE: Openldap2.4.16 performance issue
--On August 19, 2010 9:39:58 PM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: I agree with you. Please suggest me what to do for resolution of this issue. Unfortunately, Dieter is ignoring potential issues with known problems with BDB, and the fact that a number of serious issues were fixed since OpenLDAP 2.4.16. Please go answer my list of questions I mailed you yesterday. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: Openldap2.4.16 performance issue
Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: I agree with you. Please suggest me what to do for resolution of this issue. Frankly, this is simple unix system administration. A few questions: 1. hardware related - type of storage - raid level, if any - file system of disk(s) - type of network, 100MB, 1G, 10G 2. is this host running on a virtual machine or on bare metal. - if virtual machine, -- what type -Dieter Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Please find the below answers: [r...@abc openldap-data-ge_cw]# du -sh *.bdb 3.6Mbr.bdb 72K cn.bdb 32K displayName.bdb 234Mdn2id.bdb 104Kgr.bdb 419Mid2entry.bdb 56K mail.bdb 1.4MobjectClass.bdb 2.9Mpf.bdb 212Kpr.bdb 72K sn.bdb 72K uid.bdb I have seen the problems you describe before. Although a configured cache size of 250M and a database size of some 660M is not sufficient, it still is not such a bottleneck. To my experience a heavy cpu load is most likely based on heavy disk operations. If moving the transaction logs onto a separate disk didn't solve it, look for other concurrent read/write operations. Check whether the logs report constantly deadlocks. In some cases a journaling file system reduced performance. I experienced rather bad results with xfs. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
Re: Openldap2.4.16 performance issue
Quanah Gibson-Mount qua...@zimbra.com writes: --On August 19, 2010 9:39:58 PM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: I agree with you. Please suggest me what to do for resolution of this issue. Unfortunately, Dieter is ignoring potential issues with known problems with BDB, and the fact that a number of serious issues were fixed since OpenLDAP 2.4.16. Please go answer my list of questions I mailed you yesterday. Quanah, I quite agree with you, there have been problems with early 2.4 versions of OpenLDAP and BDB, and I am quite aware of the fixed issues since 2.4.13. But let's solve hardware problems first, if there are any. If it is assured that hardware and filesystem as the culprid can be excluded, the focus can put on actual OpenLDAP versions. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
Re: Openldap2.4.16 performance issue
--On August 19, 2010 8:21:01 PM +0200 Dieter Kluenter die...@dkluenter.de wrote: Quanah Gibson-Mount qua...@zimbra.com writes: --On August 19, 2010 9:39:58 PM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: I agree with you. Please suggest me what to do for resolution of this issue. Unfortunately, Dieter is ignoring potential issues with known problems with BDB, and the fact that a number of serious issues were fixed since OpenLDAP 2.4.16. Please go answer my list of questions I mailed you yesterday. Quanah, I quite agree with you, there have been problems with early 2.4 versions of OpenLDAP and BDB, and I am quite aware of the fixed issues since 2.4.13. But let's solve hardware problems first, if there are any. If it is assured that hardware and filesystem as the culprid can be excluded, the focus can put on actual OpenLDAP versions. Except for that fact that I know of people who saw this exact issue with earlier OpenLDAP versions, and we know for a fact there are issues with unpatched BDB 4.7, and there are issues with BDB 4.8 prior to 4.8.30. So it is *extremely* relevant to (a) ensure he is on a current release and (b) that he's using a valid BDB version. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: Openldap2.4.16 performance issue
Quanah Gibson-Mount qua...@zimbra.com writes: --On August 19, 2010 8:21:01 PM +0200 Dieter Kluenter die...@dkluenter.de wrote: Quanah Gibson-Mount qua...@zimbra.com writes: --On August 19, 2010 9:39:58 PM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: I agree with you. Please suggest me what to do for resolution of this issue. Unfortunately, Dieter is ignoring potential issues with known problems with BDB, and the fact that a number of serious issues were fixed since OpenLDAP 2.4.16. Please go answer my list of questions I mailed you yesterday. Quanah, I quite agree with you, there have been problems with early 2.4 versions of OpenLDAP and BDB, and I am quite aware of the fixed issues since 2.4.13. But let's solve hardware problems first, if there are any. If it is assured that hardware and filesystem as the culprid can be excluded, the focus can put on actual OpenLDAP versions. Except for that fact that I know of people who saw this exact issue with earlier OpenLDAP versions, and we know for a fact there are issues with unpatched BDB 4.7, and there are issues with BDB 4.8 prior to 4.8.30. So it is *extremely* relevant to (a) ensure he is on a current release and (b) that he's using a valid BDB version. and I saw this issue with Linux KVM, RAID5 and xfs a few times. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
Re: Openldap2.4.16 performance issue
Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi Dieter, I need to tune any parameter in DB_CONFIG file for this or not? Because I am using default DB_CONFIG file. Just edit DB_CONFIG set_log_dir /mountpoint/path/to/ -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
RE: Openldap2.4.16 performance issue
Hi Dieter, I need to tune any parameter in DB_CONFIG file for this or not? Because I am using default DB_CONFIG file. Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Dieter Kluenter Sent: Tuesday, August 17, 2010 1:12 PM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi All, I need help for openldap slapd 200% cpu utilization issue. I have configured three openldap servers(2 Masters and 1 Slave). I have configure PEN load balancer for failover setup on 2 master openldap servers.It means application server first of all hit the loadbalancer port and than PEN LB will forwad that request to Master 1 or 2 openldap server. Hardware configuration on all boxes: OS: RHEL5(x86_64) RAM: 2GB Number of CPU: 2(Intel(R) Xeon(R)2.00GHz) Number of BDB databases: 2 Databse1: Number of users : 83 Number of dns: 83 Database2: Number of users: 2000 Number of dns: 80 My Application1 using Databse1 and Application2 using Databse2. Application2 just authenticating the users and store last 10 password history only, It means Application1 is not using openldap too much. In Application2(90% dependent on openldap) every user have 1000+ sub leafs entries. when I want to do major changes(number of leafs write) into this, it.s got hanged and got socket closed error in application logs and CPU utilization goes 200% and RAM 52%. My DB_CONFIG file is below: Most likely to many concurrent write operations. You should move the db logfiles onto a different disk. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
Re: Openldap2.4.16 performance issue
Singh, Devender (GE Capital, consultant) wrote: Hi Chu, Please help me on my below issue. It’s very urgent. If you have a support contract with us, you can contact us at supp...@symas.com for help. Otherwise, people help on this list as their time and interest allows. Thanks Regards,// *Devender Singh* Senior Unix Administrator,// *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Singh, Devender (GE Capital, consultant) *Sent:* Tuesday, August 17, 2010 5:12 AM *To:* openldap-technical@openldap.org *Subject:* Openldap2.4.16 performance issue Hi All, I need help for openldap slapd 200% cpu utilization issue. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
RE: Openldap2.4.16 performance issue
Off the top of my head: 1. What indexes have been created? Do they match the attributes that your applications use most often? 2. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade. 3. Configure LB for active-active instead of active-passive? 4. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server. 5. Upgrade from 2.4.16 to 2.4.xx? From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Wednesday, August 18, 2010 3:15 PM To: h...@symas.com Cc: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue Hi Chu, Please help me on my below issue. It's very urgent. Thanks Regards, Devender Singh Senior Unix Administrator, From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Tuesday, August 17, 2010 5:12 AM To: openldap-technical@openldap.org Subject: Openldap2.4.16 performance issue Hi All, I need help for openldap slapd 200% cpu utilization issue. I have configured three openldap servers(2 Masters and 1 Slave). I have configure PEN load balancer for failover setup on 2 master openldap servers.It means application server first of all hit the loadbalancer port and than PEN LB will forwad that request to Master 1 or 2 openldap server. Hardware configuration on all boxes: OS: RHEL5(x86_64) RAM: 2GB Number of CPU: 2(Intel(R) Xeon(R)2.00GHz) Number of BDB databases: 2 Databse1: Number of users : 83 Number of dns: 83 Database2: Number of users: 2000 Number of dns: 80 My Application1 using Databse1 and Application2 using Databse2. Application2 just authenticating the users and store last 10 password history only, It means Application1 is not using openldap too much. In Application2(90% dependent on openldap) every user have 1000+ sub leafs entries. when I want to do major changes(number of leafs write) into this, it.s got hanged and got socket closed error in application logs and CPU utilization goes 200% and RAM 52%. My DB_CONFIG file is below: # $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.3.2.4 2007/12/18 11:53:27 ghenry Exp $ # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. # # See the Oracle Berkeley DB documentation # http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_config.html # for detail description of DB_CONFIG syntax and semantics. # # Hints can also be found in the OpenLDAP Software FAQ # http://www.openldap.org/faq/index.cgi?file=2 # in particular: # http://www.openldap.org/faq/index.cgi?file=1075 # Note: most DB_CONFIG settings will take effect only upon rebuilding # the DB environment. # one 0.25 GB cache set_cachesize 0 268435456 1 # Data Directory #set_data_dir db # Transaction Log settings set_lg_regionmax 262144 set_lg_bsize 2097152 #set_lg_dir logs # Note: special DB_CONFIG flags are no longer needed for quick # slapadd(8) or slapindex(8) access (see their -q option). Please help me here that what I need to do for better performance. Thanks in advance. My contact number is +919650477441 Thanks Regards, Devender Singh Senior Unix Administrator,
RE: Openldap2.4.16 performance issue
Devender, You did see this email reply, right: Singh, Devender (GE Capital, consultant) devender.sin...@ge.commailto:devender.sin...@ge.com writes: Hi Dieter, I need to tune any parameter in DB_CONFIG file for this or not? Because I am using default DB_CONFIG file. Just edit DB_CONFIG set_log_dir /mountpoint/path/to/ -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.demailto:7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6 Have you tried that yet? Any change? - chris From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Wednesday, August 18, 2010 3:15 PM To: h...@symas.com Cc: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue Hi Chu, Please help me on my below issue. It's very urgent. Thanks Regards, Devender Singh Senior Unix Administrator, From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Tuesday, August 17, 2010 5:12 AM To: openldap-technical@openldap.org Subject: Openldap2.4.16 performance issue Hi All, I need help for openldap slapd 200% cpu utilization issue. I have configured three openldap servers(2 Masters and 1 Slave). I have configure PEN load balancer for failover setup on 2 master openldap servers.It means application server first of all hit the loadbalancer port and than PEN LB will forwad that request to Master 1 or 2 openldap server. Hardware configuration on all boxes: OS: RHEL5(x86_64) RAM: 2GB Number of CPU: 2(Intel(R) Xeon(R)2.00GHz) Number of BDB databases: 2 Databse1: Number of users : 83 Number of dns: 83 Database2: Number of users: 2000 Number of dns: 80 My Application1 using Databse1 and Application2 using Databse2. Application2 just authenticating the users and store last 10 password history only, It means Application1 is not using openldap too much. In Application2(90% dependent on openldap) every user have 1000+ sub leafs entries. when I want to do major changes(number of leafs write) into this, it.s got hanged and got socket closed error in application logs and CPU utilization goes 200% and RAM 52%. My DB_CONFIG file is below: # $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.3.2.4 2007/12/18 11:53:27 ghenry Exp $ # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. # # See the Oracle Berkeley DB documentation # http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_config.html # for detail description of DB_CONFIG syntax and semantics. # # Hints can also be found in the OpenLDAP Software FAQ # http://www.openldap.org/faq/index.cgi?file=2 # in particular: # http://www.openldap.org/faq/index.cgi?file=1075 # Note: most DB_CONFIG settings will take effect only upon rebuilding # the DB environment. # one 0.25 GB cache set_cachesize 0 268435456 1 # Data Directory #set_data_dir db # Transaction Log settings set_lg_regionmax 262144 set_lg_bsize 2097152 #set_lg_dir logs # Note: special DB_CONFIG flags are no longer needed for quick # slapadd(8) or slapindex(8) access (see their -q option). Please help me here that what I need to do for better performance. Thanks in advance. My contact number is +919650477441 Thanks Regards, Devender Singh Senior Unix Administrator, This message is private and confidential. If you have received it in error, please notify the sender and remove it from your system.
RE: Openldap2.4.16 performance issue
Yes I did it, but not getting good performance. I restart slapd every time when cpu goes 200%. Thanks Regards, Devender Singh Senior Unix Administrator, From: Chris Jacobs [mailto:chris.jac...@apollogrp.edu] Sent: Thursday, August 19, 2010 4:17 AM To: Singh, Devender (GE Capital, consultant); h...@symas.com Cc: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue Devender, You did see this email reply, right: Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi Dieter, I need to tune any parameter in DB_CONFIG file for this or not? Because I am using default DB_CONFIG file. Just edit DB_CONFIG set_log_dir /mountpoint/path/to/ -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6 Have you tried that yet? Any change? - chris From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Wednesday, August 18, 2010 3:15 PM To: h...@symas.com Cc: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue Hi Chu, Please help me on my below issue. It's very urgent. Thanks Regards, Devender Singh Senior Unix Administrator, From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Tuesday, August 17, 2010 5:12 AM To: openldap-technical@openldap.org Subject: Openldap2.4.16 performance issue Hi All, I need help for openldap slapd 200% cpu utilization issue. I have configured three openldap servers(2 Masters and 1 Slave). I have configure PEN load balancer for failover setup on 2 master openldap servers.It means application server first of all hit the loadbalancer port and than PEN LB will forwad that request to Master 1 or 2 openldap server. Hardware configuration on all boxes: OS: RHEL5(x86_64) RAM: 2GB Number of CPU: 2(Intel(R) Xeon(R)2.00GHz) Number of BDB databases: 2 Databse1: Number of users : 83 Number of dns: 83 Database2: Number of users: 2000 Number of dns: 80 My Application1 using Databse1 and Application2 using Databse2. Application2 just authenticating the users and store last 10 password history only, It means Application1 is not using openldap too much. In Application2(90% dependent on openldap) every user have 1000+ sub leafs entries. when I want to do major changes(number of leafs write) into this, it.s got hanged and got socket closed error in application logs and CPU utilization goes 200% and RAM 52%. My DB_CONFIG file is below: # $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.3.2.4 2007/12/18 11:53:27 ghenry Exp $ # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. # # See the Oracle Berkeley DB documentation # http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_config.html http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_config.html # for detail description of DB_CONFIG syntax and semantics. # # Hints can also be found in the OpenLDAP Software FAQ # http://www.openldap.org/faq/index.cgi?file=2 http://www.openldap.org/faq/index.cgi?file=2 # in particular: # http://www.openldap.org/faq/index.cgi?file=1075 http://www.openldap.org/faq/index.cgi?file=1075 # Note: most DB_CONFIG settings will take effect only upon rebuilding # the DB environment. # one 0.25 GB cache set_cachesize 0 268435456 1 # Data Directory #set_data_dir db # Transaction Log settings set_lg_regionmax 262144 set_lg_bsize 2097152 #set_lg_dir logs # Note: special DB_CONFIG flags are no longer needed for quick # slapadd(8) or slapindex(8) access (see their -q option). Please help me here that what I need to do for better performance. Thanks in advance. My contact number is +919650477441 Thanks Regards, Devender Singh Senior Unix Administrator, This message is private and confidential. If you have received it in error, please notify the sender and remove it from your system.
RE: Openldap2.4.16 performance issue
--On August 19, 2010 4:23:31 AM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: Yes I did it, but not getting good performance. I restart slapd every time when cpu goes 200%. What is the size of your *.bdb files? Are you on a 32-bit or 64-bit server? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
RE: Openldap2.4.16 performance issue
Please find the below answers: [r...@abc openldap-data-ge_cw]# du -sh *.bdb 3.6Mbr.bdb 72K cn.bdb 32K displayName.bdb 234Mdn2id.bdb 104Kgr.bdb 419Mid2entry.bdb 56K mail.bdb 1.4MobjectClass.bdb 2.9Mpf.bdb 212Kpr.bdb 72K sn.bdb 72K uid.bdb [r...@abc openldap-data-ge_cw]# getconf LONG_BIT 32 Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. -Original Message- From: Quanah Gibson-Mount [mailto:qua...@zimbra.com] Sent: Thursday, August 19, 2010 4:34 AM To: Singh, Devender (GE Capital, consultant); Chris Jacobs Cc: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue --On August 19, 2010 4:23:31 AM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: Yes I did it, but not getting good performance. I restart slapd every time when cpu goes 200%. What is the size of your *.bdb files? Are you on a 32-bit or 64-bit server? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
RE: Openldap2.4.16 performance issue
Please find the answers: 1. What indexes have been created? Do they match the attributes that your applications use most often? --- All required attributes related application indexed(equality) 2. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade.-We can increase it by 3-4 GB only 3. Configure LB for active-active instead of active-passive?-I have configured PEN LB as a active-active(both master are active-active) 4. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server.(I have configured loglevel 256, if I set loglevel -1, it will slow the read write speed from/to openldap server and also eating cpu. ) 5. Upgrade from 2.4.16 to 2.4.xx?---I don't think that up gradation will solve this issue. Here my concern is: I think I need to set below parameter in DB_CONFIG file: set_cachesize (but what should be the tuned value according to my data?) Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com http://www.sdgc.com/ Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Siddhartha Jain Sent: Thursday, August 19, 2010 4:17 AM To: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue Off the top of my head: 6. What indexes have been created? Do they match the attributes that your applications use most often? 7. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade. 8. Configure LB for active-active instead of active-passive? 9. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server. 10. Upgrade from 2.4.16 to 2.4.xx? From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Wednesday, August 18, 2010 3:15 PM To: h...@symas.com Cc: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue Hi Chu, Please help me on my below issue. It's very urgent. Thanks Regards, Devender Singh Senior Unix Administrator, From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Tuesday, August 17, 2010 5:12 AM To: openldap-technical@openldap.org Subject: Openldap2.4.16 performance issue Hi All, I need help for openldap slapd 200% cpu utilization issue. I have configured three openldap servers(2 Masters and 1 Slave). I have configure PEN load balancer for failover setup on 2 master openldap servers.It means application server first of all hit the loadbalancer port and than PEN LB will forwad that request to Master 1 or 2 openldap server. Hardware configuration on all boxes: OS: RHEL5(x86_64) RAM: 2GB Number of CPU: 2(Intel(R) Xeon(R)2.00GHz) Number of BDB databases: 2 Databse1: Number of users : 83 Number of dns: 83 Database2: Number of users: 2000 Number of dns: 80 My Application1 using Databse1 and Application2 using Databse2. Application2 just authenticating the users and store last 10 password history only, It means Application1 is not using openldap too much. In Application2(90% dependent on openldap) every user have 1000+ sub leafs entries. when I want to do major changes(number of leafs write) into this, it.s got hanged and got socket closed error in application logs and CPU utilization goes 200% and RAM 52%. My DB_CONFIG file is below: # $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.3.2.4 2007/12/18 11:53:27 ghenry Exp $ # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. # # See the Oracle Berkeley DB documentation # http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/d b_config.html http://www.oracle.com/technology
RE: Openldap2.4.16 performance issue
--On August 19, 2010 4:41:14 AM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: 5. Upgrade from 2.4.16 to 2.4.xx?---I don’t think that up gradation will solve this issue. I disagree, I think it could have a significant result in your issue. Here my concern is: I think I need to set below parameter in DB_CONFIG file: set_cachesize (but what should be the tuned value according to my data?) Based on what you wrote before, I'd set it to use 500MB --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
RE: Openldap2.4.16 performance issue
If you want any other information, I can give you. I need permanent solution:) Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. -Original Message- From: Quanah Gibson-Mount [mailto:qua...@zimbra.com] Sent: Thursday, August 19, 2010 4:59 AM To: Singh, Devender (GE Capital, consultant); Siddhartha Jain; openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue --On August 19, 2010 4:41:14 AM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: 5. Upgrade from 2.4.16 to 2.4.xx?---I don’t think that up gradation will solve this issue. I disagree, I think it could have a significant result in your issue. Here my concern is: I think I need to set below parameter in DB_CONFIG file: set_cachesize (but what should be the tuned value according to my data?) Based on what you wrote before, I'd set it to use 500MB --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
RE: Openldap2.4.16 performance issue
As per the client requirement there is no need of substring indexing Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Keutel, Jochen Sent: Thursday, August 19, 2010 5:26 AM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Hi, are you sure that just equality index is sufficient? Most applications do substring searches, e.g. sn=sin*. I'd recommend to add substr to the often used attributes. Regards, Jochen. Am 19.08.2010 01:11, schrieb Singh, Devender (GE Capital, consultant): Please find the answers: 1. What indexes have been created? Do they match the attributes that your applications use most often? --- All required attributes related application indexed(equality) 2. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade.-We can increase it by 3-4 GB only 3. Configure LB for active-active instead of active-passive?-I have configured PEN LB as a active-active(both master are active-active) 4. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server.(I have configured loglevel 256, if I set loglevel -1, it will slow the read write speed from/to openldap server and also eating cpu. ) 5. Upgrade from 2.4.16 to 2.4.xx?---I don't think that up gradation will solve this issue. Here my concern is: I think I need to set below parameter in DB_CONFIG file: */set_cachesize (/*but what should be the tuned value according to my data?) Thanks Regards,// *Devender Singh* Senior Unix Administrator,// (SOA Support Team)// /___ _/// *SDG Software India Pvt. Ltd*// A-10, Sector 2, Noida 201301, U.P, India// M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020// website : www.sdgc.com http://www.sdgc.com/// /___ _/// *Please Note:* The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Siddhartha Jain *Sent:* Thursday, August 19, 2010 4:17 AM *To:* openldap-technical@openldap.org *Subject:* RE: Openldap2.4.16 performance issue Off the top of my head: 6. What indexes have been created? Do they match the attributes that your applications use most often? 7. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade. 8. Configure LB for active-active instead of active-passive? 9. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server. 10. Upgrade from 2.4.16 to 2.4.xx? *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Singh, Devender (GE Capital, consultant) *Sent:* Wednesday, August 18, 2010 3:15 PM *To:* h...@symas.com *Cc:* openldap-technical@openldap.org *Subject:* RE: Openldap2.4.16 performance issue Hi Chu, Please help me on my below issue. It's very urgent. Thanks Regards,// *Devender Singh* Senior Unix Administrator,// *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Singh, Devender (GE
RE: Openldap2.4.16 performance issue
In that case, you should ask the *client* to give you a solution. Seriously, if you do not have complete control over LDAP configuration or if *client* dictates certain config parameters then it is best to report this as a bug to your application team. Do you have a test instance to do test config changes on? -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Wednesday, August 18, 2010 5:02 PM To: Keutel, Jochen; openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue As per the client requirement there is no need of substring indexing -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Keutel, Jochen Sent: Thursday, August 19, 2010 5:26 AM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Hi, are you sure that just equality index is sufficient? Most applications do substring searches, e.g. sn=sin*. I'd recommend to add substr to the often used attributes. Regards, Jochen. Am 19.08.2010 01:11, schrieb Singh, Devender (GE Capital, consultant): Please find the answers: 1. What indexes have been created? Do they match the attributes that your applications use most often? --- All required attributes related application indexed(equality) 2. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade.-We can increase it by 3-4 GB only 3. Configure LB for active-active instead of active-passive?-I have configured PEN LB as a active-active(both master are active-active) 4. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server.(I have configured loglevel 256, if I set loglevel -1, it will slow the read write speed from/to openldap server and also eating cpu. ) 5. Upgrade from 2.4.16 to 2.4.xx?---I don't think that up gradation will solve this issue. Here my concern is: I think I need to set below parameter in DB_CONFIG file: */set_cachesize (/*but what should be the tuned value according to my data?) Thanks Regards,// *Devender Singh* Senior Unix Administrator,// (SOA Support Team)// /___ _/// *SDG Software India Pvt. Ltd*// A-10, Sector 2, Noida 201301, U.P, India// M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020// website : www.sdgc.com http://www.sdgc.com/// /___ _/// *Please Note:* The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Siddhartha Jain *Sent:* Thursday, August 19, 2010 4:17 AM *To:* openldap-technical@openldap.org *Subject:* RE: Openldap2.4.16 performance issue Off the top of my head: 6. What indexes have been created? Do they match the attributes that your applications use most often? 7. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade. 8. Configure LB for active-active instead of active-passive? 9. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server. 10. Upgrade from 2.4.16 to 2.4.xx? *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Singh, Devender (GE Capital, consultant) *Sent:* Wednesday, August 18, 2010 3:15 PM *To:* h...@symas.com *Cc:* openldap-technical@openldap.org *Subject:* RE: Openldap2.4.16 performance issue Hi Chu, Please help me on my below issue. It's very urgent. Thanks Regards,// *Devender Singh* Senior Unix Administrator,// *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Singh, Devender (GE Capital, consultant) *Sent:* Tuesday, August 17, 2010 5:12 AM *To:* openldap-technical@openldap.org *Subject:* Openldap2.4.16 performance issue Hi All, I need help for openldap slapd 200% cpu utilization issue. I have configured three openldap servers(2 Masters and 1 Slave). I have configure PEN load
RE: Openldap2.4.16 performance issue
Before migration, the application was running fine without any issue on IBM Tivoli directory server 5.2. Yes we have a test server. Please suggest me configuration level changes. Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Siddhartha Jain Sent: Thursday, August 19, 2010 5:58 AM To: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue In that case, you should ask the *client* to give you a solution. Seriously, if you do not have complete control over LDAP configuration or if *client* dictates certain config parameters then it is best to report this as a bug to your application team. Do you have a test instance to do test config changes on? -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Wednesday, August 18, 2010 5:02 PM To: Keutel, Jochen; openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue As per the client requirement there is no need of substring indexing -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Keutel, Jochen Sent: Thursday, August 19, 2010 5:26 AM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Hi, are you sure that just equality index is sufficient? Most applications do substring searches, e.g. sn=sin*. I'd recommend to add substr to the often used attributes. Regards, Jochen. Am 19.08.2010 01:11, schrieb Singh, Devender (GE Capital, consultant): Please find the answers: 1. What indexes have been created? Do they match the attributes that your applications use most often? --- All required attributes related application indexed(equality) 2. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade.-We can increase it by 3-4 GB only 3. Configure LB for active-active instead of active-passive?-I have configured PEN LB as a active-active(both master are active-active) 4. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server.(I have configured loglevel 256, if I set loglevel -1, it will slow the read write speed from/to openldap server and also eating cpu. ) 5. Upgrade from 2.4.16 to 2.4.xx?---I don't think that up gradation will solve this issue. Here my concern is: I think I need to set below parameter in DB_CONFIG file: */set_cachesize (/*but what should be the tuned value according to my data?) Thanks Regards,// *Devender Singh* Senior Unix Administrator,// (SOA Support Team)// /___ _/// *SDG Software India Pvt. Ltd*// A-10, Sector 2, Noida 201301, U.P, India// M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020// website : www.sdgc.com http://www.sdgc.com/// /___ _/// *Please Note:* The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. *From:* openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] *On Behalf Of *Siddhartha Jain *Sent:* Thursday, August 19, 2010 4:17 AM *To:* openldap-technical@openldap.org *Subject:* RE: Openldap2.4.16 performance issue Off
RE: Openldap2.4.16 performance issue
1) You seriously need to use OpenLDAP 2.4.23. I don't care if you don't think that'll solve the issue or not. ;) 2) You need to state which backend you are using (back-hdb, back-bdb, etc) 3) You need to state your updated DB_CONFIG, based on the cachesize info I suggested earlier 4) You need to state what version of BDB back-hdb/back-bdb is using. This is critical to know 5) You need to state if you have modified the threads setting for slapd.conf or related for cn=config, depending which you are using. --Quanah --On August 19, 2010 6:05:19 AM +0530 Singh, Devender (GE Capital, consultant) devender.sin...@ge.com wrote: Before migration, the application was running fine without any issue on IBM Tivoli directory server 5.2. Yes we have a test server. Please suggest me configuration level changes. Thanks Regards, Devender Singh Senior Unix Administrator, (SOA Support Team) SDG Software India Pvt. Ltd A-10, Sector 2, Noida 201301, U.P, India M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020 website : www.sdgc.com Please Note: The e-mail content is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you have erroneously received this message, please delete it immediately and notify the sender. Before opening any attachments please check them for viruses and defects. -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Siddhartha Jain Sent: Thursday, August 19, 2010 5:58 AM To: openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue In that case, you should ask the *client* to give you a solution. Seriously, if you do not have complete control over LDAP configuration or if *client* dictates certain config parameters then it is best to report this as a bug to your application team. Do you have a test instance to do test config changes on? -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Singh, Devender (GE Capital, consultant) Sent: Wednesday, August 18, 2010 5:02 PM To: Keutel, Jochen; openldap-technical@openldap.org Subject: RE: Openldap2.4.16 performance issue As per the client requirement there is no need of substring indexing -Original Message- From: openldap-technical-boun...@openldap.org [mailto:openldap-technical-boun...@openldap.org] On Behalf Of Keutel, Jochen Sent: Thursday, August 19, 2010 5:26 AM To: openldap-technical@openldap.org Subject: Re: Openldap2.4.16 performance issue Hi, are you sure that just equality index is sufficient? Most applications do substring searches, e.g. sn=sin*. I'd recommend to add substr to the often used attributes. Regards, Jochen. Am 19.08.2010 01:11, schrieb Singh, Devender (GE Capital, consultant): Please find the answers: 1. What indexes have been created? Do they match the attributes that your applications use most often? --- All required attributes related application indexed(equality) 2. In this age of cheap RAM, 2GB RAM for a server seems puny. Latest Dell R710s come packed with 32-64GB RAM. Consider a hardware upgrade.-We can increase it by 3-4 GB only 3. Configure LB for active-active instead of active-passive?-I have configured PEN LB as a active-active(both master are active-active) 4. Turn on logging (any), run a small sample from the application and see what transactions eat up most cycles on the LDAP server.(I have configured loglevel 256, if I set loglevel -1, it will slow the read write speed from/to openldap server and also eating cpu. ) 5. Upgrade from 2.4.16 to 2.4.xx?---I don't think that up gradation will solve this issue. Here my concern is: I think I need to set below parameter in DB_CONFIG file: */set_cachesize (/*but what should be the tuned value according to my data?) Thanks Regards,// *Devender Singh* Senior Unix Administrator,// (SOA Support Team)// /___ _/// *SDG Software India Pvt. Ltd*// A-10, Sector 2, Noida 201301, U.P, India// M: +91-9910024231 O: +91.120.401.4000 F: +91.120.401.4020// website : www.sdgc.com http://www.sdgc.com/// /___ _/// *Please Note:* The e-mail content is intended for the sole use of the intended recipient/s and may contain material
Re: Openldap2.4.16 performance issue
Singh, Devender (GE Capital, consultant) devender.sin...@ge.com writes: Hi All, I need help for openldap slapd 200% cpu utilization issue. I have configured three openldap servers(2 Masters and 1 Slave). I have configure PEN load balancer for failover setup on 2 master openldap servers.It means application server first of all hit the loadbalancer port and than PEN LB will forwad that request to Master 1 or 2 openldap server. Hardware configuration on all boxes: OS: RHEL5(x86_64) RAM: 2GB Number of CPU: 2(Intel(R) Xeon(R)2.00GHz) Number of BDB databases: 2 Databse1: Number of users : 83 Number of dns: 83 Database2: Number of users: 2000 Number of dns: 80 My Application1 using Databse1 and Application2 using Databse2. Application2 just authenticating the users and store last 10 password history only, It means Application1 is not using openldap too much. In Application2(90% dependent on openldap) every user have 1000+ sub leafs entries. when I want to do major changes(number of leafs write) into this, it.s got hanged and got socket closed error in application logs and CPU utilization goes 200% and RAM 52%. My DB_CONFIG file is below: Most likely to many concurrent write operations. You should move the db logfiles onto a different disk. -Dieter -- Dieter Klünter | Systemberatung sip: 7770...@sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
