Re: How to improve performance with MDB backend?

2013-04-21 Thread Saša-Stjepan Bakša
Ok. tnx for your suggestion. br Sasa . On Sat, Apr 20, 2013 at 10:16 AM, devzero2000 pinto.e...@gmail.com wrote: Just for Info, you should take a look to the tuned daemon and setting a consistent profile for your workload. It can help a lot. Best 2013/4/19, Saša-Stjepan Bakša

Re: How to improve performance with MDB backend?

2013-04-20 Thread devzero2000
Just for Info, you should take a look to the tuned daemon and setting a consistent profile for your workload. It can help a lot. Best 2013/4/19, Saša-Stjepan Bakša ssba...@gmail.com: Ok. I see your point and lesson is learned. Will do as suggested. On Fri, Apr 19, 2013 at 6:07 PM, Howard

Re: How to improve performance with MDB backend?

2013-04-19 Thread Howard Chu
Quanah Gibson-Mount wrote: --On Thursday, April 18, 2013 3:59 PM + Chris Card ctc...@hotmail.com wrote: Maybe MDB performance relative to BDB degrades as the database get bigger. From your wiki page: This particular client has 25,208 entries in their LDAP database. My test database has

Re: How to improve performance with MDB backend?

2013-04-19 Thread Howard Chu
Quanah Gibson-Mount wrote: --On Thursday, April 18, 2013 8:18 PM + Chris Card ctc...@hotmail.com wrote: I'll give that a try. I had perhaps been taking the advice given in the slapd-mdb man page too seriously: Heh, yes, it seems so. ;) With writemap, you definitely do not want to exceed

RE: How to improve performance with MDB backend?

2013-04-19 Thread Chris Card
Date: Fri, 19 Apr 2013 00:03:30 -0700 From: h...@symas.com To: qua...@zimbra.com; ctc...@hotmail.com; openldap-technical@openldap.org Subject: Re: How to improve performance with MDB backend? Quanah Gibson-Mount wrote: --On Thursday, April 18, 2013

Re: How to improve performance with MDB backend?

2013-04-19 Thread Nick Milas
On 19/4/2013 2:00 μμ, Chris Card wrote: I tried reducing the maxsize, but it made no difference to the performance. So I'm still at the point where writes to BDB are roughly 4 times faster than writes to MDB. Any more suggestions? Could it be possibly related to the OS / filesystem used?

Re: How to improve performance with MDB backend?

2013-04-19 Thread Saša-Stjepan Bakša
First test with your sugestions. I am using Phyton program writen by me to add data to server. Server is Centos 6.2 based (hardware described in my first post) Python runs on separeate dual core PC with 1Gb connection to servers. Servers are configured as N-way Multymaster Test startTest stop

Re: How to improve performance with MDB backend?

2013-04-19 Thread Nick Milas
On 18/4/2013 6:16 μμ, Quanah Gibson-Mount wrote: For me, MDB writes are a minimum of 65 times faster than writes with BDB/HDB, even when BDB/HDB use an SHM key. Can you please share your compilation options (or spec file, if applicable) and test setup complete configuration so interested

Re: How to improve performance with MDB backend?

2013-04-19 Thread Saša-Stjepan Bakša
For second test I have changed I/O scheduler only Nothing changed in terms of adding speed! Test startTest stop Test durationNum usersUser/sec 19.4.2013 19.4.2013min 5759,00742837128,99 14:37:45 16:13:34 sec 95,98333 [root@spr1 /]# cat

Re: How to improve performance with MDB backend?

2013-04-19 Thread Howard Chu
Saša-Stjepan Bakša wrote: First test with your sugestions. I am using Phyton program writen by me to add data to server. Server is Centos 6.2 based (hardware described in my first post) Python runs on separeate dual core PC with 1Gb connection to servers. Servers are configured as N-way

Re: How to improve performance with MDB backend?

2013-04-19 Thread Howard Chu
Saša-Stjepan Bakša wrote: How do you measure or compare speed? I have a Python program which add a predefined number of users (1mil and each user consists of 8 DNs and 3 alias DNs) and for Modify test I have also Python program which modify 1 attribute with random value under one DN changed

Re: How to improve performance with MDB backend?

2013-04-19 Thread Saša-Stjepan Bakša
Ok. I see your point and lesson is learned. Will do as suggested. On Fri, Apr 19, 2013 at 6:07 PM, Howard Chu h...@symas.com wrote: You really need to learn something more about system administration; you clearly don't know what to investigate but this is all fundamental sysadmin

RE: How to improve performance with MDB backend?

2013-04-18 Thread Chris Card
I am also interested in this thread. I've been doing some comparison tests between MDB and BDB using openldap 2.4.35, and find that the BDB backend is consistently much faster for writes. For example, modifying 1 DNs and changing one attribute using ldapmodify takes about 80 seconds with an

RE: How to improve performance with MDB backend?

2013-04-18 Thread Quanah Gibson-Mount
--On Thursday, April 18, 2013 2:27 PM + Chris Card ctc...@hotmail.com wrote: I am also interested in this thread. I've been doing some comparison tests between MDB and BDB using openldap 2.4.35, and find that the BDB backend is consistently much faster for writes. For example, modifying

RE: How to improve performance with MDB backend?

2013-04-18 Thread Chris Card
Date: Thu, 18 Apr 2013 08:16:03 -0700 From: qua...@zimbra.com To: ctc...@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend? --On Thursday, April 18, 2013 2:27 PM + Chris Card ctc...@hotmail.com

RE: How to improve performance with MDB backend?

2013-04-18 Thread Quanah Gibson-Mount
--On Thursday, April 18, 2013 3:29 PM + Chris Card ctc...@hotmail.com wrote: Did you make the changes to the DB flags for MDB that I noted the other day? For me, MDB writes are a minimum of 65 times faster than writes with BDB/HDB, even when BDB/HDB use an SHM key. Yes. I have these

RE: How to improve performance with MDB backend?

2013-04-18 Thread Chris Card
Date: Thu, 18 Apr 2013 08:50:23 -0700 From: qua...@zimbra.com To: ctc...@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend? --On Thursday, April 18, 2013 3:29 PM + Chris Card ctc...@hotmail.com

RE: How to improve performance with MDB backend?

2013-04-18 Thread Quanah Gibson-Mount
--On Thursday, April 18, 2013 3:59 PM + Chris Card ctc...@hotmail.com wrote: Maybe MDB performance relative to BDB degrades as the database get bigger.  From your wiki page: This particular client has 25,208 entries in their LDAP database. My test database has over 3 million entries

RE: How to improve performance with MDB backend?

2013-04-18 Thread Chris Card
Date: Thu, 18 Apr 2013 09:13:42 -0700 From: qua...@zimbra.com To: ctc...@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend? --On Thursday, April 18, 2013 3:59 PM + Chris Card ctc...@hotmail.com

RE: How to improve performance with MDB backend?

2013-04-18 Thread Quanah Gibson-Mount
--On Thursday, April 18, 2013 4:57 PM + Chris Card ctc...@hotmail.com wrote: I'm using openldap 2.4.35. olcDbMaxSize: 429496729600 and that's the size of the file on the disk too: -rw--- 1 ldap ldap 429496729600 Apr 18 16:52 data.mdb -rw--- 1 ldap ldap         8192 Apr 18 17:52

RE: How to improve performance with MDB backend?

2013-04-18 Thread Chris Card
Date: Thu, 18 Apr 2013 10:13:10 -0700 From: qua...@zimbra.com To: ctc...@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend? --On Thursday, April 18, 2013 4:57 PM + Chris Card ctc...@hotmail.com

RE: How to improve performance with MDB backend?

2013-04-18 Thread Quanah Gibson-Mount
--On Thursday, April 18, 2013 5:21 PM + Chris Card ctc...@hotmail.com wrote: I routinely test with DBs in the range of 3 million to 6 million entries, and it is always substantially faster than bdb/hdb. du -c -h data.mdb returns 16G So drop your maxsize to 32GB from 400GB. Clearly you

RE: How to improve performance with MDB backend?

2013-04-18 Thread Chris Card
Date: Thu, 18 Apr 2013 10:41:40 -0700 From: qua...@zimbra.com To: ctc...@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend? --On Thursday, April 18, 2013 5:21 PM + Chris Card ctc...@hotmail.com wrote: I routinely test with DBs

RE: How to improve performance with MDB backend?

2013-04-18 Thread Quanah Gibson-Mount
--On Thursday, April 18, 2013 8:18 PM + Chris Card ctc...@hotmail.com wrote: I'll give that a try. I had perhaps been taking the advice given in the slapd-mdb man page too seriously: Heh, yes, it seems so. ;) With writemap, you definitely do not want to exceed actual RAM. --Quanah

Re: How to improve performance with MDB backend?

2013-04-16 Thread Saša-Stjepan Bakša
Well, hardware is described in my first post. Now I am waiting for HP blades and SSD array :-) so if you need/want some tests to be done it can be arranged (in idle time). Br Sasa On Mon, Apr 15, 2013 at 7:09 PM, Michael Ströder mich...@stroeder.comwrote: Saša-Stjepan Bakša wrote: Now I

Re: How to improve performance with MDB backend?

2013-04-15 Thread Saša-Stjepan Bakša
How do you measure or compare speed? I have a Python program which add a predefined number of users (1mil and each user consists of 8 DNs and 3 alias DNs) and for Modify test I have also Python program which modify 1 attribute with random value under one DN changed sequentially from 1 to 1mil)

Re: How to improve performance with MDB backend?

2013-04-09 Thread Quanah Gibson-Mount
--On Tuesday, April 09, 2013 2:07 PM +0200 Saša-Stjepan Bakša ssba...@gmail.com wrote: Tnx. I will add what you suggest and try to run modify test with that. I didn't done that already because my mdb database is overflowed. I have set it to 100GB but it wasn't enough. I am not sure why. Are

Re: How to improve performance with MDB backend?

2013-04-09 Thread Saša-Stjepan Bakša
I am sorry replaying directly. I didn't check who was in TO field. OK I will try Re24. Tnx again. Sasa On Tue, Apr 9, 2013 at 4:15 PM, Quanah Gibson-Mount qua...@zimbra.comwrote: --On Tuesday, April 09, 2013 2:07 PM +0200 Saša-Stjepan Bakša ssba...@gmail.com wrote: Tnx. I will add what