Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-14 Thread David Boreham
On 9/14/2013 12:44 PM, Howard Chu wrote: Ah totally forgot about that, it's been a couple years since I looked inside that code. LMDB updates record counts on every write op so returning the count is zero-cost. (Ironically we don't use this fact to optimize filter evaluation order in OpenLDAP.

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-14 Thread Howard Chu
Rich Megginson wrote: On 09/12/2013 07:08 PM, David Boreham wrote: > On 9/11/2013 11:41 AM, Howard Chu wrote: >> >> Just out of curiosity, why is keeping a count per key a problem? If >> you're using BDB duplicate key support, can't you just use >> cursor->c_count() to get this? I.e., BDB alrea

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-13 Thread Rich Megginson
On 09/13/2013 02:39 PM, David Boreham wrote: On 9/13/2013 2:18 PM, Rich Megginson wrote: On 09/12/2013 07:08 PM, David Boreham wrote: On 9/11/2013 11:41 AM, Howard Chu wrote: Just out of curiosity, why is keeping a count per key a problem? If you're using BDB duplicate key support, can't you

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-13 Thread David Boreham
On 9/13/2013 2:18 PM, Rich Megginson wrote: On 09/12/2013 07:08 PM, David Boreham wrote: On 9/11/2013 11:41 AM, Howard Chu wrote: Just out of curiosity, why is keeping a count per key a problem? If you're using BDB duplicate key support, can't you just use cursor->c_count() to get this? I.e.

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-13 Thread Rich Megginson
On 09/12/2013 07:08 PM, David Boreham wrote: On 9/11/2013 11:41 AM, Howard Chu wrote: Just out of curiosity, why is keeping a count per key a problem? If you're using BDB duplicate key support, can't you just use cursor->c_count() to get this? I.e., BDB already maintains key counts internall

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-12 Thread David Boreham
On 9/11/2013 11:41 AM, Howard Chu wrote: Just out of curiosity, why is keeping a count per key a problem? If you're using BDB duplicate key support, can't you just use cursor->c_count() to get this? I.e., BDB already maintains key counts internally, why not leverage that? afaik you need t

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-12 Thread David Boreham
On 9/9/2013 11:19 AM, Rich Megginson wrote: Thanks everyone for the comments. I have added Noriko's suggestion: http://port389.org/wiki/Design/Fine_Grained_ID_List_Size David, Ludwig: Does the current design address your concerns, and/or provide the necessary first step for further refinements

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-12 Thread Ludwig Krispenz
On 09/12/2013 04:40 PM, Rich Megginson wrote: On 09/12/2013 07:39 AM, thierry bordaz wrote: On 09/10/2013 04:35 PM, Ludwig Krispenz wrote: On 09/10/2013 04:29 PM, Rich Megginson wrote: On 09/10/2013 01:47 AM, Ludwig Krispenz wrote: On 09/09/2013 07:19 PM, Rich Megginson wrote: On 09/09/20

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-12 Thread Rich Megginson
On 09/12/2013 07:39 AM, thierry bordaz wrote: On 09/10/2013 04:35 PM, Ludwig Krispenz wrote: On 09/10/2013 04:29 PM, Rich Megginson wrote: On 09/10/2013 01:47 AM, Ludwig Krispenz wrote: On 09/09/2013 07:19 PM, Rich Megginson wrote: On 09/09/2013 02:27 AM, Ludwig Krispenz wrote: On 09/07/2

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-12 Thread thierry bordaz
On 09/10/2013 04:35 PM, Ludwig Krispenz wrote: On 09/10/2013 04:29 PM, Rich Megginson wrote: On 09/10/2013 01:47 AM, Ludwig Krispenz wrote: On 09/09/2013 07:19 PM, Rich Megginson wrote: On 09/09/2013 02:27 AM, Ludwig Krispenz wrote: On 09/07/2013 05:02 AM, David Boreham wrote: On 9/6/2013

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-10 Thread Ludwig Krispenz
On 09/10/2013 04:29 PM, Rich Megginson wrote: On 09/10/2013 01:47 AM, Ludwig Krispenz wrote: On 09/09/2013 07:19 PM, Rich Megginson wrote: On 09/09/2013 02:27 AM, Ludwig Krispenz wrote: On 09/07/2013 05:02 AM, David Boreham wrote: On 9/6/2013 8:49 PM, Nathan Kinder wrote: This is a good id

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-10 Thread Rich Megginson
On 09/10/2013 01:47 AM, Ludwig Krispenz wrote: On 09/09/2013 07:19 PM, Rich Megginson wrote: On 09/09/2013 02:27 AM, Ludwig Krispenz wrote: On 09/07/2013 05:02 AM, David Boreham wrote: On 9/6/2013 8:49 PM, Nathan Kinder wrote: This is a good idea, and it is something that we discussed briefl

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-10 Thread Ludwig Krispenz
On 09/09/2013 07:19 PM, Rich Megginson wrote: On 09/09/2013 02:27 AM, Ludwig Krispenz wrote: On 09/07/2013 05:02 AM, David Boreham wrote: On 9/6/2013 8:49 PM, Nathan Kinder wrote: This is a good idea, and it is something that we discussed briefly off-list. The only downside is that we need

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-09 Thread Ludwig Krispenz
On 09/07/2013 05:02 AM, David Boreham wrote: On 9/6/2013 8:49 PM, Nathan Kinder wrote: This is a good idea, and it is something that we discussed briefly off-list. The only downside is that we need to change the index format to keep a count of ids for each key. Implementing this isn't a big

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-06 Thread David Boreham
On 9/6/2013 8:49 PM, Nathan Kinder wrote: This is a good idea, and it is something that we discussed briefly off-list. The only downside is that we need to change the index format to keep a count of ids for each key. Implementing this isn't a big problem, but it does mean that the existing in

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-06 Thread Nathan Kinder
On 09/06/2013 05:30 PM, David Boreham wrote: On 9/6/2013 3:05 PM, Rich Megginson wrote: Please review and comment: http://port389.org/wiki/Design/Fine_Grained_ID_List_Size This looks interesting. I suppose this is similar to a SQL database's concept of index statistics, and also query hints

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-06 Thread David Boreham
On 9/6/2013 3:05 PM, Rich Megginson wrote: Please review and comment: http://port389.org/wiki/Design/Fine_Grained_ID_List_Size This looks interesting. I suppose this is similar to a SQL database's concept of index statistics, and also query hints supplied by the client. Perhaps more of a "se

Re: [389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-06 Thread Noriko Hosoi
Rich Megginson wrote: Please review and comment: http://port389.org/wiki/Design/Fine_Grained_ID_List_Size -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel Hi Rich, A nice design! It looks promising to solve the sticky prob

[389-devel] RFC: New Design: Fine Grained ID List Size

2013-09-06 Thread Rich Megginson
Please review and comment: http://port389.org/wiki/Design/Fine_Grained_ID_List_Size -- 389-devel mailing list 389-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-devel