Re: mailboxes.db backend comparison
On May 18, 2006, at 12:11 PM, Andrew Morgan wrote: On Wed, 17 May 2006, Wesley Craig wrote: On 17 May 2006, at 14:21, Andrew Morgan wrote: My most recent test was to rebuild the mupdate master mailboxes.db from my backend server. skiplist - 20-25 minutes berkeley - 3 minutes How many mailboxes are there? Is there also a speed difference when running 'time ctl_mboxlist -mw' from your backends? -Patrick Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailboxes.db backend comparison
On Mon, 22 May 2006, Patrick Radtke wrote: On May 18, 2006, at 12:11 PM, Andrew Morgan wrote: On Wed, 17 May 2006, Wesley Craig wrote: On 17 May 2006, at 14:21, Andrew Morgan wrote: My most recent test was to rebuild the mupdate master mailboxes.db from my backend server. skiplist - 20-25 minutes berkeley - 3 minutes How many mailboxes are there? About 145000. Is there also a speed difference when running 'time ctl_mboxlist -mw' from your backends? That's what I did. :) Andy Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailboxes.db backend comparison
On May 22, 2006, at 1:55 PM, Andrew Morgan wrote: On Mon, 22 May 2006, Patrick Radtke wrote: On May 18, 2006, at 12:11 PM, Andrew Morgan wrote: On Wed, 17 May 2006, Wesley Craig wrote: On 17 May 2006, at 14:21, Andrew Morgan wrote: My most recent test was to rebuild the mupdate master mailboxes.db from my backend server. skiplist - 20-25 minutes berkeley - 3 minutes How many mailboxes are there? About 145000. Is there also a speed difference when running 'time ctl_mboxlist -mw' from your backends? That's what I did. :) haha, yup:) I should have been clearer. I want to know the diff speed, not the rebuild speed. So how long does ctl_mboxlist -mw take to run when the mupdate master is in sync with the backend. For example, with backend and murder master in sync, ctl_mboxlist takes 16 seconds to run here. If I'm rebuilding the murder master db from scratch then it takes 2+ hours. We have 782443 mailboxes, divided up across a bunch of backends. I would like the speed of berkley, but I don't trust it to be stable. -Patrick Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailboxes.db backend comparison
On Mon, 22 May 2006, Patrick Radtke wrote: haha, yup:) I should have been clearer. I want to know the diff speed, not the rebuild speed. So how long does ctl_mboxlist -mw take to run when the mupdate master is in sync with the backend. For example, with backend and murder master in sync, ctl_mboxlist takes 16 seconds to run here. If I'm rebuilding the murder master db from scratch then it takes 2+ hours. We have 782443 mailboxes, divided up across a bunch of backends. I would like the speed of berkley, but I don't trust it to be stable. For a berkeley backend on the mupdate master: [EMAIL PROTECTED]:~$ time /usr/local/cyrus/bin/ctl_mboxlist -mw real0m5.268s user0m1.404s sys 0m2.556s For a skiplist backend: [EMAIL PROTECTED]:~$ time /usr/local/cyrus/bin/ctl_mboxlist -mw real0m4.575s user0m1.384s sys 0m2.576s Still, since skiplist takes so long to populate initially, a failure of the mupdate master can have a pretty large effect on service. Andy Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailboxes.db backend comparison
I have a patch set to ctl_mboxlist that outputs mailboxes.db on the backend in a format that may be concatenated and reloaded on the mupdate master. Using this method, we can rebuild our mupdate master from the collective backends in a matter of minutes, even with 800K mailboxes. :wes On 22 May 2006, at 18:28, Andrew Morgan wrote: Still, since skiplist takes so long to populate initially, a failure of the mupdate master can have a pretty large effect on service. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailboxes.db backend comparison
On Wed, 17 May 2006, Wesley Craig wrote: On 17 May 2006, at 14:21, Andrew Morgan wrote: My most recent test was to rebuild the mupdate master mailboxes.db from my backend server. skiplist - 20-25 minutes berkeley - 3 minutes How long does it take to LIST the contents of each database? I don't know of a way to test this on the mupdate master, so instead I converted one of my frontends from skiplist to berkeley. A simple lm command in cyradm took approximately 15 seconds for both formats. Andy Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailboxes.db backend comparison
On 18 May 2006, at 12:11, Andrew Morgan wrote: On Wed, 17 May 2006, Wesley Craig wrote: On 17 May 2006, at 14:21, Andrew Morgan wrote: My most recent test was to rebuild the mupdate master mailboxes.db from my backend server. skiplist - 20-25 minutes berkeley - 3 minutes How long does it take to LIST the contents of each database? I don't know of a way to test this on the mupdate master, so instead I converted one of my frontends from skiplist to berkeley. A simple lm command in cyradm took approximately 15 seconds for both formats. Any difference in the resources (CPU, disk IO) consumed? Perhaps do lm 10-20 times to get enough data to demonstrate a difference. :wes Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
mailboxes.db backend comparison
I've been doing some more testing of Cyrus Murder operation. My most recent test was to rebuild the mupdate master mailboxes.db from my backend server. Both the backend and the mupdate master were configured for a skiplist mailboxes.db format. The size of the mailboxes.db is 146382 mailboxes. Then, I configured the mupdate master to use berkeley for mailboxes.db. Here are the times it took to do the full sync from the backend to the mupdate master: skiplist - 20-25 minutes berkeley - 3 minutes I know many people have decided to use skiplist for all their Cyrus databases, but this seems like a clear advantage for Berkeley DB. Any thoughts? Andy Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailboxes.db backend comparison
On 17 May 2006, at 14:21, Andrew Morgan wrote: My most recent test was to rebuild the mupdate master mailboxes.db from my backend server. skiplist - 20-25 minutes berkeley - 3 minutes How long does it take to LIST the contents of each database? :wes Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
