Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-05-26 Thread Charles Marcus
On 2010-05-26 1:21 PM, Mark Moseley wrote: > I've attached a slightly cropped rrd graph of NFS read bytes/sec on 6 > mail netapps from one of our datacenters. See if you can spot where we > started moving IMAP to dovecot over the course of about a month :) Wow, that's impressive... we had similar

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-05-26 Thread Timo Sirainen
On Tue, 2010-05-25 at 23:27 -0700, Tim Traver wrote: > >> Since Exim wouldn't be touching the index files, is it safe to leave > >> exim as-is and let it handle the deliveries to the maildirs natively? > >> Exim's already got access to everything it needs including the quota. > >> I just want to ma

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-05-25 Thread Tim Traver
On 3/17/2010 1:21 PM, Timo Sirainen wrote: > On Wed, 2010-03-17 at 13:19 -0700, Mark Moseley wrote: > > >> Since Exim wouldn't be touching the index files, is it safe to leave >> exim as-is and let it handle the deliveries to the maildirs natively? >> Exim's already got access to everything it n

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-05-25 Thread Mark Moseley
On Wed, Mar 17, 2010 at 3:12 PM, Mark Moseley wrote: > On Wed, Mar 17, 2010 at 1:09 PM, Timo Sirainen wrote: >> I don't know if anyone has run local indexes in larger setups, so I >> can't really give any good answers. The worst case of rebuilding the >> whole index isn't anyway any worse than w

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
On Wed, Mar 17, 2010 at 1:09 PM, Timo Sirainen wrote: > On Wed, 2010-03-17 at 12:15 -0700, Mark Moseley wrote: >> From the wiki and from the >> thread, it sounds like this just affects index files. One thing I >> didn't see in the thread (though it'd be easy to miss in a thread that >> long) is w

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Timo Sirainen
On Wed, 2010-03-17 at 13:27 -0700, Mark Moseley wrote: > I'll definitely keep that in mind. I should be able to keep things > pretty segregated in terms of POP3 alone or IMAP alone but my big > worry is that Courier POP3+Dovecot IMAP scenario. I know a large installation that was (is?) using Dov

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
On Wed, Mar 17, 2010 at 12:43 PM, Tony Rutherford wrote: > On 3/16/2010 7:36 PM, Timo Sirainen wrote: >> >> On 17.3.2010, at 1.01, Mark Moseley wrote: >> >> >>> >>> * Since Dovecot 2.0 seems like it's just around the corner, that's all >>> I've been testing, and indeed all I've even looked at. >>>

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Timo Sirainen
On Wed, 2010-03-17 at 13:19 -0700, Mark Moseley wrote: > Since Exim wouldn't be touching the index files, is it safe to leave > exim as-is and let it handle the deliveries to the maildirs natively? > Exim's already got access to everything it needs including the quota. > I just want to make sure I

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
Oops, forgot to ask one other thing On Tue, Mar 16, 2010 at 4:36 PM, Timo Sirainen wrote: > On 17.3.2010, at 1.01, Mark Moseley wrote: >> * Exim: We currently deliver all of our mail via Exim on separate >> servers. Our POP3/IMAP servers only do POP3/IMAP and the Exim mail >> servers deliver

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Timo Sirainen
On Wed, 2010-03-17 at 12:15 -0700, Mark Moseley wrote: > >> * Our #1 main motivation for looking Dovecot is relief for our > >> currently overtaxed NFS servers, mostly in the form of the index > >> files. Benchmarking dovecot looks great, even with the index files in > >> the maildir. > > > > Have

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Tony Rutherford
On 3/16/2010 7:36 PM, Timo Sirainen wrote: On 17.3.2010, at 1.01, Mark Moseley wrote: * Since Dovecot 2.0 seems like it's just around the corner, that's all I've been testing, and indeed all I've even looked at. Yes, hopefully it's coming soon :) * Our #1 main motivation for lo

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
First off, thanks for the reply. I appreciate it greatly! On Tue, Mar 16, 2010 at 4:36 PM, Timo Sirainen wrote: > On 17.3.2010, at 1.01, Mark Moseley wrote: > >> * Since Dovecot 2.0 seems like it's just around the corner, that's all >> I've been testing, and indeed all I've even looked at. > > Ye

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-16 Thread Timo Sirainen
On 17.3.2010, at 1.01, Mark Moseley wrote: > * Since Dovecot 2.0 seems like it's just around the corner, that's all > I've been testing, and indeed all I've even looked at. Yes, hopefully it's coming soon :) > * Our #1 main motivation for looking Dovecot is relief for our > currently overtaxed N

[Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-16 Thread Mark Moseley
Hello to the list! I've been asked to spec out the feasibility and, if feasible, plan a migration from Courier to Dovecot for both POP3 and IMAP for about 4 million mailboxes. I've been trying to absorb all the dovecot-related info I could over the past couple of weeks from the docs and from the li

Re: [Dovecot] Migration questions...

2009-05-21 Thread Richard Hobbs
pod wrote: > Richard Hobbs writes: > >> 19. Once everything is working perfectly, send an email to the entire >> company instructing them what to do after the outage and arrange an >> outage and do the following steps as soon as the outage begins: >> >> a. Unplug DMZ switch from firewall to

Re: [Dovecot] Migration questions...

2009-05-20 Thread pod
Richard Hobbs writes: > 19. Once everything is working perfectly, send an email to the entire > company instructing them what to do after the outage and arrange an > outage and do the following steps as soon as the outage begins: > > a. Unplug DMZ switch from firewall to make delivered mail

Re: [Dovecot] Migration questions...

2009-05-20 Thread Richard Hobbs
Hello again, people, I have been reading all of the emails sent back and forth recently, and reading many web pages, and I have a new draft of my migration plan. If anyone has the time and generosity to read through this latest plan for me and let me know if you spot any potential problems with it

Re: [Dovecot] Migration questions...

2009-05-15 Thread Giuliano Gavazzi
On F 15 May, 2009, at 12:30 , Steffen Kaiser wrote: On Fri, 15 May 2009, Giuliano Gavazzi wrote: well, using an LDA makes a little more cumbersome to check the local recipient at RCPT time. Huh? exim won't try local deliver unless it has decided it is a local recipient. You won't get ove

Re: [Dovecot] Migration questions...

2009-05-15 Thread Jakob Hirsch
Seth Mattinen, 2009-05-15 09:39: (right?) and so can dovecot, so should i use dovecot's "deliver" mechanism, or exim's own internal mechanism? Only dovecot 'deliver' will update the index on delivery. That's obvious. But dovecot (AFAIK) updates (not rebuilds) indexes when it sees new message

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Seth Mattinen wrote: > Richard Hobbs wrote: >> Seth Mattinen wrote: >>> Phillip Macey wrote: On 14/05/2009 5:11 PM, Steffen Kaiser wrote: > On Wed, 13 May 2009, Richard Hobbs wrote: >> The main complaint we have from users is that their IMAP Inbox, with >> 5000 emails in it takes a

Re: [Dovecot] Migration questions...

2009-05-15 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, May 15, 2009 at 11:28:42AM +, Richard Hobbs wrote: [...] > > Only dovecot 'deliver' will update the index on delivery. > > Do does this mean that it's slightly slower to actually deliver the mail > with dovecot (because it's writing two

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Seth Mattinen wrote: > Richard Hobbs wrote: >> Curtis Maloney wrote: >>> Phillip Macey wrote: > Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder > are a wimp. Problems start if the user: We are having some performancec issues on our server at the moment - all I c

Re: [Dovecot] Migration questions...

2009-05-15 Thread Steffen Kaiser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 15 May 2009, Giuliano Gavazzi wrote: well, using an LDA makes a little more cumbersome to check the local recipient at RCPT time. Huh? exim won't try local deliver unless it has decided it is a local recipient. You won't get overquota sta

Re: [Dovecot] Migration questions...

2009-05-15 Thread Giuliano Gavazzi
On F 15 May, 2009, at 09:39 , Seth Mattinen wrote: This raises an interesting question for me actually... given that we've now decided dovecot and maildir is the way forward for us, which delivery method should we use in exim? exim can support maildir, (right?) and so can dovecot, so should i

Re: [Dovecot] Migration questions...

2009-05-15 Thread Seth Mattinen
Richard Hobbs wrote: > Seth Mattinen wrote: >> Phillip Macey wrote: >>> On 14/05/2009 5:11 PM, Steffen Kaiser wrote: On Wed, 13 May 2009, Richard Hobbs wrote: > The main complaint we have from users is that their IMAP Inbox, with > 5000 emails in it takes ages to appear, and no amount

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Seth Mattinen wrote: > Phillip Macey wrote: >> On 14/05/2009 5:11 PM, Steffen Kaiser wrote: >>> On Wed, 13 May 2009, Richard Hobbs wrote: The main complaint we have from users is that their IMAP Inbox, with 5000 emails in it takes ages to appear, and no amount of coaxing will convinc

Re: [Dovecot] Migration questions...

2009-05-15 Thread Seth Mattinen
Richard Hobbs wrote: > Curtis Maloney wrote: >> Phillip Macey wrote: Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder are a wimp. Problems start if the user: >>> We are having some performancec issues on our server at the moment - >>> all I can put it down to is the larg

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Curtis Maloney wrote: > Phillip Macey wrote: >>> Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder >>> are a wimp. Problems start if the user: > >> We are having some performancec issues on our server at the moment - >> all I can put it down to is the large size of some maildirs.

Re: [Dovecot] Migration questions...

2009-05-14 Thread Phillip Macey
On 15/05/2009 10:33 AM, Timo Sirainen wrote: On May 14, 2009, at 8:19 PM, Phillip Macey wrote: On 15/05/2009 9:49 AM, Timo Sirainen wrote: Do you have POP3 users? What clients do your users typically use? 50% pop. 50% imap.. Roughly. With POP3 users those kinds of I/O bursts can happen wh

Re: [Dovecot] Migration questions...

2009-05-14 Thread Timo Sirainen
On May 14, 2009, at 8:19 PM, Phillip Macey wrote: On 15/05/2009 9:49 AM, Timo Sirainen wrote: On Fri, 2009-05-15 at 09:35 +1000, Phillip Macey wrote: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls -l

Re: [Dovecot] Migration questions...

2009-05-14 Thread Phillip Macey
On 15/05/2009 9:49 AM, Timo Sirainen wrote: On Fri, 2009-05-15 at 09:35 +1000, Phillip Macey wrote: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls -ld Maildir/cur` might show a directory >20Mb in size

Re: [Dovecot] Migration questions...

2009-05-14 Thread Timo Sirainen
On Fri, 2009-05-15 at 09:35 +1000, Phillip Macey wrote: > We are having some performancec issues on our server at the moment - all > I can put it down to is the large size of some maildirs. Eg. `ls -ld > Maildir/cur` might show a directory >20Mb in size for some of our users > (20-30k emails). >

Re: [Dovecot] Migration questions...

2009-05-14 Thread Seth Mattinen
Phillip Macey wrote: > On 14/05/2009 5:11 PM, Steffen Kaiser wrote: >> On Wed, 13 May 2009, Richard Hobbs wrote: >>> The main complaint we have from users is that their IMAP Inbox, with >>> 5000 emails in it takes ages to appear, and no amount of coaxing will >>> convince them to split their inbox

Re: [Dovecot] Migration questions...

2009-05-14 Thread Curtis Maloney
Phillip Macey wrote: Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder are a wimp. Problems start if the user: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls -ld Maildir/cur` might show

Re: [Dovecot] Migration questions...

2009-05-14 Thread Phillip Macey
On 14/05/2009 5:11 PM, Steffen Kaiser wrote: On Wed, 13 May 2009, Richard Hobbs wrote: The main complaint we have from users is that their IMAP Inbox, with 5000 emails in it takes ages to appear, and no amount of coaxing will convince them to split their inbox into multiple folders. Oh, we ser

Re: [Dovecot] Migration questions...

2009-05-14 Thread Stewart Dean
I'd point out that the big *practical* issue with mbox is the reality of big inboxes. While you can restrict the hoi polloi to something limited like a quota of under 60MB (and remember that inbox is one big honking file), the powers that be will not allow themselves to be so limited...nor wil

Re: [Dovecot] Migration questions...

2009-05-14 Thread Charles Marcus
On 5/14/2009, Steffen Kaiser (skdove...@smail.inf.fh-brs.de) wrote: > b) try to move all their 20'000+ messages at once to an archive > folder once a year, when their quota limit is exceeded. This can make > the whole server irresponsible slow, because I have the mail_log > plugin running as well.

Re: [Dovecot] Migration questions...

2009-05-14 Thread Seth Mattinen
Richard Hobbs wrote: > > That's also good to know... i like to do a job right instead of relying > on faster hardware, as i'm sure you all do too, but it's good to know > that if i make one or two "non-optimal" choices along the way, it'll > probably be lightning fast anyway! > > The main complai

Re: [Dovecot] Migration questions...

2009-05-14 Thread Steffen Kaiser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 13 May 2009, Richard Hobbs wrote: difference for you. A single server with recent hardware should be able to easily handle hundreds of simultaneous users. That's also good to know... i like to do a job right instead of relying on faster ha

Re: [Dovecot] Migration questions...

2009-05-13 Thread Timo Sirainen
On Wed, 2009-05-13 at 09:56 -0700, Scott Silva wrote: > > mbox - "slow" to delete mails - indexing will help this problem, but the > > filesystem will still have work to do in order to join the two halves of > > the file. > Actually, I think a new file is written with everything re-written except t

Re: [Dovecot] Migration questions...

2009-05-13 Thread Scott Silva
on 5-13-2009 8:55 AM Richard Hobbs spake the following: > Timo Sirainen wrote: >> On May 13, 2009, at 9:57 AM, Richard Hobbs wrote: >> Depends on the usage, but it's significantly better performing than UW-IMAP. Dovecot+mbox is also significantly faster than UW-IMAP+mbox. >>> OK... so Dov

Re: [Dovecot] Migration questions...

2009-05-13 Thread Richard Hobbs
Timo Sirainen wrote: > On May 13, 2009, at 9:57 AM, Richard Hobbs wrote: > >>> Depends on the usage, but it's significantly better performing than >>> UW-IMAP. Dovecot+mbox is also significantly faster than UW-IMAP+mbox. >> >> OK... so Dovecot is certainly significantly faster that uw-imapd in bot

Re: [Dovecot] Migration questions...

2009-05-13 Thread Timo Sirainen
On May 13, 2009, at 9:57 AM, Richard Hobbs wrote: Depends on the usage, but it's significantly better performing than UW-IMAP. Dovecot+mbox is also significantly faster than UW-IMAP+mbox. OK... so Dovecot is certainly significantly faster that uw-imapd in both cases, but is dovecot fastest

Re: [Dovecot] Migration questions...

2009-05-13 Thread Richard Hobbs
Hi All, Replies to everyone below (to keep the number of emails down)... Ed W wrote: > Richard Hobbs wrote: >> Hi All, >> >> We are soon to migrate our mail server from one piece of hardware >> to another and we would like to take this opportunity to optimize >> things. >> > > Can I recommend you

Re: [Dovecot] Migration questions...

2009-05-12 Thread Curtis Maloney
Richard Hobbs wrote: My colleague has mentioned something of interest... can dovecot keep the index files in RAM? If so, the performance will obviously be *so* much better than running them off the hard disks. My understanding was that in-memory indicies are discarded on logout. They're of ben

Re: [Dovecot] Migration questions...

2009-05-12 Thread Ed W
Richard Hobbs wrote: Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. Can I recommend you add virtualisation to your todo list. I use linux-vserver, but there are plenty other ideas out ther

Re: [Dovecot] Migration questions...

2009-05-12 Thread Timo Sirainen
On May 12, 2009, at 6:41 AM, Richard Hobbs wrote: Single-dbox is the highest performing, but note that it's not as much tested as mbox and Maildir code. I think it should work ok, but I'm not aware of any larger installations using dbox currently. So in case you find a problem, you might ha

Re: [Dovecot] Migration questions...

2009-05-12 Thread Lou Duchez
Blast, forgot something: the "Simple.pm" referenced in the script is this thing: http://search.cpan.org/~jpaf/Net-IMAP-Simple-0.93/Simple.pm Download it, compile it, put it somewhere that the script can find it. I don't know whether this would help with the migration, but I routinely solve a

Re: [Dovecot] Migration questions...

2009-05-12 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, May 12, 2009 at 10:09:06AM +, Richard Hobbs wrote: > Hello, [...] > That'd good to know. Do you happen to know where I can get a copy of > this "external script" you speak of? Will it simply be included in the > debian package (probably)?

Re: [Dovecot] Migration questions...

2009-05-12 Thread Lou Duchez
I don't know whether this would help with the migration, but I routinely solve a similar problem. I have implemented mail failover between two servers -- which are configured with identical sets of mailboxes -- and every 10 minutes or so, a script grabs any E-Mails from the other server and st

Re: [Dovecot] Migration questions...

2009-05-12 Thread Richard Hobbs
Sorry people - i'm replying to my own email again... my reply is below! Richard Hobbs wrote: > Hello, > > Timo Sirainen wrote: >> On Mon, 2009-05-11 at 15:05 +, Richard Hobbs wrote: >>> Hi All, >>> >>> We are soon to migrate our mail server from one piece of hardware to >>> another and we wou

Re: [Dovecot] Migration questions...

2009-05-12 Thread Richard Hobbs
Hello, Timo Sirainen wrote: > On Mon, 2009-05-11 at 15:05 +, Richard Hobbs wrote: >> Hi All, >> >> We are soon to migrate our mail server from one piece of hardware to >> another and we would like to take this opportunity to optimize things. >> As a result, we would like to replace "uw-imapd"

Re: [Dovecot] Migration questions...

2009-05-11 Thread Timo Sirainen
On Mon, 2009-05-11 at 15:05 +, Richard Hobbs wrote: > Hi All, > > We are soon to migrate our mail server from one piece of hardware to > another and we would like to take this opportunity to optimize things. > As a result, we would like to replace "uw-imapd" and "qpopper" with > "dovecot". The

Re: [Dovecot] Migration questions...

2009-05-11 Thread Richard Hobbs
Hello, Apologies for replying to my own email so soon, but I've had other thoughts as well... Our server is going to have 4 hard disks. These can be configured into 2 RAID 1 (mirror) arrays, a single RAID 5 array, or a single RAID 0+1 array. Previously, we thought that two RAID 1 arrays would be

[Dovecot] Migration questions...

2009-05-11 Thread Richard Hobbs
Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. As a result, we would like to replace "uw-imapd" and "qpopper" with "dovecot". The version we will be installing is 1.1.13-2, as this is what is avail