Re: [Dovecot] best practises for mail systems
On Tue, Jun 05, 2012 at 11:59:47PM +1100, Костырев Александр Алексеевич wrote: > > I'm more worried about right design of mailstorage.. should I use some > cluster fs with all mail of all users > or should I split mailstorage across servers and somehow avoid long downtime > if one of servers goes down. A clusterfs gives you active/active high availability and balanced distribution of users over your servers, at the cost of somewhat degraded I/O performance all the time. If a single node will be able to serve your load, I think it's much more sensible to create a passive/standby availability solution based on a local filesystem (XFS). If you need to split your mailstorage across servers, you can do active/standby server pairs -- but then it gets difficult to balance your users over your servers, and you *might* want to cheat and use a clusterfs instead.. -jf
Re: [Dovecot] best practises for mail systems
We once try to use similar solution as your first. 3 servers for LVS -HA This master server redirect users for 2 or 3 dovecot backends.. The mail storage were maildir ontop of OCFS2 Our problem were that OCFS2 were too slow. We could not handle many users. So we took an step back and now use only user one server. But still thinking in go back to the first one. with LVS When using LVS try to sticky user to the same backend, LVs can do ths by source ip. Where i work we have problens on testign storage. If you have any advices for testing disk performance, i will be thankfull. I wil be glad to answer anything else. []'sf.rique On Tue, Jun 5, 2012 at 9:59 AM, Костырев Александр Алексеевич < a.kosty...@serverc.ru> wrote: > I think LVS is just fine and it is not a SPOF 'cause it is actually 2 > servers: > active master --> and standby slave. > LVS supports real time replication of connections from master to slave, > so if master dies slave knows which IP was connected to which dovecot > server. > > I'm more worried about right design of mailstorage.. should I use some > cluster fs with all mail of all users > or should I split mailstorage across servers and somehow avoid long > downtime if one of servers goes down. > > > -Original Message- > From: dovecot-boun...@dovecot.org [mailto:dovecot-boun...@dovecot.org] On > Behalf Of Matthias-Christian Ott > Sent: Tuesday, June 05, 2012 11:28 PM > To: dovecot@dovecot.org > Subject: Re: [Dovecot] best practises for mail systems > > On 2012-06-05 05:14, Костырев Александр Алексеевич wrote: > > On each host system we created one VM and passed through 3x2TB disks > into it. > > > > > > > > In guests vms on top of this disks we made XFS and fired up glusterfs > with distributed replicated volumes for our mailstorage. > > > > so it looks like this: > > > > > > > > vm1replicate vm2 > > > > disk1 > disk4 > > > > disk2 > disk5 > > > > disk3 > disk6 > > > > > > > > in each vm we mounted glusterfs and pointed dovecot to that dir for mail > creation (as ltmp) and imap4 user access. > > > > also we use exim as smtp. > > > > > > > > So, with glusterfs as mailstorage we can go for LVS to load balancing > for exim and dovecot. > > > > so wherenever one of host systems (hence one of mail vms) goes down, > users don't notice that > > > > 'cause LVS points them to working smtp and imap4 servers > > > > and they get their mail 'cause of glusterfs. > > [...] > > Cons: > > > > - not quite sure if glusterfs is production ready solution 'cause I've > experienced split-brains during setting it up > > > > - IO performance issue. Though we didn't yet run any io tests, but > glusterfs uses fuse to mount on clients. And guys on #gluster told me > writing to the glusterfs mount will not be strictly local io. > > I'm not familiar with LVS, but from the project description it seems > that you need a "front server" that does the load balancing, so you > either have to run at least two of these servers in parallel or add to > your cons that you introduced a single point of failure. But you > mentioned that you only have two servers, so you really can do this. > > I would rather ensure high availability by running the two servers as > masters and using either IP address takeover or DNS failover (with > dynamic DNS) and either use Dovecot's replication (I haven't tested it > yet and I'm not sure what happens in case of IP address takeover) or a > file system that can handle these kinds of errors (e.g. Coda). You could > do load balancing via round-robin DNS. This only protects you against > the failure of single machine and because IMAP sessions are not > replicated between the servers, connections will get reset if one server > fails, but it's cost-effective and uses software that already exists. > > Regards, > Matthias-Christian >
Re: [Dovecot] best practises for mail systems
I think LVS is just fine and it is not a SPOF 'cause it is actually 2 servers: active master --> and standby slave. LVS supports real time replication of connections from master to slave, so if master dies slave knows which IP was connected to which dovecot server. I'm more worried about right design of mailstorage.. should I use some cluster fs with all mail of all users or should I split mailstorage across servers and somehow avoid long downtime if one of servers goes down. -Original Message- From: dovecot-boun...@dovecot.org [mailto:dovecot-boun...@dovecot.org] On Behalf Of Matthias-Christian Ott Sent: Tuesday, June 05, 2012 11:28 PM To: dovecot@dovecot.org Subject: Re: [Dovecot] best practises for mail systems On 2012-06-05 05:14, Костырев Александр Алексеевич wrote: > On each host system we created one VM and passed through 3x2TB disks into it. > > > > In guests vms on top of this disks we made XFS and fired up glusterfs with > distributed replicated volumes for our mailstorage. > > so it looks like this: > > > > vm1replicate vm2 > > disk1 > disk4 > > disk2 > disk5 > > disk3 > disk6 > > > > in each vm we mounted glusterfs and pointed dovecot to that dir for mail > creation (as ltmp) and imap4 user access. > > also we use exim as smtp. > > > > So, with glusterfs as mailstorage we can go for LVS to load balancing for > exim and dovecot. > > so wherenever one of host systems (hence one of mail vms) goes down, users > don't notice that > > 'cause LVS points them to working smtp and imap4 servers > > and they get their mail 'cause of glusterfs. > [...] > Cons: > > - not quite sure if glusterfs is production ready solution 'cause I've > experienced split-brains during setting it up > > - IO performance issue. Though we didn't yet run any io tests, but glusterfs > uses fuse to mount on clients. And guys on #gluster told me writing to the > glusterfs mount will not be strictly local io. I'm not familiar with LVS, but from the project description it seems that you need a "front server" that does the load balancing, so you either have to run at least two of these servers in parallel or add to your cons that you introduced a single point of failure. But you mentioned that you only have two servers, so you really can do this. I would rather ensure high availability by running the two servers as masters and using either IP address takeover or DNS failover (with dynamic DNS) and either use Dovecot's replication (I haven't tested it yet and I'm not sure what happens in case of IP address takeover) or a file system that can handle these kinds of errors (e.g. Coda). You could do load balancing via round-robin DNS. This only protects you against the failure of single machine and because IMAP sessions are not replicated between the servers, connections will get reset if one server fails, but it's cost-effective and uses software that already exists. Regards, Matthias-Christian
Re: [Dovecot] best practises for mail systems
On 2012-06-05 05:14, Костырев Александр Алексеевич wrote: > On each host system we created one VM and passed through 3x2TB disks into it. > > > > In guests vms on top of this disks we made XFS and fired up glusterfs with > distributed replicated volumes for our mailstorage. > > so it looks like this: > > > > vm1replicate vm2 > > disk1 > disk4 > > disk2 > disk5 > > disk3 > disk6 > > > > in each vm we mounted glusterfs and pointed dovecot to that dir for mail > creation (as ltmp) and imap4 user access. > > also we use exim as smtp. > > > > So, with glusterfs as mailstorage we can go for LVS to load balancing for > exim and dovecot. > > so wherenever one of host systems (hence one of mail vms) goes down, users > don't notice that > > 'cause LVS points them to working smtp and imap4 servers > > and they get their mail 'cause of glusterfs. > [...] > Cons: > > - not quite sure if glusterfs is production ready solution 'cause I've > experienced split-brains during setting it up > > - IO performance issue. Though we didn't yet run any io tests, but glusterfs > uses fuse to mount on clients. And guys on #gluster told me writing to the > glusterfs mount will not be strictly local io. I'm not familiar with LVS, but from the project description it seems that you need a "front server" that does the load balancing, so you either have to run at least two of these servers in parallel or add to your cons that you introduced a single point of failure. But you mentioned that you only have two servers, so you really can do this. I would rather ensure high availability by running the two servers as masters and using either IP address takeover or DNS failover (with dynamic DNS) and either use Dovecot's replication (I haven't tested it yet and I'm not sure what happens in case of IP address takeover) or a file system that can handle these kinds of errors (e.g. Coda). You could do load balancing via round-robin DNS. This only protects you against the failure of single machine and because IMAP sessions are not replicated between the servers, connections will get reset if one server fails, but it's cost-effective and uses software that already exists. Regards, Matthias-Christian
Re: [Dovecot] best practises for mail systems
On 5.6.2012, at 6.14, Костырев Александр Алексеевич wrote: > - not quite sure if glusterfs is production ready solution 'cause I've > experienced split-brains during setting it up Last I've heard glusterfs causes corruption problems with Dovecot. You should try stress testing it with imaptest: http://imapwiki.org/ImapTest
[Dovecot] best practises for mail systems
hello! Can someone point me to some best practices in building high-available scalable mail system or! share your own success stories. I've read article in LJ "Building a Scalable High-Availability E-Mail System with Active Directory and More" but it seemed to be outdated and there's a single point of failure (Master node). What I want to achieve: high-available, horizontaly scalable, with no single point of failure mail solution. Available hardware: intel mfsys25 modular server with 2 storage controllers, 2 switches, 4 power supply blocks with - 2 blade-servers in mfsys with: 2xIntel Xeon E5620 @ 2.40GHz with 8 cores each - promise vtrak e610s (2 storage controllers, 2 power supply blocks) - 6x 2TB SATA Hitachi HDS72302 We decided to go for KVM virtualization and glusterfs for live migration for vm image but that's not what this is all about :) We installed centos on host systems. for now while we could think of two ways to go: The first way (currently at testing stage): On each host system we created one VM and passed through 3x2TB disks into it. In guests vms on top of this disks we made XFS and fired up glusterfs with distributed replicated volumes for our mailstorage. so it looks like this: vm1 replicate vm2 disk1 > disk4 disk2 > disk5 disk3 > disk6 in each vm we mounted glusterfs and pointed dovecot to that dir for mail creation (as ltmp) and imap4 user access. also we use exim as smtp. So, with glusterfs as mailstorage we can go for LVS to load balancing for exim and dovecot. so wherenever one of host systems (hence one of mail vms) goes down, users don't notice that 'cause LVS points them to working smtp and imap4 servers and they get their mail 'cause of glusterfs. Pros: - high-available - horizontaly scalable - with no single point of failure Cons: - not quite sure if glusterfs is production ready solution 'cause I've experienced split-brains during setting it up - IO performance issue. Though we didn't yet run any io tests, but glusterfs uses fuse to mount on clients. And guys on #gluster told me writing to the glusterfs mount will not be strictly local io. The second way: split up the users mail with: two back-end VMs each other on DIFFERENT host system with - fat mailstorage with raid1+linear mode (mdadm)+XFS - dovecot/exim-back-ends and two VMs for nginx-based proxy servers for imap4 and smtp - nginx can redirect user to right back-end through HTTP-php-based logic. Pros: - we split up not only load for exim/dovecot but users mail IOs too - no split-brains Cons: - If one of the host systems (hence one of back-end VMs with storage) goes down, half of our users is unhappy P.S. Sorry if this place is way wrong to ask for such things.
