Re: [Dovecot] Proxy IMAP/POP/ManageSieve/SMTP in a large cluster enviroment
Aliet Santiesteban Sifontes put forth on 7/18/2010 4:22 PM: 6- Mail backend. - ( n servers)RHEL5/6 using Dovecot. Now for functional role 6 Mail Backend we have some dilemmas. - Recommended scalable filesystem to use for such scenario(Clustered or not). You have 3 choices: NFS, a cluster filesystem, or each of those with VMWare ESX. I recommend the cluster filesystem solution, specifically SGI CXFS, atop ESX (if you can afford it): http://www.sgi.com/pdfs/2816.pdf http://www.vmware.com/products/vsphere/esxi-and-esx/index.html -GFS2?? We have very bad experiences with GFS1 and maildir, GFS2 doesn't seems to improve this also. Using some techniques for session afinity with a backend server seems to help with the locking problems of GFS and the cache. Using GFS many IMAP SMTP servers can in parallel write or read to user mailboxes, if GFS can perform well we prefer this, will have to proxy o use director for the cache problem. CXFS doesn't suffer these problems associated with GFS1/2. - Ext4, Ext3 well tested with many setups but only one backend server can access one ext4 lun, so we have to proxy a user always to a ip backend address or VIP in a case of failure cluster software has to move the server to another node, the cluster service is formed by a IP Address and one or many Luns wich moves from node to node in a case of failure. Again, CXFS eliminates this problem as all clients own all LUNs all the time. Or are you talking about booting the cluster member machines from SAN LUNs, i.e. Boot from SAN, /boot and / reside on a SAN LUN? In that case, using VMWare ESX server with High Availability and Vmotion to move or restart failed cluster guests would be a much better more manageable solution as each guests' system disk would reside within a VMWare virtual disk which would physically be a single SAN LUN. You could stick 100 or 1000 virtual machine disk images on that single LUN, each of them being the boot disk for one of your mail cluster hosts. - Recommended scalable method for partition divition. - Directory Hashing(How it actually works the theory behind this), we see the wiki but we need to understand the theory to balance the directory structure. - Using some criteria ex Lun1(a-h users), Lun2(i-m users) You're talking about the mail store directory structure for the IMAP servers here right? I wouldn't do the split you describe above. It will cause more problems than it will solve. Again, you can partition and format a single SAN LUN, say 10TB, with CXFS and have all back end IMAP server hosts mount this same filesystem. You can put well over 1 million directories and files within this CXFS filesystem without seeing performance degradation due to CXFS overhead. - Proxy or Director is a must for a Clustered Filsystem or for the solution of a user belonging to one server. - In case we use a Proxy with dovecot we know we can use it with IMAP/POP but are not sure for ManageSieve and more important for SMTP. The You should try to proxy all IMAP/POP client connections to the same Dovecot server, not necessarily from a data integrity/file locking standpoint (if using CXFS locking issues are moot), but more from a performance and efficiency standpoint due to Dovecot indexing. Regarding inbound SMTP, you don't need to worry about locking here either. You will have both, or 4, or however many MX hosts connected to the SAN and all of them will run the CXFS client, Postfix and Dovecot/LDA/ManageSieve. They will all mount the same SAN LUN that the backend IMAP servers do, and Postfix/Dovecot LDA will drop the mail into the appropriate user directory after Sieve scripts are run. You will create a directory within the user mail storage LUN where you will store ManageSieve scripts. This allows for all inbound SMTP servers to read/write the directory where the ManageSieve scripts are kept. Thus, you'll run ManageSieve on all your MX hosts. It won't matter if you proxy MS operations or not because they are so infrequent and you'll not have any locking issues. But by running MS on all the MX hosts you enable redundancy. The only thing you need a director for in this case is routing requests around a downed MX host. question is, how can we proxy for the same user IMAP/POP/ManageSieve and SMTP delivery to mailbox at the same time, can the incoming mail gateways send the email to the proxy servers and dovecot proxy the request for smtp or lmtp appply the same proxy criteria that uses for the protocols IMAP and POP and send the email to the correct backend server and this server do the final delivery?? I mean proxy all the protocols for a user IMAP/POP/SMTP-LMTP?? Any example... No. See above, and, IMAP/POP requests would be proxied to the backend IMAP servers. Inbound SMTP and ManageSieve requests should be routed to the MX hosts, no need to proxy or balance them with any affinity. This is because SMTP/ManageSieve
Re: [Dovecot] Proxy IMAP/POP/ManageSieve/SMTP in a large cluster enviroment
On Sun, 2010-07-18 at 18:22 -0300, Aliet Santiesteban Sifontes wrote: - In case we use a Proxy with dovecot we know we can use it with IMAP/POP but are not sure for ManageSieve and more important for SMTP. The question is, how can we proxy for the same user IMAP/POP/ManageSieve and SMTP delivery to mailbox at the same time, can the incoming mail gateways send the email to the proxy servers and dovecot proxy the request for smtp or lmtp appply the same proxy criteria that uses for the protocols IMAP and POP and send the email to the correct backend server and this server do the final delivery?? I mean proxy all the protocols for a user IMAP/POP/SMTP-LMTP?? Any example... ManageSieve can be proxied exactly the same as IMAP/POP3. Use LMTP for mail delivery and then you don't need to worry about SMTP. So you can use Dovecot proxying for IMAP, POP3, ManageSieve and LMTP.
[Dovecot] Proxy IMAP/POP/ManageSieve/SMTP in a large cluster enviroment
Hi to all in the list, we are trying to do some tests lab for a large scale mail system having this requirements: - Scale to maybe 1million users(Only for testing). - Server side filters. - User quotas. - High concurrency. - High performance and High Availability. We plan to test this using RHEL5 and maybe RHEL6. As a storage we are going to use an HP EVA 8400 FC(8 GB/s) We defined this functional roles. 1- Outgoing SMTP gateway servers. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using Postfix Latest version Amavisd-new Clamav Spamassasin maybe some other filters. 2- Incomming SMTP gateway servers. - Load balanced with Piranha LVS using Direct Routing. - RHEL5/6 using Postfix Latest version Amavisd-new Clamav Spamassasin( n servers) maybe some other filters. 3- Webmail Cluster farm. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using RHCS Active/Active. Apache2 PHP5 Horde. 4- Openldap load balanced openldap cluster. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using Openldap. 5- IMAP/POP/ManageSieve/SMTP Proxy or Director. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using Dovecot - Postfix(Director or Proxy). 6- Mail backend. - ( n servers)RHEL5/6 using Dovecot. Now for functional role 6 Mail Backend we have some dilemmas. - Recommended scalable filesystem to use for such scenario(Clustered or not). -GFS2?? We have very bad experiences with GFS1 and maildir, GFS2 doesn't seems to improve this also. Using some techniques for session afinity with a backend server seems to help with the locking problems of GFS and the cache. Using GFS many IMAP SMTP servers can in parallel write or read to user mailboxes, if GFS can perform well we prefer this, will have to proxy o use director for the cache problem. - Ext4, Ext3 well tested with many setups but only one backend server can access one ext4 lun, so we have to proxy a user always to a ip backend address or VIP in a case of failure cluster software has to move the server to another node, the cluster service is formed by a IP Address and one or many Luns wich moves from node to node in a case of failure. - Recommended scalable method for partition divition. - Directory Hashing(How it actually works the theory behind this), we see the wiki but we need to understand the theory to balance the directory structure. - Using some criteria ex Lun1(a-h users), Lun2(i-m users) - Proxy or Director is a must for a Clustered Filsystem or for the solution of a user belonging to one server. - In case we use a Proxy with dovecot we know we can use it with IMAP/POP but are not sure for ManageSieve and more important for SMTP. The question is, how can we proxy for the same user IMAP/POP/ManageSieve and SMTP delivery to mailbox at the same time, can the incoming mail gateways send the email to the proxy servers and dovecot proxy the request for smtp or lmtp appply the same proxy criteria that uses for the protocols IMAP and POP and send the email to the correct backend server and this server do the final delivery?? I mean proxy all the protocols for a user IMAP/POP/SMTP-LMTP?? Any example... Right now we have doubts on this stuff, we are really going to appreciate any help or advice on this... best regards
Re: [Dovecot] Proxy IMAP/POP/ManageSieve/SMTP in a large cluster enviroment
Hello, just my comment on you topic: if I'd design such setup, I'd expect, that sooner or later some additional flexibility is needed, that there will be same tasks you never thought about in advance. I'm not sure, if in such case I'd rely on Postfix. Postfix might be fast, but it is by no means as flexibile as Exim is. Missing performance (of Exim, if this matters) you can buy using more servers. Missing flexibility you can't get easily. Just my two cents - and because you asked ☺ -- Heiko signature.asc Description: Digital signature
Re: [Dovecot] Proxy IMAP/POP/ManageSieve/SMTP in a large cluster enviroment
BTW, and I'm not sure, if you still need amavis, as direct scanning (using clamav or some other scanner) will be faster. Same is for Spamassassin, as long as you use it as filter and not just as some evaluator. -- Heiko signature.asc Description: Digital signature
