Re: [Dovecot] Hints for a NFS-Setup
On 18.4.2012, at 1.29, Kelsey Cummings wrote: On 03/30/12 07:35, Timo Sirainen wrote: IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. Timo, having the LDA flow through the director with maildir and NFS is only an issue if you want to use the dovecot delivery agent since it manipulates the indexes on delivery, right? This shouldn't be a problem if you are running a completely distinct delivery cluster running a LDA like procmail. Right. If indexes or dovecot-uidlist isn't updated then it doesn't matter.
Re: [Dovecot] Hints for a NFS-Setup
On Wed, Apr 04, 2012 at 02:02:02PM +0200, Patrick Westenberg wrote: Timo Sirainen schrieb: Another director. They're meant to connect to each others and do LB/HA. But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down? We use hardware loadbalancers in front of the directors. If a director goes down, the hardware loadbalancers sees that. We also use the poolmon script to tell the directors if a certain imap server is missing. Cor
Re: [Dovecot] Hints for a NFS-Setup
Timo Sirainen schrieb: On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: MTA --(lmtp)--\ /--(lmtp)-- backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)-- backend2 --/ IMAP-User -- frontend1 --\ /--(imap)-- backend1 --\ -- director -- -- NFS IMAP-User -- frontend2 --/ \--(imap)-- backend2 --/ The director above must be the same one for both IMAP and MTA. I know and it is meant to be the same in this drawing. However, in this scenario the director is a single point of failure. What would be the best way to get loadbalancing and high-availability? Patrick
Re: [Dovecot] Hints for a NFS-Setup
On 4.4.2012, at 14.38, Patrick Westenberg wrote: Timo Sirainen schrieb: On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: MTA --(lmtp)--\ /--(lmtp)-- backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)-- backend2 --/ IMAP-User -- frontend1 --\ /--(imap)-- backend1 --\ -- director -- -- NFS IMAP-User -- frontend2 --/ \--(imap)-- backend2 --/ The director above must be the same one for both IMAP and MTA. I know and it is meant to be the same in this drawing. However, in this scenario the director is a single point of failure. What would be the best way to get loadbalancing and high-availability? Another director. They're meant to connect to each others and do LB/HA.
Re: [Dovecot] Hints for a NFS-Setup
Timo Sirainen schrieb: Another director. They're meant to connect to each others and do LB/HA. But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down? Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director.
Re: [Dovecot] Hints for a NFS-Setup
On 4.4.2012, at 15.02, Patrick Westenberg wrote: Timo Sirainen schrieb: Another director. They're meant to connect to each others and do LB/HA. But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down? Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. I don't know if Postfix supports that. Typically people use a load balancer (cluster).
Re: [Dovecot] Hints for a NFS-Setup
On Wed, 4 Apr 2012 15:06:29 +0300 Timo Sirainen articulated: On 4.4.2012, at 15.02, Patrick Westenberg wrote: Timo Sirainen schrieb: Another director. They're meant to connect to each others and do LB/HA. But what about my MTAs? How can I tell my two postfix servers that there are two directors and it should/can use the other one if the first is down? Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. I don't know if Postfix supports that. Typically people use a load balancer (cluster). Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action. -- Jerry ♔ Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __
Re: [Dovecot] Hints for a NFS-Setup
On 4.4.2012, at 15.29, Jerry wrote: Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. I don't know if Postfix supports that. Typically people use a load balancer (cluster). Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action. I have a guess though, or alternatively a suggestion if it already doesn't work like that: lmtp:tcp:lmtp.example.com:24 where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places.
Re: [Dovecot] Hints for a NFS-Setup
Quoting Timo Sirainen t...@iki.fi: On 4.4.2012, at 15.29, Jerry wrote: Now I use relay_transport = lmtp:unix:private/dovecot-lmtp. Even if I change this to lmtp:tcp:1.2.3.4:24 it is still only one director. I don't know if Postfix supports that. Typically people use a load balancer (cluster). Perhaps posting on the Postfix forum and asking Wietse Venema would be the logical course of action. I have a guess though, or alternatively a suggestion if it already doesn't work like that: lmtp:tcp:lmtp.example.com:24 where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. Personally I use MX dns entries, lowest mx entry is localhost, then others.
Re: [Dovecot] Hints for a NFS-Setup
Timo Sirainen schrieb: where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn.
Re: [Dovecot] Hints for a NFS-Setup
On 4.4.2012, at 16.04, Patrick Westenberg wrote: Timo Sirainen schrieb: where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. MX lookups are intended for SMTP, I very much doubt LMTP uses them?
Re: [Dovecot] Hints for a NFS-Setup
Quoting Timo Sirainen t...@iki.fi: On 4.4.2012, at 16.04, Patrick Westenberg wrote: Timo Sirainen schrieb: where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. MX lookups are intended for SMTP, I very much doubt LMTP uses them? Yes, but postfix lmtp is the same binary as smtp, at least currently.
Re: [Dovecot] Hints for a NFS-Setup
On 4.4.2012, at 16.57, Patrick Domack wrote: Quoting Timo Sirainen t...@iki.fi: On 4.4.2012, at 16.04, Patrick Westenberg wrote: Timo Sirainen schrieb: where lmtp.example.com expands to your two IP addresses. DNS server handles load balancing by returning IPs in round robin and Postfix handles fallbacking to the second IP if the first one doesn't work. Dovecot has similar behavior in several places. As far as I understand, lmtp.example.com with out records would result in a MX-lookup. If postfix behaves for lmtp like it does for smtp, it will try all listed MX (my directors) in turn. MX lookups are intended for SMTP, I very much doubt LMTP uses them? Yes, but postfix lmtp is the same binary as smtp, at least currently. That's about how to handle incoming lmtp vs. smtp connections. The outgoing smtp vs. lmtp connections are always handled by the same binary anyway.
Re: [Dovecot] Hints for a NFS-Setup
On Fri, 2012-03-30 at 21:37 +0200, Patrick Westenberg wrote: Nick Warr schrieb: I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production. Yes, with director it should be something like that: MTA --(lmtp)--\ /--(lmtp)-- backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)-- backend2 --/ IMAP-User -- frontend1 --\ /--(imap)-- backend1 --\ -- director -- -- NFS IMAP-User -- frontend2 --/ \--(imap)-- backend2 --/ The director above must be the same one for both IMAP and MTA. So what happens if us...@example.tld receives a mail? - The director decides to connect to backend1 which in turn stores the mail on the NFS share and the index file locally? - Then, us...@example.tld connects to one of the frontends. Does the director know that, earlier, this user received a mail and proxies him to backend1 too? Yes. Director process actually doesn't know anything about mail protocols, it simply does username - backend mapping. The regular Dovecot IMAP/POP3/LMTP/doveadm proxying code handles the rest.
Re: [Dovecot] Hints for a NFS-Setup
Patrick Westenberg schrieb: Nick Warr schrieb: I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production. Yes, with director it should be something like that: MTA --(lmtp)--\ /--(lmtp)-- backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)-- backend2 --/ IMAP-User -- frontend1 --\ /--(imap)-- backend1 --\ -- director -- -- NFS IMAP-User -- frontend2 --/ \--(imap)-- backend2 --/ So what happens if us...@example.tld receives a mail? - The director decides to connect to backend1 which in turn stores the mail on the NFS share and the index file locally? - Then, us...@example.tld connects to one of the frontends. Does the director know that, earlier, this user received a mail and proxies him to backend1 too? Timo? This post is from March, right before you introduced the stones :D
Re: [Dovecot] Hints for a NFS-Setup
Il 30/03/2012 13.31, Patrick Westenberg ha scritto: Hi everyone, as I have often trouble with OCFS2 I want to switch to NFS but I'm not sure how to rebuild my cluster with regard to locking and indexing problems. By now my I have a four server configuration (there are another 2 servers for outgoing mail but they can be ignored): MTA(MX10) --(lmtp/socket)-- local dovecot -- iSCSI-LUN with OCFS2 MTA(MX10) --(lmtp/socket)-- local dovecot -- iSCSI-LUN with OCFS2 IMAP-User --(imap)-- IMAP-Server1 (local dovecot) -- iSCSI-LUN/OCFS2 IMAP-User --(imap)-- IMAP-Server2 (local dovecot) -- iSCSI-LUN/OCFS2 As far as I understood I will get poor performance if I'd just switch from OCFS2 to NFS (while keeping this configuration) with 4 hosts accessing the NFS-share and the index files on it and it is recommended to assign users to a specific host (http://wiki2.dovecot.org/NFS). I'm uncertain what's the meaning of user in this context. Is it an IMAP-User or every incoming mail? An IMAP-User assigned to a specific IMAP-Server is ok for me and I could store and profit from local index files. However, I want my incoming mailservers to be equally receiving mails. Both should accept mails for every mailbox but in this case I won't have local indexes. I would appreciate any hints. Patrick If you've got a load balancer, it should be fairly easy to do simple IP stickiness, with a long enough timeout, most IMAP and POP3 users will stay on the same server.. I'm sure there is some load balancing software that's also L7 aware, and could direct by username (though you'd probably have to have the LB terminate the SSL, not the server behind it). SMTP wouldn't have to be balanced in the same way, you could just use round robin in that case.. I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production.
Re: [Dovecot] Hints for a NFS-Setup
On 30.3.2012, at 15.38, Nick Warr wrote: As far as I understood I will get poor performance if I'd just switch from OCFS2 to NFS (while keeping this configuration) with 4 hosts accessing the NFS-share and the index files on it and it is recommended to assign users to a specific host (http://wiki2.dovecot.org/NFS). I'm uncertain what's the meaning of user in this context. Is it an IMAP-User or every incoming mail? IMAP, POP3, LDA so everything. If you've got a load balancer, it should be fairly easy to do simple IP stickiness, with a long enough timeout, most IMAP and POP3 users will stay on the same server.. I'm sure there is some load balancing software that's also L7 aware, and could direct by username (though you'd probably have to have the LB terminate the SSL, not the server behind it). IP stickiness isn't enough if user uses more than one IMAP client, which is pretty common nowadays. And doesn't help at all with LDA. I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production. Anything else except Dovecot director will cause corruption with NFS. Several really large sites already use director in production.
Re: [Dovecot] Hints for a NFS-Setup
Nick Warr schrieb: I think some of the new Dovecot (director?) software is user aware, but I don't know if it's quite ready for production. Yes, with director it should be something like that: MTA --(lmtp)--\ /--(lmtp)-- backend1 --\ -- director -- -- NFS MTA --(lmtp)--/ \--(lmtp)-- backend2 --/ IMAP-User -- frontend1 --\ /--(imap)-- backend1 --\ -- director -- -- NFS IMAP-User -- frontend2 --/ \--(imap)-- backend2 --/ So what happens if us...@example.tld receives a mail? - The director decides to connect to backend1 which in turn stores the mail on the NFS share and the index file locally? - Then, us...@example.tld connects to one of the frontends. Does the director know that, earlier, this user received a mail and proxies him to backend1 too?
