Re: [Dovecot] Different but probably related issue
Hi Marc,
[+list since I'm unlikely to be able to solve this problem myself]
I am trying to setup a debian testing (wheeze) mail server using
postfix, dovecot and amavisd-new with spamassassin. I have everything
working fine, using mdbox mailboxes and system users. As a final touch
for this setup, I wanted to be able to train the (global) bayes
database directly through IMAP.
Hence, I installed your plugin (directly from the official debian
repositories) and set it up to report mails to spamassassin. I am
using the pipe backend to call a wrapper script, that stores the
mail into a temporary file and launches sa-learn to learn it. My tests
indicate that this is working properly.
Ok, nice.
However, when the dovecot-antispam plugin is enabled, I have a weird
problem sending emails. This is, whenever my MUA tries to save the
just sent message to the Sent folder, dovecot shows the following
error:
Hmm, ok, let's see
---
Dovecot's error log:
---
Jun 4 22:35:14 aiur dovecot: imap(user): Error: mdbox
/home/user/.mdbox/mailboxes/Sent/dbox-Mails: map uid lost for uid 0
Jun 4 22:36:06 aiur dovecot: imap(user): Error:
/home/user/.mdbox/mailboxes/Spam/dbox-Mails/dovecot.index reset, view is
now inconsistent
Jun 4 22:36:09 aiur dovecot: imap(user): Error: Log synchronization error
at seq=8,offset=27592 for /home/user/.mdbox/storage/dovecot.map.index:
Append with UID 56056, but next_uid = 56057
Jun 4 22:36:09 aiur dovecot: lda(user): Error: Log synchronization error
at seq=8,offset=27592 for /home/user/.mdbox/storage/dovecot.map.index:
Append with UID 56056, but next_uid = 56057
Jun 4 22:36:10 aiur dovecot: imap(user): Error:
/home/user/.mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is
now inconsistent
As a result, the MUA hangs for a while (some minute and a half). After
that it closes the IMAP session properly, but I am left with two
copies of the sent email in the Sent folder: one that is marked as
unread and one that is not.
Curious. I think the problem is likely the mdbox storage... There have
always been some issues with it and the antispam plugin when combined.
The first issue was that we couldn't access the raw text or
something ... not sure what's up now.
IMAP Conversation (as logged by roundcube webmail)
[04-Jun-2012 22:35:14 +0200]: [4A68] C: A0005 APPEND INBOX.Sent (\Seen)
{519+}
[04-Jun-2012 22:35:14 +0200]: [4A68] C: Received: from
cpe-76-169-183-245.socal.res.rr.com ([76.169.183.245])
by server.domain.tld
with HTTP (HTTP/1.1 POST); Mon, 04 Jun 2012 22:35:14 +0200
MIME-Version: 1.0
...
[04-Jun-2012 22:36:10 +0200]: [4A68] S: A0005 OK [APPENDUID
1338488996 4274] Append completed.
That looks ... pretty normal.
At this point, I do not know what else to try or how to fix this
problem. Thus, I have had to disable your plugin for now. Do you have
any ideas on how to proceed? I can give you access to this machine if
need be (it's a personal server).
Unfortunately, I don't. I can only suggest, as a test, trying with some
other storage format -- I only use Maildir -- to see if the problem is
really in the interaction with mdbox. I'm fairly sure that's likely the
problem, maybe the plugin doesn't pass something through append that is
needed by mdbox, but I've never even attempted to understand mdbox.
Maybe Timo can comment. Timo, you can find the latest code here:
http://git.sipsolutions.net/?p=dovecot-antispam.git;a=summary
johannes
Re: [Dovecot] best practises for mail systems
On 5.6.2012, at 6.14, Костырев Александр Алексеевич wrote: - not quite sure if glusterfs is production ready solution 'cause I've experienced split-brains during setting it up Last I've heard glusterfs causes corruption problems with Dovecot. You should try stress testing it with imaptest: http://imapwiki.org/ImapTest
Re: [Dovecot] INBOX help needed, dovecot + squirrelmail
Den 2012-06-05 04:33, j...@rahul.net skrev: Im trying to figure out how to get dovecot to deliver to my mail_location (example: /opt/imapdata/j/jeff/INBOX/inbox) AND work with squirrelmail. Ive worked on this for hours reading the docs etc with no luck so far. namespace is set to in squirrelmail, but it must be INBOX. run conf.pl and fix it :=)
Re: [Dovecot] dovecot-metadata-9 released
Hello Nick!
I am sorry - I forgot to mention that you need attached patch for dovecot.
Kind regards,
Dennis
Am Dienstag, 5. Juni 2012, 11:28:27 schrieb Nick Rosier:
Hi Dennis,
I'm trying to compile the plugin on FreeBSD 9 with Dovecot 2.1.7 and get
the following error:
libtool: compile: gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I..
-I/usr/local/include/dovecot -g -O2 -MT mailbox-ext.lo -MD -MP -MF
.deps/mailbox-ext.Tpo -c mailbox-ext.c -fPIC -DPIC -o
.libs/mailbox-ext.o mailbox-ext.c:25:19: error: missing binary operator
before token (
mailbox-ext.c: In function 'mailbox_get_guid_string': mailbox-ext.c:32:
error: 'MAIL_GUID_128_SIZE' undeclared (first use in this function)
mailbox-ext.c:32: error: (Each undeclared identifier is reported only
once mailbox-ext.c:32: error: for each function it appears in.)
mailbox-ext.c:33: warning: implicit declaration of function
'mailbox_get_guid'
*** Error code 1
Stop in /root/work/dovecot-metadata-plugin-6fe39779d758/src. *** Error
code 1
Removing DOVECOT_PREREQ and forcing to use the 2.1 definition fixes
that (I couldn't find anywhere where that macro was defined).
Next I get another error, again caused by the DOVECOT_PREREQ:
libtool: compile: gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I..
-I/usr/local/include/dovecot -g -O2 -MT imap-metadata-plugin.lo -MD -MP
-MF .deps/imap-metadata-plugin.Tpo -c imap-metadata-plugin.c -fPIC
-DPIC -o .libs/imap-metadata-plugin.o
imap-metadata-plugin.c: In function 'is_valid_rfc5464_entry_name':
imap-metadata-plugin.c:162: warning: comparison is always false due to
limited range of data type
imap-metadata-plugin.c:513:19: error: missing binary operator before
token (
imap-metadata-plugin.c: In function 'cmd_getmetadata':
imap-metadata-plugin.c:516: warning: passing argument 2 of
'mail_namespace_find' from incompatible pointer type
imap-metadata-plugin.c: In function 'setmetadata_helper':
imap-metadata-plugin.c:596: warning: 'return' with a value, in function
returning void
imap-metadata-plugin.c:672:19: error: missing binary operator before
token (
imap-metadata-plugin.c: In function 'cmd_setmetadata':
imap-metadata-plugin.c:675: warning: passing argument 2 of
'mail_namespace_find' from incompatible pointer type
*** Error code 1
Am I missing something on my system?
Rgds,
N.
Dennis Schridde wrote:
Hello everyone!
I just released dovecot-metadata-8, which is an implementation of RFC 5464
(IMAP METADATA), allowing to add comments/annotations/metadata to folders
of an email account.
2012-06-04: Version 9
* Added Dovecot 2.1 compatibility
* Fixed compliance with RFC 5464 Section 3.2
* Separated backend code into library
* Synced code of imap-annotatemore with imap-metadata
* Improved error messages
* Several bugfixes (incl. segfaults)
* Minor cleanups
Please get the code from [1] and send me an email for any problem you
find.
For more information please refer to my email from Sun, 12 Jun 2011
15:55:57 +0200 titled dovecot-metadata-8 released.
Kind regards,
Dennis
[1] http://hg.dovecot.org/dovecot-metadata-plugin# HG changeset patch
# User Dennis Schridde devuran...@gmx.net
# Date 1319973808 -3600
# Node ID b144c7d3bb67b44c59aa53965ceb70c38e4f21b3
# Parent b88e776bfb03011fe3b2825361d59cf90dd95562
Add macros DOVECOT_VERSION_{MAJOR,MINOR} to config.h to allow version checks in the preprocessor
Version number set as PACKAGE_VERSION is assumed to be D.D* with D=digits
diff -r b88e776bfb03 -r b144c7d3bb67 configure.in
--- a/configure.in Sat Oct 29 13:29:30 2011 +0200
+++ b/configure.in Sun Oct 30 12:23:28 2011 +0100
@@ -281,6 +281,9 @@
AC_DEFINE_UNQUOTED(DOVECOT_STRING, $PACKAGE_STRING, Dovecot string)
AC_DEFINE_UNQUOTED(DOVECOT_VERSION, $PACKAGE_VERSION, Dovecot version)
+AC_DEFINE([DOVECOT_VERSION_MAJOR], regexp(AC_PACKAGE_VERSION, [^\([0-9]+\)\.\([0-9]+\)], [\1]), [Dovecot major version])
+AC_DEFINE([DOVECOT_VERSION_MINOR], regexp(AC_PACKAGE_VERSION, [^\([0-9]+\)\.\([0-9]+\)], [\2]), [Dovecot minor version])
+
AC_CHECK_HEADERS(strings.h stdint.h unistd.h dirent.h malloc.h inttypes.h \
sys/uio.h sys/sysmacros.h sys/resource.h sys/select.h libgen.h \
sys/quota.h sys/fs/ufs_quota.h ufs/ufs/quota.h jfs/quota.h sys/fs/quota_common.h \
# HG changeset patch
# User Dennis Schridde devuran...@gmx.net
# Date 1319974793 -3600
# Node ID 4ee2e23710fb04f95874a61ddf32f04a9cb2afbb
# Parent b144c7d3bb67b44c59aa53965ceb70c38e4f21b3
Add DOVECOT_PREREQ to src/lib/macros.h - convenience macro to test the version of dovecot
diff -r b144c7d3bb67 -r 4ee2e23710fb src/lib/macros.h
--- a/src/lib/macros.h Sun Oct 30 12:23:28 2011 +0100
+++ b/src/lib/macros.h Sun Oct 30 12:39:53 2011 +0100
@@ -193,4 +193,14 @@
#define i_unreached() \
i_panic(file %s: line %d: unreached, __FILE__, __LINE__)
+/*
+ Convenience macros to test the versions of dovecot.
+*/
+#if defined DOVECOT_VERSION_MAJOR defined DOVECOT_VERSION_MINOR
+#
[Dovecot] dsync backup doubles quota
Hi everyone, I recognized a very strange behavior when doing backups of my mdbox mailboxes. After the backup the quota for each mailbox is twice as much as before the backup and I have to recalculate the quota to get the former/correct information. root@mb01:~# doveadm quota get -u t...@example.com User quota STORAGE 5 10240 User quota MESSAGE 11- root@mb01:~# doveadm backup -u t...@example.com mdbox:/home/example.com/test root@mb01:~# doveadm quota get -u t...@example.com User quota STORAGE10 10240 User quota MESSAGE22 - root@mb01:~# doveadm quota get -u t...@example.com root@mb01:~# doveadm quota get -u t...@example.com User quota STORAGE 5 10240 User quota MESSAGE11 - Is this a bug or normal behavior? Regards Patrick
Re: [Dovecot] best practises for mail systems
I think LVS is just fine and it is not a SPOF 'cause it is actually 2 servers: active master -- and standby slave. LVS supports real time replication of connections from master to slave, so if master dies slave knows which IP was connected to which dovecot server. I'm more worried about right design of mailstorage.. should I use some cluster fs with all mail of all users or should I split mailstorage across servers and somehow avoid long downtime if one of servers goes down. -Original Message- From: dovecot-boun...@dovecot.org [mailto:dovecot-boun...@dovecot.org] On Behalf Of Matthias-Christian Ott Sent: Tuesday, June 05, 2012 11:28 PM To: dovecot@dovecot.org Subject: Re: [Dovecot] best practises for mail systems On 2012-06-05 05:14, Костырев Александр Алексеевич wrote: On each host system we created one VM and passed through 3x2TB disks into it. In guests vms on top of this disks we made XFS and fired up glusterfs with distributed replicated volumes for our mailstorage. so it looks like this: vm1replicate vm2 disk1 disk4 disk2 disk5 disk3 disk6 in each vm we mounted glusterfs and pointed dovecot to that dir for mail creation (as ltmp) and imap4 user access. also we use exim as smtp. So, with glusterfs as mailstorage we can go for LVS to load balancing for exim and dovecot. so wherenever one of host systems (hence one of mail vms) goes down, users don't notice that 'cause LVS points them to working smtp and imap4 servers and they get their mail 'cause of glusterfs. [...] Cons: - not quite sure if glusterfs is production ready solution 'cause I've experienced split-brains during setting it up - IO performance issue. Though we didn't yet run any io tests, but glusterfs uses fuse to mount on clients. And guys on #gluster told me writing to the glusterfs mount will not be strictly local io. I'm not familiar with LVS, but from the project description it seems that you need a front server that does the load balancing, so you either have to run at least two of these servers in parallel or add to your cons that you introduced a single point of failure. But you mentioned that you only have two servers, so you really can do this. I would rather ensure high availability by running the two servers as masters and using either IP address takeover or DNS failover (with dynamic DNS) and either use Dovecot's replication (I haven't tested it yet and I'm not sure what happens in case of IP address takeover) or a file system that can handle these kinds of errors (e.g. Coda). You could do load balancing via round-robin DNS. This only protects you against the failure of single machine and because IMAP sessions are not replicated between the servers, connections will get reset if one server fails, but it's cost-effective and uses software that already exists. Regards, Matthias-Christian
Re: [Dovecot] best practises for mail systems
We once try to use similar solution as your first. 3 servers for LVS -HA This master server redirect users for 2 or 3 dovecot backends.. The mail storage were maildir ontop of OCFS2 Our problem were that OCFS2 were too slow. We could not handle many users. So we took an step back and now use only user one server. But still thinking in go back to the first one. with LVS When using LVS try to sticky user to the same backend, LVs can do ths by source ip. Where i work we have problens on testign storage. If you have any advices for testing disk performance, i will be thankfull. I wil be glad to answer anything else. []'sf.rique On Tue, Jun 5, 2012 at 9:59 AM, Костырев Александр Алексеевич a.kosty...@serverc.ru wrote: I think LVS is just fine and it is not a SPOF 'cause it is actually 2 servers: active master -- and standby slave. LVS supports real time replication of connections from master to slave, so if master dies slave knows which IP was connected to which dovecot server. I'm more worried about right design of mailstorage.. should I use some cluster fs with all mail of all users or should I split mailstorage across servers and somehow avoid long downtime if one of servers goes down. -Original Message- From: dovecot-boun...@dovecot.org [mailto:dovecot-boun...@dovecot.org] On Behalf Of Matthias-Christian Ott Sent: Tuesday, June 05, 2012 11:28 PM To: dovecot@dovecot.org Subject: Re: [Dovecot] best practises for mail systems On 2012-06-05 05:14, Костырев Александр Алексеевич wrote: On each host system we created one VM and passed through 3x2TB disks into it. In guests vms on top of this disks we made XFS and fired up glusterfs with distributed replicated volumes for our mailstorage. so it looks like this: vm1replicate vm2 disk1 disk4 disk2 disk5 disk3 disk6 in each vm we mounted glusterfs and pointed dovecot to that dir for mail creation (as ltmp) and imap4 user access. also we use exim as smtp. So, with glusterfs as mailstorage we can go for LVS to load balancing for exim and dovecot. so wherenever one of host systems (hence one of mail vms) goes down, users don't notice that 'cause LVS points them to working smtp and imap4 servers and they get their mail 'cause of glusterfs. [...] Cons: - not quite sure if glusterfs is production ready solution 'cause I've experienced split-brains during setting it up - IO performance issue. Though we didn't yet run any io tests, but glusterfs uses fuse to mount on clients. And guys on #gluster told me writing to the glusterfs mount will not be strictly local io. I'm not familiar with LVS, but from the project description it seems that you need a front server that does the load balancing, so you either have to run at least two of these servers in parallel or add to your cons that you introduced a single point of failure. But you mentioned that you only have two servers, so you really can do this. I would rather ensure high availability by running the two servers as masters and using either IP address takeover or DNS failover (with dynamic DNS) and either use Dovecot's replication (I haven't tested it yet and I'm not sure what happens in case of IP address takeover) or a file system that can handle these kinds of errors (e.g. Coda). You could do load balancing via round-robin DNS. This only protects you against the failure of single machine and because IMAP sessions are not replicated between the servers, connections will get reset if one server fails, but it's cost-effective and uses software that already exists. Regards, Matthias-Christian
Re: [Dovecot] INBOX help needed, dovecot + squirrelmail
Benny Pedersen m...@junc.org wrote: Den 2012-06-05 04:33, j...@rahul.net skrev: Im trying to figure out how to get dovecot to deliver to my mail_location (example: /opt/imapdata/j/jeff/INBOX/inbox) AND work with squirrelmail. Ive worked on this for hours reading the docs etc with no luck so far. namespace is set to in squirrelmail, but it must be INBOX. run conf.pl and fix it :=) Thanks Benny. I didnt see 'namespace' in my configure for squirrelmail 1.4.22, but if you meant Folder Defaults-Default Folder Prefix = INBOX. I just tried that and I still get: Error: chdir(/opt/imapdata/j/jeff/INBOX/inbox) failed: Not a directory Was that the setting you meant or was there another I missed? Thanks /mf/home/jeep/shell/.signature
Re: [Dovecot] INBOX help needed, dovecot + squirrelmail
j...@rahul.net (Jeff Lacki) wrote: Thanks Benny. I didnt see 'namespace' in my configure for squirrelmail 1.4.22, but if you meant Folder Defaults-Default Folder Prefix = INBOX. I just tried that and I still get: Error: chdir(/opt/imapdata/j/jeff/INBOX/inbox) failed: Not a directory Was that the setting you meant or was there another I missed? Thanks Nevermind, I found the problem after your suggestion. Turns out my DB was returning a home directory of: /opt/imapdata/j/jeff/INBOX/inbox from when I was playing with something earlier, that got me past that issue, however I still dont know why its not giving me maildir instead of mbox. But thank you for helping me fix that issue! Jeff /mf/home/jeep/shell/.signature
Re: [Dovecot] INBOX help needed, dovecot + squirrelmail
Den 2012-06-05 15:41, j...@rahul.net skrev: Error: chdir(/opt/imapdata/j/jeff/INBOX/inbox) failed: Not a directory this error is not squirrelmail :=) # dovecot.conf namespace: type: private inbox: yes list: yes subscriptions: yes if you use sql auth in dovecot then the maildir must not end in / else it will be a mbox file mail_location: maildir:/home/vmail/%d/%u/.maildir ~ must be set to mail_location: maildir:/home/vmail/%d/%u/ and the .maildir comes from sql concat if i remember my own setup :=) squirrelmail will work without INBOX. but namespace in dovecot must math it
[Dovecot] [ Re: best practises for mail systems]
Hello, If disk space and bandwidth are affordable (and from your setup it seems that they are affordable as you have everything locally) I would split the mail storage completely and use replication in between n-master servers (n=2 for your case). The replication is not yet fully tested, but Timo is actively working on this feature. The fear of lossing the imap session does not make sense (at least to me) as the client will reconnect automatically in the background. Like this you have no SPOF and no split-brain and you get the flexibility (if needed) to geographically distribute your servers in the the future. Keep each server with its own ip, connect to them via DNS (round robin etc etc). We are currently experimenting with a setup similar to this one, but with geographically distributed servers (trans-continental) (bandwidth limited and high cost). Best regards, Andrei We once try to use similar solution as your first. 3 servers for LVS -HA This master server redirect users for 2 or 3 dovecot backends.. The mail storage were maildir ontop of OCFS2 Our problem were that OCFS2 were too slow. We could not handle many users. So we took an step back and now use only user one server. But still thinking in go back to the first one. with LVS When using LVS try to sticky user to the same backend, LVs can do ths by source ip. Where i work we have problens on testign storage. If you have any advices for testing disk performance, i will be thankfull. I wil be glad to answer anything else. []'sf.rique On Tue, Jun 5, 2012 at 9:59 AM, ëÏÓÔÙÒÅ× áÌÅËÓÁÎÄÒ áÌÅËÓÅÅ×ÉÞ a.kosty...@serverc.ru wrote: I think LVS is just fine and it is not a SPOF 'cause it is actually 2 servers: active master -- and standby slave. LVS supports real time replication of connections from master to slave, so if master dies slave knows which IP was connected to which dovecot server. I'm more worried about right design of mailstorage.. should I use some cluster fs with all mail of all users or should I split mailstorage across servers and somehow avoid long downtime if one of servers goes down. -Original Message- From: dovecot-boun...@dovecot.org [mailto:dovecot-boun...@dovecot.org] On Behalf Of Matthias-Christian Ott Sent: Tuesday, June 05, 2012 11:28 PM To: dovecot@dovecot.org Subject: Re: [Dovecot] best practises for mail systems On 2012-06-05 05:14, ëÏÓÔÙÒÅ× áÌÅËÓÁÎÄÒ áÌÅËÓÅÅ×ÉÞ wrote: On each host system we created one VM and passed through 3x2TB disks into it. In guests vms on top of this disks we made XFS and fired up glusterfs with distributed replicated volumes for our mailstorage. so it looks like this: vm1replicate vm2 disk1 disk4 disk2 disk5 disk3 disk6 in each vm we mounted glusterfs and pointed dovecot to that dir for mail creation (as ltmp) and imap4 user access. also we use exim as smtp. So, with glusterfs as mailstorage we can go for LVS to load balancing for exim and dovecot. so wherenever one of host systems (hence one of mail vms) goes down, users don't notice that 'cause LVS points them to working smtp and imap4 servers and they get their mail 'cause of glusterfs. [...] Cons: - not quite sure if glusterfs is production ready solution 'cause I've experienced split-brains during setting it up - IO performance issue. Though we didn't yet run any io tests, but glusterfs uses fuse to mount on clients. And guys on #gluster told me writing to the glusterfs mount will not be strictly local io. I'm not familiar with LVS, but from the project description it seems that you need a front server that does the load balancing, so you either have to run at least two of these servers in parallel or add to your cons that you introduced a single point of failure. But you mentioned that you only have two servers, so you really can do this. I would rather ensure high availability by running the two servers as masters and using either IP address takeover or DNS failover (with dynamic DNS) and either use Dovecot's replication (I haven't tested it yet and I'm not sure what happens in case of IP address takeover) or a file system that can handle these kinds of errors (e.g. Coda). You could do load balancing via round-robin DNS. This only protects you against the failure of single machine and because IMAP sessions are not replicated between the servers, connections will get reset if one server fails, but it's cost-effective and uses software that already exists. Regards, Matthias-Christian !DSPAM:4fce037e104291424646138!
Re: [Dovecot] INBOX help needed, dovecot + squirrelmail
Den 2012-06-05 17:03, j...@rahul.net skrev: from when I was playing with something earlier, that got me past that issue, however I still dont know why its not giving me maildir instead of mbox. remove last / in sql query auth path (concated here) dovecot have it well explained in wiki
Re: [Dovecot] auth trouble
On Jun 4, 2012, at 8:45 PM, Joseph Tam wrote: If dovecot-auth is getting input from a local socket, then rhost information is irrelevant since the host doing the asking is the server itself (maybe from another daemon connected to a remote host). Thanks for the confirmation of my suspicions Maybe someone is brute forcing your server's Postfix authenticated SMTP service since Postfix can be configured to use Dovecot's SASL authentication framework. and for the suggestion -- I do have Postfix using Dovecot-Auth checking for SASL. I think I'm going to re-install and run Tripwire... -- Glenn English hand-wrapped from my Apple Mail
Re: [Dovecot] [ Re: best practises for mail systems]
On 2012-06-05 17:33, Michescu Andrei wrote: The fear of lossing the imap session does not make sense (at least to me) as the client will reconnect automatically in the background. I agree, in practice this is not an issue compared to the unavailability of the service, but on longer IMAP sessions (e.g. transferring a big file) the connection loss is noticeable. Like this you have no SPOF and no split-brain and you get the flexibility (if needed) to geographically distribute your servers in the the future. Keep each server with its own ip, connect to them via DNS (round robin etc etc). This depends on the resolver, operating systems and clients you want to support, because I read that not all networks generate proper ICMP/ICMPv6 Destination Unreachable messages and instead simple drop the packets, so that the clients first try to connect to the failed server until timeout and then connects to the second server. Since IMAP is a stateful protocol the latency of the initial connect to the failed server can be ignored, but if you want to eliminate this, you can use dynamic DNS to automatically remove the corresponding RRs (depending on your situation you need an external monitoring server for this to avoid problems in case of net splits). We are currently experimenting with a setup similar to this one, but with geographically distributed servers (trans-continental) (bandwidth limited and high cost). I also have some plans for a similar setup in the near future. Can you share your results on the mailing list? I'm especially interested if failover via DNS works in practice (I did some searches, but I'm not fully convinced of it, but it seems quite simple compared to other solutions). Regards, Matthias-Christian
Re: [Dovecot] [ Re: best practises for mail systems]
Hello, I agree, in practice this is not an issue compared to the unavailability of the service, but on longer IMAP sessions (e.g. transferring a big file) the connection loss is noticeable. It is noticeable for somebody that really waits for a large email. For the standard user there is nothing visible because the synchronization starts / fails and starts again... In corporate environment the servers are close and the network is generally configured to have proper Destination Unreachable. For road-warriors, the main concern is the uplink/downlink and generally not the couple of seconds lost due to time-out. For the DNS... use fast-flux-like configuration and any proper resolver will behave correctly (at least in my experience). For the road-warrior setup: DNS with geoip, and all locations with split-dns (internally HA setup with failover on external locations). Unfortunately the classical HA setup (with heart-beat monitor, update DNS etc etc) it is not designed to be internet-proof (internet like in WAN). The initial design of the internet was to be able to operate even when significant segments are unavailable. Picture the following scenario: master servers on each continent. Catastrophic failure of the trans-continental network = 5 big disconnected chunks of network fully functional. Any HA setup that I saw will fail miserably. The simplest design with fully replicated masters will continue to work. Obviously planning for the scenario above is an overkill for most of the companies out there. Once you trow in the advantage of have the emails close to you anywhere where you go, then it starts making sense. And you can top it up by segmenting you user base to replicate only the users that are on the go, or are important enough. As for the current status of the ideal implementation: waiting for Timo to finalize the refactoring of dsync. As a temporary solution: rsync replication with master-slave model (not master-master). This design makes sense to us, but I'm sure that it is under-optimal for most other uses. Andrei Like this you have no SPOF and no split-brain and you get the flexibility (if needed) to geographically distribute your servers in the the future. Keep each server with its own ip, connect to them via DNS (round robin etc etc). This depends on the resolver, operating systems and clients you want to support, because I read that not all networks generate proper ICMP/ICMPv6 Destination Unreachable messages and instead simple drop the packets, so that the clients first try to connect to the failed server until timeout and then connects to the second server. Since IMAP is a stateful protocol the latency of the initial connect to the failed server can be ignored, but if you want to eliminate this, you can use dynamic DNS to automatically remove the corresponding RRs (depending on your situation you need an external monitoring server for this to avoid problems in case of net splits). We are currently experimenting with a setup similar to this one, but with geographically distributed servers (trans-continental) (bandwidth limited and high cost). I also have some plans for a similar setup in the near future. Can you share your results on the mailing list? I'm especially interested if failover via DNS works in practice (I did some searches, but I'm not fully convinced of it, but it seems quite simple compared to other solutions). Regards, Matthias-Christian !DSPAM:4fce5ae0149132093961185!
Re: [Dovecot] [ Re: best practises for mail systems]
On 5.6.2012, at 23.33, Michescu Andrei wrote: I agree, in practice this is not an issue compared to the unavailability of the service, but on longer IMAP sessions (e.g. transferring a big file) the connection loss is noticeable. It is noticeable for somebody that really waits for a large email. And there is actually some (any!) way this could be avoided?... One server dies, another continues sending the mail? I have had some thoughts about transferring idling Dovecot connections between processes / servers so that clients wouldn't notice it, but I haven't even thought about moving active (long-running) connections.
Re: [Dovecot] auth trouble
On Tue, Jun 05, 2012 at 09:38:49AM -0600, Glenn English wrote: On Jun 4, 2012, at 8:45 PM, Joseph Tam wrote: If dovecot-auth is getting input from a local socket, then rhost information is irrelevant since the host doing the asking is the server itself (maybe from another daemon connected to a remote host). Thanks for the confirmation of my suspicions What suspicions were confirmed? Maybe someone is brute forcing your server's Postfix authenticated SMTP service since Postfix can be configured to use Dovecot's SASL authentication framework. And these brute force attempts would be logged, each one. and for the suggestion -- I do have Postfix using Dovecot-Auth checking for SASL. I think I'm going to re-install and run Tripwire... I think you are overreacting. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
Re: [Dovecot] auth trouble
Glenn English wrote:
Maybe someone is brute forcing your server's Postfix authenticated
SMTP service since Postfix can be configured to use Dovecot's SASL
authentication framework.
and for the suggestion -- I do have Postfix using Dovecot-Auth checking
for SASL.
I think I'm going to re-install and run Tripwire...
Tripwire? If the purpose of your query is to automate blocking of brute
forcers, this software is not what you want (which detects tampering of
critical system files).
I suggest trying to find where Postfix failed login reports go, then use
your fail2ban or what-have-you to detect and block hosts that repeatedly
fail authentication.
(First Google hit I did on this subject)
http://scottlinux.com/2011/05/26/prevent-postfix-brute-force/
The log entries might look like
{timestamp} {servername} postfix/smtpd[{pid}]: lost connection after
AUTH
from {remote-hostname}[{remote-ip}]
Joseph Tam jtam.h...@gmail.com
Re: [Dovecot] auth trouble
On Jun 5, 2012, at 3:53 PM, /dev/rob0 wrote: What suspicions were confirmed? At first I thought that somebody was TCP'ing in and somehow turning off the remote IP in the log so I couldn't block it. Then an answer from another mailing list, and a little thinking, made it occur to me that maybe my server had been penetrated. And these brute force attempts would be logged, each one. They are, with no rhost. And there are other brute force attempts that *do* have IPs. I think you are overreacting. I really hope so. What's your thinking? Have you seen this before? And most important: what is it, how does it work, and how do I get rid of it and keep it from coming back? -- Glenn English hand-wrapped from my Apple Mail
Re: [Dovecot] [ Re: best practises for mail systems]
05.06.2012 23:33, Michescu Andrei написал: Picture the following scenario: master servers on each continent. Catastrophic failure of the trans-continental network = 5 big disconnected chunks of network fully functional. Any HA setup that I saw will fail miserably. The simplest design with fully replicated masters will continue to work. Dispute the original topic, I'd say this looks like a good service idea, as many company may pay for such a service if it can be set up specifically for their needs (routing, logs, backups, redirections). Gmail (and other big guys like them) won't be that fine-tunable (having point to service many customers with the same type of control), and companies sometime just won't deal with such a Big Brother to store their corporate mail due to internal regulations (read - 'corporate paranoia'). But the replication between points of presence (5 big datacenters, one per continent, won't be good topology) will be painful and we easily face split-brain situation, whichever replicaton scheme I can imagine. Yours, Alexander
