[Dovecot] Advise on upgrading from a jurassic version - please help.
Hi all, I have inherited an old Dovecot installation which is causing headaches almost every day. I know that one of the rules says "Don't bother asking questions about v0.99.x versions. They're no longer supported."...but please bear with me, this will be quick as I only need some advise from experienced Dovecot gurus out there. I have read the Dovecot documentation and there are instructions to upgrade from 0.99.x to 1.x and so on... my question is: can I upgrade from 0.99.11 to 2.x directly or is it a massive leap? If so, what do I have to keep in mind? This is a production system so I should not break anything... or at least have a rollback plan... Thanks a lot in advance! Regards, Martin
Re: [Dovecot] dovecot Digest, Vol 107, Issue 20 Fscking warnings
Yes that is the google thread that I saw. I don't see the relevance of your reference to dsync. As I read the man pages for dsync it is used to sync separate servers, to make backups or to convert mailbox formats. When I upgraded from 1.2.15 to 2.1.1 I saw nothing in the doco to suggest that dsync was relevant to my scenario. In a previous thread here (Log sync errors), Timo suggested that the migration fix was to delete all .imap directories. My understanding was that this should fix any differences between 1.2.15 files and 2.1.1. If that were the case, mentioning the migration again would seem irrelevant. However, it seems that deleting the .imap files did not fix the log sync errors or the fscking warnings. Both are still happening continuously. Cheers, Stephen On Thu, 8 Mar 2012 08:26:55 PM dovecot-requ...@dovecot.org wrote: > Date: Wed, 07 Mar 2012 15:03:35 -0600 > From: Stan Hoeppner > To: dovecot@dovecot.org > Subject: Re: [Dovecot] Fscking warnings > Message-ID: <4f57cd27.3000...@hardwarefreak.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On 3/6/2012 5:07 PM, Stephen Davies wrote: > > Google tells me that these "should go away" but they don't. > > > > > > > > Seems to happen continuously while a user is viewing email. > > Is this thread what "Google tells you"? > > http://dovecot.org/list/dovecot/2010-October/053909.html > > Timo is the creator of Dovecot, if you didn't know. So you can take his > words for gospel. Also note his last statement in that thread: > > "The next time you could do it with dsync to avoid these kind of > problems." > > It would seem you omitted a very important detail from your problem > report, which is that you recently performed a migration. Please don't > omit such critical details in future requests for help. Provide as much > relevant detail as possible. This speeds the process up for everyone, > and avoids guesswork on our part. > > -- > Stan > > > -- > > Message: 10 > Date: Thu, 8 Mar 2012 00:26:55 +0100 > From: "Marc" > To: > Subject: [Dovecot] FW: Centos 6 + dovecot 2 + mail.app + imap -- = Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia.Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583
[Dovecot] Has dovecot 2.1.1 been built and tested on AIX 6.1???
I have downloaded and built dovecot 2.1.1 using gcc on AIX 6.1.
(The output of "dovecot -n" is at the bottom of this email.)
I'm trying "baby steps" to get it up, before I give it the final configuration.
(My apologies: I was pointed to RFC3501 and told to get an IMAP server,
build it, configure it, and bring it up)
What is currently occurring when I start dovecot is:
Error: service(pop3-login): listen(::, 110) failed: Address already in
use
Error: service(pop3-login): listen(::, 995) failed: Address already in
use
Error: service(imap-login): listen(::, 143) failed: Address already in
use
Error: service(imap-login): listen(::, 993) failed: Address already in
use
Fatal: Failed to start listeners
Using TRUSS and recompiling with log messages I've determined that dovecot is
successfully creating and binding to AF_INET sockets... but is failing
when trying to do the "bind" the same port to an AF_INET6 socket.
The failure is "EADDRINUSE".
The logic in the dovecot sources seems driven off of the define of HAVE_IPV6
(defined in config.h by configure)
So, the questions I have are:
- Is this the correct behavior
- If this is the correct behavior, has this been tested against AIX 6.1,
and if so, does anyone have an idea of what I did wrong...???
If it has not been tested against AIX 6.1 and is NOT the correct behavior,
should I just change "config.h", and undefined HAVE_IPV6 ... or is there a
better way to move beyond this issue... (like a change to "configure")???
Thanks,
-tony
Here is the output of "dovecot -n":
# 2.1.1: /attic/usr/local/etc/dovecot/dovecot.conf
# OS: AIX 1 00C30F654C00
default_login_user = dovecot
disable_plaintext_auth = no
namespace {
inbox = yes
location =
mailbox {
special_use = \Drafts
name = Drafts
}
mailbox {
special_use = \Junk
name = Junk
}
mailbox {
special_use = \Sent
name = Sent
}
mailbox {
special_use = \Sent
name = Sent Messages
}
mailbox {
special_use = \Trash
name = Trash
}
prefix =
name = inbox
}
passdb {
args = scheme=CRYPT username_format=%u /attic/usr/local/etc/dovecot/users
driver = passwd-file
}
service anvil-auth-penalty {
name = anvil
}
service auth-worker {
name = auth-worker
}
service auth-client {
name = auth
}
service config {
name = config
}
service dict {
name = dict
}
service login/proxy-notify {
name = director
}
service dns-client {
name = dns_client
}
service doveadm-server {
name = doveadm
}
service imap {
name = imap-login
}
service login/imap {
name = imap
}
service indexer-worker {
name = indexer-worker
}
service indexer {
name = indexer
}
service ipc {
name = ipc
}
service lmtp {
name = lmtp
}
service log-errors {
name = log
}
service pop3 {
name = pop3-login
}
service login/pop3 {
name = pop3
}
service login/ssl-params {
name = ssl-params
}
service stats-mail {
name = stats
}
ssl_cert =
Re: [Dovecot] Shared folder prefix listed multiple times with dovecot 2.1.1
On 8.3.2012, at 21.18, Markus Petri wrote: > after upgrading from 2.0.18 to 2.1.1 I noticed that I could not use > shared folders with mutt anymore. 2.1 lists the shared namespace prefix > once per user sharing an folder in LIST "" "%". > > I also noticed, that with 2.1 the user folder (Shared/) is no > longer tagged as \NoSelect. > > Is this the intended behaviour and mutt simply cannot cope with it or > is it a dovecot problem? Both. Dovecot shouldn't send duplicates, but mutt shouldn't break even if it did. Also Dovecot probably should add \Noselect, especially if the mailbox isn't really selectable (there's some weirdness between shared/user being equal to shared/user/INBOX, but I'm not sure what to do about it).
[Dovecot] Shared folder prefix listed multiple times with dovecot 2.1.1
Hi,
after upgrading from 2.0.18 to 2.1.1 I noticed that I could not use
shared folders with mutt anymore. 2.1 lists the shared namespace prefix
once per user sharing an folder in LIST "" "%".
I also noticed, that with 2.1 the user folder (Shared/) is no
longer tagged as \NoSelect.
Is this the intended behaviour and mutt simply cannot cope with it or
is it a dovecot problem?
Here an example with three users sharing a folder to the logged in user
with Dovecot 2.1.1:
2 LIST "" "*"
* LIST (\HasNoChildren) "/" "INBOX"
* LIST (\HasChildren) "/" "Shared/test"
* LIST (\HasNoChildren) "/" "Shared/test/Share"
* LIST (\HasChildren) "/" "Shared/test2"
* LIST (\HasNoChildren) "/" "Shared/test2/Share2"
* LIST (\HasChildren) "/" "Shared/test3"
* LIST (\HasNoChildren) "/" "Shared/test3/Share3"
2 OK List completed.
2 LIST "" "%"
* LIST (\HasNoChildren) "/" "INBOX"
* LIST (\Noselect \HasChildren) "/" "Shared"
* LIST (\Noselect \HasChildren) "/" "Shared"
* LIST (\Noselect \HasChildren) "/" "Shared"
2 OK List completed.
The same three users and config with Dovecot 2.0.18:
2 LIST "" "*"
* LIST (\HasNoChildren) "/" "INBOX"
* LIST (\Noselect \HasChildren) "/" "Shared/test"
* LIST (\Noselect \HasChildren) "/" "Shared/test2"
* LIST (\Noselect \HasChildren) "/" "Shared/test3"
* LIST (\HasNoChildren) "/" "Shared/test/Share"
* LIST (\HasNoChildren) "/" "Shared/test2/Share2"
* LIST (\HasNoChildren) "/" "Shared/test3/Share3"
2 OK List completed.
2 LIST "" "%"
* LIST (\HasNoChildren) "/" "INBOX"
* LIST (\Noselect \HasChildren) "/" "Shared"
2 OK List completed.
Markus
# 2.1.1: /opt/dovecot-2.1/etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-1-amd64 x86_64 Debian wheezy/sid
auth_mechanisms = plain login
disable_plaintext_auth = no
listen = 192.168.56.11
mail_location = maildir:~/Maildir
mail_plugins = acl
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave namespace { inbox = yes location =
prefix =
separator = /
subscriptions = yes
type = private
}
namespace {
inbox = no
list = children
location = maildir:%%h/Maildir:INDEX=~/Maildir/index/shared/%%u
prefix = Shared/%%u/
separator = /
subscriptions = no
type = shared
}
passdb {
args = /opt/dovecot-2.1/etc/dovecot/passwd
driver = passwd-file
}
plugin {
acl = vfile
acl_anyone = allow
acl_shared_dict = file:/var/lib/vdovecot/shared-mailboxes.db
}
protocols = imap
service auth {
unix_listener auth-userdb {
mode = 0600
user = vdovecot
}
}
ssl = no
userdb {
args = /opt/dovecot-2.1/etc/dovecot/passwd
driver = passwd-file
}
verbose_proctitle = yes
protocol imap {
mail_plugins = acl imap_acl
}
[Dovecot] disabling SSLv2 in dovecot 1.2.17
I've set up a list of ciphers that excludes SSLv2 ciphers (and other weak ones) in the hope of preventing SSLv2 connections: ssl_cipher_list = TLSv1+HIGH : !SSLv2 : RC4+MEDIUM : !aNULL : !eNULL : !3DES : @STRENGTH However, this doesn't prevent the SSLv2 connection being allowed as our Nessus scans show and I'm tasked with trying to plug that "hole". I see Dovecot2 had the following change a year or so ago, in file src/login-common/ssl-proxy-openssl.c: - SSL_CTX_set_options(ssl_ctx, SSL_OP_ALL); + SSL_CTX_set_options(ssl_ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2); I tried making the same change to dovecot1's src tree on our test system and it seems to have the desired effect; however I am very hesitant about putting this into our production system without seeking advice here first :-) Have I missed anything that's obviously bad about doing this please? Thanks again, Steve Platt
Re: [Dovecot] Pop3 ordering in mdbox
On 8.3.2012, at 19.56, Christoph Bußenius wrote: > On 03/04/2012 03:10 PM, Timo Sirainen wrote: >> BTW. The script should some day be updated for Dovecot v2.0.13+ which >> supports storing separate POP3 and IMAP message order. > > Oh, I was not aware that this feature exists. > > I was just experimenting with the "O" flag in dovecot-uidlist to see how the > conversion script can be updated. I was wondering if this is only > implemented for Maildir? Yeah, for now it's only for Maildir. Probably wouldn't be difficult to implement for dbox by adding it as dbox metadata (although how to add it there? dsync can't copy that).
Re: [Dovecot] migrating/converting from system users -> virtual users
Thank you for your help, Timo. > use Dovecot v2.0's dsync I gather from your reply that it's OK to use Dovecot 2.0 utilities (eg dsync) on a dovecot (v1) installation; presumably with its own configuration file(s). > You could set mail_drop_priv_before_exec=yes ... chgrp vmail ... Yes, I think we could do that; I should have thought of it myself, thanks again. I think there was one other problem with the automatic conversion which I've now remembered: I note that the first time a user connects to th eimap service dovecot creates their (virtual) home directory for them with all the right permissions. That's great and I use the existence of that directory as an indication to our MTA that the user wants delivery into the dovecot store rather than their old system mailbox. However once I tried using the convert plugin the process fails because (it seems) the conversion tries to take place before the home directory has been created. Is there any configuration change that might change this order? Can I configure the convert plugin on LDA delivery, for example, instead of as part of the "protocol imap" section? Many thanks, Steve Platt
[Dovecot] Pop3 ordering in mdbox
On 03/04/2012 03:10 PM, Timo Sirainen wrote: BTW. The script should some day be updated for Dovecot v2.0.13+ which supports storing separate POP3 and IMAP message order. Oh, I was not aware that this feature exists. I was just experimenting with the "O" flag in dovecot-uidlist to see how the conversion script can be updated. I was wondering if this is only implemented for Maildir? Our migration process involves: 1) Converting the maildir from Courier using the Perl script 2) Converting to mdbox using dsync -R backup The POP3 ordering seems to get lost during the second step. I.e., if Dovecot is set up to server POP3 mails from a maildir having "O" flags, the POP3 ordering is as intended. After changing the configuration to mdbox format and converting the mails using dsync, the POP3 ordering is different. Is this known or am I missing something? (I tried Dovecot 2.1.1.) Cheers, Christoph -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik Technische Universität München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching
Re: [Dovecot] v2.1 latest hg: untagged reply to namespace command
On 07.03.2012 19:17, wrote e-frog:
# 2.1.1 (94de7605f50f)
1 namespace
* NAMESPACE (("" "/")("virtual/" "/")) NIL NIL
* OK Namespace completed.
Please note that the "OK Namespace completed." is send untagged.
Ok, it's working again today with 2.1.1 (7a26c427fc78).
Re: [Dovecot] dot named folders
Am 08.03.2012 17:27, schrieb Micah Anderson: > Willie Gillespie writes: > >> On 03/07/2012 12:43 PM, Micah Anderson wrote: >>> >>> When a user makes a folder called 'x.y' it actually creates a folder >>> called 'x' with a folder called 'y' inside, rather than a folder called >>> 'x.y'. I'm guessing this has to do with an internal folder separator >>> namespace configuration, but I'm a bit confused by how this works. >> >> Correct. >> Similar to how in Linux, I could create a folder >> mkdir test1/test2 >> It will create test2 inside of test1. >> >> The difference being that IMAP doesn't necessarily need the parent mailbox to >> exist, where Linux would throw an error if test1/ didn't exist first. >> >> So basically, as far as I know, you can't have a folder with a "." in the >> name >> with the namespaces you have set up. > > That makes sense, however I'm not sure that I need these namespaces any > longer if I no longer am using the maildir format (mdbox). > > In either case, it seems like the internal folder separator should not > be exposed to the user like this. What is happening now is the user gets > something other than they expect (a folder within a folder, instead of a > folder with a dot in the name) because of some unknown internal > configuration. > > If moving to mdbox is not enough to remove these namespace > configurations that cause this, then it would be good if the user was > unable to create such a folder, because it was prohibited, rather than > creating something other than they expect. > > micah > http://wiki.dovecot.org/Plugins/Listescape may help -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria
Re: [Dovecot] seeking advice: dovecot versions; mailbox formats.
Vincent Schut writes: > Debian currently has dovecot 1.2.15 in its repositories; not that much > newer... No, Debian has 1.2.15 in its /stable (squeeze)/ repositories, there are newer versions available in other Debian repositories. micah
Re: [Dovecot] dot named folders
Willie Gillespie writes: > On 03/07/2012 12:43 PM, Micah Anderson wrote: >> >> When a user makes a folder called 'x.y' it actually creates a folder >> called 'x' with a folder called 'y' inside, rather than a folder called >> 'x.y'. I'm guessing this has to do with an internal folder separator >> namespace configuration, but I'm a bit confused by how this works. > > Correct. > Similar to how in Linux, I could create a folder > mkdir test1/test2 > It will create test2 inside of test1. > > The difference being that IMAP doesn't necessarily need the parent mailbox to > exist, where Linux would throw an error if test1/ didn't exist first. > > So basically, as far as I know, you can't have a folder with a "." in the name > with the namespaces you have set up. That makes sense, however I'm not sure that I need these namespaces any longer if I no longer am using the maildir format (mdbox). In either case, it seems like the internal folder separator should not be exposed to the user like this. What is happening now is the user gets something other than they expect (a folder within a folder, instead of a folder with a dot in the name) because of some unknown internal configuration. If moving to mdbox is not enough to remove these namespace configurations that cause this, then it would be good if the user was unable to create such a folder, because it was prohibited, rather than creating something other than they expect. micah
Re: [Dovecot] OT: Distrowars - WAS: Re: seeking advice: dovecot versions; mailbox formats.
On 2012-03-08 8:53 AM, Vincent Schut wrote: But maybe you also have something useful to say on the questions I *did* ask? About dovecot versions, and/or maildir vs. dbox for example? As the subject said, I was seeking advice, not rant nor war... Yeah, sorry, and I wasn't offended, I just dislike it when someone says something like that without clarification... As for version, it is generally recommended for obvious reasons to stay within the confines of your distros package manager unless you are comfortable installing from source. I've never used Debian, so can't speak to which repos you can safely use or the implications if you do... As for what mailbox format, there is no more 'dbox', it is either sdbox (like mbox one file per folder) or mdbox (multiple files per folder) - that said, mdbox seems to be the best general purpose, but my understanding is it can complicate things if something goes wrong, but it seems to be very solid. -- Best regards, Charles
Re: [Dovecot] Shared mboxes
On 3/7/2012 3:47 PM, Stan Hoeppner wrote: On 3/6/2012 3:01 PM, Steve Campbell wrote: I've experienced that type of locked mailbox before on the old server. Users insist on accessing their email account as a pop account on their desktop with the "check for new mail every so many minutes" turned on and still keep their smartphones on while accessing it as an imap account so they can still download the files to their desktop when they return. Using IMAP on the phone and POP on the PC doesn't make any sense. Is there a (valid) reason why these people insist on this phone/IMAP and PC/POP setup? This seems seriously counter intuitive/productive. The bulk of these type users are sales staff. They use their desktop when their in the office. For years, the only type of email account we used was pop just because that was the way it was. We used horde for webmail, which read these type of accounts just fine. Once they needed email in the field, it was necessary to either set up their phones to use pop and keep email on the server so that they could download the email to their desktop, or use imap on the phones. They typically don't use any folders they've created on the imap account when accessing mail on the desktop. It would be a nightmare going to each desktop, finding a time when each and every user would have the time to allow us to change things, and switching all of the accounts. It may not seem to be a good way of doing things, but it's just the way our system here has evolved. Now that we're down to skeleton-type staffing, it's not easy to find the time and manpower to accomplish change when it "ain't broke". The occasional locked mailbox was easier to resolve that the massive change to all user's accounts. This all came about because I installed a new server to replace the old, and dovecot became the pop/imap server. So just to clarify, is it OK to have a maildir account setup on this server for these shared/imap access only accounts along with the mbox accounts already on there? Yes. With Dovecot it is possible to specify mail_location on a per user basis: http://wiki.dovecot.org/MailLocation You can even do a split mailbox type setup per user using multiple namespaces, for example specifying that INBOX use mbox with all other mail being stored in maildir format: http://wiki.dovecot.org/Namespaces Thanks for the patience and help Sure thing. Again, thanks for the help.
Re: [Dovecot] OT: Distrowars - WAS: Re: seeking advice: dovecot versions; mailbox formats.
On 03/08/2012 01:03 PM, Charles Marcus wrote: On 2012-03-08 4:56 AM, Vincent Schut wrote: The old server was running gentoo linux (which is mainly the culprit of the old dovecot version: gentoo was too much trouble to keep updating); Please stop with the FUD... I've been running gentoo for 8+ years, and it is a *breeze* to keep updated, *especially* long term (since it is a 'rolling release' type of distro)... Right. I should've known I shouln't mention anyone's favourite distro... :-) Hey, listen, sorry I offended you... its really nothing I have against gentoo, I'm sorry it might have sounded like that. It's just that I appeared not to have the time and energy to do regular updates, and when I tried to update something some months later, I had problems which I had no time and energy to start solving. Thus I decided a rolling distro was no good combination for my server and me. Which is why I will switch to a less rolling distro. That's really all there is to say about. I do still have a rolling distro which-will-not-be-named on my desktop, which I can and do update often and easy. Yes, it actually does require some minimum amount of attention from the admin, like, say, once per week or once per month updates - buy so should *any* system... and yes, it does require a little more willingness to learn and 'get your hands dirty' (especially for the installation), but it is well worth it. Yes, I have learned lots from some years with gentoo. No bad feelings. Just bad combo this time. Oh - and Portage rocks... :) Well, yes, so does granite. Or iron maiden. Or whatever. As long as you like it :-) But maybe you also have something useful to say on the questions I *did* ask? About dovecot versions, and/or maildir vs. dbox for example? As the subject said, I was seeking advice, not rant nor war... Best, Vincent.
Re: [Dovecot] seeking advice: dovecot versions; mailbox formats.
On 08.03.2012 10:56, Vincent Schut wrote: Debian currently has dovecot 1.2.15 in its repositories; not that much newer... I read in the docs about the auto-generated-from-hg debian dovecot packages for 2.0, 2.1 and 2.2. Which leaves me to the choice what version to use... OK, 2.2 is development, which leaves the choice to: 1.2.15; 2.0.x, or 2.1.x. I would appreciate any consideration or thoughts on what version to choose. On several production machines we are using dovecot from debian testing repos, so 2.0.x. It's working stable for us and is quite easy to maintain. Please be careful and very selectively install packages from testing. If possible, the package dependences should be installed from stable/security. -- Adam Szpakowski
Re: [Dovecot] dsync replication available for testing
Hi --
On 08.03.2012 12:35, Timo Sirainen wrote:
On Thu, 2012-03-08 at 11:26 +0100, Michael Grimm wrote:
You can do for example:
service config {
unix_listener config {
user = vmail
}
}
I will try that later.
It seems to me, that whenever a larger number of mails arrive on
both
servers simultaneously, the replicator gets into trouble [1]. I am
unsure if one can expect that a replicator should deal with such
stress,
though. Or?
Were these mails delivered via LMTP or dovecot-lda?
LMTP
The locks should prevent duplicates I think, so there's something
still
going wrong.
Just to be sure that I didn't misunderstand your proposed
configuration:
@mx1:
plugin {
mail_replica = remote:vm...@mx2.example.tld
}
@mx2:
plugin {
mail_replica = remote:vm...@mx1.example.tld
}
I do need to define one mail_replica plugin at each server pointing to
the
other one, correct?
Regards,
Michael
Re: [Dovecot] duplicates with multiple To/CC and sieve redirect copy
On 3/8/2012 12:56 PM, Leo Baltus wrote:
Op 23/02/2012 om 02:15:48 +0100, schreef Stephan Bosch:
The repository is at: http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate
This plugin is only a few hours old, experimental, and largely
untested, so test it thoroughly before considering to use this. Read
the INSTALL file for compile and installation instructions.
Comments are welcome.
I did some very basic testing and it seems to work fine.
The example in spec-bosch-sieve-duplicate.txt however says:
if duplicate {
fileinto :create "Trash/Duplicate";
}
This assumes the hierarchy separator is '/', but in Maildir this defaults to '.'
So this leads to:
failed to store into mailbox 'Trash/Duplicate': Invalid mailbox name
I am not sure if this a bug or not, I suppose you know the rfc's better
than I do, is the sieve language supposed to be agnostic of the
internals of the storage-engine (dovecot)?
For Sieve, the mailbox name is pretty much opaque. Usually, it matches
what is used through IMAP.
http://tools.ietf.org/html/rfc5228#section-4.1
So, in your case, just use "Trash.Duplicate" instead.
Regards,
Stephan.
[Dovecot] OT: Distrowars - WAS: Re: seeking advice: dovecot versions; mailbox formats.
On 2012-03-08 4:56 AM, Vincent Schut wrote: The old server was running gentoo linux (which is mainly the culprit of the old dovecot version: gentoo was too much trouble to keep updating); Please stop with the FUD... I've been running gentoo for 8+ years, and it is a *breeze* to keep updated, *especially* long term (since it is a 'rolling release' type of distro)... Yes, it actually does require some minimum amount of attention from the admin, like, say, once per week or once per month updates - buy so should *any* system... and yes, it does require a little more willingness to learn and 'get your hands dirty' (especially for the installation), but it is well worth it. Oh - and Portage rocks... :) -- Best regards, Charles
Re: [Dovecot] duplicates with multiple To/CC and sieve redirect copy
Op 23/02/2012 om 02:15:48 +0100, schreef Stephan Bosch:
> On 2/22/2012 12:15 AM, Adam Szpakowski wrote:
> >On 22.02.2012 00:09, Timo Sirainen wrote:
> >>Well, it would be possible to build a doveadm script that
> >>deletes the duplicates after delivery, but currently there's no
> >>implementation to avoid delivering duplicate Message-IDs in the
> >>first place.
> >>
> >>I don't really like such a Message-ID-based deduplication
> >>feature enabled by default, but something like this could be
> >>nice:
> >>
> >>fileinto :copy :x-deduplicate "boss";
> >>
> >>Anyway, probably not going to be implemented anytime soon.
> >Maybe there is a way to use a procmail with something like this:
> >
> >:0 Wh: msgid.lock
> >| formail -D 8192 .msgid.cache
> >
> >But is there a safe way to use it together with sieve? Using
> >Pigeonhole Sieve Pipe Plugin?
> >
>
> There are a few options:
>
> * You can use Procmail as primary delivery agent and invoke
> dovecot-lda/sieve from within Procmail once Procmail has determined
> that it is not a duplicate.
>
> * Invoke procmail from Sieve using the pipe extension (i.e. the
> other way around). This has the disadvantage that Procmail will
> have to take care of final delivery, meaning the Dovecot indexes are
> not updated.
>
> * For Pigeonhole v0.3 there is the possibility to "filter" the
> message through Procmail using the sieve_extprograms plugin, but I
> haven't actually tested something like that.
>
> * I've just created an alternative that implements something similar
> to the Procmail code you posted above, but from within Sieve itself.
> It is a custom language extension called vnd.dovecot.duplicate and
> it adds the "duplicate" test. This test keeps track of which
> Message-IDs it has seen before in earlier deliveries and yields a
> true result if the message was seen before, e.g.:
>
> require "vnd.dovecot.duplicate";
>
> if duplicate {
> discard;
> }
>
> Read the specification for details ("name" argument is not yet implemented):
>
> http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate/raw-file/4b1dbda4d3fc/doc/rfc/spec-bosch-sieve-duplicate.txt
>
> The repository is at: http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate
>
> This plugin is only a few hours old, experimental, and largely
> untested, so test it thoroughly before considering to use this. Read
> the INSTALL file for compile and installation instructions.
>
> Comments are welcome.
>
I did some very basic testing and it seems to work fine.
The example in spec-bosch-sieve-duplicate.txt however says:
if duplicate {
fileinto :create "Trash/Duplicate";
}
This assumes the hierarchy separator is '/', but in Maildir this defaults to '.'
So this leads to:
failed to store into mailbox 'Trash/Duplicate': Invalid mailbox name
I am not sure if this a bug or not, I suppose you know the rfc's better
than I do, is the sieve language supposed to be agnostic of the
internals of the storage-engine (dovecot)?
--
Leo Baltus, internetbeheerder /\
NPO ICT Internet Services/NPO/\
Sumatralaan 45, 1217 GP Hilversum, Filmcentrum, west \ /\/
beh...@omroep.nl, 035-6773555 \/
Re: [Dovecot] dsync replication available for testing
On Thu, 2012-03-08 at 11:26 +0100, Michael Grimm wrote:
> Let me start with replicator's configuration ...
>
> > Below is a configuration for virtual user setup.
> [...]
> > service doveadm {
> > # if you're using a single virtual user, set this to
> > # start ssh as vmail (not root)
> > user = vmail
> > }
>
> ... that led to the following complaints at start-up:
>
> | dovecot: master: Dovecot v2.1.1 (d66568d34e40) starting up
> | dovecot: doveadm: Error: Error reading configuration:
> net_connect_unix(/var/run/dovecot/config) failed: Permission denied
> | [...]
> | (repeatedly, presumably for the number of users in userdb?)
You can do for example:
service config {
unix_listener config {
user = vmail
}
}
> Now some observations regarding replicator:
>
> 1) I see a lot of error messages whenever replicator is in action
> like (although everything is being synced correctly):
>
> | mail dovecot: dsync-local(test): Error: remote:
> dsync-remote(test): Info: save: box=INBOX, uid=27,
> msgid=<3v2jfh5kv4z...@example.tld>, size=547, from=t...@example.tld
> (admin), flags=()
>
> | mail dovecot: dsync-local(test): Error: remote:
> dsync-remote(test): Info: flag_change: box=TEST, uid=27568,
> msgid=<20120307144810.6360a74f...@example.tld>, size=435,
> from=t...@example.tld, flags=(\Seen)
>
> JFTR: I do have mail_log plugin activated.
Hmm. Right. I guess all the logging should go to the log files instead
of via the ssh pipe. Of course that would also require that dsync has
write access to your log files.
> It seems to me, that whenever a larger number of mails arrive on both
> servers simultaneously,
> the replicator gets into trouble [1]. I am unsure if one can expect
> that a replicator should
> deal with such stress, though. Or?
Were these mails delivered via LMTP or dovecot-lda?
The locks should prevent duplicates I think, so there's something still
going wrong.
Re: [Dovecot] dsync replication available for testing
Hi --
On 04.03.2012 11:44, Timo Sirainen wrote:
In dovecot-2.1 hg you can now test dsync-based replication.
Everything isn't finished yet, but it appears to work and I've
enabled
it for my @dovecot.fi mails.
I did give it a try starting some days ago, and I can confirm that you
are right,
dsync replication can be used, but there are some issues, see below.
Let me start with replicator's configuration ...
Below is a configuration for virtual user setup.
[...]
service doveadm {
# if you're using a single virtual user, set this to
# start ssh as vmail (not root)
user = vmail
}
... that led to the following complaints at start-up:
| dovecot: master: Dovecot v2.1.1 (d66568d34e40) starting up
| dovecot: doveadm: Error: Error reading configuration:
net_connect_unix(/var/run/dovecot/config) failed: Permission denied
| [...]
| (repeatedly, presumably for the number of users in userdb?)
Therefore, I modified dsync_remote_cmd ...
dsync_remote_cmd = ssh -p 1234 -l vmail %{host} doveadm dsync-server
-u%u -l%{lock_timeout} -n%{namespace}
... and used an empty 'service doveadm { }' instead. That worked, but I
would
love to run doveadm as vmail user (security), though. How should I do
that without
running into the error messages above?
Now some observations regarding replicator:
1) I see a lot of error messages whenever replicator is in action
like (although everything is being synced correctly):
| mail dovecot: dsync-local(test): Error: remote:
dsync-remote(test): Info: save: box=INBOX, uid=27,
msgid=<3v2jfh5kv4z...@example.tld>, size=547, from=t...@example.tld
(admin), flags=()
| mail dovecot: dsync-local(test): Error: remote:
dsync-remote(test): Info: flag_change: box=TEST, uid=27568,
msgid=<20120307144810.6360a74f...@example.tld>, size=435,
from=t...@example.tld, flags=(\Seen)
JFTR: I do have mail_log plugin activated.
Some testing results:
1) I ran a test by sending locally produced mails every other minute on
both servers
simultaneously. That test ran for ~5 hours. All mails became synced
correctly, and
no losses were observable, but some duplicates.
2) I did send 100 small test mails from a distant server to my
mailservers (mx1 and mx2):
a) replicator and dsync deactivated: received 100 distinct mails (57
at mx1, 43 at mx2).
b) now, replicator active: 172 mails (100 distinct, a lot of
duplicates (up to 8
incarnations of the very same mail).
Ok, 2b) is a rather 'mailbomb-like' scenario, but it worries me a
bit: One of my users
is receiving mails from a mailing list that sends individual mails
batch-wise ...
3) replicator active: 1000 mails sent ended in 4523 mails at every
server. Well, that was
a mailbomb :-)
4) replicator active: 100 (and even 1000) locally produced mails at one
server only: all
100 (and 1000 mails) became synced, prefectly well, without
duplicates.
5) replicator active: 100 locally produced mails at both servers
simultaneously: 341 mails,
thus a lot of multiple incarnations.
(This test differed from 1) because all mails were sent in one
batch.)
Final note to these tests: It doesn't matter whether sieve with
redirecting, or sieve with
redirecting and copying, or no sieve at all has been involved.
It seems to me, that whenever a larger number of mails arrive on both
servers simultaneously,
the replicator gets into trouble [1]. I am unsure if one can expect
that a replicator should
deal with such stress, though. Or?
Résumé: The overall performance of replicator is very good from my
point of view for my
conditions (handful users, average workload of roughly 1000
mails a day).
Thank you for replicator and regards,
Michael
[1] JFTR: I did similar tests in the past with dsync running from cron
every other minute
with similar results.
Re: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror
HI -- On 05.03.2012 10:56, Timo Sirainen wrote: On 4.3.2012, at 13.54, Timo Sirainen wrote: On 4.3.2012, at 13.41, Michael Grimm wrote: By "undeletable" do you mean you have mails that always come back after expunging them? Yes. Deleting by the client will return them after the next dsync run. Luckily this just started happening to me as well. After some debugging I found and fixed the problem: http://hg.dovecot.org/dovecot-2.1/rev/f549cd60fec9 I can confirm, that you fixed that issue successfully. Thanks and regards, Michael
[Dovecot] seeking advice: dovecot versions; mailbox formats.
Hi, I'm currently migrating our old (colocated) mail server (running a [terribly outdated, I know] dovecot 1.1.11) to a new VPS (virtual private server). The old server was running gentoo linux (which is mainly the culprit of the old dovecot version: gentoo was too much trouble to keep updating); the new server will run debian (stable: 6). Debian currently has dovecot 1.2.15 in its repositories; not that much newer... I read in the docs about the auto-generated-from-hg debian dovecot packages for 2.0, 2.1 and 2.2. Which leaves me to the choice what version to use... OK, 2.2 is development, which leaves the choice to: 1.2.15; 2.0.x, or 2.1.x. I would appreciate any consideration or thoughts on what version to choose. On a related note, there is the possibility to switch from maildir to dbox. I did not really find much pros or cons, except from performance and standards-compliance (ability to use e.g. mutt on the server itself). Any thoughts? About the server: we're just a small company. Think about 15 accounts, normal mail traffic, sometimes relatively large attachments (20mb+). Some accounts have many folders; some accounts are very large (5Gb+). Storage is on ext3, raid10. Performance has never been an issue; reliability and ease of maintenance is more important. Thanks, Vincent Schut.
