Re: [Dovecot] Converting a POP3 client to IMAP
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 12 Feb 2013, Kristen J. Webb wrote: Our current email is outsourced and so we have used POP3 to keep all email on our systems for many years. Is there a best practice for moving mail such that we can take advantage of the hard link feature for all those older copies that are currently in POP3 files? It would also be nice to be able to auto file things. I don't see the sieve filters being activated when I copy mail in Thunderbird. you could use fetchmail to get the messages from the POP3 box and pass them to Dovecot deliver. So you can use Sieve filters (or not, if you turn Sieve off for this). Then let the messages spool to Maildir. Then use fdupes to find duplicates and hardlink them. Kind regards, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURtHEV3r2wJMiz2NAQLv7wf/afBQrT5EvVzXux4zoPo/ZJTt8upHaueV fQiuIdqmzGSiOgrXpZyxOcL9fsBFOQYuYVzPDyISbOdUj8Pf+9pjTdkDlRDKiOYC IXBWj/7I5x64V1Mpy+4TkSnqA8NSB3Tzu0Rp0xOwTa73QwSqjuVw7oO0oqD4nLWj 5CEvk6laEmEsEO/XrlEcuA9OzK32oztx+CxCnTWPrH30ARL4GfckObFkiPOR57XD JXKVJ+PigvPdsUK7aT1/V8ow4an9cjfsCwr4uxeMprfdHfoyu6ghphVl1iB8wCDi Gt7kDU70hR4eK7wArl9J3RLnsr0Ap5o8EHXl+OqLFkM5NCsKYdqjDQ== =9qoB -END PGP SIGNATURE-
Re: [Dovecot] backing up maildir dovecot files
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 12 Feb 2013, David Mehler wrote: Thanks, if I did: rsync --force --delete-after -tPrlHpogEAXz /folder/ /backups/newmailstore/ could I then tar up the newmailstore folder with something like tar zcf without messing anything up? keep in mind, that the backup of the Maildir with rsync is no 100% consistent snapshot, because of the filename renames; otherwise: yes. On 2/12/13, Steffen Kaiser wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 11 Feb 2013, Reindl Harald wrote: Am 11.02.2013 22:37, schrieb Steve Litt: On Mon, 11 Feb 2013 21:47:57 +0100 Reindl Harald wrote: Am 11.02.2013 21:34, schrieb David Mehler: I'm having to migrate servers. Both are dovecot2 systems. I'm wanting to copy over my mail store from one system to the other. I'd like to preserve dates/times of emails. These are maildir setups on both boxes, I'd like to be as transparent to the end user as possible. I currently have the first dovecot system offline and the second dovecot system is offline. assuming that BOTH servers are down as fro any other transfers like mysql-datadirs and any critical things which should go save and fast by preserve attributes: rsync --force --delete-after -tPrlHpogEAXz /folder/ root@newserver:/folder/ # -z compress # -t timestamps # -P progress # -r recursive # -l links # -H hard-links # -p permissions # -o owner # -g group # -E executability # -A acls # -X xtended attributes That's how I'd do it also. I think if you didn't mind including -D, which from my understanding is --special and --devices, neither of which I'd expect in a maildir, and if you wanted to do your deletion manually after the fact, would this be equivalent? well, i have them ususally in my "rsync.sh" but did not expect such files in a maildir, but yes it does not hurt rsync -PaHAX /folder/ The man page says -a (--archive) is equivalent to -rlptgoD, and you have all but D listed up there. My thinking on later manually deleting the old maildir, instead of letting rsync do it, is that if somehow, some weird thing goes wrong, I have the old one for backup. Who knows, maybe I copied the thing on top of the wrong other maildir and have to back it out -- I'd have the old maildir as a reference of which files. you missunderstand "--delete-after" this is for delete files in the TARGET folder which is not or no longer in the source and IMHO very very important if you want sync folders 1:1 because old artefacts can have very bad effects without you merge folders and if i know "hey my source contains exactly what i need, not more and lot less" this is not what i would like and never did in 10 years IT If you use "--delete-after" you could add "--fuzzy", which theoretically could save transfers because of filename renames. Anyway: Any form of "--delete" is required for Maildir, IMHO, because Labels, Tags, Keywords (or whatever the MUA calls it) and status information (seen, read, deleted) are reflected by the filename. Hence, if one does not "--delete", the _same_ message might pop up in the Maildir multiple times but with different status and/or tags. Same applies to new messages, because they are storred in /new and later moved to /cur. So one ends with the same message in "new" and in "cur". - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURn+wV3r2wJMiz2NAQJYnwgAwGfCHBFglm800nbaHFhsdWeuVkhGJjnW l0QtbmQLaSdQOkWwdVkmNEQkJLmwvfhddG0a9GblofI/zn8EBN+3EDR/CM0d5Y8f jy1yqemT26sFTVC6NlTqhDF8zq/Oi5WK7ftUfYfrt90MtmMD1rzmBQ5Q/N+Tteae TeE0jeHNL0rHl9DoAV9AkBzhJPnYYodSK2lA1oewazlzRpzJSiHYmgVh4RiSmsyk 4DAtch5ZrqLnsx1A/mUHAfDvdk66j2Os0bLXqbHu9ZAzz5Xb9bbNfpu52u1Ukg/o TfXfiGA4aRix6I1OGePdXmi/qXTzPVLBzhhIZdMiJ2CKCWNFbIYC7w== =ugv1 -END PGP SIGNATURE- - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURs+tV3r2wJMiz2NAQIvngf/c37Hbg30lIq0wFPCif9IAG+BO8yDRAa3 lYY2t7DlszWopeEBvV22HjUC8SlzywECYEhBFRWhJVfqNyu55oI867of+RZ6lO0A lNcThmNixX5IsFiUnN9S9NYrTENC2qhccPdrTCAcm6A6CtaR1ydyeZDxlmvmTWe4 Za60LVqlsIVoZQ146yGE5nSIctix8JOE5kfGO5NurYXTfHt9CrPj5JvlPmRUdasp Mtn+QpxaxzFa8NGECSThKDVilMPFvqfT+JrhjWnkM1v31hNjJj4F3DYhk1L0YlAL 4wUjskXq1ytoM8+k9c8rqCy5DvM7g8N2ip0c0Buvjt+RctaHE4F9jg== =moaG -END PGP SIGNATURE-
Re: [Dovecot] FreeBSD, Dovecot and ZFS
Am 12.02.2013 22:25, schrieb Jason Lock: > Mail Issues - FreeBSD > > Hello, my apologies if this may be the wrong forum but hoping that maybe > someone might be able to provide some insight. > > Have a very sporadic and strange issue with our mail servers running Dovecot > on FreeBSD. There are three servers hosting Dovecot with FreeBSD as the > underlying operating system. All three connect to a NAS server, again > running FreeBSD and ZFS. > > When the specific issue occurs, clients connecting to check mail via POP3 or > IMAP experience long delays and timeouts. To the point where POP3 Logins > fail due to the timeouts. The issue is further compounded by clients > increasing the number of attempts to check mail. > > Part of the frustration in attempting to diagnose the issue is not knowing > the root cause or symptom that initates the issue. > > Wondering if anyone has experienced anything similar, or suggestions on ways > to help identify the root cause > show your debug logs, dovecot confs, nfs setups are special, what did you setup to honor nfs Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] FreeBSD, Dovecot and ZFS
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 2/12/13 1:25 PM, Jason Lock wrote: > Mail Issues - FreeBSD > > Hello, my apologies if this may be the wrong forum but hoping that > maybe someone might be able to provide some insight. > > Have a very sporadic and strange issue with our mail servers > running Dovecot on FreeBSD. There are three servers hosting > Dovecot with FreeBSD as the underlying operating system. All three > connect to a NAS server, again running FreeBSD and ZFS. Speaking for NFS on FreeBSD, please note that the FreeBSD NFS client and server generally respects application's sync() operations which could cause delays, if the backing pool is not configured with any fast ZIL device, ideally a SSD. Normally a 'gstat -I 1s' on the NFS server will give you busy disks with small throughput, in this case, a ZIL would be helpful. Before buying a SSD, you may want to try disabling 'sync' on dataset (be careful though, that doing so would increase the chance of data loss in the event of power loss, make sure to revert the change with 'zfs inherit sync' on the datasets once you are done with experiments) and see if that helps, if so, then a SSD ZIL would be helpful; higher end systems can use a Fusion IO or ZeusRAM for that purpose. Another possible cause is compression, at this time, FreeBSD's ZFS compression thread runs at kernel priority and that would cause the system to do ZFS compression before doing everything else. If you use compression=gzip, you will see stalls more often, for short term, please consider using compression=lzjb or have dovecot to do the compression; for mid term, please use my recent LZ4 port that would consume way less CPU and have better compression ratio than lzjb; for long term, we are going to implement a scheduling policy which is similar to Solaris's System Duty Cycle and have the compatibility shims to emulate that. Lastly, try increasing your NFS iod numbers. The system default is very conservative and may become a bottleneck when you have a lot of users on one system. Hope this helps... If you have further questions about FreeBSD please consider posting to freebsd-sta...@freebsd.org . Cheers, -BEGIN PGP SIGNATURE- iQEcBAEBCAAGBQJRGuGyAAoJEG80Jeu8UPuzWdYH/iybDczQnLzvlzKfrGn/Kdm7 197P2wK4phktNJVrNjYKFYc8CqelcLoSfZiRGDc3CxpmKnrwRkUWnZMXfpYUVJVL 3SSpmNY73h4atrEpyyoKmz4tIxJp7c24IB8201j/vc09yEaooy2dc9pJrJ5cAcjz d7eSidcQQk8VdqCBk8haDpB2igZZXWz8zgZ8mKPcqSkGuxZkzZwZCkZc7XfPsCGK GKswLtXutHA19mKs+wBLJ1r+ZQxabI8ZwQi1V2+VBFz4a5edkQJVbW6VvFSmzjO9 HfbW0uO+wJztSHiceKEUiJ7bIm5ygolhA2BvZ5WlY2VkejXbN6nfQkOlvmDeS0s= =sMYy -END PGP SIGNATURE-
Re: [Dovecot] Marking messages read / retaining date with Sieve
At 3PM -0500 on 12/02/13 you (Bret Martin) wrote:
>
> To close the loop on this, I ended up doing the following, although it
> seems to fail on mailboxes with large numbers of messages (on the
> order of 10,000 or so; I didn't test carefully enough to find the
> exact number)
>
> use Mail::IMAPClient;
> use Socket;
> use strict;
>
> socketpair( my $dovecot, my $client, AF_UNIX, SOCK_STREAM, PF_UNSPEC );
>
> unless ( fork() ) {
> open( STDIN, '<&', $client );
> open( STDOUT, '>&', $client );
> exec( '/usr/lib/dovecot/imap' );
> }
> close( $client );
>
> my $imap = Mail::IMAPClient->new( Socket => $dovecot );
>
> foreach my $folder( sort $imap->folders() ) {
> print( "$folder\n" );
> $imap->select( $folder );
> $imap->set_flag( 'Seen', $imap->search( 'ALL' ) );
> }
You're not specifying Ranges in the ->new call, and you're calling
->search in list context; either of these will force ->search to return
a complete list of message numbers, which you are then trying to pass
back in a single IMAP command. You want to either loop through the
messages or set Ranges and call ->search in scalar context to get an
(object which stringifies to an) IMAP range specification, which will be
much shorter than the whole list.
Ben
Re: [Dovecot] FreeBSD, Dovecot and ZFS
At 9PM + on 12/02/13 you (Jason Lock) wrote: > Mail Issues - FreeBSD > > Hello, my apologies if this may be the wrong forum but hoping that > maybe someone might be able to provide some insight. This may turn out to be something better addressed on freebsd-stable, but this is a perfectly good place to start. > Have a very sporadic and strange issue with our mail servers running > Dovecot on FreeBSD. There are three servers hosting Dovecot with > FreeBSD as the underlying operating system. All three connect to a > NAS server, again running FreeBSD and ZFS. Over NFS, I assume? What version, what mount options, and what type of authentication? What locking strategies is Dovecot using? Are there any suspicious messages in syslog on either machine? > When the specific issue occurs, clients connecting to check mail via > POP3 or IMAP experience long delays and timeouts. To the point where > POP3 Logins fail due to the timeouts. The issue is further compounded > by clients increasing the number of attempts to check mail. Are the delays happening before or after login? If you can provoke this and get a 'procstat -k' for the relevant dovecot process this might be helpful. If 'long' delays means several minutes, running something along the lines of 'procstat -k $(pgrep -U dovecot -U doveauth)' every minute or so for a while might be one way to catch this, though this will collect a lot of data rather fast so you will need some way to locate the relevant entry. Ben
[Dovecot] Converting a POP3 client to IMAP
I have a dovecot 2.0 dovecot server set up for https/imap with spam, av, and some preliminary sieve filters in place. Our current email is outsourced and so we have used POP3 to keep all email on our systems for many years. I have run tests to move mail to the new imap server and that seems to work pretty well. However, we have lots of mail going back over a decade and used a lot of aliases that forwarded email to many users. I find the moving mail multiple times using Thunderbird results in multiple copies of the same email. Is there a best practice for moving mail such that we can take advantage of the hard link feature for all those older copies that are currently in POP3 files? It would also be nice to be able to auto file things. I don't see the sieve filters being activated when I copy mail in Thunderbird. Many thanks in advance for any guidance. Kris -- This message is NOT encrypted Mr. Kristen J. Webb Chief Technology Officer Teradactyl LLC. 2450 Baylor Dr. S.E. Albuquerque, New Mexico 87106 Phone: 1-505-338-6000 Email: kw...@teradactyl.com Web: http://www.teradactyl.com Providers of Scalable Backup Solutions for Unique Data Environments NOTICE TO RECIPIENTS: Any information contained in or attached to this message is intended solely for the use of the intended recipient(s). If you are not the intended recipient of this transmittal, you are hereby notified that you received this transmittal in error, and we request that you please delete and destroy all copies and attachments in your possession, notify the sender that you have received this communication in error, and note that any review or dissemination of, or the taking of any action in reliance on, this communication is expressly prohibited. Regular internet e-mail transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate, and it should not be relied upon as such. If you prefer to communicate with Teradactyl LLC. using secure (i.e., encrypted and/or digitally signed) e-mail transmission, please notify the sender. Otherwise, you will be deemed to have consented to communicate with Teradactyl via regular internet e-mail transmission. Please note that Teradactyl reserves the right to intercept, monitor, and retain all e-mail messages (including secure e-mail messages) sent to or from its systems as permitted by applicable law.
Re: [Dovecot] sieve /editheader / addheader :last crash
On 2/9/2013 8:20 PM, Peter Rindfuss wrote:
Am 2013-02-09 09:27, schrieb Stephan Bosch:
I'm unable to reproduce this so far. Could you show me more details of
your setup?
* Output from: dovecot -n
* Your exact test message
* Your exact test Sieve script
* The sieve-test command line
Hi Stephan,
attached are "dovecot -n", a test message, and three very similar
scripts one of which causes the crash and two not.
I'm inclined to say that the message itself does not matter.
This ist the sieve-test cmd line:
sieve-test -t - -a 'b...@wzb.eu' -l /wzb/mail/brot/Maildir -e -x
+editheader /wzb/mail/brot/sieve/{bad|good1|good2}.sieve mail
Initially I encountered the problem in real deliveries, not in
sieve-test. It is always reproducible.
I will take another look at this in a few days.
Regards,
Stephan.
[Dovecot] Is there any way to mask or hide the INBOX label for Dovecot 1.2.x Shared Mailboxes?
Hello, I'm setting up a new Dovecot sever to replace an old Cyrus server, and need to keep things as consistent as possible. I have a couple local accounts, like junkmail and notjunkmail that users get shared access to, and can drop things in to train sa-update. However, when users share over junkmail's inbox, it's displayed in the hierarchy: • Other Users • junkmail • INBOX Is there any way to get junkmail's INBOX to just show up at Other Users/junkmail and not require that extra depth? • Other Users • junkmail I'm running Debian Squeeze and the standard dovecot package available therein: > # uname -a > Linux internet.digitalquay.com.au 2.6.32-5-xen-amd64 #1 SMP Sun Sep 23 > 13:49:30 UTC 2012 x86_64 GNU/Linux > #dovecot --version > 1.2.15 I'm running in a configuration where all mail is owned by the vmail user, due to the impression I'm under, that it's a better way if there is to be shared mailboxes. Relevant bits of dovecot -n: > mail_uid: vmail > mail_gid: vmail > mail_location: maildir:/var/spool/dovecot/user/%u > mail_plugins: acl imap_acl > namespace: > type: private > separator: / > inbox: yes > list: yes > subscriptions: yes > namespace: > type: shared > separator: / > prefix: Other Users/%%u/ > location: > maildir:/var/spool/dovecot/user/%%u:INDEX="/var/spool/dovecot/user/%u/Other\ > Users/%%u" > list: children > auth default: > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: passwd > args: uid=vmail gid=vmail blocking=yes > mail=maildir:/var/spool/dovecot/user/%u Am I barking up the wrong tree, or is this possible? Thanks, Jay
Re: [Dovecot] FreeBSD, Dovecot and ZFS
On 2013-02-12 4:25 PM, Jason Lock wrote: When the specific issue occurs, clients connecting to check mail via POP3 or IMAP experience long delays and timeouts. To the point where POP3 Logins fail due to the timeouts. The issue is further compounded by clients increasing the number of attempts to check mail. WAG would be similar issues you can face when using NFS with multiple servers accessing it (file locking issues). The solution would be to use Director to make sure users are always directed to the same server. http://wiki2.dovecot.org/Director If that isn't the problem, then much more info would be needed (ie, doveconf -n output, logs exhibiting the problem, etc)... -- Best regards, */Charles /*
Re: [Dovecot] Imap process crash: assertion failed: (full_fs_access)
On Mon, 11 Feb 2013, Joseph Tam wrote:
Timo Sirainen writes:
> Any guess at what would cause this?
> > Feb 7 21:20:53 server dovecot: imap(user): Panic: file
>mailbox-list-fs-iter.c: line 447 (fs_list_get_roots): assertion
>failed: (full_fs_access)
LIST command that had invalid parameters that somehow got through the
first validity check. What namespace config do you have?
I got more info from the user that had this problem. He reported that he
was was setting up a new Email client (Nexus4 Android) and he sent me a
screenshot of his setup. Everything look ordinary except perhaps the
IMAP prefix set to "mail".
...
I've asked the user to change his prefix, and he'll probably report back.
He reported that removing the mail prefix solved it for him. I then
proceeded to update dovecot from 2.1.14 -> 2.1.15, and asked him to
change it back to the previoux prefix ("mail"). That also worked.
So I don't know what unwedged him, but it's working now.
Joseph Tam
[Dovecot] FreeBSD, Dovecot and ZFS
Mail Issues - FreeBSD Hello, my apologies if this may be the wrong forum but hoping that maybe someone might be able to provide some insight. Have a very sporadic and strange issue with our mail servers running Dovecot on FreeBSD. There are three servers hosting Dovecot with FreeBSD as the underlying operating system. All three connect to a NAS server, again running FreeBSD and ZFS. When the specific issue occurs, clients connecting to check mail via POP3 or IMAP experience long delays and timeouts. To the point where POP3 Logins fail due to the timeouts. The issue is further compounded by clients increasing the number of attempts to check mail. Part of the frustration in attempting to diagnose the issue is not knowing the root cause or symptom that initates the issue. Wondering if anyone has experienced anything similar, or suggestions on ways to help identify the root cause
Re: [Dovecot] Marking messages read / retaining date with Sieve
On Jan 30, 2013, at 8:39 PM, Ben Morrow wrote:
[...]
> Mail::IMAPClient (which I usually prefer) will work with a
> preauthenticated socket, but you need to create a socketpair explicitly,
> fork and exec dovecot/imap with one end of the pair on STDIN/STDOUT,
> then pass the other end to Mail::IMAPClient->new as the Socket
> parameter. Something like this (I've left out error checking)
[...]
Thanks again for your help!
To close the loop on this, I ended up doing the following, although it seems to
fail on mailboxes with large numbers of messages (on the order of 10,000 or so;
I didn't test carefully enough to find the exact number)
use Mail::IMAPClient;
use Socket;
use strict;
socketpair( my $dovecot, my $client, AF_UNIX, SOCK_STREAM, PF_UNSPEC );
unless ( fork() ) {
open( STDIN, '<&', $client );
open( STDOUT, '>&', $client );
exec( '/usr/lib/dovecot/imap' );
}
close( $client );
my $imap = Mail::IMAPClient->new( Socket => $dovecot );
foreach my $folder( sort $imap->folders() ) {
print( "$folder\n" );
$imap->select( $folder );
$imap->set_flag( 'Seen', $imap->search( 'ALL' ) );
}
For the moment the failure on large folders was easier to handle by just doing
those folders manually, although I'm curious if anyone knows the reason that
might fail.
--Bret
Re: [Dovecot] POLL: v2.2 to allow one mail over quota?
Le 11 févr. 2013 à 02:16, Timo Sirainen a écrit : > [...] > > I wonder if there's a better name for this than "last_extra".. Hello Timo, If I've correctly understood, the idea is to accept an additional message provided there is some free space AND the size of that message additional doesn't exceed above last_extra. An additional message, even if over-quota strictly speaking, might thus be tolerated. Or the idea of an absolute quota may thus come with some tolerance. quota_tolerance? Axel
Re: [Dovecot] backing up maildir dovecot files
Hello, Thanks, if I did: rsync --force --delete-after -tPrlHpogEAXz /folder/ /backups/newmailstore/ could I then tar up the newmailstore folder with something like tar zcf without messing anything up? Thanks. Dave. On 2/12/13, Steffen Kaiser wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On Mon, 11 Feb 2013, Reindl Harald wrote: > >> Am 11.02.2013 22:37, schrieb Steve Litt: >>> On Mon, 11 Feb 2013 21:47:57 +0100 >>> Reindl Harald wrote: Am 11.02.2013 21:34, schrieb David Mehler: > I'm having to migrate servers. Both are dovecot2 systems. I'm > wanting to copy over my mail store from one system to the other. > I'd like to preserve dates/times of emails. These are maildir > setups on both boxes, I'd like to be as transparent to the end user > as possible. I currently have the first dovecot system offline and > the second dovecot system is offline. assuming that BOTH servers are down as fro any other transfers like mysql-datadirs and any critical things which should go save and fast by preserve attributes: rsync --force --delete-after -tPrlHpogEAXz /folder/ root@newserver:/folder/ # -z compress # -t timestamps # -P progress # -r recursive # -l links # -H hard-links # -p permissions # -o owner # -g group # -E executability # -A acls # -X xtended attributes >>> >>> That's how I'd do it also. I think if you didn't mind including -D, >>> which from my understanding is --special and --devices, neither of >>> which I'd expect in a maildir, and if you wanted to do your deletion >>> manually after the fact, would this be equivalent? >> >> well, i have them ususally in my "rsync.sh" but did not expect >> such files in a maildir, but yes it does not hurt >> >>> rsync -PaHAX /folder/ >>> >>> The man page says -a (--archive) is equivalent to -rlptgoD, and you >>> have all but D listed up there. >>> >>> My thinking on later manually deleting the old maildir, instead of >>> letting rsync do it, is that if somehow, some weird thing goes wrong, I >>> have the old one for backup. Who knows, maybe I copied the thing on top >>> of the wrong other maildir and have to back it out -- I'd have the old >>> maildir as a reference of which files. >> >> you missunderstand "--delete-after" >> >> this is for delete files in the TARGET folder which is not >> or no longer in the source and IMHO very very important >> if you want sync folders 1:1 because old artefacts can have >> very bad effects >> >> without you merge folders and if i know "hey my source contains >> exactly what i need, not more and lot less" this is not what i >> would like and never did in 10 years IT > > If you use "--delete-after" you could add "--fuzzy", which theoretically > could save transfers because of filename renames. > > Anyway: Any form of "--delete" is required for Maildir, IMHO, because > Labels, Tags, Keywords (or whatever the MUA calls it) and status > information (seen, read, deleted) are reflected by the filename. Hence, if > one does not "--delete", the _same_ message might pop up in the Maildir > multiple times but with different status and/or tags. Same applies to new > messages, because they are storred in /new and later moved to > /cur. So one ends with the same message in "new" and in "cur". > > - -- > Steffen Kaiser > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.11 (GNU/Linux) > > iQEVAwUBURn+wV3r2wJMiz2NAQJYnwgAwGfCHBFglm800nbaHFhsdWeuVkhGJjnW > l0QtbmQLaSdQOkWwdVkmNEQkJLmwvfhddG0a9GblofI/zn8EBN+3EDR/CM0d5Y8f > jy1yqemT26sFTVC6NlTqhDF8zq/Oi5WK7ftUfYfrt90MtmMD1rzmBQ5Q/N+Tteae > TeE0jeHNL0rHl9DoAV9AkBzhJPnYYodSK2lA1oewazlzRpzJSiHYmgVh4RiSmsyk > 4DAtch5ZrqLnsx1A/mUHAfDvdk66j2Os0bLXqbHu9ZAzz5Xb9bbNfpu52u1Ukg/o > TfXfiGA4aRix6I1OGePdXmi/qXTzPVLBzhhIZdMiJ2CKCWNFbIYC7w== > =ugv1 > -END PGP SIGNATURE- >
[Dovecot] I can't sync with dsync and imapc
Hello, I am deploying a new dovecot server to replace and old one, both of the with dovecot 2.1.9. Now I want to synchronize imap mailbox from the old to the new one. So, in my new server I run (whith a $ sudo dsync -v -D -o imapc_host="" -o imapc_password="" -o imapc_features=rfc822.size backup -R -f -u myu...@um.es imapc: doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_backend_rescan (this is usually intentional, so just ignore this message) doveadm(myu...@um.es): Debug: Added userdb setting: plugin/quota_rule=*:storage=10G doveadm(myu...@um.es): Debug: Effective uid=2012, gid=204, home=/home/buzones/b1/an/myuser doveadm(myu...@um.es): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled doveadm(myu...@um.es): Debug: Quota root: name=User quota backend=dict args=:file:/home/buzones/b1/an/myuser/quota doveadm(myu...@um.es): Debug: Quota rule: root=User quota mailbox=* bytes=10737418240 messages=0 doveadm(myu...@um.es): Debug: Quota rule: root=User quota mailbox=Trash bytes=+1073741824 messages=0 doveadm(myu...@um.es): Debug: dict quota: user=myu...@um.es, uri=file:/home/buzones/b1/an/myuser/quota, noenforcing=0 doveadm(myu...@um.es): Debug: Namespace : type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/home/buzones/b1/an/myuser/mdbox:INDEX=/home/indexes/i1/an/myuser:ALT=/home/alt/a1/an/myuser doveadm(myu...@um.es): Debug: fs: root=/home/buzones/b1/an/myuser/mdbox, index=/home/indexes/i1/an/myuser, control=, inbox=, alt=/home/alt/a1/an/myuser doveadm(myu...@um.es): Debug: Namespace : Using permissions from /home/buzones/b1/an/myuser/mdbox: mode=0700 gid=-1 doveadm(myu...@um.es): Debug: acl: initializing backend with data: vfile doveadm(myu...@um.es): Debug: acl: acl username = myu...@um.es doveadm(myu...@um.es): Debug: acl: owner = 1 doveadm(myu...@um.es): Debug: acl vfile: Global ACL directory: (none) doveadm(myu...@um.es): Debug: Namespace : type=private, prefix=BORRADOS., sep=., inbox=no, hidden=yes, list=no, subscriptions=yes location=mdbox:/home/buzones/b1/an/myuser/expunged doveadm(myu...@um.es): Debug: fs: root=/home/buzones/b1/an/myuser/expunged, index=, control=, inbox=, alt= doveadm(myu...@um.es): Debug: Namespace BORRADOS.: Using permissions from /home/buzones/b1/an/myuser/expunged: mode=0700 gid=-1 doveadm(myu...@um.es): Debug: acl: initializing backend with data: vfile doveadm(myu...@um.es): Debug: acl: acl username = myu...@um.es doveadm(myu...@um.es): Debug: acl: owner = 1 doveadm(myu...@um.es): Debug: acl vfile: Global ACL directory: (none) dsync(myu...@um.es): Debug: Effective uid=2012, gid=204, home=/home/buzones/b1/an/myuser dsync(myu...@um.es): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled dsync(myu...@um.es): Debug: Quota root: name=User quota backend=dict args=:file:/home/buzones/b1/an/myuser/quota dsync(myu...@um.es): Debug: Quota rule: root=User quota mailbox=* bytes=10737418240 messages=0 dsync(myu...@um.es): Debug: Quota rule: root=User quota mailbox=Trash bytes=+1073741824 messages=0 dsync(myu...@um.es): Debug: dict quota: user=myu...@um.es, uri=file:/home/buzones/b1/an/myuser/quota, noenforcing=0 dsync(myu...@um.es): Debug: Namespace : type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=imapc: dsync(myu...@um.es): Debug: imapc: root=, index=, control=, inbox=, alt= dsync(myu...@um.es): Debug: imapc(myotis30.um.es:143): Looking up IP address dsync(myu...@um.es): Debug: imapc(myotis30.um.es:143): Connecting to 155.54.211.169:143 dsync(myu...@um.es): Debug: imapc(myotis30.um.es:143): Server capabilities: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN dsync(myu...@um.es): Debug: imapc(myotis30.um.es:143): Authenticating
[Dovecot] Authentication with many Windows AD
Dear All, I'm a new user of Dovecot and I need support with the authentication process. I use Dovecot 2.0.9 and Postfix 2.6.6 on host x86_64-redhat-linux-gnu 2.6.32-279.14.1.el6.x86_64 CentOS release 6.3 (Final) I need to select the validation method depending on the user. I have users that have to be authenticate with UNIX account, and I have others thats have to be authenticate with Windows AD. For example, some sysadmins and special accounts needs validate with Unix accounts. Other normal users have to authenticate with Windows AD, I have many Windows controllers. Some users authenticate to a specific Windows AD and other users are validated in another one. How I can do this? Thank you very much. Antonio Alberola
[Dovecot] Dict quota timeout error: Has anything changed in the latest versions?
Hi,
I'm using dict quota with mysql for track them via SQL query instead of
read maildirsize file. Primary quota and enforcing is provided via
userdb and is "maildir" type.
Since dovecot 2.1.10-13 (but perhaps even before) when a new user log in
via POP/IMAP or receive an email for the first time dovecot created the
entry for the users in the table "quota" of dovecot database with quota
usage and number of messages (before of this login/delivery the user has
no entry in the quota table of dovecot DB).
After the update to 2.1.14 when a new user login or receive an email for
the first time dovecot was unable to create (immediately) the entry in
quota table and in the log print these errors:
Feb 12 11:03:49 master: Info: Dovecot v2.1.15 starting up (core dumps
disabled)
Feb 12 11:04:02 auth-worker(31421): Info: mysql(localhost): Connected to
database vpopmail
Feb 12 11:04:02 pop3-login: Info: Login: user=,
method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=31422, secured,
session=
Feb 12 11:04:08 pop3(ales...@domain.com): Info: Disconnected: Logged out
top=0/0, retr=0/0, del=0/9, size=18618
Feb 12 11:04:41 pop3-login: Info: Login: user=,
method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=31424, secured,
session=
Feb 12 11:05:11 pop3(ales...@domain.com): Info: Disconnected: Logged out
top=0/0, retr=0/0, del=1/9, size=18618
Feb 12 11:05:11 dict: Info: mysql(10.10.3.19): Connected to database dovecot
[ here is when I "quit" from telnet session]
Feb 12 11:05:41 pop3(ales...@domain.com): Error:
read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after
30 seconds
Feb 12 11:06:11 pop3(ales...@domain.com): Error:
read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after
30 seconds
Feb 12 11:06:41 pop3(ales...@domain.com): Error:
read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after
30 seconds
Feb 12 11:07:11 pop3(ales...@domain.com): Error:
read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after
30 seconds
Feb 12 11:07:41 pop3(ales...@domain.com): Error:
read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after
30 seconds
Feb 12 11:08:11 pop3(ales...@domain.com): Error:
read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after
30 seconds
Feb 12 11:08:41 pop3(ales...@domain.com): Error:
read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout after
30 seconds
[ only now telnet close the session ]
This is the telnet session:
# telnet 0 110
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
+OK Dovecot ready.
user ales...@domain.com
+OK
pass ciao
+OK Logged in.
list
+OK 8 messages:
1 1594
2 1594
3 1594
4 1594
5 1594
6 1594
7 3755
8 3714
.
dele 8
+OK Marked to be deleted.
quit
[ here dovecot wait about 3 minuts before close the connecctions ]
+OK Logging out, messages deleted.
Connection closed by foreign host.
After 3 minutes of wait in the mysql database dovecot insert the entry
for the new users:
mysql> use dovecot
Database changed
mysql> select * from quota where username='ales...@domain.com';
+--+-+--+
| username | bytes | messages |
+--+-+--+
| ales...@domain.com | 2144030 | 15 |
+--+-+--+
Errors like
Error: read(/usr/local/dovecot-2.1/var/run/dovecot/dict) failed: Timeout
after 30 seconds
were present also before the upgrade but they were occasional, now I had
to disable "dict quota".
Can this be fixed?
My dovecot configuration:
# cat /usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext
connect = host=10.10.3.19 dbname=dovecot user=dovecot password=XXX
map {
pattern = priv/quota/storage
table = quota
username_field = username
value_field = bytes
}
map {
pattern = priv/quota/messages
table = quota
username_field = username
value_field = messages
}
map {
pattern = shared/expire/$user/$mailbox
table = expires
value_field = expire_stamp
fields {
username = $user
mailbox = $mailbox
}
}
# dovecot -n
# 2.1.15: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.6
auth_cache_size = 512 k
auth_worker_max_count = 60
default_login_user = nobody
dict {
acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf
sqlquota =
mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
dotlock_use_excl = no
first_valid_gid = 89
first_valid_uid = 89
last_valid_gid = 89
last_valid_uid = 89
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
log_path = /var/log/dovecot/dovecot.log
mail_fsync = always
mail_location = maildir:~/Maildir
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = quota acl zlib
maildir_very_dirty_syncs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress com
[Dovecot] dsync 2.1.10 does not correctly handle renames of mailbox subtrees
Hi,
It seems that dsync 2.1.10 does not correctly handle renames of mailbox
subtrees. The scenario is as follows.
There are two servers, A and B, both running dovecot 2.1.10 and configured
to use mdbox. Automatic replication between them is disabled, ie.
replication plugin is not active and replicator service
has process_min_avail = 0. I perform the following actions:
1. On A: doveadm mailbox create -u test_user a a/b a/b/c d
2. On A: doveadm sync -u test_user -d
The result is as expected: on both A and B there are a/b/c and d mailboxes.
Now I connect to A via IMAP and rename a/b to d/b. The result is as
expected: on A there are a and d/b/c mailboxes. Next I execute on A
'doveadm sync -u mailstoragetest -d' and now the result is not as expected:
there are a/b/c and d/b subtrees on both A and B and a/b is just a
directory, ie. it does not contain dbox-Mails (the rest are proper
mailboxes).
dovecot.mailbox.logs have the following records appended to them:
A:
#2112: rename 33259783092361d18c60ba1c7cfe973c (2013-02-12 10:42:55) - this
probably is the rename of a/b to d/b (recursive, ie. including children -
in this case a/b/c => d/b/c) by the IMAP client
#2136: rename ee2be6da91ae879476b152d7ca041f1b (1970-01-01 01:00:00) -
rename of d/b/c to a/b/c by dsync (unexpected/incorrect)
B:
#1104: rename 33259783092361d18c60ba1c7cfe973c (1970-01-01 01:00:00) -
rename of a/b to d/b (non recursive) by dsync
What seems to be missing is the record in A's dovecot.mailbox.log
describing the rename of a/b/c to d/b/c (or in general case the set of
records describing the renames of all the children of a/b). Would appending
such records allow dsync to make correct decisions as to which mailboxes
and on which servers to rename?
Thanks for any help.
# 2.1.10: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.1
auth_master_user_separator = *
auth_username_format = %Ln
default_vsz_limit = 2 G
dotlock_use_excl = no
dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l5
log_path = /var/log/dovecot.log
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_access_groups = mail
mail_location = mdbox:~/mdbox:ALT=/var/vmail/alt/%n
mail_plugins = acl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
separator = /
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
plugin {
acl = vfile
}
protocols = imap lmtp pop3
service aggregator {
fifo_listener replication-notify-fifo {
mode = 0666
}
unix_listener replication-notify {
mode = 0666
}
}
service auth {
executable = /usr/lib/dovecot/auth
unix_listener auth-client {
mode = 0660
}
unix_listener auth-master {
mode = 0600
}
unix_listener auth-userdb {
mode = 0666
}
user = root
}
service dict {
unix_listener dict {
mode = 0666
}
}
service doveadm {
process_min_avail = 1
service_count = 1024
}
service imap-login {
chroot = login
executable = /usr/lib/dovecot/imap-login
inet_listener imap {
address = *
port = 143
}
service_count = 0
user = dovecot
}
service imap {
executable = /usr/lib/dovecot/imap
}
service lmtp {
inet_listener lmtp {
port = 24
}
process_limit = 48
}
service pop3-login {
chroot = login
executable = /usr/lib/dovecot/pop3-login
inet_listener pop3 {
address = *
port = 110
}
service_count = 0
user = dovecot
}
service pop3 {
executable = /usr/lib/dovecot/pop3
}
ssl = no
userdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
protocol imap {
imap_client_workarounds = tb-extra-mailbox-sep
mail_plugins = acl
}
protocol lmtp {
mail_plugins = acl
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
mail_plugins = acl
}
--
Karol Jurak
Re: [Dovecot] backing up maildir dovecot files
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 11 Feb 2013, Reindl Harald wrote: Am 11.02.2013 22:37, schrieb Steve Litt: On Mon, 11 Feb 2013 21:47:57 +0100 Reindl Harald wrote: Am 11.02.2013 21:34, schrieb David Mehler: I'm having to migrate servers. Both are dovecot2 systems. I'm wanting to copy over my mail store from one system to the other. I'd like to preserve dates/times of emails. These are maildir setups on both boxes, I'd like to be as transparent to the end user as possible. I currently have the first dovecot system offline and the second dovecot system is offline. assuming that BOTH servers are down as fro any other transfers like mysql-datadirs and any critical things which should go save and fast by preserve attributes: rsync --force --delete-after -tPrlHpogEAXz /folder/ root@newserver:/folder/ # -z compress # -t timestamps # -P progress # -r recursive # -l links # -H hard-links # -p permissions # -o owner # -g group # -E executability # -A acls # -X xtended attributes That's how I'd do it also. I think if you didn't mind including -D, which from my understanding is --special and --devices, neither of which I'd expect in a maildir, and if you wanted to do your deletion manually after the fact, would this be equivalent? well, i have them ususally in my "rsync.sh" but did not expect such files in a maildir, but yes it does not hurt rsync -PaHAX /folder/ The man page says -a (--archive) is equivalent to -rlptgoD, and you have all but D listed up there. My thinking on later manually deleting the old maildir, instead of letting rsync do it, is that if somehow, some weird thing goes wrong, I have the old one for backup. Who knows, maybe I copied the thing on top of the wrong other maildir and have to back it out -- I'd have the old maildir as a reference of which files. you missunderstand "--delete-after" this is for delete files in the TARGET folder which is not or no longer in the source and IMHO very very important if you want sync folders 1:1 because old artefacts can have very bad effects without you merge folders and if i know "hey my source contains exactly what i need, not more and lot less" this is not what i would like and never did in 10 years IT If you use "--delete-after" you could add "--fuzzy", which theoretically could save transfers because of filename renames. Anyway: Any form of "--delete" is required for Maildir, IMHO, because Labels, Tags, Keywords (or whatever the MUA calls it) and status information (seen, read, deleted) are reflected by the filename. Hence, if one does not "--delete", the _same_ message might pop up in the Maildir multiple times but with different status and/or tags. Same applies to new messages, because they are storred in /new and later moved to /cur. So one ends with the same message in "new" and in "cur". - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBURn+wV3r2wJMiz2NAQJYnwgAwGfCHBFglm800nbaHFhsdWeuVkhGJjnW l0QtbmQLaSdQOkWwdVkmNEQkJLmwvfhddG0a9GblofI/zn8EBN+3EDR/CM0d5Y8f jy1yqemT26sFTVC6NlTqhDF8zq/Oi5WK7ftUfYfrt90MtmMD1rzmBQ5Q/N+Tteae TeE0jeHNL0rHl9DoAV9AkBzhJPnYYodSK2lA1oewazlzRpzJSiHYmgVh4RiSmsyk 4DAtch5ZrqLnsx1A/mUHAfDvdk66j2Os0bLXqbHu9ZAzz5Xb9bbNfpu52u1Ukg/o TfXfiGA4aRix6I1OGePdXmi/qXTzPVLBzhhIZdMiJ2CKCWNFbIYC7w== =ugv1 -END PGP SIGNATURE-
