LMTP BUG
Hello, just setting up mail hub with Postfix & Dovecot I have found BUG in LMTP implementation. * assumptions - latest stable Postix & Dovecot, - Postfix delivers via Dovecot LMTP (virtual_transport = lmtp:127.0.0.1:24), - Dovecot userdb & passwd lookups are made via custom checkpassword interface, - each user (email account) has different UID/GID * description LMTP process must run as root to be able to deliver msgs to each user. It drops privileges temporarily when delivering and then restoring effective UID/GID to root (saved one). The problem is, when an attempt with more than ONE recipients is made. More than one recipient within single session. It seems, that Dovecot LMTP restores root privileges not between each recipient delivery attempt but after whole transaction. The only solution is throttle Postfix to send single message with multiple recipients as many small transactions - all with only ONE rcpt. lmtp_destination_recipient_limit=1 makes this magic. Otherwise you can find: Dec 12 03:30:36 vm dovecot: lmtp(3580, i...@xxx.com): Fatal: setgid(48672 from userdb lookup) failed with euid=33001, gid=43570, egid=43570: Operation not permitted (This binary should probably be called with process group set to 4867 2 instead of 43570) Can someone confirm that this is an error/bug? Thanks Pete
[Corrected] Can't get shared public folders working
I'm trying to use a shared public namespace. My mail client says it exists, and I can telnet in and SELECT it, but sieve scripts can't write to it. The configuration in /etc/dovecot/conf.d/10-mail.conf: # Shared namespace for Foo stuff namespace { type = public separator = '/' prefix = "#Foo/" location = maildir:/local/mnt/mail/shared:INDEXPVT=/local/mnt/mail/%n/shared:LAYOUT=fs # Use the default namespace for saving subscriptions. #subscriptions = no # List the shared/ namespace only if there are visible shared mailboxes. #list = children } I created the location, owned by the vmail user: $ ls -ld /local/mnt/mail/shared drwxr-xr-x 4 vmail vmail 4096 Dec 11 12:38 /local/mnt/mail/shared I created the subfolders, verified that they are there. Also, I see that Dovecot created a dovecot.mailbox.log file: $ ls -l /local/mnt/mail/shared total 12 -rw-r--r-- 1 vmail vmail 96 Dec 11 12:38 dovecot.mailbox.log drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:37 Foo-Bugs drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:38 Foo-Patches The dovecot.mailbox.log file is empty: $ more /local/mnt/mail/shared/dovecot.mailbox.log The intermediate folder is empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/ total 4 drwxr-xr-x 5 vmail vmail 4096 Dec 11 12:36 Foo-Bugs-New The mailbox has some folders in it: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/ total 12 drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 cur drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 new drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 tmp They seem to be empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/* /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/cur: The Sieve log insists the mailbox doesn't exist: $ more /local/mnt/home/Foo-bugs/.dovecot.sieve.log sieve: info: started log at Dec 11 15:35:20. error: msgid=: failed to store into mailbox '#Foo/Foo-Bugs/Foo-Bugs-New': Mailbox doesn't exist: #Foo/Foo-Bugs/Foo-Bugs-New. info: msgid=: stored mail into mailbox 'INBOX'. But if I telnet into the server, the mailbox shows up: a1 list "" * * LIST (\HasNoChildren \Junk) "/" Junk * LIST (\Noselect \HasChildren) "/" #Foo * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Patches * LIST (\HasNoChildren) "/" #Foo/Foo-Patches/Foo-Patches-New * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Bugs * LIST (\HasNoChildren) "/" #Foo/Foo-Bugs/Foo-Bugs-New * LIST (\HasNoChildren) "/" INBOX a1 OK List completed. And I can SELECT it: a2 select "#Foo/Foo-Bugs/Foo-Bugs-New" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1418343553] UIDs valid * OK [UIDNEXT 1] Predicted next UID * OK [NOMODSEQ] No permanent modsequences a2 OK [READ-WRITE] Select completed (0.006 secs). So why can't the Sieve script store into it? -- Randall Gellens Opinions are personal;facts are suspect;I speak for myself only -- Randomly selected tag: --- I must have a prodigious quantity of mind; it takes me as much as a week sometimes to make it up. --Mark Twain, _The Innocents Abroad_
Can't get shared public folders working
I'm trying to use a shared public namespace. Clients say it exists, but sieve scripts can't write to it. The configuration in /etc/dovecot/conf.d/10-mail.conf: # Shared namespace for Foo stuff namespace { type = public separator = '/' prefix = "#Foo/" location = maildir:/local/mnt/mail/shared:INDEXPVT=/local/mnt/mail/%n/shared:LAYOUT=fs # Use the default namespace for saving subscriptions. #subscriptions = no # List the shared/ namespace only if there are visible shared mailboxes. #list = children } I created the location, owned by the vmail user: $ ls -ld /local/mnt/mail/shared drwxr-xr-x 4 vmail vmail 4096 Dec 11 12:38 /local/mnt/mail/shared I created the subfolders, verified that they are there. Also, I see that Dovecot created a dovecot.mailbox.log file: $ ls -l /local/mnt/mail/shared total 12 -rw-r--r-- 1 vmail vmail 96 Dec 11 12:38 dovecot.mailbox.log drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:37 Foo-Bugs drwxr-xr-x 3 vmail vmail 4096 Dec 11 12:38 Foo-Patches The dovecot.mailbox.log file is empty: $ more /local/mnt/mail/shared/dovecot.mailbox.log The intermediate folder is empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/ total 4 drwxr-xr-x 5 vmail vmail 4096 Dec 11 12:36 Foo-Bugs-New The mailbox has some folders in it: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/ total 12 drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 cur drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 new drwxr-xr-x 2 vmail vmail 4096 Dec 11 12:36 tmp They seem to be empty: $ ls -l /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/* /local/mnt/mail/shared/Foo-Bugs/Foo-Bugs-New/cur: The Sieve log insists the mailbox doesn't exist: $ more /local/mnt/home/Foo-bugs/.dovecot.sieve.log sieve: info: started log at Dec 11 15:35:20. error: msgid=: failed to store into mailbox '#Foo/Q popper-Bugs/Foo-Bugs-New': Mailbox doesn't exist: #Foo/Foo-Bugs/Foo-Bugs-New. info: msgid=: stored mail into mailbox 'INBOX'. But if I telnet into the server, the mailbox shows up: a1 list "" * * LIST (\HasNoChildren \Junk) "/" Junk * LIST (\Noselect \HasChildren) "/" #Foo * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Patches * LIST (\HasNoChildren) "/" #Foo/Foo-Patches/Foo-Patches-New * LIST (\Noselect \HasChildren) "/" #Foo/Foo-Bugs * LIST (\HasNoChildren) "/" #Foo/Foo-Bugs/Foo-Bugs-New * LIST (\HasNoChildren) "/" INBOX a1 OK List completed. And I can SELECT it: a2 select "#Foo/Foo-Bugs/Foo-Bugs-New" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1418343553] UIDs valid * OK [UIDNEXT 1] Predicted next UID * OK [NOMODSEQ] No permanent modsequences a2 OK [READ-WRITE] Select completed (0.006 secs). So why can't the Sieve script store into it? -- Randall Gellens Opinions are personal;facts are suspect;I speak for myself only -- Randomly selected tag: --- Broad-mindedness: The result of flattening high-mindedness out.
Re: Error: mremap_anon(###) failed: Cannot allocate memory
On 12/08/2014 03:07, Teemu Huovila wrote: A config would always be useful, but I can venture a guess. Perhaps the affected users have a dovecot.index.cache file somehwere, e.g. under INBOX, that is larger than the memory limit for the lmtp process. Try increasing "default_vsz_limit" or the "service lmtp { vsz_limit }". Removing the overly large index cache file should also, temporarily, help. In case you do not get this error from the imap/pop3 processes, perhaps you have already set a higher vsz_limit for those? Teemu, Thanks for your suggestion. I checked the output of doveconf, and by default it appears the vsz_limit is set to 18446744073709551615B for each of the services, and 256M for default_vsz_limit. I checked a user in question, and their index.cache was indeed large, 123M. Seemingly needlessly so, as I deleted the dovecot files and reindexed, and now it's 6K. Thanks, I'll keep an eye on the users this affects and try to get their index.cache in order. Thanks, Andy
Migrate with Dsync
Hello List, i have a simple and maybe stupid question but, read the guide on http://wiki2.dovecot.org/Migration/Dsync now i wonder where to put this configuration ? May i oversee something but i would appreciate any hint toward solving my problem. regards, dominik
mdbox backup strategy
Hi, now we backup maildir with rdiff-backup every single day. Backup takes almost 20hours. I would like to switch to mdbox, but how to acomplish possibility of restore emails from any date what I want? Now if I need to restore mails from the day before yesterday I put the right date to parametrs of rdiff-backup restore command and I get what I want. But if I use doveadm backup I have backup only from last run. Because we have tens of TB email I couldn't do full backup every single day. Is it possible to restore state of mailbox from backup run before the last run if I use doveadm? I don't want do full backup every day, because backup storage is not unlimited. Thank you Jiri
Re: dovecot.index.log files: what are they?
On Wed, Dec 10, 2014 at 09:19:11PM +0100, Thomas Klausner wrote: > Hi! > > I have lots of these files: > > /home/wiz/Mail/my-folder-name/cur/.imap/1238738125.13533_23713.danbala:2,S/dovecot.index.log > > What are they for? > Why are they here? > Can I remove them? This was a by-product of dovecot thinking that I had mbox mailboxes, while they were maildir mailboxes. So dovecot created one of these _for every single mail_, which is why I had so many of them. Thomas
Re: dovecot & Apple Mail & maildir & lots of Mail
On Fri, Nov 21, 2014 at 10:52:38PM +0100, Thomas Klausner wrote: > I have dovecot 2.1.12 running on a mail server, and recently > configured Apple Mail to connect to it using secure IMAP, for the > first time. > > At the beginning it just showed the inbox and everything was fine, but > then I wanted to look at some of my folders and found the 'subscribe' > menu. When I opened it, Apple Mail went to discover what mailboxes > there are, and that's where my trouble began. > > It seemed to make good progress for some time (though slow -- over > days), and now lists the mailboxes from starting with letters a to d > on the left hand side (filling all the visible space, so there might > be more). However, whenever Apple Mail gets focus, the cursor becomes > a spinning rainbow circle and I can't interact with it. > > I tried deleting the mail account and setting it up again. The inbox > was shown again immediately and Apple Mail was usable, but the first > time the cursor moved over the mailboxes on the left hand side, it > froze again. > > There were two imap processes on the server, one rather idle, the > other eating CPU for about two minutes, then idling, and some time > later it disappeared too. > > My Mail directory is 31G with about 180 directories, each containing > mails in maildir format. > > Has anyone had similar issues? > > Is my maildir too big for dovecot too handle? > > How can I debug this? I finally found out what the problem was. My mail is in the folder "$HOME/Mail". For that reason, dovecot assumed I must be using mboxes (even though it only contains maildirs), and handled each mail = maildir file as a separate mbox. I've renamed "$HOME/Mail" to "$HOME/Maildir" and configured procmail and mutt to look there, and now it works fine even with Apple Mail. Thomas
stacking istreams and ostreams
Hello, I'm developing an encryption plugin for dovecot and ran into a problem with the stacking of i/o-streams. The encryption i/o-streams are working fine on any kind of mail the test suite is passing through them. But as soon as the zlib plugin is enabled the logs show an cache error: failed: Cached message size larger than expected (214 > 206, box=INBOX, UID=1) I've already double-checked the return values of ostream's sendv and istream's read function. They seem correct (and equal). If the order of the streams are changed (by changing the number in the lib-filename libxx_scrambler.so); meaning that the encryption is done before the compression (which isn't efficient) both streams are working correct without any errors. Is there some way the zlib plugin changes the cached message size? Is there some behaviour of the zlib plugin that I'm missing? Any help would be very welcome. Best regards, Philipp -- simia.tech GbR http://simiatech.com
Re: Sieve permissions issue following update [solved]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 11 Dec 2014, David Gessel wrote: and watching the logs: dovecot: lda(ges...@blackrosetech.com): sieve: msgid=: stored mail into mailbox 'INBOX' Success! :-) The permissions correction portion of the error below still seems wrong though, isn't it? And if so, a little misleading. Dec 9 00:09:59 mailhost dovecot: lda(ges...@domain.com): Error: sieve: binary save: failed to create temporary file: open(/usr/local/etc/dovecot/sieve/10-move-spam.svbin.mailhost.domain.com.114.) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +w perm: /usr/local/etc/dovecot/sieve, we're not in group 6(mail), dir owned by 143:6 mode=0775) Well, the error is not wrong by itself. An user gets a new message, in order to run the user's Sieve script, the LDA must load the sieve_before script. This is out-of-sync currently, because of the upgrade, and hence must be re-compiled and its binary form storred there. One could argue, if: a) in case of failure the binary should be written somewhere else, e.g. a temporary location and re-compiled each time a message arrives, or into the user's home dir, or ... The current way tells the admin, that something is wrong. b) sieve_before/after scripts chould be textually merged with user's scripts and storred as one combined binary in the user's directory. A change of a global script would impact all user scripts then, a message to everyone would require quite a bit CPU. Does it seem reasonable to let the port maintainer know to submit a request to include instructions in /usr/ports/UPDATING for recompiling global scripts when necessary (and how to do it)? I checked before posting to the list and the last entry for sieve is this one: You could file a bug report in your distro's bug tracking software. If these are standard locations - I mean, you did not changed the paths to point somewhere else -, the upgrade should recompile shared Sieve scripts. - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBVIlrdHz1H7kL/d9rAQLYBAf/bzt+3OLt6f236hd4N8fWOjo6dXJ5Cc5X EJOHKcyMeHIzVSl2GkM6ckKkfRuIIjmK5DW3h36JhaIx7wh2nQJZnNPj0xCub6hK 4xE/HRoqfpnhW36Z5XvPZc656N8ut+gx0phnHxk11K1iV8kPHQsNy29d9213UWVP yoVzaVLMBHYBRSMGIpU+10MRiSfFAbBce4mBWZ5Dt0bSUHXs5cDGRnRwH7HAvr6l k2xeBmLf4oME7Y6/Ja75CWcHnnMlTMCp4J//zfHQnsrV7nFjEMiESU8MH3Z0IXqL z4t9MVRdGWb17Sa4W22/LdainnxFcSKWR4dGX6bNu6qYLdApKXHzkQ== =4TlD -END PGP SIGNATURE-