[Dovecot] Assertion failure in maildir-mail.c for gzipped maildir files in 1.1.15
I attempted an upgrade from 1.1.7 to 1.1.15 and encountered an apparent regression in handling gzipped maildir files. I have tried to narrow down the conditions under which it happens and reproduce it with as simple a case as possible. The actual mail application that exposed this problem is Squirrelmail. Starting off with the index files removed from the Folder maildir directory: * OK Dovecot ready. 1 login x y 1 OK Logged in. 2 select Folder * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 13 EXISTS * 0 RECENT * OK [UIDVALIDITY 1220278111] UIDs valid * OK [UIDNEXT 14] Predicted next UID 2 OK [READ-WRITE] Select completed. 3 UID FETCH 3 (RFC822.SIZE INTERNALDATE) * 3 FETCH (UID 3 RFC822.SIZE 1085 INTERNALDATE 04-Jan-2004 09:23:45 -0500) 3 OK Fetch completed. 4 UID FETCH 4 (RFC822.SIZE INTERNALDATE) Connection closed by foreign host. The fetch of message 3, which is not gzipped succeeds, but the fetch of message 4, which is gzipped, fails with the following assertion in the log: Panic: imap: user=x, Status: file maildir-mail.c: line 85 (maildir_mail_stat): assertion failed: (fd != -1) Repeating, but first fetching only INTERNALDATE from the gzipped message succeeds, and once that value is cached in the index files, the same query that causes the failure before will now succeed, as it doesn't need to read the gzipped file any more to get INTERNALDATE: * OK Dovecot ready. 1 login x y 1 OK Logged in. 2 select Folder * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 13 EXISTS * 0 RECENT * OK [UIDVALIDITY 1220278111] UIDs valid * OK [UIDNEXT 14] Predicted next UID 2 OK [READ-WRITE] Select completed. 3 UID FETCH 4 (INTERNALDATE) * 4 FETCH (UID 4 INTERNALDATE 02-Dec-2003 10:39:18 -0500) 3 OK Fetch completed. 4 UID FETCH 4 (RFC822.SIZE INTERNALDATE) * 4 FETCH (UID 4 RFC822.SIZE 6035 INTERNALDATE 02-Dec-2003 10:39:18 -0500) 4 OK Fetch completed. 5 logout * BYE Logging out 5 OK Logout completed. Connection closed by foreign host. Now that the value is cached, the same previously failing query will continue to execute fine, as long as the indexes remain in place: * OK Dovecot ready. 1 login x y 1 OK Logged in. 2 select Folder * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 13 EXISTS * 0 RECENT * OK [UIDVALIDITY 1220278111] UIDs valid * OK [UIDNEXT 14] Predicted next UID 2 OK [READ-WRITE] Select completed. 3 UID FETCH 4 (RFC822.SIZE INTERNALDATE) * 4 FETCH (UID 4 RFC822.SIZE 6035 INTERNALDATE 02-Dec-2003 10:39:18 -0500) 3 OK Fetch completed. 4 logout * BYE Logging out 4 OK Logout completed. Connection closed by foreign host. Removing the indexes again, and this time first fetching just RFC822.SIZE first: * OK Dovecot ready. 1 login x y 1 OK Logged in. 2 select Folder * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 13 EXISTS * 0 RECENT * OK [UIDVALIDITY 1220278111] UIDs valid * OK [UIDNEXT 14] Predicted next UID 2 OK [READ-WRITE] Select completed. 3 UID FETCH 4 (RFC822.SIZE) * 4 FETCH (UID 4 RFC822.SIZE 6035) 3 OK Fetch completed. 4 UID FETCH 4 (RFC822.SIZE INTERNALDATE) Connection closed by foreign host. In this case, the assertion fails again. Fetching RFC822.SIZE alone does not populate the indexes in a way that then allows RFC822.SIZE and INTERNALDATE to be fetched, as fetching INTERNALDATE alone does. It's all confusing to me, hopefully this make some sense to others. Let me know if there is other testing I can help with. I'm running on Slackware 12.1 and have reproduced the same issue with both kernel 2.6.24.4 and 2.6.27.7. The filesystem is XFS. The content of the gzipped message does not seem to matter. David
[Dovecot] imaptest build problems
I am not sure this is the correct list, please forgive if it is not, and point me in the right direction. I downloaded the latest http://www.imapwiki.org/ImapTest/Installation from http://dovecot.org/nightly/imaptest/imaptest-latest.tar.gz cd imaptest-20090401 $./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... configure: error: ls -t appears to fail. Make sure there is not a broken alias in your environment configure: error: newly created file is older than distributed files! Check your system clock Here is the config log http://dl.getdropbox.com/u/340087/Drops/05.28.09/config-4d3972ab-233922.log Some tests: $date Thu May 28 23:37:13 PDT 2009 $ls -lt total 2688 -rw-r--r--@ 1 me staff4552 May 28 23:37 config.log drwxr-xr-x@ 28 me staff 952 May 28 23:33 src -rw-r--r--@ 1 me staff 27278 Mar 31 19:04 ChangeLog -rw-r--r--@ 1 me staff1456 Mar 31 19:04 imaptest-config.h.in -rw-r--r--@ 1 me staff 19599 Mar 31 19:04 Makefile.in -rwxr-xr-x@ 1 me staff 671381 Mar 31 19:04 configure -rw-r--r--@ 1 me staff 262586 Mar 31 19:04 aclocal.m4 -rw-r--r--@ 1 me staff 925 Mar 31 19:04 configure.in -rw-r--r--@ 1 me staff 38 Jul 12 2008 AUTHORS -rw-r--r--@ 1 me staff 211 Jul 12 2008 COPYING -rw-r--r--@ 1 me staff 26536 Jul 12 2008 COPYING.LGPL -rw-r--r--@ 1 me staff 145 Jul 12 2008 Makefile.am -rw-r--r--@ 1 me staff9498 Dec 2 2007 INSTALL -rwxr-xr-x@ 1 me staff 15936 Dec 2 2007 depcomp -rwxr-xr-x@ 1 me staff9233 Dec 2 2007 install-sh -rwxr-xr-x@ 1 me staff 11014 Dec 2 2007 missing -rw-r--r--@ 1 me staff 0 Dec 2 2007 NEWS -rw-r--r--@ 1 me staff 0 Dec 2 2007 README -rw-r--r--@ 1 me staff 0 Dec 2 2007 stamp.h.in -rwxr-xr-x@ 1 me staff 42037 Jan 7 2007 config.guess -rwxr-xr-x@ 1 me staff 30253 Jan 7 2007 config.sub -rw-r--r--@ 1 me staff 198422 Jan 7 2007 ltmain.sh -- Scott * If you contact me off list replace talklists@ with scott@ *
Re: [Dovecot] imaptest build problems
On May 29, 2009, at 2:39 AM, Scott Haneda wrote: checking whether build environment is sane... configure: error: ls - t appears to fail. Make sure there is not a broken alias in your environment Well, that's the first I've heard of this. Have you tried compiling any other program that has a configure script built by autoconf? Seems like all of them would be broken and maybe a better list would be some autoconf-related mailing list. configure: error: newly created file is older than distributed files! Check your system clock Here is the config log http://dl.getdropbox.com/u/340087/Drops/05.28.09/config-4d3972ab-233922.log Some tests: $date Thu May 28 23:37:13 PDT 2009 Do you have any NFS or other remote filesystems enabled?
Re: [Dovecot] expire: mysql table creation error
Timo Sirainen wrote: On Thu, 2009-05-28 at 08:04 +0200, John Fawcett wrote: but maybe that's not such a good primary key, probably better to do: mysql CREATE TABLE expires ( - id integer not null auto_increment, - username varchar(100) not null, - mailbox varchar(255) not null, - expire_stamp integer not null, - primary key (id), - key (username, mailbox(200)) - ); Query OK, 0 rows affected (0.00 sec) Nothing uses the id, so I don't really see a point in having primary key at all then. You could instead create a unique index on (username, mailbox), unless MySQL disallows that too. yes it works without the primary key. It is best to not have a unique index otherwise there will be insert failures for (the unlikely event) of mailboxes for the same user with mailbox names longer than 200 chars and the first 200 chars identical. A non unique index is fine. CREATE TABLE expires_test ( username varchar(100) not null, mailbox varchar(255) not null, expire_stamp integer not null, key (username, mailbox(200)) );
Re: [Dovecot] imaptest build problems
I should note, I do not build a ton of software by hand, so I struggle, and try to learn as best as I can. If I am not clear, let me know, if I need to read or research more, let me know. On May 28, 2009, at 11:50 PM, Timo Sirainen wrote: On May 29, 2009, at 2:39 AM, Scott Haneda wrote: checking whether build environment is sane... configure: error: ls - t appears to fail. Make sure there is not a broken alias in your environment Well, that's the first I've heard of this. Have you tried compiling any other program that has a configure script built by autoconf? Seems like all of them would be broken and maybe a better list would be some autoconf-related mailing list. Primarily, I have used MacPorts often, so that is where 90% of the built apps are. In the majority of those cases, it is just calling ./ configure and doing some record keeping. On this machine, I have built outside of Macports, mrtg, which needed iirc, some GD stuff. In my efforts to get ASSP working, ~10 perl modules, not using CPAN, but using ./configure. * This was a total wild effort here, but I commented out the lines that did the date check and the ls -t check and got much further, but ended up bailing out on a could not find DC config file. This was my error, I did not download the DC sources to this machine. After doing the right thing and downloading the DC sources... dovecot-1.2.rc4 $./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... configure: error: ls -t appears to fail. Make sure there is not a broken alias in your environment configure: error: newly created file is older than distributed files! Check your system clock configure: error: newly created file is older than distributed files! Check your system clock Here is the config log http://dl.getdropbox.com/u/340087/Drops/05.28.09/config-4d3972ab-233922.log Some tests: $date Thu May 28 23:37:13 PDT 2009 Do you have any NFS or other remote filesystems enabled? No, just a lowly MacBook on wifi at the moment. I have several other servers in production I am going to give this a quick go on just to be sure it is not this particular machine. -- Scott * If you contact me off list replace talklists@ with scott@ *
Re: [Dovecot] imaptest build problems
On May 29, 2009, at 3:01 AM, Scott Haneda wrote: No, just a lowly MacBook on wifi at the moment. I have several other servers in production I am going to give this a quick go on just to be sure it is not this particular machine. I'm pretty sure the problem is your specific machine, given that I've successfully compiled Dovecot many times in my own MacBook. Although I could look into what exactly causes this problem in a few days.
[Dovecot] Maildir compression utility
Does anyone know of a script to compress old mails stored in a maildir format by carrying out the steps described at http://wiki.dovecot.org/Plugins/Zlib thanks, John
Re: [Dovecot] imaptest build problems
On May 29, 2009, at 12:06 AM, Timo Sirainen wrote: On May 29, 2009, at 3:01 AM, Scott Haneda wrote: No, just a lowly MacBook on wifi at the moment. I have several other servers in production I am going to give this a quick go on just to be sure it is not this particular machine. I'm pretty sure the problem is your specific machine, given that I've successfully compiled Dovecot many times in my own MacBook. Although I could look into what exactly causes this problem in a few days. Very nice to know that you work on a MacBook, as I move closer and closer to getting all my users migrated to DC on Mac OS X. Thank you for the offer to look at it. I am guessing configure scripts are auto gen'd? That being the case, any issues I am having need to be addressed by the developers of 'autoconf' ? I stepped through the configure script a line at a time to see if I could pin it down. I have in my bashrc export CLICOLOR=1 export CLICOLOR_FORCE=1 Commenting out the first fixes it. This seems poorly documented, as the man page for `ls` does not tell me much about how this will interfere with the output values for `ls` at all. Any suggestions on how to keep my colors and still build, as well as why this only just bit me now? I have built far too many things for this to crop up only now, and also have had that color setting on all my machines since beginnings of 10.5 and maybe 10.4 at some point. Do you have any idea on where I take this issue to correctly address is for a proper fix either in the way I define colors, or in how configure scripts are made? Thank you for your help, I am off to build this out and run some tests. -- Scott * If you contact me off list replace talklists@ with scott@ *
Re: [Dovecot] Clustering dovecot?
Hi Rick, at the moment I´m building the same setup than you. I have no further experience with it, but I made a setup in our testing lab and under testing conditions it seems to run quite nice. I took 2 servers with heartbeat1 in active/passive node. Each server has its own IP, and they have a cluster IP that´s managed by heartbeat only. This cluster IP is provided in our DNS for accessing the mailstorage cluster, and only the active node has it at the time. Then I have a DRBD shared storage on the two nodes. On the DRBD storage I only put the dovecot maildir and mysql databases. The dovecot and mysql binaries are not shared and the configuration also not. DRBD, dovecot and Mysql are managed by heartbeart. There is always a danger that the connection between the 2 nodes is failing and you will have a split brain then with a big data mess. So it´s important to provide redundancy in the connections. For heartbeat, I have one dedicated LAN connection and a serial connection. For DRBD, I use 2 bonded NICs on different PCI cards. Take a look at DOPD for DRBD. This marks the passive DRBD partition outdated if the DRBD connection fails, and because heartbeat can only takeover if it can start all resources of a resource group, a failover is not possible anymore if the DRBD connection is broken, so you can´t mess up your DRBD so easy any more. If both heartbeat connections fail, you will have lots of trouble, and that´s easy to achieve with some wrong iptables if you take only LAN connections. So the serial cable is a nice thing because it´s not affected! We use heartbeat1 because we had some trouble bringing heartbeat2 to run. Heartbeat1 is not able to monitor it´s resources, so we thought about using MON for this. And to take some STONITH devices like telnet accessible power outlets to switch off the power of a failing node automatically. But this setup seems to be rather complex, which is the enemy of reliability, and we heard about people having problems with accidently automatic failovers or reboots. So in the end we decided against an automatic failover in the case a service dies. We use only the failover of heartbeat1, e.g. if the active node dies completely, there will be a failover to the passive node. And we use connection redundancy to hopefully not have a split brain. And make a good backup ;-) (Take care not to use NFS for storage if you take another setup than the here described because you can have trouble with file locking!) Our cluster is protecting against hardware problems, and against some kind of software problems. Because of DRBD, if you do a rm -rf on the maildir, you loose all data on _both_ nodes in the same second, so the protection against administration faults is not very good! Backups are really important. But if we have some trouble with the active node, and we can´t fix it in some minutes, we can try a failover to the passive node and there is a big chance that the service is running on the other node quite well. A nice things for software updates. For MTA we use Postfix. Because it´s not a good idea to put the postfix mailqueue on a DRBD (bad experiences), you will have some mails (temporarily) lost if you do a failover. So it´s a good idea to minimize the time mails are held in the queue. Because of this and because we need a longtime stable mailstorage but an always up-to-date brand new SPAM and virus filter, we decided to put 2 Postfix/Amavis/Spamassassin/Antivirus relays in front of the IMAP cluster. They´re identical, with the same MX priority in DNS, so if one of the relays fails, the other one takes the load. As I said, this solution is working only in the lab now and not yet in production, but there the failover seems to be no problem at all for the clients. So I hope I could give you some ideas. regards, Andreas -- Nur bis 31.05.: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
[Dovecot] locking fails
hello! on one of my servers (freebsd 7.0-R , Jail) I have a very strange problem I guess its something about LOCKing. some programs have issued with locking: saslauthd[38127] :main: could not lock pid lock file: /var/run/saslauthd/saslauthd.pid.lock saslauthd[38127] :main: fcntl: Invalid argument dovecot: 2009-05-29 03:45:50 Info: dovecot v1.1.15 starting up dovecot: 2009-05-29 03:45:50 Info: Generating Diffie-Hellman parameters for the first time. This may take a while.. dovecot: 2009-05-29 03:45:50 Error: ssl-build-param: fcntl(write-lock) locking failed for file /var/db/dovecot/ssl-parameters.dat.tmp: Invalid argument dovecot: 2009-05-29 03:45:50 Fatal: ssl-build-param: file_try_lock(/var/db/dovecot/ssl-parameters.dat.tmp) failed: Invalid argument deliver(u...@example.com): 2009-05-29 03:43:24 Error: fcntl(write-lock) locking failed for file /data/maildirs/example.com/user/dovecot.index.log: Invalid argument deliver(u...@example.com): 2009-05-29 03:43:24 Error: mail_index_wait_lock_fd() failed with file /data/maildirs/example.com/user/dovecot.index.log: Invalid argument deliver(u...@example.com): 2009-05-29 03:43:24 Info: msgid=blu129-w212f3c73cddc4c399ce338fd...@phx.gbl: save failed to INBOX: Internal error occurred. Refer to server log for more information. [2009-05-29 03:43:24] -rw--- 1 vmail vmail 24 May 29 03:43 /data/maildirs/example.com/user/dovecot.index.log all directorys are existent and wirtable I`m not sure if this only affects dovecot..maybe its a issue caused of the OS. anyone an idea? maybe its possible to run dovecot without these locking? thanks!! Oskar -- Nur bis 31.05.: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
Re: [Dovecot] locking fails
On May 29, 2009, at 3:46 AM, Oskar Eyb wrote: on one of my servers (freebsd 7.0-R , Jail) I have a very strange problem No NFS or any other weird filesystem? I guess its something about LOCKing. .. saslauthd[38127] :main: could not lock pid lock file: / var/run/saslauthd/saslauthd.pid.lock saslauthd[38127] :main: fcntl: Invalid argument .. dovecot: 2009-05-29 03:45:50 Error: ssl-build-param: fcntl(write- lock) locking failed for file /var/db/dovecot/ssl- parameters.dat.tmp: Invalid argument .. deliver(u...@example.com): 2009-05-29 03:43:24 Error: fcntl(write- lock) locking failed for file /data/maildirs/example.com/user/ dovecot.index.log: Invalid argument Yeah. Clearly the fcntl locking isn't working. I've no idea why. Some FreeBSD list would probably be more helpful.
Re: [Dovecot] locking fails
No NFS or any other weird filesystem? just loopbackmounts with nullfs - but until yesterday everything worked fine.. Yeah. Clearly the fcntl locking isn't working. I've no idea why. Some FreeBSD list would probably be more helpful. yeah.. work in progress (posted already to the freebsd list, waiting für answers) until its solved, is there a way with dovecot to avoid fctnl? Oskar -- Nur bis 31.05.: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
Re: [Dovecot] locking fails
On May 29, 2009, at 4:12 AM, Oskar Eyb wrote: until its solved, is there a way with dovecot to avoid fctnl? lock_method setting.
Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot
On Thu, 2009-05-28 at 21:28 -0500, Kyle Wheeler wrote: On Friday, May 29 at 09:46 AM, quoth Curtis Maloney: This is certainly one advantage dbox and maildir have -- not being limited to the FS file size limit per folder. That's not *entirely* accurate. Certainly no single message can exceed the 2GB limit even with maildir, and the other issue that begins to come up is the impact/effect of large numbers of files. Depending on the filesystem (I'm assuming ext2?), there's probably a hard limit on FC4 had ext3 (unless my memory is totally mistaken). the number of files per directory, and almost certainly there's a big Subdirectories, yes (because the link count in the inode is of quite finite size). But there never was TTBOMK a limit on the number of files (!= Directories) in ext2 (except the trivial one: The directory is as large as the largest file. But that applies probably to all filesystems - though more recent ones allow for much larger files). Let alone ext3. performance penalty for that many files. To get good performance with Maildir and really large folders, you need a filesystem that can handle large numbersof files. Ext3 has directory hashing, ReiserFS is Make sure you have the dir_index option set on that filesystem (which is probably set per default anyways these days. Otherwise you can change it on the fly with `tune2fs`). The back of my head suggests that one has to recreate the directory after changing that option (read: `mkdir new; mv old/* new; rmdir old; mv new old`. Solving the command line too long problem is left to the reader;-). good... I believe XFS and several others have tackled the problem as well (I don't know about FFS). That said... eGADS - a real life FC4 in the wild?!?! According to fedoraproject.org: For 20030101-20050607 there are a potential 863 CVE named vulnerabilities that could have affected FC4 packages. 759 (88%) of those are fixed because FC4 includes an upstream version that includes a fix, 10 (1%) are still outstanding, and 94 (11%) are fixed with a backported patch. That would make me a little nervous that's just the issues over the course of two years, ending in 2005 (FOUR years ago). I don't know what/how others do but many servers are not really in the wild but behind more recent firewalls and/or loadbalancers and/or similar equipment (like running database servers behind webservers). And running FC4 doesn't mean that that certain/some/several/many packages aren't replaced by more recent ones - for whatever reason (security, performance, newer version, newer drivers in the kernel, ...). For a pristine FC4 with lots of services directly at the Internet: I totally agree with you. Apart from the basic question if one shouldn't better run a more conservative distribution (like RHEL) in the first place where the support cycle is much longer. Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services
Re: [Dovecot] Clustering dovecot?
On Fri, 2009-05-29 at 09:41 +0200, list...@gmx.de wrote: (Take care not to use NFS for storage if you take another setup than the here If you use MailDir you will never have NFS problems, we have multiple SMTP's, multiple POP3's and 2 x webmail (IMAP) servers, all to a netapp NAS for mail storage, it works incredibly well.
Re: [Dovecot] offlineimap with dovecot and sieve
Steffen Kaiser writes: But it seems, that my sieve-scripts on the local dovecot server are not recognised by this combination. You have to simulate the delivery after synchronisation. However, I wonder how this is to work: 1) you sync news mails from A - B 2) you process them with Sieve on B, possible out them into other folders No, this doesn't work but I wnat to do). I simulate that with nnimap-split-rules in gnus (which works okay, but I think it would be nicer to do it by sieve-scripts). 3) next time you sync the filtered mails back to A, because they appear to be new. Yes and this works too. But I guess you cannot run Sieve on A? That's right, it's a CommuniGate Pro(fessionell?-) server, which doesn't support sieve. ... The new libsieve library of Stephan Bosch does include a tool called sieve-filter. It applies a Sieve script to all messages in a folder. I will test that, if it is in squeeze. Seems promising. ... Thanks for the info. -- Dieter Faulbaum Elektronenspeicherring BESSY II Albert-Einstein-Str. 15, 12489 Berlin phone +49 30 6392-4652 fax +49 30 6392-2939 mail dieter.faulb...@bessy.de
Re: [Dovecot] (no subject)
Thank you a lot for the tip. - Original Message From: Timo Sirainen t...@iki.fi To: Patrick Hemmen patrick.hem...@yahoo.de Cc: dovecot@dovecot.org Sent: Friday, May 29, 2009 12:00:36 AM Subject: Re: [Dovecot] (no subject) On Wed, 2009-05-27 at 17:38 +, Patrick Hemmen wrote: Hi, I use a OpenLDAP for authentication. To authenticate a full DN as the user name must be used, like cn=jim,ou=users,dc=example,dc=com. There are several domains, like example2.com and example3.com. I want to use Dovecot with ldap and authentication binds. For testing I use auth_bind_userdn = cn=%n,ou=users,dc=%d and the user name must provide as j...@example,dc=com. To allow the special chars (=,) in user name, I extend auth_username_chars. Now my questions. Exists a real chance to attack the ldap directory with the extended auth_username_chars? And it's possible to use authentication binds with the regular auth_username_chars and a provided user names like j...@example.com in my special ldap directory structure? Use: auth_bind_userdn = cn=%n,ou=users,dc=%Dd See %D in http://wiki.dovecot.org/Variables
Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot
Hi Scott, Are you saying that you can gzip a mailbox, and Dovecot will then somehow uncompress that and send the data back out to the Desktop Client as a normal looking IMAP box? I have never heard of this tactic, and I am not finding any references to it on google, though I am probably just not understanding it and looking at the wrong search terms. I use this to save/backup older emails. zipped mbox = Fast, small, only one file to handle, not changeable = optimal for backup Stefan
Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot
On Qui, 28 Mai 2009, Curtis Maloney wrote: But what sort of user puts 4GB of mail into a _single_ folder?? Those that never delete the emails nor move them into folders. Everything they ever received is in the INBOX. And yes, this happens a lot. -- Eduardo M KALINOWSKI edua...@kalinowski.com.br
Re: [Dovecot] Maildir compression utility
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 29 May 2009, John Fawcett wrote: Does anyone know of a script to compress old mails stored in a maildir format by carrying out the steps described at http://wiki.dovecot.org/Plugins/Zlib I made this script: http://www2.inf.fh-bonn-rhein-sieg.de/skdata/dovecot/compressMaildir.bz2 though I used it for testing a while back, not in production environment. I just added the 'Z' and ,S= stuff mentioned on the page. But nothing about maildirlock. Bye, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBSh/cAHWSIuGy1ktrAQI+aAgAscAjMAQCLdJgDP1Kor2bD0hKOr91lrrQ A9utrBmWcCe7rQzqT8AgLFDab9bnzdPX1C7uJSea+N2oJSojoMj05WpK9qmpomwP itH6gF+hyGxUcqrD6BkAWKlbgMkcFsVfXIvFajnBpvFNGMgZq8i2dGFehWb30VAO MBtkl/smUylzvi/q+O+s5OpWaB6oI8VM9zuWJun1b1I+49vNmHrGxfP3fJoFkPXk HYM3cPvEIiETrFi3jxKxF/8Dh9uzELxAjGcMYS+w816OafakfnwhPhXk8BStBmcV maCkI2XtWC3ndLgOhD3R4qvBPZ7IyywcR7yB82v1memXtszA0NLyQw== =C3yJ -END PGP SIGNATURE-
[Dovecot] Missing IMAP folders after Server reboot
Hi, Here is the situation, I hope somebody can help me. Our Email server running ubuntu 8.04.1 and dovecot 1.0.10 rebooted, and when it came back up the dovecot config had changed. Prior to the reboot, users stored their emails and folders in /var/mail/~username. Then the config became /var/mail/username for the main Inbox and /home/~username/mail for Folders. Now, none of the folders that users had created in their IMAP account are visible, and their contents are not in the main Inbox file. For example, the Drafts folder was there prior to the reboot, and now it had to be created in /home/username afterwards. These folders were not subfolders of the Inbox, but of the main IMAP root. All users were running Thunderbird as their client. Can they be recovered? Thanks, Eoin Kennedy - Sherling Sons Ltd, Jamestown Road, Inchicore, Dublin 8, Ireland. Company Reg No. 9294 Website: www.sherlingsteel.com
Re: [Dovecot] locking fails
hi! thanks, with dotlock or flock it works - expect the ssl-stuff: dovecot: 2009-05-29 15:48:49 Info: dovecot v1.1.15 starting up dovecot: 2009-05-29 15:48:49 Info: Generating Diffie-Hellman parameters for the first time. This may take a while.. dovecot: 2009-05-29 15:48:49 Error: ssl-build-param: fcntl(write-lock) locking failed for file /var/db/dovecot/ssl-parameters.dat.tmp: Invalid argu dovecot: 2009-05-29 15:48:49 Fatal: ssl-build-param: file_try_lock(/var/db/dovecot/ssl-parameters.dat.tmp) failed: Invalid argument any idea for this - is it possible to change also in ssl-build-param the lock_method? Oskar Original-Nachricht Datum: Fri, 29 May 2009 04:14:33 -0400 Von: Timo Sirainen t...@iki.fi An: Oskar Eyb fo...@gmx.de CC: dovecot@dovecot.org Betreff: Re: [Dovecot] locking fails On May 29, 2009, at 4:12 AM, Oskar Eyb wrote: until its solved, is there a way with dovecot to avoid fctnl? lock_method setting. -- Nur bis 31.05.: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
Re: [Dovecot] LDAP (AD) auth problem
If I use dn and dnpass it'd use password lookups mechanism for this access to password field on LDAP should be granted to some particular user. Unfortunately, it's impossible. Thats why I want to use auth_bind way. On Thu, May 28, 2009 at 11:57 PM, Timo Sirainen t...@iki.fi wrote: On Mon, 2009-05-25 at 17:07 +0200, Misha Volodko wrote: Using tcpdump i found a cause of it, seems dovecot binds to AD server, then drop connection and trying to do search without bind. During sniff I see response to my ldap search - LdapErr: DSID-0C090627, comment: In order to perform this operation a successful bind must be completed on the connection Dovecot drops the user bind when it does the userdb lookup. If AD doesn't support anonymous lookups, you can specify some user with dn and dnpass settings and Dovecot will use it for the userdb lookups. -- Gd Night
Re: [Dovecot] locking fails
it works - after I start ssl-build-param from the host system. :) should I ( cron ) run this periodically (how often) or only once Oskar -- Nur bis 31.05.: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
[Dovecot] Expire-Plugin segmentation fault (Re: Can expire-tool skip folders with expire time in future errors?)
Timo Sirainen wrote: Anyway it's not an error. It just means that all the rest of the mailboxes have future timestamps, and expire-tool has finished its work. I upgraded to Dovecot 1.1.15, but expire still gives me trouble. I keep seeing the following error repeatedly now: dict: db(secondary, ): unable to allocate space from the buffer cache dict: sdb.open() failed: Cannot allocate memory dict: dict: db(/var/lib/dovecot/expire.db) open failed dict: Failed to initialize dictionary 'expire' When I run the expire-tool manually or by cron job, it dies with a segmentation fault, so effectively I currently cannot use expire at all. -R
Re: [Dovecot] LDAP (AD) auth problem
No, it doesn't use password lookups. It still would do auth binding, but the userdb lookup would be done when bound to the specified dn. On May 29, 2009, at 9:56 AM, Misha Volodko wrote: If I use dn and dnpass it'd use password lookups mechanism for this access to password field on LDAP should be granted to some particular user. Unfortunately, it's impossible. Thats why I want to use auth_bind way. On Thu, May 28, 2009 at 11:57 PM, Timo Sirainen t...@iki.fi wrote: On Mon, 2009-05-25 at 17:07 +0200, Misha Volodko wrote: Using tcpdump i found a cause of it, seems dovecot binds to AD server, then drop connection and trying to do search without bind. During sniff I see response to my ldap search - LdapErr: DSID-0C090627, comment: In order to perform this operation a successful bind must be completed on the connection Dovecot drops the user bind when it does the userdb lookup. If AD doesn't support anonymous lookups, you can specify some user with dn and dnpass settings and Dovecot will use it for the userdb lookups. -- Gd Night
Re: [Dovecot] recipient delimiter and lmtp
Timo Sirainen schreef:
On Fri, 2009-05-22 at 20:45 +0200, John Fawcett wrote:
I read in the list archives about work in progress on
an lmtp delivery agent.As a previous cyrus user
where that was the preferred delivery method
from postfix, this sounds interesting.
I was wondering whether it will handle
recipient delimiters. ie will it be able to map
user+...@example.com to u...@example.com?
I'm not interested in the delivery into folder stuff,
just the plain delivery to inbox.
Hmm. Stephan, you currently have sieve_subaddress_sep setting. Should we
instead use a common setting here? Like:
protocol lda {
recipient_delimiter = +
}
With v1.2 it probably wouldn't do anything, but with v2.0 LMTP could use
it.
Sounds sensible. I can for instance make the plugin use this setting if
sieve_subaddress_sep is not set.
Regards,
--
Stephan Bosch
step...@rename-it.nl
Re: [Dovecot] offlineimap with dovecot and sieve
Steffen Kaiser wrote: The new libsieve library of Stephan Bosch does include a tool called sieve-filter. It applies a Sieve script to all messages in a folder. This is still a work in progress. It is undocumented and ill tested. In addition there is a sieve-test tool that: - -e Turns on true execution of the set of actions that results from running the script. In combination with the -l parame‐ ter, the actual delivery of messages can be tested. Note that this will not transmit any messages to remote SMTP recipients. Such actions only print the outgoing message to stdout. Yes, sieve-test is more of a test tool an not a delivery tool or mail store filter. Currently, the best way would still be to build some sort of deliver hack. At least until I finish sieve-filter. Regards, -- Stephan Bosch step...@rename-it.nl
Re: [Dovecot] recipient delimiter and lmtp
On 5/29/2009 3:22 PM, Stephan Bosch wrote:
I was wondering whether it will handle
recipient delimiters. ie will it be able to map
user+...@example.com to u...@example.com?
I'm not interested in the delivery into folder stuff,
just the plain delivery to inbox.
Hmm. Stephan, you currently have sieve_subaddress_sep setting. Should we
instead use a common setting here? Like:
protocol lda {
recipient_delimiter = +
}
With v1.2 it probably wouldn't do anything, but with v2.0 LMTP could use
it.
Sounds sensible. I can for instance make the plugin use this setting if
sieve_subaddress_sep is not set.
How hard would it be to allow the use of more than one delimiter?
Obviously this would have to be well documented, explaining the
pitfalls, and sysadmins would have to make sure that that character was
illegal when it came to creating the actual accounts...
Also, Google has a very interesting ability to ignore dots (periods) in
the local part of the address, so you can essentially create 'unique'
addresses on the fly by simply inserting a period anywhere into the
local part, that can then be filtered on at delivery time...
--
Best regards,
Charles
Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot
on 5-28-2009 4:46 PM Curtis Maloney spake the following: robforrest wrote: What is happening is that as users log in (via thunderbird), they are shown the contents of their mailboxes. However, when they try and access a given email, they get no response even if they wait upwards of 10 minutes. I believe that the problem has arisen from their huge inboxes, several users have inboxes in excess of 4GB. FC4 is quite old... can the kernel and filesystem you're using properly handle files over 4GB? This is certainly one advantage dbox and maildir have -- not being limited to the FS file size limit per folder. But what sort of user puts 4GB of mail into a _single_ folder?? (l)users! ;-P signature.asc Description: OpenPGP digital signature
Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot
on 5-29-2009 1:42 AM Bernd Petrovitsch spake the following: On Thu, 2009-05-28 at 21:28 -0500, Kyle Wheeler wrote: On Friday, May 29 at 09:46 AM, quoth Curtis Maloney: This is certainly one advantage dbox and maildir have -- not being limited to the FS file size limit per folder. That's not *entirely* accurate. Certainly no single message can exceed the 2GB limit even with maildir, and the other issue that begins to come up is the impact/effect of large numbers of files. Depending on the filesystem (I'm assuming ext2?), there's probably a hard limit on FC4 had ext3 (unless my memory is totally mistaken). the number of files per directory, and almost certainly there's a big Subdirectories, yes (because the link count in the inode is of quite finite size). But there never was TTBOMK a limit on the number of files (!= Directories) in ext2 (except the trivial one: The directory is as large as the largest file. But that applies probably to all filesystems - though more recent ones allow for much larger files). Let alone ext3. performance penalty for that many files. To get good performance with Maildir and really large folders, you need a filesystem that can handle large numbersof files. Ext3 has directory hashing, ReiserFS is Make sure you have the dir_index option set on that filesystem (which is probably set per default anyways these days. Otherwise you can change it on the fly with `tune2fs`). I don't think you can do it on the fly. Any directories created before the option was set will not have directory indexing until a fsck is done. quote Using Directory Indexing This feature improves file access in large directories or directories containing many files by using hashed binary trees to store the directory information. It's perfectly safe to use, and it provides a fairly substantial improvement in most cases; so it's a good idea to enable it: # tune2fs -O dir_index /dev/hdXY This will only take effect with directories created on that filesystem after tune2fs is run. In order to apply this to currently existing directories, we must run the e2fsck utility to optimize and reindex the directories on the filesystem: # e2fsck -D -f /dev/hdXY Note: This should work with both ext2 and ext3 filesystems. Depending on the size of your filesystem, this could take a long time. Perhaps you should go get some coffee... /quote The back of my head suggests that one has to recreate the directory after changing that option (read: `mkdir new; mv old/* new; rmdir old; mv new old`. Solving the command line too long problem is left to the reader;-). signature.asc Description: OpenPGP digital signature
Re: [Dovecot] Some questions about deliver
Le 28 mai 09 à 23:51, Timo Sirainen a écrit : On Tue, 2009-05-26 at 14:35 +0200, Axel Luttgens wrote: [...] That looks like a server configuration mistake. No, it's just a Postfix system quickly and dirtily brought to life for the sole purpose of testing deliver from within a MTA. ;-) On that occasion, I just took Postfix' default behaviors for illustrative purposes. But it could have been Sendmail, qmail, Exim... as well, or even a very esoteric homegrown system: any of those systems, especially the already operational and fine-tuned ones, have precise expectations on how their native LDA behaves. Replacing that LDA by another one is thus something far from benign and to be done with care, if possible with the help of as much info as possible about how the new LDA will behave. Hence my investigations, then my dumb questions about deliver: how does it behave, is that behavior configurable and if yes to what extent, have I missed something in the docs or in the source code, and so on. [...] At least, it would be nice to have a very precise description of how deliver is supposed to behave when facing various conditions. This would then be a documentation matter. Maybe.. Although it can be summarized pretty easily: - Invalid command line parameter gives EX_USAGE - Invalid configuration gives EX_CONFIG - User-over-quota optionally bounces or exits with EX_NOPERM with a mention about the -e option and the quota_full_tempfail setting - Anything else is EX_TEMPFAIL. Fine! I really believe that it would be worth to engrave that behavior somewhere in the docs. This could prove extremely useful to people considering to replace their existing LDA in their existing setup, by making explicit some points to take care of. As a side-effect, this might also act as a reminder of the behaviors that should not be altered in case of code rewriting. Last but not least, no more irritating questions. ;-) Thanks a lot, Axel
Re: [Dovecot] Missing IMAP folders after Server reboot
on 5-29-2009 6:51 AM Eoin Kennedy spake the following: Hi, Here is the situation, I hope somebody can help me. Our Email server running ubuntu 8.04.1 and dovecot 1.0.10 rebooted, and when it came back up the dovecot config had changed. Prior to the reboot, users stored their emails and folders in /var/mail/~username. Then the config became /var/mail/username for the main Inbox and /home/~username/mail for Folders. Now, none of the folders that users had created in their IMAP account are visible, and their contents are not in the main Inbox file. For example, the Drafts folder was there prior to the reboot, and now it had to be created in /home/username afterwards. These folders were not subfolders of the Inbox, but of the main IMAP root. All users were running Thunderbird as their client. Can they be recovered? I doubt that your mail is gone, somebody just changed where dovecot looks for it. Do you have a backup of your old config file? signature.asc Description: OpenPGP digital signature
[Dovecot] dovecot with MYSQL on Synology NAS
hello list, under my table I have a NAS station, the Synology DS-207+ there`s a linux with ipkg. the dovocot fromo ipkg didnt have mysql support. compiling seems to be also difficult, because of configure cant find libmysqlclient. experts, what to do for getting mysql support in dovecot? best regards hkwwg -- Nur bis 31.05.: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
Re: [Dovecot] dovecot with MYSQL on Synology NAS
On Sat, 2009-05-30 at 03:00 +0200, heutekinderwirdswasge...@gmx.de wrote: hello list, under my table I have a NAS station, the Synology DS-207+ there`s a linux with ipkg. the dovocot fromo ipkg didnt have mysql support. compiling seems to be also difficult, because of configure cant find libmysqlclient. experts, what to do for getting mysql support in dovecot? Dovecot should find it if it is installed, perhaps you need to install MySQL, do a locate libmysqlclient Do not know the NAS device you are using, but ipkg from memory is used on embedded devices so this tells me you might not be able to do what you want to do. Cheers
