Re: [Dovecot] too many files opened
Hello, On Wed, Aug 29, 2007 at 10:06:41AM +0800, Joshua, C.S. Chen wrote: My institute's mail server was upgraded to RHEL5 with dovecot 1.0-2.1rc5 recently. And it uses PAM authentication which points to LDAP server. Now from time to time it gets Error like dovecot: Aug 29 09:23:59 Error: auth(default): pam(joshua,192.168.177.52): pipe() failed: Too many open files This looks similar to [221572]. We don't know what was causing that though. I can imagine that even [154314] may be causing that (which is hopefully getting fixed in RHEL 5.2). [221572] https://bugzilla.redhat.com/show_bug.cgi?id=221572 [154314] https://bugzilla.redhat.com/show_bug.cgi?id=154314 passdb: driver: pam userdb: driver: passwd Please, try to use userdb ldap in dovecot.conf. That way you don't use nss_ldap, which is buggy. -- Tomas Janousek, SW Engineer, Red Hat, Inc.
[Dovecot] Bug? Expunging Symlinked Maildir w/ Lazy_expunge Enabled
Hi all, Using Dovecot 1.0.3 on RedHat Enterprise 5 (kernel 2.6.18-8.1.6.el5PAE), and NFS storage, we symlinked a Maildir folder: /mailstore/user/Maildir/.Junk - /junkstore/user/Junkmaildir Everything works fine, until we try to expunge, which produces: A04 NO BUG: Unknown internal error This only happens if lazy_expunge is enabled: mail_plugins = quota imap_quota acl lazy_expunge lazy_expunge = .EXPUNGED/ .EXPUNGED/ .EXPUNGED/ Lazy_expunge works great on non-symlinked folders. We tried version 1.1 alpha2, which actually crashes in this scenario. The only fix we've found is to disable lazy_expunge. Attached is our dovecot -n config. Anyone have an idea what might be causing this or a workaround? Thanks! Rich [EMAIL PROTECTED] # 1.0.3: /shared/dovecot.conf base_dir: /var/dovecot-mail/ log_path: /var/dovecot-mail/dovecot.log protocols: imap imaps pop3 pop3s ssl_ca_file: /adminstore/exim/ssl/instantsslroot.crt ssl_cert_file: /adminstore/exim/ssl/public-mail.crt ssl_key_file: /adminstore/exim/ssl/private-mail.key disable_plaintext_auth: no shutdown_clients: no login_dir: /var/dovecot-mail//login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_user: exim login_greeting: System ready. login_processes_count: 32 login_max_processes_count: 400 verbose_proctitle: yes mail_location: maildir:/mailstore/%Lu/Maildir:INDEX=MEMORY mail_cache_fields: mail_cache_min_mail_count: 65536 mailbox_idle_check_interval: 10 mmap_disable: yes lock_method: dotlock maildir_stat_dirs: yes maildir_copy_with_hardlinks: yes maildir_copy_preserve_filename: yes mail_executable(default): /usr/local/libexec/dovecot/rawlog /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/rawlog /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota acl lazy_expunge mail_plugins(imap): quota imap_quota acl lazy_expunge mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle imap_client_workarounds(imap): delay-newmail outlook-idle imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %Mf pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh namespace: type: private separator: . inbox: yes namespace: type: private separator: . prefix: .EXPUNGED/ location: maildir:/mailstore/%u/Expunged:INDEX=MEMORY hidden: yes auth default: mechanisms: plain login passdb: driver: pam args: exim userdb: driver: ldap args: /adminstore/configs/dovecot-ldap.conf plugin: quota: maildir:storage=25:ignore=Junk acl: vfile:/adminstore/configs/dovecot-acls
Re: [Dovecot] rule of thumb for indexing overhead
Indexes aren't normally rebuilt, they're updated. And the update overhead is practically nothing with maildir. I just wrote this: http://wiki.dovecot.org/LDA/Indexing Hi Timo... So, if I understand this correctly, if I'm using maildir, I could use exim to do the local delivery instead of dovecot's LDA, and the index update wouldn't such a big problem? In my case, local delivery and the real imap servers are in different boxes, so by having exim doing the local delivery, I'd avoid a dovecot install in the local delivery boxes. And of course, exim doesn't have to spawn dovecot's delivery agent on every siingle piece of mail. Also, are there any drawbacks of using exim to do the local delivery? Thanks a bunch, g.
Re: [Dovecot] Err. msg. while copy.: Invalid internal data.
Bazy schrieb: Dominik Springer wrote: Hi All, While copying 9360 messages from a local folder to a folder on the imap server at message 5792 the client shows the following message from server: Invalid internal data. The error is reproducible. Any suggestions? Configuration: Versio: 1.0.rc15 (Debian Package) OS: Debian etch (in XEN VM) Kernel: 2.6.18-4-xen-686 (Debian Package) FS: ext3 Mail-Client: Thunderbird 2.0.0.6 on Mac OS X 10.4.10 Do you use maildir or mbox? I had this problem when mail was stored in a single mbox file. Try changing to maildir. Hi Bazy, thx for this hint, but I'm already using the Maildir format. Dominik
Re: [Dovecot] Err. msg. while copy.: Invalid internal data.
Marcus Rueckert schrieb: On 2007-08-29 00:33:59 +0200, Dominik Springer wrote: While copying 9360 messages from a local folder to a folder on the imap server at message 5792 the client shows the following message from server: Invalid internal data. The error is reproducible. Any suggestions? Configuration: Versio: 1.0.rc15 (Debian Package) OS: Debian etch (in XEN VM) Kernel: 2.6.18-4-xen-686 (Debian Package) FS: ext3 Mail-Client: Thunderbird 2.0.0.6 on Mac OS X 10.4.10 anything in /var/log/mail? darix No - there is nothing unusual in the log. Br, Dominik
Re: [Dovecot] rule of thumb for indexing overhead
Also, are there any drawbacks of using exim to do the local delivery? I'm very interested in the answer to this question, too. So far I have found (through reading, not trying things yet) that Dovecot's quota handling is more flexible than Exim's (exim is pretty much limited to FS quotas, I think, which is no good for virtual users). Dovecot's Sieve implementation has more features than Exim's. Both of these happen to matter to me (and led me to Dovecot in the first place.) An upside for Exim doing the delivery is that you theoretically can arrange for some additional rejections to happen at SMTP time. Pragmatically, those additional rejections are typically not done at SMTP time anyhow (e.g., an explicit fail from a user Exim filter).
Re: [Dovecot] Err. msg. while copy.: Invalid internal data.
On Wed, 2007-08-29 at 00:33 +0200, Dominik Springer wrote: Hi All, While copying 9360 messages from a local folder to a folder on the imap server at message 5792 the client shows the following message from server: Invalid internal data. Internal date I suppose? v1.0.1 fixed this (applies also to COPY): - APPEND / SEARCH: If internaldate was outside valid value for time_t, we returned BAD error for APPEND and SEARCH never matched. With 64bit systems this shouldn't have happened. With 32bit systems the valid range is usually for years 1902..2037. So I guess you have one file in your maildir with a bad mtime. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] rule of thumb for indexing overhead
On Wed, 2007-08-29 at 11:44 -0700, Tom Bombadil wrote: I just wrote this: http://wiki.dovecot.org/LDA/Indexing Also, are there any drawbacks of using exim to do the local delivery? Like the wiki page says, there shouldn't be really any performance problems with that. I can't say if there are any other problems. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] dovecot dspam plugin using libdspam
Andreas, Please, do not take this poorly. I am simply asking questions to make sure this patch/plugin is a good idea in the form you suggest. I am a user of the other patch. I am wondering if this is worth it. Your patch, if it links against libdspam will bloat dovecot. What do we gain? Not every message goes through dspam (the fork, exec, etc.). It is only those that were classified incorrectly. I agree with many of your suggested changes. Additionally, most open source projects seem to use autoconf/automake. What do we gain by switching to cmake instead of making it work some how with dovecots autoconf/automake system? Depending on your answers, I will try your patch and help you clean it up. Trever Adams Andreas Schneider wrote: Hi, I've found the dovecot dspam plugin and looked at the code. I forks and calls the dspam binary for every mail. I didn't like this behavior, so I've migrated it to use libdspam. The plugin still needs more love: * Use cmake instead of a Makefile * Make the spam folder configurable in the dovecot.conf * Code cleanup and more comments. Please test. Comments and patches are welcome ;) http://www.cynapses.org/tmp/dovecot-dspam-plugin-0.1.tar.gz Cheers, -- andreas
[Dovecot] Migration woes from tpop mbox to dovecot maildir
I was hoping some one might be able to offer me some advice on a little problem I have. I am looking to A. move from mbox to maildir B. looking to move from tpop to dovecot to bring both pop and imap under the same program. anyways the problem I am having is this tpop3d mbox uses MD5 sum of the mailbox headers in hex which is option %m in the uidl section but the problem is when dovecot convert plugin runs and makes in a maildir it fails to find the md5 of this because it is only used for mboxs. if I change this from %m to %Mf it downloads every message on the clients computer which is not desirable. I was thinking only options i have is to allow it to download all them mail and or write a script to convert it to maildir or another pop servers uidl. but i dont know much about how other pop servers generate there headers. so I was hoping maybe some one has done something similar to this or could offer some input. thanks -Chris The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete this material from any computer. In accordance with industry regulations, all messages are retained and are subject to monitoring. This message has been scanned for viruses and dangerous content and is believed to be clean. Securities offered through Cantella Co., Inc., Member FINRA/SIPC. Home Office: 2 Oliver Street, 11th Floor, Boston, MA 02109 Telephone: (617)521-8630
Re: [Dovecot] rule of thumb for indexing overhead
On Wed, 29 Aug 2007 12:31:01 -0700 (PDT) WJCarpenter [EMAIL PROTECTED] wrote: Also, are there any drawbacks of using exim to do the local delivery? I'm very interested in the answer to this question, too. So far I have found (through reading, not trying things yet) that Dovecot's quota handling is more flexible than Exim's (exim is pretty much limited to FS quotas, I think, which is no good for virtual users). Dovecot's Sieve implementation has more features than Exim's. Both of these happen to matter to me (and led me to Dovecot in the first place.) We use exim all the way to local delivery. And it handles Maildir++ quotas just fine. Depending on your actual needs and userdb/authdb choices the dovecot LDA might be the better choice. But exim can do pretty much everything one needs (we don't use sieve) and the advantages of indexing during delivery are negligible for us in general and potentially negative in some scenarios (mass mail to many/all users). An upside for Exim doing the delivery is that you theoretically can arrange for some additional rejections to happen at SMTP time. Pragmatically, those additional rejections are typically not done at SMTP time anyhow (e.g., an explicit fail from a user Exim filter). I don't think there are many situations where a MTA can do better than a LDA when it comes to rejections during SMTP time. Since at least with exim local delivery happens in a separate stage AFTER SMTP has been successfully completed.Now having consistent errors, logs and configurations for all mail delivery stages is something that might you want to stick with your MTA from the edge to the mailbox. Regards, Christian -- Christian BalzerNetwork/Systems EngineerNOC [EMAIL PROTECTED] Global OnLine Japan/Fusion Network Services http://www.gol.com/
Re: [Dovecot] rule of thumb for indexing overhead
Hello, On Wed, 29 Aug 2007 18:10:00 -0700 Tom Bombadil [EMAIL PROTECTED] wrote: of indexing during delivery are negligible for us in general and potentially negative in some scenarios (mass mail to many/all users). Is this negative hypothetical or have you actually seen load spikes in situations like this? I actually did see that. In the case of exim, for each piece of mail going to the mailbox, one exim process is spawned, and this exim delivery process spawns the dovecot's LDA. The load pretty much doubles. There is that little detail of the additional processes spawned, but the overhead for that alone is not so much of an issue here (linux, plenty of CPU and memory to spare). But when you have 8 emails rushing towards your mailstorage (and yes, of course we have sensible limits on number of process, maximum load before defer, etc) your I/O will be a bottleneck and adding the additional strain of indexing on top of that is not a good idea. It is only at times of mass mails like that I see our mailstorge boxes break into a light sweat and avoiding any additional load then is just the sensible thing to do. The indexing for these mails will be spread out over hours to days (frequency of access by the clients) instead of being lumped on top of the existing peak and I/O saturation. It all boils down to your hardware and usage patterns. Our systems are plenty fast and any delay by having to (re)build the indexes is hardly noticeable. But if you have a system with less reserves, users that access mail very infrequently but get plenty of mail (so indexing at access time will be an involved process) and no mass mail spikes, dovecot LDA starts to look a lot more promising. ^_^ Regards, Christian -- Christian BalzerNetwork/Systems EngineerNOC [EMAIL PROTECTED] Global OnLine Japan/Fusion Network Services http://www.gol.com/
