[Dovecot] Dovecot truncating Inboxes?
Hi, Environment: >1 Users (LDAP) Solaris 10 / sparc imap: dovecot-1.1.10 (pam-unix, no direct LDAP access) pop: dovecot-1.0.14 on another server (pam-unix, no direct LDAP access) mta: postfix (pam-unix, no direct LDAP access) procmail (pam-unix, no direct LDAP access) Filesystems: Inboxes and IMAP-home are on two different SAM-FS volumes. Problem description: Inboxes of users are disappearing / are truncated. we are using dovecot-1.1.0 since two month as main IMAP server. It replaced our long running but old version of UW-IMAP. Till the last week we had no problems and it worked quite well and the whole mail system worked in this configuration since several years. But since a week we receive more and more complaints regarding truncated/deleted mailboxes. First, we assumed user/client errors, but as such messages arrive more frequently, we did a deep analysis of the issue. The only common component of our mail system (postfix (local), procmail, pop, imap) that all concerned users have accessed, was dovecot-imap v. 1.1.10. We have no locking problems, we use fcntl(), dotlock for all mail components. We tested this with a small perl script and locking works fine (no delivery to a locked mailbox and so on...) Finally, all indications point to a bug in dovecot. Google and searches in this mailing list did not come up with any valuable results. Are there any known bugs / configuration problems concerned to the described problem? I greatly appreciate any help! Thx in advance Harald Strack
[Dovecot] Meaning of mail_max_userip_connections?
Hi, I set mail_max_userip_connections in our IMAP configuration to mail_max_userip_connections = 10 to allow users 10 parallel connections. It seems that this also limits the amount of parallel connections from one IP but different users?! Our users mostly accessing the IMAP server by a webmailer or proxies. Thus, all users (>1) come from only 5 different IP. However, I got a lot of complaints about denied connections after setting mail_max_userip_connections = 10. Am I right with the meaning of this parameter? Thanks in advance Harry
Re: [Dovecot] Meaning of mail_max_userip_connections?
Hi Stan, thank you very much for your help! On Mon, 2010-09-27 at 04:24 -0500, Stan Hoeppner wrote: > Harald Strack put forth on 9/27/2010 3:59 AM: > > Hi, > > > > I set mail_max_userip_connections in our IMAP configuration to > > > > mail_max_userip_connections = 10 > > > > to allow users 10 parallel connections. It seems that this also limits > > the amount of parallel connections from one IP but different users?! > > > > Our users mostly accessing the IMAP server by a webmailer or proxies. > > Thus, all users (>1) come from only 5 different IP. However, I got a > > lot of complaints about denied connections after setting > > mail_max_userip_connections = 10. > > > > Am I right with the meaning of this parameter? > > More importantly, what were you attempting to accomplish by setting > this? What problem were you expecting it to solve? > > Webmail servers typically don't hold an IMAP connection open for more > than a few seconds so this setting does nothing in a webmail only > environment. We do have 1000s of parallel connections. Even a few seconds per connection needs more than 10 parallel connections. > > Proxies on the other hand, such as imapproxy, will hold concurrent > connections open for quite a while. Enabling this setting with upstream > imap proxies is a bad idea, as you've discovered. We do not use imapproxy. Our proxies behave more like NAT-gateways: the IMAP-Server get's a lot of connections from different users from the same IP. > > Again, what specific problem are you trying to solve? we have the problem that some users forked more than 100 processes (in one case we know the user was accessing the server with a custom script, some are caused by any buggy clients that do too many reconnects...). We want to limit the number of imap processes per user to 10, but not the number of processes per client IP (because of the proxies). Any idea? Thanks in advance Harry
Re: [Dovecot] Meaning of mail_max_userip_connections?
Hi Timo, On Mon, 2010-09-27 at 13:50 +0100, Timo Sirainen wrote: > On Mon, 2010-09-27 at 12:17 +0200, Harald Strack wrote: > > > > Our users mostly accessing the IMAP server by a webmailer or proxies. > > > > Thus, all users (>1) come from only 5 different IP. However, I got a > > > > lot of complaints about denied connections after setting > > > > mail_max_userip_connections = 10. > > > > > > We want to limit the number of imap processes per user to 10, but not > > the number of processes per client IP (because of the proxies). > > For that mail_max_userip_connections should have worked. If you get > complaints then it's because some client opens more than 10 connections > (or user has multiple clients open from same IP) or your webmail opens > >10 connections simultaneously. Accordingly, mail_max_userip_connections limits the number of connections from an IP. To deal with a scenario, when 400 Users behind a NAT-gateway come from the same IP (the gateway), we have to set mail_max_userip_connections = 400, right? > > You didn't say if the complains were from webmail users or from IMAP > client users.. Assuming webmail, I guess the problem is that it just > opens so many connections. Both. > With v2.0 you could specify different limits > to a certain network range (i.e. disable it for webmail, keep it for > rest). Will there also be a limit per user? > > BTW. The default for mail_max_userip_connections is 10, so do you mean > before you had it set to 0? Nearly. We had it set to 1000 and we set it to 1000 again now. best regards Harry
Re: [Dovecot] Meaning of mail_max_userip_connections?
Hi Timo, On Mon, 2010-09-27 at 14:42 +0100, Timo Sirainen wrote: > On Mon, 2010-09-27 at 15:30 +0200, Harald Strack wrote: > > > Accordingly, mail_max_userip_connections limits the number of > > connections from an IP. To deal with a scenario, when 400 Users behind a > > NAT-gateway come from the same IP (the gateway), we have to set > > mail_max_userip_connections = 400, right? > > No, wrong. It's a user+ip combination. Each different user behind the > same IP can use up to 10 connections with > mail_max_userip_connections=10. Thanks a lot for your explanation! However, now I am at the beginning again. > > BTW. What Dovecot version? If this isn't working as expected, maybe > dovecot -n output could show something /usr/local > We do not use the most recent version... but was there a bug with this parameter? # 1.2.8: /usr/local/dovecot-1.2.8/etc/dovecot.conf # OS: SunOS 5.10 sun4u base_dir: /var/run/dovecot-1.2.8 log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S listen: *:143 ssl_listen: *:993 ssl_cert_file: /usr/local/dovecot/etc/cert.pem ssl_key_file: /usr/local/dovecot/etc/key.pem verbose_ssl: yes login_dir: /var/run/dovecot-1.2.8/login login_executable: /usr/local/dovecot-1.2.8/libexec/dovecot/imap-login login_processes_count: 8 login_max_processes_count: 8192 max_mail_processes: 16084 mail_max_userip_connections: 1000 mail_privileged_group: mail mail_location: mbox:~/dovecot-home:LAYOUT=maildir++:INBOX=/var/mail/% u:INDEX=%h/dovecot-indexes mail_debug: yes mmap_disable: yes mbox_write_locks: fcntl dotlock mail_plugins: listescape imap_client_workarounds: netscape-eoh delay-newmail outlook-idle namespace: type: private separator: / inbox: yes list: yes subscriptions: yes auth default: debug: yes passdb: driver: pam userdb: driver: passwd best regards Harry
[Dovecot] Dovecot 2.0, 2.1 and 2.2.5 core dump when Quota Plugin (FS) is enabled
Hi, we are actually deploying a new IMAP-Server, targeting dovecot 2.1.17. Unfortunately, our tests with imaptest did not succeed, were ending in core dumps. We did this test: ./imaptest host=example.com port=143 user=strack pass=secret disconnect_quit no_pipelining mbox=dovecot-crlf That's what we get in dovecot's log: Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22810, session= Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22811, session= Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22812, session=<1GbR9wPmWACBu/Qf> Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22813, session= Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22814, session= Sep 10 11:17:04 imap(strack): Panic: file mbox-storage.c: line 712 (mbox_transaction_unlock): assertion failed: (mbox->box.transaction_count > 0 || mbox->mbox_lock_type == F_UNLCK) Sep 10 11:17:04 imap(strack): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0() [0x3c7824660a] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3c78246656] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3c78219eaa] -> /usr/lib64/dovecot/libdovecot-storage.so.0() [0x3c786600c0] -> /usr/lib/dovecot/lib10_quota_plugin.so(+0xb7e7) [0x7f113b5427e7] -> /usr/lib/dovecot/lib10_quota_plugin.so(+0xb218) [0x7f113b542218] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mbox_sync+0x88) [0x3c7865fea8] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mbox_transaction_save_commit_post+0x27) [0x3c78657c07] -> /usr/lib64/dovecot/libdovecot-storage.so.0() [0x3c786a5313] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x9f) [0x3c786b3cdf] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x8a) [0x3c786a4f5a] -> /usr/lib64/dovecot/libdovecot-storage.so.0() [0x3c78660121] -> /usr/lib/dovecot/lib10_quota_plugin.so(+0xb8af) [0x7f113b5428af] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x3e) [0x3c7867aa9e] -> dovecot/imap() [0x408b3a] -> dovecot/imap() [0x4088e4] -> dovecot/imap() [0x408d3c] -> dovecot/imap(cmd_append+0x139) [0x408f99] -> dovecot/imap(command_exec+0x3d) [0x41155d] -> dovecot/imap() [0x41046e] -> dovecot/imap() [0x41055a] -> dovecot/imap(client_handle_input+0x135) [0x410785] -> dovecot/imap(client_input+0x5f) [0x4110af] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x3c78252c16] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x3c78253c9f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x3c78252bb8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x3c7823e083] -> dovecot/imap(main+0x29d) [0x4195bd] Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22826, session= Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22827, session= Sep 10 11:17:04 imap-login: Info: Login: user=, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22828, session= Sep 10 11:17:05 imap(strack): Fatal: master: service(imap): child 22806 killed with signal 6 (core dumped) Sep 10 11:17:05 imap(strack): Info: Connection closed in=49 out=3832 Sep 10 11:17:05 imap(strack): Info: Connection closed in=49 out=3832 That's the backtrace of the coredump: Loaded symbols for /lib64/libnss_files.so.2 Reading symbols from /lib64/libnss_sss.so.2...Reading symbols from /usr/lib/debug/lib64/libnss_sss.so.2.debug...done. done. Loaded symbols for /lib64/libnss_sss.so.2 Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libgcc_s.so.1 Core was generated by `dovecot/imap'. Program terminated with signal 6, Aborted. #0 0x003bc02328a5 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6_4.2.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.4.x86_64 libcom_err-1.41.12-14.el6_4.2.x86_64 libgcc-4.4.7-3.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 openssl-1.0.0-27.el6_4.2.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x003bc02328a5 in raise () from /lib64/libc.so.6 #1 0x003bc0234085 in abort () from /lib64/libc.so.6 #2 0x003c78246618 in default_fatal_finish (type=, status=0) at failures.c:191 #3 0x003c78246656 in i_internal_fatal_handler (ctx=0x7fff1c87efa0, format=, args=) at failures.c:649 #4 0x003c78219eaa in i_panic (format=0x2c1e ) at failures.c:263 #5 0x003c786600c0 in mbox_transaction_unlock (box=0x1a51270, lock_id=) at mbox-storage.c:711 #6 0x7f95c08c77e7 in quota_mailbox_transaction_rollback (ctx=0x1a589e0) at quota-storage.c:142 #7 0x7f95c08c7218 in quota_mailbox_sync_notify (box=0x1a51270, uid=0, sync_typ
