Re: [Dovecot] inet_listener imaps { port = 0 } question
On 31.5.2012, at 16.58, henrixd wrote:
Why commenting out inet_listener imaps {} won't stop dovecot to listen port
993? I think this would be expected behavior. Just curious, finally got it
working with port = 0. :)
When you comment out something, Dovecot uses the default settings for it. By
default Dovecot listens on port 993.
[Dovecot] Inconsistent search results and crash on force-resync
Hi, I am seeing inconsistencies in search results (finding 2 emails when only 1 exists, finding the email when it has been moved to another folder, etc). I figured I should run force-resync to fix any issues. I ran the following: doveadm -v force-resync -u user mailbox and I got some worrysome logs. - I should mention that I have been seeing some crashes of fts-lucene in my logs. I sent a traceback of this on the mailing list 1-2 days ago under the subject [Dovecot] fts_lucene crashing. - I should also mention that all the problems I am having are only in 1 email account. This email account contains folders of over 100k emails. Do I need to tweak dovecot somehow for this? Up until now all I did was change vsz_limit to 1024 MB for service imap. Here are the logs: Jun 1 11:15:01 X dovecot: imap(form): Error: Recent flags state corrupted for mailbox INBOX2 Jun 1 11:15:01 X dovecot: imap(form): Error: /data/emails/form/mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 X dovecot: imap(form): Error: /data/emails/form/mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 X dovecot: imap(form): Error: Recent flags state corrupted for mailbox contrat Jun 1 11:15:01 X dovecot: imap(form): Error: /data/emails/form/mailboxes/contrat/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 X dovecot: imap(form): Error: /data/emails/form/mailboxes/rep_Immigation soi-mAOo-me/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:01 X dovecot: imap(form): Error: /data/emails/form/mailboxes/Templates/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 X dovecot: imap(form): Error: Recent flags state corrupted for mailbox rep_eval_positive Jun 1 11:15:02 X dovecot: imap(form): Error: /data/emails/form/mailboxes/rep_eval_positive/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 X dovecot: imap(form): Error: Recent flags state corrupted for mailbox Sent Jun 1 11:15:02 X dovecot: imap(form): Error: /data/emails/form/mailboxes/Sent/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 X dovecot: imap(form): Error: Recent flags state corrupted for mailbox form_positif Jun 1 11:15:02 X dovecot: imap(form): Error: /data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 X dovecot: imap(form): Error: /data/emails/form/mailboxes/Archives/contrat/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 X dovecot: imap(form): Error: /data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:02 X dovecot: imap(form): Error: /data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:03 X dovecot: imap(form): Error: /data/emails/form/mailboxes/Archives/form_indetermine/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:03 X dovecot: imap(form): Error: /data/emails/form/mailboxes/Archives/form_indetermine/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:15 X dovecot: imap(form): Error: Recent flags state corrupted for mailbox form_indetermine Jun 1 11:15:15 X dovecot: imap(form): Error: /data/emails/form/mailboxes/form_indetermine/dbox-Mails/dovecot.index reset, view is now inconsistent Jun 1 11:15:25 X dovecot: indexer-worker: Error: indexer-worker: /home/jd/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:210: std::string lucene::index::DocumentsWriter::closeDocStore(): Assertion `numDocsInStore*8 == directory-fileLength( (docStoreSegment + . + IndexFileNames::FIELDS_INDEX_EXTENSION).c_str() )' failed. Jun 1 11:15:25 X dovecot: indexer: Error: Indexer worker disconnected, discarding 28 requests for form Jun 1 11:15:25 X dovecot: indexer-worker(form): Fatal: master: service(indexer-worker): child 9909 killed with signal 6 (core not dumped) I have 3 questions: 1) When the log says /mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is now inconsistent should I be worried, or this will fix itself? 2) Should I expect to see Error: Recent flags state corrupted for mailbox Sent??? I ran the force-resync 3 times and I still see this message. 3) Any idea why clucene is crashing? Regards, -Joe
[Dovecot] Exposing global (default) sieve script through Managesieve
Hi folks, I'm setting up a dovecot server with managesieve support. I'd like to offer spamfiltering through a Sieve script to my users by default, but still allow them to modify the filtering rules through Managesieve. I found the sieve_global_path configuration option, which seems perfect for what I want. I can configure a default script there, which will work for all users until they set upload their own sieve script using managesieve. However, when configured like this, the user experience isn't quite perfect. When users open the managesieve interface on their client, there is no trace of the default filters, so users might think the spamfiltering is done in some other manner. Now, when they create a filtering rule (e.g., to sort out mail to mailing lists), that rule will overwrite the default spamfiltering rule causing all the spam to spill into the user's mailbox. I'm afraid that most users won't realize they have to manually recreate the spamfiltering rule to fix this. Also, they might not know how to write the rule, even if they do... I've considered a few existing ways to fix this: - Use sieve_before / sieve_after to make sure that the default script is always executed, in addition to any user-supplied scripts. This removes the surprise, but removes the option for users to tweak the spamfiltering rules. - Don't use sieve_global_path, but instead distribute the default script to each user's homedir on user creation. This prevents making changes to the default script for existing users and in my setup, user creation and (mail)homedir creation are nicely separated through an LDAP directory, I'd rather not go this route. - When using the Roundcube webmail application as the IMAP client, I can point Roundcube at the default sieve script. Now, when Roundcube sees there are no scripts through ManageSieve, it shows a (fake) default script with the correct contents. As soon as the user changes this script or creates a new script, it is actually uploaded to Dovecot, causing the edited script to be used instead of the global script. This option has the user experience I'm looking for, but having this out-of-band connection from Roundcube to the default script configured with dovecot is ugly (and tricky, since these run on different hosts in my setup). The biggest problem is of course that this only works for Roundcube, not for any other IMAP client my users might use. So, I was wondering: Wouldn't it make sense for the managesieve plugin to do something similar to roundcube: When the user has no sieve script configured, let it fake a single default script, showing the contents of the global script? Since the ManageSieve protocol doesn't seem to support any way to flag this situation, it would be fooling the clients a bit, but I'm not sure if that's really a problem. While the user has not script named default in his sieve_dir: - include a script called default in the LISTSCRIPTS output. - return the contents of the sieve_global_path in the GETSCRIPT default command. - remove any sieve symlink after a SETACTIVE default command (as if SETACTIVE was given). This causes dovecot to fall back to the sieve_global_path script. - the DELETESCRIPT default command should fail. This might confuse clients and users, since it is listed in LISTSCRIPTS but cannot be deleted, but I think most users will understand they can't delete the default script. - RENAMESCRIPT default some_name should copy the sieve_global_path script into the user's sieve_dir. This will effectively copy the script instead of renaming it (since it will still be magically listed in LISTSCRIPTS), so that might be confusing. All other commands work just like they do now (in particular, PUTSCRIPT default uploads a script called default into the user's sieve_dir, preventing all of the above from applying. As noted above, this change might cause some confusion, but I think that is manageable. On additional thing is that running SETACTIVE will not completely disable sieve processing (as would be expected), but will (again) cause the sieve_global_path script to be run. This is already the case currently, though, and should probably be considered a separate problem (whose root cause is the lack of a difference between no script script configured yet and active script disabled, both remove the sieve symlink). Also, this problem might be a feature in some setups, so fixing it might not be so easy... So, any thoughts on this? Any fundamental problems I'm missing? (Not-so) obvious alternatives? Gr. Matthijs signature.asc Description: Digital signature
[Dovecot] dovecot stats: useful data to gather
Timo, following our discussion on dovecot stats at the LinuxTag 2012 my team and I sat down and put together a list of stat items we think to be useful in daily dovecot usage. Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. Our offer to create and contribute a standalone web interface for dovecot stats stands. Here are the stats we believe to be useful: Login/Logout - total number login success/time - total number login failure/time - total number per authentication mechanism - total number plain sessions - total number STARTTLS sessions - total number of currently connected users (pop3/pop3s/imap/imaps/managesieve) - login names of connected users (not really stats, but great for actions regarding those uses e.g. force logout) - total number logout commands/time - total number BYE responses (autologout) Mailbox state - Inflow rate (number incoming messages/time) - Deleted rate (number \Deleted flagged messages/time) - Expunge rate (number Expunge operations/time) - total number current messages mailboxes normal storage - total number current messages mailboxes alt storage - total number read messages mailboxes normal storage - total number read messages mailboxes alt storage - per user number current messages mailboxes normal storage - per user number current messages mailboxes alt storage - per user number read messages mailboxes normal storage - per user number read messages mailboxes alt storage Mailbox Quota - total number persons under soft-quota per quota - total number persons above or equal soft-quota per quota - total number persons above or equal hard-quota per quota Performance - minimum time to write a message - maximum time to write a message - average time to write a message - minimum time to modify a message - maximum time to modify a message - average time to modify a message - minimum time to delete a message - maximum time to delete a message - average time to delete a message - minimum time search operations - maximum time search operations - average time search operations Regards, p@rick -- state of mind () http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht MünchenPartnerschaftsregister PR 563
[Dovecot] auth trouble
Debian Lenny, Dovecot v 1.0.15. I'm getting a lot of what I think is a local socket asking dovecot:auth to verify username/passwords: May 31 09:00:54 server dovecot-auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=admin rhost= Note the empty 'rhost='. That's why I think it's on the server. I see others that look like bots: May 30 23:08:43 server dovecot-auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=admin rhost=200.119.139.22 And I know how to promote the latter to a firewall. But with no rhost, I'm stumped... I've read books, googled, read docs, and asked for help on other mailing lists, and I've learned a lot. And I no longer think it really has much to do with Dovecot, other than the login attempt going through it to get to PAM. But has anyone here seen this before? Is my current theory correct? What did you do to make it go away? (I suspect that upgrading to Debian Squeeze might get rid of it, but I'm afraid that if I don't figure out what's going on, it might just come back.) -- Glenn English hand-wrapped from my Apple Mail
Re: [Dovecot] dovecot stats: useful data to gather
On 1.6.2012, at 23.58, Patrick Ben Koetter wrote: Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. I had thought about SNMP before also, but for the current kind of stats that are exported I couldn't think of any reasonable way to export them. Here are the stats we believe to be useful: Login/Logout - total number login success/time - total number login failure/time .. I'll look at these later in more detail, but some important questions / design decisions: Currently stats process only remembers things after Dovecot was started. I don't think getting these kind of numbers would really work like that. Perhaps all of the statistics should be permanently dumped to disk every ~minute or so + at shutdown and loaded at startup, so the numbers would at least normally always just increase since the first time Dovecot was started? Mailbox state - Inflow rate (number incoming messages/time) - Deleted rate (number \Deleted flagged messages/time) These operations/time type of things I had hoped to be able to externalize :) If stats process simply gives the raw stats, the reader could do this kind of summing up. Otherwise .. well, I guess it could maybe keep track of the current ops/last 60 secs and the reader would then have to read the value about once a minute or half or something. It wouldn't give exact results though. Performance - minimum time to write a message - maximum time to write a message - average time to write a message Within last .. day? hour? minute? ..
Re: [Dovecot] auth trouble
I forgot to include this config info: # 1.0.15: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 ssl_listen: * ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_max_processes_count: 12 mail_privileged_group: mail mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain login verbose: yes passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix -- Glenn English hand-wrapped from my Apple Mail
Re: [Dovecot] dovecot stats: useful data to gather
* Timo Sirainen dovecot@dovecot.org: On 1.6.2012, at 23.58, Patrick Ben Koetter wrote: Besides pulling together all the data we also think it would be useful to have an SNMP interface to access the stats. I had thought about SNMP before also, but for the current kind of stats that are exported I couldn't think of any reasonable way to export them. I am not an expert on SNMP, others in my office are, but as I understand it there's no need for Dovecot to export the data. AFAIK Dovecot would have to offer a subagent, which could be queried by a SNMP server. If we need more knowledge on SNMP I can ask my folks on the team to give some guidance. For the moment I found this: http://net-snmp.sourceforge.net/wiki/index.php/TUT:Writing_a_Subagent Here are the stats we believe to be useful: Login/Logout - total number login success/time - total number login failure/time .. I'll look at these later in more detail, but some important questions / design decisions: Currently stats process only remembers things after Dovecot was started. I don't think getting these kind of numbers would really work like that. Perhaps all of the statistics should be permanently dumped to disk every ~minute or so + at shutdown and loaded at startup, so the numbers would at least normally always just increase since the first time Dovecot was started? ACK. My understanding is: Statistical data are moments in time. The application provides these snapshots. It is up to other protocols (e.g. SNMP) and software (e.g. RRD) to gather and create time series and also to relate data to each other in order to come up with ratios, timelines etc. This might be a good opportunity to check out Howard's MDB database (in order to get around potential future law suits concerning BDB usage ...). http://highlandsun.com/hyc/mdb/ Mailbox state - Inflow rate (number incoming messages/time) - Deleted rate (number \Deleted flagged messages/time) These operations/time type of things I had hoped to be able to externalize :) If stats process simply gives the raw stats, the reader could do this kind of summing up. Otherwise .. well, I guess it could maybe keep track of the current ops/last 60 secs and the reader would then have to read the value about once a minute or half or something. It wouldn't give exact results though. ACK. I'd externalize them too. So dump the /time aspect and only give raw data at moment of query. Performance - minimum time to write a message - maximum time to write a message - average time to write a message Within last .. day? hour? minute? .. Concerning message write time: the time the last message had to be written. In general the stats update interval should be configurable in order to adapt it to the overall system performance. Makes no sense to bring down the server by gathering stats every nano second unless one likes self-induced DOS. ;) It would probably be a useful strategy to update internal data on every event and answer SNMP queries from memory but write the data to disc every once in a while to have them when the server restarts. Besides that I don't see a use case for sharing such data between processes such as exporting them to memcache or anything alike. Do you? p@rick -- state of mind () http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht MünchenPartnerschaftsregister PR 563
