-------- Original Message -------- Subject: Re: Dovecot/K9/Thunderbird IMAP interaction issues: replicated messages, constant reloading (TB) & missing new messages (K-9) From: Alex JOST <jost+li...@dimejo.at> To: dovecot@dovecot.org Date: Tue Jul 15 2014 13:30:31 GMT+0300 (Arabic Standard Time)
> Am 15.07.2014 03:12, schrieb David Gessel: >> * In thunderbird, I get constant downloading of messages. Every time I >> switch to the inbox from another folder, the messages start >> redownloading. This sometimes, but not always, seems to create >> duplicate messages, but I'm now fairly sure they are only created in the >> client and not on the server. Searching through various forums, it >> appears this has been a problem for Thunderbird for some time, but it >> wasn't for me on Courier. As posts about similar errors seem to have >> petered out over the last year without any clear guidance on how to fix >> it that I could find, I expect there's a simple fix I just haven't found >> yet, either in TB configuration or in Dovecot. Any hints? I tried >> Thunderfix to remove the .msf files and will be redownloading my mail >> over the thin pipe out to Iraq for the next 6 months (DOH!). One >> artifact is that when I restart thunderbird, my inbox is reported as >> having about 130 messages in it, then thunderbird starts redownloading >> headers and eventually gets back up to the 1297 currently listed both in >> the TB pane and by ls | wc -l in /cur. Is it possible there is a time >> out that needs to be extended? In TB, I've set (among other less >> apparently relevant options) >> mailnews.tcptimeout;1000 >> mail.db.idle_limit;30000000 > > How old is your Thunderbird profile. Have you tried creating a new one > and see if the problem persists? It is a new profile created with the transition from Cyrus to Dovecot about a week ago, but I used IMAPSync to move all my mail from the ancient Cyrus server to the shiny new Dovecot server. I then deleted the old profile. > > Thunderfix should not be needed by the way. Just right click on the > folder => Properties => Repair I did see that, but first thought there was something magic about "thunderfix" then decided to use it anyway because I have annual sent/recieved archives from 2000 (and one for pre-2000) so that would have been a lotta right-clicking. > > >> server # dovecot -n >> # 2.2.13: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 9.2-RELEASE amd64 >> auth_default_realm = mydomain.com >> auth_mechanisms = plain login >> auth_realms = mydomain.com mydomain2.org >> disable_plaintext_auth = no >> first_valid_gid = 5000 >> first_valid_uid = 5000 >> hostname = mydomain.com >> imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags >> imap_idle_notify_interval = 29 mins >> last_valid_gid = 5000 >> last_valid_uid = 5000 >> lda_mailbox_autocreate = yes >> listen = * >> mail_gid = 5000 >> mail_location = maildir:~/:CONTROL=/var/no-quota/%u >> mail_plugins = quota mail_log notify >> mail_temp_dir = /var/tmp >> mail_uid = 5000 >> maildir_broken_filename_sizes = yes >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave duplicate spamtest spamtestplus editheader >> mbox_lazy_writes = no >> namespace inbox { >> inbox = yes >> location = >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> mailbox Trash { >> special_use = \Trash >> } >> prefix = >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> expire = Trash >> mail_log_events = delete undelete expunge copy mailbox_delete >> mailbox_rename >> mail_log_fields = uid box msgid size >> quota = maildir:User quota >> quota_exceeded_message = Storage quota for this account has been >> exceeded, please try again later. >> quota_rule = *:storage=50G >> quota_rule2 = Trash:storage=+30%% >> quota_rule3 = Sent:storage=+30%% >> quota_warning = storage=90%% quota-warning 90 %u >> quota_warning2 = storage=75%% quota-warning 75 %u >> sieve = ~/.dovecot.sieve >> sieve_before = /usr/local/etc/dovecot/sieve/ >> sieve_dir = ~/sieve >> sieve_extensions = +spamtest +spamtestplus +relational >> +comparator-i;ascii-numeric +editheader >> } >> postmaster_address = ges...@mydomain.com >> protocols = imap lmtp sieve sieve >> sendmail_path = /usr/local/sbin/sendmail >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0666 >> user = postfix >> } >> unix_listener auth-userdb { >> group = vmail >> mode = 0600 >> user = vmail >> } >> } >> service managesieve-login { >> inet_listener sieve { >> port = 4190 >> } >> service_count = 1 >> vsz_limit = 128 M >> } >> service quota-warning { >> executable = script /usr/local/bin/quota-warning.sh >> unix_listener quota-warning { >> user = vmail >> } >> user = vmail >> } >> ssl = required >> ssl_cert = </usr/local/etc/ca/server-cert.pem >> ssl_cipher_list = >> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES256-SHA:!SSLv2 >> >> ssl_key = </usr/local/etc/ca/server-unencrypted-key.pem >> ssl_prefer_server_ciphers = yes >> userdb { >> driver = prefetch >> } >> userdb { >> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> protocol lda { >> mail_plugins = quota mail_log notify sieve >> } >> protocol imap { >> mail_max_userip_connections = 20 >> mail_plugins = quota mail_log notify imap_quota imap_zlib >> } >> protocol lmtp { >> mail_plugins = quota mail_log notify sieve >> } >> > > I don't know if it makes a difference for you as you are not using > "LAYOUT=fs" but try to disable those imap_client_workarounds. > Good catch, thank you. (I used LAYOUT=fs at first, but found behavior a little anomalous. I forgot to disable the workarounds). This may have stopped the problem of redownloading the entire mailbox (on restart I got downloading 1 of 8 instead of 1 of 170... 30 of 1330 in the status pane) . My other mailboxes still need to sync and it might take a few days to verify that things are stable. I did get 81 immediate "Warning: Maildir /mail/mydomain.com/gessel//.Bulkmail: Expunged message reappeared, giving a new UID" warnings on connect, but if this is the problem, it might stop after all the directories are resynced. I'll update with further data as it comes in. I am also somewhat suspect of QRSYNC from this post: http://dovecot.org/pipermail/dovecot/2011-August/130687.html - though this is from a long time ago and it seems a little surprising that bugs on this seemingly useful capability would persist. In any event (if useful information), my capability enumeration reads: # telnet localhost 143 * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. a login user pass a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE COMPRESS=DEFLATE QUOTA] Logged in I did try setting mail.server.default.use_condstore;false in Thunderbird, but that didn't change the behavior of reloading the entire inbox on startup so I've changed that back as that wasn't the desired result.