> -----Original Message----- > From: dovecot-boun...@dovecot.org [mailto:dovecot- > boun...@dovecot.org] On Behalf Of Michael M Slusarz > Quoting Simon Brereton <simon.brere...@buongiorno.com>: > > >> -----Original Message----- > >> From: dovecot-boun...@dovecot.org [mailto:dovecot- > >> boun...@dovecot.org] On Behalf Of Simon Brereton > >> > -----Original Message----- > >> > From: Timo Sirainen [mailto:t...@iki.fi] On Fri, 2011-09-09 at > 13:07 > >> > -0400, Simon Brereton wrote: > >> > > >> > > I have a server that's been running Courier for about 6 years > and > >> > in > >> > > all that time I think I've only ever had 1 issues where an > entire > >> > mail > >> > > box was repopped by a webmail client. However, since moving > to a > >> > new > >> > > server and dovecot 4 weeks ago, I've now had the webmail > client > >> > repop > >> > > this account 4 times (there are about 230 mails in the > account). > >> > > > >> > > Is there a setting I need to tighten to prevent/remedy this? > I > >> > have > >> > > no idea if it's happening on other accounts, but this is one > that > >> I > >> > > see. The format is maildir. There has been no changes to the > >> > webmail > >> > > client. > >> > > >> > dovecot -n output would have been nice. Also do you see anything > in > >> > error logs? > >> > >> Ah. My apologies of course. Here it is.. > >> > >> mail:~# dovecot -n > >> # 1.2.15: /etc/dovecot/dovecot.conf > >> # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.2 ext3 > >> log_timestamp: %Y-%m-%d %H:%M:%S > >> protocols: imap imaps pop3 pop3s > >> ssl_ca_file: /etc/ssl/keys/rhodes-ca.crt > >> ssl_cert_file: /etc/ssl/keys/mail.domain.net.crt > >> ssl_key_file: /etc/ssl/private/mail.domain.net.key > >> disable_plaintext_auth: no > >> login_dir: /var/run/dovecot/login > >> login_executable(default): /usr/lib/dovecot/imap-login > >> login_executable(imap): /usr/lib/dovecot/imap-login > >> login_executable(pop3): /usr/lib/dovecot/pop3-login > >> mail_privileged_group: mailsystem > >> mail_location: maildir:/var/spool/mail/virtual/%d/%n > >> maildir_very_dirty_syncs: yes > >> mbox_write_locks: fcntl dotlock > >> mail_executable(default): /usr/lib/dovecot/imap > >> mail_executable(imap): /usr/lib/dovecot/imap > >> mail_executable(pop3): /usr/lib/dovecot/pop3 > >> mail_plugins(default): quota imap_quota > >> mail_plugins(imap): quota imap_quota > >> mail_plugins(pop3): quota > >> mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > >> mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > >> mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > >> imap_client_workarounds(default): outlook-idle delay-newmail > >> imap_client_workarounds(imap): outlook-idle delay-newmail > >> imap_client_workarounds(pop3): > >> pop3_client_workarounds(default): > >> pop3_client_workarounds(imap): > >> pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh > >> lda: > >> postmaster_address: postmas...@domain.net > >> mail_plugins: quota > >> log_path: > >> info_log_path: > >> deliver_log_format: msgid=%m: %f: %$ auth default: > >> mechanisms: plain login > >> user: mailsystem > >> verbose: yes > >> passdb: > >> driver: sql > >> args: /etc/dovecot/dovecot-sql.conf > >> userdb: > >> driver: prefetch > >> userdb: > >> driver: static > >> args: uid=999 gid=115 home=/var/spool/mail/virtual/%d/%n > >> allow_all_users=yes > >> socket: > >> type: listen > >> client: > >> path: /var/spool/postfix/private/auth > >> mode: 432 > >> user: postfix > >> group: mailsystem > >> master: > >> path: /var/run/dovecot/auth-master > >> mode: 432 > >> user: mailsystem > >> group: mailsystem > >> plugin: > >> quota: maildir > >> > >> Could you make dovecot -n munge the certificate and postmaster > email > >> addresses? I'm not comfortable with that floating on the > internet.. > >> > >> The only thing I have in the logs is 2 sessions where mail was > popped > >> (note, it doesn't even add up to the 183 messages in the mail > box). > >> But those sessions are vastly longer than the regular ones (tens > of > >> minutes compared to a few seconds). Since both IPs are on the > back- > >> bone, that's quite a while to download 100 mails (none of which > are > >> over > >> > >> Sep 11 21:36:25 mail dovecot: pop3-login: Login: > >> user=<u...@domain.com>, method=PLAIN, rip=64.88.168.84, > >> lip=83.170.65.xxx, TLS Sep 11 21:36:34 mail dovecot: > >> POP3(u...@domain.com): Disconnected: Logged out top=0/0, retr=0/0, > >> del=0/183, size=14025971 Sep 11 21:43:44 mail dovecot: pop3-login: > >> Login: user=<u...@domain.com>, method=PLAIN, rip=64.88.168.84, > >> lip=83.170.65.xxx, TLS Sep 11 21:44:54 mail dovecot: > >> POP3(u...@domain.com): Disconnected: Logged out top=0/0, retr=0/0, > >> del=0/183, size=14025971 Sep 11 21:52:31 mail dovecot: pop3-login: > >> Login: user=<u...@domain.com>, method=PLAIN, rip=64.88.168.84, > >> lip=83.170.65.xxx, TLS Sep 11 22:56:01 mail dovecot: > >> POP3(u...@domain.com): Disconnected: Logged out top=0/0, > >> retr=100/9182678, del=0/183, size=14025971 Sep 11 23:08:58 mail > >> dovecot: pop3-login: Login: user=<u...@domain.com>, method=PLAIN, > >> rip=64.88.168.84, lip=83.170.65.xxx, TLS Sep 11 23:37:57 mail > >> dovecot: POP3(u...@domain.com): Disconnected: Logged out top=0/0, > >> retr=75/4748674, del=0/183, size=14025971 Sep 12 00:04:11 mail > >> dovecot: pop3-login: Login: user=<u...@domain.com>, method=PLAIN, > >> rip=64.88.168.84, lip=83.170.65.xxx, TLS Sep 12 00:04:26 mail > >> dovecot: POP3(u...@domain.com): Disconnected: Logged out top=0/0, > >> retr=0/0, del=0/183, size=14025971 Sep 12 00:07:40 mail dovecot: > >> pop3-login: Login: user=<u...@domain.com>, method=PLAIN, > >> rip=64.88.168.84, lip=83.170.65.xxx, TLS Sep 12 00:07:53 mail > >> dovecot: POP3(u...@domain.com): Disconnected: Logged out top=0/0, > >> retr=0/0, del=0/183, size=14025971 > >> > >> > >> > If you're using the default pop3_uidl_format it'll rely on IMAP > >> UIDs > >> > to stay the same, and I guess it's possible that due to some > other > >> > problem they change (that should be logged as an error/warning > >> > though). > >> > > >> > You could try setting pop3_uidl_format=%f, but it will cause > >> everyone > >> > to redownload mails. With newer Dovecot versions you could set > >> > pop3_save_uidl=yes and when you think everyone's downloaded > mails > >> once > >> > you can safely change the pop3_uidl_format. > >> > >> Sorry, I'm very new to dovecot and I'm not sure I understand. I > >> presume because neither of those keys are in the dovecot -n output > >> that they are as the defaults, yes? The account is indeed > accessed > >> by IMAP as well (from a mobile device mostly), but I don't see > >> anything fishy there either. How could I see if the IMAP UIDs > have > >> changed? > >> > >> Sep 11 21:20:32 mail dovecot: IMAP(u...@domain.com): Connection > >> closed bytes=1095/8292 > >> > >> Sep 11 21:26:03 mail dovecot: imap-login: Login: > >> user=<u...@domain.com>, method=PLAIN, rip=174.252.83.244, > >> lip=83.170.65.xxx, TLS Sep 11 22:11:20 mail dovecot: > >> IMAP(u...@domain.com): Disconnected for inactivity bytes=725/5638 > Sep > >> 11 22:17:10 mail dovecot: imap-login: Login: > user=<u...@domain.com>, > >> method=PLAIN, rip=174.252.83.244, lip=83.170.65.xxx, TLS Sep 11 > >> 23:12:06 mail dovecot: IMAP(u...@domain.com): Disconnected for > >> inactivity bytes=1471/11025 Sep 11 23:23:22 mail dovecot: imap- > login: > >> Login: user=<u...@domain.com>, method=PLAIN, rip=174.252.83.244, > >> lip=83.170.65.xxx, TLS Sep 11 23:52:52 mail dovecot: > >> IMAP(u...@domain.com): Connection closed bytes=1841/13679 Sep 12 > >> 00:08:47 mail dovecot: imap-login: Login: user=<u...@domain.com>, > >> method=PLAIN, rip=174.252.83.244, lip=83.170.65.xxx, TLS Sep 12 > >> 01:19:05 mail dovecot: imap-login: Login: user=<u...@domain.com>, > >> method=PLAIN, rip=174.252.83.244, lip=83.170.65.xxx, TLS Sep 12 > >> 02:57:01 mail dovecot: IMAP(u...@domain.com): Connection closed > >> bytes=2713/60026 Sep 12 02:57:01 mail dovecot: > IMAP(u...@domain.com): > >> Connection closed bytes=2688/18635 > >> > >> > >> There are no errors or warnings in the mail log (I have one shared > >> log file for postfix, amavis and dovecot). Reading the notes for > >> pop3_save_uidl it doesn't seem to be a dangerous option - should > I > >> turn that on? Why will it force everyone to redownload mails > >> (there's nothing about it on the wiki)? > >> > >> Thanks! > >> > >> Simon > > > > Any help would be appreciated. > > What do you mean by "repopped"? You mean downloading the entire data > of the messages from the POP3 server? This is expected behavior when > using a stateless (e.g. webmail) client. Kind of the whole reason > you don't use POP3 in the first place.
Michael - I use a spam filtering service, that uses Horde as the web-front end. Essentially, it pops all my mail accounts (that allow popping) one of which is the one I control and is now running Dovecot - though was previously running Courier. Until now, mails that the service has popped once have never been repopped. That is, I assume that when Horde does a RETR on the account it knows what it has already popped and what it new and only retrieves the new mails. Right now though, it's redownloaded them all 5 or 6 times in 4 weeks. I don't think this is a Horde issue (since that hasn't changed), which is why I didn't post there. Horde continues to be a fantastic project. >From my limited knowledge (meaning I didn't understand the rest of your mail >:) I suspect that Dovecot is doing something with the IDs that Courier wasn't >doing and that's causing Horde to see those old mails as new every now and >again. Simon > Although caching can help. e.g., Here's what the first connection to > the server looks like (this is using IMP 5 on a mailstore with 82 > messages): > > S (1315951197.4976): +OK Dovecot ready. > C (1315951197.513): [AUTH PLAIN Command - username: slusarz] S > (1315951197.5319): +OK Logged in. > C (1315951197.5325): STAT > S (1315951197.5328): +OK 82 482351 > C (1315951197.5348): UIDL > S (1315951197.5354): +OK > S (1315951197.5354): 1 000000014935d409 > S (1315951197.5354): 2 000000024935d409 > S (1315951197.5354): 3 000000114935d409 > [...] > S (1315951197.5363): 82 000000824935d409 S (1315951197.5363): . > C (1315951197.9582): TOP 1 0 > S (1315951198.0411): From u...@domain.com Thu Jun 22 11:16:26 2006 > [...] S (1315951198.0416): . > [...] > C (1315951199.0607): LIST > S (1315951199.061): +OK 82 messages: > S (1315951199.061): 1 118630 > [...] > S (1315951199.0619): . > > We need to grab all headers so we can build the envelope information > (needed to produce the mailbox listing). And the LIST command grabs > the size information (also used in the mailbox listing). > > But remember that the full headers will need to be redownloaded > *EVERY* time you reload the page unless some sort of caching is > enabled in the client. That's just the nature of POP3. (IMAP has > the same sort of issues - if the stateless client does not cache, the > envelope information must be downloaded on every access. However, > with IMAP, the network traffic is reduced - you can download only the > needed information, not all header text - and IMAP servers have the > ability to cache this information behind the scenes due to the > abstraction of the API.). > > This is where caching is pretty much essential on the webmail side. > If caching is enabled, the best-case scenario is that the the webmail > server only needs to grab the list of UIDLs on every POP3 server > access going forward - if the UIDL list has not changed, we know the > mailbox hasn't changed and the cached information is still valid. > (CONDSTORE/QRESYNC extensions for IMAP make this synchronization > check even more efficient in IMAP) > > michael