Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
I was gonna get back on this. Short answer: it's pretty much solved. Long answer: there was indeed a cron that did 'something' with the dovecot-uidlist files. It didn't touch the cache files or anything. I disabled this script and now everything works fine... Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Jan van den Berg Verzonden: donderdag 11 september 2008 21:53 Aan: Dovecot Mailing List Onderwerp: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers I can get those files next Monday without problem and compare them. So I will get back to that. I also posted some part of the rawlog earlier in the thread; dont know if it of any use Thanks, Jan - Original Message - From: "Timo Sirainen" <[EMAIL PROTECTED]> To: "Jan van den Berg" <[EMAIL PROTECTED]> Cc: "Dovecot Mailing List" Sent: Thursday, September 11, 2008 9:31 PM Subject: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
I can get those files next Monday without problem and compare them. So I will get back to that. I also posted some part of the rawlog earlier in the thread; dont know if it of any use Thanks, Jan - Original Message - From: "Timo Sirainen" <[EMAIL PROTECTED]> To: "Jan van den Berg" <[EMAIL PROTECTED]> Cc: "Dovecot Mailing List" Sent: Thursday, September 11, 2008 9:31 PM Subject: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
I can get those files next Monday without problem and compare them. So I will get back to that. I also posted some part of the rawlog earlier in the thread; dont know if it of any use Thanks, Jan - Original Message - From: "Timo Sirainen" <[EMAIL PROTECTED]> To: "Jan van den Berg" <[EMAIL PROTECTED]> Cc: "Dovecot Mailing List" Sent: Thursday, September 11, 2008 9:31 PM Subject: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
I can get those files next Monday without problem and compare them. So I will get back to that. I also posted some part of the rawlog earlier in the thread; dont know if it of any use Thanks, Jan - Original Message - From: "Timo Sirainen" <[EMAIL PROTECTED]> To: "Jan van den Berg" <[EMAIL PROTECTED]> Cc: "Dovecot Mailing List" Sent: Thursday, September 11, 2008 9:31 PM Subject: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Hi, 1.1rc3 also shows this behaviour; I actually upgraded to 1.1.3 to try and solve this.. We had to revert back because of an internally developed IMAP webclient that all of sudden couldn't show attachments anymore with 1.1.3. So we reverted back: and all was fine. So that is more likely a application bug than a Dovecot one (probably with Content-disposition: inline vs. attachment). We're still investigating. But that is beside this point. So as well 1.1rc3 as 1.1.3 has the problem of getting new headers randomly (at least so it seems). It might be NFS related because after getting new headers for a folder I can see the dovecot.index files and uidlist are recreated and a couple .nfs* files are in the dir However I can't explain it (all NFS related Dovecot optimizations are on). Cheers, Jan - Original Message - From: "Timo Sirainen" <[EMAIL PROTECTED]> To: "Jan van den Berg" <[EMAIL PROTECTED]> Cc: "'Dovecot Mailing List'" Sent: Thursday, September 11, 2008 9:11 PM Subject: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Im still trying to figure this problem. Because of other issues (non-dovecot
related but application related) we had to revert back yesterday to 1.1rc3.
This version has been running fine for 6 months.
But now I get quite a few of these errors in my logs I can't really explain
them:
Log synchronization error at seq=2,offset=164 for
/var/spool/mail/j/a/jan.vandenberg.isp/.Sentbox/dovecot.index: ext reset:
invalid record size
This is something that happens since we reverted back:
Number of times this happens in our logs:
dovecot:2679 (so far, today)
dovecot.1:3184 (yesterday)
dovecot.2:0 (tuesday)
dovecot.3:0 (etc.)
dovecot.4:5
dovecot.5:7
dovecot.6:0
dovecot.7:1
We only ran 1.1.3 for a day or so (with zero of these errors occuring).
Before that with 1.1rc3 we also had few of these errors. But now a couple of
1000 a day. It is just a small percent of our total log file: but should I
be worried and what is it?
Cheers,
Jan
- Original Message -
From: "Jan van den Berg" <[EMAIL PROTECTED]>
To: "'Dovecot Mailing List'"
Sent: Thursday, September 11, 2008 11:57 AM
Subject: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew
headers
Hi,
I dont do anything with the dovecot-uidlist.
This is part of what the rawlog looks like (with a folder that started
getting new headers):
tbgy SELECT "My Folders.Klanten"
rty3 UID FETCH 1668:* (UID FLAGS RFC822.SIZE BODY.PEEK[HEADER]
INTERNALDATE)
9d54 UID FETCH 1:1667 (UID FLAGS)
0ntr UID FETCH 1667 (UID FLAGS BODY.PEEK[] INTERNALDATE)
znim IDLE
DONE
4ogq UID FETCH 1668 (UID FLAGS BODY.PEEK[] INTERNALDATE)
5131 IDLE
How should I interpret this? I see 1668 and 1667?
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Timo Sirainen
Verzonden: woensdag 10 september 2008 20:45
Aan: Jan van den Berg
CC: 'Dovecot Mailing List'
Onderwerp: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew
headers
On Wed, 2008-09-10 at 10:29 +0200, Jan van den Berg wrote:
We run 1.1.3 and I use Outlook to connect to my IMAP box and a couple of
times per day (randomly) when I click certain folders IMAP will start
retrieving ALL headers for that folder again ('retrieving new headers')!
Typically that means that the mailbox's UIDVALIDITY has changed. Or the
messages get assigned new UIDs for some reason (e.g. Dovecot sees that
they're gone, later sees them again). You could verify these by using
e.g. rawlog (http://wiki.dovecot.org/Debugging/Rawlog) and seeing if the
UIDVALIDITY value after SELECT command changes and if the returned UIDs
get changed.
Do you do anything on the server with dovecot-uidlist files? Or
dovecot.index* files? Like deleting them once in a while as someone was
doing in a cron job? :)
Re: [Dovecot] error in 1.1.2
I've been following this thread: the bug is related with copying mail across NFS in combination with cache locking, right? Cor uses FBSD; but is this a bug that might impact other platforms as well? Cheers, Jan - Original Message - From: "Cor Bosman" <[EMAIL PROTECTED]> To: "Dovecot Mailing List" Cc: "Cor Bosman" <[EMAIL PROTECTED]> Sent: Thursday, September 11, 2008 5:48 PM Subject: Re: [Dovecot] error in 1.1.2 Conceptually at least, but I don't know if it applies cleanly without them. Cool! Ive just applied all patches you gave me to our production source, and i'll sync it to a few servers to try it out and see :) Cor
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew headers
Hi,
I dont do anything with the dovecot-uidlist.
This is part of what the rawlog looks like (with a folder that started
getting new headers):
tbgy SELECT "My Folders.Klanten"
rty3 UID FETCH 1668:* (UID FLAGS RFC822.SIZE BODY.PEEK[HEADER] INTERNALDATE)
9d54 UID FETCH 1:1667 (UID FLAGS)
0ntr UID FETCH 1667 (UID FLAGS BODY.PEEK[] INTERNALDATE)
znim IDLE
DONE
4ogq UID FETCH 1668 (UID FLAGS BODY.PEEK[] INTERNALDATE)
5131 IDLE
How should I interpret this? I see 1668 and 1667?
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Timo Sirainen
Verzonden: woensdag 10 september 2008 20:45
Aan: Jan van den Berg
CC: 'Dovecot Mailing List'
Onderwerp: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + gettingnew
headers
On Wed, 2008-09-10 at 10:29 +0200, Jan van den Berg wrote:
> We run 1.1.3 and I use Outlook to connect to my IMAP box and a couple of
> times per day (randomly) when I click certain folders IMAP will start
> retrieving ALL headers for that folder again ('retrieving new headers')!
Typically that means that the mailbox's UIDVALIDITY has changed. Or the
messages get assigned new UIDs for some reason (e.g. Dovecot sees that
they're gone, later sees them again). You could verify these by using
e.g. rawlog (http://wiki.dovecot.org/Debugging/Rawlog) and seeing if the
UIDVALIDITY value after SELECT command changes and if the returned UIDs
get changed.
Do you do anything on the server with dovecot-uidlist files? Or
dovecot.index* files? Like deleting them once in a while as someone was
doing in a cron job? :)
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + getting newheaders
Hi, Well I enabled verbose debugs and all but there isn't anything interesting in those logs... Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Patrick Nagel Verzonden: woensdag 10 september 2008 12:17 Aan: Jan van den Berg CC: 'Dovecot Mailing List' Onderwerp: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + getting newheaders -BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jan van den Berg wrote: > I've done some debugging and attached are (parts) of 2 strace files. > > correct.txt. This is when I click on a folder in Outlook and everything is > OK (no headers are retrieved etc.) > > wrong.txt. This is when I click on a folder in Outlook and it starts > retrieving all the headers again. > > Im not sure what to look for in these files. > > Cheers, > > Jan Hm ok, maybe somebody can read something out of the strace output, I can't. The dovecot log output would have been more helpful / readable. Maybe the dovecot.index and/or dovecot-uidlist file in those mailbox folders got corrupted/deleted? But I really know too little about all that... I think dovecot's log output (see http://wiki.dovecot.org/Logging) would really be a great help in figuring out what's going on. Patrick. - -- STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone:+86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779 PGP key: E883A005 https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFIx56F7yMg/OiDoAURAo3VAJwMjqzL0QVR8CVoTkIEVaiK6cB6igCfY5+Q s/UqI42myf3Zto+LtWwFgqQ= =PNFJ -END PGP SIGNATURE-
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + getting newheaders
Hi,
I've done some debugging and attached are (parts) of 2 strace files.
correct.txt. This is when I click on a folder in Outlook and everything is
OK (no headers are retrieved etc.)
wrong.txt. This is when I click on a folder in Outlook and it starts
retrieving all the headers again.
Im not sure what to look for in these files.
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens Jan
van den Berg
Verzonden: woensdag 10 september 2008 11:23
Aan: 'Dovecot Mailing List'
CC: 'Patrick Nagel'
Onderwerp: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + getting
newheaders
Hi,
Here's the dovecot -n output:
# 1.1.3: /usr/local/dovecot/etc/imap-solcon.conf
base_dir: /var/run/imap-solcon/
syslog_facility: local7
protocols: imap
listen: *:143
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /var/run/imap-solcon/login
login_executable: /usr/local/dovecot/libexec/dovecot/imap-login
login_greeting: Solcon MX.
login_process_per_connection: no
max_mail_processes: 1024
mail_location: maildir:/var/spool/mail/%1u/%1.1u/%u
mmap_disable: yes
mail_nfs_storage: yes
mail_nfs_index: yes
maildir_stat_dirs: yes
mail_executable: /usr/local/dovecot/libexec/dovecot/imap
mail_plugins: quota imap_quota
mail_plugin_dir: /usr/local/dovecot/lib/dovecot/imap
imap_client_workarounds: outlook-idle delay-newmail
auth default:
mechanisms: plain digest-md5 cram-md5 login
cache_size: 8192
cache_negative_ttl: 0
executable: /usr/local/dovecot/libexec/dovecot/dovecot-auth
user: exim
passdb:
driver: sql
args: /usr/local/dovecot/etc/imap-solcon-sql-auth.conf
userdb:
driver: prefetch
plugin:
quota: maildir
quota_rule2: Prullenbak:storage=25M
I am a bit hesitative for debugging on production I will try and see if can
set up some sort of test env. to debug.
Cheers,
Jan
-Oorspronkelijk bericht-
Van: Patrick Nagel [mailto:[EMAIL PROTECTED]
Verzonden: woensdag 10 september 2008 11:01
Aan: Jan van den Berg
CC: 'Dovecot Mailing List'
Onderwerp: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + getting new
headers
-BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Jan,
Jan van den Berg wrote:
> We run 1.1.3 and I use Outlook to connect to my IMAP box and a couple of
> times per day (randomly) when I click certain folders IMAP will start
> retrieving ALL headers for that folder again ('retrieving new headers')!
>
> I don't understand this. What triggers IMAP to do so, why not just
retrieve
> only the NEW headers?
>
>
>
> Is it Dovecot or Outlook (probably) and what can be done about this? I
> already use the Dovecot Outlook workarounds.
>
>
>
> I have also gotten word that this behavior happens with Thunderbird.
>
>
>
> This is really a problem causing a lot of load on our storage (I am not
the
> only user).
Please give us some more information. Output of `dovecot -n` would be a good
start. Also check the logs for dovecot reporting problems (and/or switch on
debug logging).
Patrick.
- --
STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/
Phone:+86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779
PGP key: E883A005 https://stshacom1.star-china.net/keys/patrick_nagel.asc
Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iD8DBQFIx4zX7yMg/OiDoAURAtWVAKCiFcHQGzyGoF3NtRzqlH78JGVPIgCeKeOV
v9ixInWS+RWG0gcMFtg5Lk8=
=1uNp
-END PGP SIGNATURE-
gettimeofday({1221039582, 65}, NULL) = 0
open("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP FIP/new",
O_RDONLY|O_LARGEFILE) = 9
fcntl64(9, F_SETSIG, 0x23) = 0
fcntl64(9, 0x402 /* F_??? */, 0x801c) = 0
stat64("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP
FIP/new", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
gettimeofday({1221039582, 111833}, NULL) = 0
open("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP FIP/cur",
O_RDONLY|O_LARGEFILE) = 14
fcntl64(14, F_SETSIG, 0x23) = 0
fcntl64(14, 0x402 /* F_??? */, 0x801c) = 0
stat64("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP
FIP/cur", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
gettimeofday({1221039582, 112415}, NULL) = 0
stat64("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP
FIP/new", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP
FIP/cur", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP FIP",
{st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
chown32("/var/spool/mail/j/a/jan.vandenberg.isp/.My Folders.KPN.KPN NRP
Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + getting new headers
Hi,
Here's the dovecot -n output:
# 1.1.3: /usr/local/dovecot/etc/imap-solcon.conf
base_dir: /var/run/imap-solcon/
syslog_facility: local7
protocols: imap
listen: *:143
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /var/run/imap-solcon/login
login_executable: /usr/local/dovecot/libexec/dovecot/imap-login
login_greeting: Solcon MX.
login_process_per_connection: no
max_mail_processes: 1024
mail_location: maildir:/var/spool/mail/%1u/%1.1u/%u
mmap_disable: yes
mail_nfs_storage: yes
mail_nfs_index: yes
maildir_stat_dirs: yes
mail_executable: /usr/local/dovecot/libexec/dovecot/imap
mail_plugins: quota imap_quota
mail_plugin_dir: /usr/local/dovecot/lib/dovecot/imap
imap_client_workarounds: outlook-idle delay-newmail
auth default:
mechanisms: plain digest-md5 cram-md5 login
cache_size: 8192
cache_negative_ttl: 0
executable: /usr/local/dovecot/libexec/dovecot/dovecot-auth
user: exim
passdb:
driver: sql
args: /usr/local/dovecot/etc/imap-solcon-sql-auth.conf
userdb:
driver: prefetch
plugin:
quota: maildir
quota_rule2: Prullenbak:storage=25M
I am a bit hesitative for debugging on production I will try and see if can
set up some sort of test env. to debug.
Cheers,
Jan
-Oorspronkelijk bericht-
Van: Patrick Nagel [mailto:[EMAIL PROTECTED]
Verzonden: woensdag 10 september 2008 11:01
Aan: Jan van den Berg
CC: 'Dovecot Mailing List'
Onderwerp: Re: [Dovecot] Outlook 2003 and Thunderbird IMAP + getting new
headers
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi Jan,
Jan van den Berg wrote:
> We run 1.1.3 and I use Outlook to connect to my IMAP box and a couple of
> times per day (randomly) when I click certain folders IMAP will start
> retrieving ALL headers for that folder again ('retrieving new headers')!
>
> I don't understand this. What triggers IMAP to do so, why not just
retrieve
> only the NEW headers?
>
>
>
> Is it Dovecot or Outlook (probably) and what can be done about this? I
> already use the Dovecot Outlook workarounds.
>
>
>
> I have also gotten word that this behavior happens with Thunderbird.
>
>
>
> This is really a problem causing a lot of load on our storage (I am not
the
> only user).
Please give us some more information. Output of `dovecot -n` would be a good
start. Also check the logs for dovecot reporting problems (and/or switch on
debug logging).
Patrick.
- --
STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/
Phone:+86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779
PGP key: E883A005 https://stshacom1.star-china.net/keys/patrick_nagel.asc
Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iD8DBQFIx4zX7yMg/OiDoAURAtWVAKCiFcHQGzyGoF3NtRzqlH78JGVPIgCeKeOV
v9ixInWS+RWG0gcMFtg5Lk8=
=1uNp
-END PGP SIGNATURE-
[Dovecot] Outlook 2003 and Thunderbird IMAP + getting new headers
Hi,
We run 1.1.3 and I use Outlook to connect to my IMAP box and a couple of
times per day (randomly) when I click certain folders IMAP will start
retrieving ALL headers for that folder again ('retrieving new headers')!
I don't understand this. What triggers IMAP to do so, why not just retrieve
only the NEW headers?
Is it Dovecot or Outlook (probably) and what can be done about this? I
already use the Dovecot Outlook workarounds.
I have also gotten word that this behavior happens with Thunderbird.
This is really a problem causing a lot of load on our storage (I am not the
only user).
Cheers,
Jan
Re: [Dovecot] Maildirsize automatically (re)build
Hi Charles, Just for testing of course. Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Charles Marcus Verzonden: maandag 10 maart 2008 16:52 Aan: Dovecot Mailing List Onderwerp: Re: [Dovecot] Maildirsize automatically (re)build On 3/10/2008, Jan van den Berg ([EMAIL PROTECTED]) wrote: > 2. I have set a quota limit of 10MB and (extra) 8MB on the Trash > folder. ? Why on earth would you set such ridiculously low quotas? -- Best regards, Charles
Re: [Dovecot] Maildirsize automatically (re)build
Wow,
Formulating the problem for the maillist really helps my find the answer
myself :)
quota_rule=maildir:storage=10M
quota_rule2=Trash:storage=8M
should be
quota_rule=*:storage=10M
quota_rule2=Trash:storage=8M
Now it works!
(Also maildirsize gets created automatically now).
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 16:36
Aan: Dovecot Mailing List
Onderwerp: Re: [Dovecot] Maildirsize automatically (re)build
Ok so basically I have two problems:
1. I dont understand why the maildirsize file isn't automatically
created when it's not there.
2. I have set a quota limit of 10MB and (extra) 8MB on the Trash folder.
Mar 10 16:09:46 mx01 dovecot: auth(default): master out: USER 4
roka-05 uid=1001gid=1001quota_rule=maildir:storage=10M
quota_rule2=Trash:storage=8M
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Loading modules from
directory: /usr/local/dovecot/lib/dovecot/imap
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Module loaded:
/usr/local/dovecot/lib/dovecot/imap/lib10_quota_plugin.so
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Module loaded:
/usr/local/dovecot/lib/dovecot/imap/lib11_imap_quota_plugin.so
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Effective uid=1001,
gid=1001, home=
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Quota root: name=
backend=maildir args=
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=maildir bytes=10485760 messages=0
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=Trash bytes=8388608 messages=0
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): maildir:
data=/var/spool/mail/r/o/roka-05
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): maildir++:
root=/var/spool/mail/r/o/roka-05, index=, control=,
inbox=/var/spool/mail/r/o/roka-05
My maildirsize file gives the right numbers:
#cat maildirsize
10485760S
13951528 318
I have ~13MB in my Inbox but when I try to delete mail (in
SquirrelMail); I get this error:
ERROR: Could not complete request.
Request: COPY 105 "Trash"
Reason Given: Quota exceeded
(The mail I try to delete is only a couple of KBs.)
There should be up to 18MB available in Trash right?
I've looked at some straces and I can see a file is linked to the /tmp
dir from the Trash folder (so write access seems OK). But this file gets
unlinked later.
Could this be a problem:
[pid 5848] rename("/var/spool/mail/r/o/roka-05/dovecot.index.tmp",
"/var/spool/mail/r/o/roka-05/dovecot.index") = 0
This is in the maildir: which is over quota. Or is it the SM
implementation that is wrong and are there any workarounds available?
Cheers,
Jan
-----Oorspronkelijk bericht-
Van: Jan van den Berg
Verzonden: maandag 10 maart 2008 14:53
Aan: Jan van den Berg; Dovecot Mailing List
Onderwerp: RE: [Dovecot] Maildirsize automatically (re)build
Of course the rule is wrong:
":ignore=Spam\ e\-mail" shouldn't be in there.
I fixed it.
But still; why isn't the maildirsize being recreated when it's not
there?
Cheers,
Jan
-Oorspronkelijk bericht-----
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 13:54
Aan: Dovecot Mailing List
Onderwerp: Re: [Dovecot] Maildirsize automatically (re)build
Right...
I made a typo:
userdb_quotarule instead of userdb_quota_rule (weird that Dovecot
doesn't complain about this).
But now I get this error:
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=maildir bytes=10485760 messages=0
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota root QUOTA: Invalid
rule: maildir:storage=10M:ignore=Spam e-mail
Cheers,
Jan
-Oorspronkelijk bericht-----
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 13:40
Aan: Dovecot Mailing List
Onderwerp: [Dovecot] Maildirsize automatically (re)build
Hi,
Im testing with 1.1RC1. I read here:
http://wiki.dovecot.org/Quota/Maildir
"If you configure quota limits in Dovecot (e.g.
quota=maildir:storage=102400(kilobytes)), Dovecot makes sure that this
header is kept up to date. If the file does not exist, it's simply
rebuilt."
This is my query:
password_query = SELECT username as user,
username as userdb_user, passwd as password,
1001 as userdb_uid, 1001 as userdb_gid,
concat('maildir:storage=', quota, 'M:ignore=Spam\ e\-mail') AS
userdb_quotarule,
'Trash:storage=20M' as userdb_quota_rule2
FROM mailboxen
WHERE username = '%n'
Also in the main config I use:
mail_plugins: quota imap_quota
plugin:
quota: maildir
So I use maildir:storage in my config. However when I delete the
maildirsize (for testing purposes) it doesn't get automatically
recreated? According to the Wiki it should right?
Cheers,
Jan
Re: [Dovecot] Maildirsize automatically (re)build
Ok so basically I have two problems:
1. I dont understand why the maildirsize file isn't automatically
created when it's not there.
2. I have set a quota limit of 10MB and (extra) 8MB on the Trash folder.
Mar 10 16:09:46 mx01 dovecot: auth(default): master out: USER 4
roka-05 uid=1001gid=1001quota_rule=maildir:storage=10M
quota_rule2=Trash:storage=8M
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Loading modules from
directory: /usr/local/dovecot/lib/dovecot/imap
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Module loaded:
/usr/local/dovecot/lib/dovecot/imap/lib10_quota_plugin.so
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Module loaded:
/usr/local/dovecot/lib/dovecot/imap/lib11_imap_quota_plugin.so
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Effective uid=1001,
gid=1001, home=
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Quota root: name=
backend=maildir args=
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=maildir bytes=10485760 messages=0
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=Trash bytes=8388608 messages=0
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): maildir:
data=/var/spool/mail/r/o/roka-05
Mar 10 16:09:46 mx01 dovecot: IMAP(roka-05): maildir++:
root=/var/spool/mail/r/o/roka-05, index=, control=,
inbox=/var/spool/mail/r/o/roka-05
My maildirsize file gives the right numbers:
#cat maildirsize
10485760S
13951528 318
I have ~13MB in my Inbox but when I try to delete mail (in
SquirrelMail); I get this error:
ERROR: Could not complete request.
Request: COPY 105 "Trash"
Reason Given: Quota exceeded
(The mail I try to delete is only a couple of KBs.)
There should be up to 18MB available in Trash right?
I've looked at some straces and I can see a file is linked to the /tmp
dir from the Trash folder (so write access seems OK). But this file gets
unlinked later.
Could this be a problem:
[pid 5848] rename("/var/spool/mail/r/o/roka-05/dovecot.index.tmp",
"/var/spool/mail/r/o/roka-05/dovecot.index") = 0
This is in the maildir: which is over quota. Or is it the SM
implementation that is wrong and are there any workarounds available?
Cheers,
Jan
-----Oorspronkelijk bericht-
Van: Jan van den Berg
Verzonden: maandag 10 maart 2008 14:53
Aan: Jan van den Berg; Dovecot Mailing List
Onderwerp: RE: [Dovecot] Maildirsize automatically (re)build
Of course the rule is wrong:
":ignore=Spam\ e\-mail" shouldn't be in there.
I fixed it.
But still; why isn't the maildirsize being recreated when it's not
there?
Cheers,
Jan
-Oorspronkelijk bericht-----
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 13:54
Aan: Dovecot Mailing List
Onderwerp: Re: [Dovecot] Maildirsize automatically (re)build
Right...
I made a typo:
userdb_quotarule instead of userdb_quota_rule (weird that Dovecot
doesn't complain about this).
But now I get this error:
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=maildir bytes=10485760 messages=0
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota root QUOTA: Invalid
rule: maildir:storage=10M:ignore=Spam e-mail
Cheers,
Jan
-Oorspronkelijk bericht-----
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 13:40
Aan: Dovecot Mailing List
Onderwerp: [Dovecot] Maildirsize automatically (re)build
Hi,
Im testing with 1.1RC1. I read here:
http://wiki.dovecot.org/Quota/Maildir
"If you configure quota limits in Dovecot (e.g.
quota=maildir:storage=102400(kilobytes)), Dovecot makes sure that this
header is kept up to date. If the file does not exist, it's simply
rebuilt."
This is my query:
password_query = SELECT username as user,
username as userdb_user, passwd as password,
1001 as userdb_uid, 1001 as userdb_gid,
concat('maildir:storage=', quota, 'M:ignore=Spam\ e\-mail') AS
userdb_quotarule,
'Trash:storage=20M' as userdb_quota_rule2
FROM mailboxen
WHERE username = '%n'
Also in the main config I use:
mail_plugins: quota imap_quota
plugin:
quota: maildir
So I use maildir:storage in my config. However when I delete the
maildirsize (for testing purposes) it doesn't get automatically
recreated? According to the Wiki it should right?
Cheers,
Jan
Re: [Dovecot] Maildirsize automatically (re)build
Of course the rule is wrong:
":ignore=Spam\ e\-mail" shouldn't be in there.
I fixed it.
But still; why isn't the maildirsize being recreated when it's not
there?
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 13:54
Aan: Dovecot Mailing List
Onderwerp: Re: [Dovecot] Maildirsize automatically (re)build
Right...
I made a typo:
userdb_quotarule instead of userdb_quota_rule (weird that Dovecot
doesn't complain about this).
But now I get this error:
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=maildir bytes=10485760 messages=0
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota root QUOTA: Invalid
rule: maildir:storage=10M:ignore=Spam e-mail
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 13:40
Aan: Dovecot Mailing List
Onderwerp: [Dovecot] Maildirsize automatically (re)build
Hi,
Im testing with 1.1RC1. I read here:
http://wiki.dovecot.org/Quota/Maildir
"If you configure quota limits in Dovecot (e.g.
quota=maildir:storage=102400(kilobytes)), Dovecot makes sure that this
header is kept up to date. If the file does not exist, it's simply
rebuilt."
This is my query:
password_query = SELECT username as user,
username as userdb_user, passwd as password,
1001 as userdb_uid, 1001 as userdb_gid,
concat('maildir:storage=', quota, 'M:ignore=Spam\ e\-mail') AS
userdb_quotarule,
'Trash:storage=20M' as userdb_quota_rule2
FROM mailboxen
WHERE username = '%n'
Also in the main config I use:
mail_plugins: quota imap_quota
plugin:
quota: maildir
So I use maildir:storage in my config. However when I delete the
maildirsize (for testing purposes) it doesn't get automatically
recreated? According to the Wiki it should right?
Cheers,
Jan
Re: [Dovecot] Maildirsize automatically (re)build
Right...
I made a typo:
userdb_quotarule instead of userdb_quota_rule (weird that Dovecot
doesn't complain about this).
But now I get this error:
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota rule: root=
mailbox=maildir bytes=10485760 messages=0
Mar 10 13:52:29 mx01 dovecot: IMAP(roka-05): Quota root QUOTA: Invalid
rule: maildir:storage=10M:ignore=Spam e-mail
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Jan van den Berg
Verzonden: maandag 10 maart 2008 13:40
Aan: Dovecot Mailing List
Onderwerp: [Dovecot] Maildirsize automatically (re)build
Hi,
Im testing with 1.1RC1. I read here:
http://wiki.dovecot.org/Quota/Maildir
"If you configure quota limits in Dovecot (e.g.
quota=maildir:storage=102400(kilobytes)), Dovecot makes sure that this
header is kept up to date. If the file does not exist, it's simply
rebuilt."
This is my query:
password_query = SELECT username as user,
username as userdb_user, passwd as password,
1001 as userdb_uid, 1001 as userdb_gid,
concat('maildir:storage=', quota, 'M:ignore=Spam\ e\-mail') AS
userdb_quotarule,
'Trash:storage=20M' as userdb_quota_rule2
FROM mailboxen
WHERE username = '%n'
Also in the main config I use:
mail_plugins: quota imap_quota
plugin:
quota: maildir
So I use maildir:storage in my config. However when I delete the
maildirsize (for testing purposes) it doesn't get automatically
recreated? According to the Wiki it should right?
Cheers,
Jan
[Dovecot] Maildirsize automatically (re)build
Hi,
Im testing with 1.1RC1. I read here:
http://wiki.dovecot.org/Quota/Maildir
"If you configure quota limits in Dovecot (e.g.
quota=maildir:storage=102400(kilobytes)), Dovecot makes sure that this
header is kept up to date. If the file does not exist, it's simply
rebuilt."
This is my query:
password_query = SELECT username as user,
username as userdb_user, passwd as password,
1001 as userdb_uid, 1001 as userdb_gid,
concat('maildir:storage=', quota, 'M:ignore=Spam\ e\-mail') AS
userdb_quotarule,
'Trash:storage=20M' as userdb_quota_rule2
FROM mailboxen
WHERE username = '%n'
Also in the main config I use:
mail_plugins: quota imap_quota
plugin:
quota: maildir
So I use maildir:storage in my config. However when I delete the
maildirsize (for testing purposes) it doesn't get automatically
recreated? According to the Wiki it should right?
Cheers,
Jan
Re: [Dovecot] Outlook 2003 marks email for deletion
Hi, It seems it might have something to do with the Junk email setting. I noticed all the marked-for-deletion mail was automatically moved to the 'Junk email' folder. But here is what I don't understand: Why my previous Courier IMAP server never did this; with the same Outlook Junk email settings. What is so specific about Dovecot that this happens? Could it be that Dovecot and Outlook have a certain negotiation about this. Outlook sends a message to Dovecot about a Junk email and Dovecot then moves this. Can this be confirmed? Can I look for this 'negotiation' in strace? Cheers, Jan -Oorspronkelijk bericht- Van: Maarten Bezemer [mailto:[EMAIL PROTECTED] Verzonden: dinsdag 4 maart 2008 13:45 Aan: Jan van den Berg CC: Dovecot Mailing List Onderwerp: Re: [Dovecot] Outlook 2003 marks email for deletion Hi Jan, On Tue, 4 Mar 2008, Jan van den Berg wrote: > I noticed that when I connect with Outlook 2003; and I start reading new > email some mails get marked (completely random) for deletion (,S -> ,ST) > ?? Did you enable content filtering in Outlook? I've seen things like this before, when Outlook decides the contents of a message are spam or otherwise unwanted. Another possible explanation could be that Outlook has an inconsistent view of the mail 'folder'. Either because you are also using different MUA's with the same IMAP account, or because Outlook's local cache gets bigger than the ANSI limit of 1 (or 2?) GB. [side note: does anybody know how to disable this cache crap? Outlook converts Base64 Unicode-messages to 'parsed' text. In case the original was e.g. in Cyrillic, the local cache will contain '?' for every character not in US-ASCII orso. Moving mails to other folders does not copy the base64 contents, but the 'translated' contents. Thus destroying the message...] Maarten
[Dovecot] Outlook 2003 marks email for deletion
Hi, Im running Dovecot 1.1RC1 (only IMAP). I noticed that when I connect with Outlook 2003; and I start reading new email some mails get marked (completely random) for deletion (,S -> ,ST) ?? I can set this mail to unread, undo the deletion read it again and it will stay OK. But other emails will still get marked. This only seems to happen with Outlook 2003 not with Outlook Express or Thunderbird and only with new unread mail. Any ideas why Outlook marks this mail for deletion? Cheers, Jan
Re: [Dovecot] Dovecot NFS Indexes and IMAP Migration
The bulk of our customers now/still use POP (we still use Courier POP); but we're planning on also offering IMAP as a regular service (so we needed a decent IMAP server). But also because we're planning on rolling out a new in-house developed webmail app. based on IMAP. For now we use SquirrelMail as webmail app. Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Charles Marcus Verzonden: vrijdag 29 februari 2008 11:41 Aan: Dovecot Mailing List Onderwerp: Re: [Dovecot] Dovecot NFS Indexes and IMAP Migration Jan van den Berg, on 2/29/2008 3:45 AM, said the following: > FYI, > > We deployed Dovecot 1.1RC1 yesterday on about 100k+ mailboxes (on > multiple frontends which connect to a storage via NFS). > We handle about one IMAP login every second (for now only from our > webmail app). So far; it's been running like a charm; we come from > Courier IMAP and Dovecot IMAP (with indexes and auth_cache) performs > much faster and puts less load on our frontends and webmail app. Hi Jan, Just out of curiosity... why only webmail? And, which webmail app do you use? -- Best regards, Charles
Re: [Dovecot] Dovecot NFS Indexes and IMAP Migration
FYI, We deployed Dovecot 1.1RC1 yesterday on about 100k+ mailboxes (on multiple frontends which connect to a storage via NFS). We handle about one IMAP login every second (for now only from our webmail app). So far; it's been running like a charm; we come from Courier IMAP and Dovecot IMAP (with indexes and auth_cache) performs much faster and puts less load on our frontends and webmail app. Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Timo Sirainen Verzonden: donderdag 28 februari 2008 18:49 Aan: Andy Dills CC: dovecot@dovecot.org Onderwerp: Re: [Dovecot] Dovecot NFS Indexes and IMAP Migration On Feb 28, 2008, at 3:25 AM, Andy Dills wrote: >> I don't think locking is an issue. NFS caching is more problematic. >> But with >> v1.1's mail_nfs_*=yes settings there should be no problems with >> storing >> indexes on NFS. > > That's good to know. Do you view 1.1 as production ready now, or > should I > wait for an official release? It sounds production ready in the RC1 > release notes, but I was curious how much remains on your "few issues > you'd like to fix" list and how relevant they are to my environment. v1.1 is already used in production in a couple of large installations, so it should probably work quite well. I think most of the crashfixes that are on my TODO list happen only after client has already disconnected.
Re: [Dovecot] Auth_cache not working?
Aha ok! I can see the cache hits and misses now: so that's fine. But I will just keep using my imapproxy to reduce I/O. Thanks Timo. Cheers, Jan -Oorspronkelijk bericht- Van: Timo Sirainen [mailto:[EMAIL PROTECTED] Verzonden: woensdag 27 februari 2008 17:38 Aan: Jan van den Berg CC: dovecot@dovecot.org Onderwerp: Re: [Dovecot] Auth_cache not working? On Wed, 2008-02-27 at 17:33 +0100, Jan van den Berg wrote: > Sort of the same thing: but more inconsistent. Sometimes I click five > folders and OE will have done 3 logins sometimes 2. > > > So new IMAP logins for (more or less) every click in Thunderbird and OE; > this is what auth_cache_size should solve right? No. Dovecot can't affect how and when clients create new connections. auth_cache only reduces number of SQL queries when authenticating. You can determine if it works by adding auth_debug=yes and checking "Hit" and "Miss" log lines.
[Dovecot] Auth_cache not working?
Hi, I have: auth_cache_size = 4096 auth_cache_ttl = 3600 I use Thunderbird connecting to my Dovecot IMAP server 1.1rc1. I click on five folders and close Thunderbird. This is my log: dovecot: Feb 27 17:26:56 Info: Dovecot v1.1.rc1 starting up dovecot: Feb 27 17:26:56 Info: auth-worker(default): mysql: Connected to auth01 (Auth) dovecot: Feb 27 17:27:07 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:08 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:10 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:11 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:13 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:20 Info: IMAP(roka-05): Disconnected: Logged out bytes=83/469 dovecot: Feb 27 17:27:20 Info: IMAP(roka-05): Disconnected: Logged out bytes=106/524 dovecot: Feb 27 17:27:20 Info: IMAP(roka-05): Disconnected: Logged out bytes=71/463 dovecot: Feb 27 17:27:20 Info: IMAP(roka-05): Disconnected: Logged out bytes=356/4887 dovecot: Feb 27 17:27:20 Info: IMAP(roka-05): Disconnected: Logged out bytes=147/939 In Outlook Express: dovecot: Feb 27 17:27:38 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:38 Info: IMAP(roka-05): Disconnected in IDLE bytes=65/112 dovecot: Feb 27 17:27:38 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:38 Info: IMAP(roka-05): Disconnected in IDLE bytes=259/1098 dovecot: Feb 27 17:27:44 Info: imap-login: Login: user=, method=PLAIN, rip=10.4.32.70, lip=10.4.32.235 dovecot: Feb 27 17:27:51 Info: IMAP(roka-05): Disconnected: Logged out bytes=1682/10349 Sort of the same thing: but more inconsistent. Sometimes I click five folders and OE will have done 3 logins sometimes 2. So new IMAP logins for (more or less) every click in Thunderbird and OE; this is what auth_cache_size should solve right? Am I applying the auth_cache_* variables wrong or is something else wrong? Cheers, Jan
Re: [Dovecot] Multiple password schemes
CRYPT and MD5-CRYPT are compatible in pretty much all operating systems.
You could just make CRYPT your default password scheme and it'll work
for both.
I did not know that. But it works!
Cheers,
Jan
-Oorspronkelijk bericht-
Van: Timo Sirainen [mailto:[EMAIL PROTECTED]
Verzonden: dinsdag 26 februari 2008 14:02
Aan: Jan van den Berg
CC: dovecot@dovecot.org
Onderwerp: Re: [Dovecot] Multiple password schemes
On Tue, 2008-02-26 at 13:27 +0100, Jan van den Berg wrote:
> This works; however I noticed because of history a lot of my users
still
> have a CRYPT password (not a MD5-CRYPT password). Now I get these
> errors:
CRYPT and MD5-CRYPT are compatible in pretty much all operating systems.
You could just make CRYPT your default password scheme and it'll work
for both.
Another possibility is to prefix the passwords with {scheme} in the
password database.
[Dovecot] Multiple password schemes
Hi, right now I have default_pass_scheme = MD5-CRYPT in my dovecot-sql-auth.conf file. This works; however I noticed because of history a lot of my users still have a CRYPT password (not a MD5-CRYPT password). Now I get these errors: dovecot: Feb 26 13:08:17 Info: auth-worker(default): sql(***, ***): MD5-CRYPT(0lifant) != 'i3/S6ZswGSfbk' Any way to make Dovecot use another (fallback) password scheme when the default scheme isn't working? Cheers, Jan
Re: [Dovecot] 1.1rc1: Maximum number of mail processes exceeded
Before you start Dovecot type this in you shell: ulimit -n 65000 I don't know how the use of the 'unlimited' word works: maybe this is only 1024? Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Ralf Hildebrandt Verzonden: maandag 25 februari 2008 10:25 Aan: dovecot@dovecot.org Onderwerp: [Dovecot] 1.1rc1: Maximum number of mail processes exceeded I'm getting "Maximum number of mail processes exceeded" messages when 512 imap Processes are active. Dovecot reports: Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 1712). Either grow the limit or change login_max_processes_count and max_mail_processes settings But in my /var/service/dovecot/run script I use: #!/bin/sh mkdir /var/core chmod 1777 /var/core echo /var/core/%p > /proc/sys/kernel/core_pattern ulimit -n unlimited ulimit -c unlimited /usr/local/sbin/dovecot -F So how can I raise the filedescriptor limit above 1024 (linux-2.6.24.2, Debian)? -- Ralf Hildebrandt ([EMAIL PROTECTED]) [EMAIL PROTECTED] Postfix - Einrichtung, Betrieb und Wartung Tel. +49 (0)30-450 570-155 http://www.arschkrebs.de StGB SS328 Absatz 2.3: Mit Freiheitsstrafe bis zu fnf Jahren oder mit Geldstrafe wird bestraft, wer eine nukleare Explosion verursacht
Re: [Dovecot] Dovecot Sieve scalability
Hi, What exactly do you mean with "deliver preps the index"? Dovecot itself preps/creates/alters the index, when retrieving mail; right? Or are you (probably) saying this is one more benefit of using 'deliver'. Anyone know of any 'deliver'/(dovecot) stresstests with regards to forking and threading of the deliver binary? Cheers, Jan -Oorspronkelijk bericht- Van: Gabriel Millerd [mailto:[EMAIL PROTECTED] Verzonden: zaterdag 23 februari 2008 4:01 Aan: Jan van den Berg Onderwerp: Re: [Dovecot] Dovecot Sieve scalability On Fri, Feb 22, 2008 at 2:55 AM, Jan van den Berg <[EMAIL PROTECTED]> wrote: > > The Dovecot/Sieve implementation is functional but not very elegant or > robust. But this can be explained because Dovecot is build for > _retrieving_ mail (imap/pop) and _not_ delivering mail. > Deliver preps the index, makes retrieving email faster. More so for the average disorganized INBOX in my only folder customers. Sieve makes a lot of things smoother for users, the customer. Especially if they expect server side rules. Obviously the rules that drop each message into a Backups or forward messages to other hosts take up what they take up. My sievec files have ancient mtimes. They are definitely not recompiled upon receiving each email. > Any thoughts on this? Are there people out there with large > Dovecot+Sieve implementations (100k+ users). Are there benchmarks > available; how well does it perform under heavy load (mails/sec)? > Depends entirely on the sieve rules in place. -- Gabriel Millerd
Re: [Dovecot] Dovecot Sieve scalability
Thanks for the detailed answer (it helps)! Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens [EMAIL PROTECTED] Verzonden: vrijdag 22 februari 2008 12:36 Aan: dovecot@dovecot.org Onderwerp: Re: [Dovecot] Dovecot Sieve scalability > Hi, > > So you call on a single binary "deliver". This binary then looks for a > .dovecot.sieve file in the user maildir, _compiles_ it to > .dovecot.sievec file, and than drops the mail according to the rules. > Calling a single binary: that's isn't threaded, doesn't fork, and that > has to compile a file (everytime?) it's called isn't scalable. Multiply > this with 100 mails(calls) per second and this is bound for problems. > > The Dovecot/Sieve implementation is functional but not very elegant or > robust. But this can be explained because Dovecot is build for > _retrieving_ mail (imap/pop) and _not_ delivering mail. > > Any thoughts on this? Are there people out there with large > Dovecot+Sieve implementations (100k+ users). Are there benchmarks > available; how well does it perform under heavy load (mails/sec)? > This does not appear to be a problem. I think Dovecot/Sieve is smart enough to only do the compilation if .dovecot.sieve is newer than .dovecot.sievec I'm not sure if this qualifies for your definition of a "large" installations but here is an example of the installation that I manage. It may not be directly transferable to your setup since the level of activity varies a lot from user base to user base. As of this writing there are some 25 concurrent pop3 sessions and some 75 concurrent IMAP sessions (which I guess is relatively low activity considered the size of user base). Anyway I have ~75.000 e-mail addresses on one Dovecot installation and an estimated 200.000 deliveries a day (no guarantee here since deliveries haven't been measured for a long time). This is all located on two SAN luns each consisting of 5x500GB Hitachi SATA disks (RAID-5). These to LUNs are NFS exported (ZFS on Solaris) over Gbit ethernet and mounted as storage for Dovecot (both indexes and mail storage). The total size of Maildir storage+indexes+cache files is ~200GB data. Given the fact that all this happens with the spindles of only ~8 disks (not counting the parity drives) each rated to just ~60 IOPS, and the fact that all operations are done through NFS, I must say that I'm impressed with the performance of Dovecot (although ZFS cache and SAN cache helps a bit). Postfix is the MTA, but local delivery, pop3 and IMAP is all dovecot. The last majority of users have sieve scripts in their homedirs. Albeit only simple ones like putting mails into Junk if the title contains one specific keywork. If you are curious about the CPU usage as well; this activity results in 5-7% percent CPU usage on the server for Dovecot and Postfix. And another 3-5% CPU usage on the mysql database that holds all the account information (for both postfix and dovecot). The server is one Sun T2000 with 8 cores each running 1Ghz. Total RAM usage for this setup is about 6-7 gigs (including the mysql cluster snatching 3 gigs and Postfix smtpd processes eating 2-3 gigs). That being said the bottleneck is mail delivery. The postfix queue is handled on separate local disks (2x73GB 10k RPM SAS in RAID-1) and under heavy load the queue can build op about twice as fast as the delivery to the Maildirs. So if the amount of daily deliveries where do double for instance then I'd definitely have to add at least one more LUN to handle it. I think that the IO penalty for delivery is more due to updating indexes than handling Sieve though...anyway it's performing nicely :) Hope this helps Regards, Mikkel
[Dovecot] Dovecot Sieve scalability
Hi, I just finished setting up a functional design with Dovecot+Sieve and it works like a charm. However I'm having serious doubts about the scalability of this. Here is part of a discussion we're having here: About Dovecot+Sieve. What happens here is that your MTA is configured to pass _all_ email to Dovecot which is configured as a LDA. In practice this means this in the Exim config: command = /usr/local/dovecot/libexec/dovecot/deliver So you call on a single binary "deliver". This binary then looks for a .dovecot.sieve file in the user maildir, _compiles_ it to .dovecot.sievec file, and than drops the mail according to the rules. Calling a single binary: that's isn't threaded, doesn't fork, and that has to compile a file (everytime?) it's called isn't scalable. Multiply this with 100 mails(calls) per second and this is bound for problems. The Dovecot/Sieve implementation is functional but not very elegant or robust. But this can be explained because Dovecot is build for _retrieving_ mail (imap/pop) and _not_ delivering mail. Any thoughts on this? Are there people out there with large Dovecot+Sieve implementations (100k+ users). Are there benchmarks available; how well does it perform under heavy load (mails/sec)? Cheers, Jan
Re: [Dovecot] sieve dues not work
# Enable mail process debugging. This can help you figure out why Dovecot
# isn't finding your mails.
mail_debug = yes
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Carsten Henkel
Verzonden: dinsdag 19 februari 2008 20:49
Aan: dovecot@dovecot.org
Onderwerp: [Dovecot] sieve dues not work
hi together,
there is a problem with my sieve:
System SuSE 10.3 with Postfix
in Postfix i have add to main.cf
mailbox_command = /usr/lib/dovecot/deliver
in /etc/dovecot/dovecot.conf is add:
protocol lda {
# Address to use when sending rejection mails.
postmaster_address = [EMAIL PROTECTED]
...
# Enabling Sieve plugin for server-side mail filtering
mail_plugins = cmusieve
}
~/.dovecot.sieve have this entry:
require "fileinto";
if header :contains "subject" ["test"] {
fileinto "Archiv";
} else {
# The rest goes into INBOX
# default is "implicit keep", we do it explicitly here
keep;
}
but it isnt work, the mails are moved to inbox, not to Archiv there are
any idea?s ? thanks a lot
log:
Feb 19 17:30:23 server postfix/smtpd[31107]: connect from
mail.gmx.net[213.165.64.20]
Feb 19 17:30:23 server postfix/policyd-weight[12323]: decided
action=PREPEND X-policyd-weight: using cached result; rate: -8.5;
<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]>; delay: 0s
Feb 19 17:30:23 server postfix/smtpd[31107]: EE21D858006:
client=mail.gmx.net[213.165.64.20]
Feb 19 17:30:24 server postfix/cleanup[31310]: EE21D858006:
message-id=<[EMAIL PROTECTED]>
Feb 19 17:30:24 server postfix/qmgr[31078]: EE21D858006:
from=<[EMAIL PROTECTED]>, size=1092, nrcpt=1 (queue active)
Feb 19 17:30:24 server postfix/smtpd[31107]: disconnect from
mail.gmx.net[213.165.64.20]
Feb 19 17:30:27 server postfix/smtpd[31323]: connect from
server.wunschradio.de[127.0.0.1]
Feb 19 17:30:27 server postfix/smtpd[31323]: 99B82858017:
client=server.wunschradio.de[127.0.0.1]
Feb 19 17:30:27 server postfix/cleanup[31310]: 99B82858017:
message-id=<[EMAIL PROTECTED]>
Feb 19 17:30:27 server postfix/smtpd[31323]: disconnect from
server.wunschradio.de[127.0.0.1]
Feb 19 17:30:27 server amavis[30697]: (30697-06) Passed CLEAN,
[213.165.64.20] <[EMAIL PROTECTED]> -> <[EMAIL PROTECTED]>,
Message-ID: <[EMAIL PROTECTED]>, mail_id: z92vEMcCmz2c, Hits:
-2.352, size: 1092, queued_as: 99B82858017, 3620 ms
Feb 19 17:30:27 server postfix/qmgr[31078]: 99B82858017:
from=<[EMAIL PROTECTED]>, size=1578, nrcpt=1 (queue active)
Feb 19 17:30:27 server deliver(casi): maildir: root exists
(/home/casi/Maildir)
Feb 19 17:30:27 server deliver(casi): maildir: root=/home/casi/Maildir,
index=/home/casi/Maildir, control=, inbox=
Feb 19 17:30:27 server postfix/smtp[31384]: EE21D858006:
to=<[EMAIL PROTECTED]>, orig_to=<[EMAIL PROTECTED]>,
relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=0.23/0/0/3.6,
dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 99B82858017)
Feb 19 17:30:27 server postfix/qmgr[31078]: EE21D858006: removed
Feb 19 17:30:27 server deliver(casi): msgid=<[EMAIL PROTECTED]>:
saved mail to INBOX
Feb 19 17:30:27 server postfix/local[31327]: 99B82858017:
to=<[EMAIL PROTECTED]>, relay=local, delay=0.27,
delays=0.09/0/0/0.19, dsn=2.0.0, status=sent (delivered to command:
/usr/lib/dovecot/deliver)
Feb 19 17:30:27 server postfix/qmgr[31078]: 99B82858017: removed
--
Laut Statistik ist die Ehe Hauptursache aller Scheidungen.
* Dies ist ein Zufallstext und hat nichts mir dem Empfänger der Mail zu tun
Re: [Dovecot] PAM libraries not found
Im still curious about this: " Also, when configuring, what does (auto) mean (--with-pam (auto)): it's confusing. It gives the impression that, like (default), this option is added 'automatically'. " Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Jan van den Berg Verzonden: vrijdag 15 februari 2008 11:32 Aan: Dovecot Mailing List Onderwerp: [Dovecot] PAM libraries not found When I add the --with-pam option in my configure command I get the following error: checking for pam_start in -lpam... no configure: error: Can't build with PAM support: libpam not found PAM is most certainly installed on the machine: ii libpam-modules0.79-5 Pluggable Authentication Modules for PAM ii libpam-runtime0.79-5 Runtime support for the PAM library ii libpam0g 0.79-5 Pluggable Authentication Modules library Dovecot from package worked just fine with PAM. Also this file already exists: cat /etc/pam.d/dovecot #%PAM-1.0 @include common-auth @include common-account @include common-session I have run ldconfig etc. And I have the following files (all in /lib): /lib/libpamc.so.0 /lib/libpamc.so.0.79 /lib/libpam_misc.so.0 /lib/libpam_misc.so.0.79 /lib/libpam.so.0 /lib/libpam.so.0.79 So it seems that configure is not checking the right places for the libpam library? How can I tell it to look in /lib? Also what does (auto) mean (--with-pam (auto)): it's confusing. It gives the impression that, like (default), this option is added 'automatically'. Cheers, Jan
Re: [Dovecot] PAM libraries not found
... wow How did I miss this? I just noticed that my sources.list is seriously messed up. Therefore no other 'pam' packages showed up in 'apt-cache search pam' (so no libpam0g-dev). I just fixed it; everything is OK now. Thx for the quick reply! Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Ralf Hildebrandt Verzonden: vrijdag 15 februari 2008 11:47 Aan: dovecot@dovecot.org Onderwerp: Re: [Dovecot] PAM libraries not found * Jan van den Berg <[EMAIL PROTECTED]>: > When I add the --with-pam option in my configure command I get the > following error: > > > > checking for pam_start in -lpam... no > > configure: error: Can't build with PAM support: libpam not found > > > > PAM is most certainly installed on the machine: > > ii libpam-modules0.79-5 > Pluggable Authentication Modules for PAM > > ii libpam-runtime0.79-5 > Runtime support for the PAM library > > ii libpam0g 0.79-5 > Pluggable Authentication Modules library And what about: libpam0g-dev ? -- Ralf Hildebrandt ([EMAIL PROTECTED]) [EMAIL PROTECTED] Postfix - Einrichtung, Betrieb und Wartung Tel. +49 (0)30-450 570-155 http://www.arschkrebs.de Quit trying to hurt (slow down) millions abusive IPs. It's a waste of time. Your MX is vastly outnumbered and is pissing into a hurricane.
[Dovecot] PAM libraries not found
When I add the --with-pam option in my configure command I get the following error: checking for pam_start in -lpam... no configure: error: Can't build with PAM support: libpam not found PAM is most certainly installed on the machine: ii libpam-modules0.79-5 Pluggable Authentication Modules for PAM ii libpam-runtime0.79-5 Runtime support for the PAM library ii libpam0g 0.79-5 Pluggable Authentication Modules library Dovecot from package worked just fine with PAM. Also this file already exists: cat /etc/pam.d/dovecot #%PAM-1.0 @include common-auth @include common-account @include common-session I have run ldconfig etc. And I have the following files (all in /lib): /lib/libpamc.so.0 /lib/libpamc.so.0.79 /lib/libpam_misc.so.0 /lib/libpam_misc.so.0.79 /lib/libpam.so.0 /lib/libpam.so.0.79 So it seems that configure is not checking the right places for the libpam library? How can I tell it to look in /lib? Also what does (auto) mean (--with-pam (auto)): it's confusing. It gives the impression that, like (default), this option is added 'automatically'. Cheers, Jan
Re: [Dovecot] Postfix, dovecot and sieve - no response from sieve
Herve probably uses the default Etch package. I don't know who maintains the package: but it certainly is old. Regards, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Charles Marcus Verzonden: dinsdag 12 februari 2008 13:35 Aan: [EMAIL PROTECTED] CC: dovecot@dovecot.org Onderwerp: Re: [Dovecot] Postfix, dovecot and sieve - no response from sieve On 2/12/2008, Hervé Hénoch ([EMAIL PROTECTED]) wrote: > Version : 1.0.rc15 Upgrade... This is very old... -- Best regards, Charles
Re: [Dovecot] Expected size of index-files?
Thanks for the replies this cleared up a lot. Right I think I will be using Dovecot for IMAP (less clients) with index files and keep on using Courier for POP3 (most clients). According to my tests this works OK on the same mailbox. Cheers, Jan -Oorspronkelijk bericht- Van: Timo Sirainen [mailto:[EMAIL PROTECTED] Verzonden: vrijdag 8 februari 2008 17:58 Aan: Jan van den Berg CC: Dovecot Mailing List Onderwerp: Re: [Dovecot] Expected size of index-files? On Fri, 2008-02-08 at 17:45 +0100, Jan van den Berg wrote: > I'm scared to use (index) files that go sort of unnoticed (it's not > calculated in the maildirsize file) and can potentially grow with no > limit. Only some truly badly behaving clients can cause them to grow infinitely. This would require the client to fetch/search more and more unique message headers. For example field1, field2, field3, .. field1000. I don't think this is worth worrying about, although I guess I should add code to prevent this. Added to TODO. > But I am also curious what ":INDEX=MEMORY" will do with 100k users. How > much MB RAM will one IMAP session on average take? The problem isn't how much memory it'll take. It's that Dovecot may use a lot more disk I/O because it has to read and parse messages to find some data instead of doing a couple of small reads from cache file to get the same data. The memory usage is actually less without indexes, because then no fields are ever cached and the space used by dovecot.index.cache isn't mapped to memory. > And will this be > removed from memory (or stay cached) when an IMAP session is closed? Operating system's buffer cache may still contain all the message files' contents that were read during the session (which may have been avoided with indexes on disk), but that's all.
Re: [Dovecot] Expected size of index-files?
(Wow you are fast!) "For webmail type setups indexes help a lot. For Outlook/Thunderbird they help a lot less." Very interesting! I'm scared to use (index) files that go sort of unnoticed (it's not calculated in the maildirsize file) and can potentially grow with no limit. But I am also curious what ":INDEX=MEMORY" will do with 100k users. How much MB RAM will one IMAP session on average take? And will this be removed from memory (or stay cached) when an IMAP session is closed? Cheers, Jan -Oorspronkelijk bericht- Van: Timo Sirainen [mailto:[EMAIL PROTECTED] Verzonden: vrijdag 8 februari 2008 17:33 Aan: Jan van den Berg CC: Dovecot Mailing List Onderwerp: Re: [Dovecot] Expected size of index-files? On Feb 8, 2008, at 6:25 PM, Jan van den Berg wrote: > That's actually quite a lot. > Consider you run 100k+ mailboxes (1GB each) and you would move to > Dovecot this will be a huge chunk out of your storage. > Is using dovecot index files really that much faster than running > ":INDEX=MEMORY" ? Where could I find some benchmarks about this? IMAP benchmarks are difficult to produce because they depend so much on what clients are used (and how many clients user is using, and how user is using the clients). For webmail type setups indexes help a lot. For Outlook/Thunderbird they help a lot less. > so, on average maybe 20%-30% of the size of your messages ? It's actually more to do with the number of messages than the size of the messages. dovecot.index.cache file is the largest one and it completely depends on what IMAP client is used. It contains only the information clients are interested in.
Re: [Dovecot] Expected size of index-files?
That's actually quite a lot. Consider you run 100k+ mailboxes (1GB each) and you would move to Dovecot this will be a huge chunk out of your storage. Is using dovecot index files really that much faster than running ":INDEX=MEMORY" ? Where could I find some benchmarks about this? Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Luuk Verzonden: vrijdag 8 februari 2008 17:19 Aan: dovecot@dovecot.org Onderwerp: Re: [Dovecot] Expected size of index-files? Tobias Balle-Petersen schreef: > Hello... > > Im considering moving my index-files to a RAID1. How big do these > files get? What should I plan for? > > Thanks, > Tobias > since 1 july 2007 i store all mails to this list in 1 folder (Maildir-format) the size of this (and the indexes) is: 31M cur 80K dovecot.index 6.8Mdovecot.index.cache 44K dovecot.index.log 136Kdovecot.index.log.2 4.0Kdovecot-keywords 164Kdovecot-uidlist 4.0Knew 4.0Ktmp so, on average maybe 20%-30% of the size of your messages ?
Re: [Dovecot] Switch from courier to dovecot
Out of curiosity how did you handle the courierimapsubscribed and
courierimapuiddb file name differences? Did you just rename them?
Cheers,
Jan
-Oorspronkelijk bericht-
Van: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Namens
Richard G Riley
Verzonden: donderdag 10 januari 2008 13:30
Aan: Dovecot Mailing List
Onderwerp: [Dovecot] Switch from courier to dovecot
I have an courier IMAP server running. To get sib dirs working I had to
rename them all to begin with a period ".".
e.g
Maildir
Maildir/{cur,new,tmp}
Maildir/.account1/{cur,new,tmp}
Maildir/.account2/{cur,new,tmp}
Maildir/.account3/{cur,new,tmp}
etc
If I uninstall Courier and install dovecot, will these "period" dirs be
seen by default?
I am fairly new to this and sometimes it seems that Maildir format
requirements vary a lot depending on client or server.
Thank you for your help.
Re: [Dovecot] Mixed Courier POP3 and Dovecot IMAP environment
In the past we also patched Courier; so Im not sure about the naming what's right and what's not (I know for a fact what we use now: courierimapuiddb) But you get the idea, right :) We currently don't support (Courier) keywords; so if we decide to support/offer this we can start afresh with Dovecot. -Oorspronkelijk bericht- Van: Timo Sirainen [mailto:[EMAIL PROTECTED] Verzonden: zondag 30 september 2007 14:08 Aan: Jan van den Berg CC: dovecot@dovecot.org Onderwerp: Re: [Dovecot] Mixed Courier POP3 and Dovecot IMAP environment On Mon, 2007-09-24 at 13:29 +0200, Jan van den Berg wrote: > - "Patch" the source so Dovecot IMAP will look for/use > 'courierimapuiddb' instead of 'dovecot-uidlist': otherwise I guess POP3 > and IMAP users will have different UID files, right? Is this smart or > will this still break the UID file? Courier POP3 uses courierpop3uiddb (or something) file for POP3 UIDLs. Unless you're using a really old Courier version. > I'm thinking about patching so the transition can be as transparent as > possible: Also if anything might go wrong in production I want to be > able to switch back to the 'old' Courier IMAP (without renaming files > etc.) I guess then you want to rename dovecot-uidlist. Courier's keyword implementation is also different from Dovecot.
Re: [Dovecot] Mixed Courier POP3 and Dovecot IMAP environment
For now, I've 'hacked'/edited the Dovecot source and everything seems to work OK for me. It makes switching back possible however; I will then need to edit the courierimapsubscribed file (INBOX.etc). Because Dovecot handles this namespace correctly and Courier doesn't. But using a proxy can be of help too; could you give some pointers/url on how to set up this proxy? Jan -Oorspronkelijk bericht- Van: Ed W [mailto:[EMAIL PROTECTED] Verzonden: vrijdag 28 september 2007 14:06 Aan: Jan van den Berg CC: dovecot@dovecot.org Onderwerp: Re: [Dovecot] Mixed Courier POP3 and Dovecot IMAP environment > I'm thinking about patching so the transition can be as transparent as > possible: Also if anything might go wrong in production I want to be > able to switch back to the 'old' Courier IMAP (without renaming files > etc.) > > Any thoughts on this I am going through the same process at the moment and I think the best solution is to start afresh I have setup a new Dovecot server with the proxy stuff enabled. I can then migrate accounts one at a time to the new machine and other users pointing at the new machine simply get proxied back to the old machine. Can't easily roll users back to the old machine, but it does mean that we can do limited testing and not need to do a big bang approach Ed W
Re: [Dovecot] 1.1 Beta 1 Works!
It's the file descriptors limit you can set/change this with ulimit -n x. Also check the ulimit entry in man bash. Cheers, Jan -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Luciano Mannucci Verzonden: dinsdag 25 september 2007 10:34 Aan: dovecot@dovecot.org Onderwerp: [Dovecot] 1.1 Beta 1 Works! Wow! I'm running 1.1 Beta 1 on OpenSuSE linux 10.2. It seems not bad at all. The only thing I've not quite well understood is this message, that pops up when I start dovecot: Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 1536). Either grow the limit or change login_max_processes_count and max_mail_processes settings Where does dovecot take this 1024 fd limit? My machine-wise limit is some orders of magnitude bigger, af far as I can tell by reading /proc/sys/fs/file-max... luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL/ E-MAIL: [EMAIL PROTECTED] / \ AND POSTINGS/ WWW: http://www.mannucci.ORG/
[Dovecot] Mixed Courier POP3 and Dovecot IMAP environment
Hi, Currently we support Courier POP and IMAP for all our users (users can use IMAP as well as POP at the same time). We are thinking about moving to Dovecot; but first moving away from Courier IMAP to Dovecot IMAP (and maybe later also move to Dovecot POP). This would create a mixed environment; after some research I have the following questions. To not break POP3 Courier and still be able to use Dovecot IMAP: - I have to use the namespace prefix in the dovecot.conf file: I read this in the wiki. - "Patch" the source so Dovecot IMAP will look for/use 'courierimapuiddb' instead of 'dovecot-uidlist': otherwise I guess POP3 and IMAP users will have different UID files, right? Is this smart or will this still break the UID file? - "Patch" the source so Dovecot IMAP will look for/use 'courierimapsubscribed' instead of 'subscriptions' of should I just go with the renaming of the files? I'm thinking about patching so the transition can be as transparent as possible: Also if anything might go wrong in production I want to be able to switch back to the 'old' Courier IMAP (without renaming files etc.) Any thoughts on this? Cheers, Jan
Re: [Dovecot] Courier migrating issues: indexes, maildirsize, update query
You're right I had this in my query:
concat('maildir:', maildrop, username) as mail
I deleted it and now this seems to work for me:
mail_location = maildir:/var/spool/mail/%1u/%1.1u/%u:INDEX=MEMORY
I must say I am _stunned_ by the/your reaction speed on the maillist
(thanks for the wiki this explains a lot).
"Each mailbox has its own separate index files. If the index files are
disabled, the same structures are still kept in the memory"
I suppose that this is how Courier also works; or would this imply that
Dovecot will be a RAM hog?
Cheers,
Jan
-Oorspronkelijk bericht-
Van: Timo Sirainen [mailto:[EMAIL PROTECTED]
Verzonden: woensdag 23 mei 2007 15:33
Aan: Jan van den Berg
CC: dovecot@dovecot.org
Onderwerp: Re: [Dovecot] Courier migrating issues: indexes,
maildirsize,update query
On Wed, 2007-05-23 at 14:36 +0200, Jan van den Berg wrote:
> About the indexes; this thoroughly confusing.
> I don't understand why Dovecot IMAP wants to use index files for a
> maildir++ implementation (this seems to defy the point of a maildir).
Hmm. I suppose I should write a wiki page about the index files.. Done:
http://wiki.dovecot.org/IndexFiles
> Still I can't get it to work (index=memory) this is my line:
>
> mail_location = maildir:/var/spool/mail/%1u/%2u/%u:INDEX=MEMORY
>
> I added INDEX=MEMORY later; all was working fine before. Now however
> IMAP or POP3 no matter what I do; the dovecot.index files are still
> generated.
What do you use as userdb? If you return "mail" from there it overrides
mail_location.
Re: [Dovecot] Courier migrating issues: indexes, maildirsize, update query
Hi Timo,
Thanks a lot for the quick response (you sure are active on the list).
I wasn't aware of the PostLoginScript option I will sure give this a
try.
So from what I understand Dovecot isn't there yet either when it comes
to quota handling (maildirsize). I'll give it another thought on what
will work best for us.
About the indexes; this thoroughly confusing.
I don't understand why Dovecot IMAP wants to use index files for a
maildir++ implementation (this seems to defy the point of a maildir).
Therefore I figured these files were used for POP3. But you made this
clear now; that it is indeed used for IMAP.
Still I can't get it to work (index=memory) this is my line:
mail_location = maildir:/var/spool/mail/%1u/%2u/%u:INDEX=MEMORY
I added INDEX=MEMORY later; all was working fine before. Now however
IMAP or POP3 no matter what I do; the dovecot.index files are still
generated. I kill the dovecot daemon; remove the index files, start the
deamon and either way (POP3 or IMAP) the indexes will re-appear after
connecting to a mailbox.
(Note: this isn't a NFS caching thing). With a trace I can see these
files are created:
9634 rename("/var/spool/mail/t/e/test-05/dovecot.index.log.newlock",
"/var/spool/mail/r/o/roka-05/dovecot.index.log") = 0
9634 rename("/var/spool/mail/t/e/test-05/dovecot.index.tmp",
"/var/spool/mail/r/o/roka-05/dovecot.index") = 0
9634 rename("/var/spool/mail/t/e/test-05/dovecot.index.cache.lock",
"/var/spool/mail/r/o/roka-05/dovecot.index.cache") = 0
Cheers,
Jan
-Oorspronkelijk bericht-
Van: Timo Sirainen [mailto:[EMAIL PROTECTED]
Verzonden: woensdag 23 mei 2007 13:39
Aan: Jan van den Berg
CC: dovecot@dovecot.org
Onderwerp: Re: [Dovecot] Courier migrating issues: indexes,
maildirsize,update query
On Wed, 2007-05-23 at 12:57 +0200, Jan van den Berg wrote:
> I'm in the process of testing Dovecot to see whether it meets our
needs
> to replace our current Courier setup which serves well over 100.000
> mailboxes (pop3 and imap: mysql with NFS)
> So far dovecot seems pretty straight forward; however I ran into a
> couple of things that I'm curious about.
>
> 1. What's the deal with
> dovecot.index/dovecot.index.log/dovecot.index.cache. I understand
> Dovecot uses this for POP3 primarily
I guess you meant to say "for IMAP", which is correct.
> but this will no doubt cause a lot of overhead on our platform.
Yes, they bring a bit too much extra overhead for standard download
+delete POP3 users. v1.1's indexes will work better with POP3 (and most
likely also with NFS).
> Also getting this to work with our NFS
> setup would be a pain (locking etc.). I've been following this
> discussion:
> http://www.dovecot.org/list/dovecot/2006-January/thread.html#10758;
and
> it ends with C. Malony suggesting that an disable index option would
be
> in place; but no further action is taken. Basically I want to get rid
of
> the index files (courier also doesn't use any). Any suggestions; will
> there be a disable-index option?
Have you read http://wiki.dovecot.org/NFS?
You can anyway disable indexes with appending :INDEX=MEMORY to
mail_location setting, but it's probably not such a good idea. POP3
requires that the messages' virtual sizes are known. Dovecot initially
calculates by reading the whole message's contents and then storing the
size to dovecot.index.cache file. If you've disabled indexes, it means
that all the messages' contents are read at the beginning of each POP3
session.
Courier solves this by keeping the message sizes in courierpop3dsizelist
file. I'm thinking about doing something similar for Dovecot v1.1 also.
Alternative way would be to add the file's virtual size into the maildir
filename itself (see ,W= in
http://wiki.dovecot.org/MailboxFormat/Maildir), but Dovecot doesn't add
them internally (but it does use them if they exist) and I'm not sure
how to configure other MDAs to do this.
> 2. One of the reasons we want to get rid of Courier is because the
> maildirsize is often incorrect. Whether this is because of a bad
Courier
> implementation or NFS issues or whatever, I haven't figured out yet.
But
Dovecot's maildirsize implementation works like Courier's, so it's
probable that it's just as broken with NFS:
/* We rely on O_APPEND working in here. That isn't NFS-safe, but
it
isn't necessarily that bad because the file is recreated once
in
a while, and sooner if corruption causes calculations to go
over quota. This is also how Maildir++ spec specifies it
should be
done.. */
> I'd like to test this function in Dovecot; but there seems to be very
> little documentation. I read the link
> http://wiki.dovecot.org/Quota/Dict but t
[Dovecot] Courier migrating issues: indexes, maildirsize, update query
Hi, (my first post to the list) I'm in the process of testing Dovecot to see whether it meets our needs to replace our current Courier setup which serves well over 100.000 mailboxes (pop3 and imap: mysql with NFS) So far dovecot seems pretty straight forward; however I ran into a couple of things that I'm curious about. 1. What's the deal with dovecot.index/dovecot.index.log/dovecot.index.cache. I understand Dovecot uses this for POP3 primarily but this will no doubt cause a lot of overhead on our platform. Also getting this to work with our NFS setup would be a pain (locking etc.). I've been following this discussion: http://www.dovecot.org/list/dovecot/2006-January/thread.html#10758; and it ends with C. Malony suggesting that an disable index option would be in place; but no further action is taken. Basically I want to get rid of the index files (courier also doesn't use any). Any suggestions; will there be a disable-index option? 2. One of the reasons we want to get rid of Courier is because the maildirsize is often incorrect. Whether this is because of a bad Courier implementation or NFS issues or whatever, I haven't figured out yet. But I'd like to test this function in Dovecot; but there seems to be very little documentation. I read the link http://wiki.dovecot.org/Quota/Dict but this info is too scarce. Any pointers on where to go? 3. In our MySQL setup we have a field with the latest poptime and pophost (IP). This info can come in handy when troubleshooting or filtering out inactive mailboxes. With our Courier setup this field gets updated by a rather elaborate script that checks the logs and runs updates queries. I could get this to work for Dovecot; but with Dovecot being more actively in development, I wonder could this be a feature; like after a user_query, another update query? Cheers, Jan
