[Dovecot] enourmous amount of disc writes
Hi List, on my mailserver (exim4, dovecot 1.0.0, postgresql, ubuntu dapper, about 500 users) I record an enourmous amount of disc writes (up to 18000 blocks written each second) in the time where most of users are active. This IO activity causes wait-states on the disc interface (RAID5) and finally slows down my mailmachine. Is there a explanation for these disc writes respectively is there a possibility to minimize this activity. Stopping Dovecot on this machine causes stopping most of the disc activity. Regards Rainer Sigl -- - Rainer Sigl, | email: s...@mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] enourmous amount of disc writes
Hi Robert, many thanks for your answer! Yes I'm aware of the necessity to upgrade the complete system. Beside this I wanted to search for a quick solution. Upgrading the system entails many other things and is in my eyes not possible on the living system. Regards Rainer Robert Schetterer wrote: Rainer Sigl schrieb: Hi List, on my mailserver (exim4, dovecot 1.0.0, postgresql, ubuntu dapper, about 500 users) I record an enourmous amount of disc writes (up to 18000 blocks written each second) in the time where most of users are active. This IO activity causes wait-states on the disc interface (RAID5) and finally slows down my mailmachine. Is there a explanation for these disc writes respectively is there a possibility to minimize this activity. Stopping Dovecot on this machine causes stopping most of the disc activity. Regards Rainer Sigl Hi Rainer first you should upgrade to the latest stable of the 1.x branch or better 1.1.x, also dapper is very old perhaps try hardy then retest, i run 2000 users on suse with dovecot 1.0.15 and mysql on sata 3ware hardware raid 1 500 GB on ext3 without any problems -- - Rainer Sigl, | email: s...@mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] enourmous amount of disc writes
Hi Robert, many thanks for your answer! Yes I'm aware of the necessity to upgrade the complete system. Beside this I wanted to search for a quick solution. Upgrading the system entails many other things and is in my eyes not possible on the living system. Did you ever measure the ioactiviy with iostat on your machine? Please tell me the output values. Which hardware do you use? I have a RAID5 with 3ware 7000 series. Regards Rainer Robert Schetterer wrote: Rainer Sigl schrieb: Hi List, on my mailserver (exim4, dovecot 1.0.0, postgresql, ubuntu dapper, about 500 users) I record an enourmous amount of disc writes (up to 18000 blocks written each second) in the time where most of users are active. This IO activity causes wait-states on the disc interface (RAID5) and finally slows down my mailmachine. Is there a explanation for these disc writes respectively is there a possibility to minimize this activity. Stopping Dovecot on this machine causes stopping most of the disc activity. Regards Rainer Sigl Hi Rainer first you should upgrade to the latest stable of the 1.x branch or better 1.1.x, also dapper is very old perhaps try hardy then retest, i run 2000 users on suse with dovecot 1.0.15 and mysql on sata 3ware hardware raid 1 500 GB on ext3 without any problems -- - Rainer Sigl, | email: s...@mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] enourmous amount of disc writes
Hi Richard, many thanks for you answer. yes, this solution did we discuss as well. For a future testing system I will try this. But I can't use it on the living system in the moment. Regards Rainer Richard Hobbs wrote: Hello, You might also want to enable noatime in the mount options for the volume housing the data... if you're using maildir, this *could* reduce the number of writes by half, theoretically! Richard. Robert Schetterer wrote: Rainer Sigl schrieb: Hi List, on my mailserver (exim4, dovecot 1.0.0, postgresql, ubuntu dapper, about 500 users) I record an enourmous amount of disc writes (up to 18000 blocks written each second) in the time where most of users are active. This IO activity causes wait-states on the disc interface (RAID5) and finally slows down my mailmachine. Is there a explanation for these disc writes respectively is there a possibility to minimize this activity. Stopping Dovecot on this machine causes stopping most of the disc activity. Regards Rainer Sigl Hi Rainer first you should upgrade to the latest stable of the 1.x branch or better 1.1.x, also dapper is very old perhaps try hardy then retest, i run 2000 users on suse with dovecot 1.0.15 and mysql on sata 3ware hardware raid 1 500 GB on ext3 without any problems -- - Rainer Sigl, | email: s...@mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ smime.p7s Description: S/MIME Cryptographic Signature
[Dovecot] [Fwd: Re: enourmous amount of disc writes]
Original Message Subject: Re: [Dovecot] enourmous amount of disc writes Date: Mon, 15 Jun 2009 18:48:50 +0200 From: Rainer Sigl s...@mpe.mpg.de To: Ralf Hildebrandt ralf.hildebra...@charite.de References: 4a3640c5.1070...@mpe.mpg.de 4a364233.6090...@schetterer.org 4a365913.6030...@mpe.mpg.de 20090615142836.gx29...@charite.de ok, ich haette gerne von iostat 1 die folgenden Werte nach dem ersten. Die erste Zeile zeigt immer einen overall average seit dem letzten Boot an. Dieser Wert ist bei mir auch rel. harmlos. Die Spitzen sind untertags. Gruss Rainer Ralf Hildebrandt wrote: * Rainer Sigl s...@mpe.mpg.de: Hi Robert, many thanks for your answer! Yes I'm aware of the necessity to upgrade the complete system. Beside this I wanted to search for a quick solution. Upgrading the system entails many other things and is in my eyes not possible on the living system. Du wärst erstaunt was alles geht... Did you ever measure the ioactiviy with iostat on your machine? Please tell me the output values. Which hardware do you use? Welches iostat output willst Du? wir haben READ 2.0MByte/s WRITE 0.5MBytes/s bei 12.000 Usern. -- - Rainer Sigl, | email: s...@mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ -- - Rainer Sigl, | email: s...@mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] Namespace Problems
Many thanks,
its working fine with your suggestion.
But only with outlook I still have problems. Today morning I found out
that outlook does work only I keep the separator line empty
with
namespace private {
# separator = .
prefix = INBOX.
inbox = yes
hidden = no
}
But the result isn't satisfying. Outlook means it cannont connect to
server after connection was established, timeouts, goes in offline modus
etc.
Rainer
Kenny Dail schrieb:
Hi all,
after an update from dovecot beta version to 1.0.0.-1 today morning I
had different effects:
thunderbird-, kmail-, Apple-clients, pine and squirrelmail could
receive email correct with imaps. But not MSOutlook!!!
Varying the namespace configurations in dovecot.conf I got different
results with MSOutlook working but with thunderbird, squirrelmail etc.
all folders are displayed but not the root Inbox etc.
Its important to know that I migrated from courier-imap to dovecot.
My well working namespace config BEFORE upgrade to 1.0.0-1:
Here my actual namespace configs:
namespace private {
separator =
prefix = INBOX.
inbox = no
hidden = no
}
namespace private {
separator = /
prefix = INBOX/
inbox = no
hidden = yes
}
namespace private {
separator = .
prefix = INBOX.
inbox = yes
hidden = yes
}
Try it using only one namespace which matches what you are really doing.
With 1.0 this works for me with all clients I've tried, including Apple.
namespace private {
separator = .
prefix = INBOX.
inbox = yes
hidden = no
}
In varying inbox and hidden keywords I found this configuration. Most
things are working with this but in squirrelmal I can't access inbox,
the main folder.
How do I have to configure that everythings works satisfying??
dovecot.conf and dovecot-sql.conf I send in the attachment.
instead of sending dovecot.conf, next time use dovecot -n
--
-
Rainer Sigl, | email: [EMAIL PROTECTED]
MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557
Postfach 1312,| fax: +49 (89) 3-3569
D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
[Dovecot] Namespace Problems
Hi all,
after an update from dovecot beta version to 1.0.0.-1 today morning I
had different effects:
thunderbird-, kmail-, Apple-clients, pine and squirrelmail could
receive email correct with imaps. But not MSOutlook!!!
Varying the namespace configurations in dovecot.conf I got different
results with MSOutlook working but with thunderbird, squirrelmail etc.
all folders are displayed but not the root Inbox etc.
Its important to know that I migrated from courier-imap to dovecot.
My well working namespace config BEFORE upgrade to 1.0.0-1:
namespace private {
#separator = /
prefix = INBOX.
# prefix =
inbox = yes
#hidden = yes
}
#the second namespace I need to work with Apple because they use /
namespace private {
separator = /
prefix = INBOX/
inbox = no
hidden = yes
}
Here my actual namespace configs:
namespace private {
separator =
prefix = INBOX.
inbox = no
hidden = no
}
namespace private {
separator = /
prefix = INBOX/
inbox = no
hidden = yes
}
namespace private {
separator = .
prefix = INBOX.
inbox = yes
hidden = yes
}
In varying inbox and hidden keywords I found this configuration. Most
things are working with this but in squirrelmal I can't access inbox,
the main folder.
How do I have to configure that everythings works satisfying??
dovecot.conf and dovecot-sql.conf I send in the attachment.
Many thanks
Rainer Sigl
--
-
Rainer Sigl, | email: [EMAIL PROTECTED]
MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557
Postfach 1312,| fax: +49 (89) 3-3569
D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
--
-
Rainer Sigl, | email: [EMAIL PROTECTED]
MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557
Postfach 1312,| fax: +49 (89) 3-3569
D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
## Dovecot configuration file
# '#' character and everything after it is treated as comments. Extra spaces
# and tabs are ignored. If you want to use either of these explicitly, put the
# value inside quotes, eg.: key = # char and trailing whitespace
# Default values are shown after each value, it's not required to uncomment
# any of the lines. Exception to this are paths, they're just examples
# with real defaults being based on configure options. The paths listed here
# are for configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var
# --with-ssldir=/etc/ssl
# Base directory where to store runtime data.
#base_dir = /var/run/dovecot/
# Protocols we want to be serving:
# imap imaps pop3 pop3s
#protocols = imap imaps
protocols = imaps pop3s
# IP or host address where to listen in for connections. It's not currently
# possible to specify multiple addresses. * listens in all IPv4 interfaces.
# [::] listens in all IPv6 interfaces, but may also listen in all IPv4
# interfaces depending on the operating system. If you want to specify ports
# for each service, you will need to configure these settings inside the
# protocol imap/pop3 { ... } section, so you can specify different ports
# for IMAP/POP3.
#listen = *
# IP or host address where to listen in for SSL connections. Defaults
# to above if not specified.
#ssl_listen =
# Disable SSL/TLS support.
#ssl_disable = no
# PEM encoded X.509 SSL/TLS certificate and private key. They're opened before
# dropping root privileges, so keep the key file unreadable by anyone but
# root.
#ssl_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
#ssl_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
ssl_cert_file = /etc/ssl/certs/mpemail.pem
ssl_key_file = /etc/ssl/private/mpemail-key.pem
# If key file is password protected, give the password here. Alternatively
# give it when starting dovecot with -p parameter.
#ssl_key_password =
# File containing trusted SSL certificate authorities. Usually not needed.
#ssl_ca_file =
# Request client to send a certificate.
#ssl_verify_client_cert = no
# How often to regenerate the SSL parameters file. Generation is quite CPU
# intensive operation. The value is in hours, 0 disables regeneration
# entirely.
#ssl_parameters_regenerate = 168
# SSL ciphers to use
#ssl_cipher_list = ALL:!LOW
# Disable LOGIN command and all other plaintext authentications unless
# SSL/TLS is used (LOGINDISABLED capability). Note that 127.*.*.* and
# IPv6 ::1 addresses are considered secure, this setting has no effect if
# you connect from those addresses.
#disable_plaintext_auth = yes
# Use this logfile instead of syslog(). /dev/stderr can be used if you want to
# use stderr for logging (ONLY /dev/stderr - otherwise it is closed).
log_path = /var/log/dovecot.log
# For informational messages, use this logfile instead of the default
info_log_path = /var/log/dovecot-info.log
# Prefix for each line written to log file. % codes are in strftime(3)
# format.
log_timestamp = %Y-%m-%d %H:%M:%S
# Syslog facility
[Dovecot] Namespace Problems repost without attachment
Hi all,
after an update from dovecot beta version to 1.0.0.-1 today morning I
had different effects:
thunderbird-, kmail-, Apple-clients, pine and squirrelmail could
receive email correct with imaps. But not MSOutlook!!!
Varying the namespace configurations in dovecot.conf I got different
results with MSOutlook working but with thunderbird, squirrelmail etc.
all folders are displayed but not the root Inbox etc.
Its important to know that I migrated from courier-imap to dovecot.
My well working namespace config BEFORE upgrade to 1.0.0-1:
namespace private {
#separator = /
prefix = INBOX.
# prefix =
inbox = yes
#hidden = yes
}
#the second namespace I need to work with Apple because they use /
namespace private {
separator = /
prefix = INBOX/
inbox = no
hidden = yes
}
Here my actual namespace configs:
namespace private {
separator =
prefix = INBOX.
inbox = no
hidden = no
}
namespace private {
separator = /
prefix = INBOX/
inbox = no
hidden = yes
}
namespace private {
separator = .
prefix = INBOX.
inbox = yes
hidden = yes
}
In varying inbox and hidden keywords I found this configuration. Most
things are working with this but in squirrelmal I can't access inbox,
the main folder.
How do I have to configure that everythings works satisfying??
Many thanks
Rainer Sigl
--
-
Rainer Sigl, | email: [EMAIL PROTECTED]
MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557
Postfach 1312,| fax: +49 (89) 3-3569
D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
[Dovecot] namespace problem
Hi all,
after an update from dovecot beta version to 1.0.0.-1 today morning I
had different effects:
thunderbird-, kmail-, Apple-clients, pine and squirrelmail could
receive email correct with imaps. But not MSOutlook!!!
Varying the namespace configurations in dovecot.conf I got different
results with MSOutlook working but with thunderbird, squirrelmail etc.
all folders are displayed but not the root Inbox etc.
Its important to know that I migrated from courier-imap to dovecot.
My well working namespace config BEFORE upgrade to 1.0.0-1:
namespace private {
#separator = /
prefix = INBOX.
# prefix =
inbox = yes
#hidden = yes
}
#the second namespace I need to work with Apple because they use /
namespace private {
separator = /
prefix = INBOX/
inbox = no
hidden = yes
}
Here my actual namespace configs:
namespace private {
separator =
prefix = INBOX.
inbox = no
hidden = no
}
namespace private {
separator = /
prefix = INBOX/
inbox = no
hidden = yes
}
namespace private {
separator = .
prefix = INBOX.
inbox = yes
hidden = yes
}
In varying inbox and hidden keywords I found this configuration. Most
things are working with this but in squirrelmal I can't access inbox,
the main folder.
How do I have to configure that everythings works satisfying??
Many thanks
Rainer Sigl
[EMAIL PROTECTED]:/etc/dovecot# dovecot -n
# /etc/dovecot/dovecot.conf
log_path: /var/log/dovecot.log
info_log_path: /var/log/dovecot-info.log
log_timestamp: %Y-%m-%d %H:%M:%S
protocols: imaps pop3s
ssl_cert_file: /etc/ssl/certs/mpemail.pem
ssl_key_file: /etc/ssl/private/mpemail-key.pem
login_dir: /var/run/dovecot/login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(pop3): /usr/lib/dovecot/pop3-login
login_chroot: no
verbose_proctitle: yes
mail_extra_groups: mail
default_mail_env: maildir:/home/virtual_mail/%u
mail_location: maildir:/home/virtual_mail/%u
mail_debug: yes
mail_executable(default): /usr/lib/dovecot/imap
mail_executable(imap): /usr/lib/dovecot/imap
mail_executable(pop3): /usr/lib/dovecot/pop3
mail_plugin_dir(default): /usr/lib/dovecot/modules/imap
mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3
pop3_uidl_format(default):
pop3_uidl_format(imap):
pop3_uidl_format(pop3): UID%u-%v
namespace:
type: private
prefix: INBOX.
namespace:
type: private
separator: /
prefix: INBOX/
hidden: yes
namespace:
type: private
separator: .
prefix: INBOX.
inbox: yes
hidden: yes
auth default:
passdb:
driver: sql
args: /etc/dovecot/dovecot-sql.conf
userdb:
driver: static
args: uid=1002 gid=1002 home=/home/virtual_mail/%u
userdb:
driver: passwd
userdb:
driver: sql
args: /etc/dovecot/dovecot-sql.conf
--
-
Rainer Sigl, | email: [EMAIL PROTECTED]
MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557
Postfach 1312,| fax: +49 (89) 3-3569
D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
Re: [Dovecot] deleted Messages don't go into Trash Folder always
Many thanks, I'm afraid I have to update first. My kubuntu dapper installation has an older version of dovecot. Rainer Timo Sirainen schrieb: On Tue, 2007-05-15 at 19:49 +0200, Rainer Sigl wrote: Hi all, most of the time deleted messages aren't moved into trash folder but go into nirwana. Sometimes it works correctly. But a restart does not guarantee this function. This effect exists with different imap clients. Can anybody help please. IMAP server doesn't move deleted messages to Trash. Some clients can be configured to do that, so if it's not working it's either a client configuration issue or there's something else wrong. You could enable mail_log plugin to see what the clients are doing (http://wiki.dovecot.org/Plugins/MailLog). Also look at Dovecot's error logs to see if there's anything (http://wiki.dovecot.org/Logging) -- - Rainer Sigl, | email: [EMAIL PROTECTED] MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
[Dovecot] deleted Messages don't go into Trash Folder always
Hi all, most of the time deleted messages aren't moved into trash folder but go into nirwana. Sometimes it works correctly. But a restart does not guarantee this function. This effect exists with different imap clients. Can anybody help please. Rainer Sigl -- - Rainer Sigl, | email: [EMAIL PROTECTED] MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
Re: [Dovecot] header-lines of subfolders have to be rescanned after each client startup
Timo Sirainen schrieb: On Thu, 2007-05-10 at 15:53 +0200, Rainer Sigl wrote: Hi all, after each startup of my different mail clients (thunderbird, outlook, ...) the contents of subfolders if I click in has to be renewed. This takes some time for large folders. I just migrated from courier. And there all subfolders were available immediately. Is there something to configure? Another effect regarding subfolders is that the client forgets, which subfolders were opened after shutdown and shows the folders closed. This means, that necessary subfolders have to be reopened after each startup. This was managed as well by my previous courier server. Many thanks, no, I don't use NFS but it may be a permission problem. I have virtual users so all the directories belong a artificial user 'exim' and also group 'exim'. Additionally I saw now that the file dovecot-uidlist has the protection 600. The creation date of dovecot-uidlist is the date when I synced first time the folders with dovecot after installation. But strangely a change of the permissions to 666 has no effect, dove-cot isn't changed. A additional info may be that I migrated von courier imap. So the mainfolder INBOX. and single subfolders have a correct changing dovecot-uidlist with 600 and exim exim. All subfolders below the upper subfolders (.Sent.2005.2-Mar) don't change dovecot-uidlist. Rainer Sounds like something's really wrong in here. Are you using NFS? Do you see dovecot-uidlist file in each subfolder directory? Does it get changed? -- - Rainer Sigl, | email: [EMAIL PROTECTED] MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
[Dovecot] header-lines of subfolders have to be rescanned after each client startup
Hi all, after each startup of my different mail clients (thunderbird, outlook, ...) the contents of subfolders if I click in has to be renewed. This takes some time for large folders. I just migrated from courier. And there all subfolders were available immediately. Is there something to configure? Another effect regarding subfolders is that the client forgets, which subfolders were opened after shutdown and shows the folders closed. This means, that necessary subfolders have to be reopened after each startup. This was managed as well by my previous courier server. Many thanks for answers Rainer Sigl -- - Rainer Sigl, | email: [EMAIL PROTECTED] MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
