dovecot error

[Rajesh M]

hello all

we were seeing an error as such in dovecot.log for the first time

we use sogo for active sync

dovecot: master: Warning: service(dict): process_limit (100) reached, client 
connections are being dropped

once we restarted sogo service this issue got resolved

could someone please guide us on how to resolve the above

please find below dovecot config file

###
# More details about Dovecot settings:
#   - http://wiki2.dovecot.org/
#   - http://wiki2.dovecot.org/Variables

# Listen addresses.
#   - '*' means all available IPv4 addresses.
#   - '[::]' means all available IPv6 addresses.
# Listen on all available addresses by default
listen = *

#base_dir = /var/run/dovecot
mail_plugins = quota mailbox_alias acl mail_log notify stats

# Enabled mail protocols.
protocols = pop3 imap sieve lmtp

default_client_limit = 2400

# User/group who owns the message files:
mail_uid = 2000
mail_gid = 2000

# Assign uid to virtual users.
first_valid_uid = 2000
last_valid_uid = 2000


# Logging. Reference: http://wiki2.dovecot.org/Logging
#
# Use syslog
syslog_facility = local5
# Log file path if we use internal log system
#log_path = /var/log/dovecot/dovecot.log

# Debug
#mail_debug = yes
#auth_verbose = yes
#auth_debug = yes
#auth_debug_passwords = yes
# Possible values: no, plain, sha1.
#auth_verbose_passwords = no

# SSL: Global settings.
# Refer to wiki site for per protocol, ip, server name SSL settings:
# http://wiki2.dovecot.org/SSL/DovecotConfiguration
ssl_protocols = !SSLv3
ssl = required
verbose_ssl = no
#ssl_ca = http://wiki2.dovecot.org/LMTP
service lmtp {
user = vmail

# For higher volume sites, it may be desirable to increase the number of
# active listener processes. A range of 5 to 20 is probably good for most
# sites.
process_min_avail = 5

# Logging.
# Require 'log_path =' in 'protocol lmtp {}' block.
executable = lmtp -L

# Listening on socket file and TCP
unix_listener /var/spool/postfix/private/dovecot-lmtp {
user = postfix
group = postfix
mode = 0600
}

inet_listener lmtp {
# Listen on localhost (ipv4)
address = 127.0.0.1
port = 24
}
}

# Virtual mail accounts.
userdb {
args = /etc/dovecot/dovecot-mysql.conf
driver = sql
}
passdb {
args = /etc/dovecot/dovecot-mysql.conf
driver = sql
}

# Master user.
# Master users are able to log in as other users. It's also possible to
# directly log in as any user using a master password, although this isn't
# recommended.
# Reference: http://wiki2.dovecot.org/Authentication/MasterUsers
auth_master_user_separator = *
passdb {
driver = passwd-file
args = /etc/dovecot/dovecot-master-users
master = yes
}

plugin {
# Quota configuration.
# Reference: http://wiki2.dovecot.org/Quota/Configuration
quota = dict:user::proxy::quotadict

# Set default quota rule if no quota returned from SQL/LDAP query.
#quota_rule = *:storage=1G
#quota_rule2 = *:messages=0
#quota_rule3 = Trash:storage=1G
#quota_rule4 = Junk:ignore

# Quota warning.
#
# If user suddenly receives a huge mail and the quota jumps from
# 85% to 95%, only the 95% script is executed.
#
# Only the command for the first exceeded limit is executed, so configure
# the highest limit first.
quota_warning = storage=100%% quota-warning 100 %u
quota_warning2 = storage=95%% quota-warning 95 %u
quota_warning3 = storage=90%% quota-warning 90 %u
quota_warning4 = storage=85%% quota-warning 85 %u

# allow user to become max 10% (or 50 MB) over quota
quota_grace = 10%%
#quota_grace = 50 M

# Custom Quota Exceeded Message.
# You can specify the message directly or read the message from a file.
#quota_exceeded_message = Quota exceeded, please try again later.
#quota_exceeded_message = http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration
# Per-user sieve settings.
sieve_dir = ~/sieve
sieve = ~/sieve/dovecot.sieve

# Global sieve settings.
sieve_global_dir = /var/vmail/sieve
# Note: if user has personal sieve script, global sieve rules defined in
#   sieve_default will be ignored. Please use sieve_before or
#   sieve_after instead.
#sieve_default =

sieve_before = /var/vmail/sieve/dovecot.sieve
#sieve_after =

# The maximum number of redirect actions that can be performed during a
# single script execution.
# The meaning of 0 differs based on your version. For pigeonhole-0.3.0 and
# beyond this means that redirect is prohibited. For older versions,
# however, this means that the number of redirects is unlimited.
sieve_max_redirects = 30

# Use recipient as vacation message sender instead of null sender (<>).
sieve_vacation_send_from_recipient = yes

# Reference: http://wiki2.dovecot.org/Plugins/MailboxAlias
mailbox_alias_old = Sent
mailbox_alias_new = Sent 

dovecot error

[Rajesh M]

dovecot.conf
Description: Binary data

error while compiling dovecot

[Rajesh M]

hello

my server is centos 6 64 bit

while installing dovecot i get this error
Failed to run: ./test-compression

what could be the reason.

could someone guide me please

Version dovecot-ce-2.3.0-1


0 / 7 tests failed
==21130== Conditional jump or move depends on uninitialised value(s)
==21130==at 0x349B00ACB2: ??? (in /usr/lib64/liblzma.so.0.0.0)
==21130==by 0x349B00AEA9: ??? (in /usr/lib64/liblzma.so.0.0.0)
==21130==by 0x349B003121: ??? (in /usr/lib64/liblzma.so.0.0.0)
==21130==by 0x349B0061A0: ??? (in /usr/lib64/liblzma.so.0.0.0)
==21130==by 0x349B005212: lzma_easy_encoder (in /usr/lib64/liblzma.so.0.0.0)
==21130==by 0x406ACF: o_stream_create_lzma (ostream-lzma.c:206)
==21130==by 0x403D89: test_compression_handler (test-compression.c:36)
==21130==by 0x4040BD: test_compression (test-compression.c:96)
==21130==by 0x408281: test_run_funcs (test-common.c:244)
==21130==by 0x4083E0: test_run (test-common.c:315)
==21130==by 0x40398B: main (test-compression.c:308)
==21130==
Failed to run: ./test-compression
make[3]: *** [check-local] Error 1
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-ce-2.3.0/src/lib-compression'
make[2]: *** [check-am] Error 2
make[2]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-ce-2.3.0/src/lib-compression'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/root/rpmbuild/BUILD/dovecot-ce-2.3.0/src'
make: *** [check-recursive] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.LCwYD9 (%check)


RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.LCwYD9 (%check)

thanks,
rajesh

concering dovecot logging

[Rajesh M]

hi

had a question concerning dovecot lda logging.

deliver_log_format = From:<%f>-<%e> :: Subject:<%s> :: Status:<%$> ::
MsgID:<%m> :: Size<%p> :: vSize<%w>

am getting %e ie envelope sender as MAILER-DAEMON in the logs

Sep 11 08:35:50 lda(n...@xxx.com): Info: sieve: 
From:- :: Subject: :: 
Status:> :: 
MsgID:<> 
:: Size<12497> :: vSize<12687>

what could be the reason ?

thanks,
rajesh

error while compiling

[Rajesh M]

hello

this may not be in the scope of this forum, so am sorry about that

i use qmailtoaster with dovecot

i am trying to update dovecot to version 2.29,
using the source rpm
rpmbuild --rebuild --define "dist .qt.el6"  dovecot-2.2.29.1-15.qt.src.rpm

but getting error as follow. help require to identify the cause please

make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/quota'
Making check in quota-clone
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/quota-clone'
make[3]: Nothing to be done for `check'.
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/quota-clone'
Making check in imap-quota
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/imap-quota'
make[3]: Nothing to be done for `check'.
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/imap-quota'
Making check in pop3-migration
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/pop3-migration'
for bin in test-pop3-migration-plugin; do \
  if !  ./$bin; then exit 1; fi; \
done
pop3 migration get hdr sha1 .. : ok
0 / 1 tests failed
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/pop3-migration'
Making check in replication
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/replication'
make[3]: Nothing to be done for `check'.
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/replication'
Making check in snarf
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/snarf'
make[3]: Nothing to be done for `check'.
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/snarf'
Making check in stats
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/stats'
make[3]: Nothing to be done for `check'.
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/stats'
Making check in imap-stats
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/imap-stats'
make[3]: Nothing to be done for `check'.
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/imap-stats'
Making check in mail-crypt
make[3]: Entering directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/mail-crypt'
for bin in test-mail-global-key test-mail-key; do \
  if ! env NOUNDEF=1  ./$bin; then exit 1; fi; \
done
try_load_keys  : ok
test_empty_keyset  : ok
0 / 2 tests failed
generate user key  : ok
generate inbox key ... : ok
cache reset .. : ok
verify keys .. : ok
test-mail-key.c:429: Assert failed: mail_crypt_get_private_key(t, 
mcp_old_box_key_id, FALSE, FALSE, , ) > 0
test: random seed #2 was 2074645272
test-mail-key(root): Error: 
mail_crypt_get_private_key(7c9a1039ea2e4fed73e81dd3ffc3fa22ea4a28352939adde7bf8ea858b00fa4f)
 failed: Cannot decrypt key 
7c9a1039ea2e4fed73e81dd3ffc3fa22ea4a28352939adde7bf8ea858b00fa4f: 
error:100AE081:elliptic curve routines:EC_GROUP_new_by_curve_name:unknown group
test-mail-key.c:438: Assert failed: privkey != NULL
old keys . : FAILED
1 / 5 tests failed
make[3]: *** [check-test] Error 1
make[3]: Leaving directory 
`/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins/mail-crypt'
make[2]: *** [check-recursive] Error 1
make[2]: Leaving directory `/root/rpmbuild/BUILD/dovecot-2.2.29.1/src/plugins'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/root/rpmbuild/BUILD/dovecot-2.2.29.1/src'
make: *** [check-recursive] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.LDWEGt (%check)


RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.LDWEGt (%check)


thanks
rajesh

questions concerning dovecot

[Rajesh M]

hello all

had some basic questions

1) what is the logic behind the numbering of files in the conf.d folder  ... is 
this to define the sequence of execution of the config files ?

2) Concerning global scripts, is the default.sieve, before

i understand that they get executed when the user does not have his private 
sieve script.

are  sieve_before,  sieve_after and sieve_global_path all globalscripts ?

thanks
rajesh

Re: dovecot config for 1500 simultaneous connection

[Rajesh M]

christian

the servers i currently own are dell servers. The servers i plan to buy are 
Dell R530, 2U rack servers with 8 x 3.5 inch drives, with 64 gb ram each, 
Hardware raid. I am  thinking of 2 X 300 gb ssds raid1 and 6 x 2 tb drives in 
raid10 for data. I do not have any experience in setting up drdb (that would be 
my next step) ... primarily using standalone servers with hardware level 
redundancy. For backup-ups i rsync data to backup servers, also use qmail taps 
for backing up sent/recd emails on a per user basis.

i am using mysql authentication backend
my mail server is qmailtoaster, ie with vpopmail, spamdyke, simscan.

you had mentioned as follows
##
With 4000 users and 5 logins/s I can't imagine anything that wouldn't be
able to:
a) put all of the relevant data into RAM and
b) thus handle auth requests in very timely manner.
The example I gave earlier about my servers with 140 logins/s
authenticates against LDAP and isn't very busy doing so while also
fitting all 2.7GB of data for about 1 million users into RAM.
##

>>>>>>>>>>>>>>>>>>how do i put all relevant data into the RAM ?

thanks
rajesh


- Original Message -
From: Christian Balzer [mailto:ch...@gol.com]
To: dovecot@dovecot.org
Sent: Tue, 14 Feb 2017 16:40:01 +0900
Subject:


Hello,

On Mon, 13 Feb 2017 22:53:23 +0530 Rajesh M wrote:

> thanks for your help
>
> happy to say that the performance dramatically improved after i use the high 
> performance settings from here
> http://wiki.dovecot.org/LoginProcess
>
That's why that page is there.

> grep Login: /var/log/mail.log.1 |wc -l
> with the mail.log being of a typical, busy day.
> 412992
>
So about 5 login/s, not that busy, but that of course also depends on your
HW and authentication backend.

> i also picked up the imap and pop3 connections during peak hours
> [root@ns1 domains]# doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print 
> m,n}'
> username# proto (pids)
> (ips)
>
> max figures i got was 535 for imap and 97 for pop
>
>
> Question 1
> i wish to improve the performance further by caching the logins. current the 
> same is kept disable because when user's change passwords then they are not 
> able to immediately login with the new password for some time. How to solve 
> this issue.
>
There is no solution, caching auth data is by default fraught with that
sort of peril.

And how did you determine that caching auth data would actually improve
things?

What is your authentication backend?

With 4000 users and 5 logins/s I can't imagine anything that wouldn't be
able to:
a) put all of the relevant data into RAM and
b) thus handle auth requests in very timely manner.

The example I gave earlier about my servers with 140 logins/s
authenticates against LDAP and isn't very busy doing so while also
fitting all 2.7GB of data for about 1 million users into RAM.

>
>
> further to the above.
> i am planning to put up a large mail server with one of the following options
>
Don't plan 1, always plan a pair and then use whatever you're comfortable
with to turn them into a HA cluster, like Pacemaker with DRBD.

> option 1
> OS and mailqueue : 2 X 600 gb 15k rpm raid 1
What kind of RAID, HW or MD?

> data : 4 X 2000 gb raid 10 for data
> backup : 2 X 2000 gb raid 10 for backup

While with Linux MD a 2 disk RAID10 would be possible, it wouldn't be
particular helpful (slower than a RAID 1 with writes).

> 32 gb ram
>
>
> option 2
> os and mail queue : 4 X 600 gb 15k rpm raid 10
Those 15k HDDs tend to be expensive.
Estimate your expected writes per day (iostat on an existing server can
give you some hints) and get 2 SSDs instead, DC S3520s if 1 DWPD is
sufficient, DC S3610s if you need 3 DWPD (doubtful), S3710 if you need 10
DWPD (very doubtful).
Similar for Samsung DC level SSDs, check their specs.

For example, my busiest mailbox servers only write about 250KB/s averaged,
which mean about 50GB/day and thus would be a fit even for a small S3520.
I'm still using 200GB S3710s because I can afford them, like the speed and
never ever want to worry about wearout.

> data : 4 X 2000 gb raid 10 for data
> 32 gb ram
>
> i am having the OS and mail queue on primary drives
>
> i understand that Raid10 give a faster write and read whereas Raid 1 will 
> have slow writes and fast reads
>
While that is true, you're missing the real difference here, the number of
HDDs. With 4 of them in the RAID10 they will be twice as fast as the 2
disk RAID1.
This can be tweaked even further with the various RAID 10 layout options,
"man md".

> the question is will there be huge performance difference between Raid1 and 
> Raid 10 when it comes small files like mail queue ?
>
Again as explained above, double the IOPS, so

Re: dovecot config for 1500 simultaneous connection

[Rajesh M]

e,
> >>> etc.
> >>> Some are significantly larger:
> >>> ---
> >>>   PID USER  PR  NI  VIRT  RES  SHR S  %CPU %MEMTIME+  COMMAND
> >>> 1033864 mail  20   0 97600  67m  54m S 0  0.1   0:01.15 imap 
> >>> ---
> >>>
> >>> But yes, as somebody who has mailbox servers with 55k+ session the average
> >>> is around 1.6MB.
> >>>
> >>>> If I want to support a max 100,000 IMAP sessions per server, I should 
> >>>> configure the server to have at least 200GBs of SWAP?
> >>>>
> >>> You will want:
> >>
> >>> 2. Understanding how to tune Dovecot and more importantly the overall
> >>> system to such a task (see that PID up there?).
> >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run
> >>> and others like atop actually killing dovecot (performance wise, not
> >>> literally) when doing their obviously flawed cleanup on exit. Some things
> >>> clearly do NOT scale well.
> >>>
> >>> My current goal is to have 100k capable servers that work well, 200k in a
> >>> failover scenario, but that won't be particular enjoyable.
> >>>
> >>> Christian
> >>>
> >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer <ch...@gol.com> wrote:
> >>>>>
> >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote:
> >>>>>
> >>>>>> hello
> >>>>>>
> >>>>>> could somebody with experience let me know the dovecot config file 
> >>>>>> settings to handle around 1500 simultaneous connections over pop3 and 
> >>>>>> 1500 connection over imap simultaneously.
> >>>>>>
> >>>>>
> >>>>> Be very precise here, you expect to see 1500 as the result of
> >>>>> "doveadm who |grep pop3 |wc -l"?
> >>>>>
> >>>>> Because that implies an ungodly number of POP3 connects per second, 
> >>>>> given
> >>>>> the typically short duration of these.
> >>>>>
> >>>>> 1500 IMAP connections (note that frequently a client will have more than
> >>>>> the INBOX open and thus have more than one session and thus process on 
> >>>>> the
> >>>>> server) are a much easier proposition, provided they are of the typical
> >>>>> long lasting type.
> >>>>>
> >>>>> So can you put a number to your expected logins per second (both 
> >>>>> protocols)?
> >>>>>
> >>>>>> my server
> >>>>>>
> >>>>>> server configuration
> >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 
> >>>>>> X 2000
> >>>>>> gb hdd for data (No raid)
> >>>>>>
> >>>>> No RAID and no other replication like DRBD?
> >>>>> Why would you even bother?
> >>>>>
> >>>>> How many users/mailboxes in total with what quota?
> >>>>>
> >>>>> 1500 IMAP sessions will eat up about 3GB alone.
> >>>>> You will want more memory, simply to keep all relevant SLAB bits 
> >>>>> (inodes,
> >>>>> dentries) in RAM.
> >>>>>
> >>>>> If you really have several hundreds logins/s,  you're facing several
> >>>>> bottlenecks:
> >>>>> 1. Login processes themselves (easily fixed by high performance mode)
> >>>>> 2. Auth processes (that will depend on your backends, method mostly)
> >>>>> 3. Dovecot master process (spawning mail processes)
> >>>>>
> >>>>> The later is a single-threaded process, so it will benefit from a faster
> >>>>> CPU core.
> >>>>> It can be dramatically improved by enabling process re-usage, see:
> >>>>> http://wiki.dovecot.org/PerformanceTuning
> >>>>>
> >>>>> However that also means more memory usage.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Christian
> >>>>>
> >>>>>>
> >>>>>> thanks
> >>>>>> rajesh
> >>>>>>
> >>>>>
> >>>>> [snip]
> >>>>> --
> >>>>> Christian BalzerNetwork/Systems Engineer
> >>>>> ch...@gol.com   Global OnLine Japan/Rakuten Communications
> >>>>> http://www.gol.com/
> >>>>
> >>>
> >>>
> >>> --
> >>> Christian BalzerNetwork/Systems Engineer
> >>> ch...@gol.com <mailto:ch...@gol.com> <mailto:ch...@gol.com 
> >>> <mailto:ch...@gol.com>>Global OnLine Japan/Rakuten Communications
> >>> http://www.gol.com/ <http://www.gol.com/> <http://www.gol.com/ 
> >>> <http://www.gol.com/>>
> >
> >
> > --
> > Christian BalzerNetwork/Systems Engineer
> > ch...@gol.com <mailto:ch...@gol.com>Global OnLine Japan/Rakuten 
> > Communications
> > http://www.gol.com/ <http://www.gol.com/>
>


--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com   Global OnLine Japan/Rakuten Communications
http://www.gol.com/

Re: dovecot config for 1500 simultaneous connection

[Rajesh M]

- Original Message -
From: Christian Balzer [mailto:ch...@gol.com]
To: dovecot@dovecot.org
Cc: 24x7ser...@24x7server.net
Sent: Fri, 10 Feb 2017 17:58:58 +0900
Subject:

On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote:

> hello
>
> could somebody with experience let me know the dovecot config file settings 
> to handle around 1500 simultaneous connections over pop3 and 1500 connection 
> over imap simultaneously.
>

Be very precise here, you expect to see 1500 as the result of
"doveadm who |grep pop3 |wc -l"?

Because that implies an ungodly number of POP3 connects per second, given
the typically short duration of these.

1500 IMAP connections (note that frequently a client will have more than
the INBOX open and thus have more than one session and thus process on the
server) are a much easier proposition, provided they are of the typical
long lasting type.

So can you put a number to your expected logins per second (both protocols)?

> my server
>
> server configuration
> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000
> gb hdd for data (No raid)
>
No RAID and no other replication like DRBD?
Why would you even bother?

How many users/mailboxes in total with what quota?

1500 IMAP sessions will eat up about 3GB alone.
You will want more memory, simply to keep all relevant SLAB bits (inodes,
dentries) in RAM.

If you really have several hundreds logins/s,  you're facing several
bottlenecks:
1. Login processes themselves (easily fixed by high performance mode)
2. Auth processes (that will depend on your backends, method mostly)
3. Dovecot master process (spawning mail processes)

The later is a single-threaded process, so it will benefit from a faster
CPU core.
It can be dramatically improved by enabling process re-usage, see:
http://wiki.dovecot.org/PerformanceTuning

However that also means more memory usage.



Christian

>
> thanks
> rajesh
>

[snip]
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com   Global OnLine Japan/Rakuten Communications
http://www.gol.com/


thanks christian

during peak times here are the results for connections

[root@ns1 domains]# doveadm who |grep imap |wc -l
username# proto (pids) (ips)
631
[root@ns1 domains]# doveadm who |grep pop3 |wc -l
username# proto (pids)  (ips)
233


could you please guide me concerning the dovecot config files settings to 
handle the above 631 imap and 233 pop connections.

number of mailboxes is around 4000 -- some users would consume 25 GB while 
others would be just around 10 MB

this is a hex core machine with hyperthreading -- so 12 cores

[root@ns1 domains]# iostat
Linux 2.6.32-431.29.2.el6.x86_64 (ns1.bizmailserver.net)02/10/2017  
_x86_64_(12 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
   2.67   0.000.65   3.430.00   
93.25

Device:tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sdd  44.95  1094.25   765.10  720884842  504041712
sdc   1.9232.15 0.03   21178186  21248
sdb  34.71  1377.37   625.54  907398402  412102224
sda  49.88   124.29  2587.32   81879548 1704506408



thanks
rajesh

dovecot config for 1500 simultaneous connection

[Rajesh M]

hello

could somebody with experience let me know the dovecot config file settings to 
handle around 1500 simultaneous connections over pop3 and 1500 connection over 
imap simultaneously.

my server

server configuration
hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000
gb hdd for data (No raid)


thanks
rajesh


my current config file

settings as such
# 2.2.7: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final)
# NOTE: Send doveconf -n output instead when asking for help.
auth_anonymous_username = anonymous
auth_cache_negative_ttl = 0
auth_cache_size = 0
auth_cache_ttl = 0
auth_debug = no
auth_debug_passwords = yes
auth_default_realm =
auth_failure_delay = 2 secs
auth_gssapi_hostname =
auth_krb5_keytab =
auth_master_user_separator =
auth_mechanisms = plain login digest-md5 cram-md5
auth_proxy_self =
auth_realms =
auth_socket_path = auth-userdb
auth_ssl_require_client_cert = no
auth_ssl_username_from_cert = no
auth_use_winbind = no
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_username_format = %Lu
auth_username_translation =
auth_verbose = no
auth_verbose_passwords = no
auth_winbind_helper_path = /usr/bin/ntlm_auth
auth_worker_max_count = 30
base_dir = /var/run/dovecot
config_cache_size = 1 M
debug_log_path =
default_client_limit = 1000
default_idle_kill = 1 mins
default_internal_user = dovecot
default_login_user = vpopmail
default_process_limit = 100
default_vsz_limit = 256 M
deliver_log_format = msgid=%m: %$
dict_db_config =
director_doveadm_port = 0
director_mail_servers =
director_servers =
director_user_expire = 15 mins
director_username_hash = %u
disable_plaintext_auth = no
dotlock_use_excl = yes
doveadm_allowed_commands =
doveadm_password =
doveadm_port = 0
doveadm_socket_path = doveadm-server
doveadm_worker_count = 0
dsync_alt_char = _
dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U
first_valid_gid = 89
first_valid_uid = 89
hostname =
imap_capability =
imap_client_workarounds =
imap_id_log =
imap_id_send = name *
imap_idle_notify_interval = 2 mins
imap_logout_format = in=%i out=%o
imap_max_line_length = 64 k
imap_metadata = no
imap_urlauth_host =
imap_urlauth_logout_format = in=%i out=%o
imap_urlauth_port = 143
imapc_features =
imapc_host =
imapc_list_prefix =
imapc_master_user =
imapc_max_idle_time = 29 mins
imapc_password =
imapc_port = 143
imapc_rawlog_dir =
imapc_ssl = no
imapc_ssl_verify = yes
imapc_user =
import_environment = TZ DEBUG_OUTOFMEM
info_log_path =
instance_name = dovecot
last_valid_gid = 0
last_valid_uid = 0
lda_mailbox_autocreate = no
lda_mailbox_autosubscribe = no
lda_original_recipient_header =
libexec_dir = /usr/libexec/dovecot
listen = *, ::
lmtp_address_translate =
lmtp_proxy = no
lmtp_rcpt_check_quota = no
lmtp_save_to_detail_mailbox = no
lock_method = fcntl
log_path = /var/log/dovecot.log
log_timestamp = "%b %d %H:%M:%S "
login_access_sockets =
login_greeting = ready.
login_log_format = %$: %s
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
session=<%{session}>
login_trusted_networks =
mail_access_groups =
mail_always_cache_fields =
mail_attachment_dir =
mail_attachment_fs = sis posix
mail_attachment_hash = %{sha1}
mail_attachment_min_size = 128 k
mail_attribute_dict =
mail_cache_fields = flags
mail_cache_min_mail_count = 0
mail_chroot =
mail_debug = no
mail_fsync = optimized
mail_full_filesystem_access = no
mail_gid =
mail_home =
mail_location =
mail_log_prefix = "%s(%u): "
mail_max_keyword_length = 50
mail_max_lock_timeout = 0
mail_max_userip_connections = 10
mail_never_cache_fields = imap.envelope
mail_nfs_index = no
mail_nfs_storage = no
mail_plugin_dir = /usr/lib64/dovecot
mail_plugins = " quota"
mail_prefetch_count = 0
mail_privileged_group =
mail_save_crlf = no
mail_shared_explicit_inbox = no
mail_temp_dir = /tmp
mail_temp_scan_interval = 1 weeks
mail_uid =
mailbox_idle_check_interval = 30 secs
mailbox_list_index = no
maildir_broken_filename_sizes = no
maildir_copy_with_hardlinks = yes
maildir_stat_dirs = no
maildir_very_dirty_syncs = no
managesieve_client_workarounds =
managesieve_implementation_string = Dovecot Pigeonhole
managesieve_logout_format = bytes=%i/%o
managesieve_max_compile_errors = 5
managesieve_max_line_length = 65536
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
master_user_separator =
mbox_dirty_syncs = yes
mbox_dotlock_change_timeout = 2 mins
mbox_lazy_writes = yes
mbox_lock_timeout = 5 mins
mbox_md5 = apop3d
mbox_min_index_size = 0
mbox_read_locks = fcntl
mbox_very_dirty_syncs = no
mbox_write_locks = dotlock fcntl
mdbox_preallocate_space = no
mdbox_rotate_interval = 0
mdbox_rotate_size = 2 M
mmap_disable = no
namespace {
  disabled = no
  hidden = no
  ignore_on_failure 

dovecot logout issues

[Rajesh M]

hi

we are using dovecot version 2.2.7 (config file given below)
centos 6, qmail, vpopmail, mysql

server configuration
hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 
gb hdd for data (No raid)

busy server with around 4000 email ids --- load is around 2 to 10

the issue is that SQUIRRELMAIL webmail users suddenly lose connection while 
they are working on the webmail.

after logging in, if the user tries to open a mail then the interface gives 
error invalid user id or password.

this happens on an extremely random basis.

also htop always shows a few delayed processes of dovecot (shown as D) on top 
-- pop3 and imap

dovecot logs do not show any login error when such a logout takes place.

the said mailbox contained just around 30 emails

and it is not related to the timeout plugin of squirrelmail either since the 
same webmail folders works on other servers of ours without any issues.

webmail load slowly in general

however when it works normally webmail is very fast and able to handle several 
10 s of thousands of emails in the inbox.

ram consumed is 2 - 5 gb during peak hours.

rebooted server but issue not solved

issue is present for the last around 1 month and was not present earlier.

help required please.

thanks
rajesh


settings as such
# 2.2.7: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final)
# NOTE: Send doveconf -n output instead when asking for help.
auth_anonymous_username = anonymous
auth_cache_negative_ttl = 0
auth_cache_size = 0
auth_cache_ttl = 0
auth_debug = no
auth_debug_passwords = yes
auth_default_realm =
auth_failure_delay = 2 secs
auth_gssapi_hostname =
auth_krb5_keytab =
auth_master_user_separator =
auth_mechanisms = plain login digest-md5 cram-md5
auth_proxy_self =
auth_realms =
auth_socket_path = auth-userdb
auth_ssl_require_client_cert = no
auth_ssl_username_from_cert = no
auth_use_winbind = no
auth_username_chars = 
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_username_format = %Lu
auth_username_translation =
auth_verbose = no
auth_verbose_passwords = no
auth_winbind_helper_path = /usr/bin/ntlm_auth
auth_worker_max_count = 30
base_dir = /var/run/dovecot
config_cache_size = 1 M
debug_log_path =
default_client_limit = 1000
default_idle_kill = 1 mins
default_internal_user = dovecot
default_login_user = vpopmail
default_process_limit = 100
default_vsz_limit = 256 M
deliver_log_format = msgid=%m: %$
dict_db_config =
director_doveadm_port = 0
director_mail_servers =
director_servers =
director_user_expire = 15 mins
director_username_hash = %u
disable_plaintext_auth = no
dotlock_use_excl = yes
doveadm_allowed_commands =
doveadm_password =
doveadm_port = 0
doveadm_socket_path = doveadm-server
doveadm_worker_count = 0
dsync_alt_char = _
dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U
first_valid_gid = 89
first_valid_uid = 89
hostname =
imap_capability =
imap_client_workarounds =
imap_id_log =
imap_id_send = name *
imap_idle_notify_interval = 2 mins
imap_logout_format = in=%i out=%o
imap_max_line_length = 64 k
imap_metadata = no
imap_urlauth_host =
imap_urlauth_logout_format = in=%i out=%o
imap_urlauth_port = 143
imapc_features =
imapc_host =
imapc_list_prefix =
imapc_master_user =
imapc_max_idle_time = 29 mins
imapc_password =
imapc_port = 143
imapc_rawlog_dir =
imapc_ssl = no
imapc_ssl_verify = yes
imapc_user =
import_environment = TZ DEBUG_OUTOFMEM
info_log_path =
instance_name = dovecot
last_valid_gid = 0
last_valid_uid = 0
lda_mailbox_autocreate = no
lda_mailbox_autosubscribe = no
lda_original_recipient_header =
libexec_dir = /usr/libexec/dovecot
listen = *, ::
lmtp_address_translate =
lmtp_proxy = no
lmtp_rcpt_check_quota = no
lmtp_save_to_detail_mailbox = no
lock_method = fcntl
log_path = /var/log/dovecot.log
log_timestamp = "%b %d %H:%M:%S "
login_access_sockets =
login_greeting = ready.
login_log_format = %$: %s
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c 
session=<%{session}>
login_trusted_networks =
mail_access_groups =
mail_always_cache_fields =
mail_attachment_dir =
mail_attachment_fs = sis posix
mail_attachment_hash = %{sha1}
mail_attachment_min_size = 128 k
mail_attribute_dict =
mail_cache_fields = flags
mail_cache_min_mail_count = 0
mail_chroot =
mail_debug = no
mail_fsync = optimized
mail_full_filesystem_access = no
mail_gid =
mail_home =
mail_location =
mail_log_prefix = "%s(%u): "
mail_max_keyword_length = 50
mail_max_lock_timeout = 0
mail_max_userip_connections = 10
mail_never_cache_fields = imap.envelope
mail_nfs_index = no
mail_nfs_storage = no
mail_plugin_dir = /usr/lib64/dovecot
mail_plugins = " quota"
mail_prefetch_count = 0
mail_privileged_group =
mail_save_crlf = no
mail_shared_explicit_inbox = no
mail_temp_dir = /tmp
mail_temp_scan_interval = 1 weeks
mail_uid =
mailbox_idle_check_interval = 30 secs
mailbox_list_index = no
maildir_broken_filename_sizes = no
maildir_copy_with_hardlinks = yes

question concerning junk folder

[Rajesh M]

hi

using centos 6.5, qmailtoaster, dovecot, spamassassin.

i found that junk folder is specified under

/etc/dovecot/conf.d/15-mailboxes.conf

the issue is that this folder is getting created automatically and genuine 
emails from both within the server and also from external domains are being 
routed to the Junk folder. all these emails are not scanned by spamassasin.

i wish to disable to creation of Junk folder and also routing of emails to the 
Junk folder.

help required please.


#
# NOTE: Assumes "namespace inbox" has been defined in 10-mail.conf.
namespace inbox {
  # These mailboxes are widely used and could perhaps be created automatically:
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Trash {
special_use = \Trash
  }
#

rajesh

Re: concerning dovecot settings for high volume server

[Rajesh M]

thanks very much urban. this was very helpful.

i have around 12500 users spread over 3 independent servers each having around 
4000+ users
i am using qmailtoaster, vpopmail, spamassassin and dovecot.

in future i am planning to consolidate all using a HA cluster.

if it is ok with you could you kindly share some information about your email 
server configuration. if you do not wish to put it on the list then you can 
directly email me.

1) is your email volume high ?
2) server hardware to support  28000 users
3) mailserver software - exim or postfix ??.
4) antispam software like spamassassin if any

also if you have faced any email re-download issues with dovecot sometimes 
randomly incase of pop3 users storing emails on the server ?


thanks
rajesh



- Original Message -
From: Urban Loesch [mailto:b...@enas.net]
To: dovecot@dovecot.org
Sent: Sun, 13 Sep 2015 09:33:14 +0200
Subject: Re: concerning dovecot settings for high volume server

Hi,

I have running dovecot with about 28k users.
Here comes my relevant config for pop3 and imap from "doveconf -n".
No problems so far.

-- snip --
default_client_limit = 2000
...

service imap-login {
   inet_listener imap {
 port = 143
   }
   process_limit = 256
   process_min_avail = 50
   service_count = 1
}
service imap {
   process_limit = 2048
   process_min_avail = 50
   service_count = 1
   vsz_limit = 512 M
}
...

service pop3-login {
   inet_listener pop3 {
 port = 110
   }
   process_limit = 256
   process_min_avail = 25
   service_count = 1
}
service pop3 {
   process_limit = 256
   process_min_avail = 25
   service_count = 1
}
...

protocol imap {
   imap_client_workarounds = tb-extra-mailbox-sep
   imap_id_log = *
   imap_logout_format = bytes=%i/%o session=<%{session}>
   mail_max_userip_connections = 40
   mail_plugins = " quota mail_log notify zlib imap_quota imap_zlib"
}

...
protocol pop3 {
   mail_plugins = " quota mail_log notify zlib"
   pop3_logout_format = bytes_sent=%o top=%t/%p, retr=%r/%b, del=%d/%m,
\ size=%s uidl_hash=%u session=<%{session}>
}
-- snip --

Regards
Urban


Am 12.09.2015 um 20:53 schrieb Rajesh M:
> hi
>
> centos 6 64 bit
>
> hex core processor with hyperthreading ie display shows 12 cores
> 16 gb ram
> 600 gb 15000 rpm drive
>
> we are having around 4000 users on a server
>
>
> i wish to allow 1500 pop3 and 1500 imap connections simultaneously.
>
> need help regarding the settings to handle the above
>
> imap-login, pop3-login
> imap pop3 service settings
>
> i recently i got an error
> imap-login: Error: read(imap) failed: Remote closed connection (process_limit 
> reached?)
>
>
> my current dovecot config file
>
> # 2.2.7: /etc/dovecot/dovecot.conf
> # OS: Linux 2.6.32-431.23.3.el6.x86_64 x86_64 CentOS release 6.5 (Final)
> auth_cache_negative_ttl = 0
> auth_cache_ttl = 0
> auth_mechanisms = plain login digest-md5 cram-md5
> default_login_user = vpopmail
> disable_plaintext_auth = no
> first_valid_gid = 89
> first_valid_uid = 89
> log_path = /var/log/dovecot.log
> login_greeting = ready.
> mail_max_userip_connections = 50
> mail_plugins = " quota"
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date ihave
> namespace {
>inbox = yes
>location =
>prefix =
>separator = .
>type = private
> }
> passdb {
>args = cache_key=%u webmail=127.0.0.1
>driver = vpopmail
> }
> plugin {
>quota = maildir:ignore=Trash
>quota_rule = ?:storage=0
> }
> protocols = imap pop3
> service imap-login {
>client_limit = 256
>process_limit = 400
>process_min_avail = 4
>service_count = 0
>vsz_limit = 512 M
> }
> service pop3-login {
>client_limit = 1000
>process_limit = 400
>process_min_avail = 12
>service_count = 0
>vsz_limit = 512 M
> }
> ssl_cert =  ssl_dh_parameters_length = 2048
> ssl_key =  userdb {
>args = cache_key=%u quota_template=quota_rule=*:backend=%q
>driver = vpopmail
> }
> protocol imap {
>imap_client_workarounds = delay-newmail
>mail_plugins = " quota imap_quota"
> }
> protocol pop3 {
>pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
>pop3_fast_size_lookups = yes
>pop3_lock_session = no
>pop3_no_flag_updates = yes
> }
>
>
> thanks very much,
>
> rajesh
>

concerning dovecot settings for high volume server

[Rajesh M]

hi

centos 6 64 bit

hex core processor with hyperthreading ie display shows 12 cores
16 gb ram
600 gb 15000 rpm drive

we are having around 4000 users on a server


i wish to allow 1500 pop3 and 1500 imap connections simultaneously.

need help regarding the settings to handle the above

imap-login, pop3-login
imap pop3 service settings

i recently i got an error
imap-login: Error: read(imap) failed: Remote closed connection (process_limit 
reached?)


my current dovecot config file

# 2.2.7: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.23.3.el6.x86_64 x86_64 CentOS release 6.5 (Final)
auth_cache_negative_ttl = 0
auth_cache_ttl = 0
auth_mechanisms = plain login digest-md5 cram-md5
default_login_user = vpopmail
disable_plaintext_auth = no
first_valid_gid = 89
first_valid_uid = 89
log_path = /var/log/dovecot.log
login_greeting = ready.
mail_max_userip_connections = 50
mail_plugins = " quota"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
namespace {
  inbox = yes
  location =
  prefix =
  separator = .
  type = private
}
passdb {
  args = cache_key=%u webmail=127.0.0.1
  driver = vpopmail
}
plugin {
  quota = maildir:ignore=Trash
  quota_rule = ?:storage=0
}
protocols = imap pop3
service imap-login {
  client_limit = 256
  process_limit = 400
  process_min_avail = 4
  service_count = 0
  vsz_limit = 512 M
}
service pop3-login {
  client_limit = 1000
  process_limit = 400
  process_min_avail = 12
  service_count = 0
  vsz_limit = 512 M
}
ssl_cert = 

Re: concerning dovecot settings for high volume server

[Rajesh M]

- Original Message -
From: Tony Morehen [mailto:tmore...@ajmconsulting.ca]
To: dovecot@dovecot.org
Sent: Sat, 12 Sep 2015 17:57:27 -0400
Subject: Re: concerning dovecot settings for high volume server

You may be running up against Linux system/user limits.  Run
$ cat /proc/sys/kernel/pid_max
and
$ ulimit -a
That should give some insight into your problem.

On 12/09/2015 2:53 PM, Rajesh M wrote:
> hi
>
> centos 6 64 bit
>
> hex core processor with hyperthreading ie display shows 12 cores
> 16 gb ram
> 600 gb 15000 rpm drive
>
> we are having around 4000 users on a server
>
>
> i wish to allow 1500 pop3 and 1500 imap connections simultaneously.
>
> need help regarding the settings to handle the above
>
> imap-login, pop3-login
> imap pop3 service settings
>
> i recently i got an error
> imap-login: Error: read(imap) failed: Remote closed connection (process_limit 
> reached?)
>
>
> my current dovecot config file
>
> # 2.2.7: /etc/dovecot/dovecot.conf
> # OS: Linux 2.6.32-431.23.3.el6.x86_64 x86_64 CentOS release 6.5 (Final)
> auth_cache_negative_ttl = 0
> auth_cache_ttl = 0
> auth_mechanisms = plain login digest-md5 cram-md5
> default_login_user = vpopmail
> disable_plaintext_auth = no
> first_valid_gid = 89
> first_valid_uid = 89
> log_path = /var/log/dovecot.log
> login_greeting = ready.
> mail_max_userip_connections = 50
> mail_plugins = " quota"
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date ihave
> namespace {
>inbox = yes
>location =
>prefix =
>separator = .
>type = private
> }
> passdb {
>args = cache_key=%u webmail=127.0.0.1
>driver = vpopmail
> }
> plugin {
>quota = maildir:ignore=Trash
>quota_rule = ?:storage=0
> }
> protocols = imap pop3
> service imap-login {
>client_limit = 256
>process_limit = 400
>process_min_avail = 4
>service_count = 0
>vsz_limit = 512 M
> }
> service pop3-login {
>client_limit = 1000
>process_limit = 400
>process_min_avail = 12
>service_count = 0
>vsz_limit = 512 M
> }
> ssl_cert =  ssl_dh_parameters_length = 2048
> ssl_key =  userdb {
>args = cache_key=%u quota_template=quota_rule=*:backend=%q
>driver = vpopmail
> }
> protocol imap {
>imap_client_workarounds = delay-newmail
>mail_plugins = " quota imap_quota"
> }
> protocol pop3 {
>pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
>pop3_fast_size_lookups = yes
>pop3_lock_session = no
>pop3_no_flag_updates = yes
> }
>
>
> thanks very much,
>
> rajesh



this is the first time i got this message since past over an year.
the error went away as soon as i restarted dovecot..
surprisingly this happened in the night ie off-office hours.

here are the values i got

also could you provide me the settings for the limits if wish to handle around 
1500 simultaneous connections of pop3 and imap each ?

the current values are as follows

[root@ns1 log]# cat /proc/sys/kernel/pid_max
49152

[root@ns1 log]# ulimit -a
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 127047
max locked memory   (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files  (-n) 1024
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 10240
cpu time   (seconds, -t) unlimited
max user processes  (-u) 127047
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited


thanks
rajesh

outlook downloading duplicate emails on particular days

[Rajesh M]

hi

centos 6 with qmail toaster and dovecot 2.2.7 for pop3 and imap

i have a strange issue which happens on all 4 of my servers

for POP3 users -- on any random day ie once in a while users spread across 
multiple domains will face a problem of outlook re-downloading ALL the emails 
in the inbox -- not just a few duplicate emails but the entire inbox.

This happens for multiple email users spread across multiple domains sitting in 
different locations.

i am searching for a solution for the last several months but nothing yet

require help please

my dovecot config file


# Settings for 'stock' qmail-toaster

disable_plaintext_auth = no
default_login_user = vpopmail
mail_max_userip_connections = 50

service imap-login {
  service_count = 0
  client_limit = 256
  process_min_avail = 4
  vsz_limit = 384M
  process_limit = 400
}

service pop3-login {
  service_count = 0
  client_limit = 1000
  process_min_avail = 12
  vsz_limit = 512M
  process_limit = 400

}

auth_cache_size = 0
auth_cache_ttl = 0
auth_cache_negative_ttl = 0
auth_mechanisms = plain login digest-md5 cram-md5
auth_username_format = %Lu

first_valid_uid = 89
first_valid_gid = 89
log_path = /var/log/dovecot.log

login_greeting = ready.

mail_plugins = $mail_plugins quota
plugin/quota = maildir
protocols = imap pop3
ssl_cert = /var/qmail/control/servercert.pem
ssl_key = /var/qmail/control/servercert.pem

# the following will likely be the default at some point
ssl_dh_parameters_length = 2048


namespace {
  inbox = yes
  location =
  prefix =
# prefix = INBOX.
# Use 'prefix = INBOX.' for Courier-IMAP compatible transparent migration.
# This forces all client apps to create subfolders underneath/within
# the inbox instead of adjacent to it. It's not necessary to be that way.
# FWIW, shubes thinks this setting is annoying/rediculous/retarded.
# As a practical example, having no prefix allows mobile users to select
# which folders get synchronized. Otherwise their whole mailbox is
# synchronized, and they have no way to choose which folders they want sync'd.
  separator = .
  type = private
}

passdb {
  args = cache_key=%u webmail=127.0.0.1
  driver = vpopmail
}

protocol imap {
  imap_client_workarounds = delay-newmail
  mail_plugins = $mail_plugins imap_quota
}

protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_fast_size_lookups = yes
  pop3_lock_session = no
# pop3_uidl_format = UID%u-%v
pop3_no_flag_updates=yes
}

userdb {
  args = cache_key=%u quota_template=quota_rule=*:backend=%q
  driver = vpopmail
}

plugin {
  quota = maildir:ignore=Trash
  quota_rule = ?:storage=0
}


thanks
raj

Re: failed login message

[Rajesh M]

thank you for guidance,

just to recap the issue was about squirrelmail giving a wrong message : 
connection dropped by imap server instead of invalid user or password

as advised i connected using command line on both my old and new servers, and 
have posted the details including the output of dovevcot -n.


1) command prompt login. i put wrong password

telnet x.x.x.x 143

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE START
TLS AUTH=PLAIN AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=CRAM-MD5] ready.

1 login a...@mydomain.com wrongpass

1 NO [AUTHENTICATIONFAILED] Authentication failed.


2) DOVECOT LOG output

CURRENT SERVER
Nov 28 08:48:39 imap-login: Info: Disconnected (auth failed, 1 attempts in 72 
secs): user=a...@mydomain.com, method=PLAIN, rip=120.62.202.70, 
lip=103.241.181.155, session=4PxKs+II2QB4PspG

OLD SERVER
Nov 28 09:06:08 imap-login: Info: Disconnected (auth failed, 1 attempts): 
user=a...@mydomain.com, method=PLAIN, rip=120.62.202.70, lip=72.35.76.156



3) DOVECOT CONFIG FILE DETAILS

# dovecot -n
# 2.2.7: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final)
auth_cache_negative_ttl = 0
auth_cache_size = 32 M
auth_cache_ttl = 30 mins
auth_mechanisms = plain login digest-md5 cram-md5
default_login_user = vpopmail
disable_plaintext_auth = no
first_valid_gid = 89
first_valid_uid = 89
log_path = /var/log/dovecot.log
login_greeting = ready.
mail_max_userip_connections = 50
mail_plugins =  quota
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
namespace {
  inbox = yes
  location =
  prefix =
  separator = .
  type = private
}
passdb {
  args = cache_key=%u webmail=127.0.0.1
  driver = vpopmail
}
plugin {
  quota = maildir:ignore=Trash
  quota_rule = ?:storage=0
}
protocols = imap pop3
service imap-login {
  client_limit = 256
  process_limit = 400
  process_min_avail = 12
  service_count = 0
  vsz_limit = 384 M
}
service pop3-login {
  client_limit = 1000
  process_limit = 400
  process_min_avail = 12
  service_count = 0
  vsz_limit = 512 M
}
ssl_cert = /var/qmail/control/servercert.pem
ssl_dh_parameters_length = 2048
ssl_key = /var/qmail/control/servercert.pem
userdb {
  args = cache_key=%u quota_template=quota_rule=*:backend=%q
  driver = vpopmail
}
protocol imap {
  imap_client_workarounds = delay-newmail
  mail_plugins =  quota imap_quota
}
protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_fast_size_lookups = yes
  pop3_lock_session = no
  pop3_no_flag_updates = yes
}

thank you for your time and patience.

rajesh


- Original Message -
From: Gedalya [mailto:geda...@gedalya.net]
To: 24x7ser...@24x7server.net,dovecot@dovecot.org
Sent: Tue, 25 Nov 2014 05:54:49 -0500
Subject: Re: failed login message

On 11/25/2014 04:49 AM, 24x7ser...@24x7server.net wrote:
 thanks for your reply

 i intentionaly put the wrong password and checked the dovecot log and the 
 message i got was

 # tail -f /var/log/dovecot.log | grep x...@yyy.com
 Nov 25 08:47:46 imap-login: Info: Aborted login (auth failed, 1 attempts in 2 
 secs): user=x...@yyy.com, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1,
 secured, session=xObTWqYIwgB/AAAB

 but in the squirrelmail login page instead of the message unknown user or 
 password incorrect i get the message

 ERROR: Connection dropped by IMAP server.
 Query: LOGOUT


 in my old server where it is working correctly dovecot logs shows the error

 Nov 25 14:46:12 imap-login: Info: Aborted login (auth failed, 1 attempts): 
 user=x...@yyy.com, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured

 and squirrelmail gives the error : unknown user or password incorrect which 
 is correct.

 is there any other place i need to check or some other config file (either 
 squirrelmail or dovecot) that i need to modify

 could you kindly guide me


1. Please carefully read http://dovecot.org/mailinglists.html
2. Please carefully follow the instructions at
http://dovecot.org/mailinglists.html. In your case, post full details
for both servers.
3. Try to manually log in to both servers using e.g. telnet in order to
see any differences in the way they respond.
Something like

$ telnet 192.168.9.11 143
Trying 192.168.9.11...
Connected to 192.168.9.11.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE STARTTLS LOGINDISABLED] Dovecot ready.
1 login gedalya 1234
* BAD [ALERT] Plaintext authentication not allowed without SSL/TLS, but
your client did it anyway. If anyone was listening, the password was
exposed.
1 NO [PRIVACYREQUIRED] Plaintext authentication disallowed on non-secure
(SSL/TLS) connections.
2 logout
* BYE Logging out
2 OK Logout completed.
Connection closed by foreign host.

(that's how it responds in my specific case as I disallow 

failed login message

[Rajesh M.]

hi

i am using dovecot for pop3 and imap along with qmail toaster

i had recently changed over from a 32 bit system to a 64 bit system.

i use squirrelmail.

the issue i face is that when the user inserts an invalid password then the 
message he gets is connection dropped by imap server  (instead of invalid 
userid or password)

such a message confuses the user.

my earlier 32 bit version used to give the correct message

i am posting this issue here because i am not sure if this is squirrelmail 
issue or a dovecot issue.

can somebody please let me know the reason for this ?

rajesh

Re: authentication issues with ipad / mac email client

[Rajesh M.]

hi

i am using dovecot version2.2.7 provided by qmailtoaster - 64 bit on centos 6.5 
64 bit system - 16 gb ram with a hex core processor.

i was using qmail pop3 on a 32 bit system for the past over 7 years and had 
absolutely no issues.

with dovecot pop3 i have started receiving a variety of problems

mac machines and ipads -- port 110 does not work at all. i have to use port 995 
only.

i have allowed plain text auth.

thunderbird works perfectly -- both imap and pop3 with port 110 and port 995

outlook 2010 also works but incase of one customer the mail download is very 
slow. i have a windows servers with mailenable mail server and the same 
customer could download perfectly from those windows servers.

i am using the default service pop3 settings

Do i need to increase the memory settings /  for pop3 login similar to imap 
login which i am missing out like what is configured for imap. (pl see config 
file below)

My mail server is busy with around 4000 mailboxes and transacting over 200k 
emails per day. atleast around 250 simultaneous pop3 connections during peak 
hours.



# dovecot -a | grep pop3_uidl_format
pop3_uidl_format = %08Xu%08Xv


[root@ns1 spamassassin]# dovecot -n
# 2.2.7: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final)
auth_cache_negative_ttl = 0
auth_cache_size = 32 M
auth_cache_ttl = 30 mins
auth_mechanisms = plain login digest-md5 cram-md5
default_login_user = vpopmail
disable_plaintext_auth = no
first_valid_gid = 89
first_valid_uid = 89
log_path = /var/log/dovecot.log
login_greeting = ready.
mail_max_userip_connections = 50
mail_plugins =  quota
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
namespace {
  inbox = yes
  location =
  prefix =
  separator = .
  type = private
}
passdb {
  args = cache_key=%u webmail=127.0.0.1
  driver = vpopmail
}
plugin {
  quota = maildir:ignore=Trash
  quota_rule = ?:storage=0
}
protocols = imap pop3

service imap-login {
  client_limit = 256
  process_limit = 400
  process_min_avail = 4
  service_count = 0
  vsz_limit = 384 M
}
ssl_cert = /var/qmail/control/servercert.pem
ssl_dh_parameters_length = 2048
ssl_key = /var/qmail/control/servercert.pem
userdb {
  args = cache_key=%u quota_template=quota_rule=*:backend=%q
  driver = vpopmail
}
protocol imap {
  imap_client_workarounds = delay-newmail
  mail_plugins =  quota imap_quota
}
protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_fast_size_lookups = yes
  pop3_lock_session = no
  pop3_no_flag_updates = yes
}


i am really puzzled because dovecot was highly recommended by all.

thanks for all your help


rajesh


- Original Message -
From: Daniel Parthey [mailto:p...@posteo.de]
To: 24x7ser...@24x7server.net,dovecot@dovecot.org
Sent: Sat, 01 Nov 2014 09:34:53 +0100
Subject: Re: authentication issues with ipad / mac email client

Hi,

please post
* output of doveconf -n
* error log messages from dovecot for failed vs. succeeded login

Regards
Daniel

authentication issues with ipad / mac email client

[Rajesh M.]

hi

we are using using qmail toaster with dovecot pop3 and imap client

outlook and thunderbird works fine

in case of several  ipad / mac email clients
imap works fine but pop3 logins are not working with 110 port

my dovecot config file is given below

could somebody help please



# Settings for 'stock' qmail-toaster

disable_plaintext_auth = no
default_login_user = vpopmail
mail_max_userip_connections = 50

service imap-login {
  service_count = 0
  client_limit = 256
  process_min_avail = 4
  vsz_limit = 384M
  process_limit = 400
}


auth_cache_size = 32M
auth_cache_ttl = 1800
auth_cache_negative_ttl = 0
auth_mechanisms = plain login digest-md5 cram-md5
auth_username_format = %Lu

first_valid_uid = 89
first_valid_gid = 89
log_path = /var/log/dovecot.log

login_greeting = ready.

mail_plugins = $mail_plugins quota
plugin/quota = maildir
protocols = imap pop3
ssl_cert = /var/qmail/control/servercert.pem
ssl_key = /var/qmail/control/servercert.pem
# the following will likely be the default at some point
ssl_dh_parameters_length = 2048


namespace {
  inbox = yes
  location =
  prefix =


# prefix = INBOX.
# Use 'prefix = INBOX.' for Courier-IMAP compatible transparent migration.
# This forces all client apps to create subfolders underneath/within
# the inbox instead of adjacent to it. It's not necessary to be that way.
# FWIW, shubes thinks this setting is annoying/rediculous/retarded.
# As a practical example, having no prefix allows mobile users to select
# which folders get synchronized. Otherwise their whole mailbox is
# synchronized, and they have no way to choose which folders they want sync'd.
  separator = .
  type = private
}

passdb {
  args = cache_key=%u webmail=127.0.0.1
  driver = vpopmail
}

protocol imap {
  imap_client_workarounds = delay-newmail
  mail_plugins = $mail_plugins imap_quota
}

protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_fast_size_lookups = yes
  pop3_lock_session = no
# pop3_uidl_format = UID%u-%v
pop3_no_flag_updates=yes
}

userdb {
  args = cache_key=%u quota_template=quota_rule=*:backend=%q
  driver = vpopmail
}

plugin {
  quota = maildir:ignore=Trash
  quota_rule = ?:storage=0

}


thank you very much,
rajesh

Re: [Dovecot] Idea: POP3 deletion as a flag

[Rajesh M]

timo

thank you very much for your wonderful work on dovecot

the new feature that you are thinking about is very useful

this is an excellent solution and is used by a quite a few large isps /
mailing providers like rediff. And i believe blackberry also does
something similar. ie if configure the blackberry today it downloads new
emails only.

when pop3 access emails sometimes in some email clients the emails start
downloading right from the begining. this can be prevented by this
solution

also when there are large mailboxes more the number of emails in the inbox
more the system gets loaded up in scanning the mails and deciding when the
last download took place. i believe pop3 performance would be far better
with this feature.

a few points though

if this option is implemented then there should be a way to disable it
tempoarilry. This is incase the user is shifting over to a new pc and
wants to donwload all old emails.

also we can set the date of download ie say allow pop3 download emails
upto  15 days old/


rajesh


 GMail doesn't delete mails when POP3 client issues a DELE command for it.
 Instead they just become invisible for future POP3 sessions, but they
 still exist for IMAP/webmail. The same could be implemented pretty easily
 for Dovecot:

  - Add a new setting to enable this: pop3_deleted_flag = $POP3Deleted
  - When DELEting a message, add this flag to the message.
  - When listing messages, skip all messages that have this flag.
  - Also hide this flag from IMAP clients(?)

 Thoughts? Probably 20 lines of new code.

Re: [Dovecot] migration to dovecot pop3

[Rajesh M]

 On 12/19/2012 6:59 PM, Joseph Tam wrote:
 Rajesh M writes:

 i am planning to migrate from qmail pop3 to dovecot pop3

 the number of users on the server is large over 5000
 ...
 also data volume is large - over 3.5 tb -- ie many users several 1000s
 of
 emails in their inbox

 If you stay on the list, you might find some people here whose
 mail metrics are several orders of magnitude larger than this.

 1) will the migration cause all the emails on the server to re-download
 ...
 4) has anybody migrated from qmail pop3 to dovecot pop3 successfully
 without emails being downloaded again

 A proper migration will avoid this.  Google is your friend

 http://lmgtfy.com/q=POP+qmail+to+dovecot+migration

 The top hits provide exactly what you're looking for.

 2) can dovecot be configured to allow only one download at a time -
 pop3
 locking with time out setting that can be customized

 We migrated 5000+ from qpopper years ago. Dovecot is sooo much better on
 system load. Yes, locking and timeouts are in the config file.


 3) can dovecot pop3 be configured so that it will allow download of
 emails
 only of the last say 15 days ie even if the same pop3 user is
 configure on
 another machine it will download only messages of the last 15 days

 That's not part of pop3, so no, it's not possible with dovecot alone.
 You could probably work out a plugin to move read mail after x days, or
 there may be one out there ? Or some imap robot could do it..
 Ken A.
 Pacific.Net


 I can't answer this, but if this is an attempt to prevent mass download,
 the answer to 1+4 makes these questions irrelevant.

 Joseph Tam jtam.h...@gmail.com


 --
 Ken Anderson
 Pacific Internet - http://www.pacific.net
 Latest Pacific.Net Status - http://twitter.com/pacnetstatus


thanks joseph and ken for your replies

ken, if possible can you please give me some more information concerning
the servers that you use.

i am using qmailtoaster my server has over 3 tb of data.

Q1) which email server software are you using for your 5000+ users

Q2) what is your server hardware configuration for handling 5000+ users

Q3) do you users store several 1000's of emails in their inboxes.

Q4) i want to try out by running dovecot pop3 on a separate port other
than 110 while qmail toaster pop3 continues to run on port 110 and try
downloading from specific a test mailbox. do you forsee any problems in
this ?

thank you once again for your kind help.

rajesh

[Dovecot] migration to dovecot pop3

[Rajesh M]

hi

i am planning to migrate from qmail pop3 to dovecot pop3

the number of users on the server is large over 5000

also data volume is large - over 3.5 tb -- ie many users several 1000s of
emails in their inbox

email clients used are outlook / outlook express / windows mail, etc

my questions

1) will the migration cause all the emails on the server to re-download

2) can dovecot be configured to allow only one download at a time - pop3
locking with time out setting that can be customized

3) can dovecot pop3 be configured so that it will allow download of emails
only of the last say 15 days ie even if the same pop3 user is configure on
another machine it will download only messages of the last 15 days

4) has anybody migrated from qmail pop3 to dovecot pop3 successfully
without emails being downloaded again

thankyou very much in advance

rajesh

[Dovecot] issues migration from dovecot 1.2 to version 2

[Rajesh M]

hi

my system is a centos 5 with qmailtoaster

i migrated my email server with around 5000 users from dovecot version 1.2
to version 2

i have two separate 2 tb hdd's storing webmail data of these users.

the load on the server goes very high over 100 during peak load times and
the imap connections get dropped frequently, webmail becomes very slow.

however pop3 download works ok

in the dovecot log file i get errors as such

Warning: Maildir /homebackup/domains///Maildir/.ALL_INBOX MAIL:
Synchronization took 71 seconds (20 new msgs, 0 flag change attempts, 0
expunge attempts)

i had configured dovecot 1.2 using source
but i had installed version 2 using qmailtoaster rpm

i am a bit confused as to what settings are to be done for a very busy server

note that the cpu system usage and ram usage are less but IO wait goes to
around 80-90 percent

can you please guide me or post some dovecot version 2 config file
settings that are relevant to a busy server.

rajesh

[Dovecot] nfs error fcntl(read-lock) locking failed for file

[Rajesh M]

hi

i am using qmailtoaster with dovecot version 2

mailbox format is maildir

i have a domain with around 5000 users which are distributed over 2 servers

webmail (squirrelmail) runs using dovecot v2 is being used from server
number one

server number 2 had all the data stored in it and also has pop and smtp
running from it. i am not using dovecot for pop as yet

on the server with dovecot i get such errors in the log file

access to data on server number 2 is via nfs

on server number 1 i get errors as such
Error: fcntl(read-lock) locking failed for file Input/output error
squirrelmail gives error imap connection closed and i am not able to login


so i set the parameters as such in the dovecot conf file and the error
stopped

mmap_disable=yes
dotlock_use_excl = yes
lock_method = dotlock

can somebody please advise me if the above is correct ?

or is it preferred to use fcntl with lockd (note that my mailbox is
maildir format)

thanks very much for your help

rajesh

[Dovecot] dovecot logging

[Rajesh M]

hi

i am using dovecot version 1.2.11 with qmail toaster

is there a way to log the number of current imap connections at a given
time using dovecot log format ?

thanks
rajesh

[Dovecot] pop passd for changing passwords in squirrelmail

[Rajesh M]

hi

when i migrated from courier to dovecot my password change plugin in
squirrelmail stopped working since courierpassd was being used in poppassd

is there a way to get this working ie some alternative solution

i don't want to use qmailadmin


thanks
rajesh

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Rajesh M]

stan

you are right ...

i am running qmail toaster for years now and this is production mail
server with over 5000 email boxes .. i dont want to migrate at all.

i recently upgraded to dovecot only for the imap part because courier was
trashing my server .. i was getting load levels of over 30-40 during peak
times

once courier was implemented my load levels were down to around 2-3

i am using squirrel mail and it has server side sort enabled

most users of mine have at the most around 100 to 200 mb of emails and
just a few users having 3-4 gb boxes

i am absolutely happy with the performance of dovecot

a 3.8 gb mail box opens in 8-10 seconds the first time and next time in
around 3 seconds

this 30 gb email box test is just to see how best dovecot can perform

my system is on centos with ext3 and is using 60 percent of drive space
so i believe it shud not be fragmented

i also feel that this is related to index and cache which is getting
expired or something like that.

i will try out what you suggested and revert

rajesh


 Ed W put forth on 6/23/2010 4:18 PM:

 Secondly 7,500 mails over 5 mins means an indexing rate of 25
 mails/sec.  This would not be out of order for a heavily fragmented
 drive which is IO bound?  Each file needs to be opened to scan the
 headers so likely you need one disk seek and I guess it's easy to be IO
 bound?  What does iotop show you during dovecot's thrashing?

 It seems he is I/O bound a degree.  If I read his answer to my disk
 subsystem
 question correctly he's storing user maildirs on a single local 1TB SATA
 drive.  However, given that his 2nd successive login is 4-5 seconds
 instead of
 5 minutes, it would seem index and cache being current are the problem,
 not
 I/O saturation.  Faster disk would always help, but it's not close to a
 total
 solution to his problem.

 Putting his maildir on a 16 disk RAID 0 stripe of the same model 1TB disk
 he
 already has would yield a 16x improvement in seek throughput, cutting his
 'stale' login time to ~20 seconds, if my math is correct.  20 seconds is
 still
 unacceptably high IMHO, though it's much better than 300 seconds.  Ok so
 lets
 assume the filesystem underlying his maildir is heavily fragmented and
 that
 defragging it would yield a 100% improvement for argument sake (50%
 improvement is almost unheard of, normal is about 20%).  He'd still be
 looking
 at a 10 second login time after spending anywhere from $3k to $8k USD on a
 16
 disk array depending on what vendor he chooses.  Throwing money and
 hardware
 at this problem isn't the proper or optimal solution.

 Dovecot2 has an mdbox option which sounds like it could be beneficial
 for your performance requirements (but it's not stable yet)

 I don't think migrating to a new mailbox format is what he really needs,
 or
 wants.  IIRC he's been using qmail for years, which means he's been using
 maildir for years.  He's likely very comfortable with maildir and probably
 wants to stick with it.

 Otherwise I guess you need to investigate dovecot's delivery agent which
 does incremental index updates at deliver time (pay the cost one email
 at a time rather than every 7,500 emails).  Or consider alternative
 filesystems which are more performant for this requirement?

 On this I completely agree with you, and I suggested it previously in the
 same
 thread I suggested dirty_syncs.  This should be his next step.  If LDA
 alone
 doesn't fix the problem, then he should try dirty_syncs again in
 conjunction
 with LDA.

 --
 Stan

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Rajesh M]

 Rajesh M wrote:
 Rajesh M wrote:
 eric

 i studied LDA a bit
  
 if i use lda that means all my 5000+ users' email index files will be
 continuously updated when every email arrives -- means a lot of writes
 to
 disk ... is that correct ?
 Yes, but I think you make it sound worse than it is. Updating the index
 as each email arrives doesn't involve rebuilding the whole index like
 what happens at login. It only updates what's there instead of
 rebuilding the whole thing. I can't speak from experience, but I expect
 that it's very efficient, and you'll never notice any increased load on
 the machine. Perhaps the equivalent of adding journaling to a
 filesystem. You can ask the dovecot list to verify this.

 Also, it just occurred to me that if you're using ext3, you should be
 sure to optimize the filesystem. Off hand, I'd use the noatime and
 nodiratime mount options (man mount) and dir_index filesystem option
 (man tune2fs).

 if yes then i dont want that since only around 200 or so people out of
 the
 above 5000 use webmail.
 Just to be clear, it affects all imap usage, not only webmail (although
 your uses may not use imap other than webmail).

 i will study a some more and revert
 I hope you can figure a way to try this out. I expect that QMT's move
 to
 dovecot in the future will include the deliver LDA, and your work here
 will be a big help getting us there.

 Let me know if I can be of any further help.

 --
 -Eric 'shubes'



 hi eric

 few questions

 i wil enable lda exclusively for this one 30 gb email box using .qmail
 file see the performance improvement .. is that ok ?

 I believe that would be fine.

 can i safely enable atime for this user maildir  ?

 Sure. That's simply a performance consideration. I don't believe that
 atime is used for much of anything, so either atime or noatime is safe.
 noatime is just a little faster because the inode (where that piece of
 information is stored) doesn't need to be updated each time a file is
 accessed.

 thanks for your help

 Sure. I'm eager to see your results.

 --
 -Eric 'shubes'



hi eric

this is what i plan to do

as per
http://wiki.dovecot.org/LDA/Qmail

i need to enter the following line in a .qmail file under the Maildir
directory of the user
|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $...@$user

i would add an email forward using qmailadmin web interface (which creates
the .qmail file) and then manually edit the .qmail file and put the above
line in it (so that ownership and permissions are taken care off)

now i would also need to edit the dovecot.conf file
could you guide me on what changes do i need to dovecot.conf file ?


rajesh

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Rajesh M]

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On Wed, 23 Jun 2010, Rajesh M wrote:

 Roundcube 0.3.1 seems to have a hard coded limit of 200 mails per page.

 it again took around 4-5 mins

 I do not know nothing about Roundcube, therefore I wonder where the 5min
 delay come from:

 a) Do you keep index files on Dovecot side?

 b) Do you deliver / manipulate the mailbox via Dovecot only? E.g. do you
 use Dovecot deliver?

 c) If you rawlog or sniff the connection and issue the same commands as
 roundcube to Dovecot via, say, telnet next morning, is the delay the same?
 http://wiki.dovecot.org/Debugging/Rawlog
 http://wiki.dovecot.org/TestInstallation?highlight=%28telnet%29

 a) and b) should avoid rereading of the mailbox.
 c) is to ensure that Roundcube does not add the delay itself.

 the extra 1000 emails were to be added to the cache .. does dovecot go
 thru all the 17 emails again to form the meta data ? or is there
 some
 cache expiry period ?

 Usually no.

 Regards,

 - --
 Steffen Kaiser
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.6 (GNU/Linux)

 iQEVAwUBTCG2V7+Vh58GPL/cAQJyHAf+MayOiuayYay2mCM5i0DJH296pbuDMAXc
 kbXKMRXFOP/KGeAYmHirL/9/63h0IjeRtSuBe0HJxmbg87aZVWOejVaXSPOPTAUR
 n8d++siuBnQ8xiDFw0LKgep2HetYFdxepDRjdPTOM/Drpv+THFRGOVnem6JYb7yw
 0XqkQG7M4pKBaxZA+0HFH4c4ADz3XykkOluTHhmU0zDXcdkK0TybfEiwGRY3R7i8
 IozQ62pKv/v2t2z2clYOXsqLe6pM14B8/2SXUX4Anrkz8AoKyocT1Ks9hOCkizEn
 bG99r7U+WObrMmbTpy6jsq//amh496ot37S4DO7DBKa3cKYqOJwwXg==
 =gqnn
 -END PGP SIGNATURE-



hi

did the tests once again on this large email box

i set
maildir_very_dirty_syncs = yes
this was done in the morning and dovecot was restarted

i logged after about 4 hours after the previous login
again it took around 5 mins to login

i was monitoring my server load which around 1.5 - 2 on my dual core dual
xeon machine

this increased to around 8-9 during the login process

the no of emails in the box had increase by around 7500 emails

setting
maildir_very_dirty_syncs  = yes

does not seem to help

i have pasted below my dovecot.conf file



base_dir = /var/run/dovecot/
protocols = imap imaps
log_path = /backup1/qmaillog/dovecot.log
#ssl_disable = no
ssl_cert_file = /var/qmail/control/servercert.pem
ssl_key_file = /var/qmail/control/servercert.pem
ssl_cipher_list = djdjjd
verbose_ssl = yes

protocol imap {
listen = *:143
ssl_listen = *:993
}
## Login processes
#login_dir = /usr/local/var/run/dovecot/login
login_user = dovecot
login_process_per_connection = no
login_processes_count = 3
login_process_size = 128
login_max_processes_count = 512
login_greeting = Ready
#login_log_format_elements = user=%u method=%m rip=%r lip=%l %c

## Mailbox locations and namespaces
mail_location = maildir:~/Maildir
namespace private {

separator = .
prefix = INBOX.
inbox = yes
}
# Mail processes
verbose_proctitle = yes
first_valid_uid = 89
last_valid_uid = 89

# Maximum number of running mail processes. When this limit is reached,
max_mail_processes = 200

# Set max. process size in megabytes. Most of the memory goes to mmap()ing
# files, so it shouldn't harm much even if this limit is set pretty high.
mail_process_size = 256

## Maildir-specific settings
maildir_very_dirty_syncs = yes

## Authentication processes
disable_plaintext_auth = yes

auth default {
mechanisms = plain login digest-md5 cram-md5
passdb vpopmail {
args = webmail=127.0.0.1
}
userdb vpopmail {
}
user = vpopmail
count = 1
ssl_require_client_cert = no
}


thanks
rajesh

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Rajesh M]

 Timo Sirainen put forth on 6/19/2010 9:43 PM:
 On 20.6.2010, at 3.33, Rajesh M wrote:

 3) next i changed squirrelmail display preference to list 500 mails per
 page
 and then clicked on inbox ... expecting dovecot to take atleast around
 1-2
 mins to show the mails

 ... the mails were displayed in just 8 seconds .. awesome performance

 Fetching metadata for 500 messages took 8 seconds? Still sounds slow.
 Although maybe it's because of maildir. I'm anyway not happy until
 Dovecot can open a mailbox with millions of messages and access them in
 nearly constant time (few milliseconds, with SSD).

 Roundcube 0.3.1 seems to have a hard coded limit of 200 mails per page.
 On a
 dual CPU 500 MHz Celeron with 384MB and a single 500GB SATA drive, running
 both RC and Dovecot 1.2.11 on the box, it takes about 5 seconds to pull
 metadata for each set of 200 messages on a 13k+ message mailbox.  This is
 using mbox.

 I agree with Timo, 8 seconds for 500 messages on that hardware seems a bit
 slow given what my lowly setup can do.  Was the box under load when you
 tested
 or idle?  What disk subsystem is the maildir stored on?  Single disk or
 high
 performance RAID?  Was the disk subsystem under load or idle when you
 tested?

 --
 Stan



stan

my machine is dual core dual processor xeon 1.6 gig proc
4 gb ram
1 tb sata for centos linux
1 tb sata for vpopmail domains data, mysql db and log data
my hdd usage is at 60 percent now.

i accessed the large email box (this is qmail tap account in which emails
are continuously added) yesterday 10 pm at night
login took around 5 mins
Around 169000 emails in the box

I logged out and logged in again after around 5 mins -- access took just
around 4-5 seconds. Around 10 emails had been added.

at 10 am in the morning today i logged in again ie around 12 hours after
above login.

it again took around 4-5 mins

i have currently over 17 emails in the mailbox ie overnight around
1000 emails were added
i logged off and access again -- the mailbox opened in around 5 seconds

so what i can conclude is that if relogin after a few minutes then the
access is very fast whereas if i do the same after a few hours then it
takes time.

i am new to dovecot and a few questions

1) is it ok that dovecot takes 5 mins every time i login if there is
significant time difference between the two logins

2) as per what timo said the metadata is cached so the data pertaining to
the extra 1000 emails were to be added to the cache .. does dovecot go
thru all the 17 emails again to form the meta data ? or is there some
cache expiry period ?

thanks
rajesh

[Dovecot] dovecot evaluation on a 30 gb mailbox

[Rajesh M]

hi all

i use qmail toaster. i recently changed over to dovecot.

posting below my evaluation on the same

my machine
dual core dual xeon 5140 processor, with 4 gb ram, centos linux with
qmailtoaster

i had setup a qmail tap account which backups up data for a client of mine

mailbox size over 30 gb

setup dovecot with squirrelmail and logged in

it had over one hundred and thirty nine thousand emails

139019 emails in the 30 gb mailbox

Reports

1) first login : took 7 minutes to parse 139019 mails but never timed out.
this was simply impossible with courier

2) the next login it took 5 seconds by which time the mail numbers
increased to 139045

3) next i changed squirrelmail display preference to list 500 mails per page
and then clicked on inbox ... expecting dovecot to take atleast around 1-2
mins to show the mails

... the mails were displayed in just 8 seconds .. awesome performance

qmailtoaster is simply waiting for a dovecot integration

i am evaluating the stability of dovecot now

in the next 2-3 weeks i post stability report

rajesh