Re: [Dovecot] kernel problem in RedHat? -- RH specific, or what linux kernels does this affect?

[Linda Walsh]

Is this redhat's version of the kernel only?  Or does it apply to other 
linux kernels and other distros?


Any idea what linux kernel versions might cause this?

(from main dovecot webpage news)

 Thu Mar 22 14:38:53 EET 2012

Red Hat/CentOS users: A recent kernel update 
 causes Dovecot to 
start failing after it has reached 1000 child processes. To fix this, 
downgrade your kernel until Red Hat releases a fixed kernel.

Re: [Dovecot] dsync is SLOW compared to rsync

[Linda Walsh]

Jeff Gustafson wrote:

On Fri, 2012-03-23 at 19:02 +0100, Christoph Bußenius wrote:
  

Hi,

maybe try "dsync -o mail_fsync=never".



That didn't seem to make much of a difference. On a 3.1GB backup it
shaved off 5 seconds. dsync's time was over 6 minutes with or without
the mail_fsync=never. rsync copied the same 3.1GB mailbox in 15 seconds.
It seems to me that dsync *should* be able to be just as fast, but it
currently is spending way too much time doing something. What is it?
...Jeff
  

---

Next -- bench "cp -ax", against rsync -axHAX when it has to copy >75% of 
the data (cp ~6-8x speed).  But for file speed, 'dd' is king, as it can 
use large buffers (~16MB gives best results on my local Gbit network), 
but it
misses all those pesky acls and extended attrs, not to mention file 
perms...*sigh*   Compare that to the I/O done 4k at a time by many older 
utils...


If I'm writing to the LOCAL HD, instead of the network, then a 1GB-4GB 
buffer size gives best results (1GB/s raid5).  Small buffers are such a 
PITA!

Re: [Dovecot] sysconfdir depreacted

[Timo Sirainen]

On 23.3.2012, at 19.22, Andraž 'ruskie' Levstik wrote:

> :2012-03-23T12:53:Timo Sirainen:
> 
>> Yes, I was also thinking about that, but it's about removing the dovecot/ 
>> suffix from other directories as well. That might be something worth doing 
>> (--without-package-suffix or something?).
> 
> I would suggest to have a --layout=gnu|opt
> 
> That would either do what it currently does(gnu) and opt to install
> everything into a single dir i.e.:
> /opt/dovecot/
> 
> With subdirs under there.

Yes, --with-layout=gnu|opt could be useful. Anyone want to volunteer to 
implement it? :)

Re: [Dovecot] sysconfdir depreacted

[Timo Sirainen]

On 24.3.2012, at 3.19, Noel Butler wrote:

>> Yes, I was also thinking about that, but it's about removing the dovecot/ 
>> suffix from other directories as well. That might be something worth doing 
>> (--without-package-suffix or something?).
> it is very easy to have a search path for config file,  it shouldn't
> take much effort at all to change that to look for the long time default
> of /etc/dovecot.conf  first, then if not there, look in /etc/dovecot/

Technically it's easy, but the result will be that more people will be 
confused. I'll get an increase of emails about "I changed dovecot.conf, but 
nothing happens?!?" My goal is to reduce the number of emails I get, not 
increase them.

> No-one is suggesting putting all the individual conf files in /etc, only
> for existence of dovecot.conf itself. 

So you don't want to remove dovecot/ suffix from all the other dirs (lib, 
libexec, etc.) only from etc? The only way I can think of how to do that is to 
add a special option just for it, and more options is generally bad:

> Which brings up another question,  may I ask why some of the options to
> disable some passwd types were removed from build process? Systems that
> dont use system password files (amongst other formats) dont need to
> build them, that's not a criticism, 'just sayin'.

There's also no harm in having that code included. They add no extra library 
dependencies. The only thing they do is to use a few kilobytes of more disk 
space, and possibly a few kilobytes of more memory (even that isn't certain).

All options just increase the number of combinations that can cause things to 
go wrong. If I add some code to be compiled optionally, it just adds more 
combinations that should be tested together to see if the code still even 
compiles. Previously I've broken SSL code many times by not testing if after 
changes Dovecot builds without OpenSSL.

So the less options there are, the more robust Dovecot is, and the less work I 
have to do to keep it working when adding new features. So I add an option only 
when there is a good use case for it and I expect more than one person to use 
it.

Re: [Dovecot] sysconfdir depreacted

[Noel Butler]

On Fri, 2012-03-23 at 12:53 +0200, Timo Sirainen wrote:

> On 23.3.2012, at 12.44, Heiko Schlichting wrote:
> 
> > Timo wrote:
> >> So the only way I can think of how to change this is to add another
> >> option to optionally remove the dovecot/ suffix from the directory, but
> >> is this really worth the trouble?
> > 
> > I would appreciate such option too. For large dedicated installations other
> > schemes than /etc/dovecot are common.
> > 
> > See http://dovecot.org/list/dovecot/2009-January/036131.html
> 
> Yes, I was also thinking about that, but it's about removing the dovecot/ 
> suffix from other directories as well. That might be something worth doing 
> (--without-package-suffix or something?).
> 


it is very easy to have a search path for config file,  it shouldn't
take much effort at all to change that to look for the long time default
of /etc/dovecot.conf  first, then if not there, look in /etc/dovecot/

No-one is suggesting putting all the individual conf files in /etc, only
for existence of dovecot.conf itself. 

There are plenty of linux and unix systems that have been using /etc for
as long as I can recall (even early redhat did), its only certain
distros that build as /etc/foo/  the ones that use rpms or debs are
obviously not running anything special (we all know no build config
process will suite all operations) there are a large number i'm sure who
use source  (besides, with debian and redhat, who knows WHAT butchering
they've done to upstreams code)...

Which brings up another question,  may I ask why some of the options to
disable some passwd types were removed from build process? Systems that
dont use system password files (amongst other formats) dont need to
build them, that's not a criticism, 'just sayin'.




signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] distributed mdbox

[list]

On Fri, 23 Mar 2012 23:03:01 +0200, Timo Sirainen  wrote:
> On 23.3.2012, at 19.43, 

> wrote:
> 
>>> Have you tried stress testing it with imaptest? Run in parallel for
both
>>> servers:
>> I did stress test it, but we have developed a "mail bot net" tool for
the
>> purpose.  I should mention this was tested using dovecot 1.2, as this
is
>> our current production version (hopefully will be upgrading soon).  Its
>> comprised of a control server that starts a bot network of client
>> machines
>> that creates pop/imap connections (smtp as well) on our test cluster of
>> dovecot (and postfix) servers.  In my test I distributed the load
across
>> a
>> two node dovecot (/postfix) cluster back ended by glusterfs, which has
>> SAN
>> storage attached to it.  I actually didn't change my configuration from
>> when I had a test NFS server connected to the test servers (mmap
>> disabled,
>> fcntl locking, etc), because glusterfs was an afterthought when we were
>> stress testing our new netapp system using NFS.  We have everything in
>> VMware, including the glusterfs servers.  Using five bot servers and
>> connecting 7 times a second from each server (35 connections per
second)
>> for both pop and imap (70 total connections per second) split between
two
>> dovecot servers I was not seeing any big issues.  The load average was
>> low,
>> and there were no errors to speak of in dovecot (or postfix).  I was
>> mounting the storage with the glusterfs native client, not using NFS
>> (which
>> I have not tested).  I would like to do a more thorough test of
glusterfs
>> using Dovecot 2.0 on some dedicated hardware and see how much further I
>> can
>> push the system.
> 
> What did the bots do? Add messages and delete messages as fast as they
> could? I guess that's mostly enough to see if things work. imaptest
anyway
> hammers the server as fast as it can with all kinds of commands.

We created two python scripts on the bots that listed all the messages in
the inbox then deleted all the messages in the inbox, one script doing pop
and the other doing imap.  The bots were also sending messages to the
server simultaneously to repopulate inboxes.  I didn't know about imaptest,
thanks!

[Dovecot] dsync redesign

[Timo Sirainen]

In case anyone is interested in reading (and maybe helping!) with a dsync 
redesign that's intended to fix all of its current problems, here are some 
possibly incoherent ramblings about it:

http://dovecot.org/tmp/dsync-redesign.txt

and even if you don't understand that, here's another document disguising as an 
algorithm class problem :) If anyone has thoughts on how to solve it, would be 
great:

http://dovecot.org/tmp/dsync-redesign-problem.txt

It only deals with saving new messages, not expunges/flag changes/etc, but 
those should be much simpler.

Re: [Dovecot] distributed mdbox

[Timo Sirainen]

On 23.3.2012, at 19.43,   wrote:

>> Have you tried stress testing it with imaptest? Run in parallel for both
>> servers:
> I did stress test it, but we have developed a "mail bot net" tool for the
> purpose.  I should mention this was tested using dovecot 1.2, as this is
> our current production version (hopefully will be upgrading soon).  Its
> comprised of a control server that starts a bot network of client machines
> that creates pop/imap connections (smtp as well) on our test cluster of
> dovecot (and postfix) servers.  In my test I distributed the load across a
> two node dovecot (/postfix) cluster back ended by glusterfs, which has SAN
> storage attached to it.  I actually didn't change my configuration from
> when I had a test NFS server connected to the test servers (mmap disabled,
> fcntl locking, etc), because glusterfs was an afterthought when we were
> stress testing our new netapp system using NFS.  We have everything in
> VMware, including the glusterfs servers.  Using five bot servers and
> connecting 7 times a second from each server (35 connections per second)
> for both pop and imap (70 total connections per second) split between two
> dovecot servers I was not seeing any big issues.  The load average was low,
> and there were no errors to speak of in dovecot (or postfix).  I was
> mounting the storage with the glusterfs native client, not using NFS (which
> I have not tested).  I would like to do a more thorough test of glusterfs
> using Dovecot 2.0 on some dedicated hardware and see how much further I can
> push the system.

What did the bots do? Add messages and delete messages as fast as they could? I 
guess that's mostly enough to see if things work. imaptest anyway hammers the 
server as fast as it can with all kinds of commands.

[Dovecot] Dovecot IMAP is broken after upgrade from 2.1.2 to 2.1.3

[Michael Neubert]

Hello,

I just upgraded my servers from Dovecot 2.1.2 to Dovecot 2.1.3-0~auto+5 
by using
Debian binaries from "http://xi.rename-it.nl/debian/ 
stable-auto/dovecot-2.1 main".


The config was not touched but now IMAP connections are not possible 
anymore (LMTP works fine).

When I try to connect to a mailbox, the connect fails.

Some log entries:

###
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x10, ret=1: 
before/accept initialization [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: 
before/accept initialization [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv2/v3 
read client hello A [xxx.xxx.xxx.xxx]

Mar 23 21:45:28 auth: Debug: auth client connected (pid=3431)
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
read client hello A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
write server hello A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
write certificate A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
write key exchange A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
write server done A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
flush data [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv3 
read client certificate A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv3 
read client certificate A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
read client key exchange A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
read finished A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
write session ticket A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
write change cipher spec A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
write finished A [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 
flush data [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x20, ret=1: SSL 
negotiation finished successfully [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 imap-login: Warning: SSL: where=0x2002, ret=1: SSL 
negotiation finished successfully [xxx.xxx.xxx.xxx]
Mar 23 21:45:28 auth: Debug: client in: AUTH1   PLAIN   
service=imapsecured lip=yyy.yyy.yyy.yyy  rip=xxx.xxx.xxx.xxx
lport=993   rport=51379

Mar 23 21:45:28 auth: Debug: client out: CONT   1
Mar 23 21:45:28 auth: Debug: client in: CONT1   
AG5lbWlAdmlzaXQtd29ybGQuZGUAUHJvNDUwLnN1
Mar 23 21:45:28 auth-worker(3433): Debug: Loading modules from 
directory: /usr/lib/dovecot/modules/auth
Mar 23 21:45:28 auth-worker(3433): Debug: Module loaded: 
/usr/lib/dovecot/modules/auth/libdriver_mysql.so
Mar 23 21:45:28 auth-worker(3433): Info: mysql(zzz.zzz.zzz.zzz): 
Connected to database dovecot
Mar 23 21:45:28 auth-worker(3433): Debug: sql(username,xxx.xxx.xxx.xxx): 
query: SELECT password, 'directory' AS userdb_home, 'mail' AS 
userdb_uid, 'mail' AS userdb_gid FROM users WHERE username = 'username' 
AND domain = 'domain' AND active = 'Y'

Mar 23 21:45:28 auth: Debug: client out: OK 1   user=username
Mar 23 21:45:28 auth: Debug: master in: REQUEST 2286813185  3394
1   4727968fd3514dd45f623ad9f944e305
Mar 23 21:45:28 auth-worker(3433): Debug: sql(username,xxx.xxx.xxx.xxx): 
SELECT home, uid, gid FROM users WHERE username = 'username' AND domain 
= 'domain'
Mar 23 21:45:28 auth: Debug: master out: USER   2286813185  
username home=directory  uid=8   gid=8
Mar 23 21:45:28 imap-login: Info: Login: user=, method=PLAIN, 
rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy, mpid=3434, TLS
Mar 23 21:45:28 imap-login: Warning: SSL alert: where=0x4008, ret=256: 
warning close notify [xxx.xxx.xxx.xxx]

Mar 23 21:45:28 imap(username): Info: Connection closed in=0 out=303
###

The MySQL authentification seems to work fine, but after this the 
connection is closed with the SSL alert.

In Dovecot 2.1.2 everything worked fine. The SSL certifcate is also correct.

Any hints are welcome to identify the problem. Thanks in advance.

Beste wishes
Michael

Re: [Dovecot] dsync is SLOW compared to rsync

[Jeff Gustafson]

On Fri, 2012-03-23 at 19:02 +0100, Christoph Bußenius wrote:
> Hi,
> 
> maybe try "dsync -o mail_fsync=never".

That didn't seem to make much of a difference. On a 3.1GB backup it
shaved off 5 seconds. dsync's time was over 6 minutes with or without
the mail_fsync=never. rsync copied the same 3.1GB mailbox in 15 seconds.
It seems to me that dsync *should* be able to be just as fast, but it
currently is spending way too much time doing something. What is it?
...Jeff

Re: [Dovecot] Dovecot 2.1.3 Proxy creates mailbox on proxy

[Ed Nitido]

Ooops, didn't email the list... it working now thanks to Timo, solution
below

On Fri, Mar 23, 2012 at 4:14 PM, Timo Sirainen  wrote:
>
>> On 23.3.2012, at 22.01, Ed Nitido wrote:
>>
>> > pass_attrs =
>> uid=user,userPassword=password,=proxy,=master=doveadmin,=pass=xx
>>
>> I guess it doesn't like the "=proxy" part. I guess I should fix it. For
>> now just set "=proxy=y".
>>
>
>

Re: [Dovecot] Advice for new dovecot / imap proxy? setup

[Timo Sirainen]

On 23.3.2012, at 20.24, Gedalya wrote:

> On 03/23/2012 02:12 PM, Luca Lesinigo wrote:
>> Il giorno 23/mar/2012, alle ore 11:50, Timo Sirainen ha scritto:
>>> Are you thinking about actual "dummy" proxying (which is normally what 
>>> Dovecot proxying is about) or about the "imapc" backend 
>>> (http://www.dovecot.fi/products/105-dovecot-imap-adaptor.html)? If you're 
>>> using Dovecot as backend servers, there's really no reason to use imapc 
>>> proxying.
>> I actually didn't know about the two different modes. I guess I would need 
>> imapc to support the older Courier-IMAP server until I migrated everything 
>> away from it, and that I could use "dummy" proxying for the newer dovecot 
>> backends.
>> I don't know if the two can be used at the same time (eg. imapc to the older 
>> backend and dummy to the newer) and/or if there is any drawback in running 
>> everything on imapc (old and new dovecot server). I'll be investigating 
>> this
> I'm using the dummy proxying with a very different backend, certainly not 
> dovecot, and it works great. For your needs (as I understand them) It's a 
> much simpler and robust solution than imapc. Try it out. The main potential 
> source of trouble is possible differences in the CAPABILITY string, but it 
> hasn't caused me any actual problems.

Right, a lot of people have done migration from Courier -> Dovecot using the 
dummy proxying. Since v2.0 the proxying automatically handles any CAPABILITY 
string issues.

Re: [Dovecot] Dovecot 2.1.3 Proxy creates mailbox on proxy

[Timo Sirainen]

On 23.3.2012, at 21.44, Ed Nitido wrote:

> I've compared doveconf -n from both Dovecot 2.0.17 and 2.1.3 and they are
> the same
> 
> Everything works when I go back to 2.0.17, but doesn't when I use 2.1.3

Set auth_debug=yes. What does it log with v2.1.3? Also what's in your 
dovecot-ldap.conf.ext?

Re: [Dovecot] Dovecot 2.1.3 Proxy creates mailbox on proxy

[Ed Nitido]

I've compared doveconf -n from both Dovecot 2.0.17 and 2.1.3 and they are
the same

Everything works when I go back to 2.0.17, but doesn't when I use 2.1.3

Re: [Dovecot] Advice for new dovecot / imap proxy? setup

[Gedalya]

On 03/23/2012 02:12 PM, Luca Lesinigo wrote:

Il giorno 23/mar/2012, alle ore 11:50, Timo Sirainen ha scritto:

Are you thinking about actual "dummy" proxying (which is normally what Dovecot proxying 
is about) or about the "imapc" backend 
(http://www.dovecot.fi/products/105-dovecot-imap-adaptor.html)? If you're using Dovecot as backend 
servers, there's really no reason to use imapc proxying.

I actually didn't know about the two different modes. I guess I would need imapc to 
support the older Courier-IMAP server until I migrated everything away from it, and that 
I could use "dummy" proxying for the newer dovecot backends.
I don't know if the two can be used at the same time (eg. imapc to the older 
backend and dummy to the newer) and/or if there is any drawback in running 
everything on imapc (old and new dovecot server). I'll be investigating this
I'm using the dummy proxying with a very different backend, certainly 
not dovecot, and it works great. For your needs (as I understand them) 
It's a much simpler and robust solution than imapc. Try it out. The main 
potential source of trouble is possible differences in the CAPABILITY 
string, but it hasn't caused me any actual problems.



We'd like to support all the recent IMAP goodies to make modern users happy 
(IMAP IDLE, LEMONADE, etc)

Dovecot doesn't support the full LEMONADE yet, but I don't know if there are 
any LEMONADE clients either.

Oh well I included it in the list because I read about it somewhere, possibly on the 
dovecot site. But what I really meant was simply "support the latest goodies" :)

Il giorno 23/mar/2012, alle ore 11:38, Miguel Tormo ha scritto:

- isolate the mailbox servers from direct external access and just run IMAP on 
them, let other systems run ssl, pop3, smtp, webmail, etc...

I don't think I understand you here. You will need to run POP3 on the mailbox 
servers if you want to give POP3 access to the mailboxes.

Don't ask me why, but I was thinking that a dovecot proxy could talk just imap 
to the backends and use that to serve both POP3 and IMAP to clients. And it's 
possibly what happens with the imapc backend, but I need to do some RTFM about 
it.


The same proxy_maybe (dummy proxy) setup works great for POP3 too. Very 
simple to set up, works like a charm. Nothing much to think about.





However, I can confirm you that IMAP IDLE does work with imap proxy.

That's great, I really want to provide the best possible "push-like" experience 
to modern clients, and as far as I know IMAP IDLE on the protocol side plus some 
notification mechanism (as opposed to regular polling) on the backend side is the way to 
go.


It will work as well as it was working with your existing courier 
server. But it will work great for accounts migrated to native dovecot.



You have my comments above, I think it is doable. In my opinion, the IMAP proxy 
part is the easiest one. MTA configuration to distribute the mails among the 
different mailbox servers can be trickier.

Actually that part is already there. Mail enters my systems via some MX servers 
(with the usual antispam and so on) and it's finally delivered via SMTP to the 
correct mail server via postfix recipient maps (that's because I already 
receive on my MXes mail for domains not hosted on my mail server, the common 
scenario is where I route a domain's mail to the customer's exchange server). 
But right now the mail server also receives direct SMTP connections from the 
clients in addition to incoming mail from my MXes and I'd really prefer to 
separate the two things.


It's a very good idea to have completely separate machines for outgoing 
mail. Once you have imap-only boxes, you can eliminate the need for an 
MTA by using the dovecot LMTP server. Your postfix transport map can 
send mail to either smtp:imap.yourdomain.com:25 or 
lmtp:imap.yourdomain.com:2525 on a per account basis, and you can get 
rid of the MTA in due time.




You could use dovecot LMTP proxy and make the MTA deliver mails through LMTP, 
thus the dovecot proxy instance will handle the sharding for delivering and for 
reading mail.

On the proxy system I plan to run postfix to implement authenticated SMTP (it 
would authenticate on dovecot) and pop/imap-before-smtp (yes we still need to 
support that :| ), but all mail will be reinjected through our MX servers to be 
scanned before final delivery (either local or external).


Since you're sending everything back to the MX, you might as well have 
your MX use LMTP, looking up the correct protocol and host from the 
database, and spend the next couple of years telling your customers to 
change their mail client configuration to use a dedicated outgoing mail 
server. It's worth the trouble.




Thanks people for the suggestions, my next stop is getting to know imapc and 
its details, and how the various other parts will fit with that (eg. giving 
pop3 service to clients).

--
Luca Lesinigo

Re: [Dovecot] Advice for new dovecot / imap proxy? setup

[Luca Lesinigo]

Il giorno 23/mar/2012, alle ore 11:50, Timo Sirainen ha scritto:
> Are you thinking about actual "dummy" proxying (which is normally what 
> Dovecot proxying is about) or about the "imapc" backend 
> (http://www.dovecot.fi/products/105-dovecot-imap-adaptor.html)? If you're 
> using Dovecot as backend servers, there's really no reason to use imapc 
> proxying.
I actually didn't know about the two different modes. I guess I would need 
imapc to support the older Courier-IMAP server until I migrated everything away 
from it, and that I could use "dummy" proxying for the newer dovecot backends.
I don't know if the two can be used at the same time (eg. imapc to the older 
backend and dummy to the newer) and/or if there is any drawback in running 
everything on imapc (old and new dovecot server). I'll be investigating this

>> We'd like to support all the recent IMAP goodies to make modern users happy 
>> (IMAP IDLE, LEMONADE, etc)
> Dovecot doesn't support the full LEMONADE yet, but I don't know if there are 
> any LEMONADE clients either.
Oh well I included it in the list because I read about it somewhere, possibly 
on the dovecot site. But what I really meant was simply "support the latest 
goodies" :)

Il giorno 23/mar/2012, alle ore 11:38, Miguel Tormo ha scritto:
>> - isolate the mailbox servers from direct external access and just run IMAP 
>> on them, let other systems run ssl, pop3, smtp, webmail, etc...
> I don't think I understand you here. You will need to run POP3 on the mailbox 
> servers if you want to give POP3 access to the mailboxes.
Don't ask me why, but I was thinking that a dovecot proxy could talk just imap 
to the backends and use that to serve both POP3 and IMAP to clients. And it's 
possibly what happens with the imapc backend, but I need to do some RTFM about 
it.

> However, I can confirm you that IMAP IDLE does work with imap proxy.

That's great, I really want to provide the best possible "push-like" experience 
to modern clients, and as far as I know IMAP IDLE on the protocol side plus 
some notification mechanism (as opposed to regular polling) on the backend side 
is the way to go.

> You have my comments above, I think it is doable. In my opinion, the IMAP 
> proxy part is the easiest one. MTA configuration to distribute the mails 
> among the different mailbox servers can be trickier.
Actually that part is already there. Mail enters my systems via some MX servers 
(with the usual antispam and so on) and it's finally delivered via SMTP to the 
correct mail server via postfix recipient maps (that's because I already 
receive on my MXes mail for domains not hosted on my mail server, the common 
scenario is where I route a domain's mail to the customer's exchange server). 
But right now the mail server also receives direct SMTP connections from the 
clients in addition to incoming mail from my MXes and I'd really prefer to 
separate the two things.

> You could use dovecot LMTP proxy and make the MTA deliver mails through LMTP, 
> thus the dovecot proxy instance will handle the sharding for delivering and 
> for reading mail.
On the proxy system I plan to run postfix to implement authenticated SMTP (it 
would authenticate on dovecot) and pop/imap-before-smtp (yes we still need to 
support that :| ), but all mail will be reinjected through our MX servers to be 
scanned before final delivery (either local or external).

Thanks people for the suggestions, my next stop is getting to know imapc and 
its details, and how the various other parts will fit with that (eg. giving 
pop3 service to clients).

--
Luca Lesinigo

Re: [Dovecot] dsync is SLOW compared to rsync

[Christoph Bußenius]

Hi,

maybe try "dsync -o mail_fsync=never".

Cheers,
Christoph

--
Christoph Bußenius
Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik
Technische Universität München
+49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

Re: [Dovecot] distributed mdbox

[list]

On Fri, 23 Mar 2012 16:06:25 +0200, Timo Sirainen  wrote:
> On 23.3.2012, at 15.39, 

> wrote:
> 
>> I was able to get dovecot working across a gluster cluster a few weeks
>> ago
>> and it worked just fine.  I would recommend using the native gluster
>> mount
>> option (need to install gluster software on clients), and using
>> distributed
>> replicated as your replication mechanism.
> 
> Have you tried stress testing it with imaptest? Run in parallel for both
> servers:
> 
> imaptest host=gluster1 user=testuser pass=testpass
> imaptest host=gluster2 user=testuser pass=testpass
> 
> http://imapwiki.org/ImapTest
> 
> And see if Dovecot logs any errors.

I did stress test it, but we have developed a "mail bot net" tool for the
purpose.  I should mention this was tested using dovecot 1.2, as this is
our current production version (hopefully will be upgrading soon).  Its
comprised of a control server that starts a bot network of client machines
that creates pop/imap connections (smtp as well) on our test cluster of
dovecot (and postfix) servers.  In my test I distributed the load across a
two node dovecot (/postfix) cluster back ended by glusterfs, which has SAN
storage attached to it.  I actually didn't change my configuration from
when I had a test NFS server connected to the test servers (mmap disabled,
fcntl locking, etc), because glusterfs was an afterthought when we were
stress testing our new netapp system using NFS.  We have everything in
VMware, including the glusterfs servers.  Using five bot servers and
connecting 7 times a second from each server (35 connections per second)
for both pop and imap (70 total connections per second) split between two
dovecot servers I was not seeing any big issues.  The load average was low,
and there were no errors to speak of in dovecot (or postfix).  I was
mounting the storage with the glusterfs native client, not using NFS (which
I have not tested).  I would like to do a more thorough test of glusterfs
using Dovecot 2.0 on some dedicated hardware and see how much further I can
push the system.

[Dovecot] recovery of mdbox folders (was: Re: distributed mdbox)

[Jim Lawson]

On 3/23/12 1:11 PM, Stan Hoeppner wrote:
> On 3/23/2012 7:13 AM, Jim Lawson wrote:
>
>
>> I'd like to move to mdbox, but it would mean the recovery scripts will
>> need to understand which files are associated with which folders, as
>> well as restoring the associated index files.  That's a to-do.
> That's an easy weekend project. ;)
>
Out of curiosity, does anyone do self-service restoration of individual
mdbox folders?  If I'm going to write a script to do it, it'd be nice to
avoid any pitfalls someone else has already run into.  :-)
We're already backing up from snapshots, so the synchronization issues
are solved (at least at backup time...)

Jim

Re: [Dovecot] sysconfdir depreacted

[Andraž 'ruskie' Levstik]

:2012-03-23T12:53:Timo Sirainen:

> Yes, I was also thinking about that, but it's about removing the dovecot/ 
> suffix from other directories as well. That might be something worth doing 
> (--without-package-suffix or something?).

I would suggest to have a --layout=gnu|opt

That would either do what it currently does(gnu) and opt to install
everything into a single dir i.e.:
/opt/dovecot/

With subdirs under there.

-- 
Andraž 'ruskie' Levstik
Source Mage GNU/Linux Games/Xorg grimoire guru
Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/
Geek/Hacker/Tinker

Be advised: causing a disturbance may result in fines, detainment, bodily harm, 
or death. Enjoy your stay.

Re: [Dovecot] distributed mdbox

[Timo Sirainen]

On 23.3.2012, at 19.11, Stan Hoeppner wrote:

> I assume you do still have some minor locking/performance issues with
> the INBOX, even with Director, when LDA and the user MUA are both
> hitting the INBOX index and mbox files.  You'll still see this with
> mdbox, but probably to a lesser degree if you use a smallish
> mdbox_rotate_size value.  To mitigate this INBOX locking you could go
> with a dual namespaces, using maildir or sdbox for the INBOX and mdbox
> for the other user mail folders.


The biggest difference is that mbox requires read locks, mdbox doesn't. mdbox 
lock waits are very similar to maildir's. Of course, I don't know about the 
cluster filesystems' internal locking, but I thought it was even worse with 
Maildir than with mbox because it had to get a read lock for each read file, 
but I guess this depends on the filesystem.

Re: [Dovecot] distributed mdbox

[Stan Hoeppner]

On 3/23/2012 7:13 AM, Jim Lawson wrote:
> On 3/23/12 3:13 AM, Stan Hoeppner wrote:
> 
>>> Speaking as an admin who has run Dovecot on top of GFS both with and
>>> without the director, I would never go back to a cluster without the
>>> director.  The cluster performs *so* much better when glocks can be
>>> cached on a single node, and this can't happen if a single user has IMAP
>>> processes on separate nodes.
>>>
>>> No, you don't strictly need the director if you have GFS, but if you can
>>> manage it, you'll be a lot happier.
>> Did/do you see the Director/glock benefit with both maildir and mdbox
>> Jim?  Do you see any noteworthy performance differences between the two
>> formats on GFS, with and without Director?  BTW, are you hitting FC or
>> iSCSI LUNs?
>>
> 
> Actually, we're all mbox.  This primarily has to do with how users do
> self-service mail recovery from backup: one folder = one file.

Yeah, mbox isn't as dead as some people contend, but it just doesn't
have legs for newer deployment architectures.

> I'd like to move to mdbox, but it would mean the recovery scripts will
> need to understand which files are associated with which folders, as
> well as restoring the associated index files.  That's a to-do.

That's an easy weekend project. ;)

> We're using fibrechannel (IBM v7000) storage, but I would expect to see
> the same thing with iSCSI.  It's mostly about different nodes contending
> over locks on the same files (although I'm sure cache locality helps a
> great deal, too.)  If you end up with imap processes for the same folder
> on different nodes, or mail delivery happening on one node and imap on
> the other, you will feel the lag in your IMAP client.  "Oh, my INBOX has
> been unresponsive for 10 seconds, I must be getting a lot of mail right
> now!"  That's an exaggeration, but not by much.

I was asking about your SAN storage unrelated to the locking issue.
Just a curiosity thing.  Note my email domain. ;)  I'm an FC fan but
iSCSI seems to be more popular in many circles, actually pretty much
market wide these days.  So when I come across another SAN user I'm
naturally curious as to what hardware they use.

Just so nobody gets the wrong idea, I wasn't advocating against Director
earlier in the thread.  I think it's fantastic and solves some critical
scalability problems.  As in your case, it allows one to use his mail
storage format of choice with a cluster filesystem while mostly avoiding
the locking headaches.  In the past one pretty much had to use maildir
with a cluster FS to avoid the locking performance killed.  But one had
to suffer the higher IOPS load on the storage.  Not always a good
tradeoff, especially for busy mail systems.

I assume you do still have some minor locking/performance issues with
the INBOX, even with Director, when LDA and the user MUA are both
hitting the INBOX index and mbox files.  You'll still see this with
mdbox, but probably to a lesser degree if you use a smallish
mdbox_rotate_size value.  To mitigate this INBOX locking you could go
with a dual namespaces, using maildir or sdbox for the INBOX and mdbox
for the other user mail folders.

-- 
Stan

[Dovecot] doveadm user -f index

[Micah Anderson]

I've configured my mail_location to have a different location for
performance reasons so they aren't in the same location as the
mail_location.

The 'doveadm user -f home' is useful to find where a user's home
directory is for various scripting purposes, but I can't seem to find a
way to determine the location of the user's indexes. 

I can do something with the output of dovecot -a to find the
mail_location and then look for a configured INDEX, but then I don't
have a good way of translating the %d/%1n/%n type string formatters into
their values for a user.

thanks for any suggestions!
micah

-- 

Re: [Dovecot] dovecot 2.1.3 dsync Unexpected finish reply

[Micah Anderson]

Micah Anderson  writes:

> dsync-local(u...@example.com): Error: Unexpected finish reply: by 
> ims-d13.mx.aol.com (8.14.1/8.14.1) with ESMTP id q2LEhqXZ017169;
> dsync-local(u...@example.com): Error: Unexpected reply from server:Wed, 
> 21 Mar 2012 10:43:52 -0400
> dsync-local(u...@example.com): Warning: Mailbox changes caused a desync. You 
> may want to run dsync again.

I'm also getting similar strange results with my regular dsync backup:

dsync-local(u...@example.com): Error: Unexpected reply from server: 0 
23bdce147b43674f8e272c449efa1242146 \Recent 1332335848

this is with 2.1.3.

micah

Re: [Dovecot] distributed mdbox

[Timo Sirainen]

On 23.3.2012, at 15.39,   wrote:

> I was able to get dovecot working across a gluster cluster a few weeks ago
> and it worked just fine.  I would recommend using the native gluster mount
> option (need to install gluster software on clients), and using distributed
> replicated as your replication mechanism.

Have you tried stress testing it with imaptest? Run in parallel for both 
servers:

imaptest host=gluster1 user=testuser pass=testpass
imaptest host=gluster2 user=testuser pass=testpass

http://imapwiki.org/ImapTest

And see if Dovecot logs any errors.

Re: [Dovecot] delivering with maildrop

[Stan Hoeppner]

On 3/23/2012 6:41 AM, Radim Kolar wrote:
> Can somebody provide maildrop syntax for using deliver-lda as final
> delivery program during sorting mail in user mailfilter?
> 
> i mean replacement  for "to" statement
> 
> if ( /^(To|Cc):.*dovecot@dovecot.org/:h )
> {
>  to $MAIL/.dovecot/
> }

Dovecot's local delivery agent uses the Sieve language:
http://wiki.dovecot.org/LDA/Sieve

The syntax is quite different than maildrop or procmail.

-- 
Stan

Re: [Dovecot] sysconfdir depreacted

[Eliezer Croitoru]

On 23/03/2012 12:53, Timo Sirainen wrote:

On 23.3.2012, at 12.44, Heiko Schlichting wrote:


Timo wrote:

So the only way I can think of how to change this is to add another
option to optionally remove the dovecot/ suffix from the directory, but
is this really worth the trouble?


I would appreciate such option too. For large dedicated installations other
schemes than /etc/dovecot are common.

See http://dovecot.org/list/dovecot/2009-January/036131.html


Yes, I was also thinking about that, but it's about removing the dovecot/ 
suffix from other directories as well. That might be something worth doing 
(--without-package-suffix or something?).

well squid is using another way such as the directory you specify and 
without the /dovecot (squid) suffix.

it's not that important.
if you do change the config directory you know where you are putting it.
i,m using the /opt/(service name)
to install most of my self complied software so idont really care about it.

but if the sysconfig directory as a directive it should be the default.

Regards,
Eliezer
--
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer  ngtech.co.il

Re: [Dovecot] distributed mdbox

[list]

On Wed, 21 Mar 2012 09:56:12 -0600, James Devine 
wrote:
> Anyone know how to setup dovecot with mdbox so that it can be used
through
> shared storage from multiple hosts?  I've setup a gluster volume and am
> sharing it between 2 test clients.  I'm using postfix/dovecot LDA for
> delivery and I'm using postal to send mail between 40 users.  In doing
> this, I'm seeing these errors in the logs
> 
> Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error:
Fixed
> index file /mnt/testuser34/mdbox/storage/dovecot.map.index:
messages_count
> 272 -> 271
> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error:
Log
> synchronization error at seq=4,offset=3768 for
> /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516,
but
> next_uid = 517
> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error:
Log
> synchronization error at seq=4,offset=4220 for
> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update
> for invalid uid=517
> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error:
Log
> synchronization error at seq=4,offset=5088 for
> /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update
> for invalid uid=517
> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning:
> fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index
> Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning:
> fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index
> 
> 
> This is my dovecot config currently:
> 
> jdevine@test-gluster-client2:~> dovecot -n
> # 2.0.13: /etc/dovecot/dovecot.conf
> # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10
> lock_method = dotlock
> mail_fsync = always
> mail_location = mdbox:~/mdbox
> mail_nfs_index = yes
> mail_nfs_storage = yes
> mmap_disable = yes
> passdb {
>   driver = pam
> }
> protocols = " imap"
> ssl_cert =  ssl_key =  userdb {
>   driver = passwd
> }

I was able to get dovecot working across a gluster cluster a few weeks ago
and it worked just fine.  I would recommend using the native gluster mount
option (need to install gluster software on clients), and using distributed
replicated as your replication mechanism.  If you're running two gluster
servers you should have a replica count of two with distributed replicated.
You should test first to make sure you can create a file in both mounts
and see it from every mount point in the cluster, as well as interact with
it.  It's also very important to make sure your servers are running with
synchronized clocks from an NTP server.  Very bad things happen to a
(dovecot or gluster) cluster out of sync with NTP.

Re: [Dovecot] dovecot-auth restaring and caching

[Angel L. Mateo]

El 22/03/12 19:57, Timo Sirainen escribió:

On 22.3.2012, at 11.55, Angel L. Mateo wrote:


The problem I'm having is that if I have no activity in the server, 
dovecot stops its auth process and when another message is received, it 
restarted it, but with an empty cache.


service auth {
   idle_kill = 0
}



	In a test server I have, this have solved the problem. In my 
productions servers it is still being restarted. Could it be another 
parameter involve in this?


service_count is set to 0.

	I have also seen that, whenever dovecot/auth is restarted, 
dovecot/config has also been restarted. Could be related?


My config related with this service auth is:

service auth {
  chroot =
  client_limit = 4096
  drop_priv_before_exec = no
  executable = auth
  extra_groups =
  group =
  idle_kill = 0
  privileged_group =
  process_limit = 1
  process_min_avail = 0
  protocol =
  service_count = 0
  type =
  unix_listener auth-client {
group =
mode = 0600
user =
  }
  unix_listener auth-login {
group =
mode = 0600
user = $default_internal_user
  }
  unix_listener auth-master {
group =
mode = 0600
user =
  }
  unix_listener auth-userdb {
group =
mode = 0666
user =
  }
  unix_listener login/login {
group =
mode = 0666
user =
  }
  user = $default_internal_user
  vsz_limit = 18446744073709551615 B
}


--
Angel L. Mateo Martínez
Sección de Telemática
Área de Tecnologías de la Información   _o)
y las Comunicaciones Aplicadas (ATICA)  / \\
http://www.um.es/atica_(___V
Tfo: 868887590
Fax: 86337

Re: [Dovecot] distributed mdbox

[Jim Lawson]

On 3/23/12 3:13 AM, Stan Hoeppner wrote:

>> Speaking as an admin who has run Dovecot on top of GFS both with and
>> without the director, I would never go back to a cluster without the
>> director.  The cluster performs *so* much better when glocks can be
>> cached on a single node, and this can't happen if a single user has IMAP
>> processes on separate nodes.
>>
>> No, you don't strictly need the director if you have GFS, but if you can
>> manage it, you'll be a lot happier.
> Did/do you see the Director/glock benefit with both maildir and mdbox
> Jim?  Do you see any noteworthy performance differences between the two
> formats on GFS, with and without Director?  BTW, are you hitting FC or
> iSCSI LUNs?
>

Actually, we're all mbox.  This primarily has to do with how users do
self-service mail recovery from backup: one folder = one file.

I'd like to move to mdbox, but it would mean the recovery scripts will
need to understand which files are associated with which folders, as
well as restoring the associated index files.  That's a to-do.

We're using fibrechannel (IBM v7000) storage, but I would expect to see
the same thing with iSCSI.  It's mostly about different nodes contending
over locks on the same files (although I'm sure cache locality helps a
great deal, too.)  If you end up with imap processes for the same folder
on different nodes, or mail delivery happening on one node and imap on
the other, you will feel the lag in your IMAP client.  "Oh, my INBOX has
been unresponsive for 10 seconds, I must be getting a lot of mail right
now!"  That's an exaggeration, but not by much.

Jim

[Dovecot] delivering with maildrop

[Radim Kolar]

Can somebody provide maildrop syntax for using deliver-lda as final 
delivery program during sorting mail in user mailfilter?


i mean replacement  for "to" statement

if ( /^(To|Cc):.*dovecot@dovecot.org/:h )
{
 to $MAIL/.dovecot/
}

Re: [Dovecot] Problems with upgrade 2.0.16 -> 2.1.3

[Timo Sirainen]

On 23.3.2012, at 12.58, Joseph Tam wrote:

> I ran into two issues trying to upgrade our dovecot installation (Solaris 10).
> 
> 1) Does not compile with OpenSSL 0.9.7
> 
>   Not a big deal, as I was able to successfully against OpenSSL 0.9.8,
>   but does dovecot require OpenSSL >= 0.9.8 now?

Hm. Maybe it's time by now? :) It could be fixed with some more #ifdefs but 
those make code more unreadable.

> 2) Dovecot's LDA does not work
> 
>   After stopping the the old dovecot, and starting dovecot 2.1.3 using 
> tghe
>   exact same config file, local mail delivery tempfails:
> 
>   Mar 23 02:51:51 server dovecot: auth: Error: getpeerucred() failed: Bad 
> address

http://hg.dovecot.org/dovecot-2.1/rev/98fd46f8d1ab fixes this?

Re: [Dovecot] sysconfdir depreacted

[Rainer Frey]

On Mar 23, 2012, at 10:51 AM, Timo Sirainen wrote:
>>> :2012-03-22T11:55:Noel Butler:
>>> 
 perhaps it should be renamed then, given it violates the known normal
 for SYSCONF dir, you've just created another form of --datadir
>>> 
>>> Not really. The way I see it works as expected.
>> 
>>The directory for installing read-only data files that pertain
>>to a single machine–that is to say, files for configuring a
>>host. Mailer and network configuration files, ‘/etc/passwd’, and
>>so forth belong here. All the files in this directory should be
>>ordinary ASCII text files. This directory should normally be
>>‘/usr/local/etc’, but write it as ‘$(prefix)/etc’. (If you are
>>using Autoconf, write it as ‘@sysconfdir@’.)

Well, I don't see that that prevents organizing the files in sysconfdir into a 
subdirectory.

>  ton of software installs into /etc// directory. [...]
> So the only way I can think of how to change this is to add another option to 
> optionally remove the dovecot/ suffix from the directory, but is this really 
> worth the trouble?

I really don't think so. What for? Nobody has shown a real-world problem with 
that subdirectory.

Re: [Dovecot] sysconfdir depreacted

[Joseph Tam]

On Fri, 23 Mar 2012, dovecot-requ...@dovecot.org wrote:


See http://dovecot.org/list/dovecot/2009-January/036131.html


Yes, I was also thinking about that, but it's about removing the
dovecot/ suffix from other directories as well.  That might be
something worth doing (--without-package-suffix or something?).


+1.  I fake it now with symlinks (e.g. etc/dovecot -> .).

Joseph Tam 

[Dovecot] Problems with upgrade 2.0.16 -> 2.1.3

[Joseph Tam]

I ran into two issues trying to upgrade our dovecot installation (Solaris 10).

1) Does not compile with OpenSSL 0.9.7

Not a big deal, as I was able to successfully against OpenSSL 0.9.8,
but does dovecot require OpenSSL >= 0.9.8 now?

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib 
-I../../src/lib-test -std=gnu99 -O3 -fomit-frame-pointer -mcpu=ultrasparc -Wall 
-W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith 
-Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -MT 
istream-openssl.lo -MD -MP -MF .deps/istream-openssl.Tpo -c istream-openssl.c  
-fPIC -DPIC -o .libs/istream-openssl.o
iostream-openssl-context.c:9:28: openssl/engine.h: No such file or 
directory
iostream-openssl-context.c: In function `ssl_iostream_deinit_global':
iostream-openssl-context.c:431: warning: implicit declaration of 
function `ENGINE_finish'
iostream-openssl-context.c:432: warning: implicit declaration of 
function `ENGINE_cleanup'
...

2) Dovecot's LDA does not work

After stopping the the old dovecot, and starting dovecot 2.1.3 using 
tghe
exact same config file, local mail delivery tempfails:

Mar 23 02:51:51 server dovecot: auth: Error: getpeerucred() failed: Bad 
address
Mar 23 02:51:51 server dovecot: auth: Error: userdb connection: Failed 
to get peer's credentials
Mar 23 02:51:51 server dovecot: lda: Error: userdb lookup(j.tam): 
Disconnected unexpectedly
Mar 23 02:51:51 server dovecot: lda: Fatal: Internal error occurred. 
Refer to server log for more information.

# Sendmail reports
stat=Deferred: local mailer (/var/dovecot/libexec/dovecot-lda) exited 
with EX_TEMPFAIL

After seeing 2) in the logs, I had to revert back to 2.0.16.  Any hints
on what could be wrong?

Joseph Tam 

# 2.0.16: /var/dovecot/etc/dovecot/dovecot.conf
# OS: SunOS 5.10 sun4u  nfs
auth_cache_negative_ttl = 10 mins
auth_cache_size = 64 k
auth_cache_ttl = 1 days
auth_failure_delay = 5 secs
auth_master_user_separator = *
auth_socket_path = /var/dovecot/run/auth-userdb
auth_username_chars = abcdefghijklmnopqrstuvwxyz01234567890.-_
auth_worker_max_count = 1
base_dir = /var/dovecot/run
default_vsz_limit = 64 M
deliver_log_format = 
first_valid_gid = 1

first_valid_uid = 1
hostname = our.mail.domain
last_valid_gid = 1
last_valid_uid = 1
lda_mailbox_autocreate = yes
log_timestamp = 
login_greeting = Ready.

mail_location = 
mbox:/nfs/home/%n/mail:INBOX=/nfs/mail/%n:INDEX=/data/dc-cache/%n
mail_nfs_storage = yes
mail_temp_dir = /var/tmp
mbox_very_dirty_syncs = yes
mbox_write_locks = dotlock_try fcntl
namespace {
  inbox = yes
  location =
  prefix =
  separator = /
}
namespace {
  hidden = yes
  list = no
  location =
  prefix = /
  separator = /
}
namespace {
  hidden = yes
  list = no
  location =
  prefix = ~/mail/
  separator = /
}
namespace {
  hidden = yes
  list = no
  location =
  prefix = mail/
  separator = /
}
passdb {
  args = /var/dovecot/etc/master-users
  driver = passwd-file
  master = yes
  pass = yes
}
passdb {
  args = /var/yp/etc/passwd
  driver = passwd-file
}
postmaster_address = mailer-dae...@our.mail.domain
protocols = imap pop3
sendmail_path = /usr/lib/sendmail
service auth-worker {
  user = dovecot
}
service auth {
  idle_kill = 1 hours
}
service imap-login {
  process_limit = 2
  service_count = 0
}
service imap {
  process_limit = 512
}
service pop3-login {
  process_limit = 1
  service_count = 0
}
service pop3 {
  process_limit = 64
}
shutdown_clients = no
ssl_cert = 

Re: [Dovecot] quota ldap

[Alain DEFRANCE]

thanks Nick

so if i understand correctly i can mix the 2 quota_rule ?

the one who came from ldap user_attrs (quota_rule=*:bytes=%$)

and the other which from quota_rule2 = Trash:storage=+3%%

in your case you add 3% quota more for Trash ?
Am i write ?

regards



> On 23/3/2012 12:20 μμ, Alain DEFRANCE wrote:
>
>> how can i add "Trash:storage= "  to have more place for deleting
>> messages like in
>>
>
> See, for example, my setup:
> http://old.nabble.com/ldap-userdb-warning-in-v2.1.1-td33544211.html
>
> I use a single conf file (because it's small and it's more intuitive
> to me).
>
> Details: http://wiki2.dovecot.org/Quota/Configuration
>
> Regards,
> Nick
>


-- 
*Alain DEFRANCE* - Ingénieur systèmes et réseaux

Direction des systèmes d'information (DiSI)

Centre d'Exploitation des Infrastructures Informatiques (CEDII)
Cellule Réseau et Expertise Systèmes

Bât Ile de France - RDC - Bureau 58
Université d'Evry Val d'Essonne
4, Bd F. Mitterrand - 91025 EVRY Cedex

Tel : 01.69.47.80.69 - Fax : 01.69.47.80.24
Mail : alain.defra...@univ-evry.fr
Site UEVE : http://www.univ-evry.fr

Re: [Dovecot] sysconfdir depreacted

[Timo Sirainen]

On 23.3.2012, at 12.44, Heiko Schlichting wrote:

> Timo wrote:
>> So the only way I can think of how to change this is to add another
>> option to optionally remove the dovecot/ suffix from the directory, but
>> is this really worth the trouble?
> 
> I would appreciate such option too. For large dedicated installations other
> schemes than /etc/dovecot are common.
> 
> See http://dovecot.org/list/dovecot/2009-January/036131.html

Yes, I was also thinking about that, but it's about removing the dovecot/ 
suffix from other directories as well. That might be something worth doing 
(--without-package-suffix or something?).

Re: [Dovecot] Advice for new dovecot / imap proxy? setup

[Timo Sirainen]

On 21.3.2012, at 16.43, Luca Lesinigo wrote:

> The issue I'm investigating right now is how to manage a single IMAP / POP / 
> SMTP / webmail  "entry point" for multiple mail servers... in other words an 
> IMAP proxy.

Are you thinking about actual "dummy" proxying (which is normally what Dovecot 
proxying is about) or about the "imapc" backend 
(http://www.dovecot.fi/products/105-dovecot-imap-adaptor.html)? If you're using 
Dovecot as backend servers, there's really no reason to use imapc proxying.

> We'd like to support all the recent IMAP goodies to make modern users happy 
> (IMAP IDLE, LEMONADE, etc)

Dovecot doesn't support the full LEMONADE yet, but I don't know if there are 
any LEMONADE clients either.

Re: [Dovecot] quota ldap

[Nikolaos Milas]

On 23/3/2012 12:20 μμ, Alain DEFRANCE wrote:


how can i add "Trash:storage= "  to have more place for deleting
messages like in



See, for example, my setup: 
http://old.nabble.com/ldap-userdb-warning-in-v2.1.1-td33544211.html


I use a single conf file (because it's small and it's more intuitive to me).

Details: http://wiki2.dovecot.org/Quota/Configuration

Regards,
Nick

Re: [Dovecot] sysconfdir depreacted

[Heiko Schlichting]

Timo wrote:
> So the only way I can think of how to change this is to add another
> option to optionally remove the dovecot/ suffix from the directory, but
> is this really worth the trouble?

I would appreciate such option too. For large dedicated installations other
schemes than /etc/dovecot are common.

See http://dovecot.org/list/dovecot/2009-January/036131.html

Heiko

Heiko SchlichtingFreie Universität Berlin
heiko.schlicht...@fu-berlin.de   Zentraleinrichtung für Datenverarbeitung
Telefon +49 30 838-54327 Fabeckstraße 32
Telefax +49 30 838454327 14195 Berlin

Re: [Dovecot] Advice for new dovecot / imap proxy? setup

[Miguel Tormo]

El Miércoles, 21 de Marzo de 2012 15:43:14 Luca Lesinigo escribió:
> Hello list.
Hello, 
> 
> I'm planning a new mail servers for our company's customers to replace the 
> oldish Courier-IMAP based one, we already started to deploy some mail 
> accounts on a dovecot-2.0 server as an early test.
> I'd like to implement the new system with dovecot-2 (I'll probably go 
> straight to dovecot-2.1.x) and I'd like to get it right from the beginning so 
> I'm here asking for some advice.
> 
> The issue I'm investigating right now is how to manage a single IMAP / POP / 
> SMTP / webmail  "entry point" for multiple mail servers... in other words an 
> IMAP proxy.
> It would be desirable for multiple reasons:
I have recently deployed a very similar setup: imap proxy, mailbox sharding... 
Although not exactly like yours. Comments below:

> - graceful migration from the current system: we'd make the mailserver 
> hostname point to the proxy (along with its SSL certificates) and then the 
> proxy would route each domain to the correct IMAP non-ssl server on our LAN. 
> No need to update customer's systems configuration and we can move one domain 
> at a time from the old to the new server, behind the scenes
This is reasonable. For example, I did this to seamless migrate lots of users 
from one server to another, migrating just a few of them at a time.
> - be ready for similar migrations in the future (eg. right now we're still 
> keeping the imap servers with the qmail MTA, but we'd like to switch to 
> postfix+dovecot in the future)
You can do the exact same thing in the future, of course.
> - be ready for sharding mail domains on multiple IMAP servers (if/when 
> current hardware reach its capacity or needs to be swapped out for new gear)
This is fairly easy to accomplish with imap proxying.
> - be ready to serve traffic over IPv6 without touching our precious mailbox 
> servers
This is doable.
> - isolate the mailbox servers from direct external access and just run IMAP 
> on them, let other systems run ssl, pop3, smtp, webmail, etc...
I don't think I understand you here. You will need to run POP3 on the mailbox 
servers if you want to give POP3 access to the mailboxes.
> 
> Ideally the 'proxy' system would run dovecot imap and pop3 (SSL protected) 
> and Roundcube webmail (PHP, on https) and just speak IMAP to the underlying 
> mail servers on our internal LAN.
> We'd like to support all the recent IMAP goodies to make modern users happy 
> (IMAP IDLE, LEMONADE, etc) and possibly implement Maildir quota on the new 
> backend mailbox server to improve our operations (currently we just run du in 
> a cronjob once a day on the current mailserver, IMAP clients including the 
> webmail do not know about quota and thus cannot show amount of free space).
I didn't implement a lemonade profile nor quotas in my setup. However, I can 
confirm you that IMAP IDLE does work with imap proxy.
> 
> In addition to that, customer's will hit the SMTP server running on that 
> 'proxy' system and this is good to keep its configuration separated from the 
> SMTP server of the actual mail servers (which has a different configuration 
> and is restricted to get connections only from our MX systems and not from 
> outside sources).
No problem with that, but this is related to the MTA configuration, not dovecot.
> 
> I'd like to know if that plan sounds reasonable or if there's something 
> stupid in it.
> Also, is the proxy going to support all kind of IMAP stuff of the backend 
> server (IDLE, CONDSTORE, Maildir quota, immediate notification of IDLE 
> clients thanks to linux inotify, etc...) or will it limit me somehow?
You have my comments above, I think it is doable. In my opinion, the IMAP proxy 
part is the easiest one. MTA configuration to distribute the mails among the 
different mailbox servers can be trickier. You could use dovecot LMTP proxy and 
make the MTA deliver mails through LMTP, thus the dovecot proxy instance will 
handle the sharding for delivering and for reading mail.

[Dovecot] quota ldap

[Alain DEFRANCE]

hello all,
i'm using quota + ldap with dovecot 2
in dovecot-ldap.conf.ext file i have the line :

user_attrs =
homeDirectory=home,uidNumber=uid,gidNumber=gid,quota=quota_rule=*:storage=%$B


how can i add "Trash:storage= "  to have more place for deleting
messages like in

90-quota.conf file ?
 quota_rule2 = Trash:storage

thanks for help
regards



-- 
*Alain DEFRANCE* - Ingénieur systèmes et réseaux

Direction des systèmes d'information (DiSI)

Centre d'Exploitation des Infrastructures Informatiques (CEDII)
Cellule Réseau et Expertise Systèmes

Bât Ile de France - RDC - Bureau 58
Université d'Evry Val d'Essonne
4, Bd F. Mitterrand - 91025 EVRY Cedex

Tel : 01.69.47.80.69 - Fax : 01.69.47.80.24
Mail : alain.defra...@univ-evry.fr
Site UEVE : http://www.univ-evry.fr

Re: [Dovecot] sysconfdir depreacted

[Timo Sirainen]

On 22.3.2012, at 10.30, Noel Butler wrote:

> On Thu, 2012-03-22 at 07:28 +0100, Andraž 'ruskie' Levstik wrote:
> 
>> :2012-03-22T11:55:Noel Butler:
>> 
>>> perhaps it should be renamed then, given it violates the known normal
>>> for SYSCONF dir, you've just created another form of --datadir
>> 
>> Not really. The way I see it works as expected. The sysconf dir is the
> 
> 
> Then you and I and a few other devs involved in other very well known
> bits of software that everyone likely uses, will have to agree to
> disagree

A ton of software installs into /etc// directory. Most Linux 
distributions installed Dovecot v1.x that way as well. And of course everyone 
expects configuration to be under /etc. The default of sysconfdir is 
PREFIX/etc/. Dovecot v2.0 really shouldn't install its stuff into PREFIX/etc/ 
but into PREFIX/etc/dovecot/. So the only way I can think of how to change this 
is to add another option to optionally remove the dovecot/ suffix from the 
directory, but is this really worth the trouble?

[Dovecot] Dovecot v2.1.3 (f30437ed63dc) Auth/Login Issues

[Thomas Leuxner]

Hi,

some change between ff5c341f8838 and f30437ed63dc seems to have broken auth:

=> Bad Login 
Mar 23 09:01:46 spectre dovecot: master: Dovecot v2.1.3 (f30437ed63dc) starting 
up
[...]
Mar 23 10:25:44 spectre dovecot: auth: Debug: auth client connected (pid=7266)
Mar 23 10:25:45 spectre dovecot: auth: Debug: client in: 
AUTH#0111#011PLAIN#011service=imap#011secured#011lip=188.138.0.199#011rip=80.187.102.243#011lport=143#011rport=62388#011resp=
Mar 23 10:25:45 spectre dovecot: auth: Debug: 
cache(t...@leuxner.net,80.187.102.243): hit: 
#011userdb_quota_rule=*:storage=5G#011userdb_acl_groups=PublicMailboxAdmins
Mar 23 10:25:45 spectre dovecot: auth: Debug: client out: 
OK#0111#011user=t...@leuxner.net
Mar 23 10:25:45 spectre dovecot: auth: Debug: master in: 
REQUEST#0113958898689#0117266#0111#011bfc44f32051961b909e2b458440d645f
Mar 23 10:25:45 spectre dovecot: auth: Debug: 
userdb-cache(t...@leuxner.net,80.187.102.243): hit:
t...@leuxner.net#011uid=5000#011gid=5000#011home=/var/vmail/domains/leuxner.net/tlx#011quota_rule=*:storage=5G#011acl_groups=PublicMailboxAdmins
Mar 23 10:25:45 spectre dovecot: auth: Debug: master out:
USER#0113958898689#011...@leuxner.net#011uid=xxx#011gid=xxx#011home=/var/vmail/domains/leuxner.net/tlx#011quota_rule=*:storage=5G#011acl_groups=PublicMailboxAdmins
Mar 23 10:25:45 spectre dovecot: imap-login: Login: user=, 
method=PLAIN, rip=80.187.102.243, lip=188.138.0.199, mpid=7267, TLS
Mar 23 10:25:45 spectre dovecot: imap(t...@leuxner.net): Connection closed in=0 
out=319uthentication/login:

=> Good Login
Mar 23 10:26:37 spectre dovecot: master: Dovecot v2.1.3 (ff5c341f8838) starting 
up
[...]
Mar 23 10:27:18 spectre dovecot: auth: Debug: Loading modules from directory: 
/usr/lib/dovecot/modules/auth
Mar 23 10:27:18 spectre dovecot: auth: Debug: auth client connected (pid=9832)
Mar 23 10:27:19 spectre dovecot: auth: Debug: client in: 
AUTH#0111#011PLAIN#011service=imap#011secured#011lip=188.138.0.199#011rip=80.187.102.243#011lport=143#011rport=51647#011resp=
Mar 23 10:27:19 spectre dovecot: auth: Debug: 
cache(t...@leuxner.net,80.187.102.243): miss
Mar 23 10:27:19 spectre dovecot: auth: Debug: passwd-file 
/var/vmail/auth.d/leuxner.net/passwd: Read 1 users in 0 secs
Mar 23 10:27:19 spectre dovecot: auth: Debug: 
passwd-file(t...@leuxner.net,80.187.102.243): lookup: user=t...@leuxner.net 
file=/var/vmail/auth.d/leuxner.net/passwd
Mar 23 10:27:19 spectre dovecot: auth: Debug: client out: 
OK#0111#011user=t...@leuxner.net
Mar 23 10:27:19 spectre dovecot: auth: Debug: master in: 
REQUEST#0113656384513#0119832#0111#0114782efcbd0324b228bb85aaae916cfe6
Mar 23 10:27:19 spectre dovecot: auth: Debug: 
userdb-cache(t...@leuxner.net,80.187.102.243): miss
Mar 23 10:27:19 spectre dovecot: auth: Debug: 
passwd-file(t...@leuxner.net,80.187.102.243): lookup: user=t...@leuxner.net 
file=/var/vmail/auth.d/leuxner.net/passwd
Mar 23 10:27:19 spectre dovecot: auth: Debug: master out:
USER#0113656384513#011...@leuxner.net#011uid=xxx#011gid=xxx#011home=/var/vmail/domains/leuxner.net/tlx#011quota_rule=*:storage=5G#011acl_groups=PublicMailboxAdmins
Mar 23 10:27:19 spectre dovecot: imap-login: Login: user=, 
method=PLAIN, rip=80.187.102.243, lip=188.138.0.199, mpid=9835, TLS

Regards
Thomas


signature.asc
Description: Digital signature

Re: [Dovecot] distributed mdbox

[Stan Hoeppner]

On 3/22/2012 11:17 AM, Jim Lawson wrote:
> On 03/22/2012 12:11 AM, Stan Hoeppner wrote:
>> On 3/21/2012 12:04 PM, Timo Sirainen wrote:
>>> The problem is most likely the same as with NFS: Server A caches data -> 
>>> server B modifies data -> server A modifies data using stale cached state 
>>> -> corruption. Glusterfs works with FUSE, and FUSE has quite similar 
>>> problems as NFS.
>>>
>>> With director you guarantee that the same mailbox isn't accessed 
>>> simultaneously by multiple servers, so this problem goes away.
>> If using "real" shared storage i.e. an FC or iSCSI SAN LUN, you could
>> use a true cluster file system such as OCFS or GFS.  Both will eliminate
>> this problem, and without requiring Dovecot director.  And you'll get
>> better performance than with Gluster, which, BTW, isn't really suitable
>> as a transactional filesystem, was not designed for such a use case.
> 
> Speaking as an admin who has run Dovecot on top of GFS both with and
> without the director, I would never go back to a cluster without the
> director.  The cluster performs *so* much better when glocks can be
> cached on a single node, and this can't happen if a single user has IMAP
> processes on separate nodes.
> 
> No, you don't strictly need the director if you have GFS, but if you can
> manage it, you'll be a lot happier.

Did/do you see the Director/glock benefit with both maildir and mdbox
Jim?  Do you see any noteworthy performance differences between the two
formats on GFS, with and without Director?  BTW, are you hitting FC or
iSCSI LUNs?

-- 
Stan