[Dovecot] Encryption solution for messages at rest

[Alan Brown]

Date: Tue, 29 Oct 2013 08:54:04 +0100
From: Robert Schetterer 
To: dovecot@dovecot.org
Subject: Re: [Dovecot] Encryption solution for messages at rest
Message-ID: <526f699c.9080...@sys4.de>
Content-Type: text/plain; charset=ISO-8859-1


you shouldnt host mail/imap services on the same servers with massive
http hosting,



You shouldn't host anything else on a webserver FULLSTOP.

Webservers are best treated as "disposable" and should be heavily 
sandboxed. Any resources they can use should be vetted and ideally set 
as "read only"


Inbound external access should be firewalled down to the webserver ports 
and OUTBOUND traffic should be firewalled too (If it has no business 
initiating external connections then block all SYNs), in order to stop 
it becoming a DDoS zombie.


It's foolish (at best) to have mail servers running on a webserver, 
because if it's compromised it can immediately be used as a spam engine 
without much further effort.


At least if it has to hand mail off to another mailserver you have a 
chance to run outbound filtering on the emitted mail without worrying 
about that being compromised too.

Re: [Dovecot] Mailbox conversion/importing - SOLVED

[Alan Brown]

On 10/06/13 12:03, Alan Brown wrote:


I've been tasked with importing a large bunch of mbox folders (about
500) into an existing mdbox setup in Dovecot 2.1

As far as I can see, dsync "mirror" or "backup" are both inappropriate
ways of doing this. Does anyone have any suggestions about how I could
proceed?


I've finally discovered doveadm import.

In this instance:  doveadm -Dv import -u [user]  mbox:/full/path/to/mbox 
old-mbox all



Relative paths don't work. :)

[Dovecot] Mailbox conversion/importing

[Alan Brown]

I've been tasked with importing a large bunch of mbox folders (about 
500) into an existing mdbox setup in Dovecot 2.1


As far as I can see, dsync "mirror" or "backup" are both inappropriate 
ways of doing this. Does anyone have any suggestions about how I could 
proceed?


Thanks in advance

Re: [Dovecot] Administrative mailbox deletions

[Alan Brown]

and "doveadm mailbox delete" doesn't like it.

Does anyone have the magic sauce needed to escape the - character?


An "escaping" that works at the *nix level (not necessarily with
doveadm, but I would assume it works there too) is, e.g.,

rm -- -foo

i.e., double dashes to nullify the "-" option flag, followed by the
filename.



That does work. Thanks.

Alan

[Dovecot] Administrative mailbox deletions

[Alan Brown]

I'm in the process of nuking a bunch of dead mailboxes after they've 
been migrated to other servers - but the accounts have been kept.


A simple shell script takes care of most of it, BUT (there's always 
a "but" isn't there?)


One user has named all his mailboxes with leading hyphens.

ie:

-foo
-bar
-bazz

and "doveadm mailbox delete" doesn't like it.

Does anyone have the magic sauce needed to escape the - character?

Thanks in advance
Alan

[Dovecot] Strange Dovecot 2.0.20 auth chokes and cores

[Alan Brown]

From: "Konrad ."



When we upgrade our kernels from 2.6.32.2 to 3.2.16 something strange
has happened.
On high traffic dovecot/auth looks like not responding.

We found a lot of this lines at the log:


[snip]


We compile Dovecot with poll instead of epoll (--with-ioloop=poll) and
this works for us.


Is any problem with epoll on 3.2.x kernels?


Yes - and it's been discussed here.

Some "bright spark" rewrote the kernel epoll code to prevent DoS attacks 
caused by "excessive forking".


That "bright spark" clearly doesn't use imap or webserver software.

Redhat backported this change to kernels released after February 2012 
and are about to release an emergency kernel release which removes it.


A number of $LARGE companies were "not happy"

As you've posted, compiling without epoll fixes the problem, but that 
doesn't help anyone using distro or repo packages.

Re: [Dovecot] Dovecot 2.1 mbox + maildir

[Alan Brown]

From: Timo Sirainen



On Mon, 2012-05-28 at 14:50 +0100, Alan Brown wrote:

>  What syntax is needed to make this work?
>
>  The 2.0 wiki recomendations don't work - I can see the inboxes or the
>  folders but not both at once and there are lots of error messages about
>  prefix clashes if I simply use the existing 2.0.20 conf file on 2.1.6

Are you saying that it works in v2.0 but not in v2.1?


yes.


Then something's
wrong. Show your doveconf -n output and what error messages you see.


Attached to this email (dovecot 2.0 and 2.1 -n outputs)

In both cases, we use the same local config file, included from 
dovecot.conf (I've included both of these too)


The error message is:

May 30 17:00:31 msslat dovecot: imap(foobar): Error: user foobar: 
Initialization failed: namespace configuration error: Duplicate 
namespace prefix: ""
May 30 17:00:31 msslat dovecot: imap(foobar): Error: Invalid user 
settings. Refer to server log for more information.
May 30 17:00:31 msslat dovecot: master: Warning: Killed with signal 15 
(by pid=18489 uid=0 code=kill)



Attempting to tweak MAIL.conf to fix this error simply resulted in 
various states of "broken", so any help untangling this will be greatly 
appreciated.




## Dovecot configuration file

# If you're in a hurry, see http://wiki2.dovecot.org/QuickConfiguration

# "doveconf -n" command gives a clean output of the changed settings. Use it
# instead of copy&pasting files when posting to the Dovecot mailing list.

# '#' character and everything after it is treated as comments. Extra spaces
# and tabs are ignored. If you want to use either of these explicitly, put the
# value inside quotes, eg.: key = "# char and trailing whitespace  "

# Default values are shown for each setting, it's not required to uncomment
# those. These are exceptions to this though: No sections (e.g. namespace {})
# or plugin settings are added by default, they're listed only as examples.
# Paths are also just examples with the real defaults being based on configure
# options. The paths listed here are for configure --prefix=/usr
# --sysconfdir=/etc --localstatedir=/var

# Protocols we want to be serving.
#protocols = imap pop3 lmtp

# A comma separated list of IPs or hosts where to listen in for connections. 
# "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces.
# If you want to specify non-default ports or anything more complex,
# edit conf.d/master.conf.
#listen = *, ::

# Base directory where to store runtime data.
#base_dir = /var/run/dovecot/

# Greeting message for clients.
#login_greeting = Dovecot ready.

# Space separated list of trusted network ranges. Connections from these
# IPs are allowed to override their IP addresses and ports (for logging and
# for authentication checks). disable_plaintext_auth is also ignored for
# these networks. Typically you'd specify your IMAP proxy servers here.
#login_trusted_networks =

# Sepace separated list of login access check sockets (e.g. tcpwrap)
#login_access_sockets = 

# Show more verbose process titles (in ps). Currently shows user name and
# IP address. Useful for seeing who are actually using the IMAP processes
# (eg. shared mailboxes or if same uid is used for multiple accounts).
#verbose_proctitle = no

# Should all processes be killed when Dovecot master process shuts down.
# Setting this to "no" means that Dovecot can be upgraded without
# forcing existing client connections to close (although that could also be
# a problem if the upgrade is e.g. because of a security fix).
#shutdown_clients = yes

# If non-zero, run mail commands via this many connections to doveadm server,
# instead of running them directly in the same process.
#doveadm_worker_count = 0
# UNIX socket or host:port used for connecting to doveadm server
#doveadm_socket_path = doveadm-server

# Space separated list of environment variables that are preserved on Dovecot
# startup and passed down to all of its child processes. You can also give
# key=value pairs to always set specific settings.
#import_environment = TZ

##
## Dictionary server settings
##

# Dictionary can be used to store key=value lists. This is used by several
# plugins. The dictionary can be accessed either directly or though a
# dictionary server. The following dict block maps dictionary names to URIs
# when the server is used. These can then be referenced using URIs in format
# "proxy::".

dict {
  #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
  #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext
}

# Most of the actual configuration gets included below. The filenames are
# first sorted by their ASCII value and parsed in that order. The 00-prefixes
# in filenames are intended to make it easier to understand the ordering.
!include conf.d/*.conf
!include ./MAIL.conf

# A config file can also tried to be included without g

Re: [Dovecot] per-mailbox message limits

[Alan Brown]

On 29/05/12 16:56, dovecot-requ...@dovecot.org wrote:

>> > This is something Timo hacked up for me a few years ago and I realised 
>> > should be on the list in case anyone else wants them.
>> > 

> Perhaps you should try mdbox some day?

If and when I get permission to implement it.

(It was hard enough getting permission to move to Maildir format. My
manager is a confirmed VMS-head who regards all Unix formats as
unreliable and the newer they are the less he trusts them.)

[Dovecot] per-mailbox message limits

[Alan Brown]

This is something Timo hacked up for me a few years ago and I realised 
should be on the list in case anyone else wants them.


The following patches will limit maildir folders to 4000 messages on 
Dovecot 2.0.* and 2.1.*


The Specfile patch is against the Cityfan Redhat EL5 SRPM but is likely 
to work on most build platforms


Changing the message limit requires a recompile. It's brutal and ugly 
but it prevents users overloading FSes that can't cope with huge numbers 
of files (EG: GFS2)




--- dovecot.spec-2.1.6  2012-05-23 11:39:11.0 +0100
+++ dovecot.spec-2.1.6-mssl 2012-05-24 18:46:37.0 +0100
@@ -32,7 +32,7 @@
 # Remember to bump dovecot_release and sieve_release with every release
 %global dovecot_epoch 1
 %global dovecot_version 2.1.6
-%global dovecot_release 
%{?prerel:0.}2.0%{?prerel:.%{prerel}}.cf.%{__distinit}%{__distvers}
+%global dovecot_release 
%{?prerel:0.}2.0%{?prerel:.%{prerel}}.cf.MSSL.%{__distinit}%{__distvers}
 %global sieve_epoch 3
 %global sieve_version 0.3.0
 %global sieve_release 6.1.cf.%{__distinit}%{__distvers}
@@ -134,6 +134,7 @@
 Patch502:  dovecot-2.0-mysqlconfig.patch
 Patch503:  dovecot-2.1-pigeonhole-bzip2.patch
 Patch504:  dovecot-2.1.rc6-fallocate64.patch
+Patch505:  ../../dovecot-mssl-2.1.6.mail-storage.c.DIFF
 BuildRequires: gcc-c++
 BuildRequires: openssl-devel, pam-devel, zlib-devel, bzip2-devel
 BuildRequires: openldap-devel, krb5-devel >= 1.3
@@ -282,6 +283,11 @@
 # http://bugzilla.redhat.com/500487
 %patch504 -p1
 
+
+# add mssl maildir patch
+%patch505 -F3 -p1
+
+
 # PAM Configuration:
 # Default PAM configuration file uses password-auth common config;
 # revert to system-auth if password-auth is not available
--- dovecot-2.1.6/src/lib-storage/mail-storage.c.orig   2012-05-23 
11:45:55.0 +0100
+++ dovecot-2.1.6/src/lib-storage/mail-storage.c2012-05-23 
12:11:28.0 +0100
@@ -22,12 +22,13 @@
 #include "mailbox-guid-cache.h"
 
 #include 
 #include 
 
 #define MAILBOX_DELETE_RETRY_SECS (60*5)
+#define MAX_MSGS_PER_MAILBOX 4000
 
 extern struct mail_search_register *mail_search_register_imap;
 extern struct mail_search_register *mail_search_register_human;
 
 struct mail_storage_module_register mail_storage_module_register = { 0 };
 struct mail_module_register mail_module_register = { 0 };
@@ -1620,22 +1621,28 @@
ctx->dest_mail = mail;
 }
 
 int mailbox_save_begin(struct mail_save_context **ctx, struct istream *input)
 {
struct mailbox *box = (*ctx)->transaction->box;
+   struct mailbox_status status;
int ret;
 
if (mail_index_is_deleted(box->index)) {
mailbox_set_deleted(box);
return -1;
}
 
if (!(*ctx)->copying_via_save)
(*ctx)->saving = TRUE;
-   if (box->v.save_begin == NULL) {
+   mailbox_get_status(box, STATUS_MESSAGES, &status);
+   if (status.messages >= MAX_MSGS_PER_MAILBOX) {
+   mail_storage_set_error(box->storage, MAIL_ERROR_NOSPACE,
+   "Mailbox full: Too many messages");
+   ret = -1;
+   } else if (box->v.save_begin == NULL) {
mail_storage_set_error(box->storage, MAIL_ERROR_NOTPOSSIBLE,
   "Saving messages not supported");
ret = -1;
} else {
ret = box->v.save_begin(*ctx, input);
}
@@ -1685,23 +1692,31 @@
 
 int mailbox_copy(struct mail_save_context **_ctx, struct mail *mail)
 {
struct mail_save_context *ctx = *_ctx;
struct mailbox *box = ctx->transaction->box;
struct mail_keywords *keywords = ctx->keywords;
+   struct mailbox_status status;
int ret;
 
*_ctx = NULL;
 
if (mail_index_is_deleted(box->index)) {
mailbox_set_deleted(box);
mailbox_save_cancel(&ctx);
return -1;
}
 
-   ret = ctx->transaction->box->v.copy(ctx, mail);
+   mailbox_get_status(box, STATUS_MESSAGES, &status);
+   if (status.messages >= MAX_MSGS_PER_MAILBOX) {
+   mail_storage_set_error(box->storage, MAIL_ERROR_NOSPACE,
+   "Mailbox full: Too many messages");
+   ret = -1;
+   } else {
+   ret = ctx->transaction->box->v.copy(ctx, mail);
+   }
if (keywords != NULL)
mailbox_keywords_unref(&keywords);
return ret;
 }
 
 int mailbox_save_using_mail(struct mail_save_context **ctx, struct mail *mail)
--- dovecot-2.0.7/src/lib-storage/mail-storage.c.ORIG   2010-11-11 
19:46:14.0 +
+++ dovecot-2.0.7/src/lib-storage/mail-storage.c2010-11-11 
19:46:19.0 +
@@ -15,20 +15,21 @@
 #include "mail-storage-settings.h"
 #include "mail-namespace.h"
 #include "mail-search.h"
 #include "mail-search-register.h"
 #include "mailbox-search-result-private.h"
 
 #include 
 #include 
 
 #define MAILBOX_DELETE_RETRY_SECS (6

[Dovecot] Dovecot 2.1 mbox + maildir

[Alan Brown]

What syntax is needed to make this work?

The 2.0 wiki recomendations don't work - I can see the inboxes or the 
folders but not both at once and there are lots of error messages about 
prefix clashes if I simply use the existing 2.0.20 conf file on 2.1.6


The layout I have is:

Inboxes in mbox format - /var/spool/mail/%u

Folders in maildir format - /var/spool/imap/%u/Maildir/

Control and index folders are under separate trees (/var/spool/control & 
/var/spool/indexes)


There's also a mbox area mainly for moving oversized inboxes 
(/var/spool/imap/%u/mbox/ - prefixed as OLD)


Presumably I'm not the only one who's had problems of this type?

Thanks in advance for any help
Alan

[Dovecot] OS/Distro wars

[Alan Brown]

DO NOT FEED THE TROLLS

Re: [Dovecot] IMAP SPECIAL-USE extension

[Alan Brown]

> Date: Wed, 07 Dec 2011 00:49:49 +0200
> From: Timo Sirainen 
> Subject: Re: [Dovecot] IMAP SPECIAL-USE extension
>
> I did this: http://hg.dovecot.org/dovecot-2.1/rev/9b9a206395f7

Just to add to the confusion:

Pine/Alpine uses "sent-mail"

:(

Re: [Dovecot] dovecot Digest, Vol 101, Issue 26

[Alan Brown]

From: Timo Sirainen 
Subject: Re: [Dovecot] general advice sought
Message-ID: <1315830847.7326.48.camel@hurina>
Content-Type: text/plain; charset="ISO-8859-15"

On Mon, 2011-09-12 at 13:11 +0100, Alan Brown wrote:

I'd like to hear the thoughts of list members on which type of storage
method seems "best" for inboxes and for folders.

The filesystem is GFS2 and for various reasons I can't change it.

..
If I migrate to other formats (eg mdbox), then it needs to be able to be 
done on the fly. (Taking the mailservers down for a day won't go down 
well, even an hour raises howls).


Have you had any trouble with Dovecot's index files in your current
setup? Any errors at all? 


Zero. The issues which have arisen are:

1: Really slow access to messages in large folders (GFS2 issue)

2: Users trashing their inbox then demanding we drop everything to 
restore "all my important mail"



If not, I'd think mdbox will work fine. You
can do it incrementally per-user (and you really should try it first
with only a few users). http://wiki2.dovecot.org/Tools/Dsync explains
how to do it on the fly.


I thought so too, just wanted to hear opinions on doing it vs other 
approaches. :)

[Dovecot] general advice sought

[Alan Brown]

I'd like to hear the thoughts of list members on which type of storage
method seems "best" for inboxes and for folders.

The filesystem is GFS2 and for various reasons I can't change it.

Inboxes - currently Mbox format. Some users have upwards of 5000
messages in there (the largest is about 18k entries) and thanks to 
attachments some inboxes are 40-200Mb with a few sitting at 2-3Gb. 
(Suicidal, I know but enforcing rules is politik-laden and getting 
academics to behave sensibly is like herding cats)


Folders: Mdir format. Thanks to GFS2 filesystem limitations(*) I'm 
currently enforcing a limit of 4000 messages/folder. It'd be nice to 
have a "better" way of storing these which doesn't drive up backup loads

tremendously.

The mail spool has around 6 million files in the folders, covering about
400Gb.

GFS2 filesystem limitations are painful.

While there are no hard limits, It really doesn't cope well with a lot
of files in any given directory thanks to limitations in the way that
cluster locking is propagated, There are marked slowdowns about 100
files, this becomes awful above 1000 and effectively unusable above
about 8000 entries as the entire IO system gets halted momentarily when
such directories are opened and the directory may take up to 5 minutes
to return a "ls" result.

Some users had upwards of 80k files in their mail folders (~40k
messages) and accessing these brought the entire system to a halt.

If I migrate to other formats (eg mdbox), then it needs to be able to be 
done on the fly. (Taking the mailservers down for a day won't go down 
well, even an hour raises howls).


Any suggestions?

Re: [Dovecot] IO rate quotas?

[Alan Brown]

From: Timo Sirainen 
Subject: Re: [Dovecot] IO rate quotas?


So a single process that is reading files fast enough from disk can
cause disk IO to spike in a way that makes all other processes wait for
available disk IO?


It's not a single process.

Thunderbird's newest defaults are to syncronize every folder and it will 
open one connectrion per folder in order to do that.


Appropriate disk/swappiness/readahead/cache tuning is required - along 
with enough ram to cope with several users trying this at once.


The underlaying filesystem is also important. as a for-instance, this 
kind of load with maildir format will effectively _kill_ a GFS box.



Our fix was to install systemwide defaults on managed systems to not 
synchronize (not needed, the home directories are on network storage 
anyway) and to ask people to check/unset this on laptops/etc

Re: [Dovecot] dovecot Digest, Vol 94, Issue 44

[Alan Brown]

Date: Wed, 16 Feb 2011 07:04:55 + (GMT)
From: Spyros Tsiolis 
Subject: [Dovecot] [Off Topic] - Any backup solutions ?
To: Dovecot 
Message-ID: <627531.34045...@web27203.mail.ukl.yahoo.com>
Content-Type: text/plain; charset=utf-8




OK, I know this is not for this list; However, reading all these
mails on the topic of filesystems, I thought that it's a good
place to ask about backup software on linux.



Bacula works very well and scales to both small and large setups 
extremely easily.


www.bacula.org

Re: [Dovecot] dovecot Digest, Vol 93, Issue 41

[Alan Brown]

> From: Stan Hoeppner 
> Subject: Re: [Dovecot] SSD drives are really fast running Dovecot
>
>
> Yes.  Go with a cluster filesystem such as OCFS or GFS2 and an 
inexpensive SAN
> storage unit that supports mixed SSD and spinning storage such as the 
Nexsan

> SATABoy with 2GB cache:  http://www.nexsan.com/sataboy.php

I can't speak for OCFS2, but after several years' experience with the 
filesystem I strongly recommend NOT using GFS/GFS2. Its locking model is 
incredibly slow (500 locks/second on a filesystem mounted with quotas 
enabled and noatime) and results in dire performance - plus there's a 
known crash vulnerability if files are repeatedly renamed in large 
directories (this bites us regularly...)


GFS2 isn't an "enterprise" filesystem by any stretch of the imagination, 
despite what a number of enthusiastic salespeople might try to convince 
you of. We're lucky to keep the GFS servers up for more than a week at a 
time.

Re: [Dovecot] dovecot Digest, Vol 91, Issue 18

[Alan Brown]

From: Timo Sirainen 
Subject: Re: [Dovecot] emails getting mangled when dragging from
Exchange account to IMAP shared folders



On Wed, 2010-11-03 at 16:18 -0700, Scott Goodwin wrote:

FYI, I got rawlog working and it shows the same break in the raw logs as in
the broken headers.
Below is a snippet from the rawlog (names and other identifiers redacted).
 The offending sequence is always in the References headers section, and you
can see the line breaks there that show this.  So it sounds like this can't
be an issue with Dovecot, am I right?


Yeah, sounds like Outlook breaks with huge headers. That's one huge
References header you have.


Summarising mail standards (WRT headers)

There is a limit of 4096 characters per line.

If Outlook is breaking at less than that it's a bug.

If the References: line is longer than that then it should have been 
truncated by the sending MUA - there's no provision for multiline 
References headers (which are a non-standardised import from Usenet anyway).

Re: [Dovecot] 2.0.3 Bug - hanging issue

[Alan Brown]

> From: Timo Sirainen 

Subject: Re: [Dovecot] 2.0.3 Bug - hanging issue

On 25.10.2010, at 14.13, Alan Brown wrote:


>  Oct 25 12:36:34 msslat dovecot: imap(keh2): Error: user keh2: Initialization 
failed: Namespace '#mbox/': mkdir(/stage/mail/imap1/keh2/mail) failed: Permission 
denied (euid=2291(keh2) egid=101(luci) missing +w perm: /stage/mail/imap1, euid is 
not dir owner)
>  
>  Because the parent directory was rwxr-xr-x root.root

Right..


>  The problem is that for some reason dovecot then wedged and a lot of other 
stuff piled up behind it, causing people to complain their mail wasn't coming 
through. Load average climbed to over 100.
>  
>  Creating the user directories freed things up...

So the problem is that if there's a misconfiguration, load goes up? I don't 
really know if there's much to do about this. I guess it could wait a second or 
so before disconnecting client to avoid it reconnecting back too fast..



The client wasn't reconnecting as far as I can tell.

It looked to me like the server process wedged after this error - we 
only got the error message logged once.


The problem is that after it occurred no new clients could complete 
connections, but existing clients worked ok. Incoming connections were 
accepted, but then nothing would happen.


As far as I can tell the parent would fork new child processes but they 
couldn't complete initialisation. Mail clients simply complained of a 
timeout during connection. This was affecting all users.



With regard to error disconnections - yes a delay is a good idea.

Re: [Dovecot] 2.0.3 Bug - hanging issue

[Alan Brown]

> From: Charles Marcus  > On 2010-10-25 9:13 
AM, Alan Brown wrote:



>  >  We had this this morning:
 2.0 is still getting a good number of bug fixes so if you're going to
 use it you should be prepared to upgrade to the latest and see if the
 problem is already fixed before reporting a possible bug.


I have been, but I've seen no similar reports and it may still be in 2.0.6.

I will update to that version during a planned maintenance window tomorrow.

Re: [Dovecot] dovecot Digest, Vol 90, Issue 102

[Alan Brown]

> From: "Roderick A. Anderson" 
> Subject: [Dovecot] RHEL5/CentOS5 YUM repo, rpm, or spec file for 2.0?
>
> I don't remember sing any mention come across the list reference the
> Subject line and nothing shows up within the first three pages of a
> Google search.
>
> Anyone know of a YUM repo. RPM or spec file for Dovecot 2.0 and friends?


Try the following 2 URLs:

http://www.city-fan.org/ftp/contrib/yum-repo/

http://www.city-fan.org/ftp/contrib/mail/


ATrpms has builds too, but they're usually out of date.

[Dovecot] 2.0.3 Bug - hanging issue

[Alan Brown]

We had this this morning:

Oct 25 12:36:34 msslat dovecot: imap(keh2): Error: user keh2: 
Initialization failed: Namespace '#mbox/': 
mkdir(/stage/mail/imap1/keh2/mail) failed: Permission denied 
(euid=2291(keh2) egid=101(luci) missing +w perm: /stage/mail/imap1, euid 
is not dir owner)
Oct 25 12:36:34 msslat dovecot: imap(keh2): Error: Invalid user 
settings. Refer to server log for more information.
Oct 25 12:36:35 msslat dovecot: imap-login: Login: user=, 
method=PLAIN, rip=128.40.73.160, lip=128.40.73.13, mpid=26736
Oct 25 12:36:35 msslat dovecot: imap(keh2): Error: user keh2: 
Initialization failed: Namespace '#mbox/': 
mkdir(/stage/mail/imap1/keh2/mail) failed: Permission denied 
(euid=2291(keh2) egid=101(luci) missing +w perm: /stage/mail/imap1, euid 
is not dir owner)
Oct 25 12:36:35 msslat dovecot: imap(keh2): Error: Invalid user 
settings. Refer to server log for more information.


Because the parent directory was rwxr-xr-x root.root

The problem is that for some reason dovecot then wedged and a lot of 
other stuff piled up behind it, causing people to complain their mail 
wasn't coming through. Load average climbed to over 100.


Creating the user directories freed things up...

Re: [Dovecot] Limit access to dovecot by domains?

[Alan Brown]

> Date: Fri, 15 Oct 2010 15:09:57 +1100
> From: Jobst Schmalenbach 
> Subject: Re: [Dovecot] Limit access to dovecot by domains?
> To: Timo Sirainen 
> Cc: dovecot@dovecot.org
> Message-ID: <20101015040957.ga3...@senna.barrett.com.au>
> Content-Type: text/plain; charset=us-ascii
>
> On Thu, Oct 14, 2010 at 03:31:23PM +0100, Timo Sirainen (t...@iki.fi) 
wrote:

> > > On Wed, 2010-10-13 at 18:08 +1100, Jobst Schmalenbach wrote:
> > >
> >> > > Maybe I could include a script that would check the reverse 
DNS record

> >> > > of a connected IP and then I could filter?
> > >
> > > Wonder if tcpwrappers would work? You could use that with Dovecot 
v2.0.

>
> I have read a few things about this, it looks like its not so good to 
do it this way,
> besides having proper written daemons running from (x)inted is a 
system overhead.


Huh? What are you talking about?

If dovecot has tcpwrapper support and is compiled -DTCPWRAP then it can 
run as a standalone daemon and will consult the hosts.allow/deny files, 
no need for inetd of any type. man 3 hosts_access


man 5 hosts_access for details on tuning. Tcpwrapper tuning is far more 
powerful than people realise.

Re: [Dovecot] Upgraded dovecot - now it insists on imap TLS

[Alan Brown]

Marcus Rueckert wrote:

> > What am I missing?
> dovecot -n output would help.

It would, but I found/fixed the problem already.

I had to set the following:

protocol imap {
disable_plaintext_auth=no
}

The default has changed from "no" to "yes" between 1.1.20 and 2.0.1, but 
the conversion routine didn't pick it up (the wiki Documentation 
suggests the default is still "no". Perhaps this should be updated?)


This still requires TLS on port 993, so there's no additional hole opened.

Ironically, we published plans some time back to disable local-access 
plaintext imap logins in October and have been warning users to change 
their settings but 90% of them hadn't yet done so.

[Dovecot] Upgraded dovecot - now it insists on imap TLS

[Alan Brown]

This is definitely an operator error issue, but I'm under a bit of pressure.

I updated our 1.1.20 installation to 2.0.1 over the weekend and 
discovered this morning that it's insisting on TLS imap on port 143 as 
well as port 993


What am I missing?

Thanks in advance
Alan

Re: [Dovecot] dovecot Digest, Vol 89, Issue 27

[Alan Brown]

> > Timo Sirainen  Wrote:
> >
> >  > On Mon, 2010-09-06 at 14:26 +0100, Alan Brown wrote:
> >  > > Is there any way of enforcing a Maildir per-folder message 
limit in

> >  > > Dovecot?
> >
> >  > No.
> >
> > Would you consider it as a feature request?

> What if you just create a script that automatically archives older 
mails to /archive/-mm mailboxes when number of messages in a 
mailbox increases too high for too long? Or something like that.


I'm looking at doing that anyway, however it would be much easier to 
train the users if the server says "folder full"

Re: [Dovecot] dovecot Digest, Vol 89, Issue 25

[Alan Brown]

Timo Sirainen  Wrote:

> On Mon, 2010-09-06 at 14:26 +0100, Alan Brown wrote:
> > Is there any way of enforcing a Maildir per-folder message limit in
> > Dovecot?

> No.

Would you consider it as a feature request?

> > We're finding major performance hits once a threshold of 
files/directory

> > is exceeded. This applies across all common filesystems with the
> > threshold differing depending on the FS. GFS2 pretty much _STOPS_ 
for 10

> > minutes when users put 65k+ messages in one folder...

> mdbox would probably work nicely.  :)

Yes, it probably would, however I have several layers of resistance to 
get past in order to implement that (ie, up to 2 years to get approval 
to switch to dbox)


> > I'm currently running 1.1.20 but looking to update (RHEL5 cluster
> > installation)

> With v1.2+ you could also try maildir_very_dirty_syncs=yes

That won't help much for initial directory opens, which is where most of 
the pain is. Past that point, the index of ~/Maildir/.folder/cur is 
mostly cached until you hit about 16,000 messages(files) on GFS2 and 
some people are insisting on 32k+ files in one folder with the odd few 
trying 65k or more (I'm surprised their clients don't break).


For info sake: we have Dovecot running on a pair of dedicated dual E5530 
(quad-core X86 1300MHZ FSB cpu) servers (RH cluster+GFS2) with 24Gb RAM 
apiece and ~2TB of 8Gb/s san-attached storage available.


There are only ~250 accounts but there's around 600Gb in the maildir 
folder areas and another 300Gb in a separate dedicated inbox area - and 
I believe there's another 1TB of stuff in client local folders we're 
trying to pull back to the servers in order to simplify backups.


Even with this kind of horsepower available, over-large folders can 
render the setup completely unresponsive for minutes at a time. I know 
this is a case of "work smarter, not harder", but I'm facing major 
delays until we're able to change the on-disk storage format.


Alan

Re: [Dovecot] dovecot Digest, Vol 89, Issue 26

[Alan Brown]

Timo Sirainen  Wrote:

>On Mon, 2010-09-06 at 12:37 +0100, William Blunn wrote:
> > Whilst documenting LAYOUT=maildir++ under dbox, that got me thinking:
> >
> > Can we specify :LAYOUT=maildir++ with mbox?

> Yes, ever since LAYOUT was added.

> > If I have it right, this should then remove the problem of not being
> > able to have messages and mail subfolders in the same mail folder.

> Yes. Also it might be possible to use DIRNAME with LAYOUT=fs to solve
> that problem, but I'm less sure about that. I anyway doubt most mbox
> users want to switch to a whole new layout.

Mbox users might not, but Maildir or dbox users might want to, 
especially if users have a lot of folders (One of mine has ~2500)


Directories definitely get slower to list with more files/subdirectories 
in them and there are knee points beyond which they get substantially 
slower (Where the knee is depends on the type of FS but it's usually a 
power of 2)

[Dovecot] Quotas - per folder message limits?

[Alan Brown]

Is there any way of enforcing a Maildir per-folder message limit in 
Dovecot?


We're finding major performance hits once a threshold of files/directory 
is exceeded. This applies across all common filesystems with the 
threshold differing depending on the FS. GFS2 pretty much _STOPS_ for 10 
minutes when users put 65k+ messages in one folder...


I'm currently running 1.1.20 but looking to update (RHEL5 cluster 
installation)