Re: [Dovecot] IO rate quotas?
Wow, thanks everyone for the replies. Already using XFS as a filesystem with noatime. Switching to faster disks/raids/etc is not easy, we rent hosted server in Germany, to make real changes i'd have to get my own server etc. I'll play around with disk schedulers and report back. Thanks again for all the help. On Mon, Apr 11, 2011 at 12:58 PM, Alan Brown wrote: > > From: Timo Sirainen > Subject: Re: [Dovecot] IO rate quotas? > >> So a single process that is reading files fast enough from disk can >> cause disk IO to spike in a way that makes all other processes wait for >> available disk IO? > > It's not a single process. > > Thunderbird's newest defaults are to syncronize every folder and it will > open one connectrion per folder in order to do that. > > Appropriate disk/swappiness/readahead/cache tuning is required - along with > enough ram to cope with several users trying this at once. > > The underlaying filesystem is also important. as a for-instance, this kind > of load with maildir format will effectively _kill_ a GFS box. > > > Our fix was to install systemwide defaults on managed systems to not > synchronize (not needed, the home directories are on network storage anyway) > and to ask people to check/unset this on laptops/etc > > > >
[Dovecot] IO rate quotas?
Hi
Okay so we've been having this issue since forever and I figured why
the heck not ask it here since it's mostly related to dovecot. The
thing is, we have a huge amount of public folders (at the moment,
around 1100). Now, with dovecot indexing and caching we're mostly
okay, also being careful with things like Thunderbird 3's default
'Download everything' option ("Keep messages for this account on this
computer") and such. However once in a while, someone goes rogue, we
install a new version of thunderbird, someone accidentally sets up an
email account in thunderbird mistakingly leaving the download
everything option on. This causes high IO on our server, and a single
user can quickly more or less kill the whole server in mere minutes,
load average quickly spiking to 30-40-50 and everything becomes
sloow (obviously).
Is there any way I can limit the amount of IO a certain user can use?
Thanks and best regards,
KT
P.S.
my dovecot -n is:
# 2.0.9: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-2-xen-amd64 x86_64 Debian 5.0.8
auth_mechanisms = plain login
disable_plaintext_auth = no
listen = *
mail_location = maildir:~/Maildir
mail_plugins = acl
maildir_very_dirty_syncs = yes
namespace {
inbox = yes
location =
prefix =
separator = .
type = private
}
namespace {
location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
prefix = shared.
separator = .
subscriptions = no
type = public
}
namespace {
location = maildir:/home/_shared/sys:INDEX=~/Maildir/sys
prefix = sys.
separator = .
subscriptions = no
type = public
}
passdb {
driver = pam
}
plugin {
acl = vfile
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename flag_change append
mail_log_fields = uid box msgid from subject size vsize flags
mail_log_group_events = no
}
protocols = imap pop3
service imap {
executable = imap
}
ssl_cert =
Re: [Dovecot] 2.0.7: emails read emails become unread
Yes I have dovecot-shared files in their appropriate spaces.
I'll turn on debug mode tomorrow and see if I get any errors related to indexes.
KTamas
On Tue, Nov 16, 2010 at 6:33 PM, Timo Sirainen wrote:
> On Tue, 2010-11-16 at 12:32 +0100, Kádár Tamás (KTamas) wrote:
>
>> My users are reporting that in public folders, emails that were past
>> read suddenly got unread, sometimes even after marking them as read
>> again.
>
> Any errors in Dovecot's logs?
>
>> Didn't have such problems with 2.0.6 I think.
>
> I don't think I've changed anything related to this..
>
>> namespace {
>> location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
>
> You do have dovecot-shared files for all those folders
> in /home/_shared/projects/, right?
>
> Anyway, losing seen flags for shared mailboxes basically means that
> Dovecot ignored/lost the index files in ~/Maildir/_shared/ for some
> reason.
>
>
>
[Dovecot] 2.0.7: emails read emails become unread
Hi
My users are reporting that in public folders, emails that were past
read suddenly got unread, sometimes even after marking them as read
again. Didn't have such problems with 2.0.6 I think.
My dovecot -n:
# 2.0.7: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-2-xen-amd64 x86_64 Debian 5.0.6
auth_mechanisms = plain login
disable_plaintext_auth = no
listen = *
mail_location = maildir:~/Maildir
mail_plugins = acl
maildir_very_dirty_syncs = yes
namespace {
inbox = yes
location =
prefix =
separator = .
type = private
}
namespace {
location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
prefix = shared.
separator = .
subscriptions = no
type = public
}
passdb {
driver = pam
}
plugin {
acl = vfile
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename flag_change append
mail_log_fields = uid box msgid from subject size vsize flags
mail_log_group_events = yes
}
protocols = imap pop3
service imap {
executable = imap
}
ssl_cert =
Re: [Dovecot] Weird glitches with public folders - Dovecot not using per-user INDEX?
I've put the dovecot-shared files in place, restarted dovecot, but it's still not working :( Are there any extra steps I have to perform? Best regards Tamas On Mon, Oct 4, 2010 at 4:48 PM, Timo Sirainen wrote: > On Mon, 2010-10-04 at 10:43 -0400, Charles Marcus wrote: >> > It's not currently copied from parent folder, but rather the namespace >> > root directory. Se create /home/_shared/projects/dovecot-shared >> >> I can see an argument for having it first look in the parent folder and >> copy one from there if it exists, if not, then look in the namespace >> root - any chance of this in the future? > > Yeah, I guess that would be better. Added it to v2.1 TODO list. > > >
Re: [Dovecot] Weird glitches with public folders - Dovecot not using per-user INDEX?
Further investigation shows that the users DOES have indexes and
caches for these problematic folders and dovecot does seem to be using
them:
-rw-rw-rw-1 someuser mailusers 9496 Oct 1 20:09 dovecot.index
-rw-rw-rw-1 someuser mailusers 310272 Oct 4 12:16 dovecot.index.cache
-rw-rw-rw-1 someuser mailusers 30680 Oct 4 12:22 dovecot.index.log
(except, well, instead, changes still happen in the public directory
itself for the \Seen flags.)
2010/10/2 Kádár Tamás (KTamas) :
> sorry. Using the latest version (2.0.5) on Debian Lenny. Here's the
> full output of dovecot -n:
>
> mail:~# dovecot -n
> # 2.0.5: /usr/local/etc/dovecot/dovecot.conf
> # OS: Linux 2.6.26-2-xen-amd64 x86_64 Debian 5.0.6
> auth_mechanisms = plain login
> disable_plaintext_auth = no
> listen = *
> mail_debug = yes
> mail_location = maildir:~/Maildir
> mail_plugins = acl
> maildir_very_dirty_syncs = yes
> namespace {
> inbox = yes
> location =
> prefix =
> separator = .
> type = private
> }
> namespace {
> location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
> prefix = shared.
> separator = .
> subscriptions = no
> type = public
> }
> passdb {
> driver = pam
> }
> plugin {
> acl = vfile
> mail_log_events = delete undelete expunge copy mailbox_delete
> mailbox_rename flag_change append
> mail_log_fields = uid box msgid from subject size vsize flags
> mail_log_group_events = yes
> }
> protocols = imap pop3
> service imap {
> executable = imap
> }
> ssl_cert = ssl_key = userdb {
> driver = passwd
> }
> verbose_proctitle = yes
> protocol imap {
> mail_max_userip_connections = 10
> mail_plugins = acl imap_acl
> }
>
> Best regards,
> Tamas
> On Sat, Oct 2, 2010 at 8:45 PM, Charles Marcus
> wrote:
>> At a bare minimum, it is always a good idea to at least say what version
>> you're using...
>>
>> On 2010-10-02 10:29 AM, Kádár Tamás (KTamas) wrote:
>>> Hi
>>>
>>> I have this public folder:
>>>
>>> namespace {
>>> location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
>>> prefix = shared.
>>> separator = .
>>> subscriptions = no
>>> type = public
>>> }
>>>
>>> Access is controlled by ACLs, everyone who has access is 'lrwstipek'
>>> except for some people who also have 'xa' privileges.
>>>
>>> As you can see it is set to put its index to the user's Maildir so
>>> they can have per-user \Seen flags. And that is the case -- most of
>>> the time anyway. But for some folders, and I can't figure out why
>>> those, they just don't work: if someone sets a message read it
>>> immediately becomes read for everyone else, too, and vice versa if
>>> someone marks a message unread, it becomes unread everywhere. Further
>>> investigation shows that in these problematic folders, indeed all
>>> users directly manipulate the flag of the email (i.e. X marks email Z
>>> read, ",S" appears immediatelly in the filename).
>>>
>>> Anyone knows what I'm doing wrong? I can't seem to find anything
>>> different with these problematic folders, ACLs look right etc.
>>
>> --
>>
>> Best regards,
>>
>> Charles
>>
>
Re: [Dovecot] Weird glitches with public folders - Dovecot not using per-user INDEX?
Thanks! Tamas On Mon, Oct 4, 2010 at 4:15 PM, Timo Sirainen wrote: > On Mon, 2010-10-04 at 16:08 +0200, Kádár Tamás (KTamas) wrote: > >> However, that raises a question: how can I make the creation of >> dovecot-shared default when I create a public subfolder? I've just >> tried creating a new folder and dovecot-shared doesn't get created, >> even though its parent folder has it... > > It's not currently copied from parent folder, but rather the namespace > root directory. Se create /home/_shared/projects/dovecot-shared > >
Re: [Dovecot] Blackberries
How can one 'nudge' the account back into push from polling? I think that's the case with my blackberry. Thanks, Tamas On Mon, Oct 4, 2010 at 11:30 AM, interfaSys sàrl wrote: > On 04/10/2010 10:00, dovecot-requ...@dovecot.org wrote: >> Anyone has had differing experiences with different blackberries? >> >> Someone with Blackberry Gold/Bold get only 1 e-mail >> yet regualr blackberries get multiple copies of the same e-mials. >> >> Is this a blackberry bug or is their something I have to set in Dovecot? > > I've only experienced this with Thunderbird...maybe the "regular" > berries have not configured their BIS accounts properly? > > The only problem I'm aware of with Blackberry devices is that if the BIS > servers can't login because Dovecot is being upgraded (version > mismatch), then they'll go into polling mode (no more instant emails). > > Cheers, > > Olivier >
Re: [Dovecot] Weird glitches with public folders - Dovecot not using per-user INDEX?
That's it! Silly me, didn't think of that one. Indeed, dovecot-shared is missing from the folder. However, that raises a question: how can I make the creation of dovecot-shared default when I create a public subfolder? I've just tried creating a new folder and dovecot-shared doesn't get created, even though its parent folder has it... It's really important because folders get created every day around here and they really need to inherit everything, including this from its parent. Thanks and best regards, Tamas On Mon, Oct 4, 2010 at 3:55 PM, Timo Sirainen wrote: > On Sat, 2010-10-02 at 16:29 +0200, Kádár Tamás (KTamas) wrote: > >> As you can see it is set to put its index to the user's Maildir so >> they can have per-user \Seen flags. And that is the case -- most of >> the time anyway. But for some folders, and I can't figure out why >> those, they just don't work: > > Do those folders have dovecot-shared file? Per-user \Seen flag is used > only if that exists. > > >
Re: [Dovecot] Blackberries
Thanks. I've deleted and re-created the account, hopefully it fixes it. Best regards, Tamas On Mon, Oct 4, 2010 at 12:13 PM, interfaSys sàrl wrote: > To check if you're in polling mode, do a "doveadm who" and if you don't > see a BIS server connected to your account, then you don't get instant > emails anymore. > > I haven't figured out if the reason it can't switch back is because of > something on the Dovecot side or simply because BIS doesn't have a way > to switch back. > The reason being that, sometimes, a few accounts manage to connect again. > I've tried to repair mailboxes, index, etc. Nothing worked. > > On the RIM side, there doesn't seem anybody we can talk to. People on > public forums don't have a clue. Their beta program is for apps. It > costs money to open a support ticket. > > The only solution is to delete the account on the phone and to re-create > it. Annoying. > > On 04/10/2010 11:03, Kádár Tamás (KTamas) wrote: >> How can one 'nudge' the account back into push from polling? I think >> that's the case with my blackberry. >> >> Thanks, >> Tamas >> >> On Mon, Oct 4, 2010 at 11:30 AM, interfaSys sàrl >> wrote: >>> On 04/10/2010 10:00, dovecot-requ...@dovecot.org wrote: >>>> Anyone has had differing experiences with different blackberries? >>>> >>>> Someone with Blackberry Gold/Bold get only 1 e-mail >>>> yet regualr blackberries get multiple copies of the same e-mials. >>>> >>>> Is this a blackberry bug or is their something I have to set in Dovecot? >>> >>> I've only experienced this with Thunderbird...maybe the "regular" >>> berries have not configured their BIS accounts properly? >>> >>> The only problem I'm aware of with Blackberry devices is that if the BIS >>> servers can't login because Dovecot is being upgraded (version >>> mismatch), then they'll go into polling mode (no more instant emails). >>> >>> Cheers, >>> >>> Olivier >>> >
Re: [Dovecot] Weird glitches with public folders - Dovecot not using per-user INDEX?
sorry. Using the latest version (2.0.5) on Debian Lenny. Here's the
full output of dovecot -n:
mail:~# dovecot -n
# 2.0.5: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-2-xen-amd64 x86_64 Debian 5.0.6
auth_mechanisms = plain login
disable_plaintext_auth = no
listen = *
mail_debug = yes
mail_location = maildir:~/Maildir
mail_plugins = acl
maildir_very_dirty_syncs = yes
namespace {
inbox = yes
location =
prefix =
separator = .
type = private
}
namespace {
location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
prefix = shared.
separator = .
subscriptions = no
type = public
}
passdb {
driver = pam
}
plugin {
acl = vfile
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename flag_change append
mail_log_fields = uid box msgid from subject size vsize flags
mail_log_group_events = yes
}
protocols = imap pop3
service imap {
executable = imap
}
ssl_cert = wrote:
> At a bare minimum, it is always a good idea to at least say what version
> you're using...
>
> On 2010-10-02 10:29 AM, Kádár Tamás (KTamas) wrote:
>> Hi
>>
>> I have this public folder:
>>
>> namespace {
>> location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
>> prefix = shared.
>> separator = .
>> subscriptions = no
>> type = public
>> }
>>
>> Access is controlled by ACLs, everyone who has access is 'lrwstipek'
>> except for some people who also have 'xa' privileges.
>>
>> As you can see it is set to put its index to the user's Maildir so
>> they can have per-user \Seen flags. And that is the case -- most of
>> the time anyway. But for some folders, and I can't figure out why
>> those, they just don't work: if someone sets a message read it
>> immediately becomes read for everyone else, too, and vice versa if
>> someone marks a message unread, it becomes unread everywhere. Further
>> investigation shows that in these problematic folders, indeed all
>> users directly manipulate the flag of the email (i.e. X marks email Z
>> read, ",S" appears immediatelly in the filename).
>>
>> Anyone knows what I'm doing wrong? I can't seem to find anything
>> different with these problematic folders, ACLs look right etc.
>
> --
>
> Best regards,
>
> Charles
>
[Dovecot] Weird glitches with public folders - Dovecot not using per-user INDEX?
Hi
I have this public folder:
namespace {
location = maildir:/home/_shared/projects:INDEX=~/Maildir/_shared
prefix = shared.
separator = .
subscriptions = no
type = public
}
Access is controlled by ACLs, everyone who has access is 'lrwstipek'
except for some people who also have 'xa' privileges.
As you can see it is set to put its index to the user's Maildir so
they can have per-user \Seen flags. And that is the case -- most of
the time anyway. But for some folders, and I can't figure out why
those, they just don't work: if someone sets a message read it
immediately becomes read for everyone else, too, and vice versa if
someone marks a message unread, it becomes unread everywhere. Further
investigation shows that in these problematic folders, indeed all
users directly manipulate the flag of the email (i.e. X marks email Z
read, ",S" appears immediatelly in the filename).
Anyone knows what I'm doing wrong? I can't seem to find anything
different with these problematic folders, ACLs look right etc.
Thanks in advance,
Tamas
