Batching quota clone updates

[micah anderson]

Hi,

I like the quota count+clone configuration. However, I'm wondering if
the cloning process could batch its updates into chunks, or spread them
out over time?

The clone updates happen whenever the quota is updated, typically this
is fine, but when a tsunami of spam comes resulting in delivery to every
user on the system, then can be quite overwhelming for a sql server to
do an update for every user.

Thanks!

-- 
micah

Re: LDA-based vacation responses bouncing

[micah anderson via dovecot]

> Thanks for any suggestions,

Looks like my virtual_mailbox_domains setting in postfix was not doing
the proper lookup, so it thought my domain was local. I've fixed that,
so this is no longer an issue.

LDA-based vacation responses bouncing

[micah anderson via dovecot]

I've got dovecot vacataion auto-responses working via sieve, but when
the responses are sent, they are sent as if they are to be locally
delivered, which causes them to bounce for domains that I am not local
for. How can I get dovecot's sieve vacation response to check the domain
is local or not?

Here is an example of what happens:

The LDA notices it needs to send a vacation response:
2019-05-14T10:50:47.827466-07:00 brant dovecot: 
lda(u...@example.org)<14817>: sieve: 
msgid=<20190514135045.012486@localhost>: sent vacation response to 


but that response was sent via the postfix pipe through the dovecot lda
transport... because that is local, and the domain it is sending to is
not local, it bounces:

2019-05-14T10:50:47.92-07:00 brant postfix/pipe[14821]: C998381539: 
to=, relay=dovecot, delay=0.11, delays=0.04/0.01/0/0.06, 
dsn=5.1.1, status=bounced (user unknown)

Postfix is set to do this:

dovecot  unix-   n   n   -  -   pipe
  flags=DRhu user=mail argv=/usr/bin/spamc --connect-retries=1 -H -d 10.0.1.92 
-s 1024 -t 100 -u ${recipient} -e /usr/lib/dovecot/dovecot-lda -f ${sender} 
-d ${recipient}


require ["fileinto", "vacation"];
vacation
  :days 2
  :subject "Helllo"
"Hi";

Here is my doveconf -n:

# 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.4 ()
# OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.8 
auth_worker_max_count = 120
default_client_limit = 6144
default_process_limit = 256
default_vsz_limit = 512 M
dict {
  expire = mysql:/etc/dovecot/dovecot-dict-sql.conf
  quota = mysql:/etc/dovecot/dovecot-dict-sql.conf
}
disable_plaintext_auth = no
first_valid_gid = 8
first_valid_uid = 8
last_valid_gid = 8
last_valid_uid = 8
listen = *
login_greeting = howdy, ready.
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k
mail_location = mdbox:~/mdbox
mail_plugins = expire quota zlib
maildir_very_dirty_syncs = yes
namespace {
  inbox = yes
  location = 
  prefix = 
  separator = .
}
namespace {
  alias_for = 
  hidden = yes
  inbox = no
  list = no
  location = 
  prefix = INBOX.
  separator = .
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  expire = Trash
  expire2 = Trash/*
  expire3 = Spam
  expire_dict = proxy::expire
  quota = dict:Your mail quota::proxy::quota
  quota_rule = *:bytes=100663296
  quota_rule2 = Trash:bytes=+20%%
  quota_rule3 = Spam:bytes=+10%%
  quota_rule4 = INBOX.Trash:bytes=+20%%
  quota_rule5 = INBOX.Spam:bytes=+10%%
  quota_rule6 = INBOX.restored_from_backups:bytes=+500%%
  sieve = ~/.dovecot.sieve
  sieve_after = /var/lib/dovecot/sieve/after.sieve
  sieve_before = /var/lib/dovecot/sieve/default.sieve
  sieve_dir = ~/sieve
  zlib_save = gz
  zlib_save_level = 6
}
protocols = " imap pop3"
service anvil {
  unix_listener anvil {
group = dovecot
mode = 0660
user = root
  }
}
service auth {
  client_limit = 6144
  unix_listener auth-userdb {
group = mail
mode = 0600
user = mail
  }
}
service dict {
  unix_listener dict {
group = mail
mode = 0600
user = mail
  }
}
service imap-login {
  process_min_avail = 10
  service_count = 0
  vsz_limit = 512 M
}
service imap-postlogin {
  executable = script-login /usr/local/sbin/postlogin_imap
}
service imap {
  executable = imap imap-postlogin
  process_limit = 6144
}
service pop3-login {
  process_min_avail = 10
  service_count = 0
  vsz_limit = 512 M
}
service pop3-postlogin {
  executable = script-login /usr/local/sbin/postlogin_pop
}
service pop3 {
  executable = pop3 pop3-postlogin
}
service stats {
  unix_listener stats-writer {
group = mail
mode = 0660
  }
}
ssl_cert = 

Re: corrupted indexes rebuilding over and over

[micah]

"Grubert, Gordon" <gordon.grubert+li...@uni-greifswald.de> writes:

> Hi,
>
> we had the same problem after a system failure. Our problem could be
> solved by
>
> rm /PATH/USER/mdbox/storage/dovecot.map.index*
> doveadm -D -v force-resync -u USER INBOX

I thought about doing this, but mdbox documentation[0] says:

  Note that with dbox the Index files actually contain significant data
  which is held nowhere else. Index files for both single-dbox and
  multi-dbox contain message flags and keywords. For multi-dbox, the
  index file also contains the map_uids which link (via the "map index")
  to the actual message data. This data cannot be automatically
  recreated, so it is important that Index files are treated with the
  same care as message data files.

The 'map index' for mdbox is contained in this dovecot.map.index* files
and it says that it cannot be automatically recreated, so if I nuke
them, will force-resync be able to re-create them? Is force-resync not
'automatic'?

thanks,
micah

corrupted indexes rebuilding over and over

[micah]

Hi,

On saturday one of our dovecot machines had an OOPS, and we had to
powercycle it. When it came up, it did a filesystem fsck (ext4) and
fixed a couple things, but nothing in lost+found.

Now, about 50 users have a problem where their indexes are
corrupted. Dovecot tries to fix them, but for some reason it is
failing. The users are seeing their mails duplicated and they cannot
delete mails. I can see in the logs dovecot detecting this and trying to
rebuild operations during delivery and imap login, but its repeating
over and over and over. For one user, I've seen more than 4k lines in
the logs about this.

If i try to do doveadm force-resync -u username INBOX

I get this:

doveadm(nuffsaid): Warning: mdbox /n/nuffsaid/mdbox/storage: Inconsistency in 
map index (3494,32332 != 3494,383536)
doveadm(nuffsaid): Warning: fscking index file 
/srv/dovecot_indexes/n/nuffsaid/storage/dovecot.map.index
doveadm(nuffsaid): Warning: mdbox /n/nuffsaid/mdbox/storage: rebuilding indexes
doveadm(nuffsaid): Warning: fscking index file 
/srv/dovecot_indexes/n/nuffsaid/storage/dovecot.map.index
doveadm(nuffsaid): Warning: mdbox /n/nuffsaid/mdbox/storage: Inconsistency in 
map index (3494,32332 != 3494,384024)
doveadm(nuffsaid): Warning: fscking index file 
/srv/dovecot_indexes/n/nuffsaid/storage/dovecot.map.index

but then if i do it again, I get the same thing, over and over and
over (the numbers change a little).

This is dovecot 2.2.27 with mdbox format what can I do to fix this?
Its driving users mad, and they keep going over quota as their space
fills.

thanks!
micah

Benchmarking auth

[micah anderson]

Hello,

I'm attempting to use imaptest to test auths/sec on my imap server, to
simulate the number I'm getting now on a new server. Based on my
readings of dovecot stats outputs, my current machine is doing somewhere
between 11.6 and 196 logins/sec at its busiest, and I want to make sure
the new server can handle that amount.

With imaptest I attempt to simulate this by just simulating login/logout
speed, but I'm not sure I understand the results and where things are
being blocked.

For testing purposes, I set:

  mail_max_userip_connections = 6000
  mail_max_userip_connections = 6000
  auth_worker_max_count = 240

If you have some additional suggestions for what I can also tweak here
to improve this, I would like to know.

I then ran imaptest with:

imaptest clients=196 user=test pass=testpw host=127.0.0.1 port=14300 - select=0 
seed=123 secs=300

It doesn't seem like I can simulate *only* authentications/logins. I
have to pair them with logouts. It makes me wonder how I can simulate
the amount of load on my current live system on this system.

How do I read the output? I don't really understand what is being shown
here:

Logi Logo 
100% 100% 
   00 100/100 [51%]

does this mean that it tried 100 logins and 100 logouts and 51%
succeeded? What are the first two columns?
   
   77 104/121 [61%]
   66 131/139 [70%]
   33 137/148 (84 stalled >3s) [75%]
84 stalled for more than 3 seconds, how does this relate to the other
values presented?

   55 155/163 (79 stalled >3s) [83%]
   11 166/166 (107 stalled >3s) [84%]
   22 168/172 (128 stalled >3s) [87%]
   65 179/190 (140 stalled >3s) [96%]
  22   22 182/196 (136 stalled >3s)
   43 195/196 (133 stalled >3s)
5845  129 ms/cmd avg

I really wonder if these first two columns are ms/cmds?

It then continues:

Logi Logo 
100% 100% 
  10   11 195/196 (131 stalled >3s)
   78 194/196 (150 stalled >3s)
   33 194/196 (171 stalled >3s)
  109 192/196 (166 stalled >3s)
  22   22 182/196 (156 stalled >3s)
   89 193/196 (153 stalled >3s)
 - 2 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 7 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 8 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 9 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 11 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 12 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 14 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 22 stalled for 16 secs in command: 1 LOGIN "test" "testpw"
 - 25 stalled for 16 secs in command: 1 LOGIN "test" "testpw"

... it starts to build up like this, until I hit control-c twice:

 - 100 stalled for 17 secs in command: 1 LOGIN "test" "testpw"
^CInfo: Received second SIGINT - stopping immediately
11347  118 ms/cmd avg

Totals:
Logi Logo 
100% 100% 
 130  134 

What are these totals?

Thanks!
micah

making a plugin encrypt index data

[micah anderson]

Hello,

I'm encrypting mail on disk using a plugin[0], but the index files are
not encrypted (specifically the dovecot.index.cache can be read).

I want to do is encrypt index on disk, so I'm looking for how a plugin
can achieve that by hooking into the right locations. Is that easily
possible in a plugin?

I can turn off those indexes by passing INDEX=MEMORY, but that isn't
possible if I use sdbox/mdbox.

thanks for any suggestions!
micah

0. https://0xacab.org/riseuplabs/tofu-scrambler

Re: overview zlib efficiency?

[micah]

Andrew McGlashan  writes:

> On 16/03/2016 2:01 AM, Götz Reinicke - IT Koordinator wrote:
>> Hi,
>> 
>> may be someone has already done that: Do you have a script(?) tool which
>> shows the efficiency of the mail compression if zlib is used?
>> 
>> Something that shows the uncompressed size vrs. the compressed.
>
> Remember one thing; emails are stored in plain text, the same text that
> they are normally transmitted b/w servers.

Emails are not stored in plaintext if you are using dbox/mdbox.

RE: Dovecot Bulletin

[micah]

Using doveadm copy is an interesting solution because of the
hardlinking. I wonder how fast it is, because almost every time I use
the -A flag, the iterations over 100k users takes a long time.

We email the 'bulletin' to all of our users, everyone gets a copy, that
way an admin doesn't need to do it, but it does mean that we duplicate
the data quite a lot.

Kevin Kershner  writes:

> It also got the bulletin out to new users without admin intervention.
>
> Sent from Mobile
> 
> From: Doug Hardie
> Sent: ‎2/‎22/‎2016 4:02
> To: Dovecot Mailing List; Timo 
> Sirainen
> Cc: Kevin Kershner
> Subject: Re: Dovecot Bulletin
>
>
>> On 20 February 2016, at 18:14, Timo Sirainen  wrote:
>>
>> On 21 Feb 2016, at 02:50, Kevin Kershner  wrote:
>>>
>>> I'd like to revisit and old post if I may, will/does Dovecot support the old
>>> qpopper "Bulletin" ability?
>>>
>>> Basically I need a simple way of posting bulletins to all domain users.
>>> Qpopper maintained a bulletin db for each user and sent them the next
>>> bulletin in sequence.
>>
>> I guess there could be a plugin that does this check on each login. But 
>> would it actually be useful? Why would it be better than simply sending the 
>> mail to all the users? For example:
>>
>> doveadm save -A < bulletin.txt
>
> The reasons for bulletins as I see it are:
>
> 1.  The doveadm save command is undocumented.  It does show a cryptic line in 
> the output of the command "doveadm".  However, it doesn't give any clue what 
> it does or how to provide the message.  Your note above provides considerably 
> more information on that command.   I tested it and it works as you have 
> indicated though.
>
> 2.  The doveadm save command causes the email to be saved in each user's 
> mailbox.  If you have a lot of users, thats a lot of wasted disk space.  
> Qpopper's bulletins only kept one copy and every user downloaded from that 
> copy.  All that was retained per user was a counter of the last bulletin's 
> sequence number that was downloaded.
>
> — Doug

migration from zimbra to dovecot

[micah]

Hello,

Are there any migration scripts to handle UID conversion for moving from
a zimbra setup to dovecot? It would be nice to move all the mail without
having re-download it all!

thanks!
micah

ps - asking for a friend

Re: Thanks for Dovecot

[micah]

Dave McGuire <mcgu...@neurotica.com> writes:

> On 10/13/2015 05:04 PM, Gedalya wrote:
>>> Thanks for making Dovecot.
>>>
>>> I just transitioned from Debian Wheezy to Void Linux. It was fairly
>>> easy to get Dovecot working on my Void box, and having Dovecot makes
>>> all of my email activities easier by doing one thing and doing it right.
>>>
>>> Thank you for such great software.
>>>
>>> SteveT
>> 
>> Hey, you know what? It's never a bad time to join in and say a simple:
>> Thank you!
>
>   Agreed!  Thank you!

I want to jump on the puppy pile of love... I've used a lot of different
IMAP software over the decades, and dovecot has been a breath of fresh
air and a huge relief compared to what I've suffered through in the
past!

Thanks Timo for all your hard work, it really is appreciated!

micah

Re: Couldn't set transaction log view (seq 9..13)

[micah]

Hi Timo,

Thanks for your quick response, apologies for my slow one.

Timo Sirainen t...@iki.fi writes:

 I'm using 2.2.13 on Debian stable, and I perform regular dsync
 backups. Typically if there is an error, dsync can recover from it the
 next time it is run, but I've been encountering a new error that doesn't
 seem to self-heal:
 
 # /usr/bin/dsync -u micah backup ssh -i /root/.ssh/id_rsa 
 backmaildir@internal.example /usr/bin/dsync -u micah
 dsync-remote(micah): Error: 
 /srv/maildirbackups/m/micah/daily.1/mailboxes/INBOX/dbox-Mails/dovecot.index:
  Couldn't set transaction log view (seq 9..13)
 dsync-remote(micah): Error: Failed to read transaction log for mailbox INBOX
 
 Re-running the dsync never fixes the issue.
 
 I'm happy to provide any additional information if so required to help
 track down this problem!

 Can you create a tarball of such mailbox directories that reproduces the 
 problem? Probably this would work:

  - copy source and destination dboxes to /tmp/test/1 and /tmp/test/2
  - delete all the mailboxes except INBOX

I noticed an interesting bit of information doing this: the users that
are having a problem have no dbox-Mails directory in their
source mailboxes/INBOX directory - but on the destination where I am
sending the backups, there is one. If I remove the backup's INBOX/ and
then re-do the dsync, things work again.

  - delete all the u.* files from the INBOXes. This probably doesn't
  affect the dsync run - if it does run the mails instead through
  http://dovecot.org/tools/dbox-anonymize.pl

I have no u.* files on either side. There are m.* files though?

  - delete dovecot.index.cache files
  - make sure you can still reproduce this error by running: doveadm -o
  mail=sdbox:/tmp/test/1 sync sdbox:/tmp/test/2

did you mean s/sdbox/mdbox/ here?

also, I was using 'backup' and not sync, I dont know if that matters?

I tried this with some that had issues, but I couldn't reproduce the
error this way :p

  - tar czf test.tar.gz /tmp/test

 Then send the tarball to me and I can fix the bug. Oh, also it would
 be a good idea to test with a newer Dovecot version since there are
 some indexing fixes since v2.2.13..

I'm working on getting a newer version of dovecot installed to test
this.

 Also I added some code that should give a better-debuggable error message if 
 this happens: http://hg.dovecot.org/dovecot-2.2/rev/8146fdc0de34

great, this will certainly be helpful!

micah

Re: setting service_count results in Permission denied

[micah]

Steffen Kaiser skdove...@smail.inf.fh-brs.de writes:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On Mon, 6 Apr 2015, Micah Anderson wrote:

 service imap { service_count = 100 }

 where do you put this config line?

I put it as I indicated in the config block below:

 service imap {
  # Most of the memory goes to mmap()ing files. You may need to increase this
  # limit if you have huge mailboxes.
  #vsz_limit = 256M
  # Max. number of IMAP processes (connections)
  process_limit = 4096
  service_count = 100

^^^ here

  executable = imap imap-postlogin
 }


 please run:

 doveconf -a without_service_count
 add service_count
 doveconf -a with_service_count

 Then compare the section service imap and imap-postlogin for
 changes.

Thanks for the idea. I just did this (I dont need to restart dovecot, am
I right?)... but the only thing that changes is 'service imap', changing
service_count = 1 to service_count = 100

setting service_count results in Permission denied

[Micah Anderson]

The load on our machines has gotten too high, so I started to see log
messages that directed me to look at this page for config changes:

https://pad.riseup.net/redirect#http%3A//wiki2.dovecot.org/SocketUnavailable

One of the things I wanted to do on that page was to set the:

service imap { service_count = 100 }

but when I added that to the service imap block, and restarted dovecot,
I see this in the logs: Error:
net_connect_unix(/run/dovecot/imap-postlogin) failed: Permission denied

I dont understand how this impacts permissions. Its clearly this
setting, because when I remove it, things work again. I found nothing
else in the logs indicating a configuration problem.

This is what the config block looked like:

service imap {
  # Most of the memory goes to mmap()ing files. You may need to increase this
  # limit if you have huge mailboxes.
  #vsz_limit = 256M
  # Max. number of IMAP processes (connections)
  process_limit = 4096
  service_count = 100
  executable = imap imap-postlogin
}

service imap-postlogin {
  # the script process runs as the user specified here (v2.0.14+):
  #user = $default_internal_user

  executable = script-login /usr/local/sbin/postlogin_imap

  # this UNIX socket listener must use the same name as given to imap executable
  unix_listener imap-postlogin {
  }
}

This is with dovecot 2.2.13, thanks for any help you can provide!

micah

Re: Install process breaks its self by design

[micah]

Merlin at Desktop Masters mer...@desktopmasters.com writes:

 Correct
 Correct
 Correct

 And when I went to the page it lead me here.  Whomever is making the 
 .deb packages needs to be made aware of this.  Can you please point me 
 into a direction?

http://bugs.debian.org

Specifically:
https://www.debian.org/Bugs/Reporting

Couldn't set transaction log view (seq 9..13)

[micah anderson]

Hello,

I'm using 2.2.13 on Debian stable, and I perform regular dsync
backups. Typically if there is an error, dsync can recover from it the
next time it is run, but I've been encountering a new error that doesn't
seem to self-heal:

# /usr/bin/dsync -u gae backup ssh -i /root/.ssh/id_rsa 
backmaildir@internal.example /usr/bin/dsync -u micah
dsync-remote(micah): Error: 
/srv/maildirbackups/m/micah/daily.1/mailboxes/INBOX/dbox-Mails/dovecot.index: 
Couldn't set transaction log view (seq 9..13)
dsync-remote(micah): Error: Failed to read transaction log for mailbox INBOX

Re-running the dsync never fixes the issue.

I'm happy to provide any additional information if so required to help
track down this problem!

micah

Limit confusion

[Micah Anderson]

Hi,

On one of my dovecot servers, i've got a limit problem... and I'm a
little confused about which knobs I should adjust to solve it.

I've been getting both of these:

Error: net_connect_unix(imap) failed: Resource temporarily unavailable -
http://wiki2.dovecot.org/SocketUnavailableess

This one caused me to go read that wiki page, but none of those seem to
apply to me:

 . i'm running 2.2.9
 . i dont have service imap { client_limit }, although I do have service auth ( 
client_limit = 1024 }
 . my dovecot/auth process shows: dovecot/auth [91 wait, 0 passdb, 0 userdb]
 . it doesn't appear the master process is using very much cpu (in fact
   the system is mostly low load in general)
 . I  have service_count = 0 set for imap-login for the 'high performance mode'

dovecot: master: Warning: service(imap): process_limit (2560) reached,
client connections are being dropped

The second is because I have process_limit of 2560 for the imap
service... I can raise that to 4096, but I suspect there are other
limits I need to juggle?

Thanks a lot for any suggestions!


I have ulimit -n 2048 set for the dovecot processes (in
/etc/default/dovecot) and the following is my dovecot -n:

# 2.2.9: /etc/dovecot/dovecot.conf
auth_verbose = yes
default_process_limit = 256
default_vsz_limit = 512 M
dict {
  expire = mysql:/etc/dovecot/dovecot-dict-sql.conf
  quota = mysql:/etc/dovecot/dovecot-dict-sql.conf
}
disable_plaintext_auth = no
first_valid_gid = 8
first_valid_uid = 8
last_valid_gid = 8
last_valid_uid = 8
listen = *
login_greeting = howdy, ready.
mail_location = mdbox:~/mdbox:INDEX=/srv/dovecot_indexes/%d/%1n/%n
mail_plugins = expire quota zlib
maildir_very_dirty_syncs = yes
namespace {
  inbox = yes
  location = 
  prefix = 
  separator = .
}
namespace {
  alias_for = 
  hidden = yes
  inbox = no
  list = no
  location = 
  prefix = INBOX.
  separator = .
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  antispam_allow_append_to_spam = yes
  antispam_backend = pipe
  antispam_debug_target = syslog
  antispam_pipe_program = /usr/local/bin/train_spam
  antispam_pipe_program_args = -d;10.0.1.109
  antispam_pipe_program_notspam_arg = ham
  antispam_pipe_program_spam_arg = spam
  antispam_pipe_tmpdir = /var/tmp
  antispam_signature = X-Spam-Flag
  antispam_signature_missing = error
  antispam_spam_pattern_ignorecase = SPAM
  antispam_trash_pattern = trash;Trash;Deleted *
  expire = Trash
  expire2 = Trash/*
  expire3 = Spam
  expire_dict = proxy::expire
  quota = dict:Your mail quota::proxy::quota
  quota_rule = *:bytes=100663296
  quota_rule2 = Trash:bytes=+20%%
  quota_rule3 = Spam:bytes=+10%%
  quota_rule4 = INBOX.Trash:bytes=+20%%
  quota_rule5 = INBOX.Spam:bytes=+10%%
  sieve = ~/.dovecot.sieve
  sieve_after = /var/lib/dovecot/sieve/after.sieve
  sieve_before = /var/lib/dovecot/sieve/default.sieve
  sieve_dir = ~/sieve
  zlib_save = gz
  zlib_save_level = 6
}
protocols =  imap pop3
service anvil {
  unix_listener anvil {
group = dovecot
mode = 0660
user = root
  }
}
service auth {
  client_limit = 1024
  unix_listener auth-userdb {
group = mail
mode = 0600
user = mail
  }
}
service dict {
  unix_listener dict {
group = mail
mode = 0600
user = mail
  }
}
service imap-login {
  process_min_avail = 10
  service_count = 0
  vsz_limit = 512 M
}
service imap-postlogin {
  executable = script-login /usr/local/sbin/postlogin_imap
}
service imap {
  executable = imap imap-postlogin
  process_limit = 4096
}
service pop3-login {
  process_min_avail = 10
  service_count = 0
  vsz_limit = 512 M
}
service pop3-postlogin {
  executable = script-login /usr/local/sbin/postlogin_pop
}
service pop3 {
  executable = pop3 pop3-postlogin
}
service stats {
  fifo_listener stats-mail {
mode = 0600
user = mail
  }
}
ssl_cert = /etc/certs/wildcard/cert.pem
ssl_cipher_list = HIGH:-ADH:-aNULL
ssl_key = /etc/certs/wildcard/key.pem
userdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
verbose_proctitle = yes
protocol lda {
  mail_plugins = expire quota zlib sieve
  plugin {
quota_exceeded_message = Sorry, your message cannot be delivered to that 
person because their mailbox is full. If you can contact them another way, you 
may wish to tell them of this problem.
quota_rule7 = INBOX:bytes=+10%%
  }
}
protocol imap {
  imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
  mail_max_userip_connections = 10
  mail_plugins = expire quota zlib imap_quota imap_zlib antispam
  plugin {
quota_exceeded_message = You are over quota. To avoid losing mail, 
immediately empty your Trash and Sent folders and delete emails with large 
attachments. 
  }
}
protocol pop3 {
  mail_plugins = expire quota zlib
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_uidl_format = %g
}

[Dovecot] default client_limit

[Micah Anderson]

I recently upgraded my dovecot from 2.1 to 2.2, and when I started, I
received this message:

doveconf: Warning: service auth { client_limit=1000 } is lower than required 
under max. load (1024)

Searching through my configs, I do not have 1024 set anywhere. 

In order to stop this I set client_limit=1024 in my auth{} block... this
seemed odd that the defaults disagreed with each other.

However, it made me realize that my system could probably handle more,
if it were properly tuned, and the different limit settings at their
defaults are probably smaller than they could be for my server and the
load it is under. So, I'm wondering how I can accurately set some of
these values?

The different possible things that can be set are:

default_process_limit (for some reason I have this set to 256)
default_client_limit (default)
default_vsz_limit (I have this set to 512M)

for the imap-login service, I have:
  service_count = 0
  process_min_avail = 10
  vsz_limit = 512M

for service pop3-login, these are set:
  service_count = 0
  process_min_avail = 10
  vsz_limit = 512M

for service imap,
  process_limit = 2048

service auth, I've just set the client_limit=1024

I've also got a ulimit -n set to 2048 and in my sysctl.conf, I have the
following:

fs.inotify.max_user_instances = 2048

because I was seeing this in the logs:
Warning: Inotify instance limit for user 8 (UID mail) exceeded, disabling. 
Increase /proc/sys/fs/inotify/max_user_instances


What I am looking for is a few things:

 . is there a formula for setting a proper value for some of these?
 . will dovecot warn me when I've reached a limit, so I can adjust
 these? 

thanks for any suggestions or improvements, I'm pretty sure this system
can handle a lot more than it is doing right now, I just am unclear on
how to tune it up to do that.

micah
 

[Dovecot] doveadm mailbox create error codes

[Micah Anderson]

Hello,

I'm trying to find out what the different 'doveadm mailbox create' error
codes are. I've been looking through the source, but I can't seem to
find them.

I did determine that '65' happens when Error: Can't create mailbox x:
Mailbox already exists, but I've also received error 67, and the more
mysterious error code 16640.

Thanks for any pointers!
micah

[Dovecot] Migrating from dovecot 2.1 to 2.2

[Micah Anderson]

Due to the incompatibilities with the dsync protocol version between 2.1
and 2.2, I am unable to use dsync to move users to a new machine where
we have 2.2.5 setup. This wouldn't normally be a problem, as I could
just rsync the user's mdbox and dovecot would happily do the right
thing. However, on the new system I've placed the indexes into a faster
storage medium, where on the older system the indexes were included in
their mail directory.

I can rsync over the entire thing, including their old indexes, and
dovecot will try to do the right thing and build new indexes, but what
happens is all mail is marked as unread, and I get some old index cruft
left around in the directories.

How can I preserve the seen/unseen flags in such a migration? 

Thanks!
micah

[Dovecot] More fine-grained connection limitations?

[Micah Anderson]

There is the mail_max_userip_connections setting, which helps for
dealing with number of connections at any given time, but I'm looking
for something that will help deal with users who configure their mail
clients to connect too frequently.

For example, I've seen users who configured their clients to check
(IMAP) mail every 3 seconds. This is far too frequent and puts
unnecessary load on the server. Even once per minute seems excessive to
me.

It would be really great if there were a way to tarpit those users to
slow them down, or send them an imap message saying they are connecting
to often and connections have been disabled for the next 5 minutes or
something.

Micah

ps - what happens to a user when they hit mail_mx_userip_connections?
Further connections are just denied, or dropped?

[Dovecot] Dovecot antispam plugin for 2.2

[Micah Anderson]

Hello,

I recently was looking to update to dovecot 2.2 and I have been happily
using the dovecot-antispam plugin in 2.1, but found that there have been
some changes in dovecot that make this version not compile against 2.2.

I'm wondering if Johannes you have any plans to update it, or if others
have managed to figure out what is necessary to get it to work?

The particular piece that I am currently stuck on is that the
dict_init() in dovecot now requires more arguments than in previous
versions, it now requires this:

int dict_init(const char *uri, enum dict_data_type value_type,
  const char *username, const char *base_dir, struct dict **dict_r,
  const char **error_r);

but previously, we only needed four arguments. Now we need to know what
to supply for '**dict_r' and '**error_r'. Any advise/help would be appreciated:

make[1]: Entering directory 
`/home/micah/debian/dovecot-antispam/dovecot-antispam-2.0+20120225'
cc -DHAVE_CONFIG_H -I/usr/include/dovecot/ -I/usr/include/dovecot/src/ 
-I/usr/include/dovecot/src/lib/ -I/usr/include/dovecot/src/lib-storage/ 
-I/usr/include/dovecot/src/lib-mail/ -I/usr/include/dovecot/src/lib-imap/ 
-I/usr/include/dovecot/src/lib-dict/ -I/usr/include/dovecot/src/lib-index/ 
-I/usr/include/dovecot/src/imap/ -o dovecot-version dovecot-version.c
./dovecot-version  dovecot-version.h
  CC   antispam-storage.o
In file included from antispam-storage-2.2.c:20:0,
 from antispam-storage.c:3:
antispam-plugin.h: In function ‘string_dict_init’:
antispam-plugin.h:288:2: error: too few arguments to function ‘dict_init’
  return dict_init(uri, DICT_DATA_TYPE_STRING, username, NULL);
  ^
In file included from antispam-plugin.h:13:0,
 from antispam-storage-2.2.c:20,
 from antispam-storage.c:3:
/usr/include/dovecot/dict.h:34:5: note: declared here
 int dict_init(const char *uri, enum dict_data_type value_type,
 ^


micah

Re: [Dovecot] DRAC plugin for Dovecot-2.x

[micah anderson]

Hello,

USUDA Hisashi us...@designet.co.jp writes:

 Hello All,

 I released the DRAC plugin for dovecot-2.x.

 http://sourceforge.jp/projects/dovecot2-drac/

 It's based the plugin for dovecot-1.1:
 http://dovecot.org/patches/1.1/drac.c

I recently tried to get your DRAC plugin to work for 2.2.5, but it fails
to compile as it is looking for network.h include file, and that doesn't
appear to exist any longer in 2.2.

Do you have plans to make a newer version of this plugin?

thanks,
micah

Re: [Dovecot] dsync timeout?

[micah anderson]

Sean Kamath kam...@moltingpenguin.com writes:

 On Jan 30, 2013, at 3:46 PM, micah anderson mi...@riseup.net wrote:
 Seems that only the above process was still around and no other dsync
 processes. I have three machines that all have this happening it seems.
 
 I wonder if there is a ssh configuration option I could set to make
 these die off.

 If the ssh process isn't sending anything, and just waiting for read()s, and 
 keepalives are turned off, the SSH session might never know the remote side 
 is long gone. . .

This time I managed to capture a process that was stuck and look at it
from the server side, and the client side:

on the server:

2000 19470  0.0  0.0   7512  3816 ?Ss   Feb05   0:01 /usr/bin/dsync 
dsync-server -E -u foo
# strace -s 1024 -F -p 19470
Process 19470 attached - interrupt to quit
write(2, dsync-remote(foo): Error: mdbox 
/srv/maildirbackups/foo/daily.1/storage: Duplicate GUID 
96860517f68aa94f8b5197f19f0b in m.41:682501 and m.37:653225\n, 167

on the client:

root 19001  0.0  0.0  41308  1600 ?SFeb05   0:00 ssh -i 
/root/.ssh/backmaildir_id_rsa backmaildir@hoopoe-pn /usr/bin/dsync -u foo server

# strace -s 1024 -F -p 19001
Process 19001 attached - interrupt to quit select(8, [4], [], NULL, NULL

interestingly, now that I've been watching this more, the same users
keep getting wedged. 

When I attempt to do a dsync of that user by hand, I get this:

dsync-local(foo): Error: Unexpected reply from server: 13   
d2a100118c45d24f760f97f19f0b3561128 \Recent 1353980259

I tried one of the other users that was stuck, and it gave me:

dsync-remote(bar): Error: Corrupted dbox file 
/srv/maildirbackups/bar/daily.1/storage/m.130 (around offset=22532): msg header 
has bad magic value

This looks like there is something corrupted with the dbox for the user
on the client side, is there something I can do to repair those?

 If any data were transmitted, it would discover the remote side is turned off.

One thing I am doing is using a ssh controlmaster socket, and if I kill
the process on the client's side, the server side process also dies.

micah

Re: [Dovecot] dsync timeout?

[micah anderson]

Sean Kamath kam...@moltingpenguin.com writes:

 On Jan 30, 2013, at 3:46 PM, micah anderson mi...@riseup.net wrote:
 Seems that only the above process was still around and no other dsync
 processes. I have three machines that all have this happening it seems.
 
 I wonder if there is a ssh configuration option I could set to make
 these die off.

 If the ssh process isn't sending anything, and just waiting for read()s, and 
 keepalives are turned off, the SSH session might never know the remote side 
 is long gone. . .

 If any data were transmitted, it would discover the remote side is turned off.

 See man ssh_config and the option TCPKeepAlive.

 BTW: Since it's not on the command line, it's likely in /etc/ssh_config or 
 /etc/ssh/ssh_config.  Or ~/.ssh/config.

In /etc/ssh/sshd_config on the server I'm sending to, TCPKeepAlive yes
is set.

The default on this system, according to the man page, seems to be to
have TCPKeepAlive set. 

Perhaps I should set ServerAliveInterval?

micah

[Dovecot] dsync timeout?

[Micah Anderson]

I'm using dsync for a regular backup. The backup system flocks so that
two cannot run at the same time, which is generally a good thing. The
problem is that it seems like dsync sometimes goes off into the weeds
and never comes back, leaving a process running and doing nothing
forever, hogging the lock and causing my backups never to run again. I
just finally figured out that was what was causing the backups not to
run on this system was this process:

root 17836  0.0  0.0  40888  1600 ?S 2012   0:00 ssh -i 
/root/.ssh/backmaildir_id_rsa backmaildir@arg /usr/bin/dsync -u foobar server

yeah, that has been running since 2012 :(

root:/tmp# strace -p 17836
Process 17836 attached - interrupt to quit
select(8, [4], [], NULL, NULL

very exciting...

There doesn't seem to be a timeout in dsync, but perhaps there should
be? At this point my only option is to write a cronjob that will look
for dsync processes that are over a certain amount of time old and then
kill them, after I do that I will need to take a shower because that is
a very dirty solution :P

thanks for any ideas, or help!
micah
-- 

[Dovecot] Error: Internal quota calculation error

[Micah Anderson]

Hello,

I'm using 2.1.7 with seive and mysql quotas. We had an outage the other
day where the database server where quotas are stored was not available
for a short period of time. 

In dovecot land, the following types of errors occured in that scenario:

Oct 26 22:19:01 grosbeak dovecot: lda(exam...@riseup.net): Error: Internal 
quota calculation error
Oct 26 22:19:01 grosbeak dovecot: lda(exam...@riseup.net): Error: sieve: 
msgid=20122132765181x.abcce...@example.com: failed to store into mailbox 
'Trash': Internal error occurred. Refer to server log for more information. 
[2012-10-26 22:19:01]
Oct 26 22:19:01 grosbeak dovecot: lda(exam...@riseup.net): Error: sieve: script 
/maildir/e/example/.dovecot.sieve failed with unsuccessful implicit keep (user 
logfile /maildir/e/example/.dovecot.sieve.log may reveal additional details)

I expect that there would be quota calculation errors as dovecot could
not reach the database server, but what worried me was the 'failed to
store into mailbox' message from sieve. The 'Trash' mailbox in this
particular seive script is the correct location for the message to be
filed into, but the worrisome message is the 'failed with unsuccessful
implicit keep'.

I looked through all the message-ids that reported this error and I
found that the messages were properly delivered in the end, so this
might be some issue interacting between sieve, dovecot and quota and
just causing a scary message that can be ignored?

thanks for any information you can provide, dovecot is great!

micah
-- 



pgpwF2qsdIGgP.pgp
Description: PGP signature

Re: [Dovecot] mailboxes missing after upgrade from 2.1.4 to 2.1.7

[Micah Anderson]

Micah Anderson mi...@riseup.net writes:

 I mentioned this on the #dovecot irc channel, but I thought I would post
 here so I can provide more details.

 The basic problem is that when I upgraded from 2.1.4 to 2.1.7, some
 users no longer are able to see their folders in pine/alpine. The
 folders are actually there, and are subscribed (according to doveadm)
 and its possible to do operations on those folders, if you know the
 name (for example, opening the folder works fine if you know the name).

 When I downgrade to 2.1.4 again, the folders appear again, so its
 clearly something that changed between the two versions. I looked
 through the changelog, and found a few possibile culprits, I think
 pine/alpine use imapc, so that could be related:

I was able to solve this by changing the alpine folder-collection
configuration to *not* have /INBOX.[] at the end and instead have just
[]!

micah

[Dovecot] mailboxes missing after upgrade from 2.1.4 to 2.1.7

[Micah Anderson]

 in #dovecot, I attempted to add 'rawlog' to:

service imap-postlogin {
  executable = script-login /usr/local/sbin/postlogin_imap
}

to get:

service imap-postlogin {
  executable = script-login rawlogin /usr/local/sbin/postlogin_imap
}

but with that, it doesn't even open the INBOX folder, and there is
nothing in the user's rawlog folder, I get this error:

dovecot: imap-postlogin: Error: Fatal: This process must not be run as root
it didn't open even the INBOX imap folder

I went over the http://wiki2.dovecot.org/Debugging/Rawlog and made sure
everything was correct there, but it still didn't work right.

Any ideas or suggestions of things to try would be greatly appreciated!

thanks,
micah
-- 

Re: [Dovecot] Dsync automation

[Micah Anderson]

Jitendra Bhaskar jeetuind...@gmail.com writes:

 I am using dovecot-2.1.3 on centos 5.7. Its running fine. I have two
 systems for mail server one is for backup. Currently I am running rsync
 daily. But I want to use dsync. I went through all the
 http://wiki2.dovecot.org/Tools/Dsync and
 http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.htmlbut
 I couldn't understood how to automatize dsync. Can anybody give me any
 idea ho to do it ?

As far as I can tell, doing a for loop around 'doveadm user \*' is the
only way. That will do a dsync of all the users. On the backup server
you will need to decide what to do with the data. I've been rotating it
on a daily basis to try and get several daily backups, then some weekly,
and monthly.

I've been working on a backupninja[0] handler to do dsync backups, but
its still rough.

micah

0. https://labs.riseup.net/code/projects/backupninja


-- 

[Dovecot] 2.1.3: doveadm mailbox delete lost the -s option

[Micah Anderson]

Looks like the recently added -s (for unsubscribing) option to doveadm
mailbox delete went missing:

root@vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored
delete: invalid option -- 's'
doveadm mailbox delete [-u user|-A] [-S socket_path] [-s] mailbox [...]

although the option still is shown as valid :)

micah


-- 

Re: [Dovecot] dsync redesign

[Micah Anderson]

Charles Marcus cmar...@media-brokers.com writes:

 On 2012-03-27 11:47 AM, Micah Anderson mi...@riseup.net wrote:
 One would be the ability to perform *intelligent* incremental /
 rotated backups. I can do this now by running a dsync backup
 operation and then doing manual hardlinking or moving of the backup
 directories (daily.1, daily.2, weekly.1, monthly.1, etc.), but it
 would be more intelligent if this were baked into the backup process.

 There are already numerous tools that do this flawlessly - I've been using
 rsnapshot (which uses rsync) for this for years.

Are you snapshotting your filesystem (using LVM, or SAN, or similar)
before doing rsnapshot? Because if you aren't then rsync will not
assuredly get everything in a consistent state.

 I don't know if Timo should be spending his time reinventing the wheel.

dsync backup is already here, and it is quite useful.

 I'm much more interested in dsync working flawlessly to keep one or more
 secondary servers in sync, and leave backups to backup software.

I'm not against that idea, I just have not yet found a good way to use
any backup software in such a way to handle large numbers of user's
mail.

 Although, one interesting piece that I am hopeful I'll be able to implement 
 soon
 (with Timo's professional help) is the ability to easily and automatically map
 my rsnapshot snapshots directory to a read-only 'Backups' namespace that
 automatically shows the snapshots by date and time as they are produced. This
 way users could 'go back in time' anytime they wanted without having to call
 me... :)

Interesting idea, would be a great one to share with the community if
you decide to do so.

micah

Re: [Dovecot] dsync redesign

[Micah Anderson]

Timo Sirainen t...@iki.fi writes:

 In case anyone is interested in reading (and maybe helping!) with a dsync 
 redesign that's intended to fix all of its current problems, here are some 
 possibly incoherent ramblings about it:

thank you for opening this discussion about dsync!

besides the problems I've encountered with dsync, there are a couple
things that I think would be great to build into the new vision of the
protocol. 

One would be the ability to perform *intelligent* incremental/rotated
backups. I can do this now by running a dsync backup operation and then
doing manual hardlinking or moving of the backup directories (daily.1,
daily.2, weekly.1, monthly.1, etc.), but it would be more intelligent if
this were baked into the backup process.

Secondly, being able to filter out mailboxes could result in much more
efficient syncing. Now there is the capability to operate on only
specific mailboxes, but this doesn't scale well when I am trying to
backup thousands of users and I want to omit the Spam and Trash folders
from the sync. I would have to get a mailbox list of each user, and then
iterate over each mailbox for each user, skipping the Spam and Trash
folders, forking a new 'dsync backup' for each of their mailboxes, for
each user.

Lastly, there isn't a good method for restoring backups. I can reverse
the backup process, onto the user's live mailbox, but that brings the
user into an undesirable state (eg. their mailbox state one day
ago). Better would be if their backup could be restored in such a way
that the user can resolve the missing pieces manually, as they know
best.

thanks again for your work on this, from my position dovecot is an
amazing piece of software, the only part that seems to have some issues
is dsync and I applaud the effort to redesign to fix things!

micah

Re: [Dovecot] dovecot 2.1.3 dsync Unexpected finish reply

[Micah Anderson]

Micah Anderson mi...@riseup.net writes:

 dsync-local(u...@example.com): Error: Unexpected finish reply: by 
 ims-d13.mx.aol.com (8.14.1/8.14.1) with ESMTP id q2LEhqXZ017169;
 dsync-local(u...@example.com): Error: Unexpected reply from server:Wed, 
 21 Mar 2012 10:43:52 -0400
 dsync-local(u...@example.com): Warning: Mailbox changes caused a desync. You 
 may want to run dsync again.

I'm also getting similar strange results with my regular dsync backup:

dsync-local(u...@example.com): Error: Unexpected reply from server: 0 
23bdce147b43674f8e272c449efa1242146 \Recent 1332335848

this is with 2.1.3.

micah

[Dovecot] doveadm user -f index

[Micah Anderson]

I've configured my mail_location to have a different location for
performance reasons so they aren't in the same location as the
mail_location.

The 'doveadm user -f home' is useful to find where a user's home
directory is for various scripting purposes, but I can't seem to find a
way to determine the location of the user's indexes. 

I can do something with the output of dovecot -a to find the
mail_location and then look for a configured INDEX, but then I don't
have a good way of translating the %d/%1n/%n type string formatters into
their values for a user.

thanks for any suggestions!
micah

-- 

[Dovecot] dovecot 2.1.3 dsync Unexpected finish reply

[Micah Anderson]

-get failed: box=INBOX uid=69135 
guid=c5800130d2556a4f59421d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69136 
guid=83a91e08b4566a4f19711d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69137 
guid=50cf9721f95e6a4f7e441d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69138 
guid=fda2a82886606a4f88171d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69139 
guid=97ee1d1ad1636a4fc94d1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69140 
guid=4f50671f85666a4f30611d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69141 
guid=0fea590fb4666a4f7a6b1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69142 
guid=f3210b02a5676a4ffa1f1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69143 
guid=8d325a06686a6a4f2b361d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69144 
guid=0cbf1839f1756a4f8f681d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69145 
guid=d6209a2898796a4f671a1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69146 
guid=576ade31da7d6a4f5f571d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69147 
guid=70a15b34247e6a4f445a1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69148 
guid=3ff92631cd886a4ffd631d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69149 
guid=3ff2081568916a4f134d1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69150 
guid=d1a67b0907ab6a4f54601d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69151 
guid=3d4cb1197ee96a4fbf5f1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=INBOX uid=69152 
guid=aae2542818266b4f7d1e1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2077 
guid=c5a0930248e8694fb77c1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2078 
guid=dce63e1a6bee694f47021d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2079 
guid=dde63e1a6bee694f47021d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2080 
guid=dee63e1a6bee694f47021d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2081 
guid=dfe63e1a6bee694f47021d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2082 
guid=a110b53585056a4fe81b1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2083 
guid=a210b53585056a4fe81b1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2084 
guid=93afeb1f7d0b6a4f694e1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2085 
guid=ef2bb1098a2e6a4fcf1b1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2086 
guid=d8ba601bee2e6a4f982f1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2087 
guid=d9ba601bee2e6a4f982f1d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2088 
guid=7e5b483b433d6a4fd9361d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2089 
guid=7f5b483b433d6a4fd9361d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2090 
guid=b08c5205ba646a4f10671d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2091 
guid=805b483b433d6a4fd9361d12b59f
dsync-local(u...@example.com): Error: msg-get failed: box=sent-mail uid=2092 
guid=815b483b433d6a4fd9361d12b59f
dsync-local(u...@example.com): Error: Unexpected finish reply: by 
ims-d13.mx.aol.com (8.14.1/8.14.1) with ESMTP id q2LEhqXZ017169;
dsync-local(u...@example.com): Error: Unexpected reply from server:Wed, 21 
Mar 2012 10:43:52 -0400
dsync-local(u...@example.com): Warning: Mailbox changes caused a desync. You 
may want to run dsync again.

those final Unexpected finish reply and Unexpected reply from server
are a bit surprising results. 

Running dsync again seems to resolve things, but there seems to be a bug
here that is causing unexpected results to leak through to the dsync
process?

thanks,
micah

Re: [Dovecot] Antispam plugin not compatible with Dovecot 2.1

[Micah Anderson]

Eugene Paskevich eug...@raptor.kiev.ua writes:

 On Wed, 18 Jan 2012 18:31:49 +0200, Timo Sirainen t...@iki.fi wrote:

 On Wed, 2012-01-18 at 18:19 +0200, Eugene Paskevich wrote:
  mailbox.c: In function 'antispam_save_begin':
  mailbox.c:138:12: error: 'struct mail_save_context' has no member named
  'copying'
 
  The copying should be changed to copying_via_save.

 Thank you, Timo.
 Would #if DOVECOT_IS_GE(2,1) suffice or do I need anything more specific?

 Where do you expect to find such macro? ;) Hm. Perhaps I should try to
 add one.

 Heh. That's Johannes' package private macro... :)

I notice that Johannes hasn't made a 2.1 version of the anti-spam
plugin, Eugene were you able to build one successfully? If so, would you
be willing to share your changes that were required to make it work?

thanks,
micah

Re: [Dovecot] dot named folders

[Micah Anderson]

Robert Schetterer rob...@schetterer.org writes:

 Am 08.03.2012 17:27, schrieb Micah Anderson:
 Willie Gillespie wgillespie+dove...@es2eng.com writes:
 
 On 03/07/2012 12:43 PM, Micah Anderson wrote:

 When a user makes a folder called 'x.y' it actually creates a folder
 called 'x' with a folder called 'y' inside, rather than a folder called
 'x.y'. I'm guessing this has to do with an internal folder separator
 namespace configuration, but I'm a bit confused by how this works.

 Correct.
 Similar to how in Linux, I could create a folder
 mkdir test1/test2
 It will create test2 inside of test1.

 The difference being that IMAP doesn't necessarily need the parent mailbox 
 to
 exist, where Linux would throw an error if test1/ didn't exist first.

 So basically, as far as I know, you can't have a folder with a . in the 
 name
 with the namespaces you have set up.
 
 That makes sense, however I'm not sure that I need these namespaces any
 longer if I no longer am using the maildir format (mdbox). 
 
 In either case, it seems like the internal folder separator should not
 be exposed to the user like this. What is happening now is the user gets
 something other than they expect (a folder within a folder, instead of a
 folder with a dot in the name) because of some unknown internal
 configuration. 
 
 If moving to mdbox is not enough to remove these namespace
 configurations that cause this, then it would be good if the user was
 unable to create such a folder, because it was prohibited, rather than
 creating something other than they expect.
 
 micah
 

 http://wiki.dovecot.org/Plugins/Listescape
 may help

Interesting, thanks for the pointer, although I think I prefer if users
are just prohibited from making a 'folder.withadot' and told that it is
prohibited right away, rather than giving them a way to do it.

micah

-- 

Re: [Dovecot] mdbox + gzip and rsync

[Micah Anderson]

Jean-Daniel Beaubien jd.beaub...@gmail.com writes:

 After reading the following paragraph from the dovecot doc, I've been
 wondering how it would affect rsync (when combined with gzip):

 Expunging a message only decreases the message's refcount. The space is
 later freed in purge step. This is typically done in a nightly cronjob
 when there's less disk I/O activity. The purging first finds all files that
 have refcount=0 mails. Then it goes through each file and copies the
 refcount0 mails to other mdbox files (to the same files as where newly
 saved messages would also go), updates the map index and finally deletes
 the original file. So there is never any overwriting or file truncation.

Interesting, so it would be recommended to those using mdbox format to
run a 'dovadm purge -A' every night to clean up these unused files? It
seems like without this, mail storage usage will just grow infinitely.

It does appear that using an rsync backup process for mdbox would not be
able to detect this and backups would also grow infinitely.

micah

-- 

Re: [Dovecot] dot named folders

[Micah Anderson]

Willie Gillespie wgillespie+dove...@es2eng.com writes:

 On 03/07/2012 12:43 PM, Micah Anderson wrote:

 When a user makes a folder called 'x.y' it actually creates a folder
 called 'x' with a folder called 'y' inside, rather than a folder called
 'x.y'. I'm guessing this has to do with an internal folder separator
 namespace configuration, but I'm a bit confused by how this works.

 Correct.
 Similar to how in Linux, I could create a folder
 mkdir test1/test2
 It will create test2 inside of test1.

 The difference being that IMAP doesn't necessarily need the parent mailbox to
 exist, where Linux would throw an error if test1/ didn't exist first.

 So basically, as far as I know, you can't have a folder with a . in the name
 with the namespaces you have set up.

That makes sense, however I'm not sure that I need these namespaces any
longer if I no longer am using the maildir format (mdbox). 

In either case, it seems like the internal folder separator should not
be exposed to the user like this. What is happening now is the user gets
something other than they expect (a folder within a folder, instead of a
folder with a dot in the name) because of some unknown internal
configuration. 

If moving to mdbox is not enough to remove these namespace
configurations that cause this, then it would be good if the user was
unable to create such a folder, because it was prohibited, rather than
creating something other than they expect.

micah

Re: [Dovecot] seeking advice: dovecot versions; mailbox formats.

[Micah Anderson]

Vincent Schut sc...@sarvision.nl writes:

 Debian currently has dovecot 1.2.15 in its repositories; not that much
 newer...

No, Debian has 1.2.15 in its /stable (squeeze)/ repositories, there are
newer versions available in other Debian repositories.

micah

[Dovecot] dot named folders

[Micah Anderson]

When a user makes a folder called 'x.y' it actually creates a folder
called 'x' with a folder called 'y' inside, rather than a folder called
'x.y'. I'm guessing this has to do with an internal folder separator
namespace configuration, but I'm a bit confused by how this works.

I'm using 2.0.15 with mdbox and this is what I have configured for my
namespaces:

namespace {
  separator = .
  prefix =
  inbox = yes
}

namespace {
  separator = .
  prefix = INBOX.
  inbox = no
  hidden = yes
  list = no
}

I migrated from courier maildirs, so perhaps I no longer need some of
these now that the conversion is finished?

thanks for any suggestions, I've got my head mixed up on this issue,
micah


-- 

Re: [Dovecot] expunging

[micah anderson]

On Sat, 18 Feb 2012 18:52:57 +0200, Timo Sirainen t...@iki.fi wrote:
 On 17.2.2012, at 20.25, Micah Anderson wrote:
 
  /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d
  /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d
  /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d
  
  It seems like it does not use the iterate query, but rather it looks
  into the database at the expires table and rather it iterates over every
  user mentioned there. This is a problem because I've got multiple
  dovecot machines with different sets of users on them, using the same
  table, so that means that its doing stat() calls on each system for
  every user mentioned there and failing to find the user on the
  filesystem (because the user is on another system). In these cases it
  spits out an error: doveadm(user): Info: User no longer exists, skipping
 
 Upgrade to v2.1:
 
 * Expire plugin: Only go through users listed by userdb iteration.
   Delete dict rows for nonexistent users, unless
   expire_keep_nonexistent_users=yes.

Great, i'll give that version a spin, thanks!

Is there a similar dict row cleanup for quota?

micah

Re: [Dovecot] 2.1 upgrade docs?

[Micah Anderson]

Marc Perkel m...@perkel.com writes:

 Just wondering if there's any upgrade docs for 2.1 online?

yes there are.

(hint: search terms dovecot upgrade)

[Dovecot] expunging

[Micah Anderson]

I am having trouble scaling some regular dovecot cleanup operations on
our servers. On a daily basis, I'm wanting to do this on each server,
which contains its own isolated set of user storage:

/usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d
/usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d
/usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d

but these are a very expensive operations. For example, just doing the
Spam expunge takes 30 minutes (or more, depending on the load) of heavy
disk operations, on each machine it is run on. There are approximately
20k users on each machine. 

It seems like it does not use the iterate query, but rather it looks
into the database at the expires table and rather it iterates over every
user mentioned there. This is a problem because I've got multiple
dovecot machines with different sets of users on them, using the same
table, so that means that its doing stat() calls on each system for
every user mentioned there and failing to find the user on the
filesystem (because the user is on another system). In these cases it
spits out an error: doveadm(user): Info: User no longer exists, skipping

For users that do exist on this system, it seems to do something on the
order of 15 stat() calls, at minimum. 

What are some things I can do to make this less of an expensive process?
If I had a shared storage system that each machine used, this would cut
down on the resource waste because I'd only need to run the query once,
but unfortuntely, that isn't how these systems were designed.

thanks for any ideas, tips etc.
micah


pgpcDBpKXjq4y.pgp
Description: PGP signature

[Dovecot] dovecot-antispam: pipe backend with spamc failing to parse args

[Micah Anderson]

I'm trying to get the pipe backend to work with the new
dovecot-antispam (I was very pleased to see the new version, thanks
Johannes!)

Before I would run the following over the maildir: 

spamc -d 10.0.1.109 -L ham  $file 21  /dev/null

now, i'm using dovecot-antispam and mdbox and have the following
configured:

antispam_backend = pipe
antispam_signature = X-Spam-Flag
antispam_signature_missing = error
antispam_trash_pattern = trash;Trash;Deleted *
antispam_spam_pattern_ignorecase = SPAM
antispam_pipe_tmpdir = /var/tmp
antispam_pipe_program_spam_arg = -L spam
antispam_pipe_program_notspam_arg = -L ham
antispam_pipe_program = /usr/bin/spamc
antispam_pipe_program_args = -d 10.0.1.109

Now, when I move a message to Spam, dovecot-antispam detects it and
attempts to run, but it fails:

Dec 14 16:22:45 grosbeak [username 10.0.1.74 uid]: antispam: run program
failed with exit code 64

Exit code 64 is spamc's exit code for command line usage error - but I
can't see exactly the command that is being constructed by
dovecot-antispam to know where the error is.

Any ideas?
micah

-- 



pgpvqEZKd1stL.pgp
Description: PGP signature

Re: [Dovecot] overriding userdb connection host for doveadm-user

[micah anderson]

On Thu, 08 Dec 2011 06:22:58 +0200, Timo Sirainen t...@iki.fi wrote:
 On Wed, 2011-12-07 at 13:02 -0500, Micah Anderson wrote:
  I've got my users in a replicated database setup and dovecot configured
  with two connect lines in dovecot-sql.conf:
  
  connect = host=127.0.0.1 port=3306 dbname=users user=user password=pass
  connect = host=127.0.0.2 port=3306 dbname=users user=user password=pass
  
  this works really well to help balance the load. 
 
 If you actually have it like that, it should only be using 127.0.0.2, as
 the second connect setting overrides the first one.. 

What I actually have is:

connect = host=127.0.0.1 port=3306 dbname=users user=user password=pass
connect = host=127.0.0.1 port=3307 dbname=users user=user password=pass

two different port configurations (I'm running stunnel), rather than two
different host entries[0]. 

The right way would have been:
 
 connect = host=127.0.0.1 host=127.0.0.2 port=...

So, if I just have two different ports, would the right way be:

connect = host=127.0.0.1 port=3306 port=3307 dbname...

?

micah


0. When I was anonymizing the db connection information in the original
post, I accidentally changed the ip and decided it didn't matter for the
question I was asking, so just left it. Now I see that might be
confusing.


pgpN0jQlsZCIa.pgp
Description: PGP signature

[Dovecot] overriding userdb connection host for doveadm-user

[Micah Anderson]

I've got my users in a replicated database setup and dovecot configured
with two connect lines in dovecot-sql.conf:

connect = host=127.0.0.1 port=3306 dbname=users user=user password=pass
connect = host=127.0.0.2 port=3306 dbname=users user=user password=pass

this works really well to help balance the load. 

However, when a new user is created, the replication information
sometimes has not made it to the slave database before my user creation
process tries to send that new user an email about their new email
account. If it hasn't and the email gets sent, then it bounces because
postfix thinks the user doesn't exist yet. 

Replication usually happens within seconds, but could be delayed due to
problems.

I would like a way to see if the new user's information has been
replicated yet before sending the email. I was thinking I could use
'doveadm user' to lookup the user, but I am not sure that I can specify
one of the specific 'connect' lines in my configuration to check. Is
there a way to override that configuration variable to do this test? I
tried various incarnations of passing '-o connect=host='... but doveadm
user would just ignore them.

thanks for any suggestions!
micah

ps - yes, I can make my user creation process just 'sleep' for an
arbitrary amount of time and by that time the replication should have
occured, but that is a bit of a shot in the dark, and doesn't handle
longer replication delays that could potentially happen.

-- 



pgp5kLjzEjaju.pgp
Description: PGP signature

Re: [Dovecot] dovecot-lda quota rule

[Micah Anderson]

micah anderson mi...@riseup.net writes:

 On Wed, 16 Nov 2011 11:09:12 -0500, micah anderson mi...@riseup.net wrote:
 On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen t...@iki.fi wrote:
  On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote:
   I really like the feature where you can define quota rules with percents
   which trigger off of the default values[0] (so you can set the Trash to
   allow for 10% more of the user's quota for example). 
   
   What I would really love in dovecot would be for the ability to
   configure a quota rule for dovecot-lda. I would like to configure things
   so we don't bounce emails for users until they are well over quota, the
   IMAP quota plugin is a really great way to notify people that they are
   over quota because it fails to write to other folders that should be
   enough to get people's attention that they need to deal with things, but
   bouncing is harsh.
  
  You can give LDA-specific quota settings. Like maybe:
  
  protocol lda {
plugin {
  quota_rule2 = INBOX:storage=200%
}
  }

 I just tried doing this, and I started bouncing mail for everyone
 because they were over quota (even when they arent), so I am clearly
 missing something in how I have it configured. This is what I have:

Solved, thanks to tss on irc... My problem was the dovecot-lda
invocation was not passing the -d parameter, so no userdb quota rules
were being applied.

micah

Re: [Dovecot] dovecot-lda quota rule

[micah anderson]

On Wed, 16 Nov 2011 11:09:12 -0500, micah anderson mi...@riseup.net wrote:
 On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen t...@iki.fi wrote:
  On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote:
   I really like the feature where you can define quota rules with percents
   which trigger off of the default values[0] (so you can set the Trash to
   allow for 10% more of the user's quota for example). 
   
   What I would really love in dovecot would be for the ability to
   configure a quota rule for dovecot-lda. I would like to configure things
   so we don't bounce emails for users until they are well over quota, the
   IMAP quota plugin is a really great way to notify people that they are
   over quota because it fails to write to other folders that should be
   enough to get people's attention that they need to deal with things, but
   bouncing is harsh.
  
  You can give LDA-specific quota settings. Like maybe:
  
  protocol lda {
plugin {
  quota_rule2 = INBOX:storage=200%
}
  }

I just tried doing this, and I started bouncing mail for everyone
because they were over quota (even when they arent), so I am clearly
missing something in how I have it configured. This is what I have:

namespace {
  separator = .
  prefix =
  inbox = yes
}

namespace {
  separator = .
  prefix = INBOX.
  inbox = no
  hidden = yes
  list = no
}


plugin {
  quota = dict:Your mail quota::proxy::quota
}

dict {
  quota = mysql:/etc/dovecot/dovecot-dict-sql.conf
}

plugin {
  quota_exceeded_message = You are over quota. To avoid losing mail, 
immediately empty your Trash and Sent folders and delete emails with large 
attachments.
  quota_rule = *:bytes=100663296 # in case someone doesn't have sql quota
  quota_rule2 = Trash:bytes=+20%% # let trash be larger by 20%
  quota_rule3 = Spam:bytes=+10%% # let spam be larger by 10%
  quota_rule4 = INBOX.Trash:bytes=+20%% # repeat the above for namespaces
  quota_rule5 = INBOX.Spam:bytes=+10%% # same
  quota_rule6 = INBOX.restored:bytes=+500%% # allow for temporary folder
}

...

protocol lda {
  # Space separated list of plugins to load (default is global mail_plugins).
  mail_plugins = $mail_plugins
  plugin {
quota_rule7 = INBOX:bytes=+10%%
  }
}

Maybe I should have that quota_rule7 be *:bytes=+10%% ?

thanks for any suggestions for what I am missing here!
micah


pgp0AMtVVmc58.pgp
Description: PGP signature

Re: [Dovecot] Indexes to MLC-SSD

[Micah Anderson]

Dovecot-GDH ghandidrivesahum...@rocketfish.com writes:

 If I/O performance is a concern, you may be interested in ZFS and Flashcache.

 Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read 
 Cache)
 ZFS does run on Linux http://zfs-fuse.net

 Flashcache: https://github.com/facebook/flashcache/

That site has no information about what flashcache is.

[Dovecot] doveadm import assertion failed

[Micah Anderson]

Restoring a user's mailbox yesterday resulted in 'doveadm import'
panic'ing with an assertion failure and giving a backtrace:

/usr/bin/doveadm import -u user 
mdbox:/maildir/riseup.net/a/user/.daily.1/mdbox restored_from_backups/daily1 
all
doveadm(user): Error: Transaction log 
/maildir/riseup.net/a/user/.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log:
 duplicate transac
tion log sequence (3)
doveadm(user): Error: Transaction log 
/maildir/riseup.net/a/user/.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log:
 duplicate transac
tion log sequence (3)
doveadm(user): Panic: file mail-transaction-log-file.c: line 187 
(mail_transaction_log_file_add_to_list): assertion failed: ((*p)-hdr.file_se
q  file-hdr.file_seq)
doveadm(user): Error: Raw backtrace: 
/usr/lib/dovecot/libdovecot.so.0(+0x3fd1a) [0x7f05152bbd1a] - 
/usr/lib/dovecot/libdovecot.so.0(default_f
atal_handler+0x32) [0x7f05152bbe02] - 
/usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f051529519f] - 
/usr/lib/dovecot/libdovecot-storage.so
.0(+0xa596a) [0x7f05159e696a] - 
/usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_file_open+0x1f8) 
[0x7f05159e7bc8] - /usr/lib/
dovecot/libdovecot-storage.so.0(mail_transaction_log_find_file+0xd0) 
[0x7f05159e46e0] - /usr/lib/dovecot/libdovecot-storage.so.0(mail_transa
ction_log_view_set+0xcb) [0x7f05159e8bdb] - 
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0xbe) 
[0x7f05159dff6e] - /usr/lib/
dovecot/libdovecot-storage.so.0(mail_index_map+0x86) [0x7f05159d1816] - 
/usr/lib/dovecot/libdovecot-storage.so.0(+0x9a6da) [0x7f05159db6da] 
- /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x56) 
[0x7f05159db956] - /usr/lib/dovecot/libdovecot-storage.so.0(mail_
index_sync_begin+0x1e) [0x7f05159dc21e] - 
/usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x121) 
[0x7f05159bbb41] - /usr/lib/dove
cot/libdovecot-storage.so.0(mdbox_sync+0x46) [0x7f05159bc1f6] - 
/usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x43) [0x7f
05159bc293] - /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) 
[0x7f0515974031] - /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) 
[0x7f05159748b7] - /usr/bin/doveadm(doveadm_mail_iter_init+0x8b) [0x40c9db] - 
/usr/bin/doveadm() [0x40c284] - /usr/bin/doveadm() [0x40a673] - 
/usr/bin/doveadm(doveadm_mail_single_user+0x61) [0x40a731] - 
/usr/bin/doveadm() [0x40a95d] - /usr/bin/doveadm(doveadm_mail_try_run+0x141) 
[0x40ad71] - /usr/bin/doveadm(main+0x381) [0x4107e1] - 
/lib/libc.so.6(__libc_start_main+0xfd) [0x7f0514b2dc4d] - /usr/bin/doveadm() 
[0x409e59]

This is with 2.0.15.

Micah


-- 



pgpF07fgRXSq0.pgp
Description: PGP signature

Re: [Dovecot] dsync problems

[micah anderson]

On Tue, 15 Nov 2011 22:43:24 +0200, Timo Sirainen t...@iki.fi wrote:
 On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote:
  When a user renames their username, I am using dsync to copy their mail
  over to the new username's mail location[0]. 
  
  Some of the dsyncs are failing with errors that I dont know how to work
  with, for example:
  
  dsync(username): Error: Trying to open a non-listed mailbox with 
  guid=41fcd40303c8a64e43237ef44c7a
  dsync(username): Error: msg iteration failed: Couldn't open mailbox 
  41fcd40303c8a64e43237ef44c7a
  dsync(username): Error: Trying to open a non-listed mailbox with 
  guid=41fcd40303c8a64e43237ef44c7a
 
 These shouldn't really happen. Something's going internally wrong with
 dsync. Can you reproduce this reliably somehow?

Well, I dont know if I can do it reliably, but its been happening a
lot. One point of information that might be useful is that these users
were converted from courier maildir to mdbox, and their courier bits are
still around in the source mailbox (I haven't become brave enough to
remove them yet).

 
  The errors cause a non-zero exit code from dsync, which causes my rename
  script to bail out. What are these errors, and how can I fix them?
 
 Does a second dsync on error succeed? :)

Before I tried it again, I looked at their mailboxes:

# doveadm mailbox list -u username
Trash_084ed82bc59ca54eb5377ef44c7a
Sent
Drafts
INBOX_094ed82bc59ca54eb5377ef44c7a
INBOX

Then I tried it again, and I got an error:

dsync(username): Info: INBOX: only in dest 
(guid=14bf0409fa08c04e68297ef44c7a)
dsync(username): Error: Trying to open a non-listed mailbox with 
guid=14bf0409fa08c04e68297ef44c7a
dsync(username): Error: msg iteration failed: Couldn't open mailbox 
14bf0409fa08c04e68297ef44c7a
dsync(username): Error: Trying to open a non-listed mailbox with 
guid=14bf0409fa08c04e68297ef44c7a

and the mailboxes:

# doveadm mailbox list -u username
Trash_084ed82bc59ca54eb5377ef44c7a
Sent
Drafts
INBOX_094ed82bc59ca54eb5377ef44c7a

I tried it a third time, and... it worked, no error, and now their
mailbox list:

# doveadm mailbox list -u username
Trash
Sent
Drafts
INBOX

This particular user only has one test email

 
  0. Why use dsync instead of a simple mv operation? This seems to be
  necessary for two corner cases:
  
  1. dovecot creates the new mailbox automatically when the user logs in
  or receives a mail, so if the user changes their mail and logs in or
  receives an email before the move has been done, then the mailbox is
  created and then a move command will fail.
  
  2. If there has been new mail created under the new name, we can't just
  simply remove the stuff that is automatically created there and replace
  it with the old things because we could potentially be removing mail
  that has been delivered in the mean time.
 
 You could temporarily change the permissions for the home directory so
 that no new mailboxes/mails could be created during the move (e.g. 0700
 root).

The problem is there are a number of users on the system and all the
mail is stored under /srv/mailstorage/letter/username. So if foo
wants to change their username to bar -- I dont have a deterministic
way of determining that bar exists yet because mail could be delivered
or they could login and dovecot would create it and I can't set
/srv/mailstorage/letter 0700 root or nobody would be able to receive
mail.

micah



pgpRFlbnVOgtn.pgp
Description: PGP signature

Re: [Dovecot] dovecot-lda quota rule

[micah anderson]

On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen t...@iki.fi wrote:
 On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote:
  I really like the feature where you can define quota rules with percents
  which trigger off of the default values[0] (so you can set the Trash to
  allow for 10% more of the user's quota for example). 
  
  What I would really love in dovecot would be for the ability to
  configure a quota rule for dovecot-lda. I would like to configure things
  so we don't bounce emails for users until they are well over quota, the
  IMAP quota plugin is a really great way to notify people that they are
  over quota because it fails to write to other folders that should be
  enough to get people's attention that they need to deal with things, but
  bouncing is harsh.
 
 You can give LDA-specific quota settings. Like maybe:
 
 protocol lda {
   plugin {
 quota_rule2 = INBOX:storage=200%
   }
 }

Aha! Just what I was looking for. I presume the number following
'quota_rule' must be an unique number across all quota_rules in my
config?

 or even not enforcing quota at all:
 
 protocol lda {
   plugin {
 quota = maildir:User quota:noenforcing
   }
 }

I was not aware of the noenforcing option, that answered another
question I had not asked yet, so thanks for that! :)

micah


pgpcuI0ivRtaB.pgp
Description: PGP signature

Re: [Dovecot] dovecot-lda quota rule

[micah anderson]

On Wed, 16 Nov 2011 18:27:20 +0200, Timo Sirainen t...@iki.fi wrote:
 On Wed, 2011-11-16 at 11:09 -0500, micah anderson wrote:
   protocol lda {
 plugin {
   quota_rule2 = INBOX:storage=200%
 }
   }
  
  Aha! Just what I was looking for. I presume the number following
  'quota_rule' must be an unique number across all quota_rules in my
  config?
 
 No only unique, but all of the numbers must exist. For example
 quota_rule2 isn't used if quota_rule doesn't exist. Or quota_rule10
 isn't used, unless quota_rule2..9 also exist.

Ah, interesting, good to know. I presume that 'quota_rule' stands in for
'quota_rule1' and one does not need to specify a quota_rule1 if one has
a 'quota_rule'.

micah


pgp86Wcne4iz6.pgp
Description: PGP signature

[Dovecot] dsync problems

[Micah Anderson]

When a user renames their username, I am using dsync to copy their mail
over to the new username's mail location[0]. 

Some of the dsyncs are failing with errors that I dont know how to work
with, for example:

dsync(username): Error: Trying to open a non-listed mailbox with 
guid=41fcd40303c8a64e43237ef44c7a
dsync(username): Error: msg iteration failed: Couldn't open mailbox 
41fcd40303c8a64e43237ef44c7a
dsync(username): Error: Trying to open a non-listed mailbox with 
guid=41fcd40303c8a64e43237ef44c7a
dsync(username): Error: Trying to open a non-listed mailbox with 
guid=42fcd40303c8a64e43237ef44c7a
dsync(username): Error: Trying to open a non-listed mailbox with 
guid=43fcd40303c8a64e43237ef44c7a
dsync(username): Error: Trying to open a non-listed mailbox with 
guid=44fcd40303c8a64e43237ef44c7a
ERROR: dsync failed, exit code: 256

The errors cause a non-zero exit code from dsync, which causes my rename
script to bail out. What are these errors, and how can I fix them?

Thanks,
micah


0. Why use dsync instead of a simple mv operation? This seems to be
necessary for two corner cases:

1. dovecot creates the new mailbox automatically when the user logs in
or receives a mail, so if the user changes their mail and logs in or
receives an email before the move has been done, then the mailbox is
created and then a move command will fail.

2. If there has been new mail created under the new name, we can't just
simply remove the stuff that is automatically created there and replace
it with the old things because we could potentially be removing mail
that has been delivered in the mean time.

I'd be really interested if people had suggestions for a better
mechanism, or perhaps a way to have dovecot not create the new mail
location automatically.

-- 



pgp54e2yahexo.pgp
Description: PGP signature

[Dovecot] Deleting a mailbox and all its children

[Micah Anderson]

I'm looking at scripting a mechanism to delete a mailbox, which is easy
with 'doveadm mailbox delete' -- however it gets complicated when there
are children of arbitrary depth, for example, I may have this:

restored.daily1.INBOX
restored.daily1.Sent
restored.daily1.Mystuff
restored.daily1.Mystuff.foo

I would like to delete the 'restored' mailbox and all its children (and
unsubscribe the folders), but the 'doveadm mailbox delete' command wont
delete any children.

I'm surprised that there isn't a -r (recursive) option to 'doveadm
mailbox delete', but perhaps I am missing the easier way to handle this?

thanks for any ideas or suggestions!
micah



-- 



pgpruGvZod8bb.pgp
Description: PGP signature

Re: [Dovecot] Deleting a mailbox and all its children

[micah anderson]

On Wed, 16 Nov 2011 03:15:02 +0200, Timo Sirainen t...@iki.fi wrote:
 On 16.11.2011, at 3.04, Micah Anderson wrote:
 
  I'm looking at scripting a mechanism to delete a mailbox, which is easy
  with 'doveadm mailbox delete' -- however it gets complicated when there
  are children of arbitrary depth, for example, I may have this:
  
  restored.daily1.INBOX
  restored.daily1.Sent
  restored.daily1.Mystuff
  restored.daily1.Mystuff.foo
  
  I would like to delete the 'restored' mailbox and all its children (and
  unsubscribe the folders), but the 'doveadm mailbox delete' command wont
  delete any children.
  
  I'm surprised that there isn't a -r (recursive) option to 'doveadm
  mailbox delete', but perhaps I am missing the easier way to handle this?
 
 Well, how about something like:
 
 doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete
 doveadm mailbox delete restored

That wont work because 'restored.daily1.Mystuff.foo' wont get removed
before 'restored.daily1.Mystuff' is attempted, resulting in the
following:

doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete
doveadm(micah): Error: Can't delete mailbox restored.daily1.Mystuff: Mailbox 
has children, delete them first

The problem is the subfolders need to be removed first and then walk up
the tree. 

perhaps I can do:

doveadm mailbox list 'restored.*' | sort -r | xargs -n1 doveadm mailbox delete
doveadm mailbox delete restored

although I'm not sure if the reverse sort is going to work in all cases.

micah



pgpjj3dO5EJlx.pgp
Description: PGP signature

[Dovecot] dovecot-lda quota rule

[Micah Anderson]

I really like the feature where you can define quota rules with percents
which trigger off of the default values[0] (so you can set the Trash to
allow for 10% more of the user's quota for example). 

What I would really love in dovecot would be for the ability to
configure a quota rule for dovecot-lda. I would like to configure things
so we don't bounce emails for users until they are well over quota, the
IMAP quota plugin is a really great way to notify people that they are
over quota because it fails to write to other folders that should be
enough to get people's attention that they need to deal with things, but
bouncing is harsh.

Is there a way to do this now that I haven't seen? 

thanks!
micah


0. http://wiki2.dovecot.org/Quota/Configuration
-- 



pgpaJaa1mOJFt.pgp
Description: PGP signature

Re: [Dovecot] Accessing a strange mailbox

[micah anderson]

Hi, thanks for the reply!

On Fri, 04 Nov 2011 21:34:03 +0200, Timo Sirainen t...@iki.fi wrote:
 On Fri, 2011-10-21 at 10:50 -0400, Micah Anderson wrote:
  I have a user who has a mailbox called:
  
  A-B
  
  It seemed to work in courier fine, they managed to create it, and there
  are mails in it. However, dovecot is not letting the user access it, the
  IMAP server gives an error. I tried to rename it but I would also get an
  error:
  
  # doveadm mailbox rename 'A-B' AtoB
  doveadm(root): Error: Can't rename mailbox A-B to AtoB: Mailbox 
  doesn't exist: A-B
 
 Maybe it's not named that? What does it look like in filesystem? Worked
 fine with me:

I moved the user from courier maildir to mdbox, and I've still got the
maildir folders around, the filesystem shows it as:

drwx--   6 mail mail  4096 Oct 20 07:56 .A-B

I just realized my mistake, my 'doveadm mailbox rename' command wasn't
limited to the particular user (I was not passing -u username) so it
was looking in the wrong namespace for that user.

Once I specified the user, it worked properly.

Micah


pgpGTC8lpFFv5.pgp
Description: PGP signature

[Dovecot] Accessing a strange mailbox

[Micah Anderson]

I have a user who has a mailbox called:

A-B

It seemed to work in courier fine, they managed to create it, and there
are mails in it. However, dovecot is not letting the user access it, the
IMAP server gives an error. I tried to rename it but I would also get an
error:

# doveadm mailbox rename A-\B AtoB
doveadm(root): Error: Can't rename mailbox A-B to AtoB: Mailbox doesn't 
exist: A-B
# doveadm mailbox rename A-B AtoB
doveadm(root): Error: Can't rename mailbox A- to AtoB: Mailbox doesn't 
exist: A-
# doveadm mailbox rename 'A-B' AtoB
doveadm(root): Error: Can't rename mailbox A-B to AtoB: Mailbox doesn't 
exist: A-B
# doveadm mailbox rename A-B AtoB
doveadm(root): Error: Can't rename mailbox A-B to AtoB: Mailbox doesn't 
exist: A-B
# 

How can I deal with this mailbox (using mdbox here)?

Thanks!

-- 



pgpVuAtT478AK.pgp
Description: PGP signature

[Dovecot] Safely restoring backups

[Micah Anderson]

I'm trying to determine what the best way to restore mail with mdbox
is. Restoring using maildir was trivial, I just used rsync
--ignore-existing which wrote any mails that were removed and didn't
touch things that already existed[1]. With mdbox things have become more
complicated, and I haven't found a way to restore mail that doesn't
result in many message duplicates.

My backup setup is simple, I'm doing daily rsync backups of user's
mailstores, as well as weekly backups, so I end up having on my backup
server daily.1, daily.2, daily.3, daily.4, weekly.1, weekly.2... each
containing the entire contents of the user's mdbox.

The different restore methods I've tried are:

1. I tried rsyncing the different backup directories back to the mail
storage host, and then doing:

'dsync -R backup -u $user mdbox:/path/to/to/daily.1/mdbox'[2]
'dsync -R backup -u $user mdbox:/path/to/to/daily.2/mdbox'

This works ok, but it gives duplicates of every mail that already exists
for every daily/weekly I dsync. This is what the rsync --ignore-existing
avoided. This is particularly annoying if I restore the weekly, and
multiple daily directories, because then you get a copy for every one
you restore. I had thought that the individual message's GIDs would keep
them from being duplicated?

2. I also tried to use 'doveadm import' in two different ways. The
first way I created a 'restored_from_backups' folder and then import
each of the daily.#, weekly.# mdboxes into their own subfolder within
that folder, for example:

'doveadm import -u $username mdbox:/path/to/daily.1/mdbox 
restored_from_backups/daily1 all'
'doveadm import -u $username mdbox:/path/to/daily.2/mdbox 
restored_from_backups/daily2 all'
... etc.

I then go through and subscribe the new folders[3] and the user ends up
with a folder structure like this:

restored_from_backups/
  daily.1/
INBOX
Trash
folder1
folder2
  daily.2
INBOX
Trash
folder1
folder2
etc.

This works fine, except that it results in the user having a entire
duplicate copies of their mailbox for each daily/weekly that I
restore. That will quickly bring people over quota.

3. I also tried to use 'doveadm import' to import all the different
directories all into the same restored_from_backups directory, so there
are no subdirectories for each daily/weekly under restored_from_backups,
like so:

'doveadm mailbox create -u $user -s restored_from_backups'
'doveadm -v -D import -u $user mdbox:/path/to/daily.1/mdbox 
restored_from_backups all'

I then go through and subscribe the folders[3].

Using this method, the 'restored_from_backups' mailbox is created, and
populated with the folders. The only problem with this method is the
same as method #1: for every backup I restore, mails are duplicated.

Is there a way I can restore things from backup and avoid duplicates?
Is there another method I should try?

thanks for any ideas, pointers, suggestions for improvement, etc.

micah


1. this would of course bring back mails that were deleted, but that was
fine as the user could deal with that.

2. yes, I know I could setup dsync on the backup server, and use dsync
directly to pull the mails from there, but this is complicated in my
situation due to how the backupserver works.

3. why doesn't doveadm import have a -s option to subscribe? 

-- 



pgpt0Z8N5J07x.pgp
Description: PGP signature

Re: [Dovecot] doveadm mailbox subscribe wildcard

[Micah Anderson]

A.L.E.C a...@alec.pl writes:

 On 2011-10-06 23:23, Micah Anderson wrote:
 # doveadm mailbox list -s -u bob dovecot* # doveadm mailbox
 subscribe -u bob dovecot*

 RFC3501. Wildcards are allowed only in LIST/LSUB commands.

Aha, thanks!

I'll use the for loop method to achieve what I want.

micah

-- 


pgp6eY116W31a.pgp
Description: PGP signature

Re: [Dovecot] doveadm mailbox list shows file system directories as mailboxes

[Micah Anderson]

Jürgen Obermann juergen.oberm...@hrz.uni-giessen.de writes:

 Hello.
 In our dovecvot we use mbox format with the default filesystem layout.
 Therefore it is not possible to have mailboxes which are subfolders of  other
 mailboxes containing messages.
 The command doveadm mailbox list includes the file system  directories, that
 contain only subfolders, as normal mailboxes in the  output:

 archiv
 archiv/daemon
 archiv/dovecot

Did you perhaps do a 'doveadm mailbox subscribe' with a wildcard and the
shell expanded it?

micah

[Dovecot] lazy expunge namespaces

[Micah Anderson]

I turned on lazy_expunge and configured a namespace following the
wiki[0] instructions for mdbox configuration:

namespace {  
  inbox = yes
  location =
  prefix =
  separator = .
} 
namespace {
  hidden = yes
  inbox = no
  list = no
  location =
  prefix = INBOX.
  separator = .
} 
namespace {
  location = mdbox:~/mdbox:MAILBOXDIR=expunged
  prefix = EXPUNGED.
  separator = .
} 

This created an EXPUNGED mailbox in my client (thunderbird), which was
greyed out. When I deleted things, they got put in the EXPUNGED mailbox,
as expected, but I could not access them to try and restore them. The
server would say that the mailbox did not exist.

micah

0. http://wiki2.dovecot.org/Plugins/Lazyexpunge
-- 



pgpLuAxubmLRp.pgp
Description: PGP signature

Re: [Dovecot] dsync with quotas

[Micah Anderson]

Micah Anderson mi...@riseup.net writes:

 I have been working on converting people from courier maildir - dovecot
 mdbox and during some of the dsync runs I'm seeing the
 quota_exceeded_message be printed as an Error:

 dsync(user): Error: Can't save message to mailbox INBOX: You are over 
 quota. To avoid losing mail, immediately empty your Trash and Sent folders 
 and \
 delete emails with large attachments. 
 dsync(user): Info: INBOX: Couldn't keep all uids^M

 Its possible the user was over quota on the originating courier side,
 but I would still like to migrate their mail proprely to mdbox, but it
 seems like being over quota is inhibiting that. Are the quota
 calculations including both the maildir files as well as the converted
 mdbox files, resulting in a double counting? 

 I'm not entirely sure if the messages above indicate that the migration
 failed for that user or not, so I've been manually increasing their
 quota, then redoing the dsync mirror until it works properly.

 I wonder if it would be better if I turned off quota entirely during
 migration so I don't run into this problem?

Sorry to 'bump' this subject, but I'm still confused about this issue
and nobody replied, perhaps nobody has any ideas? :)

micah

[Dovecot] dsync with quotas

[Micah Anderson]

I have been working on converting people from courier maildir - dovecot
mdbox and during some of the dsync runs I'm seeing the
quota_exceeded_message be printed as an Error:

dsync(user): Error: Can't save message to mailbox INBOX: You are over quota. 
To avoid losing mail, immediately empty your Trash and Sent folders and \
delete emails with large attachments. 
dsync(user): Info: INBOX: Couldn't keep all uids^M

Its possible the user was over quota on the originating courier side,
but I would still like to migrate their mail proprely to mdbox, but it
seems like being over quota is inhibiting that. Are the quota
calculations including both the maildir files as well as the converted
mdbox files, resulting in a double counting? 

I'm not entirely sure if the messages above indicate that the migration
failed for that user or not, so I've been manually increasing their
quota, then redoing the dsync mirror until it works properly.

I wonder if it would be better if I turned off quota entirely during
migration so I don't run into this problem?

thanks,
micah

-- 



pgprlwVpl83vL.pgp
Description: PGP signature

Re: [Dovecot] v2.0.15 released

[Micah Anderson]

Simon Brereton simon.brere...@buongiorno.com writes:

 I'll bite..  But it's probably not your concern.  When will we be able
 to get stable 2.x packages for Debian?  Apt installs 1.2.15-7

2.x packages are available in Debian Wheezy and Sid. A backport to
Squeeze (stable) will be made once the 1.x transition packages make it
into Wheezy (~10 days from now).

micah

Re: [Dovecot] convert maildrop filter to sieve

[Micah Anderson]

Aleksandar Lazic al-dove...@none.at writes:

 Dear list member,

 does anybody have a script which can convert maildrop filter to sive
 filter?

I would also like this, but I'm afraid it may be too complicated due to
the different possibilities. Moving to sieve from maildrop would be
great, but in practice quite difficult to do when you have a large
number of people with customized mailfilters.

micah

Re: [Dovecot] moving mail out of alt storage

[Micah Anderson]

Timo Sirainen t...@iki.fi writes:

 On Wed, 2011-09-14 at 23:17 -0400, Micah Anderson wrote:
 I moved some mail into the alt storage:
 
 doveadm altmove -u jo...@example.com seen savedbefore 1w
 
 and now I want to move it back to the regular INBOX, but I can't see how
 I can do that with either 'altmove' or 'mailbox move'.

 Is this sdbox or mdbox? With sdbox you could simply mv the files. Or
 apply patch: http://hg.dovecot.org/dovecot-2.0/rev/1910c76a6cc9

This is mdbox, which is why I am not sure how to operate because I am
used to individual files as is with maildir.

micah

[Dovecot] movming mail out of alt storage

[Micah Anderson]

I moved some mail into the alt storage:

doveadm altmove -u jo...@example.com seen savedbefore 1w

and now I want to move it back to the regular INBOX, but I can't see how
I can do that with either 'altmove' or 'mailbox move'.

thanks!
micah


-- 



pgpsYHjQ0YeIm.pgp
Description: PGP signature

Re: [Dovecot] quota percents

[Micah Anderson]

Timo Sirainen t...@iki.fi writes:

 On 12.9.2011, at 19.17, Micah Anderson wrote:

 However, it doesn't seem to work in practice, because I have a user that
 is at 99% of quota, with nothing in the Trash who cannot move a 77KB
 message into the Trash without getting the quota_exceeded message and
 refusing to move it.
 
 dovecot -n output?
 
 namespace {
  inbox = yes
  location = 
  prefix = 
  separator = .
 }
 namespace {
  hidden = yes
  inbox = no
  list = no
  location = 
  prefix = INBOX.
  separator = .
 }

 Maybe the client is using the INBOX. namespace and trying to copy to 
 INBOX.Trash instead of Trash? You could try adding another quota rule for 
 INBOX.Trash. I should probably also change the quota code to follow 
 alias_for directives. Then you could have only a single Trash, if you add 
 alias_for =  to the INBOX. namespace.

ok, I made an INBOX.Trash quota rule:

90-quota.conf:  quota_rule = *:bytes=24117248
90-quota.conf:  quota_rule2 = Trash:bytes=+10%%
90-quota.conf:  quota_rule2 = INBOX.Trash:bytes=+10%%
90-quota.conf:  quota_rule3 = INBOX.Spam:bytes=+20%%

and that seemed to solve the problem!

micah


ps - its very annoying to test quota with thunderbird, it doesn't update
the information very reliably.

-- 

Re: [Dovecot] quota percents

[Micah Anderson]

Timo Sirainen t...@iki.fi writes:

 On Fri, 2011-09-09 at 16:05 -0400, Micah Anderson wrote:

 its incorrect because if you change it to what it suggests ('+10%') it
 wont work because that wont be 10% more, rather, it is set to bytes=+10:

 Well, that's a more complex problem. The %% escaping needs to be done in
 dovecot.conf, but it doesn't need to be done elsewhere. If this said
 that use +10%% and someone gives that in SQL user_query, it will
 break. So I'm not changing the warning.

Ok, that makes sense.

 However, it doesn't seem to work in practice, because I have a user that
 is at 99% of quota, with nothing in the Trash who cannot move a 77KB
 message into the Trash without getting the quota_exceeded message and
 refusing to move it.

 dovecot -n output?

NOTE: i turned off the quota plugins while this is sorted, so you wont
see them enabled below.

# 2.0.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.2 
auth_default_realm = example.net
auth_verbose = yes
default_process_limit = 256
default_vsz_limit = 512 M
dict {
  expire = mysql:/etc/dovecot/dovecot-dict-sql.conf
  quota = mysql:/etc/dovecot/dovecot-dict-sql.conf
}
disable_plaintext_auth = no
first_valid_gid = 8
first_valid_uid = 8
last_valid_gid = 8
last_valid_uid = 8
listen = *
login_greeting = howdy, ready.
mail_location = mdbox:~/mdbox
mail_plugins = expire
maildir_very_dirty_syncs = yes
namespace {
  inbox = yes
  location = 
  prefix = 
  separator = .
}
namespace {
  hidden = yes
  inbox = no
  list = no
  location = 
  prefix = INBOX.
  separator = .
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  expire = Trash
  expire2 = Trash/*
  expire3 = Spam
  expire_dict = proxy::expire
  quota = dict:Your mail quota::proxy::quota
  quota_exceeded_message = You are over quota. To avoid losing mail, 
immediately empty your Trash and Sent folders and delete emails with large 
attachments. 
  quota_rule = *:bytes=24117248
  quota_rule2 = Trash:bytes=+50%%
  quota_rule3 = Spam:bytes=+20%%
}
postmaster_address = postmas...@example.net
protocols = imap pop3
service dict {
  unix_listener dict {
group = mail
mode = 0600
user = mail
  }
}
service imap-login {
  process_min_avail = 10
  service_count = 0
  vsz_limit = 512 M
}
service imap-postlogin {
  executable = script-login /maildir/postlogin_imap
  user = $default_internal_user
}
service pop3-postlogin {
  executable = script-login /maildir/postlogin_pop
  user = $default_internal_user
}
ssl_cert = /etc/certs/wildcard/cert.pem
ssl_cipher_list = HIGH:-ADH:-aNULL
ssl_key = /etc/certs/wildcard/key.pem
userdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
verbose_proctitle = yes
protocol imap {
  imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
  mail_plugins = expire
}
protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_uidl_format = %g
}

[Dovecot] quota percents

[Micah Anderson]

I've noticed that http://wiki2.dovecot.org/Quota/Configuration is out of
date, it says:

plugin {
  quota = maildir:User quota
  quota_rule = *:storage=1GB
  # 10% of 1GB = 100MB
  quota_rule2 = Trash:storage=10%%
  # 20% of 1GB = 200MB
  quota_rule3 = Spam:storage=20%%
}

but if you use the '10%%' notation, dovecot complains, and *also* incorrectly:

dovecot: imap(t...@example.net): Warning: quota root mail quota rule 
Trash:bytes=10%: obsolete configuration for rule 'bytes=10%' should be changed 
to 'bytes=+10%'

its incorrect because if you change it to what it suggests ('+10%') it
wont work because that wont be 10% more, rather, it is set to bytes=+10:

dovecot: imap(t...@example.net): Debug: Quota rule: root=mail quota 
mailbox=Trash bytes=+10 messages=0

clearly, that isn't right... so I guess it needs to be changed to
'+10%%', setting that provides:

dovecot: imap(t...@example.net): Debug: Quota rule: root=mail quota mailbox=* 
bytes=2211724 messages=0
dovecot: imap(t...@example.net): Debug: Quota rule: root=mail quota 
mailbox=Trash bytes=+221172 (10%) messages=0

that seems ok, its saying the Trash mailbox is set to have 221172bytes
(215 kilobytes) of quota. So, clearly
http://wiki2.dovecot.org/Quota/Configuration needs to be updated.

However, it doesn't seem to work in practice, because I have a user that
is at 99% of quota, with nothing in the Trash who cannot move a 77KB
message into the Trash without getting the quota_exceeded message and
refusing to move it.

so... how do I get this to work?

thanks!
micah

-- 



pgpJVKcKQpvwj.pgp
Description: PGP signature

[Dovecot] sorting order changed during migration from courier

[Micah Anderson]

Hello,

I've been working to migrate to dovecot (debian stable: 1:1.2.15-7) from
courier (debian old-stable: 4.4.0-2). I've setup everything and migrated
a test user over to the new system, and ran the
courier-dovecot-migrate.pl script. 

The user is using alpine (the free version of pine) and their sort order
changed on move to dovecot. Originally it was the default in alpine,
which is to sort based on arrival. If he changes the sort order in
alpine to sort based on date, then things are back to how he expects
it. I noticed that there are no known issues with pine
(http://wiki.dovecot.org/Clients#Pine).

Ideally, I would prefer if all the users who are migrated aren't
presented with a different sort order than they were used to, as the
support burden will be quite high. 

What is it about this migration that could have caused that, and more
importantly, how can I fix it? 

I moved the mail from the original system to this system via rsync
-a. The old system was an i686 machine and ext3, this new one is amd64
and ext4.

thanks for any ideas!
micah


# dovecot -n
# 1.2.15: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.2 
protocols: imap imaps pop3 pop3s
disable_plaintext_auth: no
verbose_ssl: yes
login_dir: /var/run/dovecot/login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(pop3): /usr/lib/dovecot/pop3-login
login_greeting: howdy, ready.
login_processes_count: 10
login_max_processes_count: 256
first_valid_uid: 8
last_valid_uid: 8
first_valid_gid: 8
last_valid_gid: 8
mail_uid: 8
mail_gid: 8
mail_location: maildir:~/
maildir_copy_preserve_filename: yes
maildir_very_dirty_syncs: yes
mbox_write_locks: fcntl dotlock
mail_executable(default): /maildir/postlogin_imap
mail_executable(imap): /maildir/postlogin_imap
mail_executable(pop3): /maildir/postlogin_pop
mail_process_size: 512
mail_plugins(default): expire quota imap_quota
mail_plugins(imap): expire quota imap_quota
mail_plugins(pop3): expire quota
mail_plugin_dir(default): /usr/lib/dovecot/modules/imap
mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3
imap_client_workarounds(default): tb-extra-mailbox-sep delay-newmail
imap_client_workarounds(imap): tb-extra-mailbox-sep delay-newmail
imap_client_workarounds(pop3): 
pop3_client_workarounds(default): 
pop3_client_workarounds(imap): 
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
namespace:
  type: private
  separator: .
  prefix: INBOX.
  inbox: yes
  list: yes
  subscriptions: yes
lda:
  postmaster_address: postmas...@riseup.net
auth default:
  default_realm: riseup.net
  verbose: yes
  passdb:
driver: sql
args: /etc/dovecot/dovecot-sql.conf
  userdb:
driver: prefetch
dict:
  expire: mysql:/etc/dovecot/dovecot-dict-expire.conf



-- 



pgpASf7RZiWzd.pgp
Description: PGP signature