[Dovecot] Crash in settings parser (assertion failed: (src_count == ccount))
Lutz Schildt posted this to the Gentoo bugzilla recently: http://bugs.gentoo.org/show_bug.cgi?id=415571 There's a backtrace and conf files attached. I was able to find this possibly-related post from a while ago: http://dovecot.org/list/dovecot/2010-September/052963.html
[Dovecot] Dovecot usage values differ from actual disk usage
Hi,
in my dovecot setup the accounting database table shows wrong
values which do not correspond with the actual disk space used.
The disk usage says 2.6 Gigabytes:
mail01:~# du -sh /mail/dovecot/example.org/username
2.6G /mail/dovecot/example.org/username
While the doveadm quota get says 7 Gigabytes for the same user:
mail01:~# doveadm -f tab quota get -u usern...@example.org
Quota name Type Value Limit %
User quota STORAGE 7065208 - 0
User quota MESSAGE 282024 - 0
mysql> select * from dovecot.dovecot_usage where
username="usern...@example.org";
+--++--+
| username | storage| messages |
+--++--+
| usern...@example.org | 7234773131 | 282024 |
+--++--+
1 row in set (0.00 sec)
What might be the reason for such huge differences (several gigabytes)
between storage value in dovecot_usage table and disk usage value?
After a quota recalc everything is correct:
mail01:~# doveadm quota recalc -u usern...@example.org
mail01:~# doveadm -f tab quota get -u usern...@example.org
Quota name Type Value Limit %
User quota STORAGE 2630434 - 0
User quota MESSAGE 100714 - 0
mysql> select * from dovecot.dovecot_usage where
username="usern...@example.org";
+--++--+
| username | storage| messages |
+--++--+
| usern...@example.org | 2693565239 | 100714 |
+--++--+
1 row in set (0.00 sec)
How can I keep the accounting data consistent with actual disk usage,
without running a recalc cronjob every few hours?
There's running a dovecot director and mailbox instance on each of our servers
with the mails being stored on NFS and received from postfix via LMTP through
the director lmtp port 20024.
Configuration of director (dovecot-director.conf) and
mailbox (dovecot.conf) is attached.
Kind regards
Daniel
# 2.0.20: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS
auth_cache_negative_ttl = 0
auth_cache_size = 10 M
auth_cache_ttl = 1 mins
auth_verbose = yes
auth_verbose_passwords = sha1
deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$
dict {
quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
login_greeting = Mailbox
login_log_format = mailbox: login: %$: %s
login_trusted_networks = 10.129.3.0/24
mail_debug = yes
mail_gid = vmail
mail_home = /mail/dovecot/%d/%n
mail_location = mdbox:~/mail
mail_log_prefix = "mailbox: mail: %s(%u): "
mail_plugins = quota
mail_privileged_group = vmail
mail_uid = vmail
managesieve_implementation_string = Sieve
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
mdbox_rotate_interval = 1 weeks
mdbox_rotate_size = 50 M
mmap_disable = yes
passdb {
args = /etc/dovecot/conf.d/dovecot-sql.conf.ext
driver = sql
}
plugin {
quota = dict:User quota::proxy::quota
quota_rule = *:storage=10G
quota_rule2 = Trash:storage=+100M
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
protocols = imap pop3 lmtp sieve
service dict {
unix_listener dict {
group = vmail
mode = 0660
}
}
service imap-login {
inet_listener imap {
port = 19143
}
}
service lmtp {
inet_listener lmtp {
address = *
port = 19024
}
}
service managesieve-login {
inet_listener sieve {
port = 19200
}
}
service pop3-login {
inet_listener pop3 {
port = 19110
}
}
service quota-warning {
executable = script /usr/local/bin/quota-warning
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl = no
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/conf.d/dovecot-sql.conf.ext
driver = sql
}
verbose_proctitle = yes
protocol imap {
imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
mail_plugins = quota imap_quota
}
protocol lmtp {
mail_plugins = quota sieve
}
# 2.0.20: /etc/dovecot-director/dovecot-director.conf
# OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS
auth_verbose = yes
auth_verbose_passwords = sha1
base_dir = /var/run/dovecot-director
deliver_log_format = director: deliver: msgid=%m from=%f: %$
director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190
director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190
lmtp_proxy = yes
login_greeting = Mail Balancer
login_log_format = director: login: %$: %s
login_trusted_networks = 10.129.3.0/24
mail_debug = yes
mail_gid = vmail
mail_home = /mail/dovecot/%d/%n
mail_location = mdbox:~/mail
mail_log_prefix = "director: mail: %s(%u): "
mail_privileged_g
Re: [Dovecot] index IO patterns
On 5/12/2012 2:32 AM, Cor Bosman wrote: >> Mail is always a random IO workload, unless your mailbox count is 1, >> whether accessing indexes or mail files. Regarding the other two >> questions, you'll likely need to take your own measurements. > > Wait, maybe there is a misunderstanding. I mean the IO inside one > index file, not across the different mailboxes. So within 1 index > file that covers a mailbox with say 10.000 emails, how does the IO > occur. I would guess pretty random as well, but on the other hand > i guess in some ways it could be pretty linear too. If dovecot keeps > most changes in memory and writes it all back in 1 go. I don't see how this is relevant to designing an index storage system. Whether index file updates are sequential or random, they become random at the 2nd user and more so from there. So either way, your storage system will see a random IO pattern, and that's what you need to engineer the system for, not the single user index file update pattern. You've already expressed interest in SSD, which takes care of this concern. -- Stan
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 20:42, Reindl Harald wrote: > > Am 12.05.2012 20:35, schrieb Luuk@dovecot: >> On 12-05-2012 19:48, Gedalya wrote: >>> >>> I understand you sent an email to yourself? >>> If you gained one $message_size upon reception and lost 2*$message_size >>> upon deletion from Inbox+Sent, I would suspect quota plugin isn't active >>> when the message is being delivered. How are messages delivered? Dovecot >>> LDA, LMTP or an external MTA? >>> You really should use LDA or LMTP to handle quota accounting. >> >> Yes, i was sending it to meself... >> Postfix delivers mail using procmail. >> Next thing i need to read about mailservers are the LDA/LMTP pages > > this should be one of the first things > > nobody needs procmail/shellscripts to get quota with postfix > and a MDA with LMPT support wokring these days > > usually the better way implementing services is to draw > what exactly is needed and read docs how all this things > can be achived Everything (besides quota ;) is working as needed > > example from dbmail (dovecot should be the same) > /etc/postfix/master.cf: > dbmail-lmtp unix - - n -- lmtp -o > disable_dns_lookups=yes > > set the postfix-transport to "dbmail-lmtp:127.0.0.1:24" for each domain which > should > be delivered to LMTP and you are done, "dbmail-lmtp" is only a name > > > > i have a line like that in my master.cf, i reads: lmtp unix - - n - - lmtp but (apparantly) is not used. Before starting to use that, i should make sure all received mail keeps coming in at the right place...
Re: [Dovecot] Quota, message is never send?
Am 12.05.2012 20:35, schrieb Luuk@dovecot: > On 12-05-2012 19:48, Gedalya wrote: >> >> I understand you sent an email to yourself? >> If you gained one $message_size upon reception and lost 2*$message_size >> upon deletion from Inbox+Sent, I would suspect quota plugin isn't active >> when the message is being delivered. How are messages delivered? Dovecot >> LDA, LMTP or an external MTA? >> You really should use LDA or LMTP to handle quota accounting. > > Yes, i was sending it to meself... > Postfix delivers mail using procmail. > Next thing i need to read about mailservers are the LDA/LMTP pages this should be one of the first things nobody needs procmail/shellscripts to get quota with postfix and a MDA with LMPT support wokring these days usually the better way implementing services is to draw what exactly is needed and read docs how all this things can be achived example from dbmail (dovecot should be the same) /etc/postfix/master.cf: dbmail-lmtp unix - - n -- lmtp -o disable_dns_lookups=yes set the postfix-transport to "dbmail-lmtp:127.0.0.1:24" for each domain which should be delivered to LMTP and you are done, "dbmail-lmtp" is only a name signature.asc Description: OpenPGP digital signature
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 19:48, Gedalya wrote: > > I understand you sent an email to yourself? > If you gained one $message_size upon reception and lost 2*$message_size > upon deletion from Inbox+Sent, I would suspect quota plugin isn't active > when the message is being delivered. How are messages delivered? Dovecot > LDA, LMTP or an external MTA? > You really should use LDA or LMTP to handle quota accounting. > Yes, i was sending it to meself... Postfix delivers mail using procmail. Next thing i need to read about mailservers are the LDA/LMTP pages
Re: [Dovecot] Quota, message is never send?
On 5/12/2012 1:34 PM, Luuk@dovecot wrote:
On 12-05-2012 18:40, Gedalya wrote:
On 5/12/2012 12:21 PM, Luuk@dovecot wrote:
On 12-05-2012 18:00, Gedalya wrote:
On 5/12/2012 11:42 AM, Luuk@dovecot wrote:
In the log, the following message showed:
2012-05-12 17:11:43 imap(luuk): Error: quota:
net_connect_unix(/var/run/dovecot/quota-warning.sh) failed: No such
file
or directory
I think i have to revert the changes, and retest.;)
Wiki says: quota_warning =
You define a listening socket in the unix_listener line in the service
definition, and refer to that socket in your quota_warning plugin
configuration.
Duane wasn't saying to delete the path, he said to use the socket name.
So, just quota-warning without the .sh
plugin {
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=90%% quota-warning 90 %u
quota_warning3 = storage=10%% quota-warning 10 %u
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = dovecot
}
user = dovecot
}
ok, after this error, i must have seen them all:
2012-05-12 18:08:31 imap(luuk): Error: quota:
net_connect_unix(/var/run/dovecot/quota-warning) failed: Permission
denied (euid=1000(luuk) egid=100(users) missing +r perm:
/var/run/dovecot/quota-warning, dir owned by 0:0 mode=0755)
opensuse:/var/run/dovecot # ls -l /var/run/dovecot/quota-warning
srw--- 1 dovecot root 0 May 12 18:18 /var/run/dovecot/quota-warning
opensuse:/var/run/dovecot #
Changing the permissions using chmod +rw /var/run/dovecot/quota-warning
is reset when dovecot is restarted.
You should probably add mode = 0666 like so:
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = dovecot
mode = 0666
}
user = dovecot
}
Indeed you can't use chmod from the command line because these sockets
are created by dovecot when it starts up.
ok, it seems to work now ;)
but my quota-usage is dropping harder that it should
i started at 71%,
received a file, it went to 74%
deleted the message in Inbox
deleted the message from Sent
and not the quota is at 68%.
Recovery is possible with:
doveadm quota recalc -u
I understand you sent an email to yourself?
If you gained one $message_size upon reception and lost 2*$message_size
upon deletion from Inbox+Sent, I would suspect quota plugin isn't active
when the message is being delivered. How are messages delivered? Dovecot
LDA, LMTP or an external MTA?
You really should use LDA or LMTP to handle quota accounting.
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 18:40, Gedalya wrote:
> On 5/12/2012 12:21 PM, Luuk@dovecot wrote:
>> On 12-05-2012 18:00, Gedalya wrote:
>>> On 5/12/2012 11:42 AM, Luuk@dovecot wrote:
In the log, the following message showed:
2012-05-12 17:11:43 imap(luuk): Error: quota:
net_connect_unix(/var/run/dovecot/quota-warning.sh) failed: No such
file
or directory
I think i have to revert the changes, and retest.;)
>>> Wiki says: quota_warning = >> name>
>>>
>>>
>>> You define a listening socket in the unix_listener line in the service
>>> definition, and refer to that socket in your quota_warning plugin
>>> configuration.
>>>
>>> Duane wasn't saying to delete the path, he said to use the socket name.
>>> So, just quota-warning without the .sh
>>>
>>> plugin {
>>>quota_warning = storage=95%% quota-warning 95 %u
>>>quota_warning2 = storage=90%% quota-warning 90 %u
>>>quota_warning3 = storage=10%% quota-warning 10 %u
>>> }
>>>
>>> service quota-warning {
>>>executable = script /usr/local/bin/quota-warning.sh
>>>unix_listener quota-warning {
>>> user = dovecot
>>>}
>>>user = dovecot
>>> }
>>>
>>>
>> ok, after this error, i must have seen them all:
>> 2012-05-12 18:08:31 imap(luuk): Error: quota:
>> net_connect_unix(/var/run/dovecot/quota-warning) failed: Permission
>> denied (euid=1000(luuk) egid=100(users) missing +r perm:
>> /var/run/dovecot/quota-warning, dir owned by 0:0 mode=0755)
>>
>> opensuse:/var/run/dovecot # ls -l /var/run/dovecot/quota-warning
>> srw--- 1 dovecot root 0 May 12 18:18 /var/run/dovecot/quota-warning
>> opensuse:/var/run/dovecot #
>>
>> Changing the permissions using chmod +rw /var/run/dovecot/quota-warning
>> is reset when dovecot is restarted.
>>
>
> You should probably add mode = 0666 like so:
>
> service quota-warning {
> executable = script /usr/local/bin/quota-warning.sh
> unix_listener quota-warning {
> user = dovecot
> mode = 0666
> }
> user = dovecot
> }
>
> Indeed you can't use chmod from the command line because these sockets
> are created by dovecot when it starts up.
>
ok, it seems to work now ;)
but my quota-usage is dropping harder that it should
i started at 71%,
received a file, it went to 74%
deleted the message in Inbox
deleted the message from Sent
and not the quota is at 68%.
Recovery is possible with:
doveadm quota recalc -u
Re: [Dovecot] index IO patterns
On 5/12/2012 2:26 AM, Cor Bosman wrote: > The indexes are doing a lot of iops on the metrocluster, and it's a bit > of an expensive option for something it's not even that good at. This clears things up a bit. > Im aiming for something with 2 servers, each with a 12 disk enclosure > with SSD for fast random io with 10G network interfaces, 24 core, 48GB > memory. AMD is a great platform and I laud your preference for it. > I just want to test some io patterns on different hardware/software > solutions, including the metrocluster itself, before we commit to > a specific solution. Im slightly leaning towards DRBD right now. A DRBD cluster simply doubles your costs--twice the disks/enclosures, twice the servers, and adds another layer of redundancy software to the storage stack. It can be even more if one decides to cluster 3-6 or more DRBD servers. Have you considered something like a Nexsan E18? In 2U it gives you dual PSUs, dual active/active RAID controllers each w/ 2GB BBWC, 2x8Gb FC and 2x1GbE iSCSI ports per controller. Optionally you can replace the FC ports with the same number of 10GbE iSCSI ports. It offers up to 18 100/200/400GB SLC SSDs, or up to 36/78 of these SSDs w/the E18X or E60X expansion chassis. http://www.nexsan.com/en/products/e-series/~/media/Nexsan/Files/products/e-series/library/NexsanESeriesDSpdf.ashx http://www.nexsan.com/en/products/e-series/tech-specs.aspx http://www.nexsan.com/products/e-series.aspx You'd simply create a single RAID1+0 array of all 18 SSDs, export it as a LUN on each iSCSI port, configure SCSI multipath and the iSCSI initiator on each Dovecot host, install GFS2/OCFS2, format the LUN and go. With 18x200GB SSDs you'll get 1.8T of net capacity and well north of 100K sustained real world random r/w block IOPS. And without needing two beefy dual socket AMD server chassis mirrored with DRBD. And of course you'll still want to use Dovecot Director to avoid locking issues. Contact the Nexsan European office to see about an evaluation unit: http://www.nexsan.com/about/contact/locations.aspx Disclaimer: I've never worked for Nexsan nor any affiliate. I'm simply a past customer very satisfied with their products and philosophy/strategy. -- Stan
Re: [Dovecot] Quota, message is never send?
On 5/12/2012 12:21 PM, Luuk@dovecot wrote:
On 12-05-2012 18:00, Gedalya wrote:
On 5/12/2012 11:42 AM, Luuk@dovecot wrote:
In the log, the following message showed:
2012-05-12 17:11:43 imap(luuk): Error: quota:
net_connect_unix(/var/run/dovecot/quota-warning.sh) failed: No such file
or directory
I think i have to revert the changes, and retest.;)
Wiki says: quota_warning =
You define a listening socket in the unix_listener line in the service
definition, and refer to that socket in your quota_warning plugin
configuration.
Duane wasn't saying to delete the path, he said to use the socket name.
So, just quota-warning without the .sh
plugin {
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=90%% quota-warning 90 %u
quota_warning3 = storage=10%% quota-warning 10 %u
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = dovecot
}
user = dovecot
}
ok, after this error, i must have seen them all:
2012-05-12 18:08:31 imap(luuk): Error: quota:
net_connect_unix(/var/run/dovecot/quota-warning) failed: Permission
denied (euid=1000(luuk) egid=100(users) missing +r perm:
/var/run/dovecot/quota-warning, dir owned by 0:0 mode=0755)
opensuse:/var/run/dovecot # ls -l /var/run/dovecot/quota-warning
srw--- 1 dovecot root 0 May 12 18:18 /var/run/dovecot/quota-warning
opensuse:/var/run/dovecot #
Changing the permissions using chmod +rw /var/run/dovecot/quota-warning
is reset when dovecot is restarted.
You should probably add mode = 0666 like so:
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = dovecot
mode = 0666
}
user = dovecot
}
Indeed you can't use chmod from the command line because these sockets
are created by dovecot when it starts up.
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 18:00, Gedalya wrote:
> On 5/12/2012 11:42 AM, Luuk@dovecot wrote:
>> In the log, the following message showed:
>> 2012-05-12 17:11:43 imap(luuk): Error: quota:
>> net_connect_unix(/var/run/dovecot/quota-warning.sh) failed: No such file
>> or directory
>>
>> I think i have to revert the changes, and retest.;)
>
> Wiki says: quota_warning = name>
>
>
> You define a listening socket in the unix_listener line in the service
> definition, and refer to that socket in your quota_warning plugin
> configuration.
>
> Duane wasn't saying to delete the path, he said to use the socket name.
> So, just quota-warning without the .sh
>
> plugin {
> quota_warning = storage=95%% quota-warning 95 %u
> quota_warning2 = storage=90%% quota-warning 90 %u
> quota_warning3 = storage=10%% quota-warning 10 %u
> }
>
> service quota-warning {
> executable = script /usr/local/bin/quota-warning.sh
> unix_listener quota-warning {
> user = dovecot
> }
> user = dovecot
> }
>
>
ok, after this error, i must have seen them all:
2012-05-12 18:08:31 imap(luuk): Error: quota:
net_connect_unix(/var/run/dovecot/quota-warning) failed: Permission
denied (euid=1000(luuk) egid=100(users) missing +r perm:
/var/run/dovecot/quota-warning, dir owned by 0:0 mode=0755)
opensuse:/var/run/dovecot # ls -l /var/run/dovecot/quota-warning
srw--- 1 dovecot root 0 May 12 18:18 /var/run/dovecot/quota-warning
opensuse:/var/run/dovecot #
Changing the permissions using chmod +rw /var/run/dovecot/quota-warning
is reset when dovecot is restarted.
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 18:02, Gedalya wrote: > On 5/12/2012 11:59 AM, Luuk@dovecot wrote: >> 2012-05-12 17:50:22 imap(luuk): Error: quota: >> net_connect_unix(/usr/local/bin/quota-warning.sh) failed: Permission >> denied (euid=1000(luuk) egid=100(users) missing +w perm: >> /usr/local/bin/quota-warning.sh, dir owned by 0:0 mode=0755) > > This is net_connect_unix(), it's trying to write to a socket. But it > ended up with a path to a script, not a socket. > i just read you other post, and re-test ;)
Re: [Dovecot] Quota, message is never send?
On 5/12/2012 11:59 AM, Luuk@dovecot wrote: 2012-05-12 17:50:22 imap(luuk): Error: quota: net_connect_unix(/usr/local/bin/quota-warning.sh) failed: Permission denied (euid=1000(luuk) egid=100(users) missing +w perm: /usr/local/bin/quota-warning.sh, dir owned by 0:0 mode=0755) This is net_connect_unix(), it's trying to write to a socket. But it ended up with a path to a script, not a socket.
Re: [Dovecot] Quota, message is never send?
On 5/12/2012 11:42 AM, Luuk@dovecot wrote:
In the log, the following message showed:
2012-05-12 17:11:43 imap(luuk): Error: quota:
net_connect_unix(/var/run/dovecot/quota-warning.sh) failed: No such file
or directory
I think i have to revert the changes, and retest.;)
Wiki says: quota_warning = name>
You define a listening socket in the unix_listener line in the service
definition, and refer to that socket in your quota_warning plugin
configuration.
Duane wasn't saying to delete the path, he said to use the socket name.
So, just quota-warning without the .sh
plugin {
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=90%% quota-warning 90 %u
quota_warning3 = storage=10%% quota-warning 10 %u
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = dovecot
}
user = dovecot
}
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 17:42, Luuk@dovecot wrote:
> On 12-05-2012 15:55, Gedalya wrote:
>> On 05/12/2012 04:50 AM, Luuk@dovecot wrote:
>>> It seems i am at 79% of my quota
>>> so, i added a line to conf.d/90-quota.conf, to get a wraning when quota
>>> exceeds 10%:
>> Quota warnings are only sent when you _cross_ the limit, they are not
>> sent (again) when you are already over the limit.
>> So, in your case you will get your next warnings at 90% and 95%.
>>
>
> OK, so i tweaked my rules a bit, and send meself a huge mail to make
> sure i cross a border. (I've overdone this, because the pdf got base64
> encoded, and stored in Sent AND in INBOX ;)
>
>
> I also followed the suggestion form 'Duane Hill' and delete the path
> before the quota-warning.sh. I hope he did mean that when he was
> referring to the docs ;)
>
> In the log, the following message showed:
> 2012-05-12 17:11:43 imap(luuk): Error: quota:
> net_connect_unix(/var/run/dovecot/quota-warning.sh) failed: No such file
> or directory
>
> I think i have to revert the changes, and retest. ;)
>
2012-05-12 17:50:22 imap(luuk): Error: quota:
net_connect_unix(/usr/local/bin/quota-warning.sh) failed: Permission
denied (euid=1000(luuk) egid=100(users) missing +w perm:
/usr/local/bin/quota-warning.sh, dir owned by 0:0 mode=0755)
^C
opensuse:/etc/dovecot # ll /usr/local/bin/quota-warning.sh
-rwxr-xr-x 1 root root 301 Apr 9 16:09 /usr/local/bin/quota-warning.sh
Why does this script need +w ?? or am i misreading the error message?
opensuse:/home/luuk # dovecot -n | grep -i quota
mail_plugins = mail_log notify fts fts_squat quota
quota = maildir:User quota
quota_rule = *:storage=800M
quota_rule2 = *:messages=6
quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 %u
quota_warning2 = storage=90%% /usr/local/bin/quota-warning.sh 90 %u
quota_warning3 = storage=72%% /usr/local/bin/quota-warning.sh 72 %u
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
mail_plugins = mail_log notify fts fts_squat quota imap_quota
opensuse:/home/luuk #
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 15:55, Gedalya wrote: > On 05/12/2012 04:50 AM, Luuk@dovecot wrote: >> It seems i am at 79% of my quota >> so, i added a line to conf.d/90-quota.conf, to get a wraning when quota >> exceeds 10%: > Quota warnings are only sent when you _cross_ the limit, they are not > sent (again) when you are already over the limit. > So, in your case you will get your next warnings at 90% and 95%. > OK, so i tweaked my rules a bit, and send meself a huge mail to make sure i cross a border. (I've overdone this, because the pdf got base64 encoded, and stored in Sent AND in INBOX ;) I also followed the suggestion form 'Duane Hill' and delete the path before the quota-warning.sh. I hope he did mean that when he was referring to the docs ;) In the log, the following message showed: 2012-05-12 17:11:43 imap(luuk): Error: quota: net_connect_unix(/var/run/dovecot/quota-warning.sh) failed: No such file or directory I think i have to revert the changes, and retest. ;)
Re: [Dovecot] Quota, message is never send?
On 05/12/2012 04:50 AM, Luuk@dovecot wrote: It seems i am at 79% of my quota so, i added a line to conf.d/90-quota.conf, to get a wraning when quota exceeds 10%: Quota warnings are only sent when you _cross_ the limit, they are not sent (again) when you are already over the limit. So, in your case you will get your next warnings at 90% and 95%.
Re: [Dovecot] Quota, message is never send?
On Saturday, May 12, 2012 at 12:37:25 UTC, dove...@vosslamber.nl confabulated:
> On 12-05-2012 10:50, Luuk@dovecot wrote:
>> I think i have quota setup correctly, but finally time arrived to check
>> that ;)
>>
> ...
>> But this message is never send, what am i missing (in my config)?
>>
> dovecot -n:
> # 2.0.16: /etc/dovecot/dovecot.conf
> # OS: Linux 3.1.10-1.9-desktop x86_64 openSUSE 12.1 (x86_64)
> auth_debug = yes
> auth_mechanisms = plain login cram-md5
> info_log_path = /var/log/dovecot/dovecot-debug.log
> log_path = /var/log/dovecot/dovecot.log
> log_timestamp = "%Y-%m-%d %H:%M:%S "
> mail_debug = yes
> mail_location = maildir:~/Maildir
> mail_plugins = mail_log notify fts fts_squat quota
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope
> encoded-character vacation subaddress comparator-i;ascii-numeric
> relational regex imap4flags copy include variables body enotify
> environment mailbox date ihave
> passdb {
> args = scheme=cram-md5 /etc/cram-md5.pwd
> driver = passwd-file
> }
> plugin {
> quota = maildir:User quota
> quota_rule = *:storage=700M
> quota_rule2 = *:messages=6
> quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 %u
> quota_warning2 = storage=90%% /usr/local/bin/quota-warning.sh 90 %u
> quota_warning3 = storage=10%% /usr/local/bin/quota-warning.sh 10 %u
> sieve = ~/.dovecot.sieve
> sieve_dir = ~/sieve
> }
> protocols = imap lmtp
> service imap-login {
> inet_listener imap {
> port = 143
> }
> inet_listener imaps {
> port = 993
> ssl = yes
> }
> process_min_avail = 0
> service_count = 1
> }
> service quota-warning {
> executable = script /usr/local/bin/quota-warning.sh
> unix_listener quota-warning {
> user = dovecot
> }
> user = dovecot
> }
> ssl_cert = ssl_key = userdb {
> driver = passwd
> }
> verbose_proctitle = yes
> protocol imap {
> mail_plugins = mail_log notify fts fts_squat quota imap_quota
> }
Tell the quota warnings to use the quota-warning service you have
defined:
plugin {
...
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=90%% quota-warning 90 %u
quota_warning3 = storage=10%% quota-warning 10 %u
...
}
That's the way it is demonstrated in the default configuration files.
--
If at first you don't succeed...
...so much for skydiving.
Re: [Dovecot] Quota, message is never send?
On 12-05-2012 10:50, Luuk@dovecot wrote:
> I think i have quota setup correctly, but finally time arrived to check
> that ;)
>
...
> But this message is never send, what am i missing (in my config)?
>
dovecot -n:
# 2.0.16: /etc/dovecot/dovecot.conf
# OS: Linux 3.1.10-1.9-desktop x86_64 openSUSE 12.1 (x86_64)
auth_debug = yes
auth_mechanisms = plain login cram-md5
info_log_path = /var/log/dovecot/dovecot-debug.log
log_path = /var/log/dovecot/dovecot.log
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_debug = yes
mail_location = maildir:~/Maildir
mail_plugins = mail_log notify fts fts_squat quota
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
passdb {
args = scheme=cram-md5 /etc/cram-md5.pwd
driver = passwd-file
}
plugin {
quota = maildir:User quota
quota_rule = *:storage=700M
quota_rule2 = *:messages=6
quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 %u
quota_warning2 = storage=90%% /usr/local/bin/quota-warning.sh 90 %u
quota_warning3 = storage=10%% /usr/local/bin/quota-warning.sh 10 %u
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
protocols = imap lmtp
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
process_min_avail = 0
service_count = 1
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = dovecot
}
user = dovecot
}
ssl_cert =
[Dovecot] Quota, message is never send?
I think i have quota setup correctly, but finally time arrived to check
that ;)
opensuse:/etc/dovecot # doveadm quota get -u luuk
Quota name Type Value Limit %
User quota STORAGE 571973 716800 79
User quota MESSAGE 4 - 0
It seems i am at 79% of my quota
so, i added a line to conf.d/90-quota.conf, to get a wraning when quota
exceeds 10%:
plugin {
quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 %u
quota_warning2 = storage=90%% /usr/local/bin/quota-warning.sh 90 %u
quota_warning3 = storage=10%% /usr/local/bin/quota-warning.sh 10 %u
}
But this message is never send, what am i missing (in my config)?
Below is some info from a debug.log:
2012-05-12 10:40:51 imap(luuk): Debug: Effective uid=1000, gid=100,
home=/home/luuk
2012-05-12 10:40:51 imap(luuk): Debug: Quota root: name=User quota
backend=maildir args=
2012-05-12 10:40:51 imap(luuk): Debug: Quota rule: root=User quota
mailbox=* bytes=734003200 messages=0
2012-05-12 10:40:51 imap(luuk): Debug: Quota warning: bytes=697303040
(95%) messages=0 reverse=no command=/usr/local/bin/quota-warning.sh 95 luuk
2012-05-12 10:40:51 imap(luuk): Debug: Quota warning: bytes=660602880
(90%) messages=0 reverse=no command=/usr/local/bin/quota-warning.sh 90 luuk
2012-05-12 10:40:51 imap(luuk): Debug: Quota warning: bytes=73400320
(10%) messages=0 reverse=no command=/usr/local/bin/quota-warning.sh 10 luuk
2012-05-12 10:40:51 imap(luuk): Debug: maildir++:
root=/home/luuk/Maildir, index=, control=, inbox=/home/luuk/Maildir, alt=
Re: [Dovecot] index IO patterns
On 12.5.2012, at 10.32, Cor Bosman wrote: >> Mail is always a random IO workload, unless your mailbox count is 1, >> whether accessing indexes or mail files. Regarding the other two >> questions, you'll likely need to take your own measurements. > > Wait, maybe there is a misunderstanding. I mean the IO inside one > index file, not across the different mailboxes. So within 1 index > file that covers a mailbox with say 10.000 emails, how does the IO > occur. I would guess pretty random as well, but on the other hand > i guess in some ways it could be pretty linear too. If dovecot keeps > most changes in memory and writes it all back in 1 go. Usually the index files are small enough that I think OS reads the whole files into memory anyway. Anyway..: * dovecot.index: The header is always accessed first. After that it's accessed as necessary. Many IMAP clients fetch all message flags when selecting mailbox, so this causes a sequential read of the entire file. Also with mmap_disable=yes the whole file is always read into memory. * dovecot.index.log: Usually the last few kilobytes of the file are read into memory when mailbox is opened, and after that data is appended and read from it. In some situations the reader might seek to an older data (e.g. to beginning) and read the rest of the file sequentially. * dovecot.index.cache: Accessed randomly, depending on what data is needed to be looked up. Typically clients fetch only the last few messages, so the end of the file is accessed sequentially. Writes are typically appends + rewrites, but currently there are also a few more complex things which I want to get rid of (perhaps for v2.2).
Re: [Dovecot] index IO patterns
> Mail is always a random IO workload, unless your mailbox count is 1, > whether accessing indexes or mail files. Regarding the other two > questions, you'll likely need to take your own measurements. Wait, maybe there is a misunderstanding. I mean the IO inside one index file, not across the different mailboxes. So within 1 index file that covers a mailbox with say 10.000 emails, how does the IO occur. I would guess pretty random as well, but on the other hand i guess in some ways it could be pretty linear too. If dovecot keeps most changes in memory and writes it all back in 1 go. Cor
Re: [Dovecot] index IO patterns
> >> Alternatively, does anyone have any experience with other redundant storage >> options? Im thinking things like MooseFS, DRBD, etc? > > You seem to be interested in multi-site clustering/failover solutions, > not simply redundant storage. These two are clustering software > solutions but DRBD is not suitable for multi-site use, and MooseFS > doesn't seem to be either. MooseFS is based heavily on FUSE, so > performance will be far less than optimal. MooseFS is a distributed > filesystem, and as with all other distributed/cluster filesystems its > metadata performance will suffer, eliminating maildir as a mail store > option. > > Can you provide more specifics on your actual storage architecture needs? There are some people in our company that like MooseFS, so i'll just include it in the tests and let that speak for itself :) We are not looking for multisite solutions. Then we may as well stay with the metrocluster. I dont even care if it has to be in the same rack. It's only for the indexes, not the mail store itself which will stay on the metrocluster. In the very worst case, when the whole site explodes, i can always tell dovecot to use memory for indexes temporarily :) The indexes are doing a lot of iops on the metrocluster, and it's a bit of an expensive option for something it's not even that good at. Im aiming for something with 2 servers, each with a 12 disk enclosure with SSD for fast random io with 10G network interfaces, 24 core, 48GB memory. I just want to test some io patterns on different hardware/software solutions, including the metrocluster itself, before we commit to a specific solution. Im slightly leaning towards DRBD right now. Cor
