Re: Does domain quota work and how?
Kai Schaetzl wrote on Mon, 20 Aug 2018 13:32:59 +0200: > So, is this config correct for domain quota dictionary-based and how > should it work? Hello, resolving a problem and knowing that there is a problem are two different things. I would still like to know if this config is correct and how the domain quota should work, e.g. either user quota or domain quota or are user and domain quota handled on the same level and if either one is hit the quota is reached? Only then I would be able to check if the quota handling is working like it should. As an additional information. I noticed today that I didn't get any figures in the quota and quota_domain tables. Actually, I had noticed this earlier, but didn't find the time to troubleshoot this. But doveadm was showing me figures. As it turned out I had still set the connect string only for these two tables to the testing database while all other connect strings pointed to the correct database. I simply changed the two files and dovecot started immediately to use the different database, without a restart. The tables where empty at this stage. Dovecot recalculated the quotas and filled the tables. Good. After some checking I can confirm that the user quotas seem to be correct, e.g. I counted the message numbers for each user of a domain and compared it with the table. That's easier than to compare sizes. But the domain quotas are not correct. Dovecot used the quota of the *first* user to fill the domain quota. It didn't add the other users to it. Note, this is without doveadm, just happens if Dovecot encounters empty records I suppose. So, the domain quota is getting recalculated wrongly, but not (as I said earlier about doveadm) by using the last user, but the first. Ok, then I tried a doveadm quota recalc -A again and, lo and behold, it changed the domain quota values to show the last user's quota. So, the recalc in dovecot itself uses the first user, the recalc in doveadm uses the last user. But both are wrong. Kai
Re: Does domain quota work and how?
On 20.08.2018 14:32, Kai Schaetzl wrote: > So, is this config correct for domain quota dictionary-based and how > should it work? > > Kai > > > We are still investigating your issue. Aki
Re: Does domain quota work and how?
So, is this config correct for domain quota dictionary-based and how should it work? Kai
Re: Does domain quota work and how?
Aki Tuomi wrote on Thu, 16 Aug 2018 21:03:44 +0300 (EEST):
> Can you provide doveconf -n please?
Thanks for the quick reply. I think I already provided the most relevant
portions of config. I ran a doveconf -n and it doesn't reveal important
information like sql setup. I'll do my best to sum up the relevant parts
below, anyway.
But first I would like to know how the dictionary-based domain quota is
intended to work as that is mentioned nowhere. Is it like in the either/or
example where they say that dovecot can only act on one of the quotas at a
given time or is it intended to be used like the filesystem aquota works?
So, there are two things:
First: I do not quite understand how it is supposed to work. Once I knew I
could test my setup whether it acts on the limits correctly.
The second is the apparent bug with recalc. I omitted to say that the
updating of both quotas by dovecot when mail comes in *does* seem to work
correctly. But if I run a recalc it all gets messed up. This has been
reported a few times over time, so I guess it's not specific to my setup.
So, here's the relevant bits again (and a bit more):
dict {
quota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf
quota_domain = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf
}
mail_home = /var/vmail/%d/%n
mail_location = maildir:/var/vmail/%d/%n/mail/
mail_plugins = " quota"
plugin {
quota = dict:user::proxy::quota
quota2 = dict:domain:%d:proxy::quota_domain
quota_rule = *:storage=100M
quota_rule2 = Trash:storage=+10%%
quota_rule3 = Spam:ignore
quota2_rule = *:storage=1000M
quota2_rule2 = Trash:storage=+10%%
quota2_rule3 = Spam:ignore
quota_vsizes = yes
(+ quota_status, _warning etc. not relevant ...)
}
service dict {
unix_listener dict {
group = vmail
mode = 0600
user = vmail
}
}
service quota-status {
client_limit = 1
executable = quota-status -p postfix
inet_listener {
port = 10024
}
}
userdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
/etc/dovecot/dovecot-dict-sql-user.conf
connect = ...
map {
pattern = priv/quota/storage
table = quota
username_field = username
value_field = bytes
}
map {
pattern = priv/quota/messages
table = quota
username_field = username
value_field = messages
}
/etc/dovecot/dovecot-dict-sql-domain.conf
connect =
map {
pattern = priv/quota/storage
table = quota_domain
username_field = domain
value_field = bytes
}
map {
pattern = priv/quota/messages
table = quota_domain
username_field = domain
value_field = messages
}
/etc/dovecot/dovecot-sql.conf
user_query = SELECT CONCAT('/var/vmail/',m.maildir) AS home, CONCAT
('maildir:/var/vmail/',m.maildir,'mail/') AS mail, 200 AS uid, 200 AS gid,
concat('*:bytes=', m.quota) AS quota_rule, concat('*:bytes=',
(s.ServerPopQuota*1024*1024)) AS quota2_rule FROM mailbox AS m LEFT JOIN
server AS s ON (m.ServerID = s.ServerID) WHERE (m.username = '%u' OR
m.local_part = "%n") AND m.active = '1'
doveadm quota get of a test domain:
Username Quota name TypeValue Limit %
example...@example.com user STORAGE 5567 10 5
example...@example.com user MESSAGE70 - 0
example...@example.com domain STORAGE 33426 1024000 3
example...@example.com domain MESSAGE 1292 - 0
example_s...@example.com user STORAGE 33403 10 33
example_s...@example.com user MESSAGE 1280 - 0
example_s...@example.com domain STORAGE 33426 1024000 3
example_s...@example.com domain MESSAGE 1292 - 0
Note: these are values after a recalc and after the users (both) received
some more test mails. So, the figures in general are not correct (if I sum
them up), but you see that user quota is getting updated separate for each
user and that domain quota is equal for both but not equal with any of the
users. Now, if I would do a recalc -A it would look like this:
example...@example.com user STORAGE 5567 10 5
example...@example.com user MESSAGE70 - 0
example...@example.com domain STORAGE 33403 1024000 3
example...@example.com domain MESSAGE 1280 - 0
example_s...@example.com user STORAGE 33403 10 33
example_s...@example.com user MESSAGE 1280 - 0
example_s...@example.com domain STORAGE 33403 1024000 3
example_s...@example.com domain MESSAGE 1280 - 0
So, the last user processed determiens the domain values.
Actually, the user values would be recalculated correctly after the recalc
and might be different from above and the domain quota then follows that
by using the last processed user as the single source for it.
Thanks!
Kai
Re: Does domain quota work and how?
On Thu, 16 Aug 2018, Aki Tuomi wrote: ps. Here is the symbols your mail got, if it's any use to you. ... SUBJECT_ENDS_QUESTION Subject ends with a question (1) I find this scoring humorous, as one of this list's purpose is to ask questions. Joseph Tam
Re: Does domain quota work and how?
> On 16 August 2018 at 20:51 Kai Schaetzl wrote:
>
>
> Hello,
>
> I'm in the process of moving users from an old dovecot setup with system
> users to a modern virtual user setup.
>
> I can't find good documentation on domain quota.
> It is mentioned in the docs as
> quota2 = dict:domain:%d:proxy::quota_domain
> but not much more.
> Searching the mailing list or Google doesn't reveal much more info.
> Searching the German mailing list at jpberlin finds mostly information
> where they say that it doesn't work.
> So, does it or does it not?
>
> And if it works, does it work like I'm used to?
> e.g. with system users and aquota the filesystem quota kicks in whenever
> either the user or the domain quota is reached. That makes sense. It
> prohibits that single users can "fill" the whole domain (group) quota.
>
> I found one particular example where they say the dict quota works
> different, e.g. it's an either/or. They give this example for the
> user_query (the relevant part):
>
> CONCAT('*:bytes=', \
> IF(mailbox.quota = 0, domain.maxquota*1024000, mailbox.quota)) \
> as quota_rule
>
> So, this works as a default domain quota that is overridden on a user by
> user basis. But this means that a user quota that is set cannot honor the
> domain quota. So, in theory all users together can reach a maximum quota
> that is way over the domain quota. Not desirable.
>
> Concluding from the example in 90-quota.conf I want to set up this
> user_query (relevant part):
>
> concat('*:bytes=', m.quota) AS quota_rule, concat('*:bytes=',
> (s.quota*1024*1024)) AS quota2_rule
>
> (with m and s being the two tables for user and domain quota and s holding
> the domain quota in MB)
>
> Does this work together with this quota plugin setup and would provide
> both a user and domain quota?
>
> quota = dict:user::proxy::quota
> quota2 = dict:domain:%d:proxy::quota_domain
> quota_rule = *:storage=100M # default
> quota2_rule = *:storage=1000M # default
>
> I would like to know if this is supposed to work (and how) before I start
> testing in vain.
>
> One thing that I can confirm from several statements on the mailing lists
> is that the recalc option of doveadm doesn't work correctly. A
> doveadm quota recalc -A
> sets the used domain quota to the user used quota that it handles last for
> the domain (and a recalc -u sets the whole domain used quota to the one of
> that user). Which sets the domain quota completely wrong. I tried
> iterating thru the various users but got the same result as with -A. So,
> this is clearly broken, at least from my point of view.
>
> I'm using dovecot 2.3 from this repo:
> deb https://repo.dovecot.org/ce-2.3-latest/ubuntu/xenial xenial main
>
> I installed it because of the native blowfish encryption support. So far,
> it seems to be running great. Except for this particular aspect.
>
> Thanks!
>
> Kai
>
>
Can you provide doveconf -n please?
Aki
Does domain quota work and how?
Hello,
I'm in the process of moving users from an old dovecot setup with system
users to a modern virtual user setup.
I can't find good documentation on domain quota.
It is mentioned in the docs as
quota2 = dict:domain:%d:proxy::quota_domain
but not much more.
Searching the mailing list or Google doesn't reveal much more info.
Searching the German mailing list at jpberlin finds mostly information
where they say that it doesn't work.
So, does it or does it not?
And if it works, does it work like I'm used to?
e.g. with system users and aquota the filesystem quota kicks in whenever
either the user or the domain quota is reached. That makes sense. It
prohibits that single users can "fill" the whole domain (group) quota.
I found one particular example where they say the dict quota works
different, e.g. it's an either/or. They give this example for the
user_query (the relevant part):
CONCAT('*:bytes=', \
IF(mailbox.quota = 0, domain.maxquota*1024000, mailbox.quota)) \
as quota_rule
So, this works as a default domain quota that is overridden on a user by
user basis. But this means that a user quota that is set cannot honor the
domain quota. So, in theory all users together can reach a maximum quota
that is way over the domain quota. Not desirable.
Concluding from the example in 90-quota.conf I want to set up this
user_query (relevant part):
concat('*:bytes=', m.quota) AS quota_rule, concat('*:bytes=',
(s.quota*1024*1024)) AS quota2_rule
(with m and s being the two tables for user and domain quota and s holding
the domain quota in MB)
Does this work together with this quota plugin setup and would provide
both a user and domain quota?
quota = dict:user::proxy::quota
quota2 = dict:domain:%d:proxy::quota_domain
quota_rule = *:storage=100M # default
quota2_rule = *:storage=1000M # default
I would like to know if this is supposed to work (and how) before I start
testing in vain.
One thing that I can confirm from several statements on the mailing lists
is that the recalc option of doveadm doesn't work correctly. A
doveadm quota recalc -A
sets the used domain quota to the user used quota that it handles last for
the domain (and a recalc -u sets the whole domain used quota to the one of
that user). Which sets the domain quota completely wrong. I tried
iterating thru the various users but got the same result as with -A. So,
this is clearly broken, at least from my point of view.
I'm using dovecot 2.3 from this repo:
deb https://repo.dovecot.org/ce-2.3-latest/ubuntu/xenial xenial main
I installed it because of the native blowfish encryption support. So far,
it seems to be running great. Except for this particular aspect.
Thanks!
Kai
Re: Does domain quota work and how?
> On 16 August 2018 at 17:47 Kai Schaetzl wrote: > > > dovecot-ow...@dovecot.org wrote on Thu, 16 Aug 2018 17:14:00 +0300: > > > Message rejected by filter rule match > > Hello, > > why was this message rejected by a filter rule ? > (Posting first time to the list, yes, I subscribed.) > > Thanks, > > Kai > > Hi! I'm sorry you got bit by this, it's the spam filter. The way to get by the spam filter is to subscribe and wait about an hour for the subscription list to be updated, then you can post to the list, as we give a high negative score for anyone subscribed. Aki Tuomi ps. Here is the symbols your mail got, if it's any use to you. DMARC_NA(0)[conactive.com] SUBJECT_ENDS_QUESTION Subject ends with a question (1) REPLYTO_EQ_TO_ADDR(5) FROM_HAS_DN(0) IP_SCORE(-0.007259)[country: DE(-0.04)] MV_CASE(0.5) MIME_GOOD(-0.1)[text/plain] R_SPF_ALLOW(-0.2)[+a] R_DKIM_NA(0) GREYLIST(0)[pass,body] RCVD_TLS_LAST(0) ARC_NA(0) ASN(0)[asn:198710, ipnet:185.82.84.0/22, country:DE] PREVIOUSLY_DELIVERED(0)[dovecot@dovecot.org] FROM_EQ_ENVFROM(0) TO_DN_NONE(0) RCPT_COUNT_ONE(0)[1] BAYES_HAM(-0.01)[14.49%] TO_MATCH_ENVRCPT_ALL(0) HAS_REPLYTO(0)[dovecot@dovecot.org] RCVD_COUNT_TWO(0)[2]
