Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
On 2.8.2013, at 16.18, Axel Luttgens wrote: > Le 2 août 2013 à 14:13, Timo Sirainen a écrit : > >> I'd just do: >> >> 1) start quota-status service by e.g. connecting to it via telnet >> >> 2) gdb -p `pidof quota-status` >> b hook_mail_user_created >> cont >> >> 3) recipient=user >> >> 4) does it stop?.. if yes, keep hitting "s" to see if it goes to quota >> code. > > To be sure, tried again, but still getting quite anarchistic behaviors, > requiring some "luck" for retrieving useful info... > Could be a clang vs gdb thing; I also tried to compile the quota plugin > without optimization in the hope to bring some consistency back, without much > success. Optimization always makes things rather annoying. Especially with clang -O2 makes it just about impossible for gdb to do anything useful. You'd probably need to disable optimization for lib-storage also, not just quota plugin.
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
Le 2 août 2013 à 14:13, Timo Sirainen a écrit : > I'd just do: > > 1) start quota-status service by e.g. connecting to it via telnet > > 2) gdb -p `pidof quota-status` > b hook_mail_user_created > cont > > 3) recipient=user > > 4) does it stop?.. if yes, keep hitting "s" to see if it goes to quota > code. To be sure, tried again, but still getting quite anarchistic behaviors, requiring some "luck" for retrieving useful info... Could be a clang vs gdb thing; I also tried to compile the quota plugin without optimization in the hope to bring some consistency back, without much success. >> Still trying to have it provide me with some enlightening info, but if I may >> in the meantime paraphrase one of my initial questions on this thread: >> >> What makes doveadm-quota/lmtp and quota-status different? > > Not much.. This is what I was tempted to believe, until... ;-) >> doveadm-quota and lmtp correctly understand my quota-related settings, and >> over-qauota users are handled as such. >> >> On the other hand, quota-status always returns "action=OK" for any existing >> user, whether over-quota or not. > > I've no idea. Send your current doveconf -n and I'll see if I can > reproduce the problem with it? Thank you for your kind proposal; it would be such a relief, should you find something I'm overlooking. I provided that info at the very beginning of the thread, but I may have changed one detail or another in the meantime; I'll thus send you my current config privately. Best Regards, Axel
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
On Fri, 2013-08-02 at 10:30 +0200, Axel Luttgens wrote: > Le 1 août 2013 à 18:05, Timo Sirainen a écrit : > > > On 1.8.2013, at 19.02, Axel Luttgens wrote: > > > >> [...] > >> If yes, could it be that it is never called in my case? > > > > If not, then there's definitely some problem :) > > > >> [...] > >> Could I try to break somewhere earlier in the call chain? > > > > It should definitely stop in hook_mail_user_created, which should call > > quota_mail_user_created as one of the hooks. If not, the user then doesn't > > actually have quota plugin enabled.. > > And I'm definitely not a gdb guru. :-( I'd just do: 1) start quota-status service by e.g. connecting to it via telnet 2) gdb -p `pidof quota-status` b hook_mail_user_created cont 3) recipient=user 4) does it stop?.. if yes, keep hitting "s" to see if it goes to quota code. > Still trying to have it provide me with some enlightening info, but if I may > in the meantime paraphrase one of my initial questions on this thread: > >What makes doveadm-quota/lmtp and quota-status different? Not much.. > doveadm-quota and lmtp correctly understand my quota-related settings, and > over-qauota users are handled as such. > > On the other hand, quota-status always returns "action=OK" for any existing > user, whether over-quota or not. I've no idea. Send your current doveconf -n and I'll see if I can reproduce the problem with it?
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
Le 1 août 2013 à 18:05, Timo Sirainen a écrit : > On 1.8.2013, at 19.02, Axel Luttgens wrote: > >> [...] >> If yes, could it be that it is never called in my case? > > If not, then there's definitely some problem :) > >> [...] >> Could I try to break somewhere earlier in the call chain? > > It should definitely stop in hook_mail_user_created, which should call > quota_mail_user_created as one of the hooks. If not, the user then doesn't > actually have quota plugin enabled.. And I'm definitely not a gdb guru. :-( Still trying to have it provide me with some enlightening info, but if I may in the meantime paraphrase one of my initial questions on this thread: What makes doveadm-quota/lmtp and quota-status different? As a reminder: doveadm-quota and lmtp correctly understand my quota-related settings, and over-qauota users are handled as such. On the other hand, quota-status always returns "action=OK" for any existing user, whether over-quota or not. According to the logs, the userdb queries correctly return all needed quota-related info for the user; on the other hand, the dict service never gets launched. It could thus be inferred that quota-status is following a slightly different path for fetching/handling quota information. Knowing the difference could help to focus my miserable gdb investigations and/or to understand what may be at the fringe in my config. TIA, Axel
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
On 1.8.2013, at 19.02, Axel Luttgens wrote: > Le 1 août 2013 à 14:29, Timo Sirainen a écrit : > >> And if you're still stuck with this, set a breakpoint to >> quota_mail_user_created and step through it to figure out why >> MODULE_CONTEXT_SET() isn't being called. > > Yes, still stuck. :-( > > Did you mean function quota_mail_user_created from quota-storage.c? Yes. > If yes, could it be that it is never called in my case? If not, then there's definitely some problem :) > Desperately trying to have the program break there, without success... > > Could I try to break somewhere earlier in the call chain? It should definitely stop in hook_mail_user_created, which should call quota_mail_user_created as one of the hooks. If not, the user then doesn't actually have quota plugin enabled..
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
Le 1 août 2013 à 14:29, Timo Sirainen a écrit : > And if you're still stuck with this, set a breakpoint to > quota_mail_user_created and step through it to figure out why > MODULE_CONTEXT_SET() isn't being called. Yes, still stuck. :-( Did you mean function quota_mail_user_created from quota-storage.c? If yes, could it be that it is never called in my case? Desperately trying to have the program break there, without success... Could I try to break somewhere earlier in the call chain? TIA, Axel
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
On 19.7.2013, at 16.02, Axel Luttgens wrote:
> Le 18 juil. 2013 à 11:25, Axel Luttgens a écrit :
>
>> [...]
>> It is to be noted that no lines in the log are related to possible problems
>> encountered for launching [the dict server]. It is a bit as if quota_check()
>> in src/plugins/quota/quota-status.c always immediately returned with 1 at
>> the first test.
>> [...]
>
> Tracing with gdb, it appears this is indeed the case.
>
> Here's the beginning of quota_check():
>
> static int
> quota_check(struct mail_user *user, uoff_t mail_size, const char
> **error_r)
> {
> struct quota_user *quser = QUOTA_USER_CONTEXT(user);
> [...]
>
> if (quser == NULL) {
> /* no quota for user */
> return 1;
> }
> [...]
>
> and one has for quser:
>
> (gdb) p quser
> $1 = (struct quota_user *) 0x0
And if you're still stuck with this, set a breakpoint to
quota_mail_user_created and step through it to figure out why
MODULE_CONTEXT_SET() isn't being called.
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
Le 18 juil. 2013 à 11:25, Axel Luttgens a écrit :
> [...]
> It is to be noted that no lines in the log are related to possible problems
> encountered for launching [the dict server]. It is a bit as if quota_check()
> in src/plugins/quota/quota-status.c always immediately returned with 1 at the
> first test.
> [...]
Tracing with gdb, it appears this is indeed the case.
Here's the beginning of quota_check():
static int
quota_check(struct mail_user *user, uoff_t mail_size, const char
**error_r)
{
struct quota_user *quser = QUOTA_USER_CONTEXT(user);
[...]
if (quser == NULL) {
/* no quota for user */
return 1;
}
[...]
and one has for quser:
(gdb) p quser
$1 = (struct quota_user *) 0x0
Yet, struct user passed as argument doesn't show obvious problems (but I have
to confess the details are faaar beyond me); I reproduce it at the end of this
message.
So, either my users aren't recognized as being subjected to quotas, or
something goes wrong with macro QUOTA_USER_CONTEXT (which in turn translates
into macro MODULE_CONTEXT which I just don't understand), or both.
As a reminder, with the same configs, "doveadm quota" and lmtp do not show such
a behavior: they both take quotas into account for my users.
Any ideas?
TIA,
Axel
(gdb) p *user
$2 = {
pool = 0x7fed9b829020,
v = {
deinit = 0x10b190dd0
},
vlast = 0x7fed9b82a188,
refcount = 1,
username = 0x7fed9b829110 "john@example.com",
_home = 0x7fed9b829e08 "/_Mailstores/john.doe",
uid = 999,
gid = 999,
service = 0x7fed9b829e30 "quota-status",
local_ip = 0x0,
remote_ip = 0x0,
auth_token = 0x0,
var_expand_table = 0x7fed9b829e40,
error = 0x0,
set_info = 0x7fed9b814d60,
unexpanded_set = 0x7fed9b829138,
set = 0x7fed9b829770,
namespaces = 0x7fed9b4046b0,
storages = 0x7fed9b404780,
hooks = {
arr = {
buffer = 0x7fed9b82a130,
element_size = 8
},
v = 0x7fed9b82a130,
v_modifiable = 0x7fed9b82a130
},
mountpoints = 0x0,
default_normalizer = 0x10b0c1d00 ,
_attr_dict = 0x0,
module_contexts = {
arr = {
buffer = 0x7fed9b829da8,
element_size = 8
},
v = 0x7fed9b829da8,
v_modifiable = 0x7fed9b829da8
},
nonexistent = 0,
home_looked_up = 1,
anonymous = 0,
autocreated = 0,
initialized = 1,
mail_debug = 1,
inbox_open_error_logged = 0,
fuzzy_search = 0,
dsyncing = 0,
attr_dict_failed = 0
}
Re: [Dovecot] 2.2.4 - Some questions about and needing help with quota-status
Hello, I ended my previous message with : > [...] > Q3. What am I doing wrong? > [...] Given the details discussed in another thread (http://www.dovecot.org/list/dovecot/2013-July/091309.html), I tried by changing the user_query's SELECT from: user_query = \ SELECT DISTINCT \ nickname AS user, \ mail_home AS home, \ mail_location AS mail, \ quota_rule AS quota_rule \ FROM \ [...] to: user_query = \ SELECT DISTINCT \ nickname AS user, \ coalesce(mail_home, '/_Mailstores/' || nickname) AS home, \ mail_location AS mail, \ 'dict:Quota utilisateur:' || nickname || ':proxy::sql_quota' AS quota, \ quota_rule AS quota_rule \ FROM \ [...] while keeping dovecot.conf unchanged (see my previous message). The idea is to not rely anymore on the expansion of %u (or %n or %d) in dovecot.conf, while still keeping the ability to have per-user settings. Currently, a doveadm quota get -u john.doe yields: Quota nameTypeValue Limit % Quota utilisateur STORAGE3134 91 Quota utilisateur MESSAGE23 - 0 So, let's ask quota-status what it believes about a message with a size of 10. The reply is still "action=OK", the dict server still isn't launched, and the log shows: auth: Debug: userdb out: USER 1 john@example.com home=/_Mailstores/john.doe quota=dict:Quota utilisateur:john.doe:proxy::sql_quota quota_rule=*:storage=35000b quota-status: Debug: auth input: john@example.com home=/_Mailstores/john.doe quota=dict:Quota utilisateur:john.doe:proxy::sql_quota quota_rule=*:storage=35000b quota-status: Debug: Added userdb setting: plugin/quota=dict:Quota utilisateur:john.doe:proxy::sql_quota quota-status: Debug: Added userdb setting: plugin/quota_rule=*:storage=35000b quota-status(john@example.com): Debug: Effective uid=999, gid=999, home=/_Mailstores/john.doe quota-status(john@example.com): Debug: Quota root: name=Quota utilisateur backend=dict args=john.doe:proxy::sql_quota quota-status(john@example.com): Debug: Quota rule: root=Quota utilisateur mailbox=* bytes=35000 messages=0 quota-status(john@example.com): Debug: Quota grace: root=Quota utilisateur bytes=3500 (10%) quota-status(john@example.com): Debug: dict quota: user=john.doe, uri=proxy::sql_quota, noenforcing=0 quota-status(john@example.com): Debug: fs: root=/_Mailstores/john.doe/mboxes, index=, indexpvt=, control=, inbox=/_Mailstores/john.doe/mboxes/inbox, alt= To rule out any other side-effects potentially introduced by the user_query, I even tried with the "nickname AS user" removed from the SELECT. Even with that, the reply is "action=OK", the dict server still isn't launched, and the lines written to the log are undistinguishable from above ones... It is to be noted that no lines in the log are related to possible problems encountered for launching it. It is a bit as if quota_check() in src/plugins/quota/quota-status.c always immediately returned with 1 at the first test. Anyway, I'm still stuck. And still very interested in replies to Q1, Q2 and Q3. ;-) TIA, Axel
[Dovecot] 2.2.4 - Some questions about and needing help with quota-status
Help! I'm stuck. :-(
The config of my experimental setup appears at the end of this message; I'm
providing hereafter some more info that may not be immediately obvious.
This is dovecot 2.2.4 with changesets 9091d0f2d971 and 2be295a0b64f.
All involved databases are sqlite ones.
passdb and userdb are devised so as to change usernames.
For example, I could have a user with addresses "d...@oldexample.com",
"jo...@oldexample.com" and "john@example.com" needing to be able to log in
as "jdoe" or "u123456"; all db lookups for that user then end with name
"john.doe".
This is a single mail user setup (user/group "dovemailer", uid/gid 999).
Service lmtp has been configured to run as that user; that required a slight
adjustment at the auth-userdb socket level.
As a general rule, the quota dict appears to be correctly updated upon message
arrivals and removals (thru lmtp, pop, imap), and to be correctly
queried/interpreted by the various parts of the server.
For example, thru the userdb query, user john.doe has been given an even lower
quota limit than the already low default defined for testings:
$ sudo doveadm quota get -u john.doe
Quota nameTypeValue Limit %
Quota utilisateur STORAGE20 5 400
Quota utilisateur MESSAGE14 - 0
and is clearly recognized as being over-quota by lmtp:
$ telnet /_ROOT/var/run/dovecot/lmtp
Trying /_ROOT/var/run/dovecot/lmtp...
Connected to (null).
Escape character is '^]'.
220 almba.local Dovecot ready.
mail from:
250 2.1.0 OK
rcpt to:
250 2.1.5 OK
data
354 OK
Subject: test
.
552 5.2.2 Quota exceeded (mailbox for user is
full)
Note that both services config and dict are launched if they aren't running.
Since it has been previously seen that running quota-status as root comes with
its own problems, and since it is a single user setup anyway, I'm trying to run
it as dovemailer as well.
So, let's try to see what quota-status thinks about john.doe:
$ sudo -u _postfix telnet /_ROOT/var/spool/postfix/private/quota-policyd
Trying /_ROOT/var/spool/postfix/private/quota-policyd...
Connected to (null).
Escape character is '^]'.
Connection closed by foreign host.
Clearly, not much...
Looking in the log:
dovecot[10554]: quota-status: Fatal: Error reading configuration:
net_connect_unix(/_ROOT/var/run/dovecot/config) failed: Permission denied
This thus raises a first question:
Q1. What makes lmtp and quota-status different? How does lmtp manage to fetch
all needed info, while quota-status seems to require an access to the config
socket?
Let's then slightly adjust dovecot.conf, in the hope to make quota-status happy:
service config {
unix_listener config {
group = dovemailer
mode = 0660
}
}
Q2. Should the above really be needed, wouldn't there be a better way?
I ask, because it seems to me that I'm starting to seriously lose the benefits
of privilege separation...
Anyway, let's ask quota-status again:
$ sudo -u _postfix telnet /_ROOT/var/spool/postfix/private/quota-policyd
Password:
Trying /_ROOT/var/spool/postfix/private/quota-policyd...
Connected to (null).
Escape character is '^]'.
recipient=john@example.com
size=1
action=OK
^]
telnet> quit
Connection closed.
In the log:
dovecot[11050]: auth: Debug: userdb out: USER 1
john@example.comquota_rule=*:storage=5k
dovecot[11050]: quota-status: Debug: auth input: john@example.com
quota_rule=*:storage=5k
dovecot[11050]: quota-status: Debug: Added userdb setting:
plugin/quota_rule=*:storage=5k
dovecot[11050]: quota-status(john@example.com): Debug: Effective
uid=999, gid=999, home=/_Mailstores/john.doe
dovecot[11050]: quota-status(john@example.com): Debug: Quota root:
name=Quota utilisateur backend=dict args=john.doe:proxy::sql_quota
dovecot[11050]: quota-status(john@example.com): Debug: Quota rule:
root=Quota utilisateur mailbox=* bytes=5120 messages=0
dovecot[11050]: quota-status(john@example.com): Debug: Quota grace:
root=Quota utilisateur bytes=512 (10%)
dovecot[11050]: quota-status(john@example.com): Debug: dict quota:
user=john.doe, uri=proxy::sql_quota, noenforcing=0
dovecot[11050]: quota-status(john@example.com): Debug: fs:
root=/_Mailstores/john.doe/mboxes, index=, indexpvt=, control=,
inbox=/_Mailstores/john.doe/mboxes/inbox, alt=
It is to be noted that the config server is now launched as expected, but that
the dict server still isn't.
Trying a dirsize backend instead of the dict backend doesn't help.
The problem
