[Dovecot] 'doveadm quota get' dictionary SQL query ignores specified '@domain' part of username. bad config or bug?
Hi,
I'm setting up SQL quota usage in Dovecot2.
I've created a MAILBOX parameter table, 'PARAMS', that contains unique
@ pairs, with quota data.
mysql> select * from PARAMS;
++++-++
| ai | MAILBOX_user_domain| quota | quota_bytes |
quota_msgs |
++++-++
| 1 | myu...@domain1.com| 1073741824 | 0 |
0 |
| 2 | myu...@domain2.com| 10 | 0 |
0 |
++++-++
2 rows in set (0.00 sec)
I've specified dict usage for quota,
/etc/dovecot/dovecot.conf
!include conf.d/*.conf
protocols = imap lmtp
dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext }
created the maps,
/etc/dovecot/dovecot-dict-sql.conf.ext
connect = host=/var/run/mysql/mysql.sock dbname=my_db
user=my_user
password=my_pass
map {
pattern= priv/quota/storage
table = PARAMS
username_field = MAILBOX_user_domain
value_field= quota_bytes
}
map {
pattern= priv/quota/messages
table = PARAMS
username_field = MAILBOX_user_domain
value_field= quota_msgs
}
assigned the global values and backend,
/etc/dovecot/conf.d/90-quota.conf
plugin {
quota = dict:User Quota::proxy::quota
quota_rule = *:storage=1GB:messages=1
quota_rule2 = Trash:storage=+10%%
}
and specified the user iteration query
/etc/dovecot/sql/virtmail-userdb-sql.cf
driver = mysql
connect = host=/var/run/mysql/mysql.sock dbname=my_db
user=my_user
password=my_pass
user_query = CALL UserDBQuery('%n','%d');
iterate_query = SELECT `MAILBOX_user_domain` AS user FROM
`PARAMS`;
Verifying operation with with `doveadm`, I get a result from a !error query,
doveadm quota get -u myu...@domain1.com
Quota name TypeValue Limit %
User quota STORAGE 0 1048576 0
User quota MESSAGE 0 1 0
doveadm quota get -u myu...@domain2.com
Quota name TypeValue Limit %
User quota STORAGE 0 1048576 0
User quota MESSAGE 0 1 0
But notice that there's no specificity by *domain*. Despite being
provided the full @, matched to the map,
username_field = MAILBOX_user_domain
the Dovecot dict query seems to ignore the part, and simply
matches on the first of the tow identical parts.
Specifying "-A" doesn't help,
doveadm quota get -A
UsernameQuota name Type
Value Limit %
myu...@domain1.com User quota STORAGE 0
1048576 0
myu...@domain1.com User quota MESSAGE 0
1 0
myu...@domain2.com User quota STORAGE 0
1048576 0
myu...@domain2.com User quota MESSAGE 0
1 0
What needs to be done to get domain-specificity in the 'doveadm quota
get' query?
Is my configuration off, or incomplete, or is this a bug?
Thanks,
Rich
Re: [Dovecot] "doveadm log reopen" don't reopen separate lmtp log
On 18.10.2011, at 20.53, Henrik Larsson wrote: > >> The problem isn't lmtp itself, it's that you're using -L parameter, which >> causes LMTP processes to open the log files directly. > Just to be sure, isn't this the only way to have a separate LMTP log files? Yes. But this isn't LMTP-specific in any way. It's the same as if you wanted separate IMAP or POP3 or whatever log files. The only special cases are programs that are executed directly instead of via dovecot master process (dovecot-lda, doveadm basically), because they always open the log files directly (and die once they're finished, so they don't have the rotation problem). Hmm. Actually you could probably set service_count=1 for lmtp service and it would get rotated soon enough, because the process would die after handling one LMTP connection.
Re: [Dovecot] "doveadm log reopen" don't reopen separate lmtp log
The problem isn't lmtp itself, it's that you're using -L parameter, which causes LMTP processes to open the log files directly. Just to be sure, isn't this the only way to have a separate LMTP log files? The only way to reopen the log files is to restart those LMTP processes. "doveadm reload" should do it. Thanks, I will try this. Best regards Henrik Larsson
Re: [Dovecot] "doveadm log reopen" don't reopen separate lmtp log
The problem isn't lmtp itself, it's that you're using -L parameter, which
causes LMTP processes to open the log files directly. The only way to reopen
the log files is to restart those LMTP processes. "doveadm reload" should do it.
On 18.10.2011, at 18.32, Henrik Larsson wrote:
> I still see the issue below. Is there anyone running separate LMTP logging
> that could check if they experience the same issue?
>
> Best regards
> Henrik Larsson
>
>
> Original Message
> Subject: "doveadm log reopen" don't reopen separate lmtp log
> Date: Sun, 25 Sep 2011 00:23:49 +0200
> From: Henrik Larsson
> To: dovecot@dovecot.org
>
> Dear all
>
> I have setup separate pop3/imap log "/var/log/dovecot" and lmtp delivery log
> "/var/log/dovecot-deliver".
>
> After rotating logfiles, i run "doveadm log reopen". I see that my pop3/imap
> log "/var/log/dovecot" is used straight away, but my lmpt log
> "/var/log/dovecot-deliver" isn't used at first. After some time, usually a
> few minutes, logs are being written to this file anyway. But if I compare
> this to my maillog, it misses some deliveries just after the log rotation.
>
> Is there any problems with this configuration that should be corrected?
>
> --cut--
> # doveconf -n
> # 2.0.15: /usr/local/etc/dovecot/dovecot.conf
> # OS: FreeBSD 8.2-STABLE amd64
> auth_mechanisms = plain login digest-md5 cram-md5
> first_valid_uid = 125
> hostname = mail.larsson.it
> listen = *
> log_path = /var/log/dovecot
> mail_plugins = fts fts_squat zlib
> mail_privileged_group = postfix
> mail_temp_dir = /var/db/dovecot
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
> copy include variables body enotify environment mailbox date
> namespace {
> inbox = yes
> location =
> prefix =
> separator = .
> type = private
> }
> namespace {
> hidden = yes
> inbox = no
> list = no
> location =
> prefix = INBOX.
> separator = .
> type = private
> }
> passdb {
> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
> driver = sql
> }
> plugin {
> fts = squat
> fts_squat = partial=4 full=10
> sieve = ~/.dovecot.sieve
> sieve_dir = ~/sieve
> }
> protocols = imap pop3 lmtp sieve
> service auth-worker {
> user = $default_internal_user
> }
> service auth {
> unix_listener /home/mail/postfix/private/auth {
>group = postfix
>mode = 0666
>user = postfix
> }
> }
> service lmtp {
> executable = lmtp -L
> unix_listener /home/mail/postfix/private/dovecot-lmtp {
>group = postfix
>mode = 0660
>user = postfix
> }
> }
> ssl_cert = ssl_key = userdb {
> args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
> driver = sql
> }
> protocol lmtp {
> log_path = /var/log/dovecot-deliver
> mail_plugins = fts fts_squat zlib sieve
> }
> protocol lda {
> mail_plugins = fts fts_squat zlib sieve
> }
> protocol imap {
> mail_plugins = fts fts_squat zlib imap_zlib
> }
> --cut--
>
>
> Best regards
> Henrik Larsson
>
Re: [Dovecot] "doveadm log reopen" don't reopen separate lmtp log
I still see the issue below. Is there anyone running separate LMTP
logging that could check if they experience the same issue?
Best regards
Henrik Larsson
Original Message
Subject: "doveadm log reopen" don't reopen separate lmtp log
Date: Sun, 25 Sep 2011 00:23:49 +0200
From: Henrik Larsson
To: dovecot@dovecot.org
Dear all
I have setup separate pop3/imap log "/var/log/dovecot" and lmtp
delivery log "/var/log/dovecot-deliver".
After rotating logfiles, i run "doveadm log reopen". I see that my
pop3/imap log "/var/log/dovecot" is used straight away, but my lmpt log
"/var/log/dovecot-deliver" isn't used at first. After some time, usually
a few minutes, logs are being written to this file anyway. But if I
compare this to my maillog, it misses some deliveries just after the log
rotation.
Is there any problems with this configuration that should be corrected?
--cut--
# doveconf -n
# 2.0.15: /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 8.2-STABLE amd64
auth_mechanisms = plain login digest-md5 cram-md5
first_valid_uid = 125
hostname = mail.larsson.it
listen = *
log_path = /var/log/dovecot
mail_plugins = fts fts_squat zlib
mail_privileged_group = postfix
mail_temp_dir = /var/db/dovecot
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date
namespace {
inbox = yes
location =
prefix =
separator = .
type = private
}
namespace {
hidden = yes
inbox = no
list = no
location =
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
fts = squat
fts_squat = partial=4 full=10
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
protocols = imap pop3 lmtp sieve
service auth-worker {
user = $default_internal_user
}
service auth {
unix_listener /home/mail/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
}
service lmtp {
executable = lmtp -L
unix_listener /home/mail/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
}
}
ssl_cert =
Re: [Dovecot] mail_location in LDAP and variables
On Martes 18 Octubre 2011 16:46:28 Timo Sirainen escribió: > On Mon, 2011-10-17 at 12:12 +0200, Joseba Torre wrote: > > Hi, > > > > Is it posible to use variables in mail_location when taken from ldap? > > How? I've tried > > > > mdbox:/buzones2/'%n'/mdbox > > mdbox:/buzones2/"%n"/mdbox > > mdbox:/buzones2/%n/mdbox > > > > but %n is never replaced by its value. > > Where exactly are you trying to use this? It should get expanded. As usual, it was me trying to be a lot more complicated than needed :) I was thinking about storing the mail_location in an standard way in ldap, so that everybody in LDAP has something like irisMailbox= mdbox:/buzones/%256Hn/%n/mdbox This doesn't work, but doesn't make much sense either. (It does if used in the generic mail_location, but not in a per user value) Now when I create a new account I calculate the mail_location and store that value in LDAP. So far so good. -- Joseba Torre. Vicegerencia de TICs, área de Explotación
Re: [Dovecot] too many open files and v1.2
On Thu, 2011-10-06 at 12:42 +0100, Michael Moritz wrote: > Hi > > I've found an older thread > http://www.dovecot.org/list/dovecot/2010-March/047886.html which mentions > that this is a bug in version 1. I'm on Debian squeeze (1:1.2.15-7) and got > errors like these recently > > Oct 5 11:29:29 mail dovecot: dovecot: pipe() failed: Too many open files > Oct 5 11:29:32 mail dovecot: dovecot: pipe() failed: Too many open files > Oct 5 11:29:32 mail dovecot: dovecot: Temporary failure in creating login > processes, slowing down for now These errors come from the dovecot master process. Each child process uses up a few fds for pipes. > As a work around I've increased login_max_processes_count from 256 to 512 and > it seems to have helped (as I suspected stale opened files). That only makes it easier to reach, since now there can be more child processes eating up more fds. > My colleague thinks it's hitting the kernel open file limit ulimit -n > (currently at 1024) but I don't see how that could happen if the number of > process is nowhere near that. I don't remember how many fds each process takes in v1.x, probably between 1 and 3. So if you for a while had hundreds of imap or pop3 processes, you could run into that limit. v2.x doesn't use as many fds.
Re: [Dovecot] LDA/Postfix with quotas and without LMTP
On Tue, 2011-10-18 at 17:52 +0300, Timo Sirainen wrote: > On Sun, 2011-10-09 at 11:31 -0700, Steve Fatula wrote: > > dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) > > failed: Permission denied > > > The possibilities are: > > a) Start dovecot-lda so that the process belongs to extra group (e.g. > dovecot) and make the socket rw for that group. I don't know if Postfix > can do this for system users.. > > b) Use filesystem setgid bit for dovecot-lda > > c) Run via sudo > > The b) and c) are explained in wiki in the "multiple UIDs" section > (although for setuid-root rather than setgid-dovecot). Oh, and d) With v2.1 this works automatically
Re: [Dovecot] LDA/Postfix with quotas and without LMTP
On Sun, 2011-10-09 at 11:31 -0700, Steve Fatula wrote: > dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) > failed: Permission denied > > > Using mailbox_command in Postfix means that dovecot-lda is running as the > system user getting the mail. Which means it can't access the socket file. Of > course, one can simply use mode 666 on it, but, then people can use it to > find out information according to the doc, which I do not want. > > So, I am not sure why the doc says to add -d for per user quotas, without > also mentioning the problem with doing that. I don't see how it can work? > > Assuming I need per user quotas, and I don't want to use LMTP, is there a > good way around this without the security issue? The possibilities are: a) Start dovecot-lda so that the process belongs to extra group (e.g. dovecot) and make the socket rw for that group. I don't know if Postfix can do this for system users.. b) Use filesystem setgid bit for dovecot-lda c) Run via sudo The b) and c) are explained in wiki in the "multiple UIDs" section (although for setuid-root rather than setgid-dovecot).
Re: [Dovecot] Syntax to specify sdbox: mail_location path?
On Fri, 2011-10-07 at 11:43 -0700, mephistophe...@operamail.com wrote: > With "sdbox:" > > mail_location = sdbox:/my_mailbox_path/%d/%n > > mail gets stored in /my_mailbox_path/%d/%n/mailboxes/INBOX/dbox-Mails. > > How do I get rid of the '/mailboxes/' and '/dbox-Mails/' path components > so sdbox stores in: > > /my_mailbox_path/%d/%n/INBOX Well, there are a few settings so you could make it work that way, but why do you want to do that? There's a reason why those extra directory components were added, you shouldn't remove them.
Re: [Dovecot] mail_location in LDAP and variables
On Mon, 2011-10-17 at 12:12 +0200, Joseba Torre wrote: > Hi, > > Is it posible to use variables in mail_location when taken from ldap? How? > I've tried > > mdbox:/buzones2/'%n'/mdbox > mdbox:/buzones2/"%n"/mdbox > mdbox:/buzones2/%n/mdbox > > but %n is never replaced by its value. Where exactly are you trying to use this? It should get expanded.
Re: [Dovecot] Rare error with concurrent users searching with fts-solr
On Thu, 2011-10-13 at 10:31 +0200, Antonio Perez-Aranda wrote: > After see this error, I set a tcpdump to try to watch the SOLR XML and > GET, I can watch that Solr send a xml like this: > > > > with 200 as http status > > When the normal response is like this > > > .. Check what kind of commands Dovecot sends. Is this maybe a reply to a somewhat different (broken?) command than others? > At first, I think that the problems it's in Solr, but after get all > GET from tcpdump, I make a python script with urllib2 and send it all > to Solr with the same concurrency and I can't get any error or "empty > response" You mean send the exact same commands as Dovecot sent when this failed? It's anyway difficult for me to guess how this could be Dovecot's fault, if the command that gives the broken response normally works.
Re: [Dovecot] Auth Worker failures
On Wed, 2011-10-12 at 10:24 -0400, Simon Brereton wrote: > > > Of all the accounts on the box, it's only mine that throws this up. > Since its LIP is localhost, it could really only be for webmail - but > I don't always leave the webmail open, so I'm curious to know how this > gets there and what it is. > > Any suggestions? I find it difficult to believe I have an IMAP > process in a script somewhere (especially with my user account - the > postmaster account, I could believe, but not with my personal one).. > You could enable auth_debug_passwords=yes and see what password it tries.
Re: [Dovecot] 2.1 Plans
On Mon, 2011-10-10 at 13:53 -0700, Steve Fatula wrote: > Back in early 2010, Timo posted about a potential 2.1 feature to in > essence reduce the number of imap/pop3 processes using various > techniques since the code already supports multiple connections. It > even went as far as moving all idle connections. > > I don't see this on the 2.1 roadmap. Is it still planned perhaps for > 2.1? Thousands of imap processes going away is a great thing for me! > So, just curious. It's slowly getting there, but won't be ready for v2.1.
Re: [Dovecot] Convert-tool maildir > mbox
On Mon, 2011-10-10 at 14:10 +0200, Bertrand Jacquin wrote: > > Maybe that kind of conversion have evolved since 2.0 ? convert-tool wasn't very good. Use something else, like maybe dsync from v2.0 (even if you don't use v2.0 for anything else).
Re: [Dovecot] doveadm mailbox list shows file system directories as mailboxes
On Thu, 2011-09-22 at 11:52 +0200, Jürgen Obermann wrote: > Hello. > In our dovecvot we use mbox format with the default filesystem layout. > Therefore it is not possible to have mailboxes which are subfolders of > other mailboxes containing messages. > The command "doveadm mailbox list" includes the file system > directories, that contain only subfolders, as normal mailboxes in the > output: > > archiv > archiv/daemon > archiv/dovecot > > Did I miss something in the dovecot configuration or is there a bug in > the command "doveadm mailbox list"? It's more like a feature.. For example if there exists "archiv" directory without children, I don't think it should be invisible in the list. Maybe there could be a new parameter to hide \Noselect mailboxes.
