Re: [Dovecot] dovecot-sieve (pigeonhole) -- is is possible to re-filter INBOX when filters have been updated?
On Sat, May 25, 2013 3:53 am, Lev Serebryakov wrote: So, I need a way to run it from httpd (running with credentials of web user) on behalf mail subsystem (running with credentials of v-mail user) according command from PHP script... It could be non-trivial... There's a dovecot plugin that extends the sieve protocol and allows you to execute applications on the server. I don't know anything about it (I've never used it), but perhaps that is a path you could investigate? https://sftp.netscout.com/human.aspx?r=1484766944Arg12=fileviewArg11=1Arg07=903982830Arg06=904052910 Chris
Re: [Dovecot] dovecot-sieve (pigeonhole) -- is is possible to re-filter INBOX when filters have been updated?
On Sat, May 25, 2013 3:36 pm, Chris Richards wrote: On Sat, May 25, 2013 3:53 am, Lev Serebryakov wrote: So, I need a way to run it from httpd (running with credentials of web user) on behalf mail subsystem (running with credentials of v-mail user) according command from PHP script... It could be non-trivial... There's a dovecot plugin that extends the sieve protocol and allows you to execute applications on the server. I don't know anything about it (I've never used it), but perhaps that is a path you could investigate? woops, wrong link, use this one: http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms Chris
Re: [Dovecot] Error: dict client sent broken reply
Anyone have any thoughts on this?
Chris
On Sun, May 19, 2013 4:33 pm, Chris Richards wrote:
I've been mucking about, experimenting with the expire plugin and using a
dictionary. I've got the iteration query working when I do a normal
expunge using:
doveadm expunge -A mailbox INBOX.Trash savedbefore 1w
and expunging works as expected. However, I've got over 12,000 accounts
on this server, so I was hoping using the expire plugin to could help out.
I've configured the plugin, and things kinda work, except that somewhere
between 3700 and 3800 users, I abort with this:
doveadm(someuseraccount@somedomain): Error: dict client
(/var/run/dovecot/dict) sent broken reply
doveadm(someuseraccount@somedomain): Error: Dictionary iteration failed
doveadm: Error: Failed to iterate through some users
It consistently fails at the same user. If I delete that user from the
expire database, then it appears to fail on the next user.
I also see this in the logs:
dovecot: dict: Error: dict client: COMMIT: Can't commit while iterating
I've absolutely no idea where to go from here to troubleshoot this. Any
guidance would be appreciated.
Thanks,
Chris
doveconf -n:
# 2.1.12: /etc/dovecot/dovecot.conf
# OS: Linux 3.7.5-hardened-r1 x86_64 Gentoo Base System release 2.1 ext4
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_verbose_passwords = plain
default_process_limit = 200
dict {
quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
login_greeting = Awaiting command...
mail_location = maildir:/home/vmail/%d/%n/Maildir
mail_plugins = quota
mail_privileged_group = 100
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
expire = Trash 7
expire2 = Trash/* 7
expire3 = Spam 7
quota = dict:User quota:%u:proxy::quota
quota_rule = *:storage=200M
quota_warning = storage=99%% quota-warning 99 %n %d
quota_warning2 = storage=95%% quota-warning 95 %n %d
quota_warning3 = storage=80%% quota-warning 80 %n %d
quota_warning4 = -storage=95%% quota-warning 'less than 95' %n %d
sieve = ~/.dovecot.sieve
sieve_default = /home/vmail/dovecot/sieve/default.sieve
sieve_dir = ~/sieve
sieve_global_dir = /home/vmail/dovecot/sieve
}
protocols = imap pop3 sieve lmtp
service auth-worker {
user = $default_internal_user
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = dovecot
mode = 0666
user = dovecot
}
user = $default_internal_user
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service quota-warning {
executable = script /etc/dovecot/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl_cert = /etc/ssl/dovecot/server.pem
ssl_key = /etc/ssl/dovecot/server.key
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
protocol lmtp {
mail_plugins = sieve quota
postmaster_address = postmaster@domain
}
protocol lda {
mail_plugins = quota sieve quota
}
protocol imap {
mail_max_userip_connections = 20
mail_plugins = quota quota imap_quota
}
protocol pop3 {
mail_plugins = quota quota
}
From dovecot-dict-sql.conf.ext:
connect = host=localhost dbname=maildb user=dbuser password=dbpass
# CREATE TABLE quota (
# username varchar(100) not null,
# bytes bigint not null default 0,
# messages integer not null default 0,
# primary key (username)
# );
map {
pattern = priv/quota/storage
table = quota_usage
username_field = address
value_field = quota_bytes
}
map {
pattern = priv/quota/messages
table = quota_usage
username_field = address
value_field = quota_messages
}
# CREATE TABLE expires (
# username varchar(100) not null,
# mailbox varchar(255) not null,
# expire_stamp integer not null,
# primary key (username, mailbox)
# );
map {
pattern = shared/expire/$user/$mailbox
table = expires
Re: [Dovecot] dovecot-sieve (pigeonhole) -- is is possible to re-filter INBOX when filters have been updated?
On Fri, May 24, 2013 3:20 pm, Lev Serebryakov wrote: Hello, Dovecot. And I wonder, is here simple way to re-filter INBOX after sieve filters have been changed? Any offline (desktop) mail client could run new filters on old messages -- is here any way to do this with dovecot-sieve (pigeonhole)? Take a look at sieve-filter, which is part of the pigeonhole package since 0.3. It should provide you the capability to do what you want. Chris
[Dovecot] Error: dict client sent broken reply
I've been mucking about, experimenting with the expire plugin and using a
dictionary. I've got the iteration query working when I do a normal
expunge using:
doveadm expunge -A mailbox INBOX.Trash savedbefore 1w
and expunging works as expected. However, I've got over 12,000 accounts
on this server, so I was hoping using the expire plugin to could help out.
I've configured the plugin, and things kinda work, except that somewhere
between 3700 and 3800 users, I abort with this:
doveadm(someuseraccount@somedomain): Error: dict client
(/var/run/dovecot/dict) sent broken reply
doveadm(someuseraccount@somedomain): Error: Dictionary iteration failed
doveadm: Error: Failed to iterate through some users
It consistently fails at the same user. If I delete that user from the
expire database, then it appears to fail on the next user.
I also see this in the logs:
dovecot: dict: Error: dict client: COMMIT: Can't commit while iterating
I've absolutely no idea where to go from here to troubleshoot this. Any
guidance would be appreciated.
Thanks,
Chris
doveconf -n:
# 2.1.12: /etc/dovecot/dovecot.conf
# OS: Linux 3.7.5-hardened-r1 x86_64 Gentoo Base System release 2.1 ext4
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_verbose_passwords = plain
default_process_limit = 200
dict {
quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
login_greeting = Awaiting command...
mail_location = maildir:/home/vmail/%d/%n/Maildir
mail_plugins = quota
mail_privileged_group = 100
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
expire = Trash 7
expire2 = Trash/* 7
expire3 = Spam 7
quota = dict:User quota:%u:proxy::quota
quota_rule = *:storage=200M
quota_warning = storage=99%% quota-warning 99 %n %d
quota_warning2 = storage=95%% quota-warning 95 %n %d
quota_warning3 = storage=80%% quota-warning 80 %n %d
quota_warning4 = -storage=95%% quota-warning 'less than 95' %n %d
sieve = ~/.dovecot.sieve
sieve_default = /home/vmail/dovecot/sieve/default.sieve
sieve_dir = ~/sieve
sieve_global_dir = /home/vmail/dovecot/sieve
}
protocols = imap pop3 sieve lmtp
service auth-worker {
user = $default_internal_user
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = dovecot
mode = 0666
user = dovecot
}
user = $default_internal_user
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service quota-warning {
executable = script /etc/dovecot/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl_cert = /etc/ssl/dovecot/server.pem
ssl_key = /etc/ssl/dovecot/server.key
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
protocol lmtp {
mail_plugins = sieve quota
postmaster_address = postmaster@domain
}
protocol lda {
mail_plugins = quota sieve quota
}
protocol imap {
mail_max_userip_connections = 20
mail_plugins = quota quota imap_quota
}
protocol pop3 {
mail_plugins = quota quota
}
From dovecot-dict-sql.conf.ext:
connect = host=localhost dbname=maildb user=dbuser password=dbpass
# CREATE TABLE quota (
# username varchar(100) not null,
# bytes bigint not null default 0,
# messages integer not null default 0,
# primary key (username)
# );
map {
pattern = priv/quota/storage
table = quota_usage
username_field = address
value_field = quota_bytes
}
map {
pattern = priv/quota/messages
table = quota_usage
username_field = address
value_field = quota_messages
}
# CREATE TABLE expires (
# username varchar(100) not null,
# mailbox varchar(255) not null,
# expire_stamp integer not null,
# primary key (username, mailbox)
# );
map {
pattern = shared/expire/$user/$mailbox
table = expires
value_field = expire_stamp
fields {
address = $user
folder = $mailbox
}
}
dovecot-sql.conf.ext
driver = mysql
connect = host=/var/run/mysqld/mysqld.sock dbname=maildb user=dbuser
password=dbpass
default_pass_scheme = PLAIN
user_query = SELECT homedir AS
Re: [Dovecot] Quota not working with dict proxy
On Wed, May 15, 2013 9:15 pm, Daniel Parthey wrote:
map {
pattern = priv/quota/storage # dictionary for storage bytes
table = quota # table where to write storage count
username_field = username # username of whom storage should be
counted
value_field = bytes # number of bytes in user mailbox
}
map {
pattern = priv/quota/messages # dictionary for message count
table = quota # table where to write email count
username_field = username # username whose emails should be
counted
value_field = messages# number of messages in user mailbox
}
Regards
Daniel
I think more correctly, value_field is the name of the field in the db.
The 'storage' dictionary will always contain bytes,and the 'messages'
dictionary will always store the number of messages into the db field
named by the 'value_field' parameter.
I would guess that if you changed the pattern to 'shared/quota/messages'
then you could set the shared quota as well.
Re: [Dovecot] Quota not working with dict proxy
Are you using the same SQL table email for user lookup and quota/storage accounting? Try to use two different tables for user and quota database, because the quota accounting might have deleted an entry from the user table while it only tried to delete a row from the quota table. Regards Daniel Daniel, Per your suggestion, I created a new table, quote_usage, and changed the config files to look at it instead. Things appear to be working now. Thank you very much for your guidance. Is there any documentation that goes into more detail regarding the 'map' settings, what they mean, etc., of which you are aware? I was rather hoping to avoid digging through the code just to satisfy my curiosity. Thanks again. Chris
Re: [Dovecot] Quota not working with dict proxy
On Mon, May 13, 2013 5:55 pm, Daniel Parthey wrote: Are you using the same SQL table email for user lookup and quota/storage accounting? Try to use two different tables for user and quota database, because the quota accounting might have deleted an entry from the user table while it only tried to delete a row from the quota table. Regards Daniel I am using the same table for both user lookkup and quota accounting. I'll try creating another table for the lookup and quota accounting, but I find it disconcerting that it would be deleting entries from the database at all when all I asked it to do was recalc the quota. One wouldn't think that 'update an entry with the correct information' would equate to 'delete this row and recreate it'. Chris
[Dovecot] Quota not working with dict proxy
Hello all,
I'm sure this has been covered somewhere before, but my googlefu is not up
to the challenge.
Basically, I'm trying to configure quota plugin to use a dictionary
service (specifically proxy with mysql) so that I can store the quota
usage in a database and use that information in a lookup for postfix to
reject mail if over quota. I'm doing this because postfix+avamis+dovecot
setup with amavis re-injecting into postfix results in mail being
effectively accepted before dovecot lmtp knows if mailbox is full.
I'm currently using quota maildir:
quota = maildir:User quota
and this works;
doveadm -Df tab quota get -u 'user@domain'
Quota name TypeValue Limit %
User quota STORAGE 55388 204800 27
User quota MESSAGE 4883- 0
When I use quota dict:
quota = dict:User quota:%u:proxy::quota
I get this:
doveadm -f tab quota get -u user@domain
Quota name TypeValue Limit %
User quota STORAGE 0 204800 0
User quota MESSAGE 0 - 0
So far so good.
But manually setting database entry to values retrieved above with maildir
quota results in this:
Quota name TypeValue Limit %
User quota STORAGE 54 204800 0
User quota MESSAGE 4883- 0
I'm guessing this is returning values in KB, so that makes sense, I guess.
Attempting to recalc quota on one account using command:
doveadm -f tab quota recalc -u user@domain
returns with nothing, and when I repeat 'quota get' command, it says:
doveadm(user@domain): Fatal: User doesn't exist
Errwhat?!
Indeed, looking in the database shows the account in question was actually
DELETED!
Obviously, I've got something messed here, but I don't know what.
I need some guidance here.
Dovecot version is 2.1.12
Here is my config:
# 2.1.12: /etc/dovecot/dovecot.conf
# OS: Linux 3.7.5-hardened-r1 x86_64 Gentoo Base System release 2.1 ext4
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_verbose_passwords = plain
default_process_limit = 200
dict {
quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
login_greeting = Awaiting command...
mail_location = maildir:/home/vmail/%d/%n/Maildir
mail_plugins = quota
mail_privileged_group = 100
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
expire = Trash 7
expire2 = Trash/* 7
expire3 = Spam 7
quota = dict:User quota:%u:proxy::quota
quota_rule = *:storage=200M
quota_warning = storage=99%% quota-warning 99 %n %d
quota_warning2 = storage=95%% quota-warning 95 %n %d
quota_warning3 = storage=80%% quota-warning 80 %n %d
quota_warning4 = -storage=95%% quota-warning 'less than 95' %n %d
sieve = ~/.dovecot.sieve
sieve_default = /home/vmail/dovecot/sieve/default.sieve
sieve_dir = ~/sieve
sieve_global_dir = /home/vmail/dovecot/sieve
}
protocols = imap pop3 sieve lmtp
service auth-worker {
user = $default_internal_user
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = dovecot
mode = 0666
user = dovecot
}
user = $default_internal_user
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service quota-warning {
executable = script /etc/dovecot/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl_cert = /etc/ssl/dovecot/server.pem
ssl_key = /etc/ssl/dovecot/server.key
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
protocol lmtp {
mail_plugins = sieve quota
postmaster_address = postmaster@domain
}
protocol lda {
mail_plugins = quota sieve quota
}
protocol imap {
mail_max_userip_connections = 20
mail_plugins = quota quota imap_quota
}
protocol pop3 {
mail_plugins = quota quota
}
From dovecot-dict-sql.conf.ext:
connect = host=localhost dbname=maildb user=dbuser password=dbpass
# CREATE TABLE quota (
# username varchar(100) not null,
# bytes bigint not null default 0,
# messages integer not null default 0,
# primary key (username)
# );
map {
pattern =
Re: [Dovecot] Quota not working with dict proxy
On Sun, May 12, 2013 12:24 pm, Chris Richards wrote: Hello all, I'm sure this has been covered somewhere before, but my googlefu is not up to the challenge. More info; this is the debug output from the doveadm command: doveadm -Df tab quota recalc -u 'user@domain' doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot doveadm(root): Debug: Module loaded: usr/lib64/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_ doveadm_fts_plugin.so: undefined symbol: fts_backend_rescan (this is usually intentional, so just ignore this message) doveadm(user@domain): Debug: auth input: user@domain home=/home/vmail/domains/domain/user/ mail=maildir:/home/vmail/domains/domain/user/Maildir/ uid=1004 gid=100 quota_rule=*:storage=200M doveadm(user@domain): Debug: Added userdb setting: mail=maildir:/home/vmail/domains/domain/user/Maildir/ doveadm(user@domain): Debug: Added userdb setting: plugin/quota_rule=*:storage=200M doveadm(user@domain): Debug: Effective uid=1004, gid=100, home=/home/vmail/domains/domain/user/ doveadm(user@domain): Debug: Quota root: name=User quota backend=dict args=user@domain:proxy::quota doveadm(user@domain): Debug: Quota rule: root=User quota mailbox=* bytes=209715200 messages=0 doveadm(user@domain): Debug: Quota warning: bytes=207618048 (99%) messages=0 reverse=no command=quota-warning 99 user domain doveadm(user@domain): Debug: Quota warning: bytes=199229440 (95%) messages=0 reverse=no command=quota-warning 95 user domain doveadm(user@domain): Debug: Quota warning: bytes=167772160 (80%) messages=0 reverse=no command=quota-warning 80 user domain doveadm(user@domain): Debug: Quota warning: bytes=199229440 (95%) messages=0 reverse=yes command=quota-warning 'less than 95' user domain doveadm(user@domain): Debug: dict quota: user=user@domain, uri=proxy::quota, noenforcing=0 doveadm(user@domain): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/vmail/domains/domain/user/Maildir/ doveadm(user@domain): Debug: maildir++: root=/home/vmail/domains/domain/user/Maildir, index=, control=, inbox=/home/vmail/domains/domain/user/Maildir, alt= I don't see anything here that would explain why it is deleting my user account, but I can tell you with certainty that this command is doing SOMETHING that is resulting in the deletion of my account.
Re: [Dovecot] Random LDA failure to access auth socket
Daniel,
Just wanted to respond back and let you know that changing permissions to
dovecot:dovecot as you suggested seems to have resolved the issue; I've
not seen any more occurrences of this error.
Thanks again for your assistance!
Chris
On Sun, March 3, 2013 5:13 pm, Daniel Parthey wrote:
Hi Chris,
Chris Richards wrote:
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
user = $default_internal_user
}
In order for dovecot-lda to work, default internal user dovecot
seems to need permission for the user listing. This should work,
but you should try to narrow the permissions down:
service auth {
unix_listener auth-userdb {
group = dovecot
mode = 0666
user = dovecot
}
}
Documentation http://wiki2.dovecot.org/LDA says:
The auth-userdb socket can be used to do userdb lookups for given
usernames or
get a list of all users. Typically the result will contain the user's UID,
GID
and home directory, but depending on your configuration it may return
other
information as well. So the information is similar to what can be found
from
eg. /etc/passwd for system users. This means that it's probably not a
problem
to use mode=0666 for the socket, but you should try to restrict it more
just to
be safe.
hermes conf.d # stat /usr/libexec/dovecot/deliver
File: '/usr/libexec/dovecot/deliver' - 'dovecot-lda'
Size: 11 Blocks: 0 IO Block: 4096 symbolic
link
Device: 805h/2053d Inode: 267375 Links: 1
Access: (0777/lrwxrwxrwx) Uid: (0/root) Gid: (0/root)
Access: 2012-11-24 17:44:04.440976879 +
Modify: 2012-11-24 17:44:04.440976879 +
Change: 2012-11-24 17:44:04.440976879 +
Birth: -
deliver is a symbolic link to dovecot-lda, so its basically the same.
hermes conf.d # stat /usr/libexec/dovecot/dovecot-lda
File: '/usr/libexec/dovecot/dovecot-lda'
Size: 22432 Blocks: 48 IO Block: 4096 regular file
Device: 805h/2053d Inode: 849010 Links: 1
Access: (0755/-rwxr-xr-x) Uid: (0/root) Gid: (0/root)
Access: 2012-11-24 17:43:57.124794021 +
Modify: 2012-11-24 17:44:02.204920992 +
Change: 2012-11-24 17:44:04.444976978 +
Birth: -
No setuid/setgid flags set.
In Postfix master.cf, I have the following:
dovecot unix -n n - - pipe
flags=DRhu user=vmail:users argv=/usr/libexec/dovecot/deliver -f
${sender} -d ${user}@${nexthop}
I'm wondering why user=vmail:users does not have the desired effect
and dovecot-lda uses the effective uid dovecot and effective gid
dovecot
to do the user lookups.
Regards
Daniel
Re: [Dovecot] Random LDA failure to access auth socket
I was finally able to make this change. It doesn't appear to have broken
anything, which is a plus. ;)
I'll ping back in a couple of days or so and let you know if the issue
appears resolved.
Thanks again for your help!
Chris
On Sun, March 3, 2013 6:31 pm, Chris Richards wrote:
In order for dovecot-lda to work, default internal user dovecot
seems to need permission for the user listing. This should work,
but you should try to narrow the permissions down:
service auth {
unix_listener auth-userdb {
group = dovecot
mode = 0666
user = dovecot
}
}
I'll give this a whirl tomorrow during off-mail time. I'm thinking there
was a reason I did vmail:users, but it's not coming to me at the moment.
I'm also puzzled by why lda is using dovecot:dovecot rather than
vmail:users. Does it drop back to dovecot:dovecot and retry if the
requested uid:gid fails to work? I can't imagine where postfix would get
uid:gid of dovecot:dovecot to call with, so I don't think it's a postfix
problem.
Thanks for your help!
Chris
Re: [Dovecot] Random LDA failure to access auth socket
In order for dovecot-lda to work, default internal user dovecot
seems to need permission for the user listing. This should work,
but you should try to narrow the permissions down:
service auth {
unix_listener auth-userdb {
group = dovecot
mode = 0666
user = dovecot
}
}
I'll give this a whirl tomorrow during off-mail time. I'm thinking there
was a reason I did vmail:users, but it's not coming to me at the moment.
I'm also puzzled by why lda is using dovecot:dovecot rather than
vmail:users. Does it drop back to dovecot:dovecot and retry if the
requested uid:gid fails to work? I can't imagine where postfix would get
uid:gid of dovecot:dovecot to call with, so I don't think it's a postfix
problem.
Thanks for your help!
Chris
[Dovecot] Random LDA failure to access auth socket
Greetings all.
Please forgive me if I'm posting something that has already been
addressed, but my google-foo is not strong enough to find the solution.
I've got a dovecot server running version 2.0.19 on Gentoo Hardened. I
have Postfix as my MTA, and it is calling the Dovecot LDA to deliver the
mail. Everything is working great, mail is being delivered, and the users
are happy.
However, I am noticing that I have random entries like this:
Mar 1 20:19:23 hermes dovecot: lda: Error: userdb lookup:
connect(/var/run/dovecot/auth-userdb) failed: Permission denied
(euid=97(dovecot) egid=97(dovecot) missing +r perm:
/var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755)
Mar 1 20:19:23 hermes dovecot: lda: Fatal: Internal error occurred. Refer
to server log for more information.
Since these messages came from the server log, looking in the server log
for more information is rather pointless.
I have 10-master.conf configured as follows:
service auth {
unix_listener auth-userdb {
mode = 0600
user = vmail
group = vmail
}
In 15-lda.conf I have:
protocol lda {
mail_plugins = $mail_plugins sieve quota
}
In Postfix master.cf, I have the following:
dovecot unix -n n - - pipe
flags=DRhu user=vmail:users argv=/usr/libexec/dovecot/deliver -f
${sender} -d ${user}@${nexthop}
Permissions on the socket are:
srw--- 1 vmail vmail 0 Feb 24 23:43 auth-userdb
Judging from the log, my problem is that the LDA appears to be trying to
access the socket as the dovecot user (rather than the vmail user), but my
question is why? What logging can I use to tell me if postfix is calling
with the wrong permissions or if the lda is somehow getting confused?
Any help or suggestions greatly appreciated.
Thanks,
Gizmo
Re: [Dovecot] Random LDA failure to access auth socket
Thanks for the links: those were what I followed when I set things up.
The dovecot user is the $default_internal_user.
hermes conf.d # dovecot -n
# 2.1.9: /etc/dovecot/dovecot.conf
# OS: Linux 3.4.2-hardened-r1-bordernet x86_64 Gentoo Base System release
2.1 ext4
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@
auth_verbose_passwords = plain
disable_plaintext_auth = no
login_greeting = Awaiting command...
mail_location = maildir:/home/vmail/%d/%n/Maildir
mail_plugins = quota
mail_privileged_group = 100
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
expire = Trash 7
expire2 = Trash/* 7
expire3 = Spam 7
quota = maildir:User quota
quota_rule = *:storage=200M
quota_warning = storage=95%% quota-warning 95 %n %d
quota_warning2 = storage=80%% quota-warning 80 %n %d
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
sieve_global_dir = /home/vmail/dovecot/sieve
sieve_global_path = /home/vmail/dovecot/sieve/default.sieve
}
protocols = imap pop3 sieve
service auth-worker {
user = $default_internal_user
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
user = $default_internal_user
}
service quota-warning {
executable = script /etc/dovecot/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl_cert = /etc/ssl/dovecot/server.pem
ssl_key = /etc/ssl/dovecot/server.key
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
protocol lmtp {
mail_plugins = sieve quota
}
protocol lda {
mail_plugins = quota sieve quota
}
protocol imap {
mail_max_userip_connections = 20
mail_plugins = quota quota imap_quota
}
protocol pop3 {
mail_plugins = quota quota
}
hermes conf.d # stat /usr/libexec/dovecot/deliver
File: '/usr/libexec/dovecot/deliver' - 'dovecot-lda'
Size: 11 Blocks: 0 IO Block: 4096 symbolic link
Device: 805h/2053d Inode: 267375 Links: 1
Access: (0777/lrwxrwxrwx) Uid: (0/root) Gid: (0/root)
Access: 2012-11-24 17:44:04.440976879 +
Modify: 2012-11-24 17:44:04.440976879 +
Change: 2012-11-24 17:44:04.440976879 +
Birth: -
hermes conf.d # stat /usr/libexec/dovecot/dovecot-lda
File: '/usr/libexec/dovecot/dovecot-lda'
Size: 22432 Blocks: 48 IO Block: 4096 regular file
Device: 805h/2053d Inode: 849010 Links: 1
Access: (0755/-rwxr-xr-x) Uid: (0/root) Gid: (0/root)
Access: 2012-11-24 17:43:57.124794021 +
Modify: 2012-11-24 17:44:02.204920992 +
Change: 2012-11-24 17:44:04.444976978 +
Birth: -
Thanks in advance for your help.
Chris
On Sat, March 2, 2013 11:15 am, Daniel Parthey wrote:
Chris Richards wrote:
I've got a dovecot server running version 2.0.19 on Gentoo Hardened. I
have Postfix as my MTA, and it is calling the Dovecot LDA to deliver the
mail. Everything is working great, mail is being delivered, and the
users
are happy.
However, I am noticing that I have random entries like this:
Mar 1 20:19:23 hermes dovecot: lda: Error: userdb lookup:
connect(/var/run/dovecot/auth-userdb) failed: Permission denied
(euid=97(dovecot) egid=97(dovecot) missing +r perm:
/var/run/dovecot/auth-userdb, dir owned by 0:0 mode=0755)
Mar 1 20:19:23 hermes dovecot: lda: Fatal: Internal error occurred.
Refer
to server log for more information.
In Postfix master.cf, I have the following:
dovecot unix -n n - - pipe
flags=DRhu user=vmail:users argv=/usr/libexec/dovecot/deliver -f
${sender} -d ${user}@${nexthop}
Permissions on the socket are:
srw--- 1 vmail vmail 0 Feb 24 23:43 auth-userdb
Judging from the log, my problem is that the LDA appears to be trying to
access the socket as the dovecot user (rather than the vmail user), but
my
question is why? What logging can I use to tell me if postfix is
calling
with the wrong permissions or if the lda is somehow getting confused?
Have a look at
http://wiki2.dovecot.org/LDA
http://wiki2.dovecot.org/LDA/Postfix
Maybe there you will find some helpful hints.
Is /usr/libexec/dovecot/deliver
Re: [Dovecot] DoveCot IMAP and inconsistent state messages
Mar 31 14:00:58 myserver dovecot: IMAP(me): Maildir /home/me/.Maildir sync: UID inserted in the middle of mailbox (85953 85053, file = msg.XL7B:2,) This is a procmail configuration problem: http://wiki.dovecot.org/MailboxFormat/Maildir#procmail Just wanted to say thanks for the help. You guys have made me a hero. After reading the PostFix group and this group, and spending about a day sorting through a bunch of spam configuration stuff, I've got our mail server purring like a kitten. So I went to see the CTO this afternoon and explained the situation with the policy about RPMs. After him commenting that he had noticed a substantial drop in spam, his response was basically You seem to have a clue; do whatever you think is appropriate. Thanks again, guys. Later, Chris
Re: [Dovecot] DoveCot IMAP and inconsistent state messages
Timo Sirainen wrote: On Mon, 2008-03-31 at 15:10 -0500, Chris Richards wrote: Dovecot is version 1.0.rc15 I agree with others that it would be a good idea to upgrade from rc15, but.. I'm working on this Mar 31 14:00:58 myserver dovecot: IMAP(me): Maildir /home/me/.Maildir sync: UID inserted in the middle of mailbox (85953 85053, file = msg.XL7B:2,) This is a procmail configuration problem: http://wiki.dovecot.org/MailboxFormat/Maildir#procmail Ok, As I read this, basically you're saying that I need to append the / to my delivery dirs? Or that I need to append the / and only deliver to Maildir, instead of Maildir/new? Here's my .procmailrc # .procmailrc # routes incoming mail to appropriate destinations PATH=/usr/bin:/usr/sbin:/bin:/sbin MAILDIR=$HOME/.Maildir # all mailboxes live here DEFAULT=$MAILDIR/new/ # This is where incoming mail goes LOGFILE=$HOME/.procmail_log DELIVER=/usr/lib/dovecot/deliver SHELL=/bin/sh :0: * ^X-Spam-Flag: YESS * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* $MAILDIR/.Spam/new/ I changed the .procmailrc to append the / to the dir. Is that all I need to do? Thanks, Chris
Re: [Dovecot] DoveCot IMAP and inconsistent state messages
Chris Richards wrote: Ok, As I read this, basically you're saying that I need to append the / to my delivery dirs? Or that I need to append the / and only deliver to Maildir, instead of Maildir/new? Here's my .procmailrc # .procmailrc # routes incoming mail to appropriate destinations PATH=/usr/bin:/usr/sbin:/bin:/sbin MAILDIR=$HOME/.Maildir # all mailboxes live here DEFAULT=$MAILDIR/new/ # This is where incoming mail goes LOGFILE=$HOME/.procmail_log DELIVER=/usr/lib/dovecot/deliver SHELL=/bin/sh :0: * ^X-Spam-Flag: YESS * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* $MAILDIR/.Spam/new/ I changed the .procmailrc to append the / to the dir. Is that all I need to do? Thanks, Chris Never mind. I've figured this out. The above configuratioon seems to do Bad Things to mail delivery. I've change my config to the following, and things seem to be working: # .procmailrc # routes incoming mail to appropriate destinations PATH=/usr/bin:/usr/sbin:/bin:/sbin MAILDIR=$HOME/.Maildir # all mailboxes live here DEFAULT=$MAILDIR/new/ # This is where incoming mail goes LOGFILE=$HOME/.procmail_log DELIVER=/usr/lib/dovecot/deliver SHELL=/bin/sh :0: * ^X-Spam-Flag: YESS * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* $MAILDIR/.Spam/new/ Thanks, Chris
Re: [Dovecot] DoveCot IMAP and inconsistent state messages
Timo Sirainen wrote: On Tue, 2008-04-01 at 14:55 -0500, Chris Richards wrote: DEFAULT=$MAILDIR/new/ # This is where incoming mail goes $MAILDIR/.Spam/new/ Do these really work? They're not writing mails to new/new/ directory? I think they should have been without the new/ part. Yeah, that was what I discovered. *g* Later, Chris
Re: [Dovecot] DoveCot IMAP and inconsistent state messages
Charles Marcus wrote: No groan here... I love the flexibility Gentoo gives... you can easily leave the primary system at 'stable', then just set certain packages to unstable for the latest/greatest. My servers have been running nonstop for over 3 years, with just a few minor hiccups now and then that are quickly resolved by digging through logs and/or hitting google and/or the forums... To be fair, most distros *do* have the ability to use extra 'unstable' repos, at least for most major packages, and if I wasn't using Gentoo, I would at *least* be using those... My other box is Gentoo, and I quite like it. The biggest problem I had with it was that about a year ago when I was give control of it, it hadn't been synced in like 3 years, and it was so woefully out of date that when I tried to emerge -upDN world, it couldn't reliably upgrade because some packages no longer existed, including core packages (and the system profile). Other than that, the only problem I've had was when a Metalog (sysloger) update came out that caused my entire system to hang at boot because the portage package didn't properly move a couple of files. Later, Chris
[Dovecot] DoveCot IMAP and inconsistent state messages
I need some help troubleshooting this problem. It only shows up with IMAP connections. I initially thought it was related to SquirrelMail (because it gives me an 'EXPUNGE' error), but after attempting to send IMAP commands directly to the server as shown below, I'm thinking there is something else going on. The system configuration is PostFix as the MTA delivering to procmail in the mailbox (maildir format), with Dovecot handling POP3 and IMAP Postfix is version 2.3.3 Dovecot is version 1.0.rc15 procmail is version 3.22 2001/09/10 kernel is 2.6.18-53.1.14.el5, CentOS 5 There is additional info about the system configuration after the command sequence and log entries below. I've left my typos in, just in case they somehow bear on the problem. I omitted about 2050 messages from the mailbox FETCH listing, since they were all identical but for the message number, and I didn't figure you'd want to read all of that. :) Command sequence: [EMAIL PROTECTED] telnet locahost 143 locahost/143: Name or service not known [EMAIL PROTECTED] telnet localhost 143 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. * OK myserver awaiting command login me mypass login BAD Error in IMAP command received by server. a001 login me mypass a001 OK Logged in. a002 list Mail * a002 OK List completed. a003 list Folders * a003 OK List completed. a004 list Inbox a004 BAD Error in IMAP command LIST: Invalid arguments. a005 list Inbox * * LIST (\HasNoChildren) . INBOX a005 OK List completed. a006 list * * LIST (\HasNoChildren) . Drafts * LIST (\HasNoChildren) . Spam * LIST (\HasNoChildren) . Trash * LIST (\HasNoChildren) . Sent * LIST (\HasNoChildren) . INBOX a006 OK List completed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 2088 EXISTS * 1 RECENT * OK [UNSEEN 2088] First unseen. * OK [UIDVALIDITY 1190322195] UIDs valid * OK [UIDNEXT 89700] Predicted next UID a007 OK [READ-WRITE] Select completed. a008 FETCH l:* FLAGS a008 BAD Error in IMAP command FETCH: Invalid messageset a008 FETCH 1:* FLAGS * 1 FETCH (FLAGS (\Seen)) * 2 FETCH (FLAGS (\Seen)) * 3 FETCH (FLAGS (\Seen)) * 4 FETCH (FLAGS (\Seen)) * 5 FETCH (FLAGS (\Seen)) * 6 FETCH (FLAGS (\Seen)) . . . * 2083 FETCH (FLAGS (\Seen)) * 2084 FETCH (FLAGS (\Seen)) * 2085 FETCH (FLAGS (\Seen)) * 2086 FETCH (FLAGS (\Seen)) * 2087 FETCH (FLAGS (\Seen)) * 2088 FETCH (FLAGS (\Recent)) * BYE Mailbox is in inconsistent state, please relogin. Connection closed by foreign host. These are the corresponding entries from my mail syslog during the same time period. I've included the postfix entries, as they might be relevant to the problem. Mar 31 13:55:14 myserver dovecot: imap-login: Login: user=me, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Mar 31 13:55:33 myserver dovecot: pop3-login: Login: user=me, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy Mar 31 13:55:35 myserver dovecot: POP3(me): Disconnected: Logged out top=0/0, retr=1/18590, del=2/2089, size=10863040 Mar 31 13:56:29 myserver postfix/smtpd[26589]: connect from unknown[79.165.160.211] Mar 31 13:56:30 myserver postfix/smtpd[26589]: 1A57E2F7E6: client=unknown[79.165.160.211] Mar 31 13:56:30 myserver postfix/cleanup[26646]: 1A57E2F7E6: message-id=[EMAIL PROTECTED] Mar 31 13:56:35 myserver postfix/qmgr[2254]: 1A57E2F7E6: from=[EMAIL PROTECTED], size=61232, nrcpt=1 (queue active) Mar 31 13:56:36 myserver postfix/smtpd[26651]: connect from localhost.localdomain [127.0.0.1] Mar 31 13:56:36 myserver postfix/smtpd[26651]: C98262F893: client=localhost.localdomain [127.0.0.1] Mar 31 13:56:36 myserver postfix/cleanup[26646]: C98262F893: message-id=[EMAIL PROTECTED] Mar 31 13:56:36 myserver postfix/smtpd[26651]: disconnect from localhost.localdomain [127.0.0.1] Mar 31 13:56:36 myserver amavis[26499]: (26499-03) Passed SPAMMY, [79.165.160.211] [79.165.160.211] [EMAIL PROTECTED] - [EMAIL PROTECTED], Message-ID: [EMAIL PROTECTED], mail_id: 47TfQi44++zM, Hits: 17.608, size: 61232, queued_as: C98262F893, 1372 ms Mar 31 13:56:36 myserver postfix/smtp[26648]: 1A57E2F7E6: to=[EMAIL PROTECTED], orig_to=[EMAIL PROTECTED], relay=127.0.0.1[127.0.0.1]:10024, delay=6.8, delays=5.4/0.01/0.01/1.4, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as C98262F893)Mar 31 13:56:36 myserver postfix/qmgr[2254]: C98262F893: from=[EMAIL PROTECTED], size=62235, nrcpt=1 (queue active) Mar 31 13:56:36 myserver postfix/qmgr[2254]: 1A57E2F7E6: removed Mar 31 13:56:36 myserver postfix/local[26653]: C98262F893: to=[EMAIL PROTECTED], relay=local, delay=0.12, delays=0.1/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail) Mar 31 13:56:36 myserver postfix/qmgr[2254]: C98262F893: removed Mar 31 13:58:07 myserver dovecot: pop3-login: Login: user=someone, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy Mar 31 13:58:07 myserver dovecot: POP3(someone):
Re: [Dovecot] DoveCot IMAP and inconsistent state messages
Charles Marcus wrote: On 3/31/2008, Chris Richards ([EMAIL PROTECTED]) wrote: Dovecot is version 1.0.rc15 Upgrade please - rc15 is very old... Err, that's the newest thing in the yum repository, and if I go compiling code that isn't 'official' (i.e. doesn't come from the yum repository), I'll get myself in an awful lot of trouble. Management seems to be under the strange belief that only code from the repository it is 'safe'. So, the question is, how hard do I need to fight in order to get this done? Later, Chris
Re: [Dovecot] DoveCot IMAP and inconsistent state messages
Scott Silva wrote: on 3-31-2008 1:31 PM Chris Richards spake the following: Charles Marcus wrote: On 3/31/2008, Chris Richards ([EMAIL PROTECTED]) wrote: Dovecot is version 1.0.rc15 Upgrade please - rc15 is very old... Err, that's the newest thing in the yum repository, and if I go compiling code that isn't 'official' (i.e. doesn't come from the yum repository), I'll get myself in an awful lot of trouble. Management seems to be under the strange belief that only code from the repository it is 'safe'. So, the question is, how hard do I need to fight in order to get this done? Later, Chris If you are using a RHEL or clone, atrpms.net has a newer version in their repository. Oh bloody hell. WHY am I fighting with a dovecot version that is 1 1/2 flippin' years old?!?! Suits. I HATES 'em, I tells ya! I'll be back when I've figured out how to convince the CTO that this stupid policy is causing a lot of unnecessary grief, considering this issue is probably already fixed. Thanks for your time, guys. Later, Chris
