Re: dsync I/O Stalled
Hi guys, For anyone who have this problem it could be related to routing MTU. In my case I've found that a new router was breaking the mtu so big packet are fragmented and this cause stalled between the replicas. Thank you for your time Paolo Il 13 luglio 2021 20:39:28 CEST, "Stefan Möding" ha scritto: >Paolo Daniele writes: > >> dsync-server(aa...@bbb.bb): Error: Timeout during state=sync_mails >(send=done >> recv=mails) > >I experience the same errors in my setup. It seems to happen in >intervals >of "replication_full_sync_interval" for me. Specifically if the "full >sync" time in the output of "doveadm replicator status '*'" reaches the >full sync interval time for multiple users. > >-- >Stefan -- Inviato dal mio dispositivo Android con K-9 Mail. Perdonate la brevità.
dsync I/O Stalled
Hi guys, i've a strange problem between 2 servers. It happened after a changing on our network core routing infrastructure. We've moved to Juniper network and eBGP and iBGP of our server. One of this server is behind a firewall and communicate with another server in another country to sync our emails. Doveadmin replica seems to be fine, but sometimes i get this error message: dsync-server(aa...@bbb.bb): Error: Timeout during state=sync_mails (send=done recv=mails) It seems like the remote server has troubles to reach our local server but if i do a stupid ping i'm not losing anything. Maybe is there something am i missing? I thought that it could be related to a MTU problem but i'm not so sure and i've increased MTU on the routers involved in routing process without success. I wa missing that both servers are behind a load balancer wich is in our local DC. Can you give me some hints? Regards, Paolo -- Paolo Daniele Intendo s.r.l.E-mail: paolo.dani...@intendo.it Business Unit TLCWEB Tel.: +39 0968 430786 Via Coschi, 11Skype: paolo_daniele1 88046 - Lamezia Terme, Cz Web:http://www.tlcweb.it ItaliaWeb:http://www.intendo.it
Auth driver
(resending to the list; I apologize, I'm not using my usual email client) Hello, That's actually great news. The perspective of working in-tree didn't make me particularly happy. Could you point me to any documentation or examples? While I can find many plugins in the repo and around the Internet, I could find none which add authdb drivers. Best Regards, Riccardo P. Bestetti Da: Aki Tuomi Inviato: venerdì 9 agosto 2019 13:56 A: Riccardo Paolo Bestetti ; dovecot@dovecot.org Oggetto: Re: Auth driver On 9.8.2019 14.51, Riccardo Paolo Bestetti via dovecot wrote: > Greetings! > > I'm planning to implement a new auth driver. It's going to be, in concept, > similar to the Lua and CheckPassword drivers, in that it allows an user > program to carry out the authentication and user enumeration steps. If you do this, please make it as 3rd party repository. Dovecot auth supports plugins. Aki
Auth driver
Greetings! I'm planning to implement a new auth driver. It's going to be, in concept, similar to the Lua and CheckPassword drivers, in that it allows an user program to carry out the authentication and user enumeration steps. The rationale is that such a solution would provide better decoupling between Dovecot and the user authentication and enumeration logic. Existing integrated logic (auth/ directory in the source tree) is, even encompassing a great number of common use cases, ultimately inflexible. Writing custom drivers implementing logic, while being a solution to the inflexibility problem, is clearly not what drivers were engineered for: for one they don't support dynamic loading, making it necessary to recompile Dovecot to make changes, which is unacceptable in many situations, e.g. when using a packaged or containerized version of the software. In this email, I intend to explain what characteristics my driver will have and why it solves the proposed problem better than existing code (such as CheckPassword, Lua and proxy), in the hope to get useful feedback and I will pose some technical questions to which I could not find definitive answers by myself yet. These are going to be the characteristics of my driver (with particular emphasis to the differences to the existing drivers): - It is going to be a completely generic mechanism exposing all funcionality of Dovecot's interface, both password check and enumeration, and will fully support authentication methods with continuations. - It is going to be performant, communicating with external programs through sockets (unix or inet). - It is going to communicate using a simple, dynamic-schema, well-understood protocol, which is probably going to be JSON, being very easy to generate and parse. This is why the existing drivers do not already provide what I described: - CheckPassword is heavy (fork & 2*exec), and it's generally a mess, having initially been provided as a compatibility feature and having then been hacked to support additional functionality; it's architecturally unfeasable to support continuations with it and anyway any further extension will probably make it explode :) - Lua works but it's limited, in that it dictates a language and also dictates that the auth logic should run on the same server as Dovecot (and under Dovecot's control). It also doesn't seem to support plain password checks, but only user-password enumeration, making it non-generic (at least this is what can be inferred from the documentation and examples.) - Other possible solutions, such as (mis)using the proxy driver to communicate with an external program are not well documented and nothing guarantees the proxy protocol will not change anyway, being an internal feature. And now, the technical questions. I know these could be answered by reading the code, which is something I'm doing, but I'm finding some architectural details quite difficult to grasp due to their complexity: - From what I gather from [1], auth-worker processes only process a single request before being killed from auth. Would setting service_count to something different cause auth-workers to be reused or would it just leak the auth-workers? In the first case, is the preinit of the auth driver repeated, or is the existing instance of the auth driver reused? You could also point me to the relevant code sections which start, keep track of and initialize auth-workers. - How does a driver specify what to run in an auth-worker as opposed to in the auth process? (This is strongly related to the first question, and maybe it is more general, so you may want to answer this one first.) Please, feel free to only answer parts of my email. Any input on any of the matters is very welcome. Best regards, Riccardo P. Bestetti [1] https://wiki.dovecot.org/Services#auth-worker
Re: Dovecot behind Load Balancer
Il 10/07/19 11:44, Jean-Daniel Dupas ha scritto: Le 10 juil. 2019 à 10:24, Paolo Daniele via dovecot a écrit : Il 10/07/19 10:20, Aki Tuomi ha scritto: On 12/06/2019 20:02 Paolo Daniele via dovecot wrote: Hi, i've a question for you. I've two dovecot imap/pop server behind a zen load balancer. Load balancing is made by lx4nat so the public ip address of my load balancer contact directly the dovecot servers. Since few months i've a message from thunderbird that i've reached the imap limit login for ip. I've triend to increase the max user ip parameter but sometimes i've the same problem. It's a strange things that actually i'm able to mitigate by reduce the number of cached connections in Thunderbird but it's not normal. What do you think about that? Maybe there's some tuning that you can suggest. Thank you, Paolo Have you ensured, by checking logs, that the connections are seen by dovecot to come from public IP addresses? Also, thunderbird is known to open lots of concurrent connections. Aki Yes, connections are coming from the ip address of load balancer (also checked with a netstat -an) If connection are seen as coming from the IP address of the load balancer, isn't it normal that dovecot complains ? That means that dovecot sees all connections as coming from a single client, which would explain why you reach that limit. Yeah of course, but i've checked that i haven't reach the max_user_per_ip limit by counting dovecot process coming from that ip address. So the strange and the reason why i'm writing to you :) -- Paolo Daniele E-mail: paolo.dani...@tlcweb.it Intendo s.r.l.Mobile: +39 388 9887357 Business Unit TLCWEB Tel.: +39 0968 430786 Via Coschi, 11Skype: paolo_daniele1 88046 - Lamezia Terme, Cz Web:http://www.tlcweb.it ItaliaWeb:http://www.intendo.it
Re: Dovecot behind Load Balancer
Il 10/07/19 10:27, Sami Ketola ha scritto: On 10 Jul 2019, at 11.24, Paolo Daniele via dovecot wrote: connections are coming from the ip address of load balancer (also checked with a netstat -an) I know that thunderbird has this problem and i haven't problem since the numbers of email domain increase. I'm not sure the impact of rising the max user ip to an higher value in terms of ram consumption and cpu Any suggestion? At some point in time thunderbird opened 1 connection per email folder on destination. So if you have 20 folders thunderbird would open 20 connections. Not sure if it is still true with current thunderbird versions. There is a limit on thunderbird config that controls this. Sami I've checked the limitation on thunderbid and i've mitigated on our pcs (which have multiple email configured on the same servers) by telling thunderbird to store only one connection at the same time. But i cannot "force" or contact every customer to check what client they use and change :) This also create problems with Roundcube webmail which sometimes give me the same error. Paolo -- Paolo Daniele E-mail: paolo.dani...@tlcweb.it Intendo s.r.l.Mobile: +39 388 9887357 Business Unit TLCWEB Tel.: +39 0968 430786 Via Coschi, 11Skype: paolo_daniele1 88046 - Lamezia Terme, Cz Web:http://www.tlcweb.it ItaliaWeb:http://www.intendo.it
Re: Dovecot behind Load Balancer
Il 10/07/19 10:27, Aki Tuomi ha scritto: On 10/07/2019 11:24 Paolo Daniele via dovecot wrote: Il 10/07/19 10:20, Aki Tuomi ha scritto: On 12/06/2019 20:02 Paolo Daniele via dovecot wrote: Hi, i've a question for you. I've two dovecot imap/pop server behind a zen load balancer. Load balancing is made by lx4nat so the public ip address of my load balancer contact directly the dovecot servers. Since few months i've a message from thunderbird that i've reached the imap limit login for ip. I've triend to increase the max user ip parameter but sometimes i've the same problem. It's a strange things that actually i'm able to mitigate by reduce the number of cached connections in Thunderbird but it's not normal. What do you think about that? Maybe there's some tuning that you can suggest. Thank you, Paolo Have you ensured, by checking logs, that the connections are seen by dovecot to come from public IP addresses? Also, thunderbird is known to open lots of concurrent connections. Aki Yes, connections are coming from the ip address of load balancer (also checked with a netstat -an) I know that thunderbird has this problem and i haven't problem since the numbers of email domain increase. I'm not sure the impact of rising the max user ip to an higher value in terms of ram consumption and cpu Any suggestion? Thanks for your answer! Paolo What's your current limit? The default value is 100. Aki Actually: mail_max_userip_connections = 500 Thanks, Paolo -- Paolo Daniele E-mail: paolo.dani...@tlcweb.it Intendo s.r.l.Mobile: +39 388 9887357 Business Unit TLCWEB Tel.: +39 0968 430786 Via Coschi, 11Skype: paolo_daniele1 88046 - Lamezia Terme, Cz Web:http://www.tlcweb.it ItaliaWeb:http://www.intendo.it
Re: Dovecot behind Load Balancer
Il 10/07/19 10:20, Aki Tuomi ha scritto: On 12/06/2019 20:02 Paolo Daniele via dovecot wrote: Hi, i've a question for you. I've two dovecot imap/pop server behind a zen load balancer. Load balancing is made by lx4nat so the public ip address of my load balancer contact directly the dovecot servers. Since few months i've a message from thunderbird that i've reached the imap limit login for ip. I've triend to increase the max user ip parameter but sometimes i've the same problem. It's a strange things that actually i'm able to mitigate by reduce the number of cached connections in Thunderbird but it's not normal. What do you think about that? Maybe there's some tuning that you can suggest. Thank you, Paolo Have you ensured, by checking logs, that the connections are seen by dovecot to come from public IP addresses? Also, thunderbird is known to open lots of concurrent connections. Aki Yes, connections are coming from the ip address of load balancer (also checked with a netstat -an) I know that thunderbird has this problem and i haven't problem since the numbers of email domain increase. I'm not sure the impact of rising the max user ip to an higher value in terms of ram consumption and cpu Any suggestion? Thanks for your answer! Paolo -- Paolo Daniele E-mail: paolo.dani...@tlcweb.it Intendo s.r.l.Mobile: +39 388 9887357 Business Unit TLCWEB Tel.: +39 0968 430786 Via Coschi, 11Skype: paolo_daniele1 88046 - Lamezia Terme, Cz Web:http://www.tlcweb.it ItaliaWeb:http://www.intendo.it
Dovecot behind Load Balancer
Hi, i've a question for you. I've two dovecot imap/pop server behind a zen load balancer. Load balancing is made by lx4nat so the public ip address of my load balancer contact directly the dovecot servers. Since few months i've a message from thunderbird that i've reached the imap limit login for ip. I've triend to increase the max user ip parameter but sometimes i've the same problem. It's a strange things that actually i'm able to mitigate by reduce the number of cached connections in Thunderbird but it's not normal. What do you think about that? Maybe there's some tuning that you can suggest. Thank you, Paolo
Re: Recommended tool for migrating IMAP servers
Il 04/12/2017 17:37, Michael Slusarz ha scritto: I think Davide was asking about dsync. If so, the answer is no: dsync works only when both servers are Dovecot and needs some additional configuration to work through the network (see https://wiki2.dovecot.org/Replication). This is entirely incorrect. The source platform for dsync can be ANY IMAP/POP server. The recommended tool for migrating into Dovecot is dsync. You don't need any other tool, and other tools aren't going to preserve state so they are pretty much worthless for a real-world in-place migration. Entirely?! At most half incorrect. My apologies, I didn't know about the dsync's IMAP feature (I suppose it wasn't there from the beginning). This fact still remain: you can use dsync only when dovecot is involved. Even so (IMHO) the sentence "You don't need any other tool" is a bit too much. Who ever know all possible use cases in the world? Cheers Paolo
Re: Recommended tool for migrating IMAP servers
Il 04/12/2017 14:33, x9p ha scritto: Can I use this tool even if I do not know the other remote server typology? sure. just need both IMAP ports reachable and valid user/pass for both servers. I think Davide was asking about dsync. If so, the answer is no: dsync works only when both servers are Dovecot and needs some additional configuration to work through the network (see https://wiki2.dovecot.org/Replication). I don't know about imapsync but I suppose it is a generic IMAP tool that replicates mailboxes using IMAP protocol as a client between two servers. Cheers Paolo
Re: Sieve can't move messages in a public namespace
Hello all,
finally I upgraded to dovecot 2.2.10 (that comes with CentOS 7.4).
But the problem is still there!
Why sieve can't move a message in a public namespace complaining about
"Mailbox does not exists"?
None of you ever try? Some of you have a working setup?
I attach "dovecot -n" output.
The sieve script is:
require ["fileinto", "envelope", "mailbox"];
if header :contains ["From"] "a...@address.com" {
fileinto "Cond-UFF_FORNITORI/CONTABILITA/Ricevuta";
}
The error:
error: msgid=<5df93e60-676c-f6bb-68b8-c17fe7493...@address.com>:
failed to store into mailbox
'Cond-UFF_FORNITORI/CONTABILITA/Ricevuta': Mailbox doesn't exist:
Cond-UFF_FORNITORI/CONTABILITA/Ricevuta
Il 04/07/2017 10:34, Stephan Bosch ha scritto:
Op 7/4/2017 om 8:44 AM schreef Paolo:
Il 30/06/2017 10:46, Paolo ha scritto:
Hello,
my dovecot version is 2.1.17
I've configured some public namespaces (config attached). When a mail
arrives at a certain mail address, that mail is handled by dovecot
lmtp server and a sieve script is executed that shuold move the
message in one of the namespaces. This is the script:
require "fileinto";
if header :contains "From""exam...@example.com" {
fileinto "Cond-UFF_FORNITORI/CONTABILITA/Inviata";
}
else {
fileinto "Cond-UFF_FORNITORI/CONTABILITA/Ricevuta";
}
The problem is that the sieve script fails with the error:
sieve: info: started log at Jun 27 12:42:28.
error: msgid=<c9154db4-ff68-52a3-2639-11f09f225...@fcr.re.it>:
failed to store into mailbox
'Cond-UFF_FORNITORI/CONTABILITA/Ricevuta': Mailbox doesn't exist:
Cond-UFF_FORNITORI/CONTABILITA/Ricevuta.
I double checked config, dovecot documentation, sieve documentation,
ACL, filesystem paths, filesystem permissions, etc. etc. I'm pretty
sure it's all right.
So why isn't sieve working?
Thanks if you bother to answer & Cheers
Paolo
Nobody has clues?
Your version is very old. Can you try upgrading first?
Regards,
Stephan.
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-693.5.2.el7.x86_64 x86_64 CentOS Linux release 7.4.1708
(Core) xfs
auth_gssapi_hostname = $ALL
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = gssapi plain
default_process_limit = 300
disable_plaintext_auth = no
hostname = mail.fcr.re.it
mail_debug = yes
mail_gid = vmail
mail_location =
sdbox:/home/vmail/mailboxes/%n/dbox:ALT=/mnt/dovecot-altstorage/%n
mail_max_userip_connections = 15
mail_plugins = acl listescape
mail_shared_explicit_inbox = yes
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
namespace {
location =
maildir:/home/vmail/mailboxes/Condivise/UFF_FORNITORI-maildir:INDEX=~/Cond-indexes/.UFF_FORNITORI:CONTROL=~/Cond-control/.UFF_FORNITORI
prefix = Cond-UFF_FORNITORI/
separator = /
subscriptions = no
type = public
}
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
separator = /
type = private
}
passdb {
args = /etc/dovecot/FCR-dovecot-ldap.conf.ext
driver = ldap
}
plugin {
acl = vfile
acl_shared_dict = file:/home/vmail/mailboxes/shared-mailboxes.db
sieve = ~/.dovecot.sieve
sieve_before = /etc/dovecot/sieve-pre
sieve_max_redirects = 8
}
postmaster_address = postmas...@fcr.re.it
service auth {
client_limit = 1300
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
service imap-login {
process_min_avail = 10
}
service imap {
vsz_limit = 256 M
}
service lmtp {
executable = lmtp -L
process_min_avail = 5
}
ssl = required
ssl_cert =
Re: Sieve can't move messages in a public namespace
Il 30/06/2017 10:46, Paolo ha scritto:
Hello,
my dovecot version is 2.1.17
I've configured some public namespaces (config attached). When a mail
arrives at a certain mail address, that mail is handled by dovecot
lmtp server and a sieve script is executed that shuold move the
message in one of the namespaces. This is the script:
require "fileinto";
if header :contains "From" "exam...@example.com" {
fileinto "Cond-UFF_FORNITORI/CONTABILITA/Inviata";
}
else {
fileinto "Cond-UFF_FORNITORI/CONTABILITA/Ricevuta";
}
The problem is that the sieve script fails with the error:
sieve: info: started log at Jun 27 12:42:28.
error: msgid=<c9154db4-ff68-52a3-2639-11f09f225...@fcr.re.it>:
failed to store into mailbox
'Cond-UFF_FORNITORI/CONTABILITA/Ricevuta': Mailbox doesn't exist:
Cond-UFF_FORNITORI/CONTABILITA/Ricevuta.
I double checked config, dovecot documentation, sieve documentation,
ACL, filesystem paths, filesystem permissions, etc. etc. I'm pretty
sure it's all right.
So why isn't sieve working?
Thanks if you bother to answer & Cheers
Paolo
Nobody has clues?
Re: Sieve can't move messages in a public namespace
Il 30/06/2017 17:44, Alex JOST ha scritto: I think the :create command requires 'mailbox'. require ["fileinto","mailbox"]; Ops! Sorry. The :create clause was added after googling, but it didn't work. The problem is the same without ":create". The two mailboxes "Ricevuta" and "Inviata" does exist. I can see them and read mail in them from my Thunderbird, but sieve does not view them. Did someone face similar problem? Thanks Paolo
Sieve can't move messages in a public namespace
Hello,
my dovecot version is 2.1.17
I've configured some public namespaces (config attached). When a mail
arrives at a certain mail address, that mail is handled by dovecot lmtp
server and a sieve script is executed that shuold move the message in
one of the namespaces. This is the script:
require "fileinto";
if header :contains "From" "exam...@example.com" {
fileinto :create "Cond-UFF_FORNITORI/CONTABILITA/Inviata";
}
else {
fileinto :create "Cond-UFF_FORNITORI/CONTABILITA/Ricevuta";
}
The problem is that the sieve script fails with the error:
sieve: info: started log at Jun 27 12:42:28.
error: msgid=<c9154db4-ff68-52a3-2639-11f09f225...@fcr.re.it>: failed to
store into mailbox 'Cond-UFF_FORNITORI/CONTABILITA/Ricevuta': Mailbox doesn't exist:
Cond-UFF_FORNITORI/CONTABILITA/Ricevuta.
I double checked config, dovecot documentation, sieve documentation,
ACL, filesystem paths, filesystem permissions, etc. etc. I'm pretty sure
it's all right.
So why isn't sieve working?
Thanks if you bother to answer & Cheers
Paolo
# 2.1.17: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-419.el5 x86_64 CentOS release 5.11 (Final)
auth_gssapi_hostname = $ALL
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = gssapi plain
default_process_limit = 300
disable_plaintext_auth = no
hostname = my.mail.server
listen = *, [::]
login_greeting = IMAP/POP3 Server ready.
mail_gid = vmail
mail_location = maildir:~/Maildir
mail_max_userip_connections = 15
mail_plugins = acl listescape
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
namespace {
location =
maildir:/home/vmail/mailboxes/Condivise/DATIPHARM:INDEX=~/Cond-indexes/.DATIPHARM:CONTROL=~/Cond-control/.DATIPHARM
prefix = Cond-DATIPHARM/
separator = /
subscriptions = no
type = public
}
namespace {
location =
maildir:/home/vmail/mailboxes/Condivise/ADF:INDEX=~/Cond-indexes/.ADF:CONTROL=~/Cond-control/.ADF
prefix = Cond-ADF/
separator = /
subscriptions = no
type = public
}
namespace {
location =
maildir:/home/vmail/mailboxes/Condivise/CFO_FARMADATI:INDEX=~/Cond-indexes/.CFO:CONTROL=~/Cond-control/.CFO
prefix = Cond-CFO_FARMADATI/
separator = /
subscriptions = no
type = public
}
namespace {
location =
maildir:/home/vmail/mailboxes/Condivise/UFF_FORNITORI:INDEX=~/Cond-indexes/.UFF_FORNITORI:CONTROL=~/Cond-control/.UFF_FORNITORI
prefix = Cond-UFF_FORNITORI/
separator = /
subscriptions = no
type = public
}
namespace {
list = children
location = maildir:/home/vmail/mailboxes/%%n:INDEX=~/Cond-indexes/UTENTI/%%n
prefix = Cond-UTENTI/%%n/
separator = /
subscriptions = no
type = shared
}
namespace inbox {
inbox = yes
location =
prefix =
separator = /
subscriptions = yes
type = private
}
passdb {
args = /etc/dovecot/conf.d/auth-ldap-passdb.conf.ext.FCR
driver = ldap
}
plugin {
acl = vfile
acl_shared_dict = file:/home/vmail/mailboxes/shared-mailboxes.db
sieve = ~/.dovecot.sieve
sieve_before =
}
postmaster_address = postmas...@fcr.re.it
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
user = root
}
service imap-login {
process_min_avail = 10
}
service imap {
executable = imap
vsz_limit = 256 M
}
service lmtp {
executable = lmtp -L
process_min_avail = 5
}
service pop3-login {
process_min_avail = 10
}
service pop3 {
executable = pop3
}
ssl_cert =
Re: ot: migrating TB user's email to new laptop
On Wed, 25 May 2016, voy...@sbt.net.au wrote: On Wed, May 25, 2016 4:31 pm, Philip McGaw wrote: Did user historically have POP set up? Philip, thanks no, not for a long time, IMAP/143/StartTLS on old laptop If you still have access to the old laptop set up imap and move the emails back. yes, I have old laptop here. sorry, not sure how to, is that inside TBird, or how ? (not very familiar with TBird...) Start TB on the new laptop Close it You have created the default folder/settings in the local %appdata% folder copy the %appdata%\Thunderbird folder from the old laptop to the new Finished ;-) The new laptop is now exacly as the old (better if TB same version before migration). -- Regards, Paolo
Dovecot 2.1.7 randomly lost emails
Hi
Dovecot 2.1.7 Postfix 2.9.6 Maildir backend, emails filtered by a Postfix
frontend (Sqlgrey, Amavis).
Lmtp delivering emails using a simple sieve script.
Ubuntu 12.04 VMware virtual server.
When email are CCed to other recipients on the same domain (the same backend),
sometimes they are lost.
We hosts approx. 1500 mailboxes, the problem occurs one or two times a day.
Recently I changed the frontend, the previous one was delivering one email for
each recipient, we had a couple of error per year, we thought it was due to
client-side error.
The frequency increased with the new frontend, no more “splitting” messages for
more than one recipient.
The following sample is from a client using a POP3 connection, leaving emails
on server.
the Postfix mail.log
May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628:
from=<sen...@external.com>, size=11822371, nrcpt=2 (queue active)
May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628:
to=<user-...@domain.com>, relay=mail4.robinson.it[private/dovecot-lmtp],
delay=0.64, delays=0.45/0/0/0.19, dsn=2.0.0, status=sent (250 2.0.0
<user-...@domain.com> 0vNzHyptMlezHgAAk+lh+g Saved)
May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628:
to=user-...@domain.com>, relay=mail4.robinson.it[private/dovecot-lmtp],
delay=0.74, delays=0.45/0/0/0.29, dsn=2.0.0, status=sent (250 2.0.0
user-...@domain.com> 0vNzHyptMlezHgAAk+lh+g Saved)
May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: removed
the dovecot-info.log
May 11 01:26:55 lmtp(7859, user-...@domain.com): Info: 0vNzHyptMlezHgAAk+lh+g:
sieve:
msgid=<003a2d931fff0e48a2314a1adc87e23e8e4ba...@t49-ms-exmb02.intranet.saksroot.saksinc.com>:
stored mail into mailbox 'INBOX'
May 11 01:26:55 lmtp(7859, user-...@domain.com): Info: 0vNzHyptMlezHgAAk+lh+g:
sieve:
msgid=<003a2d931fff0e48a2314a1adc87e23e8e4ba...@t49-ms-exmb02.intranet.saksroot.saksinc.com>:
stored mail into mailbox 'INBOX'
I cannot find the email 0vNzHyptMlezHgAAk+lh+g for
user-...@domain.com<mailto:user-...@domain.com>, nor I cannot suppose problems
on the client, nothing changed on the back-end.
Please let me know if you need some more info or config dump.
Any help will be appreciated
Paolo Rinaldo
Dovecot config:
# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.11.0-15-generic x86_64 Ubuntu 12.04.4 LTS
auth_mechanisms = plain login
debug_log_path = /var/log/dovecot/dovecot-debug.log
disable_plaintext_auth = no
info_log_path = /var/log/dovecot/dovecot-info.log
log_path = /var/log/dovecot/dovecot.log
mail_plugins = " quota"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
namespace inbox {
inbox = yes
location =
prefix = INBOX.
separator = .
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
autocreate = INBOX.Trash
autocreate2 = INBOX.Spam
autocreate3 = INBOX.Sent
autocreate4 = INBOX.Drafts
autosubscribe = INBOX.Trash
autosubscribe2 = INBOX.Spam
autosubscribe3 = INBOX.Sent
autosubscribe4 = INBOX.Drafts
quota = maildir:User quota
quota_exceeded_message = Impossibile consegnare il messaggio: spazio
insufficiente sulla casella di destinazione
quota_grace = 10%%
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
sieve_global_path = /var/lib/dovecot/sieve/spam.sieve
}
protocols = imap pop3 lmtp sieve
service auth-worker {
user = vmail
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
user = dovecot
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = vmail
}
ssl_cert =
Re: object storage
> Timo Sirainen wrote: > Our typical projects have several million user accounts. I think 100k > users is around the minimum. Interesting. So che choice for object storage is driven by the number of accounts rather than the amount of data stored? For example I am heading towards 10 TB online with "just" 10k users. It wouldn't be worth? Paolo
Re: imap_logout_format improvements / ideas
# %i - total number of bytes read from client
# %o - total number of bytes sent to client
# %{fetch_hdr_count} - Number of mails with mail header data sent to
client
# %{fetch_hdr_bytes} - Number of bytes with mail header data sent to
client
# %{fetch_body_count} - Number of mails with mail body data sent to client
# %{fetch_body_bytes} - Number of bytes with mail body data sent to client
# %{deleted} - Number of mails where client added \Deleted flag
# %{expunged} - Number of mails that client expunged
# %{trashed} - Number of mails that client copied/moved to the
# special_use=\Trash mailbox.
Any thoughts on what else would be useful?
Number of mails that have been \Seen ?
I have been running Dovecot with full debug logging so I am not aware if the
same information is already available elsewhere in the log at production
loglevel.
Paolo
Different mdbox_rotate_size for primary and alternate storage
Hello. In order to speed up backups of very very old messages I would like to set two different limits for mdbox_rotate_size. Like, 50M for primary storage and 100M or larger for alternate storage. There is no mention in docs or such a possibility, so I assume it is not possible. Is that correct? While I am at it, is it possible to configure primary storage as maildir (sturdy indexes) and altstorage as mdbox (more delicate indexes)? Thanks, Paolo
Re: acl and lazy_expunge plugins are incompatible?
Hello.
Before going into deep testing of 2.2.16 (clean install tbd), I would like
to know if someone is using succesfully both ACL and Lazy_expunge plugins.
Perhaps I had a config error?
Thanks,
Paolo
Il 3 marzo 2015 alle 16.36 Paolo Cravero paolo.crav...@csi.it ha
scritto:
Hello list.
Following two previous unanswered requests
(http://www.dovecot.org/list/dovecot/2014-August/097449.html and
http://www.dovecot.org/list/dovecot/2014-May/096261.html), I fell into the
same problem: ACL and lazy_expunge plugins do not work together.
I had lazy_expunge enabled working OK. Then I activated ACL plugin, and
gave
dovecot.2 visibily over paolo.cravero's Inbox: the IMAP connection fails
on
server-side. Relevant debug log is:
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: Quota
grace: root=User quota bytes=576716800 (10%)
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: dict
quota: user=paolo.crav...@csi.it,
uri=file:/maildata1/home/csi.it/p/paolo.cravero/dovecot-quota,
noenforcing=0
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: fs:
root=/maildata1/main/csi.it/p/paolo.cravero,
index=/maildata1/indexes/csi.it/p/paolo.cravero, indexpvt=, control=,
inbox=, alt=/maildata1/altstorage/csi.it/p/paolo.cravero
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl:
initializing backend with data: vfile
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl: acl
username = paolo.crav...@csi.it
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl:
owner
= 1
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl
vfile:
Global ACLs disabled
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Fatal:
lazy_expunge: Unknown namespace: '.EXPUNGED/'
Removing the lazy_expunge, ACLs do work as expected and I can
add/remove/use
them.
Is there a known compatibility issue? Or there's something wrong in my
config and someone else is running with both plugins enabled?
Thanks,
Paolo
PS: output of dovecot -n:
# 2.2.15: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.el6.x86_64 x86_64 Red Hat Enterprise Linux Server
release 6.5 (Santiago) ext3
auth_mechanisms = plain login
disable_plaintext_auth = no
imap_client_workarounds = tb-extra-mailbox-sep
lmtp_save_to_detail_mailbox = yes
mail_attachment_dir = /maildata1/attachments
mail_attachment_hash = %{sha256}
mail_debug = yes
mail_gid = 500
mail_location =
mdbox:/maildata1/main/%d/%1n/%n:ALT=/maildata1/altstorage/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n
mail_plugins = quota mail_log notify lazy_expunge acl
mail_uid = 500
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
duplicate
mbox_write_locks = fcntl
mdbox_rotate_size = 50 M
namespace {
hidden = no
list = yes
location =
mdbox:/maildata1/main/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n:MAILBOXDIR=expunged
prefix = .EXPUNGED/
subscriptions = no
}
namespace {
list = children
location =
mdbox:/maildata1/main/%%d/%%1n/%%n:ALT=/maildata1/altstorage/%%d/%%1n/%%n:INDEX=/maildata1/indexes/%%d/%%1n/%%n
prefix = user/%%u/
separator = /
type = shared
}
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
acl = vfile
acl_shared_dict = file:/maildata1/db/shared-mailboxes.db
lazy_expunge = .EXPUNGED/
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename flag_change append
mail_log_fields = uid box msgid size flags vsize
quota = dict:User quota::file:%h/dovecot-quota
quota_grace = 10%%
quota_rule = *:storage=10G
quota_rule2 = Trash:storage=+100M
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
}
protocols = imap pop3 lmtp sieve
service auth {
unix_listener auth-userdb {
group = vmail
user = vmail
}
}
service lmtp {
inet_listener lmtp {
address = 10.102.42.114
port = 24
}
process_min_avail = 2
user = vmail
}
ssl_cert = /etc/pki/dovecot/certs/dovecot.pem
ssl_key = /etc/pki/dovecot/private/dovecot.pem
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = quota mail_log notify lazy_expunge acl sieve
}
protocol lda {
mail_plugins = quota mail_log notify lazy_expunge acl sieve
}
protocol imap {
mail_plugins = quota mail_log notify lazy_expunge
Re: Overriding dovecot.conf from Userdb Extras
Il 9 marzo 2015 alle 4.17 E.B. emailbuilde...@yahoo.com ha scritto: I thought I read that anything from dovecot.conf can be overridden in a userdb lookup. Or a passdb lookup with userdb_ prefix. But I tried for fun change log_path but it never worked. Is that because logging is special, already started logging before it comes to the passdb/userdb lookups? So are there some dovecot.conf settings that cannot be overridden? To my understanding only these extra parameters can be tweaked through the userdb/passdb: http://wiki2.dovecot.org/PasswordDatabase/ExtraFields http://wiki2.dovecot.org/UserDatabase/ExtraFields (+ mail and quota_rule) Paolo
Re: Rebuilding SIS attachment links from log
Daniel, please help me understand, since I met your same problem on 2015-02-26 as you did. 2. For each failed: read(/var/mail/attachments/aa/bb/attachmentHash-userHash line, a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists i. If attachmentHash is missing display such for possible Is this a FEATURE of Dovecot SIS? Or a known bug described somewhere? Loosing detached attachments, I mean. Can this be done via pure BASH? Need sed/awk as well? Well, you need a way to strip out the -userHash part from the string, so a little bit of sed and regex is needed. You also need to work out source and destination files. Still, personally I think the problem is somewhere else and should be fixed. Else, stop using SIS and let the SAN do the deduplication. Am I wrong? Paolo
acl and lazy_expunge plugins are incompatible?
Hello list.
Following two previous unanswered requests
(http://www.dovecot.org/list/dovecot/2014-August/097449.html and
http://www.dovecot.org/list/dovecot/2014-May/096261.html), I fell into the
same problem: ACL and lazy_expunge plugins do not work together.
I had lazy_expunge enabled working OK. Then I activated ACL plugin, and gave
dovecot.2 visibily over paolo.cravero's Inbox: the IMAP connection fails on
server-side. Relevant debug log is:
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: Quota
grace: root=User quota bytes=576716800 (10%)
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: dict
quota: user=paolo.crav...@csi.it,
uri=file:/maildata1/home/csi.it/p/paolo.cravero/dovecot-quota, noenforcing=0
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: fs:
root=/maildata1/main/csi.it/p/paolo.cravero,
index=/maildata1/indexes/csi.it/p/paolo.cravero, indexpvt=, control=,
inbox=, alt=/maildata1/altstorage/csi.it/p/paolo.cravero
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl:
initializing backend with data: vfile
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl: acl
username = paolo.crav...@csi.it
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl: owner
= 1
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Debug: acl vfile:
Global ACLs disabled
Mar 3 15:59:02 tst-msg03 dovecot: imap(doveco...@csi.it): Fatal:
lazy_expunge: Unknown namespace: '.EXPUNGED/'
Removing the lazy_expunge, ACLs do work as expected and I can add/remove/use
them.
Is there a known compatibility issue? Or there's something wrong in my
config and someone else is running with both plugins enabled?
Thanks,
Paolo
PS: output of dovecot -n:
# 2.2.15: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.el6.x86_64 x86_64 Red Hat Enterprise Linux Server
release 6.5 (Santiago) ext3
auth_mechanisms = plain login
disable_plaintext_auth = no
imap_client_workarounds = tb-extra-mailbox-sep
lmtp_save_to_detail_mailbox = yes
mail_attachment_dir = /maildata1/attachments
mail_attachment_hash = %{sha256}
mail_debug = yes
mail_gid = 500
mail_location =
mdbox:/maildata1/main/%d/%1n/%n:ALT=/maildata1/altstorage/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n
mail_plugins = quota mail_log notify lazy_expunge acl
mail_uid = 500
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave duplicate
mbox_write_locks = fcntl
mdbox_rotate_size = 50 M
namespace {
hidden = no
list = yes
location =
mdbox:/maildata1/main/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n:MAILBOXDIR=expunged
prefix = .EXPUNGED/
subscriptions = no
}
namespace {
list = children
location =
mdbox:/maildata1/main/%%d/%%1n/%%n:ALT=/maildata1/altstorage/%%d/%%1n/%%n:INDEX=/maildata1/indexes/%%d/%%1n/%%n
prefix = user/%%u/
separator = /
type = shared
}
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
acl = vfile
acl_shared_dict = file:/maildata1/db/shared-mailboxes.db
lazy_expunge = .EXPUNGED/
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename flag_change append
mail_log_fields = uid box msgid size flags vsize
quota = dict:User quota::file:%h/dovecot-quota
quota_grace = 10%%
quota_rule = *:storage=10G
quota_rule2 = Trash:storage=+100M
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
}
protocols = imap pop3 lmtp sieve
service auth {
unix_listener auth-userdb {
group = vmail
user = vmail
}
}
service lmtp {
inet_listener lmtp {
address = 10.102.42.114
port = 24
}
process_min_avail = 2
user = vmail
}
ssl_cert = /etc/pki/dovecot/certs/dovecot.pem
ssl_key = /etc/pki/dovecot/private/dovecot.pem
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = quota mail_log notify lazy_expunge acl sieve
}
protocol lda {
mail_plugins = quota mail_log notify lazy_expunge acl sieve
}
protocol imap {
mail_plugins = quota mail_log notify lazy_expunge acl imap_quota imap_acl
}
Re: mdbox attachment errors
Il 27 febbraio 2015 alle 18.18 Hardy Flor: copy file f6f4f3b882bf3488af632389d4aaba8adc332b12 from backup to /var/mail/attachments/f6/f4/hashes/f6f4f3b882bf3488af632389d4aaba8adc332b12 and create hardlink to /var/mail/attachments/f6/f4/f6f4f3b882bf3488af632389d4aaba8adc332b12-60ab750a1aa4b554da169db5accb Sure. That fix restores access to the mailbox/message/attachment. What if the file disappeared before the backup process copied it? What if 1% of 20'000 users open a ticket a day for the same reason? But ... why did it disappear at all? I got the same error on a test environment, where I am learning Dovecot (see my message sent 26/2/2015 at 14:48 CET). Nobody else has access to my server. I have not manually deleted files in the SIS partition. With a little script I ran through my mdbox m.* files, I extracted attachment hash-filenames and searched for them on the disk: 10 were missing. In some cases there is no hashes directory at all. Not even the 2nd level base path ( /attachments/a/b/hash stops at /attachments/a ). WHY??? Note that depending on how the IMAP client accesses the mailbox folder, you may be unable to see the whole folder at all! That's the case with Open-Xchange. Again: which tests can be run in order to confirm the behaviour and possibly a bug? Paolo Cravero
Re: Dovecot LDAP Take #2: Authentication failed and logging
This is the user DN: cn=Klara Fall,ou=People,dc=[domainname],dc=de According to your Dovecot configuration auth_bind_userdn = cn=%u,ou=People,dc=**[domainname]**,dc=de if you login with klarafall it will be expanded into cn=klarafall,ou=People,dc=[domainname],dc=de which is not the correct DN for Mrs Klara. So if you login with Klara Fall it should work, but that will probably mess up the things on Dovecot filesystem. I am strongly against setting a static DN when dealing with LDAP authentication. LDAP servers are optimized to serve search requests, so let yours do the job. Allow Dovecot to lookup the correct DN based on the attribute you supply (uid) and then authenticate. This should be achieved if you comment out the auth_bind_userdn line. Paolo Cravero
Re: users of dbox format
Andreas, I am interested in finding out your experiences with using the dbox format (especially mdbox) if you use this format. mdbox is THE reason why I am trying Dovecot. With mailboxes of several (tens of) GB with several k of messages I hope mdbox will speedup backups. Also SIS for attachments sounds very good, but still doesn't follow the altstorage rules (while messages go to altstorage, the extracted attachment stays in the SIS partition). I am contemplating changing my maildir setup to mdbox but I still need to make a case for it My current interest is to keep live mailbox data in Maildir format and archive in the altstorage in mdbox after 30 days or so. I will experiment that configuration next week. Nevertheless I am still puzzled by the lost data I got yesterday in my mdbox+SIS+altstorage combination (see my message and the one from Daniel Miller). I am also interested to hear real life stories. 73 Paolo Cravero
mdbox + sis + altstorage: error in attachments-connector (file not found)
Hello, first time here.
I am playing with Dovecot because of its mdbox storage, SIS feature and
altstorage ability. I authenticate against LDAP (passdb), but I don't fetch
operational data from LDAP (mail_locations are static from config files and
user_attrs = =uid=500, =gid=500, =home=/maildata1/home/%d/%1n/%n).
On a daily basis I run doveadm altmove -A savedbefore 5d
Thunderbird apparently works 100%, but only apparently. When accessing one
of those mailboxes through Open-Xchange IMAP I got an error for the first
time. Then I could reproduce it with Thunderbird as well, I simply had to
open the message mentioned in the log:
Feb 26 14:06:02 tst-msg03 dovecot: imap(paolo.crav...@csi.it): Error:
read(attachments-connector(/maildata1/altstorage/csi.it/p/paolo.cravero/storage/m.35))
failed:
read(/maildata1/attachments/40/68/40683f07fcbf1f95f09b273e472b3c7067f43210641baf6e3cce333a27b36e11-4a235a29fd05db54fd681117ea21[base64:19
b/l]) failed:
open(/maildata1/attachments/40/68/40683f07fcbf1f95f09b273e472b3c7067f43210641baf6e3cce333a27b36e11-4a235a29fd05db54fd681117ea21)
failed: No such file or directory (uid=5940, box=INBOX)
Feb 26 14:06:02 tst-msg03 dovecot: imap(paolo.crav...@csi.it): Error:
read(attachments-connector(/maildata1/altstorage/csi.it/p/paolo.cravero/storage/m.35))
failed:
read(/maildata1/attachments/40/68/40683f07fcbf1f95f09b273e472b3c7067f43210641baf6e3cce333a27b36e11-4a235a29fd05db54fd681117ea21[base64:19
b/l]) failed:
open(/maildata1/attachments/40/68/40683f07fcbf1f95f09b273e472b3c7067f43210641baf6e3cce333a27b36e11-4a235a29fd05db54fd681117ea21)
failed: No such file or directory (uid=5940, box=INBOX)
Feb 26 14:06:02 tst-msg03 dovecot: imap(paolo.crav...@csi.it): Disconnected:
Internal error occurred. Refer to server log for more information.
[2015-02-26 14:06:02] in=182 out=1266
This happens on many messages sitting in the altstorage that had an
attachment. The detached attachment file does not exist. Sometimes not even
the path is there (anymore?).
At some point in time I have run doveadm purge on the mailbox in error,
but I am not sure that is the culprit.
Have I hit a bug or done something wrong? Please suggest tests to verify
where I stand.
Config dump follows.
Paolo
# 2.2.15: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.el6.x86_64 x86_64 Red Hat Enterprise Linux Server
release 6.5 (Santiago) ext3
auth_mechanisms = plain login
disable_plaintext_auth = no
imap_client_workarounds = tb-extra-mailbox-sep
lmtp_save_to_detail_mailbox = yes
mail_attachment_dir = /maildata1/attachments
mail_attachment_hash = %{sha256}
mail_debug = yes
mail_gid = 500
mail_location =
mdbox:/maildata1/main/%d/%1n/%n:ALT=/maildata1/altstorage/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n
mail_plugins = quota mail_log notify lazy_expunge
mail_uid = 500
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave duplicate
mbox_write_locks = fcntl
mdbox_rotate_size = 50 M
namespace {
hidden = no
list = yes
location =
mdbox:/maildata1/main/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n:MAILBOXDIR=expunged
prefix = .EXPUNGED/
subscriptions = no
}
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
lazy_expunge = .EXPUNGED/
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename flag_change append
mail_log_fields = uid box msgid size flags vsize
quota = dict:User quota::file:%h/dovecot-quota
quota_grace = 10%%
quota_rule = *:storage=10G
quota_rule2 = Trash:storage=+100M
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
}
protocols = imap pop3 lmtp sieve
service auth {
unix_listener auth-userdb {
group = vmail
user = vmail
}
}
service lmtp {
inet_listener lmtp {
address = 10.102.42.114
port = 24
}
process_min_avail = 2
user = vmail
}
ssl_cert = /etc/pki/dovecot/certs/dovecot.pem
ssl_key = /etc/pki/dovecot/private/dovecot.pem
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = quota mail_log notify lazy_expunge sieve
}
protocol lda {
mail_plugins = quota mail_log notify lazy_expunge sieve
}
protocol imap {
mail_plugins = quota mail_log notify lazy_expunge imap_quota
}
Re: Subject tag [Dovecot] is gone
In this list we have Timo and many other people very skilled in dovecot and mail related stuff. I read the considerations and I suppose they are right, but ... Also there are people like me that are lower profile sysadmins. Filtering mail isn't a problem, but, in my opinion, having the tag [Dovecot] in the subject is the better solution for visual filtering. I receive 2-300 mail / day in the inbox. Often I don't read a dovecot or postfix thread if the subject doesn't interest me, but sometime the tag [Dovecot], increase the appeal of others keyword ... Pheraps, mail filtered in folders are rarely read in real time. Usually I look at it in my spare time (very reduced), or when I search for a specific argoment. A couple of friends agree with me, so I am not the only ... ;-) This is only our opinion as low profile sysadmin, Anyway, thanks to Timo and others for the great product and the support. Regards, Paolo
Re: [Dovecot] mail check
On Mon, 20 Jan 2014, Robert Schetterer wrote: Am 19.01.2014 22:36, schrieb Paolo: Is there some simple script that parsing the dovecot.info.log can give me a list of the mail checks for a user (better if reports also the type (POP/IMAP), and result of the sessions (Ok/Fail/Error/...))? Regards, Paolo Logwatch may help you http://sourceforge.net/projects/logwatch/ I am using CentOS 6.x with default dovecot-2.0.16 and logwatch-7.3.6 In default config it don't give any useful info. Tried to update the docevot script with other from the net and modifing the logs reference, but also using a specified call to it like: logwatch --service dovecot --debug 10 --logfile dovecot I obtain nothing or a complete list of sessions disabling OnlyService = (imap-login|pop3-login|dovecot) in the dovecot.conf file Probably there is some bug or I am missing something. Nothing interesting n the archive list. short way may using something like i.e. grep -i user dovecot.info.log etc That is what I am using now. Hoped in someone that had write something that summarize this output better :-) Regards, Paolo
[Dovecot] mail check
Is there some simple script that parsing the dovecot.info.log can give me a list of the mail checks for a user (better if reports also the type (POP/IMAP), and result of the sessions (Ok/Fail/Error/...))? Regards, Paolo
[Dovecot] passwd-file and MySQL
Speaking about multiple DB ... A my customer has a server (CentOS 6.x, dovecot 2.0.x, postfix, Squirrel+RoundCube with vacation and password plug-in), with about 3.000 users (about 200 different domains), all defined as system users and home as /home/mail/%user% The system works fine. Its only additional interest is authenticating users like u...@domain.tld / password (maintaining also the current user.domain / password). Is it possible and How complex configuring a similar environment? Only matter of sync /etc/shadow and DB or other problems? Regards, Paolo
Re: [Dovecot] /etc/passwd Centos + dovecot
On Tue, 25 Jun 2013, Dejan Doder wrote: Hi group , I use system users with passwords defined in /etc/passwd. How can users change their passwords ? I don't think this is dovecot related, but you can use squirrelmail + plug-in or roundcube + plugin or directly with a web interface to poppassd or usermin, or ... -- Regards, Paolo
[Dovecot] Moving index files on another disk: no troubles but need more assurances
Hi, I'm using dovecot 1.2.15 on debian squeeze with maildir, and I want to move index files out of the mailboxes disk to reduce disk I/O. I need to get the assurance that the switch is completely transparent to my clients (IMAP and POP). I read the wiki documentation and have found this thread (http://dovecot.org/pipermail/dovecot/2009-September/042665.html), which makes me confident, even if it speaks of a older version and only pop3. So I set up a test server, copied a true mailbox and changed mail location from: mail_location = maildir:~/Maildir to mail_location = maildir:~/Maildir:INDEX=/srv/indexes/%1n/%n (all users under the same domain, no need to use %d). On the IMAP side all seems to work fine, new indexes are created in the new location (dovecot.index.log on 1st access, dovecot.index.cache on message manipolation). A test client (evolution) doesn't notice at all, and I have verified via a telnet imap session that UIDs don't change. But I see that when I move the indexes the IMAP HIGHESTMODSEQ response changes and is reset to 1. Can this baffle the clients? On the rfc I don't found reference to this status walking backwards. Need I to verify that all messages have the virtual size (W=) in the file name (the mailboxes are from a previous cyrus installation, then we switch to dovecot 1.2)? What other test you suggest that checking UIDL on the pop3 side? What happens if the indexes disk disappears (broken or removed from bay or all paths down for FC/iscsi)? Can dovecot 1.2 continue with INDEX=MEMORY as for a disk full? Sorry for the many questions Paolo -- SEMEL (SErvizio di Messaging ELettronico) - AINF, Universita' di Udine
Re: [Dovecot] *notify config for 1.0.x doesn't enable inotify
On Sat, Nov 03, 2007 at 11:23:57PM +0200, Timo Sirainen wrote: instead are defined in the single linux/inotify.h. If you do like the wiki says, you should have only a single inotify.h file containing also inotify-syscall.h's contents. ahh - now I see it - too small a screen, didn't notice the ' inotify.h' ok, sorry for the noise :] Anyway, I had already recompiled changing back to (distro's kernel-headers pkg) linux/inotify.h as I see little or no point in the sys/notify.h change. As long as one's not using latest, newest glibc which (iirc) has inotify.h on it's own, that is. Not my case till, I guess, Debian Etch moves to oldstable. thanks -- paolo GPG/PGP id:0x1D5A11A4 - 04FC 8EB9 51A1 5158 1425 BC12 EA57 3382 1D5A 11A4
[Dovecot] mail-sql.tar.gz: missing mailbox-list-private.h
hi, I'm trying to compile with plugin in subject, but I get this error: In file included from mail-sql-plugin.c:4: sql-storage.h:5:34: mailbox-list-private.h: No such file or directory In file included from mail-sql-plugin.c:4: sql-storage.h:26: error: field `module_ctx' has incomplete type make[5]: *** [mail-sql-plugin.lo] Error 1 there's no such mailbox-list-private.h in the whole src tree, though seems it should be in the sql-plugin tarball. Is it in 1.1? thanks -- paolo GPG/PGP id:0x1D5A11A4 - 04FC 8EB9 51A1 5158 1425 BC12 EA57 3382 1D5A 11A4
Re: [Dovecot] caching custom headers (via plugins)
On Fri, Nov 02, 2007 at 03:48:41PM +0200, Timo Sirainen wrote: Actually it caches everything that is used. http://wiki.dovecot.org/IndexFiles and http://wiki.dovecot.org/Design/Indexes/Cache tell about the exact rules. I've read those docs, indeed, and I've run a test against 1.0.5. I see ~0.25s response time for a SEARCH on eg msg-id but ~15-35s on X-something. After that 1st run, SEARCH on same X-* takes ~same (~10kmsg mailbox). So I guessed index cache is done on the fly for any SEARCHd for item, but in memory only, since checking index files after test didn't show such X-something. But re-checking now, I realize I must have checked index files too early: my custom X-something are right there into index.cache. So I think it might already be smart enough for our purpose :) thanks -- paolo GPG/PGP id:0x1D5A11A4 - 04FC 8EB9 51A1 5158 1425 BC12 EA57 3382 1D5A 11A4
Re: [Dovecot] mail-sql.tar.gz: missing mailbox-list-private.h
On Fri, Nov 02, 2007 at 02:31:41PM +0200, Timo Sirainen wrote: On Fri, 2007-11-02 at 09:56 +0100, Paolo wrote: I'm trying to compile with plugin in subject, but I get this error: ... Is it in 1.1? Yes. mail-sql was created against many months old v1.1 code tree. I've no idea if it still compiles with the current v1.1, but in any case it won't work with v1.0. ok, thanks. -- paolo GPG/PGP id:0x1D5A11A4 - 04FC 8EB9 51A1 5158 1425 BC12 EA57 3382 1D5A 11A4
[Dovecot] *notify config for 1.0.x doesn't enable inotify
hi, the current code has changed include linux/inotify.h to sys/inotify.h since 2006-01-17, but that won't work with inotify*.h as documented in the wiki. One has to either revert back to linux/inotify.h, or add #include sys/inotify-syscalls.h to both configure.in and src/lib/ioloop-notify-inotify.c Or, just symlink sys/inotify.h - linux/inotify.h. That's on Debian 3.1 (Sarge). -- paolo GPG/PGP id:0x1D5A11A4 - 04FC 8EB9 51A1 5158 1425 BC12 EA57 3382 1D5A 11A4
[Dovecot] caching custom headers (via plugins)
hi, currently dovecot caches on disk some most common headers. Is it possible to tweak such headers list via plugins, eg to cache custom X-* headers, or do I need to hack the actual index/cache src? if so, which are the .c/.h files to touch? thanks -- paolo GPG/PGP id:0x1D5A11A4 - 04FC 8EB9 51A1 5158 1425 BC12 EA57 3382 1D5A 11A4
Re: [Dovecot] *notify config for 1.0.x doesn't enable inotify
On Fri, Nov 02, 2007 at 06:07:35PM +0200, Timo Sirainen wrote: On Fri, 2007-11-02 at 14:56 +0100, Paolo wrote: the current code has changed include linux/inotify.h to sys/inotify.h since 2006-01-17, but that won't work with inotify*.h as documented in the wiki. The wiki entry seems to create /usr/local/include/sys/inotify.h and it should be possible to include that as sys/inotify.h. Maybe the real problem is that if you tried that, the /usr/local/include isn't in the standard include lookup paths. So you should do: CPPFLAGS='-I /usr/local/include' ./configure --with-notify=inotify indeed, I spent a while to convince the autoconf to look that up, but then I realized the problem isn't the path, the test program always fails, simply because the new defs have been splitted in the 2 files mentioned in the wiki, the problem being that inotify.h doesn't #include inotify-syscalls.h on its own, and compile fails for undefined functions, eg inotify_add_watch(), which instead are defined in the single linux/inotify.h. Perhaps such #include sys/inotify-syscalls.h should be in inotify.h, but I think those .h are older versions, since current in-kernel inotify.h is just that, all in one. -- paolo GPG/PGP id:0x1D5A11A4 - 04FC 8EB9 51A1 5158 1425 BC12 EA57 3382 1D5A 11A4
Re: [Dovecot] *notify config for 1.0.x doesn't enable inotify
On Fri, Nov 02, 2007 at 11:49:03PM +0100, Marcus Rueckert wrote: instead are defined in the single linux/inotify.h. Perhaps such #include sys/inotify-syscalls.h should be in inotify.h, but I think those .h are older versions, since current in-kernel inotify.h is just that, all in one. they old splitted headers were never meant to be used in the userspace. headers in /usr/include/linux/ are in general not meant for userspace apps. yep, that's what docs suggest. But we need inotify.h. Do you know any place where to take it from, other then kernel? I see little point in placing the same file under different dir, unless there are (distro's) pkgs that provide it - eg in Debian's 3.1 there's none, in 4.0 there's klibc which places it under /usr/lib/klibc/; but then klibc docs say you're better off using linux/inotify.h if avail. So, what's the point of changing linux/inotify.h to sys/inotify.h if: 1. at best, they're just the same 2. the 1st is a standard path, the 2nd isn't (no pkgs, need to tweak by hand). -- paolo
