Re: Core-dump on SSL connection

[Richard Hector via dovecot]

On 19/06/24 20:37, Daniel Lange via dovecot wrote:

Hi Scott,

Am 19.06.24 um 06:41 schrieb Aki Tuomi via dovecot:


On 19/06/2024 06:58 EEST Scott Q. via dovecot  
wrote:


I'm on Debian 12.5 which comes with openssl 3.0.11


You can use the backport available from your trusty Debian maintainers:
https://packages.debian.org/source/stable-backports/dovecot
(currently 2.3.21+dfsg1-3~bpo12+1)

It has been patched to work with OpenSSL 3:
https://sources.debian.org/patches/dovecot/1:2.3.21%2Bdfsg1-3~bpo12%2B1/Support-openssl-3.0.patch/
and has some other upstream bugs fixed as well.


Is that the patch that Aki said doesn't always work?

I'm also on Debian 12.5 (recently upgraded), with debian-packaged 
dovecot, but I haven't seen any issues - should I be worried?


If it is a problem, would putting a proxy like nginx in front be a 
useful solution?


Cheers,
Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Dovecot logging to files causes issues

[Richard Rosner via dovecot]

Am 19.05.24 um 16:02 schrieb Alexander Dallou via dovecot:

Am 19.05.2024 um 15:55 schrieb Richard Rosner via dovecot:

Am 19.05.24 um 15:29 schrieb Friedrich Kink via dovecot:

chmod 775 /var/log/dovecot will solve the problem. Without execute permission 
the process can't access the logfile.

Why on earth does a process supposed to write to a file need execution 
permission? This most certainly is very unwelcome behavior and a bug in any 
case, no matter if it's intended by the author or not.


chmod ug+x on the /var/log/dovecot directory! Standard POSIX permissions for a 
non-root process to enter a directory.

It most certainly isn't. nginx isn't running as root, yet it can log without 
execution permissions just fine. Absolutely nothing should have execution 
permissions if they aren't meant to be executed, which should only be true for 
a very small set of files besides binaries.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Dovecot logging to files causes issues

[Richard Rosner via dovecot]

Am 19.05.24 um 15:29 schrieb Friedrich Kink via dovecot:

chmod 775 /var/log/dovecot will solve the problem. Without execute permission 
the process can't access the logfile.

Why on earth does a process supposed to write to a file need execution 
permission? This most certainly is very unwelcome behavior and a bug in any 
case, no matter if it's intended by the author or not.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Dovecot logging to files causes issues

[Richard Rosner via dovecot]

Am 19.05.24 um 04:02 schrieb Peter via dovecot:

Check the permissions of the entire path, as dovecot:

namei -l /var/log/dovecot/error.log

It might be selinux, check your audit.log file, or set selinux to permissive 
mode and see if it works:

setenforce 0


This can't be the case, there is no SELinux present by default in Debian and it 
was never installed on that server. For completeness, here's the output:

namei -l /var/log/dovecot/error.log
f: /var/log/dovecot/error.log
drwxr-xr-x root    root    /
drwxr-xr-x root    root    var
drwxr-xr-x root    root    log
drw-rw-r-- dovecot dovecot dovecot
-rw-r--r-- dovecot dovecot error.log



It might also be apparmour (sorry don't have instructions for apparmour).

The message basically means that something is preventing the dovecot user from 
writing to the file, you need to figure out what that is.


Peter


I can say that this isn't possible, as any AppArmor actions would be logged, so 
they would have showed up. And by the files sizes, Dovecot is clearly writing 
to them.

-rw-r--r--  1 dovecot dovecot    0 13. Mai 20:50 debug.log
-rw-r--r--  1 dovecot dovecot  37K 14. Mai 14:05 error.log
-rw-r--r--  1 dovecot dovecot  40K 13. Mai 21:20 info.log

So there's pretty much no possibility AppArmor could have any involvement here. Also, 
usually when AppArmor prevents access to a directory, you'd get a "file not 
found" error, not a permission denied.

For the very unlikely case that AppArmor is the cause, these are the only rules 
present for dovecot:

Dovecot has two files. In tunables you can find this:

   # @{DOVECOT_MAILSTORE} is a space-separated list of all directories
   # where dovecot is allowed to store and read mails
   #
   # The default value is quite broad to avoid breaking existing setups.
   # Please change @{DOVECOT_MAILSTORE} to (only) contain the directory
   # you use, and remove everything else.

   @{DOVECOT_MAILSTORE}=@{HOME}/Maildir/ @{HOME}/mail/ @{HOME}/Mail/ 
/var/vmail/ /var/mail/ /var/spool/mail

Which doesn't seem to be relevant for this. No idea how dovecot can put the 
mail into /maildirs/username, but since that's working I'm not complaining.
The file in abstractions only contains this:

   # used with dovecot/*

  abi ,

  capability setgid,

  deny capability block_suspend,

  # dovecot's master can send us signals
  signal receive peer=dovecot,

  owner @{run}/dovecot/config rw,

  # Include additions to the abstraction
  include if exists 

Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Dovecot logging to files causes issues

[Richard Rosner via dovecot]

I have a mailing server setup based on Debian Stable that uses postfix 
(v3.7.10) for SMTP and dovecot (v2.3.19.1 (9b53102964)) for IMAP. I now wanted 
to set dovecot to not write to syslog, but to dedicated files in 
/var/log/dovecot. While everything indicates that this happens successfully as 
the log files gain in size, I also get lots of these errors:

   May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: 
to=, relay=local, delay=3.2, delays=1.9/0.29/0/1.1, dsn=4.3.0, 
status=deferred (temporary failure. Command output: lda(user): Error: 
net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't open log 
file /var/log/dovecot/error.log: Permission denied )

If it would only log the complaints I wouldn't worry, but as long as I don't 
revert the changes in dovecot's config, mail receiving is at least vastly 
delayed, most likely stuck alltogether. So how am I supposed to set these 
settings?

I've chainged these settings in /etc/dovecot/conf.d/10-logging.conf:

   log_path = /var/log/dovecot/error.log
   debug_log_path = /var/log/dovecot/debug.log
   log_debug = category=error

The whole directory /var/log/dovecot is owned by dovecot:dovecot, permissions 
on debug.log, error.log and info.log are 644.

Best

Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Bug/Warning not sure which

[Richard Shetron]

We finally solved the problem.  For some reason dovecot stopped 
accepting mail.sgeinc.com but wanted sge.sgeing.com, the real name of 
the machine, not an alias name with the same IP address.  Why it was 
going for ssh-keygen we don't know.


On 3/3/2024 3:28 AM, Odhiambo Washington wrote:



On Sun, Mar 3, 2024 at 1:46 AM Richard Shetron <mailto:gue...@sgeinc.com>> wrote:


Hello,

My sysadmin and I spent a couple hours trying to figure out a POP3
problem that has worked for about 20 or so years.

We run our own dns for sgeinc.com <http://sgeinc.com>.
I've always used mail.sgeinc.com <http://mail.sgeinc.com> as my
incoming and outgoing server.  At
various times mail has been an alias for another machine.  It's
currently on the same address as sge.sgeinc.com
<http://sge.sgeinc.com>.  On the update forced
on us on 2/22/24 or 2/23/24 it stopped working.  It still works as an
outgoing server but incoming POP3 it stopped working.  It started
working when I changed my incoming server to sge.sgeinc.com
<http://sge.sgeinc.com>.
You might want to look into it or not.  We chased the initial problem
to, we think, ssh-keygen in /usr/local/bin/ which was Not found but is
there.


Why would dovecot need ssh-keygen? What for?

--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
  In an Internet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions: 
http://www.catb.org/~esr/faqs/smart-questions.html 
<http://www.catb.org/~esr/faqs/smart-questions.html>]

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Bug/Warning not sure which

[Richard Shetron]

Hello,

My sysadmin and I spent a couple hours trying to figure out a POP3
problem that has worked for about 20 or so years.

We run our own dns for sgeinc.com.
I've always used mail.sgeinc.com as my incoming and outgoing server.  At
various times mail has been an alias for another machine.  It's
currently on the same address as sge.sgeinc.com.  On the update forced
on us on 2/22/24 or 2/23/24 it stopped working.  It still works as an
outgoing server but incoming POP3 it stopped working.  It started 
working when I changed my incoming server to sge.sgeinc.com.
You might want to look into it or not.  We chased the initial problem 
to, we think, ssh-keygen in /usr/local/bin/ which was Not found but is 
there.


Thanks

root@sge:/usr/local/bin# dovecot -n
# 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 ()
# OS: Linux 5.4.0-172-generic x86_64 Ubuntu 20.04.6 LTS
# Hostname: sge.sgeinc.com
auth_mechanisms = plain login
mail_location = maildir:~/Maildir
namespace inbox {
    inbox = yes
    location =
    mailbox Drafts {
  special_use = \Drafts
    }
    mailbox Junk {
  special_use = \Junk
    }
    mailbox Sent {
  special_use = \Sent
    }
    mailbox "Sent Messages" {
  special_use = \Sent
    }
    mailbox Trash {
  special_use = \Trash
    }
    prefix =
}
passdb {
    driver = pam
}
protocols = " imap lmtp pop3"
service auth {
    unix_listener /var/spool/postfix/private/auth {
  mode = 0666
    }
    unix_listener auth-userdb {
  group = postfix
  mode = 0660
  user = postfix
    }
}
service pop3-login {
    inet_listener pop3 {
  port = 110
    }
}
ssl = required
ssl_cert = 

Re: Anyone Watching Actvity from this network? Attempting Dovecot Buffer Overflows?

[Richard Siddall]

Brendan Kearney wrote:


i have some rather old IpToCountry.csv files from a now defunct site. it 
mapped IP allocations to country and included the RIR, date assigned, 
etc.  this data is a few years old as the site was taken down and there 
is probably a lot of new or updated info.  a GeoDB subscription may be 
useful in the case you are looking at.


brendan



FWIW, if you look at 
https://github.com/milter-regex/milter-regex/blob/main/milter-regex-ip-prep.c 
it says you can "Download IP address allocation lists from the RIR ( 
Regional Internet Registry )


ftp://ftp.afrinic.net/pub/stats/afrinic/delegated-afrinic-latest
ftp://ftp.apnic.net/pub/stats/apnic/delegated-apnic-latest
ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest
ftp://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-latest
ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-latest";
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: No-novice with Dovecot, but need novice-like advice (was Dovecot cracked?!)

[Richard Troy]

re stuck with plain
 text until one of these things changes.


Your server is Linux and SSH client software has become quite available 
(PuTTY on Windows).


It's not an issue of possibility or ease but willingness to learn or 
desire to bother, I suppose.


I'm more than half-way through development of a simple but safe password 
changing mechanism that can run on a firewall / gateway machine and get 
the information picked up on an internal server system... The architecture 
was designed to be VERY easily be adapted to deal with pretty much any 
password scheme. Maybe I'll just finish it. Time is my issue!


(Yes, it'd be better to have it seamlessly integrated into the IMAP protocol, 
but don't forget that you'd need the *MUAs* to start supporting it as well 
before the general public will ever even learn about the new feature ...)


Yes, agreed, but if Dovecot and Postfix did their part, the rest would 
SURELY follow.


Thanks Jochen,
Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: No-novice with Dovecot, but need novice-like advice (was Dovecot cracked?!)

[Richard Troy]

However if your dovecot SASL is broken, say always permitting access with or 
without correct password, then there will be a problem


I DID find a discrepancy: smtpd_helo_restrictions did NOT have 
permit_sasl_authenticated. I made the change, of course and with that 
done, am now going to open the ports and renew my vigil for relays!


Fingers crossed!

Thanks, Jeremy - even if it doesn't work, it's a good clean shot at a fix! 
And, if that was it, it's easy to see how that could be overlooked. . .


Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: No-novice with Dovecot, but need novice-like advice (was Dovecot cracked?!)

[Richard Troy]

The problem will likely be postfix.


I actually doubt it but am VERY grateful for this remark:

However if your dovecot SASL is broken, say always permitting access with or 
without correct password, then there will be a problem


IT COULD BE?!

I don't know (or maybe recall ATM) enough about it to comment.

THANK YOU for your configuration exmaple! This I'll be on in just moments, 
thanks!


Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: No-novice with Dovecot, but need novice-like advice (was Dovecot cracked?!)

[Richard Troy]

On Thu, 8 Jun 2023, dove...@x9p.org wrote:


Logs?

Send the relevant logs so people can analyze the problem.



...The logs in full are HUGE, but I have some excerpts - Hope I caught the 
right stuff! I'll send along soon.


Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: No-novice with Dovecot, but need novice-like advice (was Dovecot cracked?!)

[Richard Troy]

On Fri, 9 Jun 2023, Sean Gallagher wrote:


It feels like you are conflating Dovecot with Postfix. Dovecot doesn't 
actually "relay" anything. (ignoring sieve and submission proxy). Relaying is 
the job of the "Mail Transfer Agent" or MTA. This is often Postfix but 
Dovecot could probably work with just about any standards-compliant mailer.


If your system is acting as an open relay, you need to look at your MTA 
configuration.


You should probably try the Postfix mailing list. Try the "postfix-users" 
list. https://www.postfix.org/lists.html




Thanks Sean,

The relaying only started and stopped when Dovecot was turned on or off.

Isn't it true that Dovecot performs an authentication function for inbound 
connect requests, the successful of which then may use the submission 
mechanism from what Postfix takes to be an internal connection to send 
emails? Is this mistaken?


However, I get your point and I've spent a lot of work on that area. And, 
you may well be right that that's where I need to turn - that is, to 
Postfix. Thanks for the link.


Richard
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: No-novice with Dovecot, but need novice-like advice (was Dovecot cracked?!)

[Richard Troy]

On Thu, 8 Jun 2023, Antonio Leding wrote:


Just curious - the first thing the hit me was “27 or so year-old…”

Fedora was released 19 years ago and Dovecot 20 — what am I missing?  And are 
you saying this box has been unchanged since ’03?




Hi Antonio,

I had a lot of that in the eamil and the on a proof-read realized it was 
FAR too long, so I cut out what you're asking about! -ugh- But since you 
seem to be a historian, I'll indulge a little:


...We really started with a couple of old Sun SPARC "pizza boxes", a DEC 
Alpha, and a few also-rans of the early '90s. ... I could get "into the 
weeds" about the OS choices of the day, but as soon as it was available we 
installed the very first version of Red Hat - paid for the disk! ... 
It was 10-BaseT, DSL / T1 networking - And getting a domain name? PAIN 
IN THE ASS! ...


We transitioned from Red Hat to Fedora as just "inertia." I barely noticed 
the difference - marketing / packaging in my view.


The site has undergone continuous maintenance, of course. Hardware ages 
out, software too, sometimes, etc. (and even people) Today we're mostly on 
Fedora Server with a mix of SuperMicro and oth but I digress...


As for pre-Dovecot? I seem to recall that from mid '99 or so we were on 
Courier, but most of us didn't use it because we had accounts on the 
"local" boxes and could just login and who needs IMAP? But, we had some 
need for it, mostly due to people who were less sophisticated - at least 
that's my view. I suppose there were other motivations.


I honestly don't recall exactly why we switched to Dovecot, but I suspect 
it was due to an early adopter who was helping with our systems at the 
time. I've got some leftover config files dating from '04 so it goes back 
at least that far. We traded off hosting hardware with the guy I'm talking 
about - he had hardware at our site and we had one or more boxes at his 
site, which helps reliability, etc...


Anyway, thanks for the trip down Memory Lane. But can you lend ANY advice 
here?


Thanks,
Richard___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

No-novice with Dovecot, but need novice-like advice (was Dovecot cracked?!)

[Richard Troy]

Hi All,

This is my first posting here, and maybe I should have found this WAY back 
in January, '23, if not LONG before. I want to be but I find it difficult 
here to be brief. ... Surely background will surely help:


A 27 or so year old Fedora / Postfix / Dovecot site I built had a major 
disaster in January and I've not yet been able to fully recover because 
Dovecot has let the damned spammers in again and again and again and 
again! OH, sure, I got it down to a trickle, but these few Russian sites 
always managed to get their spam through and I just had to shut Dovecot 
down entirely. I never found out how they got in, etc. And I've STRONGLY 
suspected Dovecot got cracked - at least the modern version in the 
youngest version for the youngest Fedora we had back in January - uh, 
Fedora Server 37 - I've forgotten the matching Dovecot version.


In the disaster, we lost /var but not /etc, so I figured recovery would be 
easy and for nearly everything, it was. But NOT Dovecot (and insofar as it 
matters, Postfix), and in these 5+ months I've tried so many things, I'm 
sure I've forgotten most of them and I don't know that a retroactive look 
is worth doing.


...I kept some notes that might be useful if anyone wants to see the 
evidence of the cracking, but in short, I kept a constant watch on the 
logs and when ANY relay happened that shouldn't, I'd instantly know it and 
shut things off entirely. However, that became untenable as I couldn't 
find the problem and had to just shut it off, pissing off users, etc, but 
I've had to do things like spend a month and a half traveling, and so 
forth and, well... Life goes on, as the saying goes.


---

NOW I want to try again.

It's my perception that it's a waste of time to even LOOK at the old 
Dovecot configuration stuff. I feel I need to REMOVE it ALL, and I could 
use some help being SURE to get it all gone. And then I think I need to do 
a FULL new installation. Overkill? IDK.


I could use some advice about SAFE ways to make changes and test to ensure 
we do NOT become an open mail relay EVER AGAIN.


ALSO WORTH SAYING is that if Dovecot were all that damned safe and secure 
I wouldn't so easily be able to propose a new feature that would make a 
HUGE difference to sites like mine: Give me a white-list of the ONLY 
accounts that can relay; NOTHING ELSE can relay. ... THAT would do it! But 
no! Neither in Postfix nor dovecot is there such a thing!


Combine that with a greylist type function where the usual IP addresses 
for particular users were let through, and new ones delayed, THAT would be 
awesome, too! And this isn't even all that hard to do - I could do it if I 
didn't already have a thousand obligations in life!


And if someone tells me I'm wrong and points me at how to do these things, 
I'll fall out of my damned chair! And after picking myself up, I'll find a 
way to send that person some sort of gift. THIS WOULD HAVE SOLVED ALL MY 
PROBLEMS. And I'm sure MANY others could use this, too!


---

THIS configuration:

I'd like to find a way to have both virtual and our existing "unix 
accounts" users.


IF we had an IMAP supported password CHANGING scheme, we'd gladly run 
encrypted passwords, but there isn't, and we haven't invented (finished 
inventing!) our own web-way to change 'em and so we're stuck with plain 
text until one of these things changes.


BTW, isn't this a HUGE and OBVIOUS hole that should have been fixed decades 
ago?! If a major provider like the Dovecot.org team added a way to update 
passwords to the IMAP protocol, all the rest of the folks would follow 
along for sure! OR, "is that a thing" and I'm just ignorant of it?


So, again, plain-text, in cram, of course. What else? Coach me on "the 
right way" if you want, but if users can't change it themselves, they'd 
rather I can retrieve it for them if needed... I'm sure the corporate 
world doesn't do it this way, but their code isn't open source, or am I 
wrong?...


---

In closing I don't actually anticipate ANY help.

My father, an even earlier computer user than me, once observed, "you can 
ask for information until you're blue in the face, and nobody will say a 
thing, but post the WRONG thing and a hundred people will post to point 
out you're wrong!"


GIVEN how EASY it is to have your email system become an instant open 
relay at the hands of the spammers out there, how the hell Dovecot can 
advertise the way it is WITHOUT a serious guide about this is just 
frustrating and laughable. But I'd love to be shown where they DO help 
with this!


Thanks for any and all help,
Richard

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Issue with one user only, exceeding connections

[Richard]

> Date: Thursday, June 09, 2022 11:07:38 -0500
> From: Jeremy Schaeffer 
>
> On 6/9/2022 10:59, Richard wrote:
>> 
>>> Date: Thursday, June 09, 2022 10:46:25 -0500
>>> From: Jeremy Schaeffer 
>>> 
>>> That was the first thing I tried, I lowered the cache connections
>>> in Thunderbird. Actually the max connections was 50, not 500, but
>>> I could see why as I do have a lot of folders, but what is odd is
>>> I have other mailboxes that have even more folders, but it's only
>>> one mailbox that is trowing the error.
>>> 
>>> "# ps -axww | grep imap" does not give me the same results -
>>> 
>>> .
>>> 
>>> 19897 ?    S  0:00 dovecot/imap
>>> 19900 ?    S  0:02 dovecot/imap
>>> 19901 ?    S  0:00 dovecot/imap
>>> 19902 ?    S  0:00 dovecot/imap
>>> .
>>> 
>>> I wish it did give me the mailbox, is there a option to get it to
>>> give me that information?
>>
>> Try "auxw" on your "ps". I.e., add in the "u" which will get you
>> the user detail in the first column, otherwise you just get the
>> process id.
>> 
>> 
> Thank you! That worked, I piped the output to a file, grep the
> username and sure enough there are 60 lines. So I guess going over
> 50 was a possibility.
> 
> Learn something new every day. I set the maximum to 100 so I should
> not have any errors on that anymore.
> 
 
Rather than simply upping the limit I think a reasonable question to
ask is why/how they are managing to do that. That's a lot of open
folders.
 
By the way, the single command:
 
 ps auxw | grep imap | cut -d" " -f1 | sort | uniq -c
 
will get you a nice list with the users and their connection counts.
 
 

Re: Issue with one user only, exceeding connections

[Richard]

> Date: Thursday, June 09, 2022 10:46:25 -0500
> From: Jeremy Schaeffer 
>
> That was the first thing I tried, I lowered the cache connections
> in Thunderbird. Actually the max connections was 50, not 500, but I
> could see why as I do have a lot of folders, but what is odd is I
> have other mailboxes that have even more folders, but it's only one
> mailbox that is trowing the error.
> 
> "# ps -axww | grep imap" does not give me the same results -
> 
> .
> 
> 19897 ?    S  0:00 dovecot/imap
> 19900 ?    S  0:02 dovecot/imap
> 19901 ?    S  0:00 dovecot/imap
> 19902 ?    S  0:00 dovecot/imap
> .
> 
> I wish it did give me the mailbox, is there a option to get it to
> give me that information?

Try "auxw" on your "ps". I.e., add in the "u" which will get you the
user detail in the first column, otherwise you just get the process
id.

Re: Issue with one user only, exceeding connections

[Richard]

> Date: Wednesday, June 08, 2022 14:14:23 -0500
> From: Jeremy Schaeffer 
>
> I keep having this issue with one user, and I have to restart
> dovecot several times a day to clear it. What I have is a postfix /
> dovecot mail server (Centos 7) and about a dozen users. All
> mailboxes are imap ssl. I monitor about 4 mailboxes on my computer
> and tablet. I use Thunderbird on the computer (cache connections at
> 2) and K9 on the tablet, but one user of the four I keep getting
> "Maximum number of connections from user+IP exceeded" and I have
> the maximum at 50 "(mail_max_userip_connections=50)" so its hard
> for me to believe I am actually exceeding it unless dovecot/client
> is not dropping connections and keeps starting new ones until it
> reaches the maximum, but again, only for one user, even though I am
> monitoring 4 on the same devices. Any idea how to troubleshoot
> this? I don't know if I should be looking at dovecot or the
> clients, or what I need to look for. It's been going on since I put
> this server in use over a year ago. I also have issues with Outlook
> clients disconnecting, just outlook, is there any recommended
> settings to make Outlook work smoother?
> 
> Thanks! - Jeremy
> 
> Config -
> 
># 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf
># OS: Linux 3.10.0-1160.11.1.el7.x86_64 x86_64 CentOS Linux release
># 7.9.2009 (Core) Hostname: ***

> }
> protocol imap {
>    mail_max_userip_connections = 50
> }
> 

On my centos-7 dovecot install (same versions you indicate you are
using), that connection-limiting line is in:

  /etc/dovecot/conf.d/20-imap.conf

while you appear to have it in the main dovecot.conf file. If you
have that in both places I'm not certain which one takes precedence,
but you might want to check. [In my setup almost all the
configuration pieces are in include files, rather than the main
.conf.]

Starting with a "ps" (and grep &etc.) it's fairly easy to see how
many connections any account has open. An open mailbox has one or two
connections and then every folder (under a mailbox) that is open will
have a connection associated with it too, which pushes up the count
quickly if someone holds a lot of folders open.

Re: doveadm pw usage

[Richard Hector]

On 24/04/22 22:45, ミユナ (alice) wrote:

ok the helps says:

pw   [-l] [-p plaintext]

i just thought it specifies the text file.

thanks for clarifying it.



Bernardo Reino wrote:
The argument to "-p" is not a file containing the password, but the 
password itself!



The downside of putting the password on the command line is that it will 
(briefly) be visible in the output of 'ps':


richard   9449  0.0  0.0   5040  3616 pts/4R+   19:27   0:00 
/usr/bin/doveconf -f service=doveadm -c /etc/dovecot/dovecot.conf -m 
doveadm -e /usr/bin/doveadm pw -p asdf


Cheers,
Richard

Re: how to setup IMAPs with letsencrypt

[Richard Hector]

On 24/04/22 13:14, ミユナ (alice) wrote:



Richard Hector wrote:

otherwise you'll have to use DNS challenge method
to support multiple hostnames on the same certificate.


Um, no I didn't. I replied to that. Please check your attributions :-)

Cheers,
Richard

Re: how to setup IMAPs with letsencrypt

[Richard Hector]

On 22/04/22 11:57, Joseph Tam wrote:

Keep in mind the subject name (CN or SAN AltNames) of your certificate
must match your IMAP server name e.g. if your certificate is
made for "www.mydomain.com", you'll have to configure your IMAP
clients to also use "www.mydomain.com" as the IMAP server name.

This typically means the web and IMAP server must reside on the
same server, otherwise you'll have to use DNS challenge method
to support multiple hostnames on the same certificate.


_A_ web server has to be there. It doesn't have to serve anything else 
useful. My mail server has a web server that only serves the LE 
challenge. Well, actually it's a proxy server that serves several other 
domains too, but there's nothing else served on that domain (at the moment).


Cheers,
Richard

Re: Dovecot and OAuth2 and gmail

[Richard]

> Date: Wednesday, April 13, 2022 17:10:36 -0400
> From: Steve Litt 
>
> Hi all,
> 
> I'm not sure Dovecot has anything to do with this, but I'd rather
> ask and know for sure.
> 
> I do the following:
> 
> Gmail IMAP=>fetchmail=>procmail=>Dovecot IMAP
> 
> Then, I view my Dovecot hosted email with Claws-Mail.
> 
> I understand that on May 31, 2022, current methods to access Gmail
> IMAP will turn into pumpkins because of insistence on OAuth2. Do I
> need to do anything to Dovecot to get ready for this Mass
> Extinction Event?
> 
> Do you think I'll need to dump fetchmail for something else?
> 

While it's a little obtuse, a reading of:



specifically the "fix problems" / "use an app password" section,
would seem to indicate that app passwords will continue to work for
gmail access after May 30. 

What google is turning off is "third-party apps or devices which ask
you to sign in to your Google Account using only your username and
password", i.e., using your google login password with "less secure"
3rd party apps.

So, you need to set up, and configure your application to use, app
passwords.

Re: Dovecot book for a newbie

[Richard Doyle]

https://www.amazon.com/Dovecot-POP3-IMAP-servers-enterprises/dp/1534895701


On 7/7/21 10:04 AM, techli...@phpcoderusa.com wrote:
> 
> 
> Hi,
> 
> Please recommend a Dovecot book for a newbie... I have a fair amount of
> Linux PHP hosting experience - LAMP virtual host configurations.  I'm
> new to BIND, Postfix, and Dovecot.
> 
> I'm running Ubuntu 20.04lts.
> 
> I have a test server almost working.  Can send but not receive.  Would
> like to understand more.  I'm guessing it is a Zone (MX) / SSL / Client
> configuration issue.
> 
> Thanks in advance!!
> 

Re: Unable to run dovecot.2.3.15

[Richard]

> Date: Monday, June 28, 2021 09:19:07 +0800
> From: Joe Wong 
>
>  I am trying to setup a new server on Centos 7 running 2.3.15, I
> compile the source with ./configure --with-pam , make then make
> install. I have another server using 2.3.8 so I copy my config
> files from there to /usr/local/etc/dovecot. I am using systemctl
> enable dovecot then systemctl start dovecot , however it failed to
> start. There is error found in /var/log/messages:
> 
> Jun 28 09:14:37 new-ns1 systemd: Started Dovecot IMAP/POP3 email
> server. Jun 28 09:14:37 new-ns1 dovecot: Error:
> unlink(/usr/var/run/dovecot/login/login) failed: Read-only file
> system (in master-settings.c:739)

Unless "/usr/var/run/..." is on a mounted filesystem, that is mounted
read-only, that "Read-only file system" message is generally an
indication of a failing disk that has gone read-only. 

So, check to see if that file system is mounted, and if so, adjust as
necessary. If not a mounted file system, then I'd check the disk, and
start looking for a replacement.

Re: Quota-status service on Director

[Richard Pablo Mérida González]

Did you solve this problem of quota status? I am in the same situation 
right now, i would appreciate you let me know any helpfull information


Sent with Aqua Mail for Android
https://www.mobisystems.com/aqua-mail

How to setup Dovecot director in a NAT-like environment?

[Richard Mader]

How can I setup Dovecot director in a NAT like environment?

My mailservers are running within docker and one of them behind a 
router, so the IP that dovecot binds to does not match the IP that other 
mailservers have to use to connect to them.


Since when running director the servers connect to each other, I'd guess 
I'd have to set the external IP of the servers in the 
"director_servers". If I do that I'm unable to start the director nodes 
since they cannnot find out which of the addresses is their own.
I had a quick peek at the sourcecode and director detects its own 
address by trying to bind to all of the addresses. Since we're behind a 
NAT, that will of course fail for its external IP.


When I set the local address or localhost in the "director_servers" 
directive, then the director nodes are able to start but they are denied 
when connecting to other nodes.

When using the NAT-internal IP I've got the following error:
Error: Remote director thinks it's someone else (connected to 
:9090, remote says it's 
:9090)

When using localhost/127.0.0.1 then I get:
Info: Connecting to :9090 (as 127.0.0.1): 
Alone in director ring - trying to connect to others
Error: connect(:9090) failed: Invalid 
argument



Is it somehow possible to tell director which of the addresses within 
"director_servers" is its own address, that it then advertises to other 
servers, instead of trying figuring that out itself by trying to bind to 
all ip-addresses on startup?
Kafka e.g. has the "advertised.listeners" property to set a custom host 
to announce to clients and to other Kafka nodes.


If there is not such a property or any other workaround, I might 
consider implementing this into director myself. Although I might need 
some help getting started since this is my first time contributing to a 
non Scala/Java or Typescript/NodeJS Project.
So far I've discovered that Github seems to be the official repo and is 
not just a mirror and I was able to compile it on my machine. Now I've 
got questions such as, where can I find the binaries that I've just 
compiled and whats the best way to get them running locally and to test 
my changes, where are unit tests located, ...?


Best Regards,
Richard

Re: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied

[Richard Hector]

Hi, I'm seeing the same, but I think it's every time the lda is called 
to deliver mail to a mailbox. Postfix runs it as vmail:vmail.


My socket is owned by root:dovecot (on Debian buster).

Should I add vmail to the dovecot group to enable it to write? Or is 
changing the socket to 0666 safer?


I'm also a little confused, because according to
https://doc.dovecot.org/configuration_manual/stats/ no statistics are 
logged by default, and I have no mentions of 'metric' anywhere in my 
config, so why is the write even attempted? (dovecot 2.3.4)


Cheers,
Richard

On 24/10/20 4:46 am, Aki Tuomi wrote:

Hi!

You can ignore it, or you can change the socket permissions to 0666.

service stats {
unix_listener stats-writer {
   mode = 0666
}
}

Aki



On 23/10/2020 17:52 mj  wrote:

 
Hi,


Nobody?

It happens so rarely, and the system appears to be running fine 
otherwise, should I just ignore it?


Still makes me wonder way it would happen at all..?

MJ

On 10/22/20 12:53 PM, mj wrote:
> Hi,
> 
> We are getting very occasional messags from dovecot:
> 
>> net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
> 
> Over the last week, the message appeared five times. (on a mail server 
> with over 100 users, to that's basically almost never)
> 
> doveconf -n below
> 
>> # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf

>> # Pigeonhole version 0.5.4 ()
>> # OS: Linux 4.19.0-10-amd64 x86_64 Debian 10.6 xfs
> 
> snip...
> 
>> service stats {

>>   unix_listener stats-reader {
>>     group = vmail
>>     mode = 0660
>>     user = vmail
>>   }
>>   unix_listener stats-writer {
>>     group = vmail
>>     mode = 0660
>>     user = vmail
>>   }
>> }
> 
> and the on-disk permissions are:
> 
>> root@dovecot:~# ls -l /var/run/dovecot/*stat*

>> srw--- 1 root  root  0 Oct  6 00:25 /var/run/dovecot/old-stats
>> prw--- 1 root  root  0 Oct  6 00:25 /var/run/dovecot/old-stats-mail
>> prw--- 1 root  root  0 Oct  6 00:25 /var/run/dovecot/old-stats-user
>> srw-rw 1 vmail vmail 0 Oct  6 00:25 /var/run/dovecot/stats-reader
>> srw-rw 1 vmail vmail 0 Oct  6 00:25 /var/run/dovecot/stats-writer
> 
> We're not sure what makes the Permission denied error happen...
> 
> Anyone with an idea?
> 
> MJ

Re: OT: SASL questions

[Richard Hector]

On 21/08/20 7:15 pm, @lbutlr wrote:
> On 21 Aug 2020, at 01:05, Richard Hector  wrote:
>> Is that a standard interface? ie can a client like postfix talk to
>> either dovecot or cyrus without knowing the difference?
> 
> Yes. Postfix does not care, though I find it is easier to setup and more 
> reliable to use dovecot (I've used both, YMMV).

Thanks - is there documentation of this protocol somewhere?
Though having just now had another look at the Postfix SASL_README, it
appears it needs support for each compiled in, suggesting there are
differences?

>> Are there others?
> 
> Those are the only two I have used. If there are others I've not seen them 
> mentioned on the postfix list that I can recall.

Postfix, AFAICS, only supports the two - but I've seen references for
IRC servers talking to an irc services server called anope, which
provides SASL somehow?

>> Is there a good reference to this somewhere, short of reading the RFCs?
> 
> The best bet is
> 
>   1) get a real cert.
>   2) copy and existing configuration

I'm not following - I'm not sure we're on the same page :-(
I already have Postfix (with a Letsencrypt cert) using Dovecot SASL
(Dovecot also uses the same cert)
Or are you talking about some other kind of cert? And are you talking
about the Postfix and/or Dovecot config?

>> And is there any option (current or proposed) to let dovecot act as a
>> client, rather than a server?
> 
> A client for…?

A SASL client - so eg Dovecot and Postfix could both talk to the same
Cyrus (or other - even another Dovecot) SASL server. One reason might be
to use password hash algorithms that Dovecot doesn't know about.

Cheers,
Richard

OT: SASL questions

[Richard Hector]

Hi all,

Apologies for the somewhat off-topic questions - I'm trying to get my
head around SASL, and what it is/does.

I can see that a (SMTP, IMAP etc) server can offer various
authentication methods, which are pre-defined and which the client can
choose from. SASL then seems to define how those work.

But dovecot and cyrus-sasl both seem to offer a client-server interface,
usable by eg postfix, which I don't see any reference to in SASL
summaries online.

Is that a standard interface? ie can a client like postfix talk to
either dovecot or cyrus without knowing the difference? Are there others?

Is there a good reference to this somewhere, short of reading the RFCs?

And is there any option (current or proposed) to let dovecot act as a
client, rather than a server?

Thanks,
Richard

Re: handling spam from gmail.

[Richard Siddall]

Marc Roos wrote:



I am sick of this gmail spam. Does anyone know a solution where I can do
something like this:

1. received email from adcpni...@gmail.com
2. system recognizes this email address has been 'whitelisted', continue
with 7.
3. system recognizes as this email never been seen before
4. auto reply with something like (maybe with a wait time of x hours):
Your message did not receive the final recipient. You are sending
from a known spam provider
network that is why we blocked your message. Please confirm that:
- you are not a spammer and
- you have permission to use the mail adress you send your message to
- you and your provider agree to uphold GDPR legislation
- you and your provider are liable for damages when breaching any of
the above.



Click link to confirm and you agree with the above
https://www.domainwithoutletsencryptcertificate.com/asdfasdfadsfaf

5. sender clicks confirm url
6. email address is added to some white list.
7. email is delivered to recipient.



This seems similar to the long-dead Active Spam Killer 
(https://directory.fsf.org/wiki/Active_Spam_Killer) with updates for GDPR.

Re: ot: copy physical mail files ?

[Richard Siddall]

Voytek Eymont wrote:

dumb question warning:

I have two email accounts, me@tld1 and me@tld2
often, get emails intended (by me) for me@tld2 sent to me@tld1

this is purely for my own preferred record keeping preference,

as I run the server, is it 'a really bad things' if I simply copy relevant
mail files from
/var/vmail/vmail1/tld1/cur to /var/vmail/vmail1/tld2.../cur ?

rather than forward emails as I have been doing?

thanks




I used to move mail files around but now use "doveadm move".

You can probably move mail from me@tld1 to me@tld2 with something like:

doveadm move -u me@tld2 INBOX user me@tld1 MAILBOX INBOX subject party

where party is a string that appears only in the subject line of the 
emails you want to move.


Richard.

Re: newbie question on a dovecot buffer

[Richard Bown via dovecot]

Many thanks for all the replies on this.
It has given a lot to consider, now for some intense reading .

On 06/11/2019 04:47, Plutocrat via dovecot wrote:

On 06/11/2019 01.41, Richard Bown via dovecot wrote:
  > Can I use Dovecot as a buffer between my mail providers Dovecot servers

and the several IMAP mail clients on my own network ?
ie,  so Dovecot would mirror the mail on my account with my mail provider, So 
my local dovecot server downloads mail and caches/stores on the local SBC and 
all the local users use IMAP from that, instead of from my mail provider.


I'm by no means an expert, but I've seen this mentioned before, and it seems 
that you may be asking about this:

https://wiki.dovecot.org/Replication

P.



--
 Best wishes /73
 Richard Bown

 Email : rich...@g8jvm.com

 HTTP  :  http://www.g8jvm.com

 ##

 Ham Call: G8JVM . QRV: 50-432 MHz + Microwave
 Maidenhead QRA: IO82SP38, LAT. 52 39.720' N LONG. 2 28.171 W
 QRV 6mtrs 200W, 4mtrs 150W, 2mtrs 300W, 70cms 200W,
 Microwave 1296MHz 110W, 2320MHz 100W, 5760MHz 10W & 10368MHz 5W
 OS: Linux Mint 19.2 x86_64 Tina, on a Dell Inspiron 3580 laptop
 ##
Come back Guy Fawkes, all is forgiven

newbie question on a dovecot buffer

[Richard Bown via dovecot]

Hi
apologies for what will seem a daft question
I'm getting problems with my mail provider who is using dovecot.
I've loaded Dovecot on a 8 core ARMHF SBC version 2.2.33.2 on Ubuntu 
18.04. The SBC is headless. and the load looks very light and only needs 
to store/cache 4GB of mail


Can I use Dovecot as a buffer between my mail providers Dovecot servers
and the several IMAP mail clients on my own network ?

ie,  so Dovecot would mirror the mail on my account with my mail 
provider, So my local dovecot server downloads mail and caches/stores on 
the local SBC and all the local users use IMAP from that, instead of 
from my mail provider.
Can I import the same directory structure as help on my mail providers 
server ?
I'm trying to avoid the timeout problem I get when using IMAP from my 
mail provider.

Can it be done ?
and can you withstand the questions on configuration, I haven't used 
Dovecot before so its a vertical learning curve

Thanks
--
 Best wishes /73
 Richard Bown

 Email : rich...@g8jvm.com

 HTTP  :  http://www.g8jvm.com

 ##

 Ham Call: G8JVM . QRV: 50-432 MHz + Microwave
 Maidenhead QRA: IO82SP38, LAT. 52 39.720' N LONG. 2 28.171 W
 QRV 6mtrs 200W, 4mtrs 150W, 2mtrs 300W, 70cms 200W,
 Microwave 1296MHz 110W, 2320MHz 100W, 5760MHz 10W & 10368MHz 5W
 OS: Linux Mint 19.2 x86_64 Tina, on a Dell Inspiron 3580 laptop
 ##
Come back Guy Fawkes, all is forgiven

Re: Password database - external verification questions

[Richard Hector via dovecot]

On 10/05/19 10:10 AM, Richard Hector via dovecot wrote:
> Hi all,
> 
> I'm currently using a PostgreSQL database for my user/password db,
> directly from dovecot. The trouble with that is that I'm stuck with
> whatever hash algorithms dovecot supports - which IIRC means (a subset
> of?) what libc has been compiled with, which can be a bit restrictive.
> 
> So I'd like to use an external tool, which would also let me integrate
> other applications (eg web apps).
> 
> PAM seems to be most suited to sharing accounts with the OS, which isn't
> what I want.
> 
> BSDAuth likewise, but I'm not using BSD.
> 
> CheckPassword looks like a somewhat convoluted protocol, but maybe the
> best bet?
> 
> IMAP - well, that's circular :-)
> 
> OAuth2 looks possible, but seems to be focused on http?
> 
> Any suggestions? And recommended implementations?
> 
> How hard is it to add extra methods?

No tips?

Are my requirements/preferences quite unusual?

Am I asking a silly question?

Am I misunderstanding/exaggerating the limitations of dovecot's/libc's
algorithms?

Thanks,
Richard

Password database - external verification

[Richard Hector via dovecot]

Hi all,

I'm currently using a PostgreSQL database for my user/password db,
directly from dovecot. The trouble with that is that I'm stuck with
whatever hash algorithms dovecot supports - which IIRC means (a subset
of?) what libc has been compiled with, which can be a bit restrictive.

So I'd like to use an external tool, which would also let me integrate
other applications (eg web apps).

PAM seems to be most suited to sharing accounts with the OS, which isn't
what I want.

BSDAuth likewise, but I'm not using BSD.

CheckPassword looks like a somewhat convoluted protocol, but maybe the
best bet?

IMAP - well, that's circular :-)

OAuth2 looks possible, but seems to be focused on http?

Any suggestions? And recommended implementations?

How hard is it to add extra methods?

Thanks,
Richard

sql table definitions

[Richard Hector via dovecot]

Hi all,

Trivial question ...

I'm using PostgreSQL for my auth db. I used the example CREATE TABLE
statement in the config file, but now I find the fields are too short. I
assume dovecot will be fine with 'text' type columns replacing the
varchars? Or failing that, I can change the length of the varchar fields?

Cheers,
Richard

Re: How to backup maildir

[Richard Hector via dovecot]

On 10/02/19 4:50 PM, Robert Moskowitz via dovecot wrote:
> I have been thinking, and reading, on how to back up my mailserver. I
> have not found any approach that seems ready to use.
> 
> 
> I have run years without any backup, but would really like to have
> something in place.  I figure I can attach a USB drive and backup to
> that, then from there rsync to something elsewhere.  Further if that USB
> drive is a full mailserver image, I actually have a 'hot backup' where I
> only have to put the backup drive into a system and boot up at the last
> backup.
> 
> But this means properly copying all of /home/vmail and probably
> /home/sieve plus the /var/lib/mysql
> 
> Are there good tools that nicely does this?  Or do I choose a time late
> at night (only I am sometimes in non-US timezones) to shut down all
> services and just use rsync?
> 
> And stopping services itself is thought provoking.  What if Dovecot,
> amavis, mysql, or whatelse is in the middle of writing out a mail file
> what happens to that file and restart.

I use dirvish, which is based on rsync, and which only syncs changed
files but uses hardlinks to keep storage and bandwidth down - it leaves
me with a full copy of the tree for each date, with links between. I run
it on a machine at home.

I don't ususally worry about stopping services; maildir should be all
right. For databases, I use the backup tools provided to back up the
running database, then use dirvish on the backups (and skip the database
files themselves).

I do do it at night, to save load on the server (it's an LXC container
on a VPS).

I believe rdiffbackup does something similar to dirvish.

Richard

Re: Connection refused (61)

[Richard]

which confirms that dovecot (nor anything else) is listening on that
port. the results from the roundcube test were misleading. always
test deeper -- don't assume that client-based test are doing what you
expect.




> Date: Thursday, July 12, 2018 00:00:31 +0200
> From: Teno Deuter 
>
> produces an empty result!
> 
> On Wed, Jul 11, 2018 at 11:57 PM, Richard
>  wrote:
>> what does the output of:
>> 
>> netstat -n | grep :587
>> 
>> run as root, show you? the -p will give the program and pid.
>> 
>> 
>> 
>>> Date: Wednesday, July 11, 2018 21:51:09 +
>>> From: Larry Rosenman 
>>> 
>>> Yep, you (probably) need to configure openSMTPD to listen on 587
>>> 
>>> (I run exim, so I can't help with that).
>> 
>> 

Re: Connection refused (61)

[Richard]

obvious correction ... sorry.

  netstat -np | grep :587


> Date: Wednesday, July 11, 2018 21:57:31 +
> From: Richard 

> what does the output of:
> 
> netstat -n | grep :587
> 
> run as root, show you? the -p will give the program and pid.
> 
> 
> 
>> Date: Wednesday, July 11, 2018 21:51:09 +
>> From: Larry Rosenman 
>> 
>> Yep, you (probably) need to configure openSMTPD to listen on 587
>> 
>> (I run exim, so I can't help with that).
> 
> 

 

Re: Connection refused (61)

[Richard]

what does the output of:

netstat -n | grep :587

run as root, show you? the -p will give the program and pid.



> Date: Wednesday, July 11, 2018 21:51:09 +
> From: Larry Rosenman 
>
> Yep, you (probably) need to configure openSMTPD to listen on 587
> 
> (I run exim, so I can't help with that).

Re: AuthDatabase CheckPassword broken?

[Richard Hector]

On 02/02/18 14:19, Mark Foley wrote:
> Script didn't run:
> 
>   File "/root/tmp/checkpwtest.py", line 8
> o?= with os.fdopen(DOVECOT_PW_FD, 'r') as s:
> ^
> SyntaxError: invalid syntax

Copy/paste error? The beginning of that line doesn't seem to be in the
original.

Richard

Re: sieve filter move wrong email to Junk folder

[Richard]

> Date: Thursday, December 14, 2017 09:47:44 -0800
> From: Gao 
>
> I use a sieve filter to move spam email to user's Junk folder:
># cat spam_to_junk.sieve
> require "fileinto";
>    if exists "X-Spam-Status" {
>    if header :contains "X-Spam-Status" "YES" {
>    fileinto "Junk";
>    stop;
>    } else {
>    }
>    }
>    if header :contains "subject" ["SPAM?"] {
>      fileinto "Junk";
>      stop;
>    }
> 
> Most time this filter works fine but occasionally it move non-spam
> in to Junk folder. Here is an example, this email is from dovecot
> mailling list and it end up in my Junk folder. Mailllog and header
> here. Would someone help me to figure out what went wrong here?
> 
> Thanks.
> 
> Gao
> 

  > X-Spam-Status: No, score=-2.9 required=5.0 
 tests=ALL_TRUSTED,BAYES_00


Because of the way you are bounding it, I suspect that the "YES" in
BAYES_00, at the end of that line, is triggering the mis-filing. 

Why not make:

   contains "X-Spam-Status" "YES"

a single string:

   contains "X-Spam-Status: YES"

that would be more precise and avoid this issue.

Re: is a self signed certificate always invalid the first time

[Richard Hector]

On 18/08/17 20:05, Stephan von Krawczynski wrote:
> On Fri, 18 Aug 2017 00:24:39 -0700 (PDT)
> Joseph Tam  wrote:
> 
>> Michael Felt  writes:
>>
>>>> I use acme.sh for all of my LetsEncrypt certs (web & mail), it is
>>>> written in pure shell script, so no python dependencies.
>>>> https://github.com/Neilpang/acme.sh  
>>>
>>> Thanks - I might look at that, but as Ralph mentions in his reply -
>>> Let's encrypt certs are only for three months - never ending circus.  
>>
>> I wouldn't characterize it as a circus.  Once you bootstrap your first
>> certificate and install the cert-renew cron script, it's not something
>> you have to pay a lot of attention to.  I have a few LE certs in use,
>> and I don't think about it anymore: it just works.
>>
>> The shorter cert lifetime also helps limit damage if your certificate
>> gets compromised.
>>
>> Joseph Tam 
> 
> Obviously you do not use clustered environments with more than one node per
> service.
> Else you would not call it "it just works", because in fact the renewal is
> quite big bs as one node must do the job while all the others must be
> _offline_.
> 

Couldn't the others just proxy to the one, for the .well-known
directory? They can continue serving up the rest of the site fine, surely?

I've worked with clusters, and with LE/certbot, but not yet both together.

Richard

Re: Permission denied when logrotating dovecot.log

[Richard]

> Date: Sunday, March 19, 2017 15:28:35 +1300
> From: Michael Heuberger 
>
> On 19/03/17 15:12, Richard wrote:
>> 
>>> Date: Sunday, March 19, 2017 14:56:01 +1300
>>> From: Michael Heuberger 
>>> 
>>> On 19/03/17 13:43, Richard wrote:
>>>>> Date: Sunday, March 19, 2017 13:32:57 +1300
>>>>> From: Michael Heuberger 
>>>>> 
>>>>> Hello guys
>>>>> 
>>>>> Having headaches here how to make logrotation for dovecot log
>>>>> files work. Having permission issues:
>>>>> 
>>>>> michael.heuberger@xxx /e/l/daily ❯❯❯ sudo logrotate -fv
>>>>> dovecot.daily
>>>>>  ⏎
>>>>> reading config file dovecot.daily
>>>>> 
>>>>> Handling 1 logs
>>>>> 
>>>>> rotating pattern: /var/log/dovecot*.log  forced from command
>>>>> line (10 rotations)
>>>>> empty log files are rotated, old logs are removed
>>>>> considering log /var/log/dovecot.log
>>>>> error: skipping "/var/log/dovecot.log" because parent directory
>>>>> has insecure permissions (It's world writable or writable by
>>>>> group which is not "root") Set "su" directive in config file to
>>>>> tell logrotate which user/group should be used for rotation.
>>>>> 
>>>>> This is my current logrotation conf for dovecot:
>>>>> 
>>>>> /var/log/dovecot*.log {
>>>>> rotate 10
>>>>> missingok
>>>>> sharedscripts
>>>>> postrotate
>>>>> doveadm log reopen
>>>>> endscript
>>>>> }
>>>>> 
>>>>> And the /var/log folder has these permissions:
>>>>> 
>>>>> drwxrwxr-x 12 root syslog   4.0K Mar 19 12:43 log
>>>>> 
>>>>> Any clues what's wrong?
>>>> As the message says:
>>>> 
>>>>   > because parent directory has insecure permissions
>>>>   > (It's world writable or writable by group which
>>>>   > is not "root") 
>>>> 
>>>>   > drwxrwxr-x 12 root syslog   4.0K Mar 19 12:43 log
>>>> 
>>>> On my RHEL derived systems, /var/log is root.root (and even then,
>>>> is not writable by group).
>>> Thank you. And what user/group/file perms does your dovecot.log
>>> file have?
>>> 
>>> - Michael
>>> 
>>> 
>> I log dovecot via syslog to [/var/log/]maillog, rather than its own
>> log file. That file is owned root.root and has permissions of 600.

> Well, I tried the same but it didn't work.
> 
> Setting my dovecot.log to 600 with root:root is breaking my mail
> system. I am then unable to receive and open emails.
> 
> Had to apply an ugly hack
> 
> /var/log/dovecot*.log {
> su syslog syslog
> create 666 syslog syslog
> rotate 10
> ...
> }
> 
> Like that anyone who wants to access/write to it, can do it and all
> works.
> 
> That's my problem. Do not know who/what/how to set this up
> correctly.
> 
> - Michael
> 

I would be inclined to just log dovecot to the syslog mail facility,
which I believe is the default (in 10-logging.conf) -- in the RHEL
setup anyway, and what I do:

   log_path = syslog

   syslog_facility = mail

Re: Permission denied when logrotating dovecot.log

[Richard]

> Date: Sunday, March 19, 2017 14:56:01 +1300
> From: Michael Heuberger 
>
> On 19/03/17 13:43, Richard wrote:
>> 
>>> Date: Sunday, March 19, 2017 13:32:57 +1300
>>> From: Michael Heuberger 
>>> 
>>> Hello guys
>>> 
>>> Having headaches here how to make logrotation for dovecot log
>>> files work. Having permission issues:
>>> 
>>> michael.heuberger@xxx /e/l/daily ❯❯❯ sudo logrotate -fv
>>> dovecot.daily
>>>  ⏎
>>> reading config file dovecot.daily
>>> 
>>> Handling 1 logs
>>> 
>>> rotating pattern: /var/log/dovecot*.log  forced from command line
>>> (10 rotations)
>>> empty log files are rotated, old logs are removed
>>> considering log /var/log/dovecot.log
>>> error: skipping "/var/log/dovecot.log" because parent directory
>>> has insecure permissions (It's world writable or writable by group
>>> which is not "root") Set "su" directive in config file to tell
>>> logrotate which user/group should be used for rotation.
>>> 
>>> This is my current logrotation conf for dovecot:
>>> 
>>> /var/log/dovecot*.log {
>>> rotate 10
>>> missingok
>>> sharedscripts
>>> postrotate
>>> doveadm log reopen
>>> endscript
>>> }
>>> 
>>> And the /var/log folder has these permissions:
>>> 
>>> drwxrwxr-x 12 root syslog   4.0K Mar 19 12:43 log
>>> 
>>> Any clues what's wrong?
>> 
>> As the message says:
>> 
>>   > because parent directory has insecure permissions
>>   > (It's world writable or writable by group which
>>   > is not "root") 
>> 
>>   > drwxrwxr-x 12 root syslog   4.0K Mar 19 12:43 log
>> 
>> On my RHEL derived systems, /var/log is root.root (and even then,
>> is not writable by group).
>
> Thank you. And what user/group/file perms does your dovecot.log
> file have?
> 
> - Michael
> 
> 

I log dovecot via syslog to [/var/log/]maillog, rather than its own
log file. That file is owned root.root and has permissions of 600.

Re: Permission denied when logrotating dovecot.log

[Richard]

> Date: Sunday, March 19, 2017 13:32:57 +1300
> From: Michael Heuberger 
>
> Hello guys
> 
> Having headaches here how to make logrotation for dovecot log files
> work. Having permission issues:
> 
> michael.heuberger@xxx /e/l/daily ❯❯❯ sudo logrotate -fv
> dovecot.daily
>  ⏎
> reading config file dovecot.daily
> 
> Handling 1 logs
> 
> rotating pattern: /var/log/dovecot*.log  forced from command line
> (10 rotations)
> empty log files are rotated, old logs are removed
> considering log /var/log/dovecot.log
> error: skipping "/var/log/dovecot.log" because parent directory has
> insecure permissions (It's world writable or writable by group
> which is not "root") Set "su" directive in config file to tell
> logrotate which user/group should be used for rotation.
> 
> This is my current logrotation conf for dovecot:
> 
> /var/log/dovecot*.log {
> rotate 10
> missingok
> sharedscripts
> postrotate
> doveadm log reopen
> endscript
> }
> 
> And the /var/log folder has these permissions:
> 
> drwxrwxr-x 12 root syslog   4.0K Mar 19 12:43 log
> 
> Any clues what's wrong?


As the message says:

  > because parent directory has insecure permissions
  > (It's world writable or writable by group which
  > is not "root") 

  > drwxrwxr-x 12 root syslog   4.0K Mar 19 12:43 log

On my RHEL derived systems, /var/log is root.root (and even then, is
not writable by group).

Re: Sieve LDA Errors (Improper Saving?)

[Richard Laager]

On 02/07/2017 06:51 PM, Stephan Bosch wrote:
> Newer versions of Pigeonhole may use a different version of the compiled
> binary format. So, for some upgrades it may be necessary to recompile.

> Anyway, for now you should be helped by just manually recompiling.

Manually recompiling fixed it. We had upgraded a while back, so the
version difference could very well be the issue. Thanks!

-- 
Richard

Sieve LDA Errors (Improper Saving?)

[Richard Laager]

I'm getting lots of errors like this (possibly on every message delivery):

imap2 dovecot: lmtp(rlaa...@wiktel.com): Error: OU02K+gQmFhUAwAAVtfydQ
: sieve: binary save: failed to create temporary file:
open(/var/lib/dovecot/sieve/junk-mail.svbin.ima
p2.852.) failed: Permission denied (euid=500(vmail) egid=500(vmail)
missing +w perm: /var/lib/dovecot/
sieve, dir owned by 0:0 mode=0755)

imap2 dovecot: lmtp(rlaa...@wiktel.com): Error: OU02K+gQmFhUAwAAVtfydQ:
sieve: The LDA Sieve plugin does not have permission to save global
Sieve script binaries; global Sieve scripts like
`/var/lib/dovecot/sieve/junk-mail.sieve' need to be pre-compiled using
the sievec tool

It's intentional in my setup that the vmail user can't write to the
global sieve script directory. But it shouldn't need to, as those are
already pre-compiled:

rlaager@imap2:/var/lib/dovecot/sieve$ ls -la
total 12
drwxr-xr-x 2 root root 4096 Nov 29 22:27 .
drwxr-xr-x 3 root root 4096 Feb  6 20:39 ..
lrwxrwxrwx 1 root root   53 Sep 12 01:35 junk-mail.sieve ->
/usr/share/wiktel-server-mail-backend/junk-mail.sieve
-rw-r--r-- 1 root root  254 Nov 29 22:27 junk-mail.svbin

rlaager@imap2:/var/lib/dovecot/sieve$ ls -la
/usr/share/wiktel-server-mail-backend/junk-mail.sieve
-rw-r--r-- 1 root root 124 Oct 31 09:34
/usr/share/wiktel-server-mail-backend/junk-mail.sieve

Note that the .svbin is from November, while the text version is from
October. Even if something is looking at the date of the symlink, that's
from September.

So the first question is... why is Dovecot trying to write the binary file?

I dug into the Pigeonhole code... I think, but am certainly not sure,
that lda_sieve_open() in
pigeonhole/src/plugins/lda-sieve/lda-sieve-plugin.c is the relevant
function calling lda_sieve_binary_save(). At the end of the function, it
has:

if (!recompile)
lda_sieve_binary_save(srctx, sbin, script);

This seems odd to me. Why is it trying to save in the "!recompile" case?
It seems like it should be saving in the "recompile" case. If I'm
reading this code right, recompile is set when it loads a corrupt sieve
binary script and needs to recompile from text.

I could be completely off, though.

Any thoughts?

-- 
Richard

RE: First steps in Dovecot; IMAP not working

[Richard]

> Date: Tuesday, October 18, 2016 13:12:39 +0200
> From: Moi 
>
> Thank you. It worked and I now have several log files to check.
> 
> In the meantime, I've tried once again to send a message to
> "mailtest" from an outside address; this time, I got an error reply:
> 
> This report relates to a message you sent with the following header
> fields:
> 
>   Message-id: <00e201d2290f$67da8c70$378fa550$@mac.com>
>   Date: Tue, 18 Oct 2016 09:15:21 +0200
>   From: Moi
>   To: 'Mail Test' 
>   Subject: Test
> 
> Your message cannot be delivered to the following recipients:
> 
>   Recipient address: mailt...@barbu.sytes.net
>   Reason: Illegal host/domain name found
> 
> 
> Yet another area with a problem; at least this is now a valid
> reason for it to not work.
> Is this a misconfiguration of my DNS server?

Assuming that "barbu.sytes.net" is the intended hostname (not
something made up to obscure the real name), there is an MX-record
for that that points to "mail.barbu.sytes.net", but there is no
A-record for the "mail." hostname. There is an A-record for
"mail.sytes.net", in case that is what you were intending, in which
case you'd need to fix the MX on "barbu.sytes.net".

Re: Dsync Header Hashing

[Richard Laager]

On 04/12/2016 04:05 PM, Timo Sirainen wrote:

I added it today: 
https://github.com/dovecot/core/commit/03af8e5325a7b4fec36414ac35949457bc426c0b


Cool. And thanks for the awesome software!

--
Richard

Re: Dsync Header Hashing

[Richard Laager]

We've completed our migration to Dovecot (yay!), so this isn't critical 
for me any more. But this change might still be a useful addition to 
Dovecot. It doesn't create any non-standard behavior (like my patch for 
non-atom flags).


On 03/07/2016 11:16 PM, Richard Laager wrote:

On 03/04/2016 08:52 AM, Timo Sirainen wrote:

On 04 Mar 2016, at 07:47, Richard Laager  wrote:

Is there any way to disable the header hashing in dsync?

...

Does the attached patch happen to work? Compiles, but untested for now.


It works with one more change on top of your patch:

diff --git a/src/doveadm/dsync/dsync-mailbox-export.c 
b/src/doveadm/dsync/dsync-mailbox-export.c
index 361cc55..0267f86 100644
--- a/src/doveadm/dsync/dsync-mailbox-export.c
+++ b/src/doveadm/dsync/dsync-mailbox-export.c
@@ -518,7 +518,7 @@ dsync_mailbox_export_init(struct mailbox *box,
 p_array_init(&exporter->expunged_seqs, pool, 16);
 p_array_init(&exporter->expunged_guids, pool, 16);

-   if (!exporter->mails_have_guids)
+   if (!exporter->mails_have_guids && !exporter->no_hdr_hashes)
 exporter->wanted_headers = dsync_mail_get_hash_headers(box);

 /* first scan transaction log and save any expunges and flag changes */



--
Richard
diff --git a/src/doveadm/dsync/dsync-brain-mailbox.c b/src/doveadm/dsync/dsync-brain-mailbox.c
index 5dadf97..f71c7aa 100644
--- a/src/doveadm/dsync/dsync-brain-mailbox.c
+++ b/src/doveadm/dsync/dsync-brain-mailbox.c
@@ -322,6 +322,12 @@ int dsync_brain_sync_mailbox_open(struct dsync_brain *brain,
 		exporter_flags |= DSYNC_MAILBOX_EXPORTER_FLAG_TIMESTAMPS;
 	if (brain->hdr_hash_v2)
 		exporter_flags |= DSYNC_MAILBOX_EXPORTER_FLAG_HDR_HASH_V2;
+	if (remote_dsync_box->messages_count == 0) {
+		/* remote mailbox is empty - we don't really need to export
+		   header hashes since they're not going to match anything
+		   anyway. */
+		exporter_flags |= DSYNC_MAILBOX_EXPORTER_FLAG_NO_HDR_HASHES;
+	}
 
 	brain->box_exporter = brain->backup_recv ? NULL :
 		dsync_mailbox_export_init(brain->box, brain->log_scan,
diff --git a/src/doveadm/dsync/dsync-mailbox-export.c b/src/doveadm/dsync/dsync-mailbox-export.c
index c013eb0..361cc55 100644
--- a/src/doveadm/dsync/dsync-mailbox-export.c
+++ b/src/doveadm/dsync/dsync-mailbox-export.c
@@ -63,6 +63,7 @@ struct dsync_mailbox_exporter {
 	unsigned int minimal_dmail_fill:1;
 	unsigned int return_all_mails:1;
 	unsigned int export_received_timestamps:1;
+	unsigned int no_hdr_hashes:1;
 };
 
 static int dsync_mail_error(struct dsync_mailbox_exporter *exporter,
@@ -163,6 +164,10 @@ exporter_get_guids(struct dsync_mailbox_exporter *exporter,
 
 	if (!exporter->mails_have_guids) {
 		/* get header hash also */
+		if (exporter->no_hdr_hashes) {
+			*hdr_hash_r = "";
+			return 1;
+		}
 		if (dsync_mail_get_hdr_hash(mail, exporter->hdr_hash_version, hdr_hash_r) < 0)
 			return dsync_mail_error(exporter, mail, "hdr-stream");
 		return 1;
@@ -505,6 +510,8 @@ dsync_mailbox_export_init(struct mailbox *box,
 		(flags & DSYNC_MAILBOX_EXPORTER_FLAG_TIMESTAMPS) != 0;
 	exporter->hdr_hash_version =
 		(flags & DSYNC_MAILBOX_EXPORTER_FLAG_HDR_HASH_V2) ? 2 : 1;
+	exporter->no_hdr_hashes =
+		(flags & DSYNC_MAILBOX_EXPORTER_FLAG_NO_HDR_HASHES) != 0;
 	p_array_init(&exporter->requested_uids, pool, 16);
 	p_array_init(&exporter->search_uids, pool, 16);
 	hash_table_create(&exporter->export_guids, pool, 0, str_hash, strcmp);
diff --git a/src/doveadm/dsync/dsync-mailbox-export.h b/src/doveadm/dsync/dsync-mailbox-export.h
index c8f9548..02c6aa9 100644
--- a/src/doveadm/dsync/dsync-mailbox-export.h
+++ b/src/doveadm/dsync/dsync-mailbox-export.h
@@ -6,7 +6,8 @@ enum dsync_mailbox_exporter_flags {
 	DSYNC_MAILBOX_EXPORTER_FLAG_MAILS_HAVE_GUIDS	= 0x02,
 	DSYNC_MAILBOX_EXPORTER_FLAG_MINIMAL_DMAIL_FILL	= 0x04,
 	DSYNC_MAILBOX_EXPORTER_FLAG_TIMESTAMPS		= 0x08,
-	DSYNC_MAILBOX_EXPORTER_FLAG_HDR_HASH_V2		= 0x10
+	DSYNC_MAILBOX_EXPORTER_FLAG_HDR_HASH_V2		= 0x10,
+	DSYNC_MAILBOX_EXPORTER_FLAG_NO_HDR_HASHES	= 0x20
 };
 
 struct dsync_mailbox_exporter *

[PATCH] imapc: Accept strings in FLAGS responses

[Richard Laager]

While non-standard, the IMAP server we are replacing returns non-system
flags as strings instead of atoms.

Prior to this change, imapc would abort processing on the first message
with a string flag.
---
 src/lib-storage/index/imapc/imapc-mailbox.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/lib-storage/index/imapc/imapc-mailbox.c 
b/src/lib-storage/index/imapc/imapc-mailbox.c
index c3e12d1..91333dc 100644
--- a/src/lib-storage/index/imapc/imapc-mailbox.c
+++ b/src/lib-storage/index/imapc/imapc-mailbox.c
@@ -310,7 +310,8 @@ static void imapc_untagged_fetch(const struct 
imapc_untagged_reply *reply,
t_array_init(&keywords, 8);
seen_flags = TRUE;
for (j = 0; flags_list[j].type != IMAP_ARG_EOL; j++) {
-   if (!imap_arg_get_atom(&flags_list[j], &atom))
+   if (!imap_arg_get_atom(&flags_list[j], &atom) &&
+   !imap_arg_get_string(&flags_list[j], &atom))
return;
if (atom[0] == '\\')
flags |= imap_parse_system_flag(atom);
-- 
2.1.4

Re: Dsync Header Hashing

[Richard Laager]

On 03/04/2016 08:52 AM, Timo Sirainen wrote:
> On 04 Mar 2016, at 07:47, Richard Laager  wrote:
>> Is there any way to disable the header hashing in dsync?
...
> Does the attached patch happen to work? Compiles, but untested for now.

It works with one more change on top of your patch:

diff --git a/src/doveadm/dsync/dsync-mailbox-export.c 
b/src/doveadm/dsync/dsync-mailbox-export.c
index 361cc55..0267f86 100644
--- a/src/doveadm/dsync/dsync-mailbox-export.c
+++ b/src/doveadm/dsync/dsync-mailbox-export.c
@@ -518,7 +518,7 @@ dsync_mailbox_export_init(struct mailbox *box,
p_array_init(&exporter->expunged_seqs, pool, 16);
p_array_init(&exporter->expunged_guids, pool, 16);
 
-   if (!exporter->mails_have_guids)
+   if (!exporter->mails_have_guids && !exporter->no_hdr_hashes)
exporter->wanted_headers = dsync_mail_get_hash_headers(box);
 
/* first scan transaction log and save any expunges and flag changes */

-- 
Richard

Dsync Header Hashing

[Richard Laager]

Is there any way to disable the header hashing in dsync?

I'm doing a one-time migration to Dovecot using imapc. The FETCHes for 
Date & Message-ID take a non-trivial amount of time and it's not clear 
to me if they have a function for a one-time migration.


--
Richard

Re: dovecot Digest, Vol 154, Issue 14

[Richard Platel]

Thanks, Hajo

This mostly works, but we can’t seem to send an arbitrary mailbox name with 
UserDB, we have to initialize it in the config:

"namespace/inbox/mailbox=Junk namespace/inbox/mailbox/Junk/name"="Spam"
seems to initialize the folder somehow so that the subsequent
"namespace/inbox/mailbox/Junk/auto"="subscribe"
and
"namespace/inbox/mailbox/Junk/special_use"="\Junk"
have something to act upon.

(the first line doesn't work quite properly, however - it ends up
causing a folder named "" to show up)

we've noticed that if we set a very minimal config for the mailbox in
the main config (that wouldn't cause the mailbox to be autocreated),
that this also initializes the folder name allowing for subsequent auto
and subscribe directives to be processed for it:
namespace inbox {
 inbox = yes
 mailbox "Spam" {
   auto = no
 }
}

is there some userdb response that will simply initialize the folder
(without doing any name remapping) in the same way that the above config
lines do?

we have tried a few things based off Hajo's first line, but nothing we
guess at seems to do the trick. ie:
"namespace/inbox/mailbox"="Spam"
"namespace/inbox/mailbox=Spam"=“Spam"

> 
> --
> 
> Message: 2
> Date: Tue, 9 Feb 2016 23:33:53 +0100
> From: Hajo Locke 
> To: dovecot@dovecot.org
> Subject: Re: Per-user special folder?
> Message-ID: <56ba6951.8010...@gmx.de>
> Content-Type: text/plain; charset=windows-1252; format=flowed
> 
> Hello,
> 
> Am 09.02.2016 um 22:28 schrieb Richard Platel:
>> Hi
>> 
>> It's possible to mark some folders as special use for IMAP in the config 
>> like:
>> 
>> namespace inbox {
>>   mailbox Spam {
>> special_use = \Junk
>>   }
>> }
>> 
>> 
>> Our webmail allows users to use an arbitrary folder for Spam, and we have 
>> this settings and we'd like to return it in from our UserDB (which is a 
>> custom dict proxy).
>> 
>> For testing were able to set a namespace parameter like "separator" by 
>> returning:
>> "namespace/inbox/separator" : "=",
>> 
>> from UserDB, but can't figure out a way to set mailbox settings,
>> 
>> "namespace/inbox/mailbox Spam/special_use" : "\Junk", for example doesn't 
>> work.
>> 
>> Is there a way to return this setting from a UserDB query?
>> 
> we use this a lot with userdb to allow individual folders marked as special.
> your userbd-query should return something like this:
> 
> namespace/inbox/mailbox=Junk namespace/inbox/mailbox/Junk/name=Spam 
> namespace/inbox/mailbox/Junk/auto=subscribe 
> namespace/inbox/mailbox/Junk/special_use=\Junk
> Spam is visible name in this case.
> 
> Hajo

Per-user special folder?

[Richard Platel]

Hi

It's possible to mark some folders as special use for IMAP in the config like:

namespace inbox {
  mailbox Spam {
special_use = \Junk
  }
}


Our webmail allows users to use an arbitrary folder for Spam, and we have this 
settings and we'd like to return it in from our UserDB (which is a custom dict 
proxy).

For testing were able to set a namespace parameter like "separator" by 
returning:
"namespace/inbox/separator" : "=",

from UserDB, but can't figure out a way to set mailbox settings,

"namespace/inbox/mailbox Spam/special_use" : "\Junk", for example doesn't work.

Is there a way to return this setting from a UserDB query?

Re: Mail to nowhere

[Richard]

> Date: Tuesday, June 16, 2015 07:21:06 PM -0400
> From: Steve Matzura 
>
> On Tue, 16 Jun 2015 22:02:29 +0200, you wrote:
> 
>> Am Dienstag, den 16.06.2015, 11:44 -0400 schrieb Steve Matzura:
>>> The next phase of my testing procedure involves the simple act of
>>> delivering mail to my test box. When I send a message to either a
>>> valid or relayed user at my remote machine's address, it never
>>> gets there. I know this virtually for sure because I'm tailing
>>> /var/log/maillog and nothing new has been added for the past
>>> hour.  I'm
>>> thinking eventually my ISP will send the message back to me as
>>> undeliverable, and in that bounced message there may be some
>>> information about why it was undeliverable, but maybe someone
>>> has a thought as to why I'm not even seeing anything attempt to
>>> connect?
>> 
>> Receiving and Sending Mail is done by the MTA like postfix or
>> exim. Not Dovecot. It's just for the IMAP/POP access by the users
>> to get the mails.
>> So look at that logs. And if there's nothing in it, then make
>> sure port 25 is actually open by the postfix/exim process.
> 
> Well, it isn't. In fact, there is no exim process running. However,
> something calling itself 'master' has port 25 open according to
> netstat, and that process has a subprocess running something called
> qmgr. I'd give more information, such as path to these programs,
> but it isn't shown.
> Time to check my Postfix config a little more closely.

Those pieces are all part of the postfix MTA (you haven't indicated
your OS, but postfix is the current default on most current linux
distributions).

As I indicated earlier, by default most (current) MTA installs only
listen on localhost, not on the external interfaces, so don't accept
off-host mail.

There is extensive postfix documentation and also a postfix mailing
list if you need assistance.

Re: Mail to nowhere

[Richard]

> Date: Tuesday, June 16, 2015 07:05:43 PM -0400
> From: Steve Matzura 
>
> On Tue, 16 Jun 2015 17:31:07 +, you wrote:
> 
>> By default, most MTAs only listen to localhost, so don't accept
>> externally derived mail. Try telnetting to port 25 on the machine
>> from some place off your machine's network. If you get a "hang",
>> it's likely a firewall issue. If you get a "connection refused"
>> message it's probably that the MTA isn't listening on the external
>> interface.
>> 
>> If the MTA responds, do a manual delivery exchange and see what
>> you get.
>> 
>> If you're not seeing anything in your maillog, this is very likely
>> an issue with your MTA's configuration, not dovecot.
> 
> Now that I think about and concentrate on what you wrote, make
> perfect sense. Add to that the fact that I received bounces from
> my ISP that all said the same thing: Unable to establish an SMTP
> connection. The return code was 4.5.1.

A 4xx indicates a transient error (i.e., within configured limits,
delivery retries will be attempted), 5xx is permanent (immediate
reject, no retries), otherwise the specific number isn't all that
telling.

>From everything you've indicated, this is an MTA issue.


 

Re: Mail to nowhere

[Richard]

> Date: Tuesday, June 16, 2015 01:13:47 PM -0400
> From: Steve Matzura 
>
> On Tue, 16 Jun 2015 11:27:58 -0500, you wrote:
> 
>> No attempt to deliver almost always means either:
>> - a DNS problem; the sender can't find the destination, or finds
>> the "wrong" destination.
> 
> That would be interesting, since I tried both the actual IP address
> and the DNS name for the test node. Neither message got through.
> 
>> - a connectivity problem; the sender can't connect to the
>> destination.  Possibly a firewall not open.
> 
> Now there's a possibility, but I am not aware of running any
> firewall on the remote machine.


By default, most MTAs only listen to localhost, so don't accept
externally derived mail. Try telnetting to port 25 on the machine
from some place off your machine's network. If you get a "hang",
it's likely a firewall issue. If you get a "connection refused"
message it's probably that the MTA isn't listening on the external
interface.

If the MTA responds, do a manual delivery exchange and see what you
get.

If you're not seeing anything in your maillog, this is very likely
an issue with your MTA's configuration, not dovecot.

Re: Multiple servers and NFS

[Richard Hector]

On 25/07/14 15:30, Eduardo Ramos wrote:
> Hi Richard,
> 
> In fact I thought it a little confusing. I had some bad experience with
> DNS RR when one of my IMAP server got down. Clients continued trying
> connect to broken server and it caused some problems. But when
> everything is ok, it works well.
> 
> I drew a diagram with my idea. What do you think?
> 
> https://dl.dropboxusercontent.com/u/41373531/mail.png

Interesting, thanks. I'd forgotten to draw in the director ring.

As I said, if we need load balancing we can do that on the router, which
as I understand it will do more or less the same thing as LVS. It might
be Cisco SLB, but I'm not sure; I'm not the router guy :-)

But what interests me most is that your diagram shows the mx servers
connecting directly to the backend servers, rather than going through
the proxy director - I thought that was a no-no. Oh, and I don't think
we want to load down our front-end MX servers with amavis, either.

Thanks for your input :-)

Richard

Re: Multiple servers and NFS

[Richard Hector]

On 25/07/14 09:12, Richard Hector wrote:

Rather than trying to draw increasingly complex diagrams in ASCII, I've
put some here (without the LVS layer):
https://walnut.gen.nz/mail-architectures.png


I've come up with a revised plan - I think we can do without LVS; SMTP 
should just work with multiple MX records, and IMAP/POP should be fine 
as well with RRDNS - the machines should be up most of the time, and if 
a customer has to click to reconnect every now and then on the rare 
occasions when they're not it's not a huge deal. Otherwise, we could 
also do load balancing on our routers.


Anyway - any comments on the sanity of this diagram most welcome :-)

https://walnut.gen.nz/mail-architecture-2.png

Richard

Re: Multiple servers and NFS

[Richard Hector]

On 25/07/14 00:01, Eduardo Ramos wrote:

You can use one or more instances of Dovecot on the same machine, as you
can see here (http://wiki2.dovecot.org/RunningDovecot) "Running Multiple
Invocations of Dovecot".

The problem with DNS round-robind is that if you server goes down, DNS
continues resolving for it. I would recommend use some balancer like
LVS+keepalived.

Consider that multi layer solution:

| LVS + keepalived || LVS + keepalived |

  ||
   --   --
   | Director 1 |   | Director 2 |
   --   --
  ||
  -   -
  | IMAP/POP/LMTP |   | IMAP/POP/LMTP |
  |Backend|   |Backend|
  -   -
  \/
   \  /
\    /
 ---|  NetAPP  |-



Thanks.

Presumably each LVS (in VRRP setup?) has to talk to both directors, and 
the directors each have to talk to both backends. ASCII art is tricky :-)


I accept that I could run multiple dovecots on the same machine, true. 
And keepalived/LVS is a good plan, thanks.


The key point I wanted to confirm is that I need to run the lmtpds on 
the same set of backend machines as the imapd/popds, and behind the same 
directors, so that all sessions relating to the same user can be 
directed to the same backend. Correct?


Rather than trying to draw increasingly complex diagrams in ASCII, I've 
put some here (without the LVS layer): 
https://walnut.gen.nz/mail-architectures.png


I suspect that A is what I need, though the docs suggest that if I turn 
off writing of index files in lmtp, I could get away with one of the 
others, right? What disadvantages are there in that? One concern is the 
ability to scale up to more servers for some particular parts of the 
chain as load dictates - we're concerned that amavis might be a 
significant candidate. I assume amavis could go either in front of or 
behind the director.


Thanks,
Richard

Re: Maildir migration - using separate INDEX and CONTROL directories?

[Richard Hector]

On 24/07/14 18:18, Steffen Kaiser wrote:


On Thu, 24 Jul 2014, Richard Hector wrote:




Is there a handy tool to move the files to the right places? Or is
there a complete list of what files need to go where?


IMHO: move all dovecot* files the migration script generates and the
"subscriptions" file into the control directories, some INBOX-related
files probably go to /.INBOX, dovecot-keywords and
dovecot-uidlist - maybe more. The migration scripts I saw til today
never create any indexes, so the assumption should be fine. And test it
with one account, move some messages around, add some subscriptions and
ACLs, ...


Thanks - seems straightforward enough.

Richard

Multiple servers and NFS

[Richard Hector]

Hi all,

For some reason, I didn't go to http://wiki2.dovecot.org/NFS until now, 
and I'm starting to get worried ...


The plan was to have multple servers (MXes) receiving mail, and 
delivering via LMTP to multiple backend dovecot servers (with amavis in 
front of dovecot; LMTP both sides). Then we'd have multiple servers for 
clients to use IMAP or POP3.


This is more or less how the system already works, except with Courier 
IMAP, and postfix on the backends, delivering to maildirs with procmail.


But with the recommendation to use the Director for both IMAP/POP3 and 
LMTP - that starts to sound like I need a whole bunch more servers to 
run Directors and proxies, and even then it might not be a good idea to 
have different servers running lmtp and imap/pop.


One possible mitigating point is that our 'load balancing' is DNS 
round-robin, so a given client will probably stick with a single 
imap/pop server anyway, but if the user has multiple clients 
(desktop/mobile etc) then they may still hit different servers.


Can someone clarify best practice for a setup needing multiple servers 
for load balancing and redundancy?


Is Courier already likely to have been suffering these problems?

Oh, the NFS server is a NetApp Filer, if that matters.

I'm using dovecot 2.2.9 from debian wheezy backports, in order to get 
the quota policy daemon support.


Thanks,
Richard

Maildir migration - using separate INDEX and CONTROL directories?

[Richard Hector]

Hi all,

I'm working on a migration from Courier to Dovecot, and the 
courier-dovecot-migrate.pl seems to work ok, but it puts all the 
metadata files (dovecot-uidlist etc) in the Maildir, while I want to put 
them in the appropriate places as specified:


mail_location = 
maildir:~/Maildir:INDEX=/var/mail/meta/index/%d/%1n/%1.1n/%n:CONTROL=/var/mail/meta/control/%d/%1n/%1.1n/%n


http://wiki2.dovecot.org/Quota/FS explains why some of these files need 
to go in different places (which is why I'm doing it), but I haven't 
seen a complete list of what files will be moved as a result of those 
directives.


Is there a handy tool to move the files to the right places? Or is there 
a complete list of what files need to go where?


Thanks,
Richard

[Dovecot] maildir compressed message fix patch

[Richard Platel]

When a compressed maildir message has a bad S= size in its filename it puts the 
user in an unrecoverable state, since maildir's do_fix_size function just does 
a stat() on the maildir file and saves the compressed size in the filename.

This (quick, rough, barely tested) patch addresses this issue, it's 
inefficient, but we're already in a hopefully rare emergency situation.

--- maildir-mail.c  2014-02-11 22:23:37.0 +
+++ maildir-mail.c.new  2014-04-24 20:41:25.0 +
@@ -8,6 +8,7 @@
 #include "maildir-filename.h"
 #include "maildir-uidlist.h"
 #include "maildir-sync.h"
+#include "compression.h"

 #include 
 #include 
@@ -640,6 +641,10 @@
 {
const char *fname, *newpath, *extra, *info, *dir;
struct stat st;
+  const struct stat * stp;
+  const struct compression_handler * handler;
+  struct istream * fstream;
+  struct istream * cstream;

fname = strrchr(path, '/');
i_assert(fname != NULL);
@@ -650,13 +655,29 @@
info = strchr(fname, MAILDIR_INFO_SEP);
if (info == NULL) info = "";

+  fstream = i_stream_create_file(path, 1024);
+  handler = compression_detect_handler(fstream);
+  if (handler != NULL && handler->create_istream != NULL)
+  {
+cstream = handler->create_istream(fstream, TRUE);
+if (i_stream_stat(cstream, TRUE, &stp) < 0)
+{
+  return -1;
+}
+st = *stp; /* dumb copy */
+i_stream_unref(&cstream);
+  }
+  else
+  {
if (stat(path, &st) < 0) {
if (errno == ENOENT)
return 0;
mail_storage_set_critical(&mbox->storage->storage,
  "stat(%s) failed: %m", path);
return -1;
-   }
+ }
+  }
+  i_stream_unref(&fstream);

newpath = t_strdup_printf("%s/%s,S=%"PRIuUOFF_T"%s", dir,
  t_strdup_until(fname, extra),

[Dovecot] zlib maildir reindex broken

[Richard Platel]

I posted to the list about this a while ago but never got a response, I have a 
bit more information now.

Dovecot 2.2.12 and other 2.2 versions are broken when using zlib and maildir.  
If messages are re-indexed, the INTERNALDATE of all messages is set to the time 
the re-index is done.
  
The problem seems to be in src/plugins/zlib/zlib-plugin.c in the function 
zlib_mail_cache_open.

During a reindex maildir_mail_get_received_date() does an i_stream_stat on the 
the i_stream_seekable stream created in zlib_mail_cache_open, but this istream 
does not know about the original maildir message file and always returns the 
current time for the file's stat times.

This is also broken on initial index, but if mail is indexed when it's received 
(ours isn't) it coincidentally gets the right time.

Re: [Dovecot] dovecot with maildir not using mtime on reindex

[Richard Platel]

Furthermore: it seems the behaviour is correct (mtime is used for internaldate) 
if the message is not compressed.

On Feb 20, 2014, at 12:09 PM, Richard Platel  wrote:

> Hi.
> 
> It seems that dovecot is using the current time, not a maildir file's mtime 
> for INTERNALDATE when a message is re-indexed:
> 
> $ cd Index
> $ rm -rf .INBOX
> $ cd ../Maildir/cur
> $ stat *
>  File: `1392914632.P54451Q0M08633.smtpin01,S=2215,W=2249:2,'
>  Size: 960 Blocks: 8  IO Block: 1048576 regular file
> Device: 36h/54d Inode: 11132959Links: 1
> Access: (0600/-rw---)  Uid: (8/mail)   Gid: (8/mail)
> Access: 2012-01-01 00:00:00.0 +
> Modify: 2012-01-01 00:00:00.0 +
> Change: 2014-02-20 16:46:20.0 +
> Birth: -
> $ telnet imap01 143
> Trying 10.5.45.1...
> Connected to imap01.dev.firefly.tucows.com.
> Escape character is '^]'.
> * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE 
> AUTH=PLAIN] Dovecot ready.
> A LOGIN rpla...@ff-dev.com 
> A OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE 
> SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT 
> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS 
> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN 
> CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA] Logged in
> A SELECT INBOX
> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags 
> permitted.
> * 1 EXISTS
> * 0 RECENT
> * OK [UNSEEN 1] First unseen.
> * OK [UIDVALIDITY 1265835133] UIDs valid
> * OK [UIDNEXT 4548] Predicted next UID
> A OK [READ-WRITE] Select completed (0.035 secs).
> A FETCH 1:* FULL
> * 1 FETCH (FLAGS () INTERNALDATE "20-Feb-2014 16:59:51 +" RFC822.SIZE 
> 2249 ENVELOPE ("Thu, 20 Feb 2014 11:43:50 -0500" "Test message" (("Richard 
> Platel" NIL "rplatel" "tucows.com")) (("Richard Platel" NIL "rplatel" 
> "tucows.com")) (("Richard Platel" NIL "rplatel" "tucows.com")) ((NIL NIL 
> "rplatel" "ff-dev.com")) NIL NIL NIL 
> "") BODY ("text" "plain" 
> ("charset" "us-ascii") NIL NIL "7bit" 23 4))
> A OK Fetch completed.
> A LOGOUT
> * BYE Logging out
> A OK Logout completed.
> Connection closed by foreign host.
> $ date
> Thu Feb 20 16:59:58 UTC 2014
> 
> Stracing the imap process, it seems dovecot does not stat the message file at 
> all.  Performing the above with an old dovecot 1 server yields the expected 
> result, the INTERNALDATE of the message is the file's mtime.
> 
> $ dovecot -n -c /he/dovecot/conf/dovecot.conf
> # 2.2.10.3: /he/dovecot/conf/dovecot.conf
> # OS: Linux 3.4.46-dom0-2.0.0 x86_64 Debian 7.0
> debug_log_path = syslog
> disable_plaintext_auth = no
> first_valid_uid = 8
> info_log_path = syslog
> lock_method = dotlock
> log_timestamp =
> mail_fsync = always
> mail_gid = mail
> mail_nfs_index = yes
> mail_nfs_storage = yes
> mail_plugins = zlib quota tc_mail_log notify tc_proc
> mail_temp_dir = /var/run/dovecot_tmp
> mail_uid = mail
> maildir_very_dirty_syncs = yes
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date ihave
> mmap_disable = yes
> namespace inbox {
>  inbox = yes
>  location =
>  prefix =
> }
> passdb {
>  args = host=localhost port=1143 
> username=%L{user}::%L{service}::%L{rip}::%L{session}
>  driver = imap
> }
> plugin {
>  antispam_backend = pipe
>  antispam_debug_target = syslog
>  antispam_pipe_program = /he/dovecot/utils/he_spamtrain.pl
>  antispam_pipe_program_args = --user=%u
>  antispam_pipe_program_notspam_arg = --falsepositive
>  antispam_pipe_program_spam_arg = --missed
>  antispam_pipe_tmpdir = /var/run/dovecot_as_tmp
>  antispam_signature_missing = move
>  antispam_spam = Spam;Inbox.Spam;INBOX.Spam;Junk;INBOX.Junk
>  antispam_trash_pattern_ignorecase = trash
>  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename 
> flag_change append
>  mail_log_fields = uid box msgid flags hetag
>  memcached_servers = 10.5.47.223,10.5.47.222
>  zlib_save = gz
>  zlib_save_level = 6
> }
> protocols = imap pop3
> service anvil {
>  unix_listener anvil-auth-penalty {
>mode = 00
>  }
> }
> service imap-l

[Dovecot] dovecot with maildir not using mtime on reindex

[Richard Platel]

Hi.

It seems that dovecot is using the current time, not a maildir file's mtime for 
INTERNALDATE when a message is re-indexed:

$ cd Index
$ rm -rf .INBOX
$ cd ../Maildir/cur
$ stat *
  File: `1392914632.P54451Q0M08633.smtpin01,S=2215,W=2249:2,'
  Size: 960 Blocks: 8  IO Block: 1048576 regular file
Device: 36h/54d Inode: 11132959Links: 1
Access: (0600/-rw---)  Uid: (8/mail)   Gid: (8/mail)
Access: 2012-01-01 00:00:00.0 +
Modify: 2012-01-01 00:00:00.0 +
Change: 2014-02-20 16:46:20.0 +
 Birth: -
$ telnet imap01 143
Trying 10.5.45.1...
Connected to imap01.dev.firefly.tucows.com.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE 
AUTH=PLAIN] Dovecot ready.
A LOGIN rpla...@ff-dev.com 
A OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT 
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND 
URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED 
I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH 
LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA] Logged in
A SELECT INBOX
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags 
permitted.
* 1 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1265835133] UIDs valid
* OK [UIDNEXT 4548] Predicted next UID
A OK [READ-WRITE] Select completed (0.035 secs).
A FETCH 1:* FULL
* 1 FETCH (FLAGS () INTERNALDATE "20-Feb-2014 16:59:51 +" RFC822.SIZE 2249 
ENVELOPE ("Thu, 20 Feb 2014 11:43:50 -0500" "Test message" (("Richard Platel" 
NIL "rplatel" "tucows.com")) (("Richard Platel" NIL "rplatel" "tucows.com")) 
(("Richard Platel" NIL "rplatel" "tucows.com")) ((NIL NIL "rplatel" 
"ff-dev.com")) NIL NIL NIL "") 
BODY ("text" "plain" ("charset" "us-ascii") NIL NIL "7bit" 23 4))
A OK Fetch completed.
A LOGOUT
* BYE Logging out
A OK Logout completed.
Connection closed by foreign host.
$ date
Thu Feb 20 16:59:58 UTC 2014

Stracing the imap process, it seems dovecot does not stat the message file at 
all.  Performing the above with an old dovecot 1 server yields the expected 
result, the INTERNALDATE of the message is the file's mtime.

$ dovecot -n -c /he/dovecot/conf/dovecot.conf
# 2.2.10.3: /he/dovecot/conf/dovecot.conf
# OS: Linux 3.4.46-dom0-2.0.0 x86_64 Debian 7.0
debug_log_path = syslog
disable_plaintext_auth = no
first_valid_uid = 8
info_log_path = syslog
lock_method = dotlock
log_timestamp =
mail_fsync = always
mail_gid = mail
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = zlib quota tc_mail_log notify tc_proc
mail_temp_dir = /var/run/dovecot_tmp
mail_uid = mail
maildir_very_dirty_syncs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
mmap_disable = yes
namespace inbox {
  inbox = yes
  location =
  prefix =
}
passdb {
  args = host=localhost port=1143 
username=%L{user}::%L{service}::%L{rip}::%L{session}
  driver = imap
}
plugin {
  antispam_backend = pipe
  antispam_debug_target = syslog
  antispam_pipe_program = /he/dovecot/utils/he_spamtrain.pl
  antispam_pipe_program_args = --user=%u
  antispam_pipe_program_notspam_arg = --falsepositive
  antispam_pipe_program_spam_arg = --missed
  antispam_pipe_tmpdir = /var/run/dovecot_as_tmp
  antispam_signature_missing = move
  antispam_spam = Spam;Inbox.Spam;INBOX.Spam;Junk;INBOX.Junk
  antispam_trash_pattern_ignorecase = trash
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename 
flag_change append
  mail_log_fields = uid box msgid flags hetag
  memcached_servers = 10.5.47.223,10.5.47.222
  zlib_save = gz
  zlib_save_level = 6
}
protocols = imap pop3
service anvil {
  unix_listener anvil-auth-penalty {
mode = 00
  }
}
service imap-login {
  inet_listener imap {
address = 0
  }
  inet_listener imaps {
port = 0
  }
  process_limit = 29
  process_min_avail = 14
  service_count = 0
}
service imap-postlogin {
  executable = script-login -d /he/dovecot/utils/post_login.sh
}
service imap {
  executable = imap imap-postlogin
  process_limit = 1270
  vsz_limit = 0
}
service pop3-login {
  inet_listener pop3 {
address = 0
  }
  inet_listener pop3s {
port = 0
  }
  process_limit = 29
  process_min_avail = 14
  service_count = 0
}
service pop3-postlogin {
  executable = script-login -d /he/dovecot/utils/post_login.sh
}
service pop3 {
  executable = pop3 pop3-postlogin
  process_limit = 206
  vsz_limit = 512 M
}
ssl = no
userdb {
  args = /

[Dovecot] EAGAIN in dict proxy

[Richard Platel]

Hello,

We’re using a custom program to manage quotas talking to dovecot via the 
dovecot dict proxy protocol over a unix socket:

plugin {
  quota = dict:User quota::proxy:/var/run/auth_proxy_dovecot/quotasocket:quota
}

Dovecot gets slammed with quota requests periodically, seemingly because 
Thunderbird thought it would be a good idea to hardcode having a quota check at 
45 seconds past the minute, and thus every Thunderbird client makes several 
GETQUOTAROOT requests simultaneously.

This causes dovecot to make many client connections to the quota proxy, and 
many of them fail with EAGAIN.  In the log we see:

Jan 15 16:52:46 imap25 dovecot: imap(rpla...@tucows.com): Error: 
net_connect_unix(/var/run/auth_proxy_dovecot/quotasocket) failed: Resource 
temporarily unavailable

And the client gets:
* QUOTAROOT Spam "User quota"[0d][0a]
* QUOTA "User quota" ()[0d][0a]
* BAD Internal quota calculation error[0d][0a]
19 OK Getquotaroot completed.[0d][0a]

Thunderbird transparently disconnects and reconnects at this point but 
obviously this is not ideal.

Writing some toy programs, I found that even making a forked server with 
several processes doing nothing but accept()ing on the listening socket, with a 
high number for the listen queue, it’s easy to overwhelm it with simultaneous 
clients who then get EAGAIN.  If the clients do indeed immediately try again, 
they are successful. (An INET listening socket does not seem to have this 
problem, incidentally)

All of this is a long-winded way of saying that I believe in 
lib-dict/dict-client.c:client_dict_connect() the call to net_connect_unix 
should be a call to net_connect_unix_with_retries() with a small timeout.

It would also be useful for us if an INET socket could be used.

Re: [Dovecot] zlib config questions

[Richard Platel]

You only need to add the zlib plugin to mail_plugins once.

As far as I know, there's no indication in the logs that mail is being 
compressed, and the filename isn't modified to indicate that it's compressed, 
but, of course, the files are zlib data on disk.

One gotcha if you're using maildir is, if you're planning to compress old mail, 
make sure that the S= and W= sizes in the filenames are correct, otherwise 
dovecot errors and closes the client session, and it can't repair the filename 
on it's own.



On Dec 6, 2013, at 12:51 AM, dovecot-requ...@dovecot.org wrote:

> Message: 6
> Date: Thu, 5 Dec 2013 13:55:40 -0800
> From: Terry Barnum 
> To: Dovecot Mailing List 
> Subject: [Dovecot] zlib config questions
> Message-ID: 
> Content-Type: text/plain; charset=us-ascii
> 
> After nearly running out of space I swapped in larger disks and then saw the 
> recent threads about zlib compression. Unfortunately I'm still confused after 
> reading .
> 
> In order to compress new email being stored do I only need to change
> 
> 10-mail.conf to this:
> mail_plugins = $mail_plugins zlib
> 
> and 90-plugin.conf to this:
> plugin {
>  zlib_save_level = 6
>  zlib_save = gz
>  ...
> 
> Or do I need to instead (or also?) add it to the list of plugins in 
> 20-imap.conf?
> 
> A simple 'sudo doveadm reload' to enable?
> 
> Once enabled, is there an indication in the logs that compression is taking 
> place?
> 
> Do new mail files have a suffix like .Z to indicate they've been compressed?
> 
> Any gotchas to be aware of? 
> 
> I'm running macports dovecot 2.2.5.
> 
> Thanks for any help.
> 
> -Terry
> 
> Terry Barnum
> digital OutPost
> http://www.dop.com

Re: [Dovecot] dovecot-antispam plugin problem with multiple messages

[Richard Platel]

This seems to fix the issue

--- a/dovecot-antispam-plugin/src/antispam-storage-2.0.c
+++ b/dovecot-antispam-plugin/src/antispam-storage-2.0.c
@@ -91,15 +91,6 @@ antispam_copy(struct mail_save_context *ctx, struct mail *mai
int ret;
bool src_trash, dst_trash;

-   if (!ctx->dest_mail) {
-   /* always need mail */
-   if (!ast->mail)
-   ast->mail = mail_alloc(t, MAIL_FETCH_STREAM_HEADER |
- MAIL_FETCH_STREAM_BODY,
-  NULL);
-   ctx->dest_mail = ast->mail;
-   }
-
i_assert(mail->box);

asbox->save_hack = FALSE;
@@ -145,7 +136,7 @@ antispam_copy(struct mail_save_context *ctx, struct mail *ma
else
ret = asbox->cfg->backend->handle_mail(
asbox->cfg, t, ast->backendctx,
-   ctx->dest_mail,
+   mail,
move_to_class(asbox->movetype));

    /*



On Nov 27, 2013, at 4:17 PM, Richard Platel  wrote:

> Hi
> 
> With dovecot 2.2.5, and dovecot-antispam built from a recent HEAD pull, when 
> copying multiple messages to or from a Spam folder, the plugin sends multiple 
> copies of the first message to the backend.  I've tried this with the pipe 
> and spool2dir backends.
> 
> For example with the spool2dir backend, via IMAP doing
> 
> A COPY 1:3 Spam
> 
> yields 3 copies of message id 1 in the dir:
> dev:imap-8.1 rplatel@imap01:/var/run/dovecot_as_tmp$ sudo -u mail md5sum *
> 28ad0a215eb7ecbd3a814a8a334d85bf  
> 001385586164-rpla...@ff-dev.com-1s
> 28ad0a215eb7ecbd3a814a8a334d85bf  
> 001385586164-rpla...@ff-dev.com-2s
> 28ad0a215eb7ecbd3a814a8a334d85bf  
> 001385586164-rpla...@ff-dev.com-3s
> 
> I see the same behaviour with the pipe backend, the pipe program is invoked 3 
> times, but with the same message content.
> 
> 

[Dovecot] dovecot-antispam plugin problem with multiple messages

[Richard Platel]

Hi

With dovecot 2.2.5, and dovecot-antispam built from a recent HEAD pull, when 
copying multiple messages to or from a Spam folder, the plugin sends multiple 
copies of the first message to the backend.  I've tried this with the pipe and 
spool2dir backends.

For example with the spool2dir backend, via IMAP doing

A COPY 1:3 Spam

yields 3 copies of message id 1 in the dir:
dev:imap-8.1 rplatel@imap01:/var/run/dovecot_as_tmp$ sudo -u mail md5sum *
28ad0a215eb7ecbd3a814a8a334d85bf  001385586164-rpla...@ff-dev.com-1s
28ad0a215eb7ecbd3a814a8a334d85bf  001385586164-rpla...@ff-dev.com-2s
28ad0a215eb7ecbd3a814a8a334d85bf  001385586164-rpla...@ff-dev.com-3s

I see the same behaviour with the pipe backend, the pipe program is invoked 3 
times, but with the same message content.

Re: [Dovecot] fts-solr indexer-worker connects to wrong solr host dovecot-2.2.4

[Richard Platel]

Did some more digging.

The problem is that the fts-solr plugin has a global solr_conn pointer, that 
persists between users.  I think this patch fixes the problem:

--- a/dovecot/fts_solr_plugin/fts-solr-plugin.c
+++ b/dovecot/fts_solr_plugin/fts-solr-plugin.c
@@ -50,6 +50,13 @@ static void fts_solr_mail_user_create(struct mail_user 
*user, const char *env)
 {
struct fts_solr_user *fuser;

+   /** solr URL may be different per-user **/
+   if (solr_conn != NULL) {
+   solr_connection_deinit(solr_conn);
+   solr_conn = NULL;
+   }
+   /**/
+
fuser = p_new(user->pool, struct fts_solr_user, 1);
if (fts_solr_plugin_init_settings(user, &fuser->set, env) < 0) {
/* invalid settings, disabling */


On 2013-10-02, at 3:28 PM, Richard Platel  wrote:

> I've confirmed that this problem still exists in 2.2.5
> 
> It seems that indexer-worker only init's plugins at startup, so the fts_solr 
> plugin is holding the url= parameter from the first user.
> 
> The problem doesn't happen if the indexer-worker process is idle-killed 
> between users.  A new process starts up with the new user's userdb settings.
> 
> I thought I could work around this problem by adjusting indexer-worker's 
> settings:
> 
> service indexer-worker {
>  service_count = 1
>  idle_kill = 1
> }
> 
> but these changes don't seem to have any effect, the indexer-worker process 
> still hangs around idling after indexing a user, and isn't idle-killed for 
> upwards of a minute.
> 
> Any help?
> 
> 
> On 2013-09-27, at 11:46 AM, Richard Platel  wrote:
> 
>> Hello.  
>> We're setting up fts solr and want to have the solr server host be set 
>> per-user via UserDB.
>> 
>> It looks like if a user connects and fts indexes mail, and then another user 
>> connects and indexes mail, indexer-worker is connecting to the first user's 
>> fts host:
>> 
>> User1, ham...@rp-auth-test.com connects, does a SEARCH for the first time, 
>> indexer-worker gets UserDB settings and correctly indexes mail on ftsvs01:
>> 
>> [...]
>> auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): lookup 
>> shared/userdb/ham...@rp-auth-test.com
>> auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): result: 
>> {"uid":"8","fts":"solr","quota_rule4":"Spam:ignore","_session":"talk15_590ec6d100042","quota_rule3":"Trash:ignore","quota_rule2":"*:messages=2684354","quota_rule":"*:storage=5242880k","mail":"maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/","fts_solr":"debug
>>  url=http://ftsvs01:8080/solr/","gid":"8"}
>> auth: Debug: userdb out: USER   1   ham...@rp-auth-test.com uid=8   
>> fts=solrquota_rule4=Spam:ignore _session=talk15_590ec6d100042   
>> quota_rule3=Trash:ignorequota_rule2=*:messages=2684354  
>> quota_rule=*:storage=5242880k   
>> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
>> fts_solr=debug url=http://ftsvs01:8080/solr/gid=8
>> indexer-worker: Debug: auth input: ham...@rp-auth-test.com uid=8 fts=solr 
>> quota_rule4=Spam:ignore _session=talk15_590ec6d100042 
>> quota_rule3=Trash:ignore quota_rule2=*:messages=2684354 
>> quota_rule=*:storage=5242880k 
>> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
>>  fts_solr=debug url=http://ftsvs01:8080/solr/ gid=8
>> indexer-worker: Debug: Added userdb setting: 
>> plugin/_session=talk15_590ec6d100042
>> indexer-worker: Debug: Added userdb setting: plugin/fts=solr
>> indexer-worker: Debug: Added userdb setting: plugin/fts_solr=debug 
>> url=http://ftsvs01:8080/solr/
>> indexer-worker: Debug: Added userdb setting: 
>> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ha
>> m...@rp-auth-test.com/
>> indexer-worker: Debug: Added userdb setting: 
>> plugin/quota_rule=*:storage=5242880k
>> indexer-worker: Debug: Added userdb setting: 
>> plugin/quota_rule2=*:messages=2684354
>> indexer-worker: Debug: Added userdb setting: plugin/quota_rule3=Trash:ignore
>> indexer-worker: Debug: Added userdb setting: plugin/quota_rule4=Spam:ignore
>> indexer-worker(ham...@rp-auth-test.com): Debug: Effective uid=8, gid=8, home=
>> indexer-worker(ham...@rp-auth-test.com): Debug: Namespace inbox: 
>&g

Re: [Dovecot] fts-solr indexer-worker connects to wrong solr host dovecot-2.2.4

[Richard Platel]

I've confirmed that this problem still exists in 2.2.5

It seems that indexer-worker only init's plugins at startup, so the fts_solr 
plugin is holding the url= parameter from the first user.

The problem doesn't happen if the indexer-worker process is idle-killed between 
users.  A new process starts up with the new user's userdb settings.

I thought I could work around this problem by adjusting indexer-worker's 
settings:

service indexer-worker {
  service_count = 1
  idle_kill = 1
}

but these changes don't seem to have any effect, the indexer-worker process 
still hangs around idling after indexing a user, and isn't idle-killed for 
upwards of a minute.

Any help?


On 2013-09-27, at 11:46 AM, Richard Platel  wrote:

> Hello.  
> We're setting up fts solr and want to have the solr server host be set 
> per-user via UserDB.
> 
> It looks like if a user connects and fts indexes mail, and then another user 
> connects and indexes mail, indexer-worker is connecting to the first user's 
> fts host:
> 
> User1, ham...@rp-auth-test.com connects, does a SEARCH for the first time, 
> indexer-worker gets UserDB settings and correctly indexes mail on ftsvs01:
> 
> [...]
> auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): lookup 
> shared/userdb/ham...@rp-auth-test.com
> auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): result: 
> {"uid":"8","fts":"solr","quota_rule4":"Spam:ignore","_session":"talk15_590ec6d100042","quota_rule3":"Trash:ignore","quota_rule2":"*:messages=2684354","quota_rule":"*:storage=5242880k","mail":"maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/","fts_solr":"debug
>  url=http://ftsvs01:8080/solr/","gid":"8"}
> auth: Debug: userdb out: USER   1   ham...@rp-auth-test.com uid=8   
> fts=solrquota_rule4=Spam:ignore _session=talk15_590ec6d100042   
> quota_rule3=Trash:ignorequota_rule2=*:messages=2684354  
> quota_rule=*:storage=5242880k   
> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
> fts_solr=debug url=http://ftsvs01:8080/solr/gid=8
> indexer-worker: Debug: auth input: ham...@rp-auth-test.com uid=8 fts=solr 
> quota_rule4=Spam:ignore _session=talk15_590ec6d100042 
> quota_rule3=Trash:ignore quota_rule2=*:messages=2684354 
> quota_rule=*:storage=5242880k 
> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
>  fts_solr=debug url=http://ftsvs01:8080/solr/ gid=8
> indexer-worker: Debug: Added userdb setting: 
> plugin/_session=talk15_590ec6d100042
> indexer-worker: Debug: Added userdb setting: plugin/fts=solr
> indexer-worker: Debug: Added userdb setting: plugin/fts_solr=debug 
> url=http://ftsvs01:8080/solr/
> indexer-worker: Debug: Added userdb setting: 
> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ha
> m...@rp-auth-test.com/
> indexer-worker: Debug: Added userdb setting: 
> plugin/quota_rule=*:storage=5242880k
> indexer-worker: Debug: Added userdb setting: 
> plugin/quota_rule2=*:messages=2684354
> indexer-worker: Debug: Added userdb setting: plugin/quota_rule3=Trash:ignore
> indexer-worker: Debug: Added userdb setting: plugin/quota_rule4=Spam:ignore
> indexer-worker(ham...@rp-auth-test.com): Debug: Effective uid=8, gid=8, home=
> indexer-worker(ham...@rp-auth-test.com): Debug: Namespace inbox: 
> type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions
> =yes 
> location=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
> indexer-worker(ham...@rp-auth-test.com): Debug: maildir++: 
> root=/mail/mailstore01/215/573/ham...@rp-auth-test.com, 
> index=/mail/index01/215/
> 573/ham...@rp-auth-test.com, indexpvt=, control=, 
> inbox=/mail/mailstore01/215/573/ham...@rp-auth-test.com, alt=
> indexer-worker(ham...@rp-auth-test.com): Debug: Ignoring unknown cache field: 
> pop3.order
> indexer-worker(ham...@rp-auth-test.com): Debug: Ignoring unknown cache field: 
> binary.parts
> indexer-worker(ham...@rp-auth-test.com): Warning: Created dotlock file's 
> timestamp is different than current time (1380294685 vs 1380294612
> ): /mail/index01/215/573/ham...@rp-auth-test.com/.INBOX/dovecot.index.log
> indexer-worker(ham...@rp-auth-test.com): Debug: http-client: request [POST 
> http://ftsvs01:8080/solr/update]: Submitted
> [...]
> 
> 
> User1 index finishes and imap searches against ftsvs01
> [.

[Dovecot] fts-solr indexer-worker connects to wrong solr host dovecot-2.2.4

[Richard Platel]

Hello.  
We're setting up fts solr and want to have the solr server host be set per-user 
via UserDB.

It looks like if a user connects and fts indexes mail, and then another user 
connects and indexes mail, indexer-worker is connecting to the first user's fts 
host:

User1, ham...@rp-auth-test.com connects, does a SEARCH for the first time, 
indexer-worker gets UserDB settings and correctly indexes mail on ftsvs01:

[...]
auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): lookup 
shared/userdb/ham...@rp-auth-test.com
auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): result: 
{"uid":"8","fts":"solr","quota_rule4":"Spam:ignore","_session":"talk15_590ec6d100042","quota_rule3":"Trash:ignore","quota_rule2":"*:messages=2684354","quota_rule":"*:storage=5242880k","mail":"maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/","fts_solr":"debug
 url=http://ftsvs01:8080/solr/","gid":"8"}
auth: Debug: userdb out: USER   1   ham...@rp-auth-test.com uid=8   
fts=solrquota_rule4=Spam:ignore _session=talk15_590ec6d100042   
quota_rule3=Trash:ignorequota_rule2=*:messages=2684354  
quota_rule=*:storage=5242880k   
mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
fts_solr=debug url=http://ftsvs01:8080/solr/gid=8
indexer-worker: Debug: auth input: ham...@rp-auth-test.com uid=8 fts=solr 
quota_rule4=Spam:ignore _session=talk15_590ec6d100042 quota_rule3=Trash:ignore 
quota_rule2=*:messages=2684354 quota_rule=*:storage=5242880k 
mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
 fts_solr=debug url=http://ftsvs01:8080/solr/ gid=8
indexer-worker: Debug: Added userdb setting: 
plugin/_session=talk15_590ec6d100042
indexer-worker: Debug: Added userdb setting: plugin/fts=solr
indexer-worker: Debug: Added userdb setting: plugin/fts_solr=debug 
url=http://ftsvs01:8080/solr/
indexer-worker: Debug: Added userdb setting: 
mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ha
m...@rp-auth-test.com/
indexer-worker: Debug: Added userdb setting: 
plugin/quota_rule=*:storage=5242880k
indexer-worker: Debug: Added userdb setting: 
plugin/quota_rule2=*:messages=2684354
indexer-worker: Debug: Added userdb setting: plugin/quota_rule3=Trash:ignore
indexer-worker: Debug: Added userdb setting: plugin/quota_rule4=Spam:ignore
indexer-worker(ham...@rp-auth-test.com): Debug: Effective uid=8, gid=8, home=
indexer-worker(ham...@rp-auth-test.com): Debug: Namespace inbox: type=private, 
prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions
=yes 
location=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
indexer-worker(ham...@rp-auth-test.com): Debug: maildir++: 
root=/mail/mailstore01/215/573/ham...@rp-auth-test.com, index=/mail/index01/215/
573/ham...@rp-auth-test.com, indexpvt=, control=, 
inbox=/mail/mailstore01/215/573/ham...@rp-auth-test.com, alt=
indexer-worker(ham...@rp-auth-test.com): Debug: Ignoring unknown cache field: 
pop3.order
indexer-worker(ham...@rp-auth-test.com): Debug: Ignoring unknown cache field: 
binary.parts
indexer-worker(ham...@rp-auth-test.com): Warning: Created dotlock file's 
timestamp is different than current time (1380294685 vs 1380294612
): /mail/index01/215/573/ham...@rp-auth-test.com/.INBOX/dovecot.index.log
indexer-worker(ham...@rp-auth-test.com): Debug: http-client: request [POST 
http://ftsvs01:8080/solr/update]: Submitted
[...]


User1 index finishes and imap searches against ftsvs01
[...]
imap(ham...@rp-auth-test.com): Debug: http-client: request [GET 
http://ftsvs01:8080/solr/select?fl=uid,score&rows=2&sort=uid+asc&q=(hdr:%22moo%22+OR+body:%22moo%22)&fq=%2Bbox:42faee1f735b1e52b321386e9ade+%2Buser:%22ham...@rp-auth-test.com%22]:
 Submitted
[...]


User2 gr...@rp-auth-test.com connects and does a SEARCH, index worker gets gets 
UserDB settings, including fts host ftsvs02, but connects to ftsvs01 (also note 
index-worker initially shows wrong user in loglines)
[...]
auth-worker(2195): Debug: dict(gr...@rp-auth-test.com): lookup 
shared/userdb/gr...@rp-auth-test.com
auth-worker(2195): Debug: dict(gr...@rp-auth-test.com): result: 
{"uid":"8","fts":"solr","quota_rule4":"Spam:ignore","_session":"cow80_609fed761","quota_rule3":"Trash:ignore","quota_rule2":"*:messages=2684354","quota_rule":"*:storage=5242880k","mail":"maildir:/mail/mailstore01/812/023/gr...@rp-auth-test.com/:INDEX=/mail/index01/812/023/gr...@rp-auth-test.com/","fts_solr":"debug
 url=http://ftsvs02:8080/solr/","gid":"8"}
auth: Debug: userdb out: USER   2   gr...@rp-auth-test.com  uid=8   
fts=solrquota_rule4=Spam:ignore _session=cow80_609fed761
quota_rule3=Trash:ignorequota_rule2=*:messages=2684354  
quota_rule=*:storage=5242880k   
mail=maildir:/mail/mailstore01/812/023/gr...@rp-auth-test.

Re: [Dovecot] Index error copying compressed message

[Richard Platel]

On 2013-09-22, at 12:35 AM, Timo Sirainen  wrote:

> On 19.9.2013, at 23.59, Richard Platel  wrote:
> 
>> Dovecot 2.2, with the zlib plugin, I think we're getting bad index entries 
>> on IMAP COPY.
>> 
>> On copying a message to an empty folder, in the dovecot error log I see:
>> 
>> Sep 19 20:34:25 imap01 dovecot: imap(gr...@rp-auth-test.com): Error: Cached 
>> message size smaller than expected (615 < 971)
>> Sep 19 20:34:25 imap01 dovecot: imap(gr...@rp-auth-test.com): Error: 
>> Corrupted index cache file 
>> /mail/index01/434/860/gr...@rp-auth-test.com/.Bup/dovecot.index.cache: 
>> Broken physical size for mail UID 0
>> Sep 19 20:34:25 imap01 dovecot: imap(gr...@rp-auth-test.com): Error: read() 
>> failed: Invalid argument (uid=0)
>> 
>> (Note this happens from the copy operation, not a subsequent access.  Also 
>> note the UID is always 0).
> 
> UID=0 means that it's trying to get the size for the mail that is still being 
> saved (so not the copy source mail). You mean you can easily reproduce this 
> simply by copying a mail to a newly created folder? I couldn't. Try if you 
> can still reproduce it with a smaller config, especially removing non-zlib 
> plugins.
> 

This was indeed a plugin configuration problem, thanks.

[Dovecot] Index error copying compressed message

[Richard Platel]

Hi.

Dovecot 2.2, with the zlib plugin, I think we're getting bad index entries on 
IMAP COPY.

On copying a message to an empty folder, in the dovecot error log I see:

Sep 19 20:34:25 imap01 dovecot: imap(gr...@rp-auth-test.com): Error: Cached 
message size smaller than expected (615 < 971)
Sep 19 20:34:25 imap01 dovecot: imap(gr...@rp-auth-test.com): Error: Corrupted 
index cache file 
/mail/index01/434/860/gr...@rp-auth-test.com/.Bup/dovecot.index.cache: Broken 
physical size for mail UID 0
Sep 19 20:34:25 imap01 dovecot: imap(gr...@rp-auth-test.com): Error: read() 
failed: Invalid argument (uid=0)

(Note this happens from the copy operation, not a subsequent access.  Also note 
the UID is always 0).

The filename for the message is:
-rw--- 2862 mail mail 615 Aug 29 15:38 
1379622865.M228140P11548.imap01,S=971,W=988:2,S

S= size looks correct:

$ zcat 1379622865.M228140P11548.imap01\,S\=971\,W\=988\:2\,S |wc
 17  51 971


doveadm dump says:
$ sudo -u mail doveadm -c /he/dovecot/conf/dovecot.conf dump 
/mail/index01/434/860/gr...@rp-auth-test.com/.Bup/
Detected file type: index
-- INDEX: /mail/index01/434/860/gr...@rp-auth-test.com/.Bup//dovecot.index
version .. = 7.3
base header size . = 120
header size .. = 208
record size .. = 12
compat flags . = 1
index id . = 1379605150 (2013-09-19 15:39:10)
flags  = 0
uid validity . = 1377629137 (2013-08-27 18:45:37)
next uid . = 14309
messages count ... = 1
seen messages count .. = 1
deleted messages count ... = 0
first recent uid . = 14308
first unseen uid lowwater  = 14309
first deleted uid lowwater = 14308
log file seq . = 6
log file tail offset . = 204
log file head offset . = 204
day stamp  = 1379548800 (2013-09-19 00:00:00)
day first uid[0] . = 1
day first uid[1] . = 0
day first uid[2] . = 0
day first uid[3] . = 0
day first uid[4] . = 0
day first uid[5] . = 0
day first uid[6] . = 0
day first uid[7] . = 0
-- Extension 0 --
name  = maildir
hdr_size  = 36
reset_id  = 0
record_offset = 0
record_size . = 0
record_align  = 0
header
 - new_check_time  = 2013-09-19 20:34:10
 - new_mtime . = 2013-09-19 20:08:51
 - new_mtime_nsecs ... = 79253
 - cur_check_time  = 2013-09-19 20:35:38
 - cur_mtime . = 2013-09-19 20:35:38
 - cur_mtime_nsecs = 22771
 - uidlist_mtime . = 2013-09-19 20:35:38
 - uidlist_mtime_nsecs = 254613000
 - uidlist_size .. = 1025178
-- Extension 1 --
name  = cache
hdr_size  = 0
reset_id  = 1379605174
record_offset = 8
record_size . = 4
record_align  = 4
-- Keywords --

-- CACHE: /mail/index01/434/860/gr...@rp-auth-test.com/.Bup//dovecot.index.cache
major version  = 1
minor version  = 1
indexid .. = 1379605150 (2013-09-19 15:39:10)
file_seq . = 1379605174 (2013-09-19 15:39:34) (24 compressions)
continued_record_count = 0
record_count . = 0
used_file_size (old) . = 108
deleted_record_count . = 0
field_header_offset .. = 32 (0x88808080 nontranslated)
-- Cache fields --
 #  Name Type Size Dec  Last used
 0: flagsbit 4 tmp  2013-09-19 20:07
 1: hdr.Message-ID   hdr - tmp  2013-09-19 20:07
 2: hdr.X-HE-Tag hdr - tmp  2013-09-19 20:07

-- RECORDS: 1
RECORD: seq=1, uid=14308, flags=0x08 (Seen)
 - ext 3 cache :  0 ()



$ sudo -u mail dovecot -c /he/dovecot/conf/dovecot.conf -n
# 2.2.4.3 (12e60e803a54+): /he/dovecot/conf/dovecot.conf
# OS: Linux 3.4.46-dom0-2.0.0 x86_64 Debian 7.0
debug_log_path = syslog
disable_plaintext_auth = no
first_valid_uid = 8
info_log_path = syslog
lock_method = dotlock
log_path = /var/run/dovecot/log-fifo
log_timestamp =
mail_fsync = always
mail_gid = mail
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = zlib quota tc_mail_log notify tc_proc stats
mail_uid = mail
maildir_very_dirty_syncs = yes
mmap_disable = yes
namespace inbox {
  inbox = yes
  location =
  prefix =
}
passdb {
  args = host=localhost port=1143 
username=%L{user}::%L{service}::%L{rip}::%L{session}
  driver = imap
}
plugin {
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename 
flag_change append
  mail_log_fields = uid box msgid flags hetag
  memcached_servers = 10.5.47.223,10.5.47.222
  quota = dict:User quota::proxy:/var/run/auth_proxy_dovecot/quotasocket:quota
  stats_command_min_time = 1 mins
  stats_domain_min_time = 12 hours
  stats_ip_min_time = 12 hours
  stats_memory_limit = 16 M
  stats_refresh = 30 secs
  stats_session_min_time = 15 mins
  stats_track_cmds = yes
  stats_user_min_time = 1 hours
  zlib_save = gz
  zlib_save_level = 6
}
protocols = imap pop3
service anvil {
  un

Re: [Dovecot] Custom quota setup

[Richard Platel]

On 2013-08-02, at 12:05 PM, Timo Sirainen  wrote:

> On 2.8.2013, at 18.56, Richard Platel  wrote:
> 
>> On 2013-08-02, at 11:34 AM, Timo Sirainen  wrote:
>> 
>>> On 2.8.2013, at 18.15, Richard Platel  wrote:
>>> 
>>>> We pass custom quota rules for each user in our userdb, and use a custom 
>>>> dict proxy program, so that program could read the file and pass a setting 
>>>> at log in time too (if, for example, there was a setting that said "offset 
>>>> the user's quota usage by X amount")
>>> 
>>> That I think would work.
>>> 
>> 
>> That's future development though?  No such setting exists now?
> 
> I'm not exactly sure what you thought of, but my idea was simply that you'd 
> add your own dict proxy in the middle which hooks into the GET command, and 
> increases its value by reading the filestoresize. You can already do that by 
> pointing to a UNIX socket different from the normal dict server socket, 
> similar as in http://wiki2.dovecot.org/AuthDatabase/Dict
> 

If, for example, the user has 2GB quota, and 750MB of mail, and 500MB of files, 
we'd like the mail client to show that the user has 2GB quota, and 1.25GB used, 
.75GB free.

I see how via the dict userdb proxy I could reduce the user's quota allowance 
by the filestorage amount:
quota_rule=*:storage=, 
and the client would show the correct free amount, but not the correct total or 
used amount. 

Is this what you mean, or is there some other setting or some other dict proxy 
entirely that I'm missing?

Re: [Dovecot] Custom quota setup

[Richard Platel]

On 2013-08-02, at 11:34 AM, Timo Sirainen  wrote:

> On 2.8.2013, at 18.15, Richard Platel  wrote:
> 
>> We pass custom quota rules for each user in our userdb, and use a custom 
>> dict proxy program, so that program could read the file and pass a setting 
>> at log in time too (if, for example, there was a setting that said "offset 
>> the user's quota usage by X amount")
> 
> That I think would work.
> 

That's future development though?  No such setting exists now?

[Dovecot] feature request: IMAP passdb prefetch

[Richard Platel]

> On 2.8.2013, at 18.20, Richard Platel  wrote:
> 
>> It'd be useful for us if the IMAP passdb could be used as a prefetch userdb.
>> 
>> The remote IMAP server could respond with something like
>> 
>> * OK key=value
>> * OK key=value
>> SEQ OK [CAPABILITY ...] Logged in.
>> 
>> Or
>> 
>> * OK 
>> SEQ OK [CAPABILITY ...] Logged in.
>> 
>> Would anyone else find this useful?
> 
> Uh. Why not simply something completely different like HTTP-based passdb?
> 

I can't find info for HTTP on the wiki, can we set that up as a success/fail 
passdb?  We do our own password auth (so we can support custom hash types, etc).

[Dovecot] feature request: IMAP passdb prefetch

[Richard Platel]

It'd be useful for us if the IMAP passdb could be used as a prefetch userdb.

The remote IMAP server could respond with something like

* OK key=value
* OK key=value
SEQ OK [CAPABILITY ...] Logged in.

Or

* OK 
SEQ OK [CAPABILITY ...] Logged in.

Would anyone else find this useful?

[Dovecot] Custom quota setup

[Richard Platel]

(Dovecot 2.2-ee) 

We have a weird quota requirement, we have file storage that we manage through 
our own APIs but want that usage to come out of the user's mail quota.

The usage is in a maildirsize like file uncreatively called filestoresize in 
the user's maildir.  

In the past we've been doing this by modifying the quota plugin and 
re-compiling, but it seems like it should be possible to do this via 
configuration.

Is there a way to add a quota setting pointing at this file for additional 
usage (not limits)?

The feature is used infrequently so it would probably be acceptable use the 
dirsize backend, but I can't figure out how to configure that to point at a 
certain directory.

We pass custom quota rules for each user in our userdb, and use a custom dict 
proxy program, so that program could read the file and pass a setting at log in 
time too (if, for example, there was a setting that said "offset the user's 
quota usage by X amount")


Thanks.

[Dovecot] question about /var/spool/mail/xxx

[Richard Feng@eBay]

this is Redhat 6.3
I installed sendmail and dovecot.
but sendmail can continue send mail as MTA
all mails go to /var/spool/mail

this is problem for dovecot. dovecot cannot get it goto dovecot inbox.

if setup mailbox poing to /var/spool/mail/%u, it is workable
but if set up maildir to somewhere else ,I cannot receive mail by dovecot.

[Dovecot] "Renaming not supported across conflicting directory permissions"

[Richard Platel]

Is it possible to disable this check in 2.1.7? (without a patch and re-compile?)

Re: [Dovecot] Fixing bad maildir message sizes

[Richard Platel]

d, $new) = each %renames)
  {
$old = basename($old);
$new = basename($new);
$list =~ s/$old/$new/;
  }
  truncate ($fh, 0);
  seek($fh, 0,0);
  print $fh $list;
  close ($fh);
}

sub _pout($@)
{
  my $h = shift;
  print $h @_ if $h;
}


# get the message size, either from the last 4 bytes if compressed, or file size
sub _uncompressed_size_quick($)
{
  my ($fn) = @_;
  my $gzid = chr(0x1f) . chr(0x8b);
  my ($flag, $buf);

  open(my $fh, '<', $fn) or return undef;
  return undef unless (sysread($fh, $flag, 2) == 2);
  unless ($flag eq $gzid) # not a compressed file, return the size-on-disk
  {
return sysseek($fh, 0, 2);
  }
  # gziped file, size is in last 4 bytes
  return undef unless (sysseek($fh, -4, 2));
  return undef unless (sysread($fh, $buf, 4));
  return unpack('V', $buf);
}
  

# get the S= and W= size by reading the whole file.
sub _uncompressed_size($)
{
  my $fn = $_[0];
  my $fh = IO::Zlib->new($fn, "rb") || IO::File->new("< $fn");
  return undef unless $fh;
  my $sz = 0; #uncompressed size
  my $wsz = 0; #uncompressed size with /n converted to /r/n
  my $read;
  my $chunk = 4096; # TODO tune
  my $buf;
  my $cusp = 0;
  while ($read = read($fh, $buf, $chunk))
  {
$sz += $read;
$wsz += $read;
$wsz += () = $buf =~ /(? \$maildir,
"folder=s" => \$folder,
"help" => sub {pod2usage(-verbose =>1 )},
"man" => sub {pod2usage(-verbose =>2 )}) 
  || pod2usage(-verbose => 1);

  $lockbin = $ENV{MAILDIRLOCK_BIN};
  pod2usage(-verbose=>1) unless ($lockbin && ($maildir || $folder || $mail));

  die "Could not execute maildirlock [$lockbin]" unless -x $lockbin;
  system("$lockbin >/dev/null 2>&1 ");
  die "Could not execute maildirlock [$lockbin], " . 
"maybe you need to set LD_LIBRARY_PATH" unless ($? >> 8) == 1;
  if ($maildir)
  {
print "Fixing maildir: [$maildir]\n";
fixmaildir($maildir, \*STDOUT);
  }
  if ($folder)
  {
print "Fixing folder: [$folder]\n";
fixfolder($folder, \*STDOUT);
  }
}

1;


On 2013-01-22, at 7:01 AM, Timo Sirainen  wrote:

> On 21.1.2013, at 21.54, Richard Platel  wrote:
> 
>> As stated in my previous message, we have some old compressed maildir 
>> messages with incorrect sizes in the filename.  These messages cause dovecot 
>> 2.x problems.
>> 
>> I'm trying to write a script to crawl all our messages, check the actual 
>> message size and if necessary, fix the filename.  However, when I do this, 
>> dovecot gives the message a new UID on next login.  If I change the filename 
>> in dovecot-uidlist, dovecot still gives a new UID on next login.  If I 
>> change dovecot-uidlist and delete the index, then the UID is preserved.
> 
> I don't really understand why deleting dovecot.index* would make a difference 
> here, except maybe as a workaround in case the user has that mailbox 
> selected, because the filenames could be cached in memory.
> 
> http://dovecot.org/tools/maildir-size-fix.pl
> http://dovecot.org/tools/maildir-size-check.sh
> 
> Those scripts kind of do what you want, except not fully, so it would be nice 
> to get one fully finished one :) The best way for the script to would would 
> be to:
> 
> * scan through a maildir, figure out what needs to be renamed to what, but 
> don't actually do it
> * lock the maildir with dovecot-uidlist.lock (src/util/maildirlock comes with 
> dovecot)
> * doveadm kick user's imap/pop3 sessions, and even better if it was possible 
> to kill -9 any pending processes
> * rename the files and update dovecot-uidlist
> * delete dovecot-uidlist.lock
> 
> This separately for each folder.
> 

[Dovecot] Fixing bad maildir message sizes

[Richard Platel]

Hello.

As stated in my previous message, we have some old compressed maildir messages 
with incorrect sizes in the filename.  These messages cause dovecot 2.x 
problems.

I'm trying to write a script to crawl all our messages, check the actual 
message size and if necessary, fix the filename.  However, when I do this, 
dovecot gives the message a new UID on next login.  If I change the filename in 
dovecot-uidlist, dovecot still gives a new UID on next login.  If I change 
dovecot-uidlist and delete the index, then the UID is preserved.

Re-indexing our millions of mailboxes is not a great solution for us.  Is there 
a good way to fix this?

[Dovecot] Zlib maildir reindex bug?

[Richard Platel]

Hi

Running dovecot 2.1.7 (from debian repo).

We have some old compressed maildir messages with the wrong S= size in the 
filename (our fault).

If I delete index files, log in to the mailbox and try to FETCH the bad 
message, dovecot complains about the incorrect message size, attempts to fix 
the filename and disconnects the client.  However, it changes the filesize to 
the size of the message on disk, not the message's uncompressed size.

For subsequent clients, the FETCH works, but dovecot logs an error (and maybe 
re-indexes the mailbox?) and the message UID changes.


Jan 14 18:40:56 imap01 dovecot: imap(b...@confidence.com): Error: Cached 
message size larger than expected (1612 > 1556)
Jan 14 18:40:56 imap01 dovecot: imap(b...@confidence.com): Error: Maildir 
filename has wrong S value, renamed the file from 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=1612:2,Sab to 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab
Jan 14 18:40:56 imap01 dovecot: imap(b...@confidence.com): Error: Corrupted 
index cache file /mail/index01/505/236/b...@confidence.com/.Sent 
Items/dovecot.index.cache: Broken physical size for mail UID 1
Jan 14 18:40:56 imap01 dovecot: imap(b...@confidence.com): Error: 
read(/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=1612:2,Sab) failed: 
Input/output error (uid=1)


Jan 14 18:41:25 imap01 dovecot: imap(b...@confidence.com): Error: Cached 
message size smaller than expected (931 < 1556)
Jan 14 18:41:25 imap01 dovecot: imap(b...@confidence.com): Error: Maildir 
filename has wrong S value, renamed the file from 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab to 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab
Jan 14 18:41:25 imap01 dovecot: imap(b...@confidence.com): Error: Corrupted 
index cache file /mail/index01/505/236/b...@confidence.com/.Sent 
Items/dovecot.index.cache: Broken physical size for mail UID 2
Jan 14 18:41:25 imap01 dovecot: imap(b...@confidence.com): Error: Cached 
message size smaller than expected (931 < 1556)
Jan 14 18:41:25 imap01 dovecot: imap(b...@confidence.com): Error: Maildir 
filename has wrong S value, renamed the file from 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab to 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab
Jan 14 18:41:25 imap01 dovecot: imap(b...@confidence.com): Error: Corrupted 
index cache file /mail/index01/505/236/b...@confidence.com/.Sent 
Items/dovecot.index.cache: Broken physical size for mail UID 2


Jan 14 18:41:53 imap01 dovecot: imap(b...@confidence.com): Error: Cached 
message size smaller than expected (931 < 1556)
Jan 14 18:41:53 imap01 dovecot: imap(b...@confidence.com): Error: Maildir 
filename has wrong S value, renamed the file from 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab to 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab
Jan 14 18:41:53 imap01 dovecot: imap(b...@confidence.com): Error: Corrupted 
index cache file /mail/index01/505/236/b...@confidence.com/.Sent 
Items/dovecot.index.cache: Broken physical size for mail UID 2
Jan 14 18:41:53 imap01 dovecot: imap(b...@confidence.com): Error: Cached 
message size smaller than expected (931 < 1556)
Jan 14 18:41:53 imap01 dovecot: imap(b...@confidence.com): Error: Maildir 
filename has wrong S value, renamed the file from 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab to 
/mail/mailstore01/505/236/b...@confidence.com/.Sent 
Items/cur/1168058702.93517743273399070372.fetchmail01,S=931:2,Sab
Jan 14 18:41:53 imap01 dovecot: imap(b...@confidence.com): Error: Corrupted 
index cache file /mail/index01/505/236/b...@confidence.com/.Sent 
Items/dovecot.index.cache: Broken physical size for mail UID 2

[Dovecot] Plugin help, number of messages in mailbox

[Richard Platel]

Hi,

We use Dovecot for IMAP and POP (but not LDA), we want to do something when a 
user has an INBOX that becomes empty, or becomes not empty (set a flag in 
memcached, but that's not really important).  

I'm writing a plugin (for Dovecot 2.1.7).  On mailbox_open() I can use 
mailbox_get_status() to get a count of messages in the mailbox, and then 
decrement this in expunge() or increment it in mailbox save_finish() (for IMAP 
APPEND or COPY commands).

However in expunge() and mailbox_save_finish, even after calling the super 
function, mailbox_get_status doesn't update the number of messages in the 
mailbox.

This is a problem if (for example) there are concurrent POP sessions.  Two POP 
sessions could get all the messages in INBOX, one could logout, calling expunge 
a few times, eventually causing my plugin to note that the inbox is empty, then 
our LDA could deliver a message, mark the INBOX not empty, then the other POP 
session could log out, call expunge and cause my plugin to mark the INBOX 
empty, when it's not.


So in summation: how can a plugin be notified of changes to a mailbox, and then 
accurately get the real number of messages in that mailbox? 

[Dovecot] Custom auth process in dovecot 2

[Richard Platel]

Hi,

I'm trying to upgrade from dovecot-1.1.x to 2.1.7.  

We have our own custom auth server process (because we want to do our own 
password validation and for other reasons) that listens on a UNIX domain socket 
and speaks the dovecot auth protocol.  

In dovecot 1.1 we could configure this with

auth external {
  socket connect {
master {
  path = /var/run/dovecot/auth.sock
}
  }
}

as per http://wiki.dovecot.org/MainConfig

I haven't been able to figure out how to do this in 2.1.7, is it possible?

Re: [Dovecot] Getting duplicates despite trying hard to match lock styles

[Richard Walker]

On 12/06/2012, Richard Walker  wrote:
> 1. Output of "doveconf -n" and a note about how I modified locking
>from the Fedora default.

Oops, I can send more of the config if necessary -- again, I
was trying to be "helpful" by cutting out the default settings.

The output of "doveconf | grep lock" is:

dotlock_use_excl = yes
lock_method = fcntl
mail_max_lock_timeout = 0
mbox_dotlock_change_timeout = 2 mins
mbox_lock_timeout = 5 mins
mbox_read_locks = fcntl
mbox_write_locks = dotlock fcntl
pop3_lock_session = no

[Dovecot] Getting duplicates despite trying hard to match lock styles

[Richard Walker]

I'm attempting to replace (a) a very old setup that has POP (qpopper)
access to inboxes and a separate UW IMAP server that provides folders,
with (b) a shiny new mail setup with dovecot providing both inboxes
and IMAP support.

For the new mail server I created a virtual machine running a minimal
Fedora 16 installation and installed sendmail, MIMEDefang,
SpamAssassin, ClamAV, procmail, and dovecot.  I have kept installing
updates as they become available.

For now I'm running the old and new mail setups in parallel; I have
configured the original sendmail server to forward copies of incoming
messages to the new sendmail running on the virtual machine.  I then
compare the results (e.g., how spam filtering is working).

I've kept as much as possible of the original _style_ of setup as
possible, which in particular means using sendmail, and message
delivery through procmail to mbox files in /var/spool/mail.  The key
difference is the use of dovecot to provide IMAP access to the inbox
and IMAP folders.

Because of the legacy setup, my desktop access to email is via
Thunderbird 2.0.0.22 on a very old Mac PowerBook G4 to work with both
old and new setups and I have two windows open to make comparison
possible.  (Yes, both mail servers are on separate computers, not on
this notebook.)

Mostly this is working fine (after a fair bit of tweaking, including
adding custom SELinux rules to get rid of all AVCs).  I put the
notebook to sleep overnight, and in the morning I open it up and see
what happens.  After a few minutes, the window with the old setup does
its POP fetch; the window with the new setup almost straightaway shows
the new messages in its version of the inbox.

Not quite: again, for legacy reasons I have some Thunderbird filters,
and I have duplicated those (still within Thunderbird) for the new
setup.  The filters are:
1. Move messages tagged as spam by SpamAssassin to the Junk folder.
2. Move messages from GeoNetwork-related senders to a "GeoNetwork"
   folder.
3. Move all remaining messages to the "In" folder.

Most mornings this works just fine.  But not always.  Sometimes I get
duplicates in the "In" and "GeoNetwork" folders of the new
dovecot-based setup.  I used to get _garbled_ duplicates (with extra
random bits of other messages at the end of the duplicates) in the new
setup, which I presumed must be due to a locking configuration
mismatch.  Having fixed that (see below) I no longer get garbled
duplicates, but I do still sometimes (including today) get identical
duplicates.  This seems to happen when one of the incoming messages
has a very large attachment - but you may wish to treat that as
hearsay.

I attach below:
0. The line from /etc/mtab on the new server that covers the
   filesystem (i.e., including /var/spool and /home).
1. Output of "doveconf -n" and a note about how I modified locking
   from the Fedora default.
2. Output of "procmail -v".
3. Sendmail procmail mailer config (for good measure; I don't think
   you need this).
4. An excerpt from /var/log/maillog on the new server showing the
   beginning of dovecot processing this morning when I opened my
   notebook.
5. A link to the dovecot raw log files of my "INBOX" and "In" folder
   processing from this morning.

You'll see from the dovecot log files that Thunderbird sends expunge
commands, but the expunged messages hang around -- indeed, the same
messages get expunged several times!  And eventually they get fetched
again -- hence the duplicates I see in Thunderbird.

Given that INBOX.out contains:

08:56:53.765423 * 537 EXISTS
08:56:53.765423 * 533 RECENT

and then, after many expunges:

08:56:58.441341 * 16 EXPUNGE
08:56:58.441341 * 11 EXPUNGE
08:56:58.441341 * 3 EXPUNGE
08:56:58.441341 * 539 EXISTS
08:56:58.441341 * 536 RECENT
08:56:58.441341 9 OK Expunge completed.

it looks like I still have a locking problem.  I have tried very hard
to understand the locking options in dovecot.conf and to match dovecot
with procmail -- apparently, there is more to do.


0. The line from /etc/mtab for the filesystem:

--
/dev/mapper/vg_f16i386serverbasic-lv_root / ext4
rw,seclabel,relatime,user_xattr,acl,barrier=1,data=ordered 0 0
--


1. doveconf -n says:

--
# 2.0.20: /etc/dovecot/dovecot.conf
# OS: Linux 3.3.6-3.fc16.i686.PAE i686 Fedora release 16 (Verne)
mail_debug = yes
mail_privileged_group = mail
namespace {
  hidden = yes
  inbox = yes
  list = no
  location = mbox:~/mail:INBOX=/var/spool/mail/%u
  prefix = "#mbox/"
  separator = /
  type = private
}
namespace {
  inbox = no
  location = maildir:~/Maildir
  prefix =
  separator = /
  type = private
}
passdb {
  driver = pam
}
service imap-login {
  inet_listener imap {
address = localhost
  }
}
service imap {
  executable = imap postlogin
}
service pop3-login {
  inet_listener pop3 {
address = localhost
  }
}
service postlogin {
  executable = script-login -d rawlog -t
}
ssl_cert = 
Copyright (c) 1997-2001, Philip A. Guenther 

Submit questi

Re: [Dovecot] Can we know when a user read our email?

[Richard]

> Date: Sunday, June 03, 2012 02:54:32 PM -0400
> From: Jerry 
> 
>> On Sun, 03 Jun 2012 20:19:20 +0200
>> Reindl Harald articulated:
>> 
>> people are mostly to stupid to realize what they
>> are trying to accomplish and why it it a bad idea
>> 
>> this is why we professionals exist and if people
>> refuse what you are explaining them kiss them
>> goodbye - irt will be better for you over the long
> 
> No offense, but considering your business attitude and disdain for
> potential clients and your opinion of them, it would be a far
> better thing if they steered clear of you all together. There are
> many considerate, intelligent, compassionate professionals out
> there who would be willing to take on the difficult client. Any
> "asshole" can service the routine, run of the mill, client. It
> takes a true professional to work with and service a difficult 
> one.

Something that seems to be missing from this discussion are
considerations of privacy and (personal) security. There are fairly
serious implications of a sender being able to tell that/when
someone has downloaded/opened a message -- including discovery of
daily patterns and potentially where the recipient is, or isn't.

I think it is our responsibility to understand these issues and
explain them to managers/clients in order to bring them along if we
refuse (as I would) to provide a capability such as this. [I always
set the sendmail "noreceipts" PrivacyOptions so it doesn't respond
to these disposition requests.]

One approach is to point out to managers/clients that if their
system is configured to return read receipts, anyone sending mail to
them on that system will be able to get these same types of
receipts. When they think about that they may not like the
implications and may reconsider their request. 

Just because it is technically possible to do something (and even if
other vendors provide the capability) does not mean that it is the
ethically or legally responsible thing to do.


   - Richard

 

[Dovecot] FIXED Re: Trouble adding sasl support via dovecot

[Richard Troy]

Hi All,

it turned out to be the order of entries in stmpd_recipient_restrictions.

Regards,
Richard


On Mon, 12 Mar 2012, Richard Troy wrote:

> Date: Mon, 12 Mar 2012 10:14:09 -0700 (PDT)
> From: Richard Troy 
> To: postfix-us...@cloud9.net, dovecot@dovecot.org
> Subject: [Dovecot] Trouble adding sasl support via dovecot
>
>
> Hello Folks,
>
> I've been the admin of a site that uses Postfix with Dovecot on RedHat
> since, oh, gosh, maybe 1996? It's been a long time. I've never built it
> from source, though, just used the rpms (and I wonder if maybe that's my
> problem now). It just works, is reliable, and lets me be a very-part-time
> administrator.
>
> Repeatedly over the last few years I've been asked to have our mail system
> "join the modern age" and provide mail sending capabilities for clients
> that aren't on our internal network - via their smart-phones, from home,
> etc. OK... Well, way back when the site was set up, smtp servers didn't do
> any kind of "auth", but along the way to solving this problem (trying to
> configure pop-before-smtp, someone mentioned that Postfix now has an auth
> mechanism that uses Dovecot and I should use that instead! Great!  ...
> Except that I've spent between 16 and 20 hours on this with no joy, and
> while I hate having to ask for help, it's time to ask what things that are
> obvious to the less ignorant that I must be doing wrong... Certainly,
> given the solid history of Postfix and Dovecot, I must be the problem!
>
> My problem statement is simply, "it should be working", but doesn't, and I
> don't get any announcement of "auth" when testing connections to Postfix
> as per directions here:
>
>http://www.postfix.org/SASL_README.html#server_test
>
> At least I haven't broken the normal functionality!
>
> I'm building a new server on the latest Fedora Core (16), but it's lacking
> in some hardware and won't be ready for a while, so I'm working with FC
> 14, running Postfix 2.5.6, and Dovecot 1.2.8. It uses the "cram-md5" auth
> scheme (which works fine and I'd hate to change it if I don't have to).
> The system has been up and functional on these versions for a couple of
> years, and quite stable, we just can't send if we're not local.
>
> When I do "postconf-a" it indicates cyrus and dovecot, so I take it that
> means Postfix has been built with sasl support. (I presume this means I
> don't have to compile it from source.)
>
> First Dovecot. Its set up to provide all protocols, but only imaps and
> pop3s have ports forwarded through the firewall. Plain-text auth is
> disabled, ssl is set to yes, ssl_listen is not specified, and the cert and
> key files are in the default locations - and work. No cipher list is used.
> Dovecot's chrooted. The protocol sections imap and pop3 take ALL the
> defaults, as does lda (I've ignored sendmail_path = /usr/lib/sendmail) as
> I don't think it matters. "auth default {" has mechanisms set to cram-md5,
> digest-md5, plain, and login, with passdb passwd-file pointing to a file
> in /etc where the cram data goes. It's not using pam, and there's an OLD
> comment in the config:
>
> # Experience says we need an empty passdb - passwd group:
>
> which is followed by passdb passwd{}. Later, there's "userdb passwd {}.
>
> All of that was configured long ago and has been functional.
>
> The changes I've made to add sasl support primarily pertain to the "socket
> listen section of "auth default". There, the master section remains
> commented out while the client section has been uncommented, the path set
> to /var/spool/postfix/private/auth, mode set to 0660, and the user and
> group have been set to postfix. ...This is all as described here:
>
> http://www.postfix.org/SASL_README.html
> and
> http://wiki.dovecot.org/HowTo/PostfixAndDovecotSASL
>
> That's it for Dovecot. Now, to Postfix itself.
>
> >From the working environ, only listening on port 25, I simply added the
> following (as per directions already cited above):
>
> smtpd_sasl_type = dovecot
> smtpd_sasl_path = private/auth
> smtpd_sasl_auth_enable = yes
> broken_sasl_auth_clients = yes
> smtpd_sasl_security_options = noanonymous, noplaintext
> smtp_sasl_security_options = noanonymous, noplaintext
> smtpd_sasl_tls_security_options = noanonymous
> smtp_sasl_tls_security_options = noanonymous
>
> And, of course, permit_sasl_authenticated was added to
> smtpd_recipient_restrictions.
>
>
> I got the impression from the baove sources that Postfix will then use
>

[Dovecot] Trouble adding sasl support via dovecot

[Richard Troy]

s, too (after trying without). On the good side,
an available Android phone, previously reading fine, but unable to send,
no longer complained when the setup was changed to the imap username and
password, same server address, TLS security type, and the server port of
25. HOWEVER, no mail has passed through it successfully, it just gives no
error whatsoever, so far, while the server's log reports "Relay access
denied."

Notably, when setting up TLS, Postfix complained when the
smtpd_tls_key_file was incorrect, but did not complain when it was
provided properly, suggesting it's reading and accepting my self-signed
certificate and private key.

Ideas, please?!

And, by the way, what's port 465 all about? Some clients propose that's
what should be used to send...


Thanks in advance for your help,
Richard

Re: [Dovecot] dovecot-managesieve

[Richard Gliebe]

On 6/13/11 3:11 PM Timo Sirainen wrote:

Hi,


Looks to me like the managesieve version isn't compatible with this
Dovecot version. I guess the earlier managesieve binaries didn't give a
better error message about mismatching versions.


Which managesieve version is compatible with my dovecot version and 
where are the Repos?


At the moment these packages are installed (with "yum install") on our 
CentOS release 5.6 (Final) box:


dovecot-1.1.20-1_98.el5
dovecot-sieve-devel-0.1.18-7.el5
dovecot-managesieve-0.11.12-0_5.1.el5
dovecot-sieve-cmu-1.1.8-11.el5
dovecot-sieve-0.1.18-7.el5


[/etc/yum.conf]
# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum.repos.d
#
[vanderkooij]
name=Vanderkooij.org
baseurl=http://yum.vanderkooij.org/el5/$basearch/
enable=1
gpgkey=http://yum.vanderkooij.org/RPM-GPG-KEY-HvdK.asc
gpgcheck=1
#
[atrpms]
name=Fedora Core $releasever - $basearch - ATrpms
baseurl=http://dl.atrpms.net/el$releasever-$basearch/atrpms/stable
gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms
gpgcheck=1

many thanks
Richard

[Dovecot] dovecot-managesieve

[Richard Gliebe]

Hi all,

I have to implement a Out of Office Tool on our dovecot IMAP Server.

Recently I updated dovecot-1.0.7-7.el5 to dovecot-1.1.20-1_98.el5 on our 
CentOS release 5.6 (Final) box via "yum update"


# cat /etc/yum.conf

[atrpms]
name=Fedora Core $releasever - $basearch - ATrpms
baseurl=http://dl.atrpms.net/el$releasever-$basearch/atrpms/stable
gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms
gpgcheck=1
#


After it, I installed dovecot-managesieve-0.11.12-0_5.1.el5 via yum install.

These are my sieve settings in /etc/dovecot.conf:

...
version_ignore=yes

# Log file to use for error messages, instead of sending them to syslog.
# /dev/stderr can be used to log into stderr.
log_path = /var/log/dovecot.log

# Protocols we want to be serving: imap imaps pop3 pop3s managesieve
# If you only want to use dovecot-auth, you can set this to "none".
#protocols = imap imaps pop3 pop3s
protocols = imaps pop3s pop3 managesieve

mail_debug=yes

 protocol managesieve {
   # Specify an alternative address:port the daemon must listen on
   # (default: *:2000)
   #listen = localhost:2000

   managesieve_logout_format = bytes ( in=%i : out=%o )

   # Duplicated from plugin section
   sieve=~/.dovecot.sieve
   sieve_storage=~/sieve
 }

 plugin {
   # Settings for the Sieve plugin
   sieve=~/.dovecot.sieve
   sieve_dir=~/sieve
 }
..

After restarting dovecot, I'm getting the following errors in dovecot.log:

dovecot: Jun 10 08:50:14 Info: dovecot v1.1.20 starting up (core dumps 
disabled)
dovecot: Jun 10 08:50:16 Error: login: Login request missing a file 
descriptor
dovecot: Jun 10 08:50:16 Error: Temporary failure in creating login 
processes, slowing down for now
dovecot: Jun 10 08:50:16 Warning: managesieve-login: managesieve-login: 
capability string is empty.
dovecot: Jun 10 08:50:16 Error: login: Login request missing a file 
descriptor
dovecot: Jun 10 08:50:16 Warning: managesieve-login: managesieve-login: 
capability string is empty.
dovecot: Jun 10 08:50:16 Error: login: Login request missing a file 
descriptor
dovecot: Jun 10 08:50:16 Warning: managesieve-login: managesieve-login: 
capability string is empty.
dovecot: Jun 10 08:50:16 Info: Created login processes successfully, 
unstalling
dovecot: Jun 10 08:50:17 Error: login: Login request missing a file 
descriptor
dovecot: Jun 10 08:50:17 Error: Temporary failure in creating login 
processes, slowing down for now


Don't know, whats going on .

next problem.
I can't login with our Thunderbird Clients to our IMAP/sieve Server 
(Port 2000) to edit Sieve Filters.


Authentication is set to:
use IMAP Username and password

Port 2000 ist listening:
# netstat -an | grep 2000
tcp0  0 0.0.0.0:20000.0.0.0:* 
LISTEN
tcp0  0 192.168.1.6:2000192.168.1.8:2882 
ESTABLISHED
tcp0  0 :::2000 :::* 
LISTEN


Out of Office:

I found a plugin for our thunderbird clients:
http://www.trustedbird.org/tb/Out_of_Office

is this the right one?

many many hints are welcome ;-)
Richard

Re: [Dovecot] Fatal: postmaster_address setting not given

[Richard Edmonds]

On 10/03/2011 12:45 a.m., Charles Marcus wrote:

On 2011-03-08 8:33 PM, Richard Edmonds wrote:

I'm getting: "Fatal: postmaster_address setting not given" errors in my
log file
dovecot ver:  1.2.12
Recently I tried to follow:
http://postfixmail.com/blog/index.php/postfixadmin-on-ubuntu-9-10/ on
ubuntu 10.10
(which works fine on ubunto 9.10)

If you aren't going to follow the official docs, but some other howto,
you should ask the other howto maintainer...


I note that in 10.10 their is no dovecot-postfix.conf so I made the
appropriate changes to dovecot.conf instead and changed master.cf
dovecot transport line since I saw someone recently had the same problem:

dovecot   unix  -   n   n   -   -   pipe
   flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -c
/etc/dovecot/dovecot.conf -f ${sender} -d ${recipient}

I still have that problem however. Authentication seems ok but delivery
fails.
Any possibility of help on this one?

dovecot -n output is generally mandatory when asking for help with your
config...

But really you should be asking this question in an ubuntu support forum
don't you think?


Thanks for reviewing.

As it turns out the problem I had was due to a difference in dovecot 
versions that I hadn't noticed. A section of the dovecot.conf file 
describing the LDA parameters in which the default

#postmaster_address = postmas...@example.com
became
#postmaster_address =
Which caused fatal failure and needed to be fixed. That's all.
I doubt anyone else will have the same problem if they read the manual 
or google for a solution and find /this/ post.


Thanks again
lockquote>
   Thanks for reviewing.
   
   As it turns out the problem I had was due to a difference in dovecot
   versions that I hadn't noticed. A section of the dovecot.conf file
   describing the LDA parameters in which the default
   #postmaster_address = postmas...@example.com
   became
   #postmaster_address =
   Which caused fatal failure and needed to be fixed. That's all.
   I doubt anyone else will have the same problem if they read the
   manual or google for a solution and find this post.
   
   Thanks again
 

[Dovecot] Fatal: postmaster_address setting not given

[Richard Edmonds]

Hi,
I'm getting: "Fatal: postmaster_address setting not given" errors in my 
log file

dovecot ver:  1.2.12
Recently I tried to follow: 
http://postfixmail.com/blog/index.php/postfixadmin-on-ubuntu-9-10/ on 
ubuntu 10.10

(which works fine on ubunto 9.10)

I note that in 10.10 their is no dovecot-postfix.conf so I made the 
appropriate changes to dovecot.conf instead and changed master.cf 
dovecot transport line since I saw someone recently had the same problem:


dovecot   unix  -   n   n   -   -   pipe
  flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -c 
/etc/dovecot/dovecot.conf -f ${sender} -d ${recipient}



I still have that problem however. Authentication seems ok but delivery 
fails.

Any possibility of help on this one?
vecot   unix  -   n   n   -   -   pipe
     flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -c
   /etc/dovecot/dovecot.conf -f ${sender} -d ${recipient}
   
   
   I still have that problem however. Authentication seems ok but
   delivery fails.
   Any possibility of help on this one?
 

Re: [Dovecot] Best way to move mail to an archive

[Richard C. Cox]

Thank you!  Very nice script.  Exactly what I was looking for!

> On Sat, 2010-12-18 at 12:21 -0500, Richard C. Cox wrote:
> > I'd like to archive e-mail as it ages into a
> > single IMAP archive directory.  Ideally, I would like to set up a cron
> > job to search for e-mail in all of my imap directories that older that
> > 'X' days and move it to the archive directory.  I know that dovecot
> > wants to manage e-mail movement thru the IMAP interface, so I'm leery of
> > just using ordinary Linux commands to move mail around and bypass the
> > IMAP protocol.  Does anybody have any recommendations about dovecot/IMAP
> > safe commands I could use to accomplish this task?
> 
> Personally, I just do it over IMAP.
> 
> http://david.woodhou.se/archivemail.sh