smtpd / mail.lmtp rewrites from address, breaking lmtp

2019-06-26 Thread Michal Krzysztofowicz
Hi All,

I’ve been using OpenSMTPD on OpenBSD to run my email server since at least 
2015, and this setup has been working really well for me since. 

I have recently upgraded my SMTP server from OpenBSD 6.3, via 6.4 to 6.5 (I 
know I dragged my feet on this one, but I was worried about breaking my email 
service - there are a few people using this server and relying on it quite a 
lot). Anyway, it turns out the changes to the config brought on by the new 
syntax were easier than I initially expected and it seems all is good again. 

The one issue I do have is with incoming email, sent from an address which 
contains a single apostrophe (‘) in the local-part of the address - e.g. 
. 

It seems OpenSMTPD rewrites the from email address by replacing characters it 
deems insecure with a colon (:). So the example address above becomes 
. 

Now, the issue is that, while the ‘:’ as well as “‘“ are both allowed 
characters in the local part, the ‘:’ is only allowed when the local-part is 
enclosed in double quotes - i.e.  is VALID, 
 is INVALID, and <“first.o:last"@example.com> is 
VALID. 

It seems that when the rewrite happens, the local part is not being enclosed in 
the double quotes.

All of the above leads to Dovecot LMTP rejecting the email because of the 
invalid from address. 

The relevant part of the smtpd.conf is below: 

action "lmtp" mda "/usr/libexec/mail.lmtp -f \"%{sender}\" -d /var/dovecot/lmtp 
%{user.username}" virtual 
action "lmtp-local" mda "/usr/libexec/mail.lmtp -f \"%{sender}\" -d 
/var/dovecot/lmtp %{user.username}" alias 

I have also tried using the “syntactic sugar” form: 

action "lmtp" lmtp "/var/dovecot/lmtp" virtual 
action "lmtp-local" lmtp "/var/dovecot/lmtp" alias 

and the result is the same - Dovecot rejects the email. 

I also tried using the :raw modifier to the actions above, like so: 

action "lmtp" mda "/usr/libexec/mail.lmtp -f \"%{sender:raw}\" -d 
/var/dovecot/lmtp %{user.username}" virtual 

but this caused an issue with running mail.lmtp, as now the shell complained 
about the lack of closing single quote character (‘) for emails which contain 
an apostrophe in the local part. 


Whether Dovecot LMTP checking the from email address is a good thing is 
disputable, and I believe they are working on removing this check altogether, 
following the logic that once the SMTP server accepted an email and is happy, 
dovecot should not care, but the current state of play is that they do, and 
OpenSMTPD rewriting the from address seems to be breaking delivery of some 
email. This may not be much of an issue in some places, but in the country I 
live in, it’s quite popular for people to have an apostrophe in their surnames, 
and thus have their email address with the apostrophe. 


To try and pin-point the issue, I ran dovecot with “lmtp_rawlog_dir” option, to 
log each lmtp session as it took place. 

This is what happens when you try and send email, via SMTPD, with the 
apostrophe in the from address:

atlantic:/var/log/dovecot# cat 20190623-211547.66274.1.in
1561324547.779544 LHLO localhost
1561324547.780609 MAIL FROM:

atlantic:/var/log/dovecot# cat 20190623-211547.66274.1.out
1561324547.778817 220 mail.example.org Server Ready.
1561324547.780444 250-mail.example.org
1561324547.780444 250-8BITMIME
1561324547.780444 250-CHUNKING
1561324547.780444 250-ENHANCEDSTATUSCODES
1561324547.780444 250-PIPELINING
1561324547.780444 250-STARTTLS
1561324547.780444 250 VRFY
1561324547.780762 501 5.5.4 Invalid FROM: Invalid character in localpart

with the relevant line in the maillog: 

Jun 14 11:57:34 atlantic smtpd[42606]: 21749fd12ac76b57 mda delivery 
evpid=56aed6237d6444a0 from= 
to= rcpt= user=me delay=0s 
result=PermFail stat=Error ("mail.lmtp: LMTP server error: 501 5.5.4 Invalid 
FROM: Invalid character in localpart")


and this is what happens when I connect to dovecot-lmtp directly and send the 
same email, correctly quoting the rewritten from address:

atlantic:/var/log/dovecot# cat 20190623-221211.18076.3.in
1561327931.935101 LHLO localhost
1561327931.935293 MAIL FROM:<"first.o:last"@example.com>
1561327931.935668 RCPT TO:
1561327931.960386 DATA
1561327931.960854 From: Fist O'Last 
1561327931.960854 To: Postmaster 
1561327931.960854 Subject: This is a test
1561327931.960854
1561327931.960854 this is a test
1561327931.960854 .
1561327931.996700 QUIT

atlantic:/var/log/dovecot# cat 20190623-221211.18076.3.out
1561327931.934775 220 mail.example.org Server Ready.
1561327931.935192 250-mail.example.org
1561327931.935192 250-8BITMIME
1561327931.935192 250-CHUNKING
1561327931.935192 250-ENHANCEDSTATUSCODES
1561327931.935192 250-PIPELINING
1561327931.935192 250-STARTTLS
1561327931.935192 250 VRFY
1561327931.935560 250 2.1.0 OK
1561327931.960207 250 2.1.5 OK
1561327931.960506 354 OK
1561327931.996534 250 2.0.0  y8HBNzv5D12cRgAA9ywqzw Saved
1561327931.996787 221 2.0.0 Bye
atlantic:/var/log/dovecot#


Am I missing something in the configuration, or is this something that needs to 
be fix

Re: Introduction

2019-06-26 Thread Gilles Chehade
On Tue, Jun 25, 2019 at 08:15:22PM -0700, tyler wrote:
> Hey all,
> 

Hey tyler,


> I'm a newbie who started his own email server as a side project. Nothing
> serious or as a job, but as a proverbial "middle finger" to gmail and yahoo,
> etc, and some personal enrichment. I've been a fan of OpenBSD for many years
> now, so OpenSMTPD was the obvious choice. This is email address runs on it,
> on a VPS.
> 

\o/


> I've really enjoyed the experience. It didn't take long to set up and after
> tinkering with my domain's DNS records seems to work flawlessly, except for
> unexplained Proofpoint blocks.
> 

These happen even to the best of us.


> Hope you all are having a wonderful day.
> 

Enjoy your stay around,

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles

-- 
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org