Re: Check outgoing emails not using TLS
On Tue, Apr 07, 2020 at 07:06:41PM -0400, Wietse Venema wrote: > Attached are an updated script, and a diff. Looks good to me. -- Viktor.
Re: Check outgoing emails not using TLS
Viktor Dukhovni:
> On Tue, Apr 07, 2020 at 11:46:33AM +0200, Michael Storz wrote:
>
> > > I should perhaps mention that the "tlstype.pl" Perl script does not
> > > handle TLS connection re-use. I've not looked at what it would take
> > > to do that.
> >
> > And it does not work for mixed-case hostnames:
> >
> > - TLS connection established to lower-case-hostname
> > - relay=mixed-case-hostname
>
> Ah, thanks. Easily enough corrected, by wrapping Perl values in
> lc($value). If you fix this and more issues, feel free to put it up on
> github somewhere... I am not planning to become a "maintainer" of this
> off-the-cuff script.
Attached are an updated script, and a diff.
Wietse
#! /usr/bin/env perl
use strict;
use warnings;
local $/ = "\n\n";
while (<>) {
my $qid;
my %tls;
my $smtp;
foreach my $line (split("\n")) {
if ($line =~ m{ postfix(?:\S*?)/qmgr\[\d+\]: (\w+): from=<.*>,
size=\d+, nrcpt=\d+ [(]queue active[)]$}) {
$qid //= $1;
next;
}
if ($line =~ m{ postfix(?:\S*?)/smtp\[(\d+)\]: (\S+) TLS connection
established to (\S+): (.*)}) {
$tls{$1}->{lc($3)} = [$2, $4];
next;
}
if ($line =~ m{.*? postfix(?:\S*?)/smtp\[(\d+)\]: (\w+): (to=.*),
relay=(\S+), (delay=\S+, delays=\S+, dsn=2\.\S+, status=sent .*)}) {
next unless $qid eq $2;
if (defined($tls{$1}->{lc($4)}) && ($tls{$1}->{lc($4)}->[2] //= $5)
eq $5) {
printf "qid=%s, relay=%s, %s -> %s %s\n", $qid, lc($4), $3,
@{$tls{$1}->{lc($4)}}[0..1];
} else {
delete $tls{$1};
printf "qid=%s, relay=%s, %s -> cleartext\n", $qid, lc($4), $3;
}
}
}
}
--- tlstype.pl-posted 2020-04-07 18:13:50.0 -0400
+++ tlstype.pl 2020-04-07 18:59:44.0 -0400
@@ -15,16 +15,16 @@
next;
}
if ($line =~ m{ postfix(?:\S*?)/smtp\[(\d+)\]: (\S+) TLS connection
established to (\S+): (.*)}) {
- $tls{$1}->{$3} = [$2, $4];
+ $tls{$1}->{lc($3)} = [$2, $4];
next;
}
if ($line =~ m{.*? postfix(?:\S*?)/smtp\[(\d+)\]: (\w+): (to=.*),
relay=(\S+), (delay=\S+, delays=\S+, dsn=2\.\S+, status=sent .*)}) {
next unless $qid eq $2;
- if (defined($tls{$1}->{$4}) && ($tls{$1}->{$4}->[2] //= $5) eq $5) {
- printf "qid=%s, relay=%s, %s -> %s %s\n", $qid, $4, $3,
@{$tls{$1}->{$4}}[0..1];
+ if (defined($tls{$1}->{lc($4)}) && ($tls{$1}->{lc($4)}->[2] //= $5)
eq $5) {
+ printf "qid=%s, relay=%s, %s -> %s %s\n", $qid, lc($4), $3,
@{$tls{$1}->{lc($4)}}[0..1];
} else {
delete $tls{$1};
- printf "qid=%s, relay=%s, %s -> cleartext\n", $qid, $4, $3;
+ printf "qid=%s, relay=%s, %s -> cleartext\n", $qid, lc($4), $3;
}
}
}
Re: Check outgoing emails not using TLS
Viktor Dukhovni: > On Tue, Apr 07, 2020 at 11:46:33AM +0200, Michael Storz wrote: > > > > I should perhaps mention that the "tlstype.pl" Perl script does not > > > handle TLS connection re-use. I've not looked at what it would take > > > to do that. > > > > And it does not work for mixed-case hostnames: > > > > - TLS connection established to lower-case-hostname > > - relay=mixed-case-hostname > > Ah, thanks. Easily enough corrected, by wrapping Perl values in > lc($value). If you fix this and more issues, feel free to put it up on > github somewhere... I am not planning to become a "maintainer" of this > off-the-cuff script. Also I'd be happy to bundle an uipdated version under $postfix/auxiliary. Wietse
Re: Check outgoing emails not using TLS
On Tue, Apr 07, 2020 at 11:46:33AM +0200, Michael Storz wrote: > > I should perhaps mention that the "tlstype.pl" Perl script does not > > handle TLS connection re-use. I've not looked at what it would take > > to do that. > > And it does not work for mixed-case hostnames: > > - TLS connection established to lower-case-hostname > - relay=mixed-case-hostname Ah, thanks. Easily enough corrected, by wrapping Perl values in lc($value). If you fix this and more issues, feel free to put it up on github somewhere... I am not planning to become a "maintainer" of this off-the-cuff script. -- Viktor.
Re: modifying outbound email headers
Wietse Venema wrote: > One correction: > > > Anyway, if you can identify all the the mail2news gateways AND your users > > > must use your server set up a transport map: > > > > > > /etc/postfix/main.cf > > > transport_maps = hash:/etc/postfix/transport > > > > > > /etc/postfix/transport: > > > mail2n...@axample.com newsgateway: > > > mail2news@other.example newsgateway: > > > > > > (or use a pcre: map if you want broader coverage). > > > > > > /etc/postfix/master.cf: > > > newsgateway unix - - n - - smtp > > > -o header_checks=pcre:/etc/postfix/news-header-checks.pcre > > That should be smtp_header_checks (make changes while delivering), > not header_checks (make changes while receiving). Thanks for the correction! Best regards Stefan -- Signal (Desktop) +4915172173279 https://keybase.io/stefan_claas
Re: modifying outbound email headers
One correction: > > Anyway, if you can identify all the the mail2news gateways AND your users > > must use your server set up a transport map: > > > > /etc/postfix/main.cf > > transport_maps = hash:/etc/postfix/transport > > > > /etc/postfix/transport: > > mail2n...@axample.com newsgateway: > > mail2news@other.example newsgateway: > > > > (or use a pcre: map if you want broader coverage). > > > > /etc/postfix/master.cf: > > newsgateway unix - - n - - smtp > > -o header_checks=pcre:/etc/postfix/news-header-checks.pcre That should be smtp_header_checks (make changes while delivering), not header_checks (make changes while receiving). Wietse > > Then populate news-header-checks.pcre with all the patterns that > > you need to modify or discard information. It will of course break > > DKIM signatures so be aware of that in your DMARC policies. > > > > Wietse > > Thanks a lot for your help, much appreciated! > > Best regards > Stefan > > -- > Signal (Desktop) +4915172173279 > https://keybase.io/stefan_claas > >
Re: modifying outbound email headers
Wietse Venema wrote: > Stefan Claas: > > Wietse Venema wrote: > > > > > Stefan Claas: > > > > Hi, > > > > > > > > my postfix mail server works perfectly so far. > > > > > > > > However, I am now facing the following problem and have tried as best > > > > as I can to find a solution to this. > > > > > > > > I run an anonymous remailer, which also allows sending emails to > > > > mail2news gateways for Usenet postings. So far so good. > > > > > > > > I would like to achieve the following: > > > > > > > > postfix should modify outgoing email headers that *only* go to mail2news > > > > gateways, using the email gateway addresses for parsing, so that the > > > > right part of the message ID, after the @ charachter, will be modified > > > > with a defined string. > > > > > > Do it in the mail2news gateway. > > > > Oh ... I would have expected a different answer from you. > > > > Like I previously replied in this thread, users are using often multiple > > mail2news gateways when posting messages to Usenet and this would then > > result in different message-ids for the same posting. > > Who said that users can only use your Postfix system to post > to news gateways? Nobody, sorry if it sounded like that! > Anyway, if you can identify all the the mail2news gateways AND your users > must use your server set up a transport map: > > /etc/postfix/main.cf > transport_maps = hash:/etc/postfix/transport > > /etc/postfix/transport: > mail2n...@axample.com newsgateway: > mail2news@other.example newsgateway: > > (or use a pcre: map if you want broader coverage). > > /etc/postfix/master.cf: > newsgateway unix - - n - - smtp > -o header_checks=pcre:/etc/postfix/news-header-checks.pcre > > Then populate news-header-checks.pcre with all the patterns that > you need to modify or discard information. It will of course break > DKIM signatures so be aware of that in your DMARC policies. > > Wietse Thanks a lot for your help, much appreciated! Best regards Stefan -- Signal (Desktop) +4915172173279 https://keybase.io/stefan_claas
Re: modifying outbound email headers
Stefan Claas: > Wietse Venema wrote: > > > Stefan Claas: > > > Hi, > > > > > > my postfix mail server works perfectly so far. > > > > > > However, I am now facing the following problem and have tried as best as I > > > can to find a solution to this. > > > > > > I run an anonymous remailer, which also allows sending emails to mail2news > > > gateways for Usenet postings. So far so good. > > > > > > I would like to achieve the following: > > > > > > postfix should modify outgoing email headers that *only* go to mail2news > > > gateways, using the email gateway addresses for parsing, so that the right > > > part of the message ID, after the @ charachter, will be modified with a > > > defined string. > > > > Do it in the mail2news gateway. > > Oh ... I would have expected a different answer from you. > > Like I previously replied in this thread, users are using often multiple > mail2news gateways when posting messages to Usenet and this would then > result in different message-ids for the same posting. Who said that users can only use your Postfix system to post to news gateways? Anyway, if you can identify all the the mail2news gateways AND your users must use your server set up a transport map: /etc/postfix/main.cf transport_maps = hash:/etc/postfix/transport /etc/postfix/transport: mail2n...@axample.com newsgateway: mail2news@other.example newsgateway: (or use a pcre: map if you want broader coverage). /etc/postfix/master.cf: newsgateway unix - - n - - smtp -o header_checks=pcre:/etc/postfix/news-header-checks.pcre Then populate news-header-checks.pcre with all the patterns that you need to modify or discard information. It will of course break DKIM signatures so be aware of that in your DMARC policies. Wietse
Re: MTA-STS?
David Mehler: > Hello, > > I just heard about this and started reading on it. Is MTA-STS > something Postfix works with? https://www.google.com/search?q=postfix+mta-sts This uses the Postfix's smtp_tls_policy_maps plugin. Wietse
Re: MTA-STS?
On Tuesday, April 7, 2020 11:15:27 AM EDT David Mehler wrote: > Hello, > > I just heard about this and started reading on it. Is MTA-STS > something Postfix works with? You need https://github.com/Snawoot/postfix-mta-sts-resolver and then yes. Scott K
Re: Tracking email status - Bounced, Deferred, Over Quota, Delivered
Owais Khan: > Thanks > > I knew this way of redirecting output. I thought it would produce me some > statistics like bounced counts, etc. > > But, it seems to bring chunks from original log. Is this the expected output > of this collate.pl? Yes. The purpose of this script is to group logfile records for the same mail delivery transaction. This will help you to find out what happened with the messages that are the subject of your investigation. If the remote SMTP server accepted email, then Postfix will log "status=sent" and the remote SMTP server response will contain an identifier that the server's administrators can use in further investigations. Examples: Apr 6 05:09:40 spike postfix/smtp[14214]: 48wl9F4n0dzJrNs: to=, relay=gmail-smtp-in.l.google.com[172.217.197.26]:25, delay=9.3, delays=6.9/0.018/1.1/1.3, dsn=2.0.0, status=sent (250 2.0.0 OK 1586164180 s21si9838137qte.81 - gsmtp) Apr 6 09:10:04 spike postfix/smtp[17719]: 48wrVh2cdHzJrNt: to=, relay=mail.cloud9.net[2604:8d00:0:1::7]:25, delay=0.24, delays=0.13/0.026/0.068/0.016, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 8DED4336382) If the remote SMTP server rejected email, then Postfix will log "status=bounced" or "status=deferred" and the remote SMTP server response will say why. Wietse
MTA-STS?
Hello, I just heard about this and started reading on it. Is MTA-STS something Postfix works with? Thanks. Dave.
Re: Tracking email status - Bounced, Deferred, Over Quota, Delivered
Thanks I knew this way of redirecting output. I thought it would produce me some statistics like bounced counts, etc. But, it seems to bring chunks from original log. Is this the expected output of this collate.pl? Thanks & Regards, Owais. -- Sent from: http://postfix.1071664.n5.nabble.com/Postfix-Users-f2.html
Re: Check outgoing emails not using TLS
Am 2020-04-06 23:53, schrieb Viktor Dukhovni: On Mon, Apr 06, 2020 at 02:53:25PM +0100, Dominic Raferd wrote: > whose output you'd send to the attached Perl script. On my system for > example: > > # bzip2 -dcf $(ls -tr /var/log/maillog*) | perl collate | perl tlstype.pl I should perhaps mention that the "tlstype.pl" Perl script does not handle TLS connection re-use. I've not looked at what it would take to do that. And it does not work for mixed-case hostnames: - TLS connection established to lower-case-hostname - relay=mixed-case-hostname We should perhaps consider logging some indication of TLS in the core delivery summary line: postfix/smtp: : to=<...>,[ orig_to=<...>,] relay=..., [ tls=:(Anonymous|Untrusted|Verified),] that is, perhaps just the security level and verification status? Collating the data from the logs is tricky, and likely more so with connection reuse (but perhaps not too bad, exercise for the reader...). Regards, Michael
