Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting KatolaZ (kato...@freaknet.org):

> I might have missed something, but there is no need to speculate: it's
> sufficient to have a look at the headers of emails posted on DNG. Most
> of them do not have a Reply-To: header set, which means that Mailman
> does not add anything (there is a config option to do that, but it's
> quite annoying TBH).

Bless you for trying to help, Enzo, but I think you've addressed an
entirely different question from the one Hendrik just raised.

Hendrik was hoping that Mailman's DMARC mitigation would not discard a
poster's existing Reply-To: header in cases where Mailman appends one of
its own pointing back to the poster.  In reply, I was citing reasons to
expect it will not discard such a header.


As I've already explained, Dng's Mailman configuration has been, since
December 6th, munging the From: header in cases where the poster's
domain has a DMARC policy of p=reject or p=quarantine, and _only_ in 
such cases.  For your reference, in Mailman v. 2.1.18 and later admin
webUI screens (Dyne.org's current version being 2.1.23), this is on
page Privacy Options, Sender Filters, item 'Action to take when anyone
posts to the list from a domain with a DMARC Reject/Quarantine Policy'
aka dmarc_moderation_action.  Applying the mitigation Mailmna's
developers recommend consists of changing default radio button 'Accept'
to "Munge From'.

And then one also changes 'dmarc_quarantine_moderation_action)' to Yes.

Mailman's developers no longer recommend the similar-looking setting
'Replace the From: header address with the list's posting address to
mitigate issues stemming from the original From: domain's DMARC or
similar policies' aka from_is_list on page General Options:  Opting for
that (older)  version of Mailman's munging kludge unconditionally
applies it to all postings whether they are from DMARC-encumbered
domains or not.

You may or may not recall that we discussed all of these matters on
Devuan-dev in a couple of threads starting October 29th.  It took until
December 6th for action to be taken, but we went through the matter a
few times in discusison.


___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[KatolaZ]

On Sun, Dec 30, 2018 at 01:18:18PM -0800, Rick Moen wrote:
> Quoting Hendrik Boom (hend...@topoi.pooq.com):
> 
> > I hope it doesn't add a reply-to header if there's already one.
> 
> I can't say for certain, but it's likely GNU Mailman would (in that
> case) add the poster's real address as an additional address within the
> existing Reply-To: address (if they even differ).  
> 

I might have missed something, but there is no need to speculate: it's
sufficient to have a look at the headers of emails posted on DNG. Most
of them do not have a Reply-To: header set, which means that Mailman
does not add anything (there is a config option to do that, but it's
quite annoying TBH).

My2Cents

KatolaZ

-- 
[ ~.,_  Enzo Nicosia aka KatolaZ - Devuan -- Freaknet Medialab  ]  
[ "+.  katolaz [at] freaknet.org --- katolaz [at] yahoo.it  ]
[   @)   http://kalos.mine.nu ---  Devuan GNU + Linux User  ]
[ @@)  http://maths.qmul.ac.uk/~vnicosia --  GPG: 0B5F062F  ] 
[ (@@@)  Twitter: @KatolaZ - skype: katolaz -- github: KatolaZ  ]


signature.asc
Description: PGP signature
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Hendrik Boom (hend...@topoi.pooq.com):

> I hope it doesn't add a reply-to header if there's already one.

I can't say for certain, but it's likely GNU Mailman would (in that
case) add the poster's real address as an additional address within the
existing Reply-To: address (if they even differ).  

I thus speculate because tat's the heuristic I see modern versions of
GNU Mailman follow when the listadmin has enabled conventional
(non-DMARC related) Reply-To: munging (the type infamous for triggering
Internet flamewars, the munging practice obsoleted by IETF in 2001 via
RFCs 2822 and 2369, but still popular on mailing lists catering to
technophobes).  So, I would expect Mailman's DMARC-mitigation Reply-To:
addition to use the same code.

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Hendrik Boom]

On Wed, Dec 26, 2018 at 08:06:52PM +, Simon Hobson wrote:
> 
> What they've down with the list (and I've seen it with other lists 
> too) is: if the mail matches some criteria, then the originator's 
> address is replaced with the list address and a reply to header is 
> added. Thus for those users on a broken mail system (such as gmail, 
> or hotmail, or ... they still get the list emails instead of not 
> seeing mails from some proportion of list users.
> The downside is what you see.

I hope it doesn't add a reply-to header if there's already one.

-- hendrik

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Hendrik Boom]

On Thu, Dec 27, 2018 at 08:49:32PM -0800, Rick Moen wrote:
> Quoting Hendrik Boom (hend...@topoi.pooq.com):
> 
> > I *hate* the way that the Google Groups mailing lists refuse to 
> > include the mailing-list headers so you cannot just 
> > reply-to-list.
> 
> *glyph of surprise*
> 
> Are you sure?  I just checked a recent example, and List-Post is there
> and completely correct and appropriate.  That is the signifier dictated
> by RFC 2369 section 3.4 as the method for posting.  (In atypical cases, the
> software administrator might cause it to point to a moderator, or to
> some other location for submission, instead of to the mailing list's
> address.  In the specific atypical case of a mailing list that does not
> allow posting, e.g., an announcements list, the List-Post field may
> contain the special value 'NO'.)

That's news!  I'll have to try it out!  I know it didn't work a while ago.

-- hendrik
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Hendrik Boom (hend...@topoi.pooq.com):

> I *hate* the way that the Google Groups mailing lists refuse to 
> include the mailing-list headers so you cannot just 
> reply-to-list.

*glyph of surprise*

Are you sure?  I just checked a recent example, and List-Post is there
and completely correct and appropriate.  That is the signifier dictated
by RFC 2369 section 3.4 as the method for posting.  (In atypical cases, the
software administrator might cause it to point to a moderator, or to
some other location for submission, instead of to the mailing list's
address.  In the specific atypical case of a mailing list that does not
allow posting, e.g., an announcements list, the List-Post field may
contain the special value 'NO'.)

Selected headers from the example posting:

Date: Fri, 21 Dec 2018 09:13:32 -0800 (PST)
From: goossbears 
To: BerkeleyLUG 
Message-Id: 
Reply-To: berkeley...@googlegroups.com
Mailing-list: list berkeley...@googlegroups.com; contact
berkeleylug+own...@googlegroups.com
List-ID: 
X-Google-Group-Id: 61884646931
List-Post: ,

List-Help: ,

List-Archive: ,

List-Unsubscribe: 
,



In this particular case, the listadmin has also made the regrettable choice
of munging Reply-To: to cater to technophobes[1] -- but that is a different
matter, and also not dictated AFAIK by Google.


[1] http://marc.merlins.org/netrants/reply-to-still-harmful.html
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Hendrik Boom]

On Thu, Dec 27, 2018 at 10:17:41PM +, Simon Hobson wrote:

> 
> Nice thought, but do you really think that the likes of Google give a sh*t 
> about some little mailing list somewhere, and which should be using Google's 
> services anyway - how dare they use their own solution !
> The reality is that the "big boys" have implemented these breakages - they 
> knew beforehand that they would break almost all forms of forwarding, but 
> their solution to that "problem" was simply to declare any form of mail 
> forwarding as "improper" and therefore breaking it wasn't their fault. I 
> can't help thinking that their marketing people saw an opportunity to make 
> life harder for small scale competitors.

I *hate* the way that the Google Groups mailing lists refuse to 
include the mailing-list headers so you cannot just 
reply-to-list.

-- hendrik
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Miles Fidelman (mfidel...@meetinghouse.net):

> Ahh... missed that.  Didn't really notice anything until this huge
> string of emails.  Sigh...

Eh, no worries.  I half-realised that's what probably happened.

[publishing SPF & DMARC/DKIM records in DNS for a mailing list host:]

> True, but it sometimes helps.  And it's easy enough if one has
> access to one's nameserver records, as anyone who runs a list
> manager usually does.

Just as a matter of personal perspective/opinion:  I watched the
introduction of DKIM (né DomainKeys) by Yahoo and considered it so
botched that I wanted nothing to do with it.  When Yahoo extended DKIM
to create DMARC, it seemed to me Yahoo had learned nothing from the
DomainKeys/DKIM experience and screwed up a second time.  

By contrast, all of the complaints against SPF (the real ones, not the
bullshit non-sequitur complaints like 'SPF doesn't block spam' and
'spamhaus domains can and do publish SPF records) divide neatly into two
categories:

1.  I object to /etc/aliases and ~/.forward breaking and refuse to use
SRS in their entries.  (Dude, wrong decade.)

2.  I want to be free to originate outgoing SMTP from arbitrary
not-previously-planned IP addresses and not have it be suspected of
forgery.  (Dude #2, good luck with that.  Also, still the wrong decade.)

Both factions kept advising me it's Bad and Wrong for me to publish an
SPF record saying 'Please reject as forged any mail purporting to be
from my domains that isn't from IP address 198.144.195.186', to which I
always responded 'Why the Gehenna is that _bad_?  It's exactly what I want
to happen, because all mail genuinely from my domain comes from my IP.
If users, even those who have shell on my machine, are forging my domain
from other IPs, I _want_ that mail to fail as forged, because it's
actually forged, and users should not try to do that.'

Anyway, as far as I'm aware, nobody is distrusting mail legitimately
from my domains for lack of DMARC attestation.  I keep asking people
suggesting DMARC what demonstrable benefit my domains would get that
they don't already get from a very clear and emphatic SPF policy, and
nobody's yet given me a compelling answer.

If things change and I _do_ see signs of penalising domains with
emphatic SPF policies but no DKIM/DMARC, then I'll reconsider.

(Above speaks, obviously, just for me and my domains.  I'm not part of
the Dyne/Dng administration team, just a friendly Devuan sysadmin and
recently-Debian-leaning Operations guy.)

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Miles Fidelman]

On 12/27/18 11:07 AM, Rick Moen wrote:


Quoting Miles Fidelman (mfidel...@meetinghouse.net):


Speaking as someone who hosts a couple of dozen email lists, I
really don't understand what the fuss is about here.

The fuss involved people having paid no attention to the announcement of
Dng's DMARC-mitigation munging starting on December 6th, and so being
confused by that and the appended Reply-To.  And some people, including
you, still aren't getting that, even though it got re-explained
yesterday, encore une fois.



Ahh... missed that.  Didn't really notice anything until this huge 
string of emails.  Sigh...






If one runs a list, and wants folks on gmail, AOL - any service that
honors p=reject - then one has to:

1. adjust headers so that list mail appears to originate from the
list manager, not from the original author

Yes.  But only where p=reject or p=quarantine..



Yes, indeed.




2. publish DKIM & SPF records for the machine hosting the list

No, there's no obligation for either of those, to ensure deliverability
of mail munged by the MLM software for DMARC-mitigation purposes.



True, but it sometimes helps.  And it's easy enough if one has access to 
one's nameserver records, as anyone who runs a list manager usually does.






Updating the mailman settings, and publishing the appropriate DNS
records, is really a no brainer for any halfway competent list
administrator.

Indeed, the Dng listadmins did exactly that, on Dec. 6th, following
instructions I provided.  The whole thing was covered here at the time,
so I'm puzzled that so many people seem to have failed completely to
read the explanations.



Sorry to add to the noise - there was just so much of it.  My bad.

Cheers,

Miles

--
In theory, there is no difference between theory and practice.
In practice, there is.   Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Simon Hobson (li...@thehobsons.co.uk):

> Perhaps I'm missing something, but doesn't SRS provide a gaping wide
> chasm for spammers to pile through ?

I would call _gaping_ chasm an exaggeration -- but it is certainly
abusable (to the extent cross-domain aliases become known or
discoverable in public).  

Someone trying to send Don Marti spam via alias 'd...@linuxmafia.com'
will implicitly rope my linuxmafia.com MTA (mail transfer agent = SMTP
daemon) into the evil task of pumping spam delivery attempts at Don's
zgp.org MTA, a regrettable case of 'Let's you and him fight' -- which is
why I've just now permanently disabled (now that I remembered the
problem) all cross-domain /etc/aliases entries.  (I've retained
intradomain aliases, such as ones that send root@, postmaster@, abuse@,
and hostmaster@ to the appropriate user mail spool.)

Relevant to this picture is the _other_ difference between MLMs (mailing
list managers) and other SMTP mail reflectors:  MLMs are _smarter_,
giving opportunities to reject or sequester abusive mail patterns the
other reflector types cannot.  E.g., even by default, GNU Mailman will
intercept and hold or reject mail with too many recipients, overly large
mail, mail implicitly addressed (mailing list address specified only in
Bcc), and a number of other similar heuristics.  Also, in this decade,
almost no mailing lists pass through without review mail from
non-subscribed addresses -- and spammers have still shied away from
making their spambots go through 3-way confirmation to join mailing
lists before trying to post to them.

So, yeah, you have a point, and I thank you for the reminder that I was
long overdue to disable those aliases.

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Simon Hobson]

Rick Moen  wrote:

> Back in the day, I gave out /etc/aliases entries to friends that
> leveraged the 'mafia' theme of my linuxmafia.com domain,

In our case it was simple alias entries ina  database queried by Postfix - but 
same effect and same problem.

> SRS (sender rewriting scheme) was SPF creator Meng Wong's kludge for
> salvaging /etc/alias and ~/.forward (when used cross-domain) from
> unintended collateral SPF damage.

Perhaps I'm missing something, but doesn't SRS provide a gaping wide chasm for 
spammers to pile through ? It always seemed to me a bit like server C getting a 
header that's been re-written in scuh a manner by server B that server C is 
expected to accept it as though server B is pinkie swearing that the forwarded 
mail is genuine and did come from server A. Or more precisely, server B 
effectively saying "this message from some other domain, well pretend it's 
coming from my domain"- so all a spammer has to do is forge (in a correct 
manner) the re-written from address and the spam bypasses SPF.
I guess that's why DKIM etc came along.

> Wong provided a Perl wrapper script to rewrite the SMTP envelope on the 
> outbound copy, emulating what MLMs do.

it was a few years ago now, so details are "a bit fuzzy" to say the least. In 
our case using Postfix, it needed some plugin to do it - and I think this 
plugin re-wrote all addresses regardless of where the email was headed. Due to 
the way the two services were done, the greylisting (part of policyd, aka 
Cluebringer) was done on the re-written address, and since this (IIRC) changed 
each day then few emails ever got the "seen this triplet before, straight 
through" treatment and so nearly all mail was delayed. Funny how users get to 
expect "instant" email even though there's never ever been any guarantee of 
instant delivery :-/

But at least my service did something that apparently the likes of Google and 
Microsoft couldn't manage - I did not have to silently delete mail that failed 
spam or embedded nasties checks. I rejected the messages so that any properly 
configured server would notify the sender that the message wasn't delivered. I 
was always proud of that bit.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Simon Hobson (li...@thehobsons.co.uk):

> Correction noted.

I truly appreciate your hearing it in the spirit intended.  Thank you,
Simon.  (We'll want to wind this up soon, as it has little to do with
Devuan, and discussion has veered away from Dng's adoption of Mailman
DMARC-mitigation munging starting 2018-12-06.)

> However, in my defence my issues (which I no longer have to deal with)
> were with mail forwarding in servers rather than mailing lists (IIRC
> our mailing list hosting had dwindled to just a couple of announce
> lists before the problem raised it's head) - so a different set of
> related issues which was primarily SPF at the time. I did get as far
> as having a look at SRS - but unfortunately the plugin for Postfix was
> incompatible with the greylisting I used due to the order of
> operations which prevented whitelisting of "known" greylisting
> triplets.

I think I know/remember a little bit about this.

Not all forwarding is alike.  MLM (mailing list manager) forwarding, the
operation where the MLM retransmits a received post to each subscriber,
involves writing an entirely new SMTP envelope on the outgoing
subscriber copies.  Other forwarding mechanisms such as /etc/aliases and
~/.forward entries do _not_.  Those just hurl the received message back
out with envelope unchanged.  (SRS was a kludge proposed for non-MLM
forwarders on account of this difference to help them preserve SPF
validity, a matter I'll return to shortly.).

Back in the day, I gave out /etc/aliases entries to friends that
leveraged the 'mafia' theme of my linuxmafia.com domain, e.g.,
'c...@linuxmafia.com' reaches Chris di Bona, then a co-worker at VA
Linux Systems and now Linux Community Manager at Google.
'd...@linuxmafia.com' was a natural fit as an alias for _Linux Journal_
editor Don Marti's personal mailbox dma...@zgp.org.  And so on.
However, following wide adoption of aggressive hardfail SPF policies,
those and all other cross-domain /etc/aliases entries more-or-less broke
(well, became selectively unreliable, depending on the sending domain),
because any mail transiting the alias would arrive at the other-domain
end-destination unable to pass SPF scrutiny for the claimed sending
domain, which in turn was because my MTA at linuxmafia.com wasn't in the
sending domain's SPF-published list of authorised sending IPs.

SRS (sender rewriting scheme) was SPF creator Meng Wong's kludge for
salvaging /etc/alias and ~/.forward (when used cross-domain) from
unintended collateral SPF damage.  Wong provided a Perl wrapper script
to rewrite the SMTP envelope on the outbound copy, emulating what MLMs
do.

At the time, I couldn't be bothered reimplementing all of those
cross-domain /etc/aliases entries using an SRS wrapper, so they have
simply become not-very-reliable reflectors, and what I tell users is 
'/etc/aliases and ~/.forward are no longer best practices for
cross-domain mail redirection, unless you're willing to do more work
than I personally am volunteering for.'

But the point is that MLM-redirection, by contrast, never had that
problem because of its smarter way of handling the envelope header.

In hindsight, SRS-wrapping seems like small potatoes compared to the
order-of-magnitude-greater hassle of DKIM and DMARC (but I personally
elect to eschew all three).

-- 
Cheers,"He who laughs last, lasts."
Rick Moen   -- Leo Rosten
r...@linuxmafia.com
McQ! (4x80)
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Simon Hobson]

Rick Moen  wrote:

> Simon, I appreciate your pitching in to attempt to answer this question.
> A few necessary corrections, though:

Correction noted. However, in my defence my issues (which I no longer have to 
deal with) were with mail forwarding in servers rather than mailing lists (IIRC 
our mailing list hosting had dwindled to just a couple of announce lists before 
the problem raised it's head) - so a different set of related issues which was 
primarily SPF at the time. I did get as far as having a look at SRS - but 
unfortunately the plugin for Postfix was incompatible with the greylisting I 
used due to the order of operations which prevented whitelisting of "known" 
greylisting triplets. Customised solutions were beyond my skill set - not to 
mention, the issues of leaving a maintenance time-bomb for any admin taking 
over*.

* When I left, a host developed a hardware issue. There was enough spare 
capacity to simply move the VM to another host - a few hours to copy the mail 
folders. Instead the know it all in charge took nearly a week to get something 
working because the concepts were beyond him. It was hard to laugh out load as 
I knew what it would be doing to the customers - many of whom I knew personally 
through having provided support over the years.


Rick Moen  wrote:

> Why messages fail DMARC is convoluted, and I'd frankly rather spend my
> time on other things.  If you are wanting to spend a lot more time on
> this, here's a fine place to start:
> https://wiki.asrg.sp.am/wiki/Mitigating_DMARC_damage_to_third_party_mail

Thanks for that, an interesting site.



Steve Litt  wrote:

> I'd suggest we ban email from gmail, yahoo, protonmail, and the rest
> that demand strict adherence to DMARC.

Nice thought, but do you really think that the likes of Google give a sh*t 
about some little mailing list somewhere, and which should be using Google's 
services anyway - how dare they use their own solution !
The reality is that the "big boys" have implemented these breakages - they knew 
beforehand that they would break almost all forms of forwarding, but their 
solution to that "problem" was simply to declare any form of mail forwarding as 
"improper" and therefore breaking it wasn't their fault. I can't help thinking 
that their marketing people saw an opportunity to make life harder for small 
scale competitors.

From the users' PoV, if a random mailing list or forwarding server doesn't work 
with such broken domains then clearly it has to be the little mailing list or 
forwarding server that's broken. For many years at a previous job we ran a mail 
server for customers - going back to before everyone and his dog were offering 
such services. We always recommended customers to create a second account in 
their mail software to (at a minimum) collect their mail - but many would 
simply refuse to countenance the complication - and instead we had to forward 
"i...@customersdomain.co.uk" to "someobscureaddress24673...@isp.com".
This worked just fine for many years - until that is, the big boys went out and 
broke it.


___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Miles Fidelman (mfidel...@meetinghouse.net):

> Speaking as someone who hosts a couple of dozen email lists, I
> really don't understand what the fuss is about here.

The fuss involved people having paid no attention to the announcement of
Dng's DMARC-mitigation munging starting on December 6th, and so being
confused by that and the appended Reply-To.  And some people, including
you, still aren't getting that, even though it got re-explained
yesterday, encore une fois.

> If one runs a list, and wants folks on gmail, AOL - any service that
> honors p=reject - then one has to:
> 
> 1. adjust headers so that list mail appears to originate from the
> list manager, not from the original author

Yes.  But only where p=reject or p=quarantine..

> 2. publish DKIM & SPF records for the machine hosting the list

No, there's no obligation for either of those, to ensure deliverability
of mail munged by the MLM software for DMARC-mitigation purposes.  

> Updating the mailman settings, and publishing the appropriate DNS
> records, is really a no brainer for any halfway competent list
> administrator.

Indeed, the Dng listadmins did exactly that, on Dec. 6th, following
instructions I provided.  The whole thing was covered here at the time,
so I'm puzzled that so many people seem to have failed completely to 
read the explanations.

> The folks who administer lists.dyne.org just need to do it.

{headdesk}

I'm sorry, but how were the several explanations of this matter,
including those just yesterday, unclear?

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting info at smallinnovations dot nl (i...@smallinnovations.nl):

> So far i have just installed DMARC one time but if i remember it
> correctly either SPF or DKIM had to be correct to accept the e-mail or
> not. To quote my source "A message will fail DMARC if the message fails
> both (1) SPF or SPF alignment and (2) DKIM or DKIM alignment.". DKIM
> would be a hard nut to crack for MLM but SPF should not be a problem
> then? Or do I overlook something?

Why messages fail DMARC is convoluted, and I'd frankly rather spend my
time on other things.  If you are wanting to spend a lot more time on
this, here's a fine place to start:
https://wiki.asrg.sp.am/wiki/Mitigating_DMARC_damage_to_third_party_mail

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Michael]

On Thursday 27 December 2018 05:06:04 am Miles Fidelman wrote:
> Speaking as someone who hosts a couple of dozen email lists, I really
> don't understand what the fuss is about here.
>
> DMARC breaks mailing lists - it's that simple.  It breaks pretty much
> anything that forwards mail.  (FYI:  Early on, it broke the IETF's
> lists.  Rather annoying, that.)
>
> If one runs a list, and wants folks on gmail, AOL - any service that
> honors p=reject - then one has to:
>
> 1. adjust headers so that list mail appears to originate from the list
> manager, not from the original author

1.) is my understanding of what TDE does.  Here are some of the {snipped} TDE 
headers:

Return-Path: 
Envelope-to: {snip}@inet-design.com
Reply-To: trinity-us...@lists.pearsoncomputing.net
From: Michael <{snip}@inet-design.com>
To: trinity-us...@lists.pearsoncomputing.net
X-Get-Message-Sender-Via: srv04.srv04-inet-design.com: authenticated_id: 
inetd/from_h
X-Authenticated-Sender: srv04.srv04-inet-design.com: {snip}@inet-design.com
 
My guess is it's either, or both together, Reply-To or Return-Path that make 
the desired behavior change.  Based upon the email ‘From: chillfan’ here that 
doesn’t work that has in the headers:

Reply-To: chill...@protonmail.com  [1]

instead of no ‘Reply-To:’ at all for messages here that do work.

(Return-Path seems to already be set.)
(Added the X- as TDE seems to authenticate me as a sender, where Devuan 
doesn’t.  No real idea if that’s relevant.)

# # #

As we seem to be way over thinking this

Suggestion:  Just try adding, or replacing if already present, the ‘Reply-To:’ 
for all messages with the value:

Reply-To: dng@lists.dyne.org

That’s not going to break the list, and will answer the question immediately.

Best,
Michael

[1] I would mangle that, but it’s clear text in the original.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Miles Fidelman]

Speaking as someone who hosts a couple of dozen email lists, I really 
don't understand what the fuss is about here.


DMARC breaks mailing lists - it's that simple.  It breaks pretty much 
anything that forwards mail.  (FYI:  Early on, it broke the IETF's 
lists.  Rather annoying, that.)


If one runs a list, and wants folks on gmail, AOL - any service that 
honors p=reject - then one has to:


1. adjust headers so that list mail appears to originate from the list 
manager, not from the original author


2. publish DKIM & SPF records for the machine hosting the list

Both Sympa (which I run) and Mailman have settings that will apply the 
appropriate header changes (Sympa had a community supported patch within 
a week, which was integrated into the next release; Mailman took a bit 
longer, with features showing up in v 2.1).


Updating the mailman settings, and publishing the appropriate DNS 
records, is really a no brainer for any halfway competent list 
administrator.  The folks who administer lists.dyne.org just need to do 
it.  (I can't believe they haven't already - but then I don't notice it 
- we run our own email server, and pointedly don't honor DMARC p=reject 
on incoming mail.)


Miles Fidelman



--

In theory, there is no difference between theory and practice.
In practice, there is.   Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[info at smallinnovations dot nl]

On 27-12-18 03:20, Rick Moen wrote:
>
> {sigh}  Nobody listens.
>
> There is nothing needing a 'fix' (unless you wish to argue with
> operators of domains publishing aggressive DMARC policies (p=reject or
> p=quarantine) and convince them that such is an unwise policy).  In a
> world where DMARC is being rolled out by many domains, mailing lists can
> either attempt to mitigate the DMARC-caused damage, or do nothing and
> let some subscribers and their readers figure out the hard way why
> certain posters aren't being received at certain domains (e.g., GMail)
> and are getting gradually auto-unsubscribed by Mailman on account of
> excessive 'bounce scores'.
>
So far i have just installed DMARC one time but if i remember it
correctly either SPF or DKIM had to be correct to accept the e-mail or
not. To quote my source "A message will fail DMARC if the message fails
both (1) SPF or SPF alignment and (2) DKIM or DKIM alignment.". DKIM
would be a hard nut to crack for MLM but SPF should not be a problem
then? Or do I overlook something?

Grtz.

Nick




signature.asc
Description: OpenPGP digital signature
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Steve Litt]

On Wed, 26 Dec 2018 20:06:52 +
Simon Hobson  wrote:

> Michael  wrote:
> 
> >> Argh.  Sending to the list this time.
> >> 
> >> Please don't set "Reply-to" on list emails.
> >> 
> >> Antony.  
> > 
> > I’m pretty sure the individuals aren’t doing it explicitly.  This
> > list just doesn’t seem to create, or override really, the headers
> > quite right.  Some messages here I hit reply (like this one) and
> > the proper “To: dng@lists.dyne.org” shows up, on others someone’s
> > name is populated in the To: box.  Other lists, you hit reply and
> > To: is always populated correctly.
> > 
> > golinux?, other admins?, is there a config option somewhere in the
> > backend to ‘fix’ this?  
> 
> Unfortunately I think it's one of those things where you have to
> break some stuff to work around the deliberate breakage implemented
> with malice aforethought by many large email providers.
> 
> The problem is SPF, DMARC, and friends. These basically provide
> information about where emails may come from - eg gmail may only come
> from Google's servers. This is a problem for any system that forwards
> email - such as mailing lists and mail servers setup to forward email
> for (say) i...@nicetownplumbers.co.uk to
> ntplumb2458...@someispmail.com.
> 
> So, someone using gmail sends a message to dng@lists.dyne.org which
> is delivered and then forwarded to all the list users. Some of those
> users will be using mail services that check SPF etc - and oh dear,
> there's an email which purports to come from gmail but it's actually
> being sent from a dyne.org server. So it gets discarded as obviously
> spam.

I'd suggest we ban email from gmail, yahoo, protonmail, and the rest
that demand strict adherence to DMARC. A person can pay $18.00/year for
his/her own domain, and that domain will give them at least one email
address, maybe more. If somebody is too cheap to pay $18/year for email
that doesn't require every list they're on to do all sorts of BS that
often leads to an accidental sending to the whole list of an intended
private response, they don't belong here. Eat only beans for four days
and you've probably saved enough to buy good email for the whole year.

On another of my mailing lists, I filtered all DMARC clusterscrewups
to /dev/null. Everyone on DNG is smart enough to understand the value
of a real email address.

SteveT

Steve Litt 
December 2018 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Michael (mb_devuan-mailingl...@inet-design.com):

> I am going to say, that the Trinity Desktop Environment (TDE) Users
> mailing list seems to send from/to ProtonMail without the reply to
> issue the Devuan Users list has.  The list admin, and owner of the
> project, is Timothy Pearson, and his publicly published email is:

If you're saying that the TDE Users mailing list doesn't apply the
Mailman-recommended DMARC mitigation, then you are correct that no
MLM-level munging of the sender's 'From: ' header (or appending of a
Reply-To: header for DMARC-mitigation reasons) will be applied to any
subscriber's mail.

Unfortunately, that would _also_ mean that TDE Users postings from domains
with aggressive DMARC policies will encounter deliverability issues to
receiving domains that enforce DMARC (e.g., to GMail).  And this bad
situation will persist for TDE Users members unles/until the listadmin
applies DMARC mitigations.

> The headers are fairly different, not that I know what specifically ‘fixes’ 
> this issue, but I’ll guess it would hopefully give a clue as to what/where 
> the fix lies.

{sigh}  Nobody listens.

There is nothing needing a 'fix' (unless you wish to argue with
operators of domains publishing aggressive DMARC policies (p=reject or
p=quarantine) and convince them that such is an unwise policy).  In a
world where DMARC is being rolled out by many domains, mailing lists can
either attempt to mitigate the DMARC-caused damage, or do nothing and
let some subscribers and their readers figure out the hard way why
certain posters aren't being received at certain domains (e.g., GMail)
and are getting gradually auto-unsubscribed by Mailman on account of
excessive 'bounce scores'.

-- 
Cheers,"He who laughs last, lasts."
Rick Moen   -- Leo Rosten
r...@linuxmafia.com
McQ! (4x80)

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Simon, I appreciate your pitching in to attempt to answer this question.
A few necessary corrections, though:


Quoting Simon Hobson (li...@thehobsons.co.uk):

> Unfortunately I think it's one of those things where you have to break
> some stuff to work around the deliberate breakage implemented with
> malice aforethought by many large email providers.

Indeed it's a regrettable but small breakage, necessitated by
(specifically) the DKIM cryptographic signing component within DMARC's
omnibus framework.[1]

> The problem is SPF, DMARC, and friends. 

Incorrect.  SPF poses _no_ challenges for mailing lists.  E.g., my
domains have since 2003 published aggressive SPF policies, and encounter
absolutely zero problems with either sent or received mailing list mail 
(to or from my domains) getting spamboxed or rejected on account of
perceived forgery.

$ dig -t txt linuxmafia.com +short
"v=spf1 ip4:198.144.195.186 -all"
$ dig -t txt unixmercenary.net +short
"v=spf1 ip4:198.144.195.186 -all"
$

That hyphen in concluding field '-all' signifies 'Dear receiving SMTP
hosts, please hardfail as conclusively forged any SMTP mail that doesn't
originate in one of the places listed' (the only place listed in this
case being a single IP).  So, given the many mailing lists I host, and
the many mailing lists I subscribe to, if SPF caused a problem for
mailing lists, I would know.

Indeed, if you understand how SPF works, you can figure out that there
is no way that SPF _could_ claim MLM-reflected (mailing list
manager-reflected) mail is forged, because the reflected copy bears a
fresh envelope header from the mailing list's domain, in contrast to
that of the sender's domain it had during the initial hop.

Simon, I really do appreciate your effort to help, but we already had
this conversation on Dng in August 2017.  You confidently made these
same mistakes and I painstakingly corrected the erroneous information, 
so it's rather unfortunate to have to go through this with you yet again.

Would you mind doing me a solid?  Please read up about antiforgery
methods before making similar claims again.


> So, someone using gmail sends a message to dng@lists.dyne.org which is
> delivered and then forwarded to all the list users. Some of those
> users will be using mail services that check SPF etc - and oh dear,
> there's an email which purports to come from gmail but it's actually
> being sent from a dyne.org server. So it gets discarded as obviously
> spam.

As to SPF, this claim is (again) flatly incorrect.

SPF validates (solely) the envelope 'From ' header (used in the SMTP
MAIL FROM operation) against forgery, not the internal 'From:' (etc.)
headers.  So, it does _not_ cause the effect you describe.

> What they've down with the list (and I've seen it with other lists
> too) is: if the mail matches some criteria, then the originator's
> address is replaced with the list address and a reply to header is
> added. Thus for those users on a broken mail system (such as gmail, or
> hotmail, or ... they still get the list emails instead of not seeing
> mails from some proportion of list users.

This is almost but not quite correct.

Retransmission by the mailing list software introduces unavoidable
header and body alterations that cause a message's DKIM cryptographic
signature to be no longer able to be validatable against the sending
domain's DKIM public key.  _If_ the sending domain has published an
aggressive DMARC policy (p=reject or p=quarantine) _and_ the
subscriber's receiving domain enforces sending domains' DMARC policies,
then the subscriber copy will be (respectively) rejected or spamboxed.


> Not sure what the criteria are - whether it's based on there being
> certain headers in the email, whether the sender domain has SPF
> records etc, or what.

Criterion used:  Sender's domain publishes an aggressive DMARC policy
(=reject or p=quarantine).


> One answer is to always use reply to all and then move/remove
> addresses so you just have a single destination of the list address.

I am unclear on what you mean, here, by 'answer' -- but the Dng mailing
list's DMARC-mitigation configuration will categorically munge the
internal 'From:' header (upon retransmission to subscribers) of any posting from
a domain with an aggressive DMARC policy, and (by way of attempt to
compensate for the regrettable but necessary damage) appending a
Reply-To: header pointing back to the poster's real posting address.

The steps you recommend (above) would have no effect whatsoever on
Mailman's DMARC mitigation.


Not intended to complain, but I comprehensively explained Dng's
alterations to do DMARC mitigation on December 6th, when the listadmins
rolled out GNU Mailman's most-recommended DMARC mitigation:
https://lists.dyne.org/lurker/message/20181206.100230.c6157b41.en.html



[1] DMARC is a mammoth superset of SPF and DKIM in which either or both
of those pre-existing antiforgery mechanisms is present along with
checking what is called 'alignment' 

Re: [DNG] Admins can you fix/set the header overrides?

[Michael]

On Wednesday 26 December 2018 02:06:52 pm Simon Hobson wrote:
> The problem is SPF, DMARC, and friends. These basically provide information
> about where emails may come from - eg gmail may only come from Google's
> servers. This is a problem for any system that forwards email - such as
> mailing lists and mail servers setup to forward email for (say)
> i...@nicetownplumbers.co.uk to ntplumb2458...@someispmail.com.

On Wednesday 26 December 2018 03:05:43 pm Rick Moen wrote:
> _Why_, and why (specifically) _your_ mail?  Actually, it's not just
> you, but rather your sending domain, protonmail.com.  Protonmail creates
> a challenge to any mailing list by publishing an aggressive DMARC
> antiforgery policy in its public DNS:

Okay, I'm not disagreeing with either of you on SPF/DMARC/etc.’s inner 
workings, as I have to wrangle those beasts myself for my clients' DNS to 
work correctly.  Literal headache every time.

I am going to say, that the Trinity Desktop Environment (TDE) Users mailing 
list seems to send from/to ProtonMail without the reply to issue the Devuan 
Users list has.  The list admin, and owner of the project, is Timothy 
Pearson, and his publicly published email is:

"Timothy Pearson" 

He's somewhat hard to get a hold of, but I can certainly provide copies of my 
message headers to the TDE users list privately to the admin(s) of this list.  
The headers are fairly different, not that I know what specifically ‘fixes’ 
this issue, but I’ll guess it would hopefully give a clue as to what/where 
the fix lies.

I was also a member of the LEAF user list for near 20 years, I don’t remember 
them ever having the issue either.  I can dig up some old headers (‘17) off 
that list as well if an admin here thinks it would help.

# # #

Admittedly, just being able to always click reply and it ‘just works’ is a 
triviality on any issue scale, but if there’s anything I can do to help get 
there, I’m in, as it’s momentous on my annoyance scale.

Best All,
Michael
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting chillfan via Dng (dng@lists.dyne.org):

> I can confirm that I haven't set a reply-to header, but this is just a
> web mail. It could be that my webmail provider just doesn't allow me
> to unset the header, but I haven't looked that far into it.

Indeed, you didn't set it in this case.


Short version:  It's a mitigation on the mailing list server for the
problem of DMARC.

Your headers as you send them were like this:

  From: chillfan >
  To: "dng@lists.dyne.org" 

Mailman is configured to alter your postings' headers (for
retransmission to all subscribers) as follows:

  From: chillfan via Dng 
  To: "dng@lists.dyne.org" 
  Reply-To: chill...@protonmail.com

_Why_, and why (specifically) _your_ mail?  Actually, it's not just
you, but rather your sending domain, protonmail.com.  Protonmail creates
a challenge to any mailing list by publishing an aggressive DMARC
antiforgery policy in its public DNS:

  $ dig -t txt _dmarc.protonmail.com +short
  "v=DMARC1; p=quarantine; fo=1;"
  $

Because of that aggressive 'p=quarantine' policy, and because Mailman
(like all other MLM = mailing list manager packages) makes changes to
postings (upon retransmission to subscribers) that unavoidably cause the
subscriber copies to fail checks of the message's DMARC cryptographic
signature.  This is a serious problem for mailing lists, causing
retransmitted mail to either be rejected (if 'p=reject') or spamboxed
(if 'p=quarantined') at any receiving domain that enforces DMARC.

Mailman's mitigation (see above example) circumvents the damage from
'p=quarantine' or 'p=reject' policies by substituting the mailing list's
domain as sending domain during retransmission.  It adds a Reply-To as 
described in the above example, in order to preserve the sender's
intended originating address as well as possible under the
circumstances.  Mailman does _not_ apply this mitigation to all
postings, only to ones from domains with p=quarantine or p=reject DMARC
policies (aggressive ones).

Admittedly, the end-result is a bit irksome, but it's the least-bad
solution to the DMARC challenge the Mailman developers have so far come
up with.

(I advised Devuan's mailing list administrators on how to handle the
DMARC problem, which was causing subscribers problems, as is happening
on mailing lists everywhere.)

-- 
Cheers,"He who laughs last, lasts."
Rick Moen   -- Leo Rosten
r...@linuxmafia.com
McQ! (4x80)
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Simon Hobson]

Michael  wrote:

>> Argh.  Sending to the list this time.
>> 
>> Please don't set "Reply-to" on list emails.
>> 
>> Antony.
> 
> I’m pretty sure the individuals aren’t doing it explicitly.  This list just 
> doesn’t seem to create, or override really, the headers quite right.  Some 
> messages here I hit reply (like this one) and the proper “To: 
> dng@lists.dyne.org” shows up, on others someone’s name is populated in the 
> To: box.  Other lists, you hit reply and To: is always populated correctly.
> 
> golinux?, other admins?, is there a config option somewhere in the backend 
> to ‘fix’ this?

Unfortunately I think it's one of those things where you have to break some 
stuff to work around the deliberate breakage implemented with malice 
aforethought by many large email providers.

The problem is SPF, DMARC, and friends. These basically provide information 
about where emails may come from - eg gmail may only come from Google's 
servers. This is a problem for any system that forwards email - such as mailing 
lists and mail servers setup to forward email for (say) 
i...@nicetownplumbers.co.uk to ntplumb2458...@someispmail.com.

So, someone using gmail sends a message to dng@lists.dyne.org which is 
delivered and then forwarded to all the list users. Some of those users will be 
using mail services that check SPF etc - and oh dear, there's an email which 
purports to come from gmail but it's actually being sent from a dyne.org 
server. So it gets discarded as obviously spam.

What they've down with the list (and I've seen it with other lists too) is: if 
the mail matches some criteria, then the originator's address is replaced with 
the list address and a reply to header is added. Thus for those users on a 
broken mail system (such as gmail, or hotmail, or ... they still get the list 
emails instead of not seeing mails from some proportion of list users.
The downside is what you see.

Not sure what the criteria are - whether it's based on there being certain 
headers in the email, whether the sender domain has SPF records etc, or what.

One answer is to always use reply to all and then move/remove addresses so you 
just have a single destination of the list address. I do ths all the time out 
of habit - partly because my mailer does somethings slightly differently with 
reply all, partly because I'm on a few lists and they all seem to do things 
differently (some have always left the senders address, some have always 
replaced it, some have always used a reply to, ...
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[chillfan via Dng]

I can confirm that I haven't set a reply-to header, but this is just a web 
mail. It could be that my webmail provider just doesn't allow me to unset the 
header, but I haven't looked that far into it.


Cheers,

chillfan

‐‐‐ Original Message ‐‐‐
On Wednesday, December 26, 2018 4:07 PM, Michael 
 wrote:

> On Wednesday 26 December 2018 03:24:24 am Antony Stone wrote:
> 

> > Argh. Sending to the list this time.
> > Please don't set "Reply-to" on list emails.
> > Antony.
> 

> I’m pretty sure the individuals aren’t doing it explicitly. This list just
> doesn’t seem to create, or override really, the headers quite right. Some
> messages here I hit reply (like this one) and the proper “To:
> dng@lists.dyne.org” shows up, on others someone’s name is populated in the
> To: box. Other lists, you hit reply and To: is always populated correctly.
> 

> golinux?, other admins?, is there a config option somewhere in the backend
> to ‘fix’ this?
> 

> Best,
> Michael
> 

> Dng mailing list
> Dng@lists.dyne.org
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng



publickey - chillfan@protonmail.com - 0xB179B25B.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

[DNG] Admins can you fix/set the header overrides?

[Michael]

On Wednesday 26 December 2018 03:24:24 am Antony Stone wrote:
> Argh.  Sending to the list this time.
>
> Please don't set "Reply-to" on list emails.
>
> Antony.

I’m pretty sure the individuals aren’t doing it explicitly.  This list just 
doesn’t seem to create, or override really, the headers quite right.  Some 
messages here I hit reply (like this one) and the proper “To: 
dng@lists.dyne.org” shows up, on others someone’s name is populated in the 
To: box.  Other lists, you hit reply and To: is always populated correctly.

golinux?, other admins?, is there a config option somewhere in the backend 
to ‘fix’ this?

Best,
Michael
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng