Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Quoting spiralofhope (spiralofh...@spiralofhope.com): > Thank you for the lesson. It all, and I think the above in particular, > is just the thing I needed to learn to approach the admins of the list > I've had such problems with. It may also be helpful to know that Mailman listadmins in _many_ (most? wouldn't know for sure) cases have been granted and know how to effectively use the Mailman admin WebUI for their supervision of the mailing list but lack access to the server shell, hence their (typical) inability to read/research Mailman's and the MTA's log entries. Also, as a personal observation about my administration of both my own (linuxmafia.com) host + mailing lists and those of several LUGs, although (to date) I've always been willing to spend time investigating subscribers' claims that the mailing list server is losing/refusing their mail, it's inevitably a significant time sink -- and the diagnostic information provided by the user tends to be vague and unreliable. Which, in turn, is in part because a high percentage of those users lack diagnostic information, e.g., I'll ask such a user what the user's outbound MTA logs show about the delivery attempts, and the user has no idea whatsoever, because the user has no access to that data. So, instead, the user _claims_ there were outbound delivery attempts, but is actually just guessing and really knows only that he/she composed mail and handed it off to outbound system software at timestamp X. I can't tell you how many times such a user swore up and down that there was an outbound MTA delivery attempt, I report finding no evidence in my receiving MTA logs, and then days later the user admits that 'Well, actually, I was merely assuming outbound delivery attempts.' So, what I'm saying is: If you're going to chew up the time and effort of a volunteer listadmin, be aware of the limits of your knowledge (especially if you are neither root on your outbound MTA box nor even have shell on it), and provide the most-exact information you have, ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On Thu, 2 Jan 2020 15:30:28 -0800 Rick Moen wrote: > Because GMail > enforces at the time of receipt the declared DMARC policies of what > is asserted to be the source domain of an arriving mail, and because > yahoo.com has an r=reject DMARC policy and its declared roster of > authorised origins for yahoo.com mail doesn't include Dng's MTA host, > Gmail 55x-rejects Gmail User's copy. He/she never sees Yahoo User's > posting. Worse, Mailman takes note of the 55x rejetion, and > increments GMail User's bounce score, in effect sanctioning Gmail > User for Yahoo User's domain's (IMO) problem-causing antiforgery > procedures. > > After a few such incidents, Gmail User gets his/her delivery disabled > and eventually unsubscribed. Thank you for the lesson. It all, and I think the above in particular, is just the thing I needed to learn to approach the admins of the list I've had such problems with. Now I can move away from "nefarious shadowbanning" to actual troubleshooting. (This topic is new to me. I always thought email was wholly unreliable and we'd one day just use PGP for actual authenticity.) ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
Quoting spiralofhope (spiralofh...@spiralofhope.com): > If an email address successfully receives a few emails but then gets > automatically unsubscribed later, could this be why? That would be a possible reason (but not on this mailing list since the implementation of DMARC migitation a bit over a year ago). GNU Mailman records the fact of it incrementing a subscriber's 'bounce score' (and eventually disabling subscription delivery, and then as a later stage unsubscribing, if bounce score remains persistently[0] high) into log file /var/log/mailman/bounce, which is world-readable for command-line users on the Mailman server. However, that log doesn't include the reason for the 'bounce', in part because Mailman is a bit of a dunce about such things[1], so researching the exact reason then requires also that a site admin look through the logs for the MTA associated with Mailman. > (Or would problematic settings means that no emails would ever be > received in the first place?) Your question's a bit vague. (a) If Mailman cannot reach a would-be subscriber by mail, then the person will be, by definition, unable to complete the three-way handshake process required to subscribe. (b) As a reminder, the typical scenario you are discussing (as quoted near the top of this post) involves adverse consequences _not_ primarily to the subscriber whose domain has an aggressive DMARC policy, but rather some other subscribers. Illustration: Imagine two subscribers to Dng; call them Gmail User and Yahoo User. One day, Yahoo User posts a valid posting to a Dng thread. Mailman receives it, and attempts to re-mail copies out through its local MTA to each Dng subscriber of record, including Gmail User. Because GMail enforces at the time of receipt the declared DMARC policies of what is asserted to be the source domain of an arriving mail, and because yahoo.com has an r=reject DMARC policy and its declared roster of authorised origins for yahoo.com mail doesn't include Dng's MTA host, Gmail 55x-rejects Gmail User's copy. He/she never sees Yahoo User's posting. Worse, Mailman takes note of the 55x rejetion, and increments GMail User's bounce score, in effect sanctioning Gmail User for Yahoo User's domain's (IMO) problem-causing antiforgery procedures. After a few such incidents, Gmail User gets his/her delivery disabled and eventually unsubscribed. Back in the latter half of 2018, this happened a few times and I observed people complaining to Golinux, who was in fact not in a position to read the MTA logs, hence they were complaining to the wrong party and basically shouting at the clouds. (IMO, it really didn't help that a whole lot of folks here are desktop computer users afflicted with what I call helpdesk mentality, where one imagines problems get solved by people complaining rather than doing relevant analysis.) [0] There's some logic to expire out a user's bounce score after something like a month. [1] E.g. Mailman doesn't even try to parse 45x and 55x DSNs the outbound MTA receives when the MTA attempts to deliver a particular subscriber's copy of a mailing list posting. Mailman just notes in its log that a 'bounce' event occurred and increments the user's bounce score, irrespective of what caused the non-acceptance. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
On Sat, 28 Dec 2019 23:47:49 -0800 Rick Moen wrote: > Ergo, often one of the places mailing > lists first notice delivery problems owing to aggressive DMARC > policies is among subscribers receiving their subscription mail on > GMail, who suddenly aren't getting some mailing list traffic, report > their subscriptions disabled on account of mysteriously high 'bounce > scores' or get mysteriously unsubscribed (for the same reason). If an email address successfully receives a few emails but then gets automatically unsubscribed later, could this be why? (Or would problematic settings means that no emails would ever be received in the first place?) ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On Sat, 28 Dec 2019 13:01:25 + Mark Rousell wrote: > On 28/12/2019 07:01, Steve Litt wrote: > > So, if we insist on assisting Yahoo, Gmail, Hotmail, and their ilk, > > and all their users, by incorporating DMARC > > Really, it's surely not a matter of willingly helping them. It's more > a matter of survival at all in a world where they carry a significant > proportion (possibly a majority but it's not certain) of the world's > email and where they re-make the rules to suit themselves. Just be > glad they still support SMTP at all! YMMV, but I do not need to carry a significant proportion of global emails. In fact, all those listed above and plenty others are permanently blocked on my mail server for over a decade plus because of their "free speech" stupidty of passing on what is clearly spam. SMTP wiill be a very long time dying when the names' business models are all about exploiting their marks/customers. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Quoting Hendrik Boom (hend...@topoi.pooq.com): > Can I hope that it won't append a Reply-To: header if there is already one? I really have no idea. Mailman's versions of the last few years have, if as listadmin you have the poor judgement to enforce Reply-To munging via the admin WebUI, done that forcing _additively_, appending the forced header to any existing one supplied by the sender (e.g., as a second Reply-to addressee, comma-delimited). So, I suspect the ones added for DMARC mitigation would do likewise. If you wish the answer to that specific question, maybe you should ask the GNU Mailman developers. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Quoting Bernard Rosset via Dng (dng@lists.dyne.org):
> On a more gneric topic, what I read about DMARC over here seems to
> be a bit unfair.
If you mean specifically my own postings on the subject, that's quite
arguably true, especially the stuff I wrote a bit over a year ago, when
I was well and truly furious about the destructive effect of strong
DMARC policies on the (many) mailing lists I administer, and trying to
help fellow listadmins understand and cope with the problem.
I'd be willing to consider offers to hire me to write utterly dispassionate
and exhaustive documentation, as well, at consulting rates, two-hour
minimum. But that would be a different need from the one I had been
(and recently, somewhat exhaustedly, continued) attempting to satisfy.
> DMARC is only there to *enforce* SPF and/or DKIM ("DomainKeys
> Identified Mail" hence not really "former" DomainKeys, just mere
> relabeling).
I'm a little unclear on what you're saying, here, and what your point
is. If you're saying DKIM is just a newer name for DomainKeys, but was
unchanged from DomainKeys, you are incorrect: Yahoo had produced a
draft called 'enhanced DomainKeys', and that was merged with a separate
Cisco effort called 'Identified Internet Mail' to produce DKIM in 2004.
Yes, DMARC is a defined superset of SPF and/or DKIM. DKIM, IIRC, had
the same destructive effects on mailing lists for the same reasons.
Saying DMARC is 'only there to enforce' it is rather missing the point,
IMO.
> The real protection mechanisms being considered/violated here are
> SPF and/or DKIM. DMARC's policy only triggers if *both* SPF & DKIM
> fail.
Your wording, here, is a bit ambiguous. If you are intending to
suggest that DMARC requires that a domain implement both SPF and DKIM,
that is not correct. OTOH, if you mean that DMARC fails only if neither
SPF or DKIM validates, then that is correct.
> Now, if the sender's domain supports DKIM, and provided the headers
> potentially important to the mailing list's piping are not provided
> & signed (Sender, List-*, Reply-To, etc.), ie if mere From, Subject
> are signed (which I believe is a common case), it is alright.
>
> Well. It is alright... provided mailing lists stop doing what they
> have been doing for ages, ie *modifying* protected content, either
> protected headers or body.
In other words, with the typical DKIM-attested set of headers and
content, mailing lists break short of major changes such as wrapping the
message, From: rewriting, or ceasing all message modifications, meaning
not just no more footers and subject prefixes, but also (IIRC) problems
with List-ID and similar headers.
More than a year ago, I could have written a comprehensive explanation
of all the gory details, but will confess I've dropped a lot of it from
memory since then.
> Hence, the real problem comes from violating DKIM... or having no
> DKIM set up.
Again, your wording is ambiguous. If you're suggesting that having no
DKIM set up at a sending domain is somehow problematic for that domain,
then that is incorrect. E.g., my linuxmafia.com domain does not have
DKIM setup (because I think that technology design was poorly written),
and I have no deliverability problems at all -- particularly because
my domain has a correct, strongly asserted SPF policy, and because I
follow reputable SMTP practices carefully and protect the reputation of
my sending IP address.
I'm not entirely sure what you mean, if you meant something else.
> DMARC + DKIM should do the trick, provided mailing lists (softwares)
> stop being intrusive.
'Stop being intrusive'? The nerve!
Also, the term 'DMARC + DKIM' doesn't actually make a lot of sense.
DMARC is a superset built atop either DKIM or SPF (or both).
> In the current state of my understanding of DMARC, SPF & DKIM, I
> have a hard time understanding flaming any of those protection
> mechanisms.
Well, I have no problem taking care of that need, in your absence.
No charge, sir.
> The only trouble I see here is that mailing lists have a long
> history of modifying email headers and/or content, and it has been
> deemed "normal" over years of doing so.
That's like saying the only trouble you see is that humans have a long
history of eating.
> Would you mind if I arbitrarily opened/modified your (private)
> postal mail or any written message from/to you?
This is an abuse of metaphor, and I'm having a difficult time believing
you aren't trolling.
Mailing lists are sophisticated remailer mechanisms. In postal mail
context, the proper metaphor would be an optional commercial service
you can send a letter to, where the letter would be photocopied and then
remailed to all of your friends. This isn't 'arbitrary'; the original
sender engages the services of the remailing mechanism. Nor is it
'private'.
When you signed up for Dng, you were aware that you were voluntarily
engaging the services of a software remailing service that would
generate slightly
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Quoting g4sra via Dng (dng@lists.dyne.org): > Thanks Rick, I appreciate that chain of summaries and the time it has > saved now not having to dig through archives. No problem! E-mail is a dreadful solution to the problem of collective knowledge, and I really ought to post that somewhere persistent on the Web. Maybe https://dev1galaxy.org/viewforum.php?id=7, dunno. Of course, what I wrote wasn't a proper Devuan Project doc, but rather a personal take on the matter as a friendly outsider. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On 29/12/2019 07:47, Rick Moen wrote: [snip] Thanks Rick, I appreciate that chain of summaries and the time it has saved now not having to dig through archives. Email has probably got to be one of my weakest areas of knowlege, I have learnt something today. When drawing my own conclusions I pay little heed to dissagrievements on mailing lists, but find facts really helpful. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On Sat, Dec 28, 2019 at 09:41:48PM -0800, Rick Moen wrote: > > tl;dr: Mailman will now munge the From: address if and only if the > sender's domain publishes a problematic DMARC policy, to substitute the > mailing list's address for the sender's. On those mails, Mailman > also appends a Reply-To: header pointing to the sender's real address. > No other mails will be touched. Can I hope that it won't append a Reply-To: header if there is already one? -- hendrik ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists : solved by workaround
On Sat, 28 Dec 2019 21:41:48 -0800 Rick Moen wrote: > tl;dr: Mailman will now munge the From: address if and only if the > sender's domain publishes a problematic DMARC policy, to substitute > the mailing list's address for the sender's. On those mails, Mailman > also appends a Reply-To: header pointing to the sender's real address. > No other mails will be touched. The preceding description describes what I see on my end. However, when I click "Return to sender", Claws-Mail takes me literally and sends to the munged To, which is now the mailing list. Claws does not consult the "Reply to" when I click "Return to sender." This is my problem. If I click "Reply to list" it replies only to the list, which is exactly what I want under normal situations. If I click "Reply" or "Reply to All", it sends to the list and copies the return address. I then have to prune off whichever address I don't want, and if the one I want is the return address, I need to change it from Cc to To. So my solution is procedural. I removed my "Reply to Sender" button, because in the age of DKIM it does just what I don't want, even if it's literally correct. Now, whenever I want to email somebody offlist, I'll: 1) Click Reply to all 2) Delete the mailing list address 3) Change the Cc to To The point is, if I see two addresses up there, I'll understand there's danger and delete the dng one. So, although I have the same opinion of DKIM that I've always had, my procedural workaround means I won't need to ask anyone else for help. SteveT Steve Litt December 2019 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21 ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On 29/12/2019 06:30, Rick Moen wrote:
Quoting Mark Rousell (mark.rous...@signal100.com):
That said, the mail list *does* seem to work as Steve wants.
It really doesn't.
On 28/12/2019 14:16, Mark Rousell wrote:
At least it does for my mail client (Thunderbird).
It definitely seems to be MUA-specific. The last bit from Mark is
important: the Thunderbird MUA seems to always show consistent behaviour
of its "Reply" & "Reply List" buttons.
The only thing which changes for this MUA is the set of displayed
headers above the message.
Non-DMARC-protected domains show From, Subject & To, while
DMARC-protected ones show From, Subject, Reply-To & To.
I concur with Mark on the fact this email client seems to do the job, at
least on that front.
-
On a more gneric topic, what I read about DMARC over here seems to be a
bit unfair.
DMARC is only there to *enforce* SPF and/or DKIM ("DomainKeys Identified
Mail" hence not really "former" DomainKeys, just mere relabeling).
The real protection mechanisms being considered/violated here are SPF
and/or DKIM. DMARC's policy only triggers if *both* SPF & DKIM fail.
SPF is a mechanism to ensure the envelope matches the headers & sender
machine is authorized to emit for a domain (hence protects against
impersonation).
DKIM protects against message tempering by signing body & some headers
of the emitted email.
From-munging, used to circumvent SPF, actually means
faking/modifying/impersonating the original email source.
It also happens to circumvent DKIM... and DMARC as a whole, since the
emitting domain would now be the list's one, *not* the sender's.
This From-munging is a perfect man-in-the-middle example, actually
pulling the plug on all headers checks at destination.
Now, if the sender's domain supports DKIM, and provided the headers
potentially important to the mailing list's piping are not provided &
signed (Sender, List-*, Reply-To, etc.), ie if mere From, Subject are
signed (which I believe is a common case), it is alright.
Well. It is alright... provided mailing lists stop doing what they have
been doing for ages, ie *modifying* protected content, either protected
headers or body.
That means no From header modification (no From-munging).
That means no Subject header modification (no added prefix and rather
let destination users route incoming email based on headers rather than
Subject prefix).
That means no body modification (and rather leverage List-* headers &
let MUA augment received messages based on those).
As stated before, a DMARC policy fails if *both* SPF & DKIM checks fail
or if one fail and the other is non-existent.
Hence, the real problem comes from violating DKIM... or having no DKIM
set up.
DMARC + DKIM should do the trick, provided mailing lists (softwares)
stop being intrusive.
In the current state of my understanding of DMARC, SPF & DKIM, I have a
hard time understanding flaming any of those protection mechanisms.
The only trouble I see here is that mailing lists have a long history of
modifying email headers and/or content, and it has been deemed "normal"
over years of doing so.
Would you mind if I arbitrarily opened/modified your (private) postal
mail or any written message from/to you?
My understanding might be incomplete. If so, please enlighten me &
anyone interested, by all means.
Cheers,
Bernard Rosset
https://rosset.net/
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
One note about my advice to OSI on December 1, 2018: That was, IIRC, my earliest attempt to advise fellow GNU Mailman listadmins about how to contend with the DMARC problem. A probably-mistaken small datum in what I said to OSI now sticks out: > Yahoo and Gmail are examples of sending domains with strict DMARC > policies. Correction: Either that was never true about GMail, or it was at the time of writing, but is no longer. (More likely the former.) Today: ~ $ dig -t txt _dmarc.gmail.com. +short "v=DMARC1\; p=none\; sp=quarantine\; rua=mailto:mailauth-repo...@google.com; Substring 'p=none' means _not_ an aggressive/strict DMARC policy[1] -- unlike, say, yahoo.com's published policy: :r! dig -t txt _dmarc.yahoo.com. +short "v=DMARC1\; p=reject\; pct=100\; rua=mailto:dmarc_y_...@yahoo.com\;; What _is_ true of GMail is that it enforces upon receipt at GMail all published DMARC policies of SMTP-sending domains (as relatively few SMTP-receiving domains yet do). Ergo, often one of the places mailing lists first notice delivery problems owing to aggressive DMARC policies is among subscribers receiving their subscription mail on GMail, who suddenly aren't getting some mailing list traffic, report their subscriptions disabled on account of mysteriously high 'bounce scores' or get mysteriously unsubscribed (for the same reason). Back when I was advising OSI, I probably confused the issues of GMail's strong application of _other_ domains' DMARC policies with its lack of an aggressive policy published for outbound gmail.com mail. What's linuxmafia.com's published policy, you might ask: :r! dig -t txt _dmarc.linuxmafia.com. +short "DMARC: tragically misdesigned since 2012. Check our SPF RR, instead." [1] gmail.com's 'sp=quarantine' in the DMARC TXT RR is a policy for any/all subdomains, one of innumerable baroque features that'll eat your evening if you start studying the hideous thing. -- Cheers, "Maybe the law ain’t perfect, but it’s the only Rick Moenone we got, and without it we got nuthin'." r...@linuxmafia.com -- U.S. Deputy Marshal Bass Reeves, circa 1875 McQ! (4x80) ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Quoting Andrew McGlashan via Dng (dng@lists.dyne.org): > They screw up greylisting, they screw up SPF and they screw up DMARC. They didn't screw up SPF. If you as the domain stakeholder of an SMTP-sending domain deterministically know and can specify in SPF's flexible spec format for DNS TXT records where _all_ your domain's legitimate mail will originate, then you can use SPF to good effect to make forged sending IPs detectable and rejectable at the time of SMTP receipt. I happen to be able to thus specify. It's particularly simple in my domain's case, because the sole authorised origin is one IPv4 address. Therefore... :r! dig -t txt linuxmafia.com. @ns1.linuxmafia.com. +short "v=spf1 ip4:96.95.217.99 -all" ...Works for Me[tm]. (Please note that the '-all' means my DNS recommends _permfail_ of non-compliant mail.) Occasionally, I see claims in Linux forums, including in a discussion two years ago on this mailing list, that SPF breaks on mailing lists. This is simply not true. If it'd been true, I'd have noticed at some point over the last couple of decades. Domain owners for whom SPF does _not_ work include ones who insist on originating port 25 unauthenticated SMTP from arbitary unplanned IP addresses without that mail getting rejected as a suspected of being a forgery. (Good luck with that.) For them, fortunately, even if they take that rather impractical position, SPF still doesn't hurt them: They retain the option of not publishing an SPF record, or one declaring that their mail might originate from anywhere. Oddly enough, I _can_ identify a time when my SPF record did hurt my mail delivery. It was the afternoon of December 17, 2019, when because of an ISP shutdown I had to re-IP my server for the first time in 18 years. Everything appeared to go smoothly, in part because I'd shortened TTLs to 3600 many days in advance. Less than an hour after cutover, one of my outgoing mailing list postings was rejected by luv.asn.au on grounds that my own SMTP server supposedly violated my own SPF policy. Explanation: Someone there was for some reason retaining my old SPF RR in cache longer than was supposed to happen. Problem did not recur. ;-> ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
Quoting Steve Litt (sl...@troubleshooters.com): > In my wildest imagination, I never dreamed that Mailman wouldn't give > the admin control over the text assembly of the munged from. Then, possibly you can suggest how. Why don't you test your solution on a test GNU Mailman installation, and advise Dyne.org's administrators about specifics you have in mind? Also, it would be a good idea for you to skim details of GNU Mailman's relevant documentation. To help you in that, here is the advice that I give on the subject to Mailman listadmins, identifying a specific admin WebUI configuration I recommend. In this case, it's my recommendation to _OSI's_ listadmins (which FWIW they implemented), because that was the easiest copy for me to find, but I gave near-identical advice to Devuan Project: Date: Sat, 1 Dec 2018 18:47:16 -0800 From: Rick Moen To: license-review-ow...@lists.opensource.org Subject: (forw) Re: [License-review] Approval: Server Side Public License, Version 2 (SSPL v2) I wish to strongly recommend, based on my own listadmin experience, the best available DMARC mitigation. DMARC is proving to be an utter nightmare for mailing lists, in as much as they are mail forwarders, and DMARC was IMO botched in its ability to accomodate the way they work. From memory, and so I'm probably dropping a bunch of detail: Because MLMs such as Mailman (appropriately) change the internal SMTP headers upon retransmitting the poster's mail to subscribers (notably the To: header), it no longer validates against the sender's domain if it is a DMARC-using one with a strict policy. Yahoo and Gmail are examples of sending domains with strict DMARC policies. There is an (IMO unhappy but least-bad-available) kludge setting in Mailman's admin WebUI to make the MLM compensate for DMARC brain-damage: You go to Privacy Options, Sender Filters, item 'Action to take when anyone posts to the list from a domain with a DMARC Reject/Quarantine Policy' aka dmarc_moderation_action. Change the radio button from Accept (default) to Munge from. To quote the help text: from_is_list (general): Replace the From: header address with the list's posting address to mitigate issues stemming from the original From: domain's DMARC or similar policies. Several protocols now in wide use attempt to ensure that use of the domain in the author's address (ie, in the From: header field) is authorized by that domain. These protocols may be incompatible with common list features such as footers, causing participating email services to bounce list traffic merely because of the address in the From: field. This has resulted in members being unsubscribed despite being perfectly able to receive mail. The following actions are applied to all list messages when selected here. To apply these actions only to messages where the domain in the From: header is determined to use such a protocol, see the dmarc_moderation_action settings under Privacy options... -> Sender filters. Settings: [...] Munge From This action replaces the poster's address in the From: header with the list's posting address and adds the poster's address to the addresses in the original Reply-To: header. So, for example, _if_ my sending domain linuxmafia.com had a strong DMARC policy (which it doesn't, because I hate DMARC with a passion), then the 'Munge from' setting would cause my post to license-review to get this 'From: ' header upon retransmission to subscribers: From: Rick Moen via license-review instead of the normal From: Rick Moen The reason this helps sidestep DMARC validation is that it's now no longer considered needing validation against linuxmafia.com's (hypothetical) DMARC policy, but rather lists.opensource.org's. I personally detest this solution because, when I send out my sending address on a mailing list, it is deliberately there so that people can, if necessary, contact me offlist. The kludge complicates this, albeit, if I remember correctly, it tries to compensate for the brain-damage by inserting a Reply-To as well. It should be noted that the Munge from kludge thus alters -only- the postings of subscribers from DMARC-damaged^H^H^H^W^W^W^Wusing domains, so only _some_ postings will get disfigured in this manner. Sadly, I recommend opting for this kludge, because otherwise deliverability suffers. I am specifically _not not not_ recommending the similar-looking setting 'Replace the From: header address with the list's posting address to mitigate issues stemming from the original From: domain's DMARC or similar policies' aka from_is_list on General Options. My understanding is that opting for _that_ version of the kludge unconditionally applies it to all postings whether they are from badly DMARC-impaired domains (ones with published p=reject or p=quarantine DMARC policies) or not. My recommendations: On Privacy options, Sender filters: dmarc_moderation_action: Munge from
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Quoting 'smee via Dng (dng@lists.dyne.org): > Is there a clear rule about when the sender displays as an individual, > vs "individual via DNG"? Herewith, a repost: Date: Thu, 6 Dec 2018 02:02:30 -0800 From: Rick Moen To: dng@lists.dyne.org Subject: Dng now alters (some) posts to compensate for DMARC antiforgery As of today, the esteemed Dng listadmins have made a small tweak to Mailman's operation, and have asked me to explain the change. _No_ action is required on your end. tl;dr: Mailman will now munge the From: address if and only if the sender's domain publishes a problematic DMARC policy, to substitute the mailing list's address for the sender's. On those mails, Mailman also appends a Reply-To: header pointing to the sender's real address. No other mails will be touched. Forgery of SMTP mail is a serious problem, leading to a series of proposals for extensions to the SMTP standard, to permit domains and users at mail-receipt time to detect and reject forgeries: SPF, DKIM (formerly DomainKeys), and DMARC. DMARC, from Yahoo, is the most recent of these SMTP extensions (incorporating DKIM and SPF as sub-components). Unfortunately, DMARC, when implemented by sending domains publishing a strongly asserted DMARC antiforgery policy, tends to be a disaster for mailing lists: Subscribers sending from such mail domains gradually discover that their outbound mail, when routed out through mailing list software and thus retransmitted to mailing list subscribers, gets refused (as forged) upon arrival at many of the subscribers' receiving SMTP servers. Q: Why does that mail get rejected as forged? A: Because the mailing list manager (MLM) software alters and makes additions to the sender's headers and body text, on the copies retransmitted to subscribers, with the result that the message no longer matches its DKIM cryptographic signature. Q: Which sending domains are affected? A: I referred to these as sending domains with 'strongly asserted DMARC antiforgery policies'. Specifically, this means domains that publish p=reject or p=quarantine as part of the DMARC policies in their DNS. Here's an example of the former, domain mongodb.com: $ dig -t txt _dmarc.mongodb.com [...] _dmarc.mongodb.com.300INCNAME mongodb.com.hosted.dmarc-report.com. mongodb.com.hosted.dmarc-report.com. 300 IN TXT"v=DMARC1; p=reject; rua=mailto:1eed4...@mxtoolbox.dmarc-report.com,mailto:dmarc_agg@vali.email,mailto:dmarc_repo...@mongodb.com; ruf=mailto:1eed4...@forensics.dmarc-report.com,mailto:dmarc_repo...@mongodb.com; [...] $ Q: Which receiving domains reject such mail? A: Domains that implement DMARC and respect/enforce (some) sending domains' strongly asserted DMARC policies. For example, GMail exactly enforces sending domains' published DMARC policies (if any), when it decides what arriving mail to reject as forged. Q: If Mailman rewrites my mail for transmission to subscribers, what would that look like? A: Like this (using me as an example poster) From: Rick Moen via Dng (with) Reply-To: Rick Moen instead of the normal From: Rick Moen This example is what would occur if domain linuxmafia.com had a strongly asserted DMARC policy, which in reality it doesn't (because domain owner Rick Moen doesn't like DMARC). Q: Isn't 'munging' (forcing) of the Reply-To: header by anyone but the sender been officially a bad idea ever since IETF adopted RFCs 2822 and 2369 in 2001? A: Yes. Ironic, isn't it? GNU Mailman adopted and recommended this mitigation for the problems caused by DMARC anway, as it's the least-bad response to the fundamental hostility DMARC has for mailing lists as reflectors. The Mailman developers might eventually come up with something better, but this is the best solution they have at this writing. Q: Are other MLMs also affeected? A: Yes. Q: Who decided to adopt this modification to Dng's operation? A: It was a unanimous recommendation of the Devuan Project's weekly Jitsi conference on December 5, 2018. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Quoting Mark Rousell (mark.rous...@signal100.com): > Even without having to submit a patch or knowing the full ins and out of > Mailman's DMARC mitigation, it strikes me that Steve's request was a > reasonable one. OK, cool, I nominate you to handle everything about this situation, from this point forward. Take charge, Mark. That'll save me the effort of trying to talk sense into people. > That said, the mail list *does* seem to work as Steve wants. It really doesn't. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
Is there a clear rule about when the sender displays as an individual, vs "individual via DNG"? Just during this discussion I've seen it both ways. It hasn't been a problem for me, likely because I seldom participate (that won't always be the case), but I can see it being a hassle for a regular contributor. Just something I noticed, I thought I'd share in case it's helpful, I use evolution and, every time I reply to an email from the list, it prompts that I'm replying privately and then gives four options: (1)reply privately, (2)cancel, (3)reply to group (goes to individual AND mailing list), or just (4)reply to list. Oddly it gives this option every time I reply, whether to an individual or to individual via DNG. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
On Sat, 28 Dec 2019 00:34:09 -0800 Rick Moen wrote: > Are you in the middle of submitting a patch to GNU Mailman, then? I'm > expect they will give it appropriate consideration, and give you > expert feedback (which, possibly, the rest of us will appreciate > hearing). > > OTOH, expecting Dyne.org people to hand-hack the local Mailman > installation, rather than trying to get it accepted by the > developers, and not even providing anyone with a patch, doesn't > strike me as even a tiny bit reasonable. And, gosh, I'm sorry to say > you appear to be so suggesting, which again, for the second night in > a row, makes me a little sad. In my wildest imagination, I never dreamed that Mailman wouldn't give the admin control over the text assembly of the munged from. If Mailman gives this string as a take-it-or-leave-it constant, I guess I'll just have to implement a fix on my end. Anyone know of an equivalent for procmail on the *sending* side? SteveT Steve Litt December 2019 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21 ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
Steve Litt wrote: > ... we could at least > change the munge string from: > > Firstname Lastname via Dng > > to: > > GOES TO DNG (IRT Firstname Lastname) > > So when you do "return to sender" and it crazily puts > dng@lists.dyne.org in the To field, at least that To field won't be > disguised as the user. Quick question ... Does your MUA show the full address, or does it follow the MS rule of actively hiding the actual address and only showing the name part ? I am now forced to use Outlook for mail at work, and it's a pile of steaming manure in many respects. Prior to starting work, I had to have some email exchanges with my manager so his outlook has cached both my personal and work addresses. Since Outlook makes it hard to see the email address that's behind the name, there are times when he's sent work email to my home address. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On 28/12/2019 15:02, Arnt Karlsen wrote: > On Sat, 28 Dec 2019 13:16:19 +, Mark wrote in message > <5e0755a3.80...@signal100.com>: > >> That said, the mail list *does* seem to work as Steve wants. > ..you almost nailed it with the above observation, I'd go > "the mail list does *seem* to work as Steve wants", which > is how and why Steve got fooled into doing what he did. It certainly does work as he wants it for me (as I understand his intention). :-) I.e. For senders on DMARC "p=reject" or "p=quarantined" domains, what would originally have been their From address is placed in the Reply-To field of the mailing list post. For me running Thunderbird, this ensures that when I click the Reply button my reply goes to the individual sender's Reply-To address (what was originally in their email's From field), not to the list. That is, as I understand it, what Steve wants. It could well be that Steve's mail client is failing to prioritise the Reply-To address over the From address (whereas mine priorities the Reply-To address over the >From address when I click the Reply button). For the avoidance of doubt, I have separate 'Reply to List', 'Reply All' and 'Reply' buttons. -- Mark Rousell ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On Sat, 28 Dec 2019 13:16:19 +, Mark wrote in message <5e0755a3.80...@signal100.com>: > That said, the mail list *does* seem to work as Steve wants. ..you almost nailed it with the above observation, I'd go "the mail list does *seem* to work as Steve wants", which is how and why Steve got fooled into doing what he did. -- ..med vennlig hilsen = with Kind Regards from Arnt Karlsen ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 29/12/19 12:01 am, Mark Rousell wrote: > On 28/12/2019 07:01, Steve Litt wrote: >> So, if we insist on assisting Yahoo, Gmail, Hotmail, and their >> ilk, and all their users, by incorporating DMARC > > Really, it's surely not a matter of willingly helping them. It's > more a matter of survival at all in a world where they carry a > significant proportion (possibly a majority but it's not certain) > of the world's email and where they re-make the rules to suit > themselves. Just be glad they still support SMTP at all! Sadly that is too true. They screw up greylisting, they screw up SPF and they screw up DMARC. And to make matters worse, you can easily block IP addresses and IP blocks of bad email servers unless it comes from the rotten lot as above (including Apple and Microsoft). I see plenty of forwarded junk coming through my server from Apple and it's a real pain point. I just wish everyone would stop using those rotten service providers when it comes to email :( A. -BEGIN PGP SIGNATURE- iHUEAREIAB0WIQTJAoMHtC6YydLfjUOoFmvLt+/i+wUCXgdZ/AAKCRCoFmvLt+/i +0ywAPwK9LnPkzeVNaatCEloqyHDEFDAcO08W+mGMhJdFAN1EQD/VuBBBnlmFUxv HGebU11GuFOusgjdz6YHbhrr2GwK8cU= =eaLf -END PGP SIGNATURE- ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On 28/12/2019 08:34, Rick Moen wrote: > Are you in the middle of submitting a patch to GNU Mailman, then? I'm > expect they will give it appropriate consideration, and give you expert > feedback (which, possibly, the rest of us will appreciate hearing). > > OTOH, expecting Dyne.org people to hand-hack the local Mailman installation, > rather than trying to get it accepted by the developers, and not even > providing anyone with a patch, doesn't strike me as even a tiny bit > reasonable. And, gosh, I'm sorry to say you appear to be so suggesting, > which again, for the second night in a row, makes me a little sad. > > Also, have you bothered to make sure you understand Mailman's > DMARC mitigation, before jumping in? I'm guessing 'nope'. > > (Again, just to be crystal-clear, I myself am neither a Dyne.org > administrator nor a GNU Mailman developer, further underlining my point > about how you would be best advised to address the correct people with > your, um, semi-developed notions, and not the wrong ones.) Chillax, it's Christmas (or the seasonal celebration of one's choice)! :-) Even without having to submit a patch or knowing the full ins and out of Mailman's DMARC mitigation, it strikes me that Steve's request was a reasonable one. It would help for non-standard behaviour to be more clear. That said, the mail list *does* seem to work as Steve wants. At least it does for my mail client (Thunderbird). On a message posted by someone posting to the list from a p=reject DMARC domain:- When I click 'Reply to List' the reply goes to the list. When I click 'Reply' the reply goes to the message's 'Reply-To' header contents which is the poster's personal email address. When I click 'Reply All' the reply goes to everyone mentioned in any To, Cc or Reply-To header. -- Mark Rousell ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists
On 28/12/2019 07:01, Steve Litt wrote: > So, if we insist on assisting Yahoo, Gmail, Hotmail, and their ilk, and > all their users, by incorporating DMARC Really, it's surely not a matter of willingly helping them. It's more a matter of survival at all in a world where they carry a significant proportion (possibly a majority but it's not certain) of the world's email and where they re-make the rules to suit themselves. Just be glad they still support SMTP at all! -- Mark Rousell ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
Hi, Steve. First, apologies last night if I was a bit peeved. It's just that I really had put quite a lot of effort into making sure Dyne.org people and the Dng community understood the problem, and that my recommendation was to enable a least-bad mitigation within GNU Mailman that 'munged' _only_ mail from domains that made the (IMO, bad) choice to publish highly aggressive DMARC policies, not out of any liking for Reply-To munging, but because there wasn't a better alternative. Quoting Steve Litt (sl...@troubleshooters.com): > So, if we insist on assisting Yahoo, Gmail, Hotmail, and their ilk, and > all their users, by incorporating DMARC at all, we could at least > change the munge string from: > > Firstname Lastname via Dng > > to: > > GOES TO DNG (IRT Firstname Lastname) Are you in the middle of submitting a patch to GNU Mailman, then? I'm expect they will give it appropriate consideration, and give you expert feedback (which, possibly, the rest of us will appreciate hearing). OTOH, expecting Dyne.org people to hand-hack the local Mailman installation, rather than trying to get it accepted by the developers, and not even providing anyone with a patch, doesn't strike me as even a tiny bit reasonable. And, gosh, I'm sorry to say you appear to be so suggesting, which again, for the second night in a row, makes me a little sad. Also, have you bothered to make sure you understand Mailman's DMARC mitigation, before jumping in? I'm guessing 'nope'. (Again, just to be crystal-clear, I myself am neither a Dyne.org administrator nor a GNU Mailman developer, further underlining my point about how you would be best advised to address the correct people with your, um, semi-developed notions, and not the wrong ones.) ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Again, again: DMARC is a no-win problem for mailing lists (was: Can we fix this DMARC thing?)
On Fri, 27 Dec 2019 18:19:10 -0500 Steve Litt wrote: > On Thu, 26 Dec 2019 19:12:21 -0800 > Rick Moen wrote: > > > Quoting Steve Litt (sl...@troubleshooters.com): > > > > > Seriously, this DMARC thing, or at least the way it's implemented > > > on DNG, is downright dangerous. > > > > Seriously, at the time this came up, I worked really hard, > > tirelessly, and thanklessly, and repeatedly, to explain that Dng > > was caught in a dilemma created by a mailing-list-hostile > > anti-forgery standard, a well-intentioned but (in my opinion) badly > > written piece of ancillary plumbing for SMTP and DNS. I carefully, > > painstakingly qualified what I said, and dealt with the inevitable > > people who wanted to argue merely because I expressed a viewpoint, > > who wanted in knee-jerk fashion to dismiss what I said as yet > > another subvariety of SMTP crankery, or who were the inevitable > > sort of edge-case fanatics who lurk on all technical mailing lists. > > > > I described how the architecture of DMARC left _all_ the mailing > > lists in the world in a no-win situation. I detailed how the GNU > > Mailman people had built into recent releases two separate choice of > > ways to try to mitigate the DMARC disaster. I detailed why I > > strongly recommended one of those mitigations strongly over the > > other. I very carefully disclosed the disadvantages, stressing that > > there would be some unavoidable problems resulting from the > > preferred mitigation's operation any time the mailing list poster > > is sending from a domain with a strongly asserted DMARC policy. > > > > I tirelessly repeated these explanations over a span of months, as > > the Dyne principal volunteers came to grips with the problem and > > parsed what I and others were saying. > > > > And, after a whole lot of my attempting to explain, and explain > > again, and explain again, and deal with arguments and knee-jerk > > naysaying, the Dyne principals accepted my recommendation as the > > least-bad course of action, and implemented the better of the two > > mitigations. > > > > Which brings us to the present. > > > > > > > Let me repeat: "Reply to sender" should never, ever go to the > > > list. > > > > What part of 'some unavoidable problems resulting from the preferred > > mitigation's operation any time the mailing list poster is sending > > from a domain with a strongly asserted DMARC policy' was unclear? > > > > > Did you know that for some but not all DNG email, "reply to > > > sender" sends it to the list? > > > > Did you know that most senders don't suffer the malign effects of > > strong-asserted DMARC policies in their domains' DNS? I've only > > explained that on Dng a few dozen times. Probably it didn't sink > > in. > > > > > > You're making me sorrowful, my friend. I am feeling as if all of my > > efforts to make the no-win nature of the situation, and my > > mentioning in _particular_ the great irony of my appearing to > > recommend (a very limited form of) Reply-To munging, after a > > quarter-century of trying to calmly document for the Internet why > > it's a bad idea, was time wasted. > > > > Tell you what: How about you go onto the Mailman developers' > > mailing list and bitch about how their least-bad effort to limit the > > pernicious effects of a badly written anti-forgery standard thrust > > upon them by others fails to meet your needs? Would you mind doing > > that? > > > > Part of the reason I'm asking is that you, personally, you my friend > > Mr. Litt, recently accidentally posted private mail here portraying > > me as a particularly contentious person (in your view as a denizen > > of Florida, a land of noted passive-aggressives), and thus, if I now > > argue with you, I will help support your accidental character > > assassination. (I'll be nice and call it accidental, even though it > > accords with previous personal characterisations of me you've posted > > non-accidentally.) > > > > And, well, I'm not going to. For lots of reasons including their > > being no percentage in it. Have a great holiday season. (Chag > > Chanukah sameach.) > > > > > > And, next time, _you_ get to do the heavy lifting and deal with > > people who cannot be bothered to read and understand what you said. > > > > Meanwhile, I give up. > > > > > > > I beg whomever is in charge of the DNG mailing list to fix > > > whatever's wrong with the DMARC implementation. > > > > I beg you to pay attention, next time. If I bother to explain > > anything next time. > > > > DMARC is the systemd of the email world. I'm not going to learn its > ins and outs or its best and worst workarounds. I'm not going to talk > to the mailman people. What I *am* going to repeat is that a system > that *ever* sends back to the list when you click "reply to sender" is > incredibly dangerous. So, if we insist on assisting Yahoo, Gmail, Hotmail, and their ilk, and all their users, by incorporating
