[dmarc-ietf] Review of draft-ietf-dmarc-dmarcbis-32
I finally managed to finish my review of the DMARCbis. I think it is in good enough shape to go forward but here are my comments when reviewing it: -- Section 3.2.6 is conflicting with later text. Historically, Domain Owner Assessment Policies of "p=quarantine" or "p=reject" have been higher value signals to Mail Receivers (#mail- receiver). Messages with Author Domains for which such policies exist that are not validated using the DMARC mechanism will not reach the inbox at Mail Receivers that participate in DMARC and honor the Domain Owner's expressed handling preference. I think we should add note, that even if this was true before, DMARCbis now says that in case of DMARC fail the mail receivers handling can be influenced by DMARC policy record, but also other considerations should be taken care of. -- Section 4.5 says: Per [RFC1035], a TXT record can comprise several "character-string" objects. Where this is the case, the module performing DMARC evaluation MUST concatenate these strings by joining together the objects in order and parsing the result as a single string. This text is fine when we are talking about one TXT record, but some implementors might read it to talk about multiple TXT records, so perhaps a note about that should be added. So character-strings inside one TXT records are concatenated, but each TXT record is processed separately. -- It would be better to use name of the RFC when referencing to it, and only use the number in the [RFC1234] part, i.e., instead of saying: Readers are encouraged to be familiar with the contents of [RFC5598]. it would be better to say: Readers are encouraged to be familiar with the contents of Internet Mail Archtecture ([RFC5598]). The rfc numbers 1035, 8020, 4343 etc might be familiar to writers of this document, but there are most likely lots of readers who are not as familiar with them, and expanding them in the document will make it easier for people who do not have full RFC mapping from all RFC mumbers to document title in memory, to read this document. Same should apply to the internet draft names also, as while the [I-D.ietf-dmarc-aggregate-reporting] actually tells what that document is, but when it gets replaced with RFC9951 and RFC10923 even people who are familiar with number to title mapping now, might have to update their internal mapings... I.e., instead of saying: DMARC, in the associated [I-D.ietf-dmarc-aggregate-reporting] and [I-D.ietf-dmarc-failure-reporting] documents, also specifies a reporting framework. say: DMARC, in the associated Aggregate raporting specification ([I-D.ietf-dmarc-aggregate-reporting]) and Failure Reporting specification ([I-D.ietf-dmarc-failure-reporting]) documents, also specifies a reporting framework. Getting new people to be involved is hard anyways, and making it harder by forcing everybody to learn lots of different number -> title mappings makes it even harder. I am very familiar with certain subset of the RFC numbers (ipsec, security area etc), but most of the email RFC numbers are new to me, so while I was reviewing this, I always needed to go and check those numbers, which made reading it more difficult. -- In section 4.10 step 1, there is text saying: 1. Query the DNS for a DMARC Policy Record at the appropriate starting point for the Tree Walk. A possibly empty set of records is returned. what is this "appropriate starting point" used here? It is not defined before, and for the first read it seems like something is missing. It will be defined in the 4.10.1, but adding reference to that section here would make things easier, or even move the starting point definition from there to here, or before this section. Or perhaps move the actual tree walk steps from the section 4.10 DNS Tree Walk to come after 4.10.1, i.e. add new "4.10.2 DNS Tree Walk Steps", and move text: The generic steps for a DNS Tree Walk are as follows: ... there. -- 5.4. Policy Enforcement Considerations At a minimum, Mail Receivers SHOULD add the Authentication-Results header field (see [RFC8601]), and it is RECOMMENDED when delivering messages that fail the DMARC validation check. What is this text trying to say. It seems to say authentication- results header is SHOULD, but it is also SHOULD when DMARC validations fails? Perhaps just change "...it is even more important when ...". -- 10.6. External Reporting Addresses To avoid abuse by bad
Re: [dmarc-ietf] DMARCbis WGLC - Issue 141 DMARC and What To Say About SPF -all
Scott Kitterman writes: > I hear you. Your operational issue is my system working as designed. > DMARC works on top of SPF, it doesn't change it. Yes, DMARC works on top of SPF, and DKIM and provides policy layer. We are trying to change the fact that people rely purely on SPF, and try to get them moved to use DMARC istead, and we are trying to explain that if you do SPF inside the DMARC context, you get exactly same policy results you get as when you do SPF before, except you get it better, as you have more data available. Using -all would be completely ok if everybody would be doing DMARC, but as there are some systems which do SPF outside DMARC, and there having -all might shortcircuit DMARC out from the equation, we should provide guidance to those people how they can get best results in current environment. Thus the best current practice should be use to use ~all instead of -all if you are trying to use DMARC, and want other systems to actually act based on your DMARC policy. > Anything like this belongs in an operational guidance document, not in the > protocol description. I have no problem describing the trade offs in an > appropriate document, but I don't think this is it. > -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Overall last-call comments on DMARC
Murray S. Kucherawy writes: > Some sort of contract or agreement between sender and receiver > seems to me to be unavoidable if we want to leverage ARC without > having a global domain reputation system. We don't have a > precise method to do that. We need to experiment and > standardize something to that extent, which I hope this WG can > do after publishing -bis. > > I know what "contract" means abstractly, but what does this actually > look like to someone that's looking for specific guidance? The text > you have here, by itself, is vague and I don't think many operators > will know what to do with it. For example Fastmail [1] includes per user account configuration that lists "Forwarding hosts", which affect how they do spam filtering and whether they trust ARC or not (they do have global trusted ARC list also). The M3AAWG forwarding whitepaper will propose that all mailbox providers should include similar setting, i.e., allow users to configure which hosts to trust for ARC. It was already pointed out that forwarding does not happen out of blue, there is always the user setting it up, i.e., joining the mailing list, providing the email address for alumni forwarding etc. When user does that it would also be easy for him to go to the account settings of whatever mailbox provider he uses and add that ARC host there. The mailbox provider could even detect that user is getting emails that are been forwarded and which have ARC headers, and they could even ask similar question they do now when you move mails away from spam folder, i.e., "Not spam", "This email has valid ARC signature for alumni.university.edu, have you configured this organization for forwarding emails to you, and if so do you trust this organization for doing mail authentication checks on behalf of us". What ARC really offers is that if there is ARC header from organization you trust, you can check the ARC-Authentication-Results and use them in addition to your own checks. If for example that header says SPF was pass, and you trust that domain, then you can trust that it properly did SPF checks and you can consider using ARC SPF pass as SPF pass for the email, even when it is now failing. I do not think there will ever be global trusted ARC signers list, as I do for example want to trust certain organizations / countries, and there is no point of me trusting for example microsoft.com ARC signatures, as there should not be forwarders in microsoft that should be forwarding emails to me. If there is ARC header signed by microsoft that header does not have any value for me, but will have some value for some other people. Simiarly I will trust iki.fi (non profit email forwarding service in Finland that will forward all emails I receive to my actual mailbox), but there is no point of you personally to trust iki.fi ARC signatures. Mailbox provides might want to trust iki.fi as one of our 3 members might be using your services, thus adding iki.fi to trusted forwarders makes thins easier for iki.fi members. [1] https://www.fastmail.help/hc/en-us/articles/360060591413-Spam-filtering#forwarding -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
[dmarc-ietf] WGLC contentious topics (I'm in the rough and I know it) in draft-ietf-dmarc-dmarcbis-30
Seth Blank writes: > In order from most to least contentious: > > 1. 8.6. Interoperability Considerations > > "It is therefore critical that domains that host users who might > post messages to mailing lists SHOULD NOT publish p=reject." > > While this advice represents consensus, it does not represent > operational best practice, nor where the market is moving to stop > fraud and abuse. DMARC is becoming increasingly required at the > major mail receivers, and messages that cannot get a DMARC pass are > increasingly likely to get rejected outright. This language feels > like it is creating an even worse interoperability problem-- giving > guidance to people that will guarantee their mail gets rejected by > major mail systems. These DMARC mandates arose after consensus was > called, and have changed the playing field materially. > > More accurate language that alleviates the concern would be "It is > therefore critical that domains that host users who wish for their > messages to be modified and spoofed by downstream intermediaries, > such as alumni forwarders or mailing lists, SHOULD NOT publish > p=reject. Such spoofed messages may still be rejected, regardless of > a domain owner's published DMARC policy." I do not think the text above is more accurate, I think is mostly wrong. The current text in the draft is correct (or even better would be to say MUST NOT, but I think we lost that consensus call). The only reason intermediaries might be considered to be "spoofing" anything is because of the use of SPF. It is completely possible to implement mail forwarding which keeps DKIM valid, thus DMARC is still fine, but there is no way to keep SPF valid, and as not everybody implement DKIM this will cause mails to be rejected. But we lost that battle to remove SPF in the dmarcbis too... I think we should keep the current text. > 3. 4.4. Identifier Alignment Explained > > Since then, we've discussed two other removals -- relying on SPF at all, and > not worrying about mailing list traffic at all -- both of which affect more > mailflow and deployment than strict alignment. If we're willing to seriously > discuss things that affect more mail, should we not also review the need for > strict alignment? I myself would like to get both SPF and strict alignment to be removed from the draft. Some adminstrators are still going to use the SPF regardless what DMARC says, and some of them are already using it before DMARC which means they do not care about DMARC they only care about SPF, and there is nothing we can do for them (execpt we could say they are not compliant with DMARC). > I also don't think we really reviewed strict alignment after > incorporating the tree walk into the document. Most of the uses for > strict alignment were in the "I have a large organization, and want > to make sure one part of the organization cannot spoof another" > camp-- which now the tree walk should have provided a more scalable > solution for. My understanding is that with tree walk and psd=n for the "strict" domain will allow those who want to enforce strict alingment to do so, so there is no point of having two different ways in the draft to archieve same result. I.e., if you want strict alignment, for domain a.mail.example.com, simply publish DMARC record for _dmarc.a.mail.example.com which has psd=n and that will provide you strict alignment. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] WGLC editorial review of draft-ietf-dmarc-dmarcbis-30
Alessandro Vesely writes: > On Sun 31/Mar/2024 14:22:04 +0200 Douglas Foster wrote: > > On SPF, our document should say simply, > > " a DMARC-compliant evaluator MUST NOT reject a message, based on SPF > > result, > > prior to receiving the Data section and checking for aligned and verifiable > > signatures." > > > Nonsense. Rejecting at RCPT TO is much quicker than waiting for the whole > message. People who publish -all know what they do. > > I also reject based on RBLs and private IP lists; does that affect DMARC > compliance? Yes, either one of those practices are not using DMARC to validate the messages. Of course you are allowed to do whatever extra checks you want for the incoming emails, you can even reject ever email coming in from ip-address is even number, but that is not DMARC. To implement DMARC you have to follow the rules set in the DMARC. I.e. if you are implementing DMARC you MUST follow the rules set in section 5.7.2 and the step 3 requires you to do DKIM signature verifications checks, which you can't do if you reject email before the you even see the body that contains DKIM signatures. Actually you can't do steps 1 and 2 of the 5.7.2 if you reject email before body as you do not know RFC5322.From domain, so how can you claim to be implementing DMARC if you do not even load DMARC policy record. So you can do whatever extra checks you want, but those are not part of DMARC, and should not be considered here. If you actually implement DMARC, you already MUST NOT reject a message based on the SPF results prior to receiving data section, as that is already mandated by the section 5.7.2 dmarc, so saying that again in the draft is not adding any new requirements, it is simply restating the same requirement in different words for implementors just in case they did not properly understand the section 5.7.2. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DMARCbis WGLC Issue 136 - DMARC Records Can Be CNAMEs
John Levine writes: > It appears that Todd Herr said: > >I agree that clarifying it can't hurt, obviously, ... > > I disagree, it does hurt. > > If we say you're allowed to use CNAMEs to point to DMARC records, > people are to say uh oh, is there something special here? What about > DKIM records? what about SPF records? how about SPF includes? or SPF > redirects? > > Really, there is nothing to say here, so let's not say it. We could add an example Appendix B that uses CNAME, so that would give indication, yes of course you can use CNAMEs, without explicitly adding text that might cause confusion. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DMARCbis way forward: Do we need our session at IETF 118
Murray S. Kucherawy writes: > On Sun, Oct 29, 2023 at 12:35 PM Douglas Foster < > dougfoster.emailstanda...@gmail.com> wrote: > > By contrast, the new tag cannot be effective until DMARCbis is published > and filtering software updated. This involves years. Even then, domain > owners will never have confidence that the new token support has been > implemented by all recipient evaluators. > > At least on its face, this is a big concern. A domain owner publishing "auth= > dkim" is going to get varying results as some sites update to software > supporting such a tag while others ignore it. I don't know if the potential > for some benefit is a desirable trade for the potential for some confusion. Varying meaning, those who implement auth=dkim will get extra protection, and those who do not, are left as they are now. I myself think that adding clear indication that sender always uses dkim, and evaluators can rely on that makes the DMARC better, and more secure. And as every DMARC evaluator needs to support DKIM anyways (there is no such thing as SPF only DMARC evaluator, both SPF and DKIM are mandatory to implement), there is no real difference in complexity for evaluators. > Seems like a slam dunk for SPF neutral. I said the problem and it's > solution needs to be laid out in our document because I am one of those > who did not understand it as a possible strategy > > I think I agree. This will also change the behavior of the receivers. For example spamassasin does give more points to SPF_NEUTRAL (around 0.6-0.7) than SPF_PASS (-0.001) etc. I would assume other similar software also use SPF results similarly. The reason why SPF_PASS gives only -0.001 is that it is assumed that spammers will use their own domain and thus can get SPF records to match (DMARC, DKIM and SPF are never meant to work against spammers). -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DMARCbis way forward: Do we need our session at IETF 118
John Levine writes: > It appears that Scott Kitterman said: > >>* Is there consensus on moving ahead with the idea of a way to indicate > >>which authentication method(s) the Domain Owner wants Receivers to use? If > >>so, it doesn't seem to be in the document yet. > > > >I haven't seen any valid case for it yet. It adds complexity to > >little or no benefit. > > Normally I am in favor of keeping stuff simple, but I think in this case the > argument for "DKIM only" is quite strong. Actually removing SPF completely from DMARC would simplyfy the protocol a lot, and would solve several issues, where people use DMARC with only SPF, or claim to do dmarc, but do filtering based on the SPF records before getting to the actual email, thus not checking DKIM records at all. If the DMARC would only use DKIM, that would make it clear that if you want to publish DMARC records you needs to also use DKIM, and when checking DMARC records you need to check verify DKIM signatures. Whether you do SPF in addition to that before or after would be local implementation issue, and not part of the DMARC. There were people who wanted to keep SPF as part of the DMARC, who did not even do DMARC, because the used SPF only as a first step of filtering during the MAIL FROM phase (before being able to fetch DMARC records, or checking DKIM signatures)... > There's the counterargument "so don't publish SPF" but it's on so > many checklists that even though that would be a fine idea, it's not > practical. That is unfortunately true, but if we could decouple the DMARC from SPF, then at least we could fix the situation at some point... -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
[dmarc-ietf] Reflections on IETF 117 Conference and DMARC Meeting
Hector Santos writes: > o Concerning p=reject: > >- Our focus on p=reject should be expanded to include p=quarantine > as they're both challenging. We should perhaps categorize these under > 'Restrictive Policies'. I will repeat the point I made in the mic in IETF: I think DMARC should really be explaining what the senders is doing and not trying to give instructions to the receiver. I.e., the dmarc should tell that: 1) Sender does not authenticate email (cannot be expressed in dmarc currently) 2) Sender might be authenticating email, but there is some email that significant part of email flows that are not authenticated (p=none?) 3) Sender is trying to authenticate all email, but there might be some parts where there might still be some unauthenticated emails going through (p=quarantine?) 4) Sender is authenticating all emails and emails which fails authentication are not originating from the sender or has been modified in transit (p=reject?) There was people pointing out that there would still be usefull to know the senders policy what can be done to the email after knowing whether the authentication was successfull or not, i.e., p=reject, p=quarantine etc. I am not that sure this is something that needs to be expressed. The receiver will use its own policy and other information (i.e., the actual content of the email) to decide that anyways. The policy might be that emails are accepted, but there is big warning saying the email failed authentication and can be phishing attack (I think gmail has been doing that kind of warnings, in case emails from domains which usually have dkim signature are missing dkim signature), or the policy might be that in addition that the system will render all links in email in such way they can't be clicked etc. >- I highlighted that "SPF Comes First" before DMARC or DKIM is > known. It is entirely possible that an SPF restrictive policy (-ALL, > Hard Fail) can preempt the payload transfer, causing a rejection > before the DATA is reached. DMARCbis does acknowledge this > possibility, mentioning that receivers might process SPF rejects > before DMARC is known. As those implementations do SPF outside the DMARCbis context, there is nothing we can do for them, thus we do not even need to consider them. I.e., we can safely ignore that discussion from the DMARCbis, as it is outside the scope of DMARCbis. If they are the only people who want to keep requiring SPF for DMARCbis, then we can safely remove SPF from the DMARCbis, as only people who want to keep SPF in DMARCbis are not going to be implementing DMARCbis :-) Section 5.7.2 do require that implementations MUST perform DKIM signature verification checks, and also say MUST perform SPF verification checks, so any implementation which only does one is not conforming DMARCbis. If we can't change from the MUST to not mentioning SPF at all, perhaps we should go from MUST implement SPF to MAY implement SPF, i.e., say that yes, you can still use SPF, but it is no longer mandatory to implement. As DMARC is only small piece of the whole mail recipient policy engine, the overall engine is already using lots of other pieces and can and most likely will use SPF as input anyways, regardless whether we require it in 5.7.2 or not. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Eliminating From Munging from this list
Douglas Foster writes: > Baptiste's proposal is clearly the easiest to implement: Admins inform the > group that IETF is going to stop munging on a specific date. After that date, > subscribers are switched to digest mode if the MLM or the user detects > problems. Admins switch them back when the user reports that the list > traffic has an exception in place by the subscriber's evaluator. This > approach may also be sufficient for this list, as I suspect that most of our > evaluators will already have an exemption for IETF. I do not think there is any need for admins to switch users back. Users are already able to log in to list options page (https://www.ietf.org/mailman/options/dmarc for this list) and change Set Digest Mode option on or off. So if the mailing list receives bounce from specific user, and that user is not yet using digest mode, then mailing list would turn on digest mode. If it gets bounce from the digest email sent to the user, it would do whatever it now does when it receives bounces (I think they temporarely disable deliver, and after some time they will try again and after repeated failures they will remove user from the list or something like that). If user thinks his mail system is fixed so non digested emails works, he can log in to options page for the mailing list and turn of digest mode. If he was wrong and the non-digested emails do not work, then there will be bounce, and digest mode gets automatically turned on again. There is no point of asking admins to do anything for specific users, when users can do those operations themselves. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Another p=reject text proposal
Wei Chuang writes: > 2) The proposed language calls out "“alumni forwarders”, role-based email > aliases, and mailing lists" for consideration by receivers. How should > receivers be aware that traffic failing authentication should be reconsidered? > Mailing-lists sometimes uses RFC2919 List-id headers. Can Section 5.8 [1] > call out more strongly the application of those List-id headers? Can > something be done so that receivers can identify the other scenarios e.g. > role-based email aliases and alumni-forwarders? For alumni forwarders / role-based forwarders things are different, as users who set those up, actually knows who does the forwarding, and they themself recognize that emails coming to those addresses are important to them, and they also trust (university etc) the one doing forwarding. So if the alumni forwarder / role-based forwarder adds ARC signatures, then the final recipient can whitelist that ARC signer in their setup and say that the policy code that checks the SPF/DKIM/DMARC results can fully trust the signed ARC authentication results from that signer. This of course requires that the recipient SMTP server can't simply reject the incoming email after the MAIL FROM because SPF checks do not match, they actually need to receive the email so they can check ARC and DKIM headers... -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Another p=reject text proposal
Barry Leiba writes: > > 2) As others have observed, the mailing list problem is > > exclusively an evaluator error. An evaluator's job is to allow > > safe and wanted messages while blocking unsafe or unwanted > > messages. > > I disagree. As I and others have observed, those creating the problem > are the ones who are using p=reject in a way that it was not intended > to be used. If we simply want to "fix" the mailing list issue, the mailing lists could simply refuse any subscription attempt from address which publishes p=reject, and say that your email address is not compatible with mailing lists in general, use some other address when sending emails to mailing list. On the other hand mailing list adminstrators usually try to include everybody, and not block people if they can cope with them, and thats why they do From address rewriting etc instead of just rejecting subscription requests. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Another p=reject text proposal
Murray S. Kucherawy writes: > On Fri, Jul 7, 2023 at 6:35 PM Douglas Foster < > dougfoster.emailstanda...@gmail.com> wrote: > > Consequently, the problem remains: How does an evaluator distinguish > between a legitimate list and a malicious attack? > > If we had a reliable answer to that, this would've been over ages ago. > Unfortunately, any mechanism we create for lists to distinguish their traffic > can be trivially co-opted by bad actors. I think phishing attacks using mailing list format would not be as efficient as it would be to inpersonate some other user that the intended recipient is familiar with. Mailing list are also something that quite a lot people already have special filters for, i.e., direct them to separate mailbox, allow them to go through without spam checking etc. For mailing lists users actually joined willingly, the users are familiar to, and have ability to cope. If it is mailing list they got added without their real consent, then if some of those messages gets lost because it is run through spam filtering and they get some extra spam points because no dkim signature etc the user probably do not care even if they are thrown away. The problem with DMARC checking is that it is usually done too early, and without consulting intended recipient whitelist/policy etc. Users can't add rules that say that mailing lists having DKIM signature of header.d=ietf.org are ok, and should get through even when the DMARC checks fails. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Another p=reject text proposal
Scott Kitterman writes: > You can equally argue that these receivers are merely following the > policy advice provided by the sending domain (it has reject right in > the name) and this problem is entirely generated by sender's > inappropriate use of p=reject. True, thats why the proposed text also says you SHOULD NOT put p=reject... :-) > I don't think engineering the location where the blame lands is the > right place to focus. I've done plenty of blame avoidance > engineering in my day, but I don't think it's what the IETF should > be doing. It would be much better when people would really remember that having valid or not valid DMARC/DKIM/SPF for email does not tell anything whether the email is bad/spam/unwanted. Regardless whether the DMARC/DKIM/SPF is valid you always need to use other methods to filter emails, so as you need to do that anyways while receiving emails, there is no problems of using same things also for p=reject messages. You can use the failed dmarc with p=reject as one of the inputs to feed your actual email filtering system, the dmarc p=reject SHOULD NOT BE your only email filtering system. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Another p=reject text proposal
Brotman, Alex writes: > I suspect the portion that instructs receivers to not act solely on > p=reject may be ignored by a fair set of receivers. I'm not > necessarily opposed to the language below, just that it seems odd to > create language that we know will be ignored. If receivers ignore that, then at least we can complain to them and say that you should not do that, as the RFC says you should use other information too if they want to get important forwarded emails through. For example we in iki.fi have been regularly been helping people with their broken spf etc records which break forwarding, and several times we have actually managed to explain the situation and they have change their settings. Quite often those people simply follow whatever some consultant etc suggested, and they did not understood at all that they at the same time broke other things. Quite often they do want to reduce the amoung of support calls, and if they get support calls every time some forwarded email from mailing list or from forwarding gets rejected, they most likely will notice that and fix their setup. > Additionally, I find it odd that we won't tell forwarders how to > munge messages to avoid this situation, but we will tell receivers > how to avoid this situation. There is no good way for forwards to mungle message in such way that return path/DKIM/user expectations stays intact. I myself for example find the From address mangling done by this mailing list really annoying as I need to use extra time to parse the =40 addresses to find out domain name of the sender. For mailing list this is just annoying but you can do that, but for regular forwarding you can't do that as you want to keep the DKIM signature intact. And this problem is not generated by forwarders, it is generated by the receivers who only use DMARC and no other information while rejecting emails. So there is no point of asking forwarders to fix things that was broken by DMARC... -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] easier DKIM, DMARC2 & SPF Dependency Removal
Jan Dušátko writes: > Scott, Barry, > as far as I understand, SPF are historic technology, but still have a > reason why to use it. In my opinion (and concerns), it is also necessary > to be aware of the extension of the individual protection methods > provided by senders (amount of domains). This is not a deep analysis. It > is possible that the numbers given will not be accurate. > SPF 87% > DKIM 13% > DMARC 63% > ARC 5% How were those values calculated? They are very different compared to the actual mails going through different servers. Is this calculated from the domains used, not from emails processed? How many emails was used to calculate this statistics, how many domains were there in total? > In terms of provability, each DKIM signed email will be sent from the > owner of the authorized servers. Ideally, therefore, it is not necessary > to combine SPF and DKIM, only DKIM could be sufficient. Yes. > However, the following situations are a problem. As a rule, > DKIM+DMARC is not fully implemented, part of the domains are > protected by DKIM, part by DKIM + SPF, part by SPF only. Thats why I think it is important that we get more DKIM implementations for DMARC. > - Only DKIM and DMARC are implemented, yet the SPAM distributor sends an > email without the DKIM signature and without the key information. SPAM distributors are not interesting here, DMARC/DKIM/SPF are not protecting against spam, there is no way they could ever be efficient for that. If there is no DKIM signature the message is not authenticated to originate from the domain where it claims to be, thus it will not gain from the possible whitelisting, better reputation etc associated with the domain. > - Only DKIM and DMARC are implemented, and a signed email is available. > The SPAM distributor starts repeatedly sending the same email > (equivalent to a DOS attack). It is very easy to filter identical emails, especially when they are actually signed with same dkim signature, thus you can safely mark later emails as duplicates, and not deliver them to mailbox. I do not think there is ever need to deliver duplicate emails to same mailbox, thus you can safely remove them. If the RCPT TO is different than the To-header inside the body, that is one of the checks that spam checking software quite often use to check whether the email is spam. I.e., if the To-header does not have actual address of the mailbox the message is supposed to be delivered to, then give some positive spam points to the message. If this is forwarded email (i.e. the to-address contains something like kivi...@iki.fi, instead the address where my emails finally gets forwarded to) then the user who set up the forwarding most likely already added those addresses as "valid recipient addresses" to their mail settings, thus forwarded emails do not get marked as spam. And of course if thousands of different mailboxes in your system are getting same DKIM signed message, and some of the users start marking it as spam, that is good indication that others are too, thus your spam checking software can learn from that and start processing those emails differently. > - Only DKIM and DMARC are implemented, and a signed email based on > RSA+SHA1 is available. Because it is possible to find collisions for > SHA1, and the digital signature is actually an "encrypted hash," it is > possible to send counterfeit mail with a signature that looks genuine. > The solution is to use explicitly h=sha2, but if not specified, it is > also possible to "downgrade the signature" against another key on this > domain supporting SHA1 (any SHA1-based signature will be used). In stackexchange there was question about this: https://crypto.stackexchange.com/questions/99767/how-easy-is-it-in-2022-to-find-a-sha1-collision The first answer was that using single GPU it would take about 5.3 years to find collision (year 2022). Getting 30 GPUs would drop that to 2 months, and those GPUs would cost about $20k, and the electricity for those GPUs would cost you about $2k for 2 months. Using AWS it would still take months and cost you few hundred dollars. And you need to do that for every single email you want to counterfeit. For phishing attacks this would be feasible, i.e., if you want to take real email from the CEO, change it and find collsion and send it forward taking few months and costing hundreds or thousands of dollars would be ok. So companies who actually care about the security in DKIM should NOT USE SHA1. For spammers this is not a feasible solution. Also if you are using unsecure crypto parameters for DKIM (short keys, broken hashes) you should also change those keys every now and then... > These attacks are the first that came to my mind. How we can mitigate > that threat? According to owner authorization of IP addresses, they > protect against a different kind of attack than the digital signature by > using DKIM. You did not describe any threat
Re: [dmarc-ietf] easier DKIM, DMARC2 & SPF Dependency Removal
Alessandro Vesely writes: > On Mon 26/Jun/2023 19:32:53 +0200 Douglas Foster wrote: > > DKIM+SPF says "our domain, including subdomains covered by this policy, > > will never use an ESP". (Since most ESP messages pass SPF based on the ESP > > domain) That is incorrect. It would also mean we will NEVER send email to anybody who would want to forward that email to some other place. How is that adminstrator putting the DKIM+SPF policy be sure of that? If they ever send any emails to customers, contractors, friends etc, they can never be sure those emails are not forwarded, thus they can't say DKIM+SPF. I can easyly see some big companies wanting to lock in their customers by making it hard to change email addresses, but we should build or protocols for those companies, but instead allow freedom of movement where people are reading their emails. > ESPs can provide include files for those who wish otherwise. I know that some companies in finland has included the iki.fi IP-addresses ranges to their SPF records, because they had several complains from people of SPF failures when they were sending emails to iki.fi addresses. We even added _spf.iki.fi DNS record for them to use for their include when it was requested, but I do not consider that good practice for solving issues of the SPF. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DMARC2 & SPF Dependency Removal
Tero Kivinen writes: > > What are those 0.75%, some 30k SPF - DKIM messages? Are there > > cases of DKIM random failure salvaged by SPF? > > My current analysis script does not try to calculate that, I would > need to need to add that step there and rerun the script. If I > understand correctly you would like to see cases where if there is > both SPF and DKIM, the cases where the both, only one, or neither > passed, and how many of those cases would be where dkim=fail, but > spf=pass? I rerun the statistics and yes, there is 0.84% cases where dkim failed, but spf returned eithe pass, softfail or neutral. There was also 1.12% cases where spf returned permerror but dkim returned pass, and 1.26% cases where dkim returned pass and spf returned fail, or softfail. There is of course much bigger part of emails where there was no dkim, but there was spf that passed (7.03%) or softfailed (1.08%). Here are the actual numbers for DKIM and SPF result combinations: DKIM & SPF combinations === 78.62% 3133749 dkim=pass,spf=pass 7.03% 280239 dkim=none,spf=pass 4.68% 186634 dkim=pass,spf=none 3.85% 153543 dkim=none,spf=none 1.12% 44543 dkim=pass,spf=permerror 1.08% 43212 dkim=none,spf=softfail 0.82% 32821 dkim=fail,spf=pass 0.78% 30953 dkim=pass,spf=softfail 0.61% 24221 dkim=none,spf=fail 0.48% 19329 dkim=pass,spf=fail 0.43% 17120 dkim=none,spf=neutral 0.24% 9612dkim=fail,spf=none 0.06% 2320dkim=none,spf=permerror 0.06% 2214dkim=pass,spf=neutral 0.04% 1712dkim=none,spf=temperror 0.02% 995 dkim=fail,spf=fail 0.02% 924 dkim=fail,spf=softfail 0.02% 669 dkim=temperror,spf=pass 0.01% 360 dkim=missing,spf=missing 0.00% 199 dkim=temperror,spf=temperror 0.00% 196 dkim=fail,spf=neutral 0.00% 144 dkim=missing,spf=none 0.00% 119 dkim=pass,spf=temperror 0.00% 99 dkim=missing,spf=pass 0.00% 50 dkim=fail,spf=permerror 0.00% 38 dkim=missing,spf=softfail 0.00% 14 dkim=temperror,spf=none 0.00% 10 dkim=temperror,spf=softfail 0.00% 7 dkim=missing,spf=fail 0.00% 6 dkim=fail,spf=temperror 0.00% 6 dkim=missing,spf=neutral 0.00% 1 dkim=temperror,spf=fail 0.00% 1 dkim=missing,spf=temperror I.e. 78.64% of time both DKIM and SPF passed. I also calculated statistics for all DKIM, SPF, DMARC, and ARC combinations, but there were so many of them that I do not include the full list here but here is top 30 from that list: Protocol combinations 37.74% 1504477 dkim=pass,spf=pass,dmarc=missing,arc=missing 25.37% 1011277 dkim=pass,spf=pass,dmarc=pass,arc=missing 10.96% 436838 dkim=pass,spf=pass,dmarc=none,arc=missing 3.46% 138083 dkim=none,spf=pass,dmarc=missing,arc=missing 2.15% 85799 dkim=pass,spf=none,dmarc=missing,arc=missing 2.00% 79739 dkim=pass,spf=none,dmarc=pass,arc=missing 1.96% 78279 dkim=none,spf=none,dmarc=missing,arc=missing 1.64% 65205 dkim=none,spf=pass,dmarc=none,arc=missing 1.60% 63758 dkim=pass,spf=pass,dmarc=missing,arc=pass 1.54% 61579 dkim=none,spf=pass,dmarc=pass,arc=missing 1.16% 46309 dkim=pass,spf=pass,dmarc=pass,arc=pass 1.09% 43529 dkim=none,spf=none,dmarc=fail,arc=missing 0.92% 36478 dkim=pass,spf=pass,dmarc=fail,arc=missing 0.79% 31298 dkim=none,spf=none,dmarc=none,arc=missing 0.56% 22504 dkim=none,spf=softfail,dmarc=missing,arc=missing 0.56% 22123 dkim=pass,spf=permerror,dmarc=missing,arc=missing 0.55% 21973 dkim=pass,spf=pass,dmarc=none,arc=pass 0.40% 15760 dkim=fail,spf=pass,dmarc=missing,arc=missing 0.37% 14855 dkim=none,spf=softfail,dmarc=fail,arc=missing 0.37% 14716 dkim=pass,spf=softfail,dmarc=missing,arc=missing 0.34% 13576 dkim=none,spf=fail,dmarc=missing,arc=missing 0.32% 12745 dkim=pass,spf=permerror,dmarc=none,arc=missing 0.31% 12348 dkim=pass,spf=softfail,dmarc=pass,arc=missing 0.26% 10290 dkim=none,spf=neutral,dmarc=missing,arc=missing 0.24% 9657dkim=pass,spf=permerror,dmarc=pass,arc=missing 0.23% 9367dkim=pass,spf=fail,dmarc=missing,arc=missing 0.20% 8121dkim=pass,spf=fail,dmarc=pass,arc=missing 0.20% 7785dkim=fail,spf=pass,dmarc=none,arc=missing 0.17% 6719dkim=pass,spf=none,dmarc=missing,arc=pass 0.16% 6248dkim=none,spf=pass,dmarc=fai
Re: [dmarc-ietf] DMARC2 & SPF Dependency Removal
Alessandro Vesely writes: > On Tue 13/Jun/2023 23:33:50 +0200 Tero Kivinen wrote: > > [...] > > > > As you can see 85.75% of incoming email was already signed by DKIM, > > and 86.5% of emails had SPF records that passed. So they both have > > about same amount if usage coming in to our servers. > > > What are those 0.75%, some 30k SPF - DKIM messages? Are there cases of DKIM > random failure salvaged by SPF? My current analysis script does not try to calculate that, I would need to need to add that step there and rerun the script. If I understand correctly you would like to see cases where if there is both SPF and DKIM, the cases where the both, only one, or neither passed, and how many of those cases would be where dkim=fail, but spf=pass? I will try to see if I can run the that check later. > > 0.19% 7506none,pass > > 0.15% 5910pass,none > > How do you order DKIM signatures? My understanding is that rspamd most likely uses the order of DKIM signatures in the email body. On the other hand order does not matter, as if ANY of the dkim checks pass, then the whole message passes. The reason I printed out the combinations of different dkim results was to show that there are cases where there is multiple dkim headers and some of those pass and some fail. I.e there were: 0.00% 4 pass,fail,fail,fail,fail 0.00% 2 pass,pass,pass,pass,pass,pass I.e. four emails had five dkim records, four of them failing and one passing, where another two one had six dkim records all passing. Most of the emails had oly one dkim record, and those of which had two most of them were so that both passed. -- kivi...@iki.fi ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DMARC2 & SPF Dependency Removal
Murray S. Kucherawy writes:
> On Tue, Jun 13, 2023 at 10:34 PM Tero Kivinen wrote:
>
> DKIM failures
>
> 36.34% 26619 invalid DKIM record
>
> This is staggering. Can you characterize what the most common malformations
> are?
I think most of those are missing keys. I.e., there is no key in the
dns at all for that header.d and header.s.
This might be caused by having some internal machine doing the DKIM
signing but not publishing the actual DKIM records in the dns at all.
Sometimes there is another DKIM record that will pass like this:
ARC-Authentication-Results: i=1;
MTA-v4;
dkim=none ("invalid DKIM record") header.d=ernieball.com
header.s=ci-ernieball header.b=XXX;
dkim=pass header.d=criticalimpactinc.com header.s=keyd header.b=XXX;
spf=pass (MTA-v4: XXX)
Sometimes there that was the only dkim record and then the final
result is fail:
ARC-Authentication-Results: i=1;
MTA-v4;
dkim=none ("invalid DKIM record") header.d=autostadium.fi header.s=x
header.b=XXX;
spf=pass (MTA-v4: XXX)
Note, that those are not really failures, I calculated those error
messages from dkim=none result to the statistics, as it indicates that
there was DKIM record in email, but DKIM was not set properly, so in
sense it is DKIM error, but if I remember right DKIM specification
says that not having DKIM record, or having missing keys etc in dns
are no different from each other, so both are DKIM=none...
--
kivi...@iki.fi
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DMARC2 & SPF Dependency Removal
Barry Leiba writes: > > DKIM only: ~99.5% > > DKIM + SPF: ~100% > > SPF only: ~100% > > That's interesting and disturbing if it remains consistent. The statistics I have are quite different. The failure rate is much bigger both in DKIM and SPF. Following statistics is random subset of emails going through iki.fi system, from last 30 days, consisting bit less than 4 million emails. Iki.fi is email forwarding service, so about 90% of those emails will fail SPF checks after iki.fi sends them forward. DKIM will go through unmodified, and we do not modify normal messages (spam messages might get tagged as spam depending on the members configuration), so 85.75% of emails will still have valid DKIM signature after passing iki. We do graylisting of blacklisted ip-addresses, thus spammers that do not work around graylisting are not part of the statistics. There is significant amount of mailing lists going through iki, and quickly checking that 1.58% of emails going through has spf-errors, dkim signers or similar with domain name in form of list.domain or lists.domain, so that will cause some of the SPF and DKIM failures. Note, that this only counts cases where the domain name was used in the verification and printed in the logs i.e., only in error cases. As we are using ARC, and we add ARC-Authentication-Results header to all emails as first step when they come in, and I used those headers to generate these statistics. First some generic statistics: Number of ARC-header levels === 95.61% 3811208 1 3.83% 152487 2 0.44% 17711 3 0.09% 35864 0.01% 460 5 0.01% 349 6 0.01% 207 7 0.00% 36 8 0.00% 15 9 0.00% 1 10 Mailer == 91.96% 3665744 MTA-v4 8.04% 320315 MTA-v6 0.00% 1 MSA So 3.83% of emails already had one ARC header, and 0.56% had more than one arc header, with exactly one email having 10 ARC-Authentication-Results headers. Most of the emails do not have ARC headers. 92% of traffic came in using IPv4.. Then lets compare DKIM, SPF, DMARC and ARC results DKIM summary results = 85.75% 3417541 pass 13.11% 522367 none 1.12% 44604 fail 0.02% 893 temperror SPF results = 86.50% 3447577 pass 8.78% 349947 none 1.89% 75137 softfail 1.18% 46913 permerror 1.12% 44553 fail 0.49% 19536 neutral 0.05% 2037temperror DMARC results = 62.82% 1243393 pass 30.99% 613478 none 6.05% 119800 fail 0.08% 1485temperror 0.06% 1244permerror ARC results = 91.66% 160268 pass 8.34% 14584 reject As you can see 85.75% of incoming email was already signed by DKIM, and 86.5% of emails had SPF records that passed. So they both have about same amount if usage coming in to our servers. The difference is that only 1.14% of emails had errors (fail, or temperror) in their DKIM signatures (most of those were because the email was from the mailing list that modified the body, but did not generate new DKIM header), compared to the 4.24% of emails having SPF failures (softfail, permerror, fail or temperror). Meaning there were much more emails that failed SPF than DKIM. Even if we ignore the softfails, we still have about double the emails failing (2.35%). Note, that the dmarc and arc statistics are not from all of the emails, it only includes those which actually had DMARC or ARC information. For dmarc this was about 50%, and for ARC it was only 4.3% of all emails. Here are some statistics abut the DKIM processing and the error cases. 76.75% had one DKIM signature, and over 20% had more than one signature. Here is number of DKIM signatures and their results, i.e., 22.22% of emails had two DKIM signatures both passing, and 0.34% had one signature that passed, and another that failed etc: DKIM results === 62.67% 2497633 pass 22.22% 885372 pass,pass 13.06% 520332 none 1.04% 41477 fail 0.34% 13353 pass,fail 0.19% 7506none,pass 0.15% 5910pass,none 0.07% 2635fail,fail 0.06% 2235pass,pass,pass 0.05% 2034none,none 0.03% 1296pass,pass,pass,pass 0.03% 1026pass,pass,fail 0.03% 1002fail,pass 0.02% 892 temperror 0.02% 631 pass,fail,fail 0.01% 583 pass,none,none 0.01% 369 fail,fail,fail 0.01% 356 fail,fail,pass 0.01% 335
