Re: [dmarc-ietf] Tree Walk impact
Neil, this is a delayed reply to your earlier email about my implementation efforts, and whether I am on-board with the Tree Walk. The response has been delayed while I investigated the algorithm. I began by building a cache of DMARC policies using the PSL and my mail stream as input. The first thing that jumped out was the DNS results of "timeout" (or "Server failed"). After several re-runs, I have the list of recurring problems down to 87 domains (79 from the PSL and 8 from my list). I envision a real performance bottleneck if timeouts occur in high volume due to a concentration of messages on a specific DNS server. To avoid timeouts and obtain other benefits, I concluded that a database-based cache is necessary. I envision using database cache lifetimes that are much longer than DNS TTLs, probably at least a week. This avoids the performance risk associated with short TTLs in DNS. It also provides a framework for configuring overrides, such as to correct a policy that specifies strict alignment when the actual messages from the domain require relaxed alignment. The resulting code requires a dance: (1) check for a result using the cache, (2) Return a success result if all path elements are in the database and those entries have not expired. Otherwise, return a cache failure. (3) On cache failure, walk the tree to reload the cache, then (4) requery the cache to get a final result. (5) repeat for each DKIM domain, as well as the mail from domain and message from domain. (6) Exit when a domain check produces a final result. Some domains, especially TLDs, could be given a long cache life to avoid repeated queries for unchanging data, but then the cache protocol needs to provide feedback on the portion of the domain path that needs to be re-queried, which adds complexity. Overall, this algorithm feels like an order of magnitude more difficult to write and to use than the PSL lookup. When the tree walk cache works, the result will be a database of domains, with indicators of whether the domain has a policy or not, and the policy contents when one exists. Scaling becomes the next concern. Because we are tracking descendent paths from PSL entries, rather than PSL entries themselves, the database size will be multiple orders of magnitude larger than the PSL table. The cache size can be contained by reducing cache life and increasing garbage collection, but the cache reduction reduces filtering performance and garbage collection increases general overhead. In short, I conclude that the Tree Walk has a very large cost for the comparatively small benefit of avoiding specific PSL errors. But fixing PSL errors is still desirable, so another approach is needed. What makes more sense is to collect domain lists from the email logs, possibly supplemented from other sources. Then perform Tree Walks in a background process. The background process would identify domains where the Tree Walk produces a different result than the PSL, so that the difference can be investigated and the local-copy PSL corrected.Perhaps some web service will even do the work and publish results to its clients or the whole web. As a batch algorithm, the Tree Walk has potential. As a real-time algorithm, the Tree Walk algorithm seems like a poor fit. Doug Foster On Fri, Oct 13, 2023 at 3:29 PM Neil Anuskiewicz wrote: > If I read that right it gives you what you think is a desirable outcome. > That is, this might be a strong sign that you’re at least considering > supporting DMARCbis! > > Yes, we all need to be prepared for headaches no matter which direction > this all goes. > > ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DBOUND
On Oct 16, 2023, at 6:48 PM, Neil Anuskiewicz wrote: > > >> On Oct 16, 2023, at 6:43 PM, Seth Blank >> wrote: >> >> I'm sorry, to what are you referring? I co-chair the M3AAWG technical >> committee, and am unaware of any advocacy for relitigating DBOUND... Seth, I based my statement on wording in a couple documents and I heard it from other sources but I think I’m just flat out wrong w3aawg is supporting DBOUND. I did not mean it as a pejorative, it was more interest in why they might be supporting it. I don’t anything about DBOUND. It looked like the project was a an active project, so I was curious about it and why M3AAWG might have taken an interest. But jumping from that to support was too big a leap. My goal for being on this list is to learn and that’s been great. So the next thing would be how to use what I learn to be helpful. It’s challenging to figure out how to be helpful to the WG so I’ve been thinking of other ways such as other content to help others understand. The reference to DBOUND was sparked by looking around at what w3aawg was saying as I generally think they have some good papers and a great video series on email authentication. But what I found on DBOUND isn’t endorsement. They framed DBOUND in a positive light and encouraged people to look into the project but that doesn’t mean the organization endorses it. The last thing I wanted to do is offend anyone here. I’d like to continue to learn and possibly contribute to the larger DMARCbis effort. So again, I apologize. Neil ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DBOUND
It appears that Neil Anuskiewicz said: >-=-=-=-=-=- >See section V Enaging with DBOUND is not the same thing as advocating that DBOUND do anything specific. Everyone agrees the PSL is awful, including the people who maintain it. But there's no agreement at all about what would be better. DMARC managed to get away from the PSL because unlike every other PSL application, it can put its own markers in the DNS. Quite a while ago I came up with a tricky way to put the whole PSL in the DNS and do look ups efficiently (one lookup per boundary, which typically means one lookup total.) But people who write web browsers that use it to manage cookies said no, that's too slow, and we haven't made any perceptible progress since then. R's, John >> On Oct 16, 2023, at 6:43 PM, Seth Blank >> wrote: >> > > >> I'm sorry, to what are you referring? I co-chair the M3AAWG technical >> committee, and am unaware of any advocacy for relitigating DBOUND... >> On Mon, Oct 16, 2023 at 6:36� �PM Neil Anuskiewicz >> wrote: >> >> M3AAWG is advocating for DBOUND as most of you likely know. Does it seem >> a viable alternative? We could sure use the support of M3AAWG. How could >> we earn their support or are they committed to DBOUND? >> >> Perhaps once we prove that DMARCbis works they� ll reconsider. >> >> Neil ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DBOUND
See section VOn Oct 16, 2023, at 6:43 PM, Seth Blank wrote:I'm sorry, to what are you referring? I co-chair the M3AAWG technical committee, and am unaware of any advocacy for relitigating DBOUND...On Mon, Oct 16, 2023 at 6:36 PM Neil Anuskiewicz 40marmot-tech@dmarc.ietf.org> wrote:M3AAWG is advocating for DBOUND as most of you likely know. Does it seem a viable alternative? We could sure use the support of M3AAWG. How could we earn their support or are they committed to DBOUND? Perhaps once we prove that DMARCbis works they’ll reconsider. Neil ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc -- Seth Blank | Chief Technology Officere: s...@valimail.comp: This email and all data transmitted with it contains confidential and/or proprietary information intended solely for the use of individual(s) authorized to receive it. If you are not an intended and authorized recipient you are hereby notified of any use, disclosure, copying or distribution of the information included in this transmission is prohibited and may be unlawful. Please immediately notify the sender by replying to this email and then delete it from your system. ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DBOUND
On Oct 16, 2023, at 6:43 PM, Seth Blank wrote:I'm sorry, to what are you referring? I co-chair the M3AAWG technical committee, and am unaware of any advocacy for relitigating DBOUND...On Mon, Oct 16, 2023 at 6:36 PM Neil Anuskiewicz 40marmot-tech@dmarc.ietf.org> wrote:M3AAWG is advocating for DBOUND as most of you likely know. Does it seem a viable alternative? We could sure use the support of M3AAWG. How could we earn their support or are they committed to DBOUND? Perhaps once we prove that DMARCbis works they’ll reconsider. Neil ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc In this document on the site, toward the bottom they encourage support for it. I found the same thing in another similar document. https://www.m3aawg.org/sites/default/files/m3aawg_present_and_future_of_the_public_suffix_list.docx_.pdfNo offense intended, sir. I apologize if I’m wrong. It does appear they are supporting it by the wording.Neil___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] DBOUND
I'm sorry, to what are you referring? I co-chair the M3AAWG technical committee, and am unaware of any advocacy for relitigating DBOUND... On Mon, Oct 16, 2023 at 6:36 PM Neil Anuskiewicz wrote: > M3AAWG is advocating for DBOUND as most of you likely know. Does it seem a > viable alternative? We could sure use the support of M3AAWG. How could we > earn their support or are they committed to DBOUND? > > Perhaps once we prove that DMARCbis works they’ll reconsider. > > Neil > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc > -- *Seth Blank * | Chief Technology Officer *e:* s...@valimail.com *p:* This email and all data transmitted with it contains confidential and/or proprietary information intended solely for the use of individual(s) authorized to receive it. If you are not an intended and authorized recipient you are hereby notified of any use, disclosure, copying or distribution of the information included in this transmission is prohibited and may be unlawful. Please immediately notify the sender by replying to this email and then delete it from your system. ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
[dmarc-ietf] DBOUND
M3AAWG is advocating for DBOUND as most of you likely know. Does it seem a viable alternative? We could sure use the support of M3AAWG. How could we earn their support or are they committed to DBOUND? Perhaps once we prove that DMARCbis works they’ll reconsider. Neil ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Getting the word out and educating
> On Oct 16, 2023, at 4:53 PM, Dotzero wrote > https://www.m3aawg.org/ had sessions on DMARCbis, DKIM replay, etc. at their > meeting in Brooklyn last week. I did not attend. A number of ISPs, mailbox > providers, ESPs, etc. participate. I don't see it as something IETF organizes. > > Michael Hammer > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc That makes sense. So I could write on the topic, videos, etc., on my own speaking for nobody other than myself and that wouldn’t step on toes it sounds like. I just wanted to make sure as I’m considering creating some content meant to help. I could create content for technical audiences who might have basic knowledge of email authentication and, because of their role, they want a primer on what’s coming down the pike. Neil___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Getting the word out and educating
On Mon, Oct 16, 2023 at 7:14 PM Neil Anuskiewicz wrote: > Outside of the official business of the WG, is there a mechanism for > educating people about DMARCbis. Why is it important? What problems does it > solve? What’s different? I think some of it might even surprise some > people: You mean we have been relying on files of public suffix domains all > this time? How exactly does the tree walk work? > > The RFC is great but won’t be enough for quite a few people. To get > adoption we need content for different audiences, I think. > > So my thought is it would be useful to have content such as articles, > videos, even a website that explains why the changes are important and how > they work. I realize the RFC does explain things but I think more will be > needed. > > Is that something people just would do on their own or is this more of a > coordinated process? > > I realize there’s dmarc.org but I think there will be a need for more > explanatory material through multiple mediums. I think good articles and > videos will be key. > > Neil > https://www.m3aawg.org/ had sessions on DMARCbis, DKIM replay, etc. at their meeting in Brooklyn last week. I did not attend. A number of ISPs, mailbox providers, ESPs, etc. participate. I don't see it as something IETF organizes. Michael Hammer ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
[dmarc-ietf] Getting the word out and educating
Outside of the official business of the WG, is there a mechanism for educating people about DMARCbis. Why is it important? What problems does it solve? What’s different? I think some of it might even surprise some people: You mean we have been relying on files of public suffix domains all this time? How exactly does the tree walk work? The RFC is great but won’t be enough for quite a few people. To get adoption we need content for different audiences, I think. So my thought is it would be useful to have content such as articles, videos, even a website that explains why the changes are important and how they work. I realize the RFC does explain things but I think more will be needed. Is that something people just would do on their own or is this more of a coordinated process? I realize there’s dmarc.org but I think there will be a need for more explanatory material through multiple mediums. I think good articles and videos will be key. Neil ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Tree Walk impact
On October 16, 2023 9:20:26 PM UTC, Neil Anuskiewicz wrote: > > >> On Oct 16, 2023, at 11:00 AM, Scott Kitterman wrote: >> >> >> >>> On October 16, 2023 5:53:13 PM UTC, Alessandro Vesely >>> wrote: On Fri 13/Oct/2023 16:35:43 +0200 Neil Anuskiewicz wrote: Thank you, sir. That’s part of the reason to cautiously transition away from the PSL. It has the feel of a throwback to a time when people thought the number of total users would be in the hundreds or thousands. Wouldn’t a cautious transition alleviate your concerns? Not everyone, everywhere will pull the switch at midnight. >>> >>> >>> Can we engage ICANN for sending a kind request to upgrade their DMARC >>> records to all PSDs? Or can we do it on their behalf? Or on IETF behalf? >>> Or? >>> >>> Is that a subject for 118? >> >> Which ICANN managed TLDs have DMARC records (PSDs which are either not TLDs >> or not ICANN managed TLDs are not anything ICANN has anything to say about)? >> >> Scott K > >Are TLD’s typically responsive to these sorts of requests? Do they each have >to be sold on the idea? It was mostly a rhetorical question. I'm reasonably confident that no such TLDs exist. Scott K ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Tree Walk impact
> On Oct 16, 2023, at 11:00 AM, Scott Kitterman wrote: > > > >> On October 16, 2023 5:53:13 PM UTC, Alessandro Vesely wrote: >>> On Fri 13/Oct/2023 16:35:43 +0200 Neil Anuskiewicz wrote: >>> Thank you, sir. That’s part of the reason to cautiously transition away >>> from the PSL. It has the feel of a throwback to a time when people thought >>> the number of total users would be in the hundreds or thousands. Wouldn’t a >>> cautious transition alleviate your concerns? Not everyone, everywhere will >>> pull the switch at midnight. >> >> >> Can we engage ICANN for sending a kind request to upgrade their DMARC >> records to all PSDs? Or can we do it on their behalf? Or on IETF behalf? >> Or? >> >> Is that a subject for 118? > > Which ICANN managed TLDs have DMARC records (PSDs which are either not TLDs > or not ICANN managed TLDs are not anything ICANN has anything to say about)? > > Scott K > > ___ > dmarc mailing list > dmarc@ietf.org > https://www.ietf.org/mailman/listinfo/dmarc Are TLD’s typically responsive to these sorts of requests? Do they each have to be sold on the idea? Neil ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Tree Walk impact
On October 16, 2023 5:53:13 PM UTC, Alessandro Vesely wrote: >On Fri 13/Oct/2023 16:35:43 +0200 Neil Anuskiewicz wrote: >> Thank you, sir. That’s part of the reason to cautiously transition away from >> the PSL. It has the feel of a throwback to a time when people thought the >> number of total users would be in the hundreds or thousands. Wouldn’t a >> cautious transition alleviate your concerns? Not everyone, everywhere will >> pull the switch at midnight. > > >Can we engage ICANN for sending a kind request to upgrade their DMARC records >to all PSDs? Or can we do it on their behalf? Or on IETF behalf? Or? > >Is that a subject for 118? Which ICANN managed TLDs have DMARC records (PSDs which are either not TLDs or not ICANN managed TLDs are not anything ICANN has anything to say about)? Scott K ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Tree Walk impact
On Fri 13/Oct/2023 16:35:43 +0200 Neil Anuskiewicz wrote: Thank you, sir. That’s part of the reason to cautiously transition away from the PSL. It has the feel of a throwback to a time when people thought the number of total users would be in the hundreds or thousands. Wouldn’t a cautious transition alleviate your concerns? Not everyone, everywhere will pull the switch at midnight. Can we engage ICANN for sending a kind request to upgrade their DMARC records to all PSDs? Or can we do it on their behalf? Or on IETF behalf? Or? Is that a subject for 118? Best Ale -- ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
