Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Alessandro Vesely wrote:
On 08/May/11 19:13, Dave CROCKER wrote:
In practical terms for the current world, what is the likelihood that
a signer has any information about the 'type' of an email address?
How can a signer possibly know that an addressee is a mailing list???
Currently, it has to query the /time-distributed database/ brought
about by the spotty subscription reminders that MLMs send on April
fools' day and similar occasions.
Seriously, since it is a civic and sometimes legal duty to confirm
subscriptions, one may wonder why that database is not maintained by
means of present-day technologies. Every MTA would then have a list
of mass mailers, cross-linked to its users, to be looked up when
whitelisting, signing, resolving complaints, and, occasionally,
attesting (un)subscriptions.
Forgive the OT.
Well, Dave's perspective is that of an independent (3rd party) free
wheeling signer. The issue at hand regarding the need to use the l=
is that of an author domain signing his own mail and thus has already
selected a s= value to use, its own or a provisioned independent
signer domain. But it the author domain doing all the picking, setup
and signing - not the signer domain. Even if its the 3rd party, the
author domain can still dictate what it wants (see the ending comment).
In our current rev1 DKIM implementation, the outbound edge MTA does
the signing and it reads a setup file keyed on the From.domain or
List-ID.domain.
That is how it knows what to sign and with what options per domain
stream and this was the quickest and no MLM software change. It was
to handle the MLM list setups on our system or a customer system with
the same software to handle its mailing list.
In Rev2 it will updated to handle external mailing list (inbound)
using a Mail Conference Type factor and this is where the l=
option MAY APPLY.
For example, for our API, the conference types are:
enum {
mtNormalPublicPrivate,
mtNormalPublic,
mtNormalPrivate,
mtFidoNetmail,
mtEmailOnly,
mtUsenetNewsgroup,
mtUsenetNewsgroupModerated,
mtInternetMailingList,
mtFidoEcho,
mtListServeForum,
mtUserEmail,
mtEND = 256
};
The operator can prepare Internet Mailing List conference where list
traffic can be stored. For this conference type, it also enabled a
Reply Network Address. For example:
Name : IETF-DKIM
Type : Internet Mailing List
Network Address: ietf-dkim@mipassoc.org
This conference can be accessible by online users, including by News
Readers MUAs if the conference option is checked:
[X] Publish as a Local Newsgroup
Any (access allowed) user reading the conference can create or reply
and the mail is exported with a forced direction to the network address.
Now when we add DKIM, new options could be for a internet mailing
list conference type:
[X] Sign Mail
[X] Use Body Length tag
With this option enabled, our edge MTA will now have the setup info
to use the L= tag when signing it only for this specific network address.
Finally, it is not unheard off in the outsource or ISP/ESP/Anti-Spam
mail service business to provide a list of user addresses for your
system. In fact, before we added better anti-spam stuff, we offered
an utility to specifically export a complete or DIFF file of
acceptable users for the hosting service to accept, do whatever to
the mail if anything, and forward to mail to their smart host/MDA
system. This is how many early operators began to add Anti-Spam
protection to their mail.
So for DKIM, a domain that wishes to outsource the signing operation
to a 3rd party, can easily possibly to start using a similar manual or
email automated method to send a full/change list of domains and
addresses it wishes to be sign under specific options like a l=. In
fact, the # of domains/addresses may even be part of the fee schedule
for the service. This will cater to domains with an fast DKIM entry
point who do not wish the deal with the internal overhead and cost to
setup/maintenance DKIM.
--
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com
___
NOTE WELL: This list operates according to
http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
-Original Message- From: ietf-dkim-boun...@mipassoc.org [mailto:ietf-dkim- boun...@mipassoc.org] On Behalf Of John R. Levine Sent: Sunday, May 08, 2011 2:02 PM To: Dave CROCKER Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output 1. State principals that are specific to the content of the draft and that give guidance about the scope and boundaries of what should be covered. 2. Make specific suggestions for specific bits of text in the draft. Despite the valiant work that Murray has put into the MLM document, my preference, which I doubt has any hope of gaining consensus, would be to throw it away and replace it by one page that says a) many lists break signatures, which isn't going to stop b) so it would be nice if they signed their mail on the way out. Everything else is either too marginal to be worth worrying about, or not a problem if a list's mail has a credible signature.* Failing that, I don't see small changes making it any better, so just ship it. +1 ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 08/May/11 19:13, Dave CROCKER wrote: In practical terms for the current world, what is the likelihood that a signer has any information about the 'type' of an email address? How can a signer possibly know that an addressee is a mailing list??? Currently, it has to query the /time-distributed database/ brought about by the spotty subscription reminders that MLMs send on April fools' day and similar occasions. Seriously, since it is a civic and sometimes legal duty to confirm subscriptions, one may wonder why that database is not maintained by means of present-day technologies. Every MTA would then have a list of mass mailers, cross-linked to its users, to be looked up when whitelisting, signing, resolving complaints, and, occasionally, attesting (un)subscriptions. Forgive the OT. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Despite the valiant work that Murray has put into the MLM document, my preference, which I doubt has any hope of gaining consensus, would be to throw it away and replace it by one page that says ... Failing that, I don't see small changes making it any better, so just ship it. +1 Hmmm, it occurs to me that the folks doing this form of +1 might mean ship it or they might mean preference for replacing with one-page doc or, failing that, ship it. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
-Original Message- From: ietf-dkim-boun...@mipassoc.org [mailto:ietf-dkim- boun...@mipassoc.org] On Behalf Of Barry Leiba Sent: Saturday, May 07, 2011 10:00 AM To: Hector Santos Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output We are spending an awful amount of time on this l= issue, whether it should be pulled, keep it and explaining how bad it is and discourage usage. Agreed. I would like to deprecate it. But we don't have consensus for going that far, and I think we're too late in the process to get ourselves mired in that. What we're doing now is just short of deprecating it -- saying that, well, you really shouldn't oughta use it, without being normative. The 6% using l= needlessly is a red flag. Yep. Happily, we (where we, here, mostly means Murray, but some others as well) are collecting stats. It's possible, later, for someone to create an individual submission for DKIM l= Considered Harmful, or some such, and perhaps if/when someone ever moves DKIM to full Standard we can actually deprecate l=. Barry, as participant Barry, I'd like to request that we specifically test for consensus on deprecating l= through the usual +1/-1 approach. No miring, just a vote. If we have consensus, great. If we don't have consensus then let's move on. My concern is that if this is not addressed now it will not be addressed in the future. My personal belief is that it should be deprecated at this time. Mike ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 5/9/2011 7:40 AM, MH Michael Hammer (5304) wrote: I'd like to request that we specifically test for consensus on deprecating l= through the usual +1/-1 approach. No miring, just a vote. This isn't my vote, but a comment: Oddly, I'm finding myself coming to believe that its use within a coordinated template for mediators might actually being helpful. This assumes, of course, that the template can be specified and gain consensus, and that signers, verifiers and mediators all are willing to implement it. Hence, this path involves significant effort. One could argue that it's cleaner to drop it now and explore re-introducing it in the effort to develop that template. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Dave CROCKER wrote: On 5/9/2011 7:40 AM, MH Michael Hammer (5304) wrote: I'd like to request that we specifically test for consensus on deprecating l= through the usual +1/-1 approach. No miring, just a vote. This isn't my vote, but a comment: Oddly, I'm finding myself coming to believe that its use within a coordinated template for mediators might actually being helpful. +1 This assumes, of course, that the template can be specified and gain consensus, and that signers, verifiers and mediators all are willing to implement it. Hence, this path involves significant effort. One could argue that it's cleaner to drop it now and explore re-introducing it in the effort to develop that template. It is really just a matter of defining the current practical use cases today, highlighting it is only useful under limited cases. From a functional standpoint, the only purpose for the body length tag (LTAG) is when the author domain has a desire and intent to maintain the author's organization branding presence and originating message integrity by allowing for additional text to be added and with a no failure expectation, targeted at an intermediary with known behaviors. However, the problem with LTAG is that it is not the only consideration and without target information, insufficient to maintain an expectation for survival. In other words, from a technical standpoint, a consideration to use LTAG must come with other DKIM related factors: - Knowing the intermediary behavior, - Knowing what headers to sign, and - Knowing what content to avoid. For example, in our MLM, the default options for creating a new list are: [X] Add Footer [...] [_] Add Header, if defined [...] [X] Use [list-name] tag in Subject: line [X] Add/Alter Reply-To set to list address [X] Keep Original To: [_] Strip Attachments. Exceptions [...] [X] Strip HTML [X] Add List-* Headers [_] Strip Dangerous HTML, Javascript code [_] DKIM Options [_] List only accepted valid DKIM submissions [X] Reject Restrictive ADSP Domains [X] Verify [X] Report [X] Sign [X] Strip original signatures If a member knew how the targeted list distribution behaved, he would then know how to 100% survive sending DKIM signed mail with signing options: - Enable l= tag - Do not sign headers: Subject and Reply-to - Create only text/plain text He can also survive it with DKIM options set: [X] DKIM Options [_] List only accepted valid DKIM submissions [X] Reject Restrictive ADSP Domains [X] Verify [X] Report [X] Sign [_] Strip original signatures thus ending up with two valid signatures. But if he also knew the message signature will be stripped and message resigned and was was ok with the the 3rd party signature, they probably may see they don't need sign mail unless the list required it. So much of this has to do with the business use cases for the author domaina the DKIM signing service providers and how they are able to expose that information in general or contractual. -- Hector Santos, CTO http://www.santronics.com http://santronics.blogspot.com ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Oddly, I'm finding myself coming to believe that its use within a coordinated template for mediators might actually being helpful. This assumes, of course, that the template can be specified and gain consensus, and that signers, verifiers and mediators all are willing to implement it. Hence, this path involves significant effort. One could argue that it's cleaner to drop it now and explore re-introducing it in the effort to develop that template. This makes sense, but I suspect it's better to design it and use a new tag with the semantics you need rather than to try to recycle l= R's, John ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On May 9, 2011, at 7:56 AM, Dave CROCKER wrote: On 5/9/2011 7:40 AM, MH Michael Hammer (5304) wrote: I'd like to request that we specifically test for consensus on deprecating l= through the usual +1/-1 approach. No miring, just a vote. This isn't my vote, but a comment: Oddly, I'm finding myself coming to believe that its use within a coordinated template for mediators might actually being helpful. This assumes, of course, that the template can be specified and gain consensus, and that signers, verifiers and mediators all are willing to implement it. Hence, this path involves significant effort. One could argue that it's cleaner to drop it now and explore re-introducing it in the effort to develop that template. It sounds like what you're suggesting would be quite different from (and more complex than) l=, and would have very different semantics compared with the current numeric definition of l=. Cheers, Steve ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
I'd like to request that we specifically test for consensus on deprecating l= through the usual +1/-1 approach. No miring, just a vote. Semantics first: we don't vote here. OK, that taken care of, it's a fair request, because there's been a lot of discussion about it. We certainly have a good base of support for deprecating l=. So I'll ask it this way, starting a new thread for it: I determine from discussion that there's enough support for deprecating l= to qualify as rough consensus *if* there's not much objection to it. It's the objection we need to gauge. Please post to this thread if you object to deprecating l= in 4871bis. You may say why you object, but please keep it brief, and let's not have a lot of discussion of it here. If there's enough objection to derail deprecation, we will leave it alone. You may also weigh in as objecting if you don't want to delay the document by doing this. I'll let this go until 25 May, or until there's enough objection that we have our answer, whichever comes first. If we decide to deprecate it, I'll ask Murray to make the edits, and then we'll need to have the working group approve the result, so I expect that'll take another two weeks or so -- say, until 11 June. Have at it. Barry, as chair ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
So I'll ask it this way, starting a new thread for it: I determine from discussion that there's enough support for deprecating l= to qualify as rough consensus *if* there's not much objection to it. It's the objection we need to gauge. Please post to this thread if you object to deprecating l= in 4871bis Sorry... I forgot to change the subject line. Please do NOT register your objection on this thread; do it on the Issue: Consider deprecating l= thread. Thanks. Barry, as forgetful chair ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 5/9/2011 1:14 PM, Steve Atkins wrote: On May 9, 2011, at 7:56 AM, Dave CROCKER wrote: Oddly, I'm finding myself coming to believe that its use within a coordinated template for mediators might actually being helpful. This assumes, of course, that the template can be specified and gain consensus, and that signers, verifiers and mediators all are willing to implement it. Hence, this path involves significant effort. One could argue that it's cleaner to drop it now and explore re-introducing it in the effort to develop that template. It sounds like what you're suggesting would be quite different from (and more complex than) l=, and would have very different semantics compared with the current numeric definition of l=. In its entirety, yes. My guess is that there is some benefit in a piece like (or the same as) l=, but the important difference is that it would be fit into an integrated mechanism, rather than just sit there piecemeal. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
-Original Message- From: ietf-dkim-boun...@mipassoc.org [mailto:ietf-dkim-boun...@mipassoc.org] On Behalf Of Barry Leiba Sent: Saturday, May 07, 2011 7:00 AM To: Hector Santos Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output The 6% using l= needlessly is a red flag. Yep. Happily, we (where we, here, mostly means Murray, but some others as well) are collecting stats. I don't know that I would red flag it so fast. There are local policy options that can mitigate the damage, and the stats don't say whether or not those are in effect. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 07/May/11 15:39, Hector Santos wrote: I would like to know why 6% of the mail use [l=] but don't need it. One possible answer is that the signing agents have no clue about that mail's destination. The easiest way to configure DKIM in order to use l= on some messages, is to enable it on _all_ messages. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Murray says... The 6% using l= needlessly is a red flag. Yep. Happily, we (where we, here, mostly means Murray, but some others as well) are collecting stats. I don't know that I would red flag it so fast. There are local policy options that can mitigate the damage, and the stats don't say whether or not those are in effect. I think I would, and here's why: Alessandro says... One possible answer is that the signing agents have no clue about that mail's destination. The easiest way to configure DKIM in order to use l= on some messages, is to enable it on _all_ messages. I think Alessandro is right: there's likely no thought at all being put into the use of l=. Signing software is set up to use it because it will make the chances that the signature verifies slightly greater... with no consideration of the consequences. Signers use it because it's the default in the signing software, or because it's easy to check the box to use it. No one is thinking of why it might not be a good idea. This is all the more reason to deprecate it as soon as is practical (though, again, I repeat: not here and now). Barry, as participant ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 5/6/2011 11:24 AM, Murray S. Kucherawy wrote: I suspect it's use of l= by a signer without regard to whether or not the mail is heading to an MLM. For example, OpenDKIM's antecedent had that as an option; only the evolution to OpenDKIM allowed you to be more specific. You are certain to be correct. In practical terms for the current world, what is the likelihood that a signer has any information about the 'type' of an email address? How can a signer possibly know that an addressee is a mailing list??? d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 5/8/2011 10:40 AM, John R. Levine wrote: In practical terms for the current world, what is the likelihood that a signer has any information about the 'type' of an email address? How can a signer possibly know that an addressee is a mailing list??? Our expert interface designers add yet another knob to the user friendly control panel, of course. http://graphics2.snopes.com/inboxer/graphics/rand.jpg How the heck did they get a photo of my living room??? But seriously... I have to say that way too much of the MLM document strikes me as overcomplicated workarounds to unlikely scenarios, all of which could be dealt with by encouraging lists to sign their outgoing mail. A document like this has a challenge to find the right balance between well-known vs. possible vs. likely issues. The challenge is exacerbated, for scenarios that entail some complexity or, as with MLMs, mediation. I think there are two reasonable ways to find the balance: 1. State principals that are specific to the content of the draft and that give guidance about the scope and boundaries of what should be covered. 2. Make specific suggestions for specific bits of text in the draft. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
1. State principals that are specific to the content of the draft and that give guidance about the scope and boundaries of what should be covered. 2. Make specific suggestions for specific bits of text in the draft. Despite the valiant work that Murray has put into the MLM document, my preference, which I doubt has any hope of gaining consensus, would be to throw it away and replace it by one page that says a) many lists break signatures, which isn't going to stop b) so it would be nice if they signed their mail on the way out. Everything else is either too marginal to be worth worrying about, or not a problem if a list's mail has a credible signature.* Failing that, I don't see small changes making it any better, so just ship it. Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly * - Yes, that includes whether you can believe the From: line. I'm agnostic about whether it's useful to include an A-R header describing the incoming state of the message. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
So if we wish to discourage l= useful, some of these text needs to be reworded, like this one in section 3.5 [...] I don't think the proposed text adds or clarifies anything that isn't already there. The semantics and use of l= are pretty well defined already. I agree. Barry, as participant ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Barry Leiba wrote: So if we wish to discourage l= useful, some of these text needs to be reworded, like this one in section 3.5 [...] I don't think the proposed text adds or clarifies anything that isn't already there. �The semantics and use of l= are pretty well defined already. I agree. Ok, no sweat. (Note, there was more than one proposed text and not the only listed in the last post.) However, for the record, I would like to provide my opinion: We are spending an awful amount of time on this l= issue, whether it should be pulled, keep it and explaining how bad it is and discourage usage. The 6% using l= needlessly is a red flag. I would like to know why 6% of the mail use it but don't need it. That will suggest 6% of the potential DKIM Market are subject to l= exploit or %6 are just not reading it correctly. Should we wait until 10%, 20%, etc? Volume analysis needs to be broken down to domain analysis to see how DKIM is being used. Consider that we don't know the domain break down of the 156524 messages. So of the 156524, there are X unique domains, what percent of those use l= needlessly? Was it higher or lower than 6%? or was it just 1 or 2 domains? -- Hector Santos, CTO http://www.santronics.com http://santronics.blogspot.com ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
We are spending an awful amount of time on this l= issue, whether it should be pulled, keep it and explaining how bad it is and discourage usage. Agreed. I would like to deprecate it. But we don't have consensus for going that far, and I think we're too late in the process to get ourselves mired in that. What we're doing now is just short of deprecating it -- saying that, well, you really shouldn't oughta use it, without being normative. The 6% using l= needlessly is a red flag. Yep. Happily, we (where we, here, mostly means Murray, but some others as well) are collecting stats. It's possible, later, for someone to create an individual submission for DKIM l= Considered Harmful, or some such, and perhaps if/when someone ever moves DKIM to full Standard we can actually deprecate l=. Barry, as participant ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Volume tends to hide many important data points at the domain level. Many times its just 1-2 domains (and their implementation) that are showing how specs are being read/used. My view in reading this complex document, many parts has become sparse with its technical information. For the available tags to consider, most readers will use 3.5 as a quick guide and that is where the first warning should be highlighted. It isn't highlighted unless you do a search for l= to see any other references for it. So if we are not going to deprecate it and talk mostly about discouraging usage, the Quick Guide 3.5 tags sections should be the very first place with this important tidbit. Barry Leiba wrote: We are spending an awful amount of time on this l= issue, whether it should be pulled, keep it and explaining how bad it is and discourage usage. Agreed. I would like to deprecate it. But we don't have consensus for going that far, and I think we're too late in the process to get ourselves mired in that. What we're doing now is just short of deprecating it -- saying that, well, you really shouldn't oughta use it, without being normative. The 6% using l= needlessly is a red flag. Yep. Happily, we (where we, here, mostly means Murray, but some others as well) are collecting stats. It's possible, later, for someone to create an individual submission for DKIM l= Considered Harmful, or some such, and perhaps if/when someone ever moves DKIM to full Standard we can actually deprecate l=. Barry, as participant -- Hector Santos, CTO http://www.santronics.com http://santronics.blogspot.com ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 5/6/11 4:35 PM, John R. Levine wrote: http://www.opendkim.org/stats/report.html#l_tag You can see the count that have l= smaller than the final message size as well as the l=0 ones, and how many of those passed or failed. That's out of 155972 signatures that used l=, and 4.36M total signatures observed, in just over eight months of data. Hmmn. If my arithmetic is right, about 95% of l= signatures didn't cover the whole body, and only a few of those were l=0. Your users must subscribe to different mailing lists than I do. John, the opendkim project gathers information from various sources, they're not necessarily the users of Murray and they're not necessarily subscribed to mailing lists. The statistics also doesn't tell which inbound mail is from legitimate sources and which inbound mail is from spammers/bad guys. Maybe the 95% you mention, are the spammers who try to abuse l= in DKIM... /rolf ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
John R. Levine wrote: http://www.opendkim.org/stats/report.html#l_tag You can see the count that have l= smaller than the final message size as well as the l=0 ones, and how many of those passed or failed. That's out of 155972 signatures that used l=, and 4.36M total signatures observed, in just over eight months of data. Hmmn. If my arithmetic is right, about 95% of l= signatures didn't cover the whole body, and only a few of those were l=0. Your users must subscribe to different mailing lists than I do. of course, we don't all live in the same levine list world. What I found in a quick grep scan of ~7000 list messages: 137 used l= with some value 3 used l=0 from the same source More details: - Sorted down to 37 domains, 36 unknown, 1 known domain, - Except for 1 known, all mail from the 36 was spam, - 20 of them had the same patterns but different domains, and - the 20 used two signatures, sha1 and sha256 The collection were saved prior to verification so I don't know off hand if they failed or the actual body counts. In my network of mail, I will say, l= is used by spammers blasting list mail to their collected emails addresses to spam. -- Hector Santos, CTO http://www.santronics.com http://santronics.blogspot.com ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 05/06/2011 08:12 AM, Rolf E. Sonneveld wrote: John, the opendkim project gathers information from various sources, they're not necessarily the users of Murray and they're not necessarily subscribed to mailing lists. The statistics also doesn't tell which inbound mail is from legitimate sources and which inbound mail is from spammers/bad guys. Maybe the 95% you mention, are the spammers who try to abuse l= in DKIM... Good luck with spammers trying abuse that. A naive filter wouldn't know at all about l= but are well acquainted with spammers inserting good looking text with spammy text. A dkim aware filter would be even less impressed as the unsigned content would stick out like a sore thumb. For all of the hysterical warnings in the spec, it would be nice to hear about even ONE successful abuse. It's been, what, 5 years now? Mike ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Might not be list traffic. But I have data for that too. Count of signatures with l= that did or didn't appear to pass through an MLM: +--+--+ | count(*) | mailing_list | +--+--+ |77246 |0 | |78853 |1 | +--+--+ That's just strange. Most of the l= signatures don't cover the whole body, and half of those didn't go through a mailing list? Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
-Original Message- From: John R. Levine [mailto:jo...@iecc.com] Sent: Friday, May 06, 2011 10:50 AM To: Murray S. Kucherawy Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output Might not be list traffic. But I have data for that too. Count of signatures with l= that did or didn't appear to pass through an MLM: +--+--+ | count(*) | mailing_list | +--+--+ |77246 |0 | |78853 |1 | +--+--+ That's just strange. Most of the l= signatures don't cover the whole body, and half of those didn't go through a mailing list? I suspect it's use of l= by a signer without regard to whether or not the mail is heading to an MLM. For example, OpenDKIM's antecedent had that as an option; only the evolution to OpenDKIM allowed you to be more specific. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
-Original Message- From: John R. Levine [mailto:jo...@iecc.com] Sent: Friday, May 06, 2011 11:43 AM To: Murray S. Kucherawy Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output +--+--+ | count(*) | mailing_list | +--+--+ |77246 |0 | |78853 |1 | +--+--+ That's just strange. Most of the l= signatures don't cover the whole body, and half of those didn't go through a mailing list? I suspect it's use of l= by a signer without regard to whether or not the mail is heading to an MLM. For example, OpenDKIM's antecedent had that as an option; only the evolution to OpenDKIM allowed you to be more specific. Except that doesn't explain why l= doesn't cover the entire body. Signing or verifying bug? Clever spammer replaying signed mail and getting away with it? Forwarders of some sort that add a footer but otherwise don't look like mailing lists? My guess is the third one. The specification for what we decide is a mailing list submission isn't bulletproof, but is listed as: - has a Precedence: list field - has a List-Id: field - has a List-Post: field - has a List-Unsubscribe: field - has a Mailing-List: field If there are other metrics that are easily detected, I could add monitoring for them and then cut off future reports at today's date, or something like that. There's also the whole thing about l= is the post-canonicalization body count, not the actual full message byte count. I'll double-check on that logic. This won't change the l= use information though, just the pass/fail reporting accuracy. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Hi, Murray, On 5/6/11 8:50 PM, Murray S. Kucherawy wrote: -Original Message- From: John R. Levine [mailto:jo...@iecc.com] Sent: Friday, May 06, 2011 11:43 AM To: Murray S. Kucherawy Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output +--+--+ | count(*) | mailing_list | +--+--+ |77246 |0 | |78853 |1 | +--+--+ That's just strange. Most of the l= signatures don't cover the whole body, and half of those didn't go through a mailing list? I suspect it's use of l= by a signer without regard to whether or not the mail is heading to an MLM. For example, OpenDKIM's antecedent had that as an option; only the evolution to OpenDKIM allowed you to be more specific. Except that doesn't explain why l= doesn't cover the entire body. Signing or verifying bug? Clever spammer replaying signed mail and getting away with it? Forwarders of some sort that add a footer but otherwise don't look like mailing lists? My guess is the third one. The specification for what we decide is a mailing list submission isn't bulletproof, but is listed as: - has a Precedence: list field - has a List-Id: field - has a List-Post: field - has a List-Unsubscribe: field - has a Mailing-List: field I assume this is a boolean OR list? /rolf ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
-Original Message- From: Rolf E. Sonneveld [mailto:r.e.sonnev...@sonnection.nl] Sent: Friday, May 06, 2011 1:48 PM To: Murray S. Kucherawy Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output My guess is the third one. The specification for what we decide is a mailing list submission isn't bulletproof, but is listed as: - has a Precedence: list field - has a List-Id: field - has a List-Post: field - has a List-Unsubscribe: field - has a Mailing-List: field I assume this is a boolean OR list? Correct. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
On 5/6/11 8:43 PM, John R. Levine wrote: +--+--+ | count(*) | mailing_list | +--+--+ |77246 |0 | |78853 |1 | +--+--+ That's just strange. Most of the l= signatures don't cover the whole body, and half of those didn't go through a mailing list? I suspect it's use of l= by a signer without regard to whether or not the mail is heading to an MLM. For example, OpenDKIM's antecedent had that as an option; only the evolution to OpenDKIM allowed you to be more specific. Except that doesn't explain why l= doesn't cover the entire body. Signing or verifying bug? Clever spammer replaying signed mail and getting away with it? Forwarders of some sort that add a footer but otherwise don't look like mailing lists? or just authoring MLM's (see par. 4.2 of http://www.ietf.org/id/draft-ietf-dkim-mailinglists-08.txt) which probably use software that incorporates some (minimalistic) type of MLM which adds one of the headers, listed by Murray. I notice I regularly get mail from marketing departments of my car dealer, smart phone provider, from Adobe, etc. that carry a 'List-Unsubscribe' header and some of them carry a Precedence header. I wouldn't be surprised if the percentage of this type of messages is greater than the percentage of the 'classic' re-sending MLM type of messages. Back to the topic of this thread: I don't think we can draw any conclusions from these statistics in relation to the description of l= in rfc4871bis. The current description in rfc4871bis works for me. /rolf ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
Rolf E. Sonneveld wrote: Back to the topic of this thread: I don't think we can draw any conclusions from these statistics in relation to the description of l= in rfc4871bis. The current description in rfc4871bis works for me. I would like to know the percentage of l=xxx where xxx equals actual body count. If its very high, that will tell me there are many references to l= with text that sounds like its expected to be added. I posted this here: http://mipassoc.org/pipermail/ietf-dkim/2011q2/016138.html So if we wish to discourage l= useful, some of these text needs to be reworded, like this one in section 3.5 bh= The hash of the canonicalized body part of the message as limited by the l= tag (base64; REQUIRED). It sounds like its expected and the REQUIRED is too close to l= which can throw someone off. I suggested the change to be: bh= The hash (base64; REQUIRED) of the canonicalized body part of the message. It is limited to the entire body length count or length explicitly set with the optional l= tag count value. If the hash is to represent the entire body with no expectation for additional unhashed text appended to the body, the l= tag SHOULD NOT be used. (See Section 9.1). My post list all the references to l= for reading and setting. -- Hector Santos, CTO http://www.santronics.com http://santronics.blogspot.com ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output
-Original Message- From: ietf-dkim-boun...@mipassoc.org [mailto:ietf-dkim-boun...@mipassoc.org] On Behalf Of Hector Santos Sent: Friday, May 06, 2011 3:33 PM To: Rolf E. Sonneveld Cc: ietf-dkim@mipassoc.org Subject: Re: [ietf-dkim] l= statistics was 23 again (sorry John) was Output Rolf E. Sonneveld wrote: Back to the topic of this thread: I don't think we can draw any conclusions from these statistics in relation to the description of l= in rfc4871bis. The current description in rfc4871bis works for me. I would like to know the percentage of l=xxx where xxx equals actual body count. Assuming you mean percentage of signatures using 'l=' where the signed length and the message length are the same, OpenDKIM's data shows that as 9008 out of 156524, or less than 6%. Absent an error in the data, that suggests to me that when l= is used, it's being used because the mail is following a path through which it is likely to be extended. So if we wish to discourage l= useful, some of these text needs to be reworded, like this one in section 3.5 [...] I don't think the proposed text adds or clarifies anything that isn't already there. The semantics and use of l= are pretty well defined already. ___ NOTE WELL: This list operates according to http://mipassoc.org/dkim/ietf-list-rules.html
