Re: Careful with those spamtools.....
Indeed. These open relay blacklist sites were always a highly questionable source for mail filtering. Quite obviously, open relays have no relationship to spam... Agreed and already in public domain: http://www.imc.org/ube-relay.html A similar criteria conceptually more correlated to spam filtering, would be a blacklist of relays that are dishonest about the previous IP address in the Received header chain. I do not think http://www.rfc-ignorant.org/ currently databases such non-compliance. Then again such a hypothetical database would be mostly useless in implementation, because dishonest proxies come and go faster than we could database them. Could test in real-time, but tests can be lied to. There are (some proprietary) reliable way to detect the dishonest proxies, but I agree with Dean, much better to just detect the spam directly. In terms of detecting spam directly, per message filters which are based solely on content, have such as high false positive cost and are subvertable with content: http://citeseer.nj.nec.com/androutsopoulos00learning.html (See Page 9 of the PDF linked at top) Filters based on bulk correlation (DCC) of content, require whitelist maintenance and are subvertable with content. Filters which required your senders to opt-in are inherently expensive to the email system, as well as generate many false positives, and are subvertable by forged headers (not to mention being patented). A brief taxonomy is here: http://www.imc.org/ube-sol.html Even if these above filter types haven't been subverted in high rates yet, they can be: http://www1.ietf.org/mail-archive/ietf/Current/msg22190.html We are working on a filtering mechanism which does not suffer from these sorts of issues, because it actually looks at what it unique about spam, not just some sometimes correlated side effects as other filters above do. I agree with Dean and I think conceptually that ALL existing anti-spam (that is currently in public domain that I am aware of) is useless and even harmful as Dean points out (in long run) because they filter things which are not spam, just sometimes (even if most of time so far) correlated to spam. I've been making points like this for a long time: http://ixazon.dynip.com/pipermail/nilsimsa/2002-December/41.html (my warnings on dangers of Bayesian anti-spam filtering, which imo caused Paul Graham to eventually add a disclaimer to his web page) Shelby Moore http://AntiViotic.com
Re: Careful with those spamtools.....
Valdis wrote: There's an even bigger problem - you have to make the difficult choice between: 1) Flag the DMZ mail server of every site that uses RFC1918 space, since the previous hop is in their 1918 space. This won't win you friends No need to flag it as being dishonest when it is not dishonest. Valdis wrote: 2) Allow a pass for 1918 space, and just accept that spammers will use a dummy RFC1918 network (of possibly 1 node looped back to itself) to look like (1). But as I said in previous post I know of at least one proprietary way to detect this. Dean wrote: I do not think that _all_ anti-spam is useless. I asserted here _how_ all existing (that are in public domain and I am aware of) can be rendered useless (eventually): http://www1.ietf.org/mail-archive/ietf/Current/msg22190.html Dean wrote: I think that content analysis holds much promise. Agreed but not by ... [phrase withheld due to IP charter of this list] ... due to information theory. I can not say more until the patent is pending. Only a few years ago, we thought that speaker-independent voice recognition was science fiction. I personally did not think so since I got into studying filtering theory in 1986. Much of my early work was due to an interest I had in robotics and A.I.. Along similar lines, I think it will be possible for intelligent agents to decide whether email is likely to be interesting to us. Yes it already exists and will hopefully be demonstrated publicly soon. I guess (according to Vernon) that means I am a Kook until demonstrated :) Shelby Moore http://AntiViotic.com
Re: Careful with those spamtools.....
Shelby wrote: But as I said in previous post I know of at least one proprietary way to detect this. Valdis wrote: Do tell. Apply to this message. I will alert you when the patent is applied for and has a link online at www.uspto.gov. I hope that will be within a few months, but the problem I have right now is getting enough non-spam data to test the algorithm outside of the theoretical. SpamArchive.org gives me spam data. But I need non-spam data also to form a non-theoretical corpus test. Can any one help? Dean wrote: No, it isn't. And it is an illegal method, because you (if you are an ISP), probably don't have permission to block non-spam mail. Conceptually agree. Concept of attempting to kill spam by stopping the relaying of all unauthenticated email when email protocols and installed base are not designed that way is like burning a forest because you want to lower the mosquito population. You kill a lot of trees and other things that the forest was designed for, and you may or may not kill a lot of mosquitos. Worse, the mosquitos can reproduce faster than the trees and other things destroyed. If you change the protocol and understand the tradeoffs of the change, then that could be okay: http://www1.ietf.org/mail-archive/ietf/Current/msg22215.html That is why I proposed a way to move non-spam bulk email (by new definition) out of email. Obviously that was not popular here at IETF and for reasons frankly that I can understand and agree with to a certain extent (although I think the alternative is worse as pointed out some where burried in that thread). That was an attempt to see if it is possible to make services such as DCC legal, because otherwise they can be misused to block non-spam bulk email. Note afaik DCC is most often implemented at ISP MTA level (per Dean's point about US federal communications law). (Note I did not say that DCC is not useful, if not misused and understand it can be subverted in future). And to legalize the blocking of bulk email at other nodes in the channel. Now I can __gleefully__ see it isn't likely to happen (gleeful that my proposal failed). Which is fabulous for the value of the anti-spam algorithm I invented recently. My point is (see link in previous posts) that all existing anti-spam in public domain that I am aware of is of dubious benefit, legality, and most have the potential (eventually if the mega spam virus occurs) to be as harmful as they can benefit. People are not kooks for working to improve a state-of-the-art which is currently all dubious hueristics. There may or may not be kooky people working on anti-spam (I don't care), but their methods can't be any more kooky than the current *published* state-of-the-art: http://www1.ietf.org/mail-archive/ietf/Current/msg22184.html I just hope that the world of people using anti-spam doesn't have to suffer a horrific experience or experiences (similar to Harald's discovery which lead to this thread, but on larger scale) to become aware of the dangers of existing tools. Thanks, Shelby Moore http://AntiViotic.com
Shannon Entropy, channel protocols, and propensity for noise
On a conceptual level, maximizing opportunity (uncertainty) to *scale* is the same as maximizing the Shannon entropy of the protocol in terms of the characteristic we want to *scale*. The following thought can also be generalized to any protocol or aspect of the protocol, not just email or scale. Some private discussions stimulated me to have a generalized thought which I personally felt is relevant (also profound, yet obvious) and important enough to share with the people who supposedly design some of the internet engineering in the STDs track. Regarding the concept of a protocol's ability to scale, as one of Dean's private criticisms of Iljitsch's request to revisit criticisms of the (probably very old) idea of forcing SMTP to accept email only from a SMTP server of DNS record for the domain in From header (I may have misstated the idea, but my thought herein still survives). My thought is as result of any design to maximize scale entropy, that we expect (as predicted by Shannon's theories) that the channel has the propensity to generate a maximized amount of noise. I think most of people would agree that spam is noise (that unsolicited is essentially noise). My thought is on a conceptual level what my previous anti-spam proposal and what Iljitsch's idea (and probably all other protocol level anti-spam proposals) are attempting to do is to reduce the entropy (opportunities, i.e. uncertainties) of the email channel. The benefits and costs (tradeoff analysis) within such a conceptual paradigm is what I leave here for thought stimulation. I am not here to argue any point, merely to postulate the conceptual relationship. Good luck. I am resigning *again* from the list, so please do not send any response directly to me. I can always pull the archives if I have desire/time to read followup posts in this thread. Shelby Moore http://AntiViotic.com Dislaimer: I am posting this only in terms of sharing a thought (probably not even a new thought), just to stimulate thought (and perhaps some discussion). So please do not feel obligated to add noise to this thread (and thus list), if your only purpose is to prove this was never thought of before. If this is not interesting nor important topic to you, then please ignore it. I have no interest in defending or nurturing this thought. It is simply stated to stimulate (or not) depending on each individual reader.
Re: Exposing the security holes in all existing anti-spam techniques (was Re: You Might...)
Sending this again, since Harald claims here: http://www1.ietf.org/mail-archive/ietf/Current/msg22200.html That he is not censoring me, yet my 3rd post on this thread: http://www1.ietf.org/mail-archive/ietf/Current/msg22198.html Has already appeared (thank you Harald) while after some hours, the 2nd post I made (this email) has not. I will also be emailing Harald privately to find out why my repeated attempts in last few hours to unsubscribe (so I would not be provoked by new posts) are not being recognized by Majordomo (still receiving email from list). Already been done, and better - Consider a virus that installs an open proxy for spammers to use. Do the lit review yourself if you can't name which one(s) did this (yes, more than one has).l Valdis what you describe is not the same as what I described. The virus must run autonomously in order to have the effect at the scale I described. Such a virus would want to be on the order of 1 million computers sending 10,000 random spams per day (or any combinations of the same product, e.g. 100,000 sending 100,000). If you assume that 10 billion spams are sent per day now, and that DCC, Bayesian, etc catch 90% of them, then 10 billion undetected spams would give inversion of performance (from 90% detected to 10% detected). Do the lit review for which famous viruses created havoc by sending around other attachment at random off a person's disk. Viral attachments are easily to block, so you would not want an attachment in the outgoing spam. Reread what I wrote last post about the potential virus could spread itself orthogonally to it's spam function. However, keep in mind that the spam can't be TOO randomized and still convey a message Conveying a message wasn't what I suggested a the virus could do. I proposed it would simply disrupt antispam systems and wreck havoc on the email system. It is a macho thing, such as the ILOVEU virus from the Philippines. I should disclose that I am currently visiting the Philippines for a conference (field research) on this (check my IP address). With a truely random content (except normal words and normal word distribution, normal in uniform statistical distribution sense), and with a huge volume, you need not care if any one reads it. The only point would be to get past the antispam systems and users who were formerly getting 90% antispam would be seeing more like 10% (missing a zero in my previous post) antispam and 90% spam. I wonder how actual field research other experts actually do on virus/spam havens. Already being done: Consider the following obfuscations seen in today's spam No I meant truely random ordering of *normal* words. I usually mean what I write. The *normal* words are needed to avoid Bayesian filters. We're quite aware of the architectural problems. We're also aware of exactly what it would take to deploy a solution Nice boast but imo you have proven otherwise in the way you handled my posts, which is going to be quite clear to independent (outside) observers, when/if the mega spam virus I described herein hits the world. Lastly I have done the full background search at ASRG (IRTF), and I did not find prior art for either the proposal I made to legitimize bulk email by moving it to pull, nor the prior art for our soon to be patent-pending anti-spam algorithm. Your search was incomplete, and here's some prior art. The one you quoted is referring to RSS which is not entirely correlated with what I proposed. I proposed using POP (or what ever the receiptient prefers) which does not require a complete overhaul of email clients as quoted as one of the complaints in thread you mention. I have long ago in the this list readily admitted that message pull has existed for a long time, such as our past discussion of usenet. Also the one you quoted does not discuss the benefits I proposed, such as the ability to define spam at other nodes in the channel than the pyschology of unsolicited and the benefits that follow such a logic. BTW, I noticed there were no reasonable objections in the thread you quoted regarding overall concept of email pull. Make sure that the claims on your patent don't cover anything in this message, as that would of course be a big no-no. You are confusing 2 different things. Please read my posts more carefully. The proposal I made here has nothing to do with the antispam algorithm we developed. I stated time-domain analysis (idea only, no details found) as the closest thing found at ASRG, but not substantial or correlated (just an idea, no details) enough to be prior art, as Vernon and you claim. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
Iljitsch van Beijnum wrote: declaring the spam problem unsolvable. I don't think it's a good idea to lend credibility to this sentiment by publishing it as an RFC. How hard is it to agree that: a) there will always be (some) spam b) there is no need for it to be 50% of all mail Vernon Schryver responded: That last sentiment is on my list. There are several currently available, independent sets of mechanisms that will keep more than 90% of all spam out of your mailbox with fewer than 0.1% false positives. If your mailbox receives on average more than 1 spam/day and you care, then fire your current ISP and hire one that offers reasonable spam defenses. If you care to invest your own time and effort maintaining filters or if you can tolerate more than 0.1% false positives, your mailbox can be practically spam free. Consider my response a new thread, Why all existing anti-spam will fail miserably or are otherwise indequate. Or consider this response, Exposing the security holes in all existing anti-spam techniques, similar to the benefits of exposing the security holes in operating systems before they are exploited. There is no sense in relying on something and making an ever increasing investment in that thing, if it is going to fail miserably at some point and force you to start over. Vernon, misses some *very* important details in his *simplistic* analysis above, which I am confident after all of you read the following, you will agree could not be left as an unchallenged statement *pretending* to be factual. 1. The DCC (Vernon's business) and all current practical anti-spam which can generate the 90% + 0.1% that Vernon claims, rely heavily on whitelisting, which is both inherently subvertable and more importantly which has a great cost to (usually not transportable) investment in maintenance, which may in many cases outweigh the *current* cost of spam: Shttp://www.ietf.org/proceedings/03mar/slides/asrg-1/sld12.htm 2. Not all people can use those existing anti-spam tools. For example, I am capable of using BrightMail on my Earthlink account but not on my hosted accounts. In order words, those existing tools don't scale every where. 3. And here is the kicker. ALL existing anti-spam methods, can be (and thus will eventually be) easily subverted. This is already in public domain else where. All someone need do is create a virus which both spreads sometimes via email and the rest of time sends large quantities of highly randomized spam. The seed would need to be truely random (e.g. cpu clock modulo milliseconds) and randomize all headers (To, From, Subject, etc) and content, using lookup tables of common domains and words. Vernon's DCC, Paul Graham's Bayesian filters, reply opt-in whitelisting, etc.. would all fail miserably. Additionally imagine all the bounced traffic (from randomized address) and especially the case where two reply opt-in whitelisting entities get caught in infinite loop (randomized From/ Reply-To addresses). Also this would probably overload the DCC servers with too many unique flooded checksums. Some script kiddie could become famous by turning all anti-spam from 90% in 1% effectiveness in days, not to mention probably overloading internet email to the point where no one could find their legitimate email. If #3 happens, those of you here at the IETF who attempted to ridule me (unsuccessfully obviously), will be realizing that my warnings of dire architectual problem are real. Lastly I have done the full background search at ASRG (IRTF), and I did not find prior art for either the proposal I made to legitimize bulk email by moving it to pull, nor the prior art for our soon to be patent-pending anti-spam algorithm. The closest prior art I found was spam is any bulk email from someone you don't know essay, and time-domain analysis idea (with no details). I am indeed working on novel anti-spam, and I do not appreciate the unprofessional suggestion (borderline libel) otherwise. If any one would like the full set of links to my research (literature review) at the ASRG, email me and I will send them to you privately. This is the last I have to say on this matter in public. I am extremely confident in the expertise of my assertions. The rest will be said with my actions and other naturally occurring events. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
Iljitsch van Beijnum wrote: declaring the spam problem unsolvable. I don't think it's a good idea to lend credibility to this sentiment by publishing it as an RFC. How hard is it to agree that: a) there will always be (some) spam b) there is no need for it to be 50% of all mail Vernon Schryver responded: That last sentiment is on my list. There are several currently available, independent sets of mechanisms that will keep more than 90% of all spam out of your mailbox with fewer than 0.1% false positives. If your mailbox receives on average more than 1 spam/day and you care, then fire your current ISP and hire one that offers reasonable spam defenses. If you care to invest your own time and effort maintaining filters or if you can tolerate more than 0.1% false positives, your mailbox can be practically spam free. Consider my response a new thread, Why all existing anti-spam will fail miserably or are otherwise indequate. Or consider this response, Exposing the security holes in all existing anti-spam techniques, similar to the benefits of exposing the security holes in operating systems before they are exploited. There is no sense in relying on something and making an ever increasing investment in that thing, if it is going to fail miserably at some point and force you to start over. Vernon, misses some *very* important details in his *simplistic* analysis above, which I am confident after all of you read the following, you will agree could not be left as an unchallenged statement *pretending* to be factual. 1. The DCC (Vernon's business) and all current practical anti-spam which can generate the 90% + 0.1% that Vernon claims, rely heavily on whitelisting, which is both inherently subvertable and more importantly which has a great cost to (usually not transportable) investment in maintenance, which may in many cases outweigh the *current* cost of spam: http://www.ietf.org/proceedings/03mar/slides/asrg-1/sld12.htm 2. Not all people can use those existing anti-spam tools. For example, I am capable of using BrightMail on my Earthlink account but not on my hosted accounts. In order words, those existing tools don't scale every where. 3. And here is the kicker. ALL existing anti-spam methods, can be (and thus will eventually be) easily subverted. This is already in public domain else where. All someone need do is create a virus which both spreads sometimes via email and the rest of time sends large quantities of highly randomized spam. The seed would need to be truely random (e.g. cpu clock modulo milliseconds) and randomize all headers (To, From, Subject, etc) and content, using lookup tables of common domains, and normal words people use in email. Vernon's DCC, Paul Graham's Bayesian filters, reply opt-in whitelisting, etc.. would all fail miserably. Additionally imagine all the bounced traffic (from randomized address) and especially the case where two reply opt-in whitelisting entities get caught in infinite loop (randomized From/ Reply-To addresses). Also this would probably overload the DCC servers with too many unique flooded checksums. Some script kiddie could become famous by turning all anti-spam from 90% in 1% effectiveness in days, not to mention probably overloading internet email to the point where no one could find their legitimate email. If #3 happens, those of you here at the IETF who attempted to ridule me (unsuccessfully obviously), will be realizing that my warnings of dire architectual problem are real. Lastly I have done the full background search at ASRG (IRTF), and I did not find prior art for either the proposal I made to legitimize bulk email by moving it to pull, nor the prior art for our soon to be patent-pending anti-spam algorithm. The closest prior art I found was spam is any bulk email from someone you don't know essay, and time-domain analysis idea (with no details). I am indeed working on novel anti-spam, and I do not appreciate the unprofessional suggestion (borderline libel) otherwise. If any one would like the full set of links to my research (literature review) at the ASRG, email me and I will send them to you privately. This is the last I have to say on this matter in public. I am extremely confident in the expertise of my assertions. The rest will be said with my actions and other naturally occurring events. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
Iljitsch van Beijnum wrote: declaring the spam problem unsolvable. I don't think it's a good idea to lend credibility to this sentiment by publishing it as an RFC. How hard is it to agree that: a) there will always be (some) spam b) there is no need for it to be 50% of all mail Vernon Schryver responded: That last sentiment is on my list. There are several currently available, independent sets of mechanisms that will keep more than 90% of all spam out of your mailbox with fewer than 0.1% false positives. If your mailbox receives on average more than 1 spam/day and you care, then fire your current ISP and hire one that offers reasonable spam defenses. If you care to invest your own time and effort maintaining filters or if you can tolerate more than 0.1% false positives, your mailbox can be practically spam free. Consider my response a new thread, Why all existing anti-spam will fail miserably or are otherwise indequate. Or consider this response, Exposing the security holes in all existing anti-spam techniques, similar to the benefits of exposing the security holes in operating systems before they are exploited. There is no sense in relying on something and making an ever increasing investment in that thing, if it is going to fail miserably at some point and force you to start over. Vernon, misses some *very* important details in his *simplistic* analysis above, which I am confident after all of you read the following, you will agree could not be left as an unchallenged statement *pretending* to be factual. 1. The DCC (Vernon's business) and all current practical anti-spam which can generate the 90% + 0.1% that Vernon claims, rely heavily on whitelisting, which is both inherently subvertable and more importantly which has a great cost to (usually not transportable) investment in maintenance, which may in many cases outweigh the *current* cost of spam: http://www.ietf.org/proceedings/03mar/slides/asrg-1/sld12.htm 2. Not all people can use those existing anti-spam tools. For example, I am capable of using BrightMail on my Earthlink account but not on my hosted accounts. In order words, those existing tools don't scale every where. 3. And here is the kicker. ALL existing anti-spam methods, can be (and thus will eventually be) easily subverted. This is already in public domain else where. All someone need do is create a virus which both spreads sometimes via email and the rest of time sends large quantities of highly randomized spam. The seed would need to be truely random (e.g. cpu clock modulo milliseconds) and randomize all headers (To, From, Subject, etc) and content, using lookup tables of common domains, and normal words people use in email. Vernon's DCC, Paul Graham's Bayesian filters, reply opt-in whitelisting, etc.. would all fail miserably. Additionally imagine all the bounced traffic (from randomized address) and especially the case where two reply opt-in whitelisting entities get caught in infinite loop (randomized From/ Reply-To addresses). Also this would probably overload the DCC servers with too many unique flooded checksums. Some script kiddie could become famous by turning all anti-spam from 90% in 1% effectiveness in days, not to mention probably overloading internet email to the point where no one could find their legitimate email. If #3 happens, those of you here at the IETF who attempted to ridule me (unsuccessfully obviously), will be realizing that my warnings of dire architectual problem are real. Lastly I have done the full background search at ASRG (IRTF), and I did not find prior art for either the proposal I made to legitimize bulk email by moving it to pull, nor the prior art for our soon to be patent-pending anti-spam algorithm. The closest prior art I found was spam is any bulk email from someone you don't know essay, and time-domain analysis idea (with no details). I am indeed working on novel anti-spam, and I do not appreciate the unprofessional suggestion (borderline libel) otherwise. If any one would like the full set of links to my research (literature review) at the ASRG, email me and I will send them to you privately. This is the last I have to say on this matter in public. I am extremely confident in the expertise of my assertions. The rest will be said with my actions and other naturally occurring events. Shelby Moore http://AntiViotic.com
Re: Exposing the security holes in all existing anti-spam techniques (was Re: You Might...)
Do you have any idea how to unsubscribe from this list? I have tried numerous times but can't get Majordomo to do it. I've tried sending unsubscribe ietf in subject and body to [EMAIL PROTECTED], [EMAIL PROTECTED] Since the list won't stop sending me email, I guess I will respond one more time... (was going to send privately as I have been with Dean debate... but what the heck!) Already been done, and better - Consider a virus that installs an open proxy for spammers to use. Do the lit review yourself if you can't name which one(s) did this (yes, more than one has).l No this is not the same as what I described. The virus must run autonomously in order to have the effect at the scale I described. Do the lit review for which famous viruses created havoc by sending around other attachment at random off a person's disk. Viral attachments are easily to block, so you would not want an attachment in the outgoing spam. Reread what I wrote last post. However, keep in mind that the spam can't be TOO randomized and still convey a message Conveying a message wasn't what I suggested a the virus could do. I proposed it would simply disrupt antispam systems and wreck havoc on the email system. It is a macho thing, such as the ILOVEU virus from the Philippines. I should disclose that I am currently visiting the Philippines for a conference on this (check my IP address). With a truely random content (except normal words and word distribution), and with a huge volume, you need not care if any one reads it. The only point would be to get past the antispam systems and users who were formerly getting 90% antispam would be seeing more like 10% (missing a zero in my previous post) antispam and 90% spam. Already being done: Consider the following obfuscations seen in today's spam No I meant truely random order of *normal* words. I usually mean what I write. The *normal* words are needed to avoid Bayesian. We're quite aware of the architectural problems. We're also aware of exactly what it would take to deploy a solution Nice boast but imo you have proven otherwise in the way you handled my posts, which is going to be quite clear to independent observers, when the virus I mentioned hits the world. Lastly I have done the full background search at ASRG (IRTF), and I did not find prior art for either the proposal I made to legitimize bulk email by moving it to pull, nor the prior art for our soon to be patent-pending anti-spam algorithm. Your search was incomplete, and here's some prior art. The one you quoted is referring to RSS which is not what I proposed. I proposed using POP (or what ever the receiptient prefers) which does not require a complete overhaul of email clients. I have long ago in the this list readily admitted that message pull has existed for a long time, such as our past discussion of usenet. Also the one you quoted does not discuss the benefits I proposed, such as the ability to define spam at other nodes in the channel than the pyschology of unsolicited and the benefits that follow such a logic. BTW, I noticed there were no reasonable objections in the thread you quoted regarding overall concept of email pull. Make sure that the claims on your patent don't cover anything in this message, as that would of course be a big no-no. You are confusing 2 different things. Please read my posts more carefully. The proposal I made here has nothing to do with the antispam algorithm we developed. I stated time-domain analysis (idea only, no details found) as the closest thing but not quite prior art at ASRG. Shelby Moore http://AntiViotic.com
Exposing the security holes in all existing anti-spam techniques (was Re: You Might...)
-domain analysis idea (with no details). I am indeed working on novel anti-spam, and I do not appreciate the unprofessional suggestion (borderline libel) otherwise. If any one would like the full set of links to my research (literature review) at the ASRG, email me and I will send them to you privately. This is the last I have to say on this matter in public. I am extremely confident in the expertise of my assertions. The rest will be said with my actions and other naturally occurring events. Shelby Moore http://AntiViotic.com
Re: Exposing the security holes in all existing anti-spam techniques (was Re: You Might...)
However, keep in mind that the spam can't be TOO randomized and still convey a message BTW, in addition to my previous rebuttal of Valdis (which seems to be blocked from the list by Harald again), I keep forgetting to mention that terrorists might be the likely source of a future virus which effectively shuts down the email system permanently by evading all antispam and overwhelming legit email. Their economic incentive would be to disrupt the economies in the West where there is heavy reliance on the internet email for business. Given that the Taliban is against all forms of modern technology in daily life (yet are astute at using it to fight), I think this is a very real possibility. Shelby Moore http://AntiViotic.com
Re: POP3 delivers, not deletes
[not private reply...in interests of sharing info] Thanks for reply. At 10:08 PM 9/8/2003 -0700, you wrote: [sent privately, since I just advised you to move discussion from the IETF list] I thought you meant regarding the anti-spam stuff, which I agreed with. Why should discussions of POP engineering be moved to IRTF??? I thought IETF stands for internet engineering task force. --On 9. september 2003 11:14 +0800 Shelby Moore [EMAIL PROTECTED] wrote: A *lot* of POP-using programs have the Leave Mail On Server option. And a lot of people have used Leave Mail On Server as a poor man's 1-folder IMAP, leading POP providers to implement mail retaining policies of the RETR it once and it's gone, whether you DELEted it or not. Do you think they also apply this policy to TOP n? Do you have any idea how prevelant this rather extreme policy might be??? I don't know much about the commercial POP sellers' current behaviour... disk has become cheaper, but mail has become fatter over the years since 1996. Well at least it is easy enough to test for. Send test message, POP, etc... Quoting more from RFC 1939 (a document I *very* much encourage reading in *detail* before basing business decisions on details of POP behaviour): I have read it several times, not taking the delete after RETR seriously because...and if you read the section you quote more carefully you see... One special case of a site policy is that messages may only be downloaded once from the server, and are deleted after this has been accomplished. This could be implemented in POP3 server software by the following mechanism: following a POP3 login by a client which was ended by a QUIT, delete all messages downloaded during the session with the RETR command. It is important not to delete messages in the event of abnormal connection termination (ie, if no QUIT was received from the client) because the client may not have successfully received or stored the messages. One merely needs to disconnect without using QUIT, if the POP server is RFC 1939 compliant. So this seems to make the whole suggestion implausible, since so easily to divert it. So the issue has been considered, it seems. Yes but that is not what I asked you. I asked you how prevelant you thought the practice is. Both listed authors are still active in the IETF, I think. I guess that means POP engineering threads belong in the IETF mailing lists, which is why I am not making a private reply. Contacting them privately for advice might prove illuminating. Why privately? Why not share the information? I did not start this thread. Thanks for sharing. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
email) instead of *BE (all bulk email). Again read the linked posts above more carefully. With a different model of spam, we aren't stopping abuse, we are merely increasing detection by having a better model of the signal. Dean Anderson wrote: Detecting abuse is quite different from making a protocol that can't be abused. This thread is not proposing that. See above. Iljitsch van Beijnum wrote: If you can detect abuse on input rather than on output, Correct that is the point of improving the model of the spam signal, so we can do things at earlier points in the channel, input to mailing lists, input to dialup accounts, ISPs, Hosts, etc. Right now, ISPs and Hosts can do nothing because they can not say that all bulk email is spam, therefor they can not be proactive in real-time. That is just one example of many benefits to improving the model the way I have proposed. Iljitsch van Beijnum wrote: detecting abuse is exactly what enables you to make an abuse-free protocol. No we can not get a 100% abuse free protocol. Information theory tells you that is impossible and I agree with Dean on that. But we can get a better model which helps us detect more abuse. Spammers can _always_ do whatever regular users can do. Mostly yes. And that is why improving the model of the spam signal is the only real way we are going to get better at detection. Actually your theories are making my proposal stronger vs all the other ways of detecting spam. My point for the last 2 years has been that any model of spam which looks at content is going to fail over the long-term because content is what legitimate mail does also. So you have to look at modeling what is actually different about the spam signal. I will make a very profound AXIOM based on information theory: AXIOM 1: The only way to reliably detect spam over the long-term is by modeling that which is unique to spam signal and not shared with legit email signal. And that is the bulkness of it and/or the low response (or read) rate. Now I wrote mostly yes above because once you move legitimate bulk email to a pull channel, then as Iljitsch van Beijnum originally pointed out, you can authenticate spammers to subscribed lists differently than legit users of that list. In email channel the signatures of spam are bulk and low response rate. In a subscribed pull channel, the spammer's signature is again what?? Think about that deeply before you try to rebutt it. (Hint: no response rate and the fact that messages which are responded to can be automatically deleted from the pull queue before many users might pull them...and other possible algorithms) Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
I started this thread. Let's please close this thread here on IETF and move it some where more appropriate. I do not know exactly where to move it to (which list I will choose exactly because I want to research more to try to avoid running into Vernon, Valdis, Spencer, and others again which will be difficult since they seem to be every where). What I may do is write the LD and make a working group some where and then apply to such a list and ask only those are really interested to follow. Sometime in the future, I will make one more post on this list to say where all my replies will be going. So if there are any more posts, I will be collecting them and making replies on them to another list in future. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
You'd do well to observe and learn. What specifically did you learn from Vernon in this thread?
Re: You Might Be An Anti-Spam Kook If ...
Vernon Schryver wrote: I've been compiling a list in the style of Jeff Foxworthy. You Might Be An Anti-Spam Kook If Please publish this as an RFC. A collection of unworkable approaches to the spam problem (anti-spam anti-patterns) is useful knowledge that should be preserved and promulgated to reduce the Anti-Spam Kook problem. On the other hand there seems to be considerable interest into declaring the spam problem unsolvable. I don't think it's a good idea to lend credibility to this sentiment by publishing it as an RFC. How hard is it to agree that: a) there will always be (some) spam b) there is no need for it to be 50% of all mail I agree. On the other hand there seems to be considerable interest into declaring the spam problem unsolvable. Amazing that Vernon says working on spam is Kooky, yet that is exactly what he does (DCC). Vernon has pretended he is an expert with this thread, with an obvious timing of condescending the serious anti-spam thread I started: http://www1.ietf.org/mail-archive/ietf/Current/msg22035.html Some disinteresting things Vernon emailed directly to me in past, which imho demonstrate he is not an expert and/or he qualifies for his own Kook test: 1. Vernon apparently got offended because I pointed out that he didn't realize that MD5 checksum on IPv4 was easily breakable via dictionary attack or that his use of it went his often public stated condescending policy of do not implement half-solutions. 2. Vernon wrote, ...it is impossible, because no two people (or at least organizations) think the same streams of bulk mail are solicited and unsolicited.. I can easily find 2 streams that 2 people/orgs can agree are solicited and unsolicited. If you give me 2 streams and 2 people/org at random, then I can not guarantee every time (but eventually and never impossible), but he said no which is always false and certainly never impossible. That just shows you the kind of errors he makes with his logic. 3. Vernon wrote, In the last couple of months, more than one person has invented a super duper wonderful perfect foolproof mechanism of using IP addresses collected by the DCC.. Forgive me, but I doubt your scheme differs significantly from the others.. Isn't the desire to discover part of what makes a scientist an expert? When we stop discovering and learning, then we decay. 4. A fundamental problem is that spam is unsolicited bulk mail, and not IP addresses...no mater how highly coorelated with spam.. The first part is true, the second part is not because we don't live in 3 dimensions. I will not tell you why beyond that, but I know why. 5. The DCC is doing quite well, thank you very much, against the current plague, but the main event is to come.. Wonder what he means by main event is to come? 6. consider me a hopeless, useless, paranoid skeptic with an insurmountable not-invented-here syndrome, and stop sending me mail Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
But, it's hard to have sympathy for your whining That is interesting interpretation of my winning a debate so completely that no one has been able to poke one logic hole in the serious anti- spam thread I started: http://www1.ietf.org/mail-archive/ietf/Current/msg22035.html Instead I think you must be referring to the person who started this silly thread. when you are told by _several_different_people_ that this idea has flaws. Which have all been resolutely refuted and defeated ad nauseum by me. I am usually just a lurker here, but this thread doesn't look like it is going anywhere useful, Makes you kind of wonder why Vernon posted a silly, condescending thread doesn't it. If you can't beat me with logical debate, then resort to other tactics... and it has lost it's entertainment value. That is an interesting interpretation of condescending. In fact, while your posts can't really be considered spam, And yours, and Vernons and everybody else who got suckered in by the person who started this condescending thread which has no engineering value. Shelby Moore http://AntiViotic.com
Re: POP3 delivers, not deletes
At 09:18 AM 9/8/2003 -0400, you wrote: Although it is theoretically possible, using POP (rather than IMAP) to leave the mail on the server until you pull it again with POP, many servers appear to clear out the mail after POPing it. The Client program users LIST and RETR to get the messages than DELE to remove them. The server/cloud program(s) don't do that. If you/anyone want to leave mesage a\on a POP3 MTA, selecting a client program (or writing one) to not DELEte them at your discression is potenetially useful. regs, Dan I am already doing it here: http://MailThentic.com/test.php See the Test Only (don't delete) flag at bottom. Shelby Moore http://AntiViotic.com (aka http://MailThentic.com)
Re: You Might Be An Anti-Spam Kook If ...
Ah. The ever popular IETFers-are-jealous-because-I'm-sexy argument. No, the argyment to stay focused on the serious engineering debate, such as in link below. when you are told by _several_different_people_ that this idea has flaws. Which have all been resolutely refuted and defeated ad nauseum by me. And to make that clear, I have written a summary for you: http://www1.ietf.org/mail-archive/ietf/Current/msg22112.html Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
Imagine if we engineered all things in real life by first expending effort publishing all the ways not to engineer them. Umm.. We don't. I believe that when I see more posts in real engineering threads than I see posts in this thread. On Mon, 08 Sep 2003 19:25:22 +0800, Shelby Moore said: Amazing that Vernon says working on spam is Kooky, yet that is exactly what he does (DCC). No.. Yes. For one thing Vernon doesn't understand the concept of dynamic set theory and entropy as it applies to mapping IPs to spam. That statement he made convinced me neither he or you have fully explored all the possibilities. Neither of you still haven't sent any links to prior art to prove your assertion that I am rehashing. You are all hot air and no facts. What he *said* was totally disregarding the things others have already tried is kooky. The problem is he thinks he knows what other people are proposing but he doesn't understand it well enough to know it is different that what has been tried already. And I doubt either of you have any clue about the mathematical ways it is different. So before you write a master set of rules, maybe you should listen and learn a little more. Which line item from his note is the DCC in violation of? Not DCC, but him as creator of DCC. The very hippocritical fact that he publishes such a document that is in violation of the core philosophy of the document, which is in essense don't think you know everything, unless you do, and you probably don't. Especially the last one he added, which he got from one Keith Moore emailed about me: you're certain none of the preceding apply to you or you see nothing wrong in some of the He is either certain or is saying he is a Kook. He fell right into his own trap. 2. Vernon wrote, ...it is impossible, because no two people (or at least organizations) think the same streams of bulk mail are solicited and unsolicited.. I can easily find 2 streams that 2 people/orgs can agree are solicited and unsolicited. If you give me 2 streams and 2 people/org at random, then I can not guarantee every time (but eventually and never Right. You can't guarantee *every time* for 2 people at random. That's his point exactly. He did not write every time. He wrote no. no means none or empty set. Learn about set theory. Also you missed the deeper point, but that is to be expected... Any scheme that requires any 2 random people to come up with the same answer on this judgment call *will* break with either a false positive or false negative for each object in the stream there's a disagreement on. Again you obviously don't have the mathematical background or don't know how to apply it. I am sure that many people said that nuclear bomb could not be created, that the earth was flat, etc... 4. A fundamental problem is that spam is unsolicited bulk mail, and not IP addresses...no mater how highly coorelated with spam.. The first part is true, the second part is not because we don't live in 3 dimensions. I will not tell you why beyond that, but I know why. Actually, 3 dimensions has absolutely nothing to do with it. You are way off. You did not even realize I was talking about the lack of time dimension, so you very likely still have no clue what I really mean. And the fact that you think it does indicates that you really don't understand the difference there. I doubt you understand how entropy applies to spam. And I doubt any of your prior art has ever looked it from that perspective. The real problem is that there exists a non-empty subset of the IP address space for which the predicate '((sends spam) XOR (sends non-spam)) ' is zero, even for an actual SMTP source. You are still stuck in binary logic. You seem to be very far behind where my thought process is on spam. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
1. Vernon apparently got offended because I pointed out that he didn't realize that MD5 checksum on IPv4 was easily breakable via dictionary attack or that his use of it went his often public stated condescending policy of do not implement half-solutions. And the applicable Kook rule from Mr. Schryver's bible that started this thread is: http://www1.ietf.org/mail-archive/ietf/Current/msg22092.html you are deeply offended when people do not agree that you have found the UFPSTTSP Which Mr. Schryver conveniently removed from his web site version (wonder why?): http://www.rhyolite.com/anti-spam/you-might-be.html Perhaps Mr. Moore should recall my claim to archive mail. He wrote this: Perhaps Mr. Schryver should recall my claim to archive mail. The point I made in #1 above is that Mr. Schryver had apparently never considered the question until I asked it. And then when I pointed out to him he got very offended, which violates one of his own rules for being a Kook: I wrote: Yes that is why I wondered why you bothered to hash the IP at all. I think this is case where you voilated your own principle of not implementing half solutions. There is no security or privacy benefit to hashing the IPs (because a brute force dictionary attack can be built in 100 hours) and there would be benefits to not hashing them such as my algorithm. Now I have to carry around a 16 GB dictionary (lookup table). He responded: I doubt you have figured out how we thought the IP address might be used. I responded: What you thought is irrelevant to facts at hand. The fact is the IPs can be reasonably extracted thus logically it is reasonably useless to hash them. } So if you had 1% of that space, or 40 million IPs in your databases } over time, then would take approx. 20 million minutes = 333,000 } hours = 15,000 days 50 years to convert all MD5 back to IPv4s. I never did figure out what Mr. Moore meant by 15,000 days. He could not have been thinking of doing on average 2 billion MD5 hashes Can you read? I wrote 1% of the space. for each of 4 billion IPv4 addresses, because that would have been silly and would have take more than 15,000 days. Ok, I'll stop feeding the troll now. I will stop feeding the troll by posting in this thread now. I predict he or his buddies won't stop posting in this thread. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
. It is only fixed when the spammer loses his account. Then the spammer gets a new account. So it isn't really fixed. So we are always going to be playing a game of whack-a-mole. That cannot be avoided by altering the protocol or the authentication scheme (information theory proves this). So it is useful, then, to work on ways of detection, Agreed. And you can not detect well if you can not define what you are detecting. improve our whack-a-mole skills. Altering protocols and authentication is a waste of time. Er... unless they improve our ability to detect. Shelby Moore http://AntiViotic.com
Re: The proper forum for evaluation of Spam proposals
while the spam discussion has been entertaining as usual, for those who Not particularly entertaining for me. - There is a proper forum for discussing anti-spam proposals. It is the mailing list of the IRTF antispam research group, [EMAIL PROTECTED] This proposal fits within their taxonomy of antispam solutions http://www.irtf.org/asrg/survey_of_proposals.htm, and the discussion is therefore appropriate to that forum. Consult the ASRG charter and web pages for guidelines for appropriate discussion. Will do. Thank you. As I said previously, I started here: http://www.faqs.org/rfcs/inet-standards.html And here: http://www.faqs.org/rfcs/editor-info.html Which recommend going through IEFT. - The charter of the IETF discussion list says that unprofessional commentary is inappropriate for the list. While it can certainly be entertaining to watch people try to define each other as kooks of various kinds, it is also definitely unprofessional commentary. Agreed 100%! Wish you would have said that earlier to the person who started the Kook thread. So is drawing conclusions in public about other people's intelligence, education and previous epxerience. Which was the whole point of making a Kook thread to try to discredit my post or using the words plonk to address my serious post. Summary: Please move over, folks - wrong arena! As I said, I'll believe that when I see more engineering posts than I see posts in the Kook thread or ones addresses my posts with the word plonk. Professionalism is I think one of the words I used to rant about what was happening. I will move over now to IRTF and expect to go through this whole nasty reception thing all over again. Yippie! :) Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
I am not talking about email spreading virues. A number of viruses appear to send spam. (not spreading). Sometimes this is autonymous. Sometime it is under control via IRC channel back to the virus operator. Further, it seems that many open proxies are installed by virus. Once the virus has control of the computer, it has or will obtain keys to private keychains, etc. It can do whatever the infected users can do. The number of 40-50 emails per IP figure comes from analysis of spam messages that get by filters, by reviewing how many messages came from the same source. A lot of spam that gets by filters is of this very low volume type. Dean this is important data for me, but not as useful until I can get you to tell me: 1. What was the time duration of the sample? 2. What was your total approximate sample size? 3. What was the total volume of this type of spam within that sample? Estimates would be fine. I would really appreciate it very much. Thanks, Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
previously in this thread, once you model spam the way I have proposed, then solicited *BE will have a distinct advantage to adopt the model. And as I point out above, it doesn't matter what spammers do, because the improved model is helpful for advancing detection in both cases. And my other point has been that when a channel gets so saturated with noise that you can not longer find the original signal reliably (as you say above the S/N ratio will depend on Nyquist, which is a very crucial point), then solicited *BE and receivers are going to need a different model, else information transmission will no longer occur reliably. So given a set of unmarked messages, some spam, some not-spam, the task is to have a program mark them in the same way that a human would if a human were reading the messages. Since humans have different definitions of spam, it would be useful if the program could accept different definitions as well. This is the realm of content analysis. You see this is the crux of the whole stagnation of anti-spam in my view. Content has nothing to do with what makes spam annoying. It is the S/N factor, i.e. that it only gets a 0.005% response rate. I am trying to shift the whole paradigm from thinking about psychology (will always be fuzzy result), to thinking and modeling the noise factor. It is a profound paradigm shift that gets you closer to a more robust solution for detection. Thus my whole motivation for an unambiguous definition (spam == all bulk email) along the channel and not just a definition at the end points (UBE). You may need a precise definition before you can begin implementation (just like you need a definition of voltage, current, etc to begin building a transmitter), Exactly. You need a definition before you can model. but you do not need a precise definition to talk about the theoretical aspects. Yes you do. Spam could be defined as UCE, CE, UBE, or BE. I have also a more complete and detailed taxonomy of spam: Those are all definitions. There are 3 types of email that we generally call spam: This is going down into the psychology line of model, which I am trying to paradigm shift away from, because it is not very well correlated to what makes spam a problem. If spam had a 5% response rate, it would no longer be a problem. Modeling the psychology is something other people are working on already. [snip] Thanks, Shelby Moore http://AntiViotic.com
Re: POP3 delivers, not deletes
This is very important to me because obviously my new service (AntiViotic.com) depends on it, to the degree that IMAP is not widely implemented. Thus I have some important questions below... Although it is theoretically possible, using POP (rather than IMAP) to leave the mail on the server until you pull it again with POP, many servers appear to clear out the mail after POPing it. The Client program users LIST and RETR to get the messages than DELE to remove them. The server/cloud program(s) don't do that. If you/anyone want to leave mesage a\on a POP3 MTA, selecting a client program (or writing one) to not DELEte them at your discression is potenetially useful. A *lot* of POP-using programs have the Leave Mail On Server option. And a lot of people have used Leave Mail On Server as a poor man's 1-folder IMAP, leading POP providers to implement mail retaining policies of the RETR it once and it's gone, whether you DELEted it or not. Do you think they also apply this policy to TOP n? Do you have any idea how prevelant this rather extreme policy might be??? That is a very worrisome revelation for me. I was aware of the aging policy you mention below from the RFC, but not of the delete after RETR policy you mention above. * Enforce a site policy regarding mail retention on the server. Sites are free to establish local policy regarding the storage and retention of messages on the server, both read and unread. For example, a site might delete unread messages from the server after 60 days and delete read messages after 7 days. Such message deletions are outside the scope of the POP3 protocol and are not considered a protocol violation. Note that the RFC does not define what a read message is, so I wonder if TOP n can apply , given that TOP n implies partial inspection (even if the entire message is returned)? Thanks, Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
IMO, this (whether Hotmail will implement a specific feature) is a fairly irrelevant (an 80 out of 80/20 rule) fork of the debate relative to the main point of the proposal, so let's try to wrap this fork up with one or two go rounds max okay. Interestingly note that Hotmail makes you pay to POP *FROM* hotmail, but no charge to POP from other accounts *TO* Hotmail. Does that give you any hint about their business model?? Yes. It's *NOT* a business model where they want to be polling a dozen servers on a regular basis for each of their customers for mail that may or may not be there, and for the average mailing list, probably is not there at any given poll. Not any more than they want to be POPing any email at all. Nobody wants to do any work they do not have to do. But if there is advantage for them, or a profit to be made, they do it. If they do not, someone else will, then they lose marketshare. They used to not POP email at all (do you remember or did you know that?). Then they discovered they were missing a big market of eyeballs. They want eyeballs, and the last thing they want to do is expend more effort than needed to get eyeballs. No disrespect intended, but that sentence is illogical. You want something and you don't want to do something that gives you what you want. You mean I guess that they would not agree to add effort just to retain existing eyeballs. Again I disagree. I think they will do what ever they have to in order to retain market share, as long as the cost doesn't kill more profit than it retains. Sure - they can even optimize the 'POP the list' check by only doing it once for all the subscribers - but they're still hitting each server for each list on a several-times a day basis. And under the current scheme, they can just *catch* one SMTP transaction with all the RCPT TO's piggybacked *when there's actual mail*. So they'd have to work a lot harder under your scheme. POPing once (one list mailing) versus processing one email with zillion RCPT TOs (one list mailing) is not a very big cost difference. One might be slightly less than the other and we really can't say which one, but it is irrelevant because the difference is insignificant. Actually it is more likely that when they POP they will get several messages at once, so less cost than catch several SMTP emails. Also they know a priori the correlation of receivers to POP, which can be optimized with time, versus having to build a new mapping table in real-time every time they process an SMTP with RCPT TO. And let's *THINK* for a moment here - what is your proposal *REALLY* going to change? We already have many estimates that 50% or so of all e-mail is spam. Let's take that as a given, and let's make the rash assumption that the rest is 25% mailing list traffic and 25% person-to-person. It be more interesting to know what the real stats are on the other 50%, because I doubt that 25% is legitimate bulk email. It seems that you live in a different (mailing list centric) world than I and most normal people live in. I join mailing lists for a short time to get something done, then I leave asap. Most of the people I know and the many thousands of customers I come into contact with, seem to not even know how to use a mailing list. With 500 million people on the internet, I would venture that 80% don't even know what a mailing list is. They may use Yahoo Personals, and not even realize it is a mailing list. Since the email is being directly deposited into the Yahoo account, they have no clue. Any way, let's follow your line of debate... So what you want to do is take the 25% of the list traffic that works just fine on the current infrastructure, No it doesn't work fine. My gf complained that she couldn't find her Yahoo Personals email amongst the 500 spams she gets per day, of course that makes me happy but that is besides the point :) and is usually quite easily whitelistable via a number of different methods - Whitelisting can be subverted by spammers: http://www.cnn.com/2003/TECH/internet/09/01/spam.chainletter/index.html ...Herrick, however, admits that the practice could be a good way to bypass e-mail filters which block messages from senders who are not known to the recipient. Spammers could use chain letters to discover the addresses of people with whom you frequently communicate. Spam purporting to be from someone in your address book would sneak by filters. If I were a spammer, I'd be working very hard to perfect this technique, he said... and move it to something totally different. And what you're left with is a 2-1 mix of spam and personal mail that you yourself admit things like the DCC and spam filters are unable to perfectly distinguish. The whole point of the change is to enable elimination of the spam which can not currently be done. See my response to John C Klensin, regarding chicken and egg and the example benefits to
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
At 01:43 AM 9/7/2003 -0400, you wrote: On Sun, 07 Sep 2003 13:07:10 +0800, Shelby Moore said: It is a wrong assumption to equate commercial email with bulk email. Which is why you're trying to rewrite how bulk email is done in order to deal with *one segment* of commercial e-mail. Now I understand fully. No I am trying to eliminate (drastically reduce) spam (UBE). And, I am trying to give receivers the ability to opt-in and opt-out of all legitimate bulk email under their own power. In my mind, that is more important than existing legitimate bulk email paradigm. I can understand you and other's vested interest in existing legitimate bulk email paradigm, but I don't agree it is more important. When spam is 99% of all email you recieve, you may begin to care also. It won't be too long from now... Fortunately what this discussion is proving to me so far, is that architectual change will be resisted fiercely by vested groups that control the design of the internet. This means that my new (soon to be patent pending) algorithm for http://AntiViotic.com will have a near monopoly in the market. So for me and users of my service it will be great. But for the rest of users, spammers will be forced to actually increase the amount of email they send in the self-feeding model of my algorithm. Unfortunately, the rest of the email universe will be imploded by an accelerating rate of spam. Just imagine what spammers will do when confronted with an insubvertable algorithm that can detect bulk. I came here to see if there was a better way, but I guess we will have to do it the hard way (and very profitable for me). This is all conjecture (vaporware) at this point...but very well researched conjecture. I'll be back here in this list later (probably a year from now) when your needs have changed to a more dire state regarding email.
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
At 01:54 AM 9/7/2003 -0400, you wrote: The evidence indicates that the senders will use whatever is more likely to result in the receiver seeing the message. This is different from seeing it where the receiver would like to see it. I get your point and it is a reasonable one that must be taken seriously. However, I think that at some point soon, users will have no choice but to either stop using email (because it doesn't work efficiently any more) or look for a solution to spam. At that point, I think they will demand change (already seeing early warning signs of that now). Also you are assuming that legitimate bulk email is very important to most receivers wherein I think for most spam is a bigger issue. Lastly, I don't think my proposal has to be any more inconvenient than existing sub/unsubscription procedures, which require trading several emails, versus inputting one POP domain in an email client. I understand the concept of diminishing utility in economics, so I take your point seriously. And I will ponder it more. The point of the usenet reference analogy is that if placing things where people have to go look was sufficient, folks would not have started using spam email. But there is a key difference. Users started preferring email without the agreement that it would have spam rate increase from 8% to 50% in 2 years. When the noise gets too high, you turn off the TV and do something more productive (remember antenae reception?). If you have a blackout every 5 minutes, you giveup trying to do anything with appliances. Etc.. They would have used other techniques. And they will if spam continues to increase. They already are. And those techniques may have much worse effects than a correct architectual definition of spam. For example, legitimate mailing list traffic is getting falsely classified as spam. Maybe not for you, perhaps because you are expert, but I assert your experience is not likely shared equally by the 500 million... Many (most) of the spammers are simply interested in any technique that will get the message in front of the recipient. Hence, they will use the path that is checked most often (the personal mailbox). Yes my proposal depends on that fact. Once you have the legitimate email separated from the spam architecturally, then you can effectively increase the cost of spamming to the point it is a non-economically viable activity. Remember the economics of spam is precisely what makes it so noisy. If spam ever got like 5% response rate, then it would not be such a big problem as it is. The fact that is averages something like 0.005% response rate is what makes it both a big noise problem and also makes it very fragile cost wise. The typical individual spammer does not make $300,000 a day. More like less than $100,000 a year. And the institutionalized spammers (the economy-of-scale bundlers) would make nice big fish for FTC. The more you can concentrate, the better. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
Don't get me wrong, but respectfully, this has *NOTHING* to do with SMTP. SMTP is not involved and not changed. Therin lies the flaw in your plan, as smtp must continually change in a distributed fashion in order to effectively reduce the amount of egregiously time-wasting email that flows through its veins. No you are counting the chickens before the eggs. If you have no more eggs, then you won't get any more chickens. The misuse of SMTP has to be attacked at the root cause, which is my proposal (go study all my posts for full logic), rather than morphing SMTP in a never ending game of cat and mouse.
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
Keith, IMHO you started an excellent line for further debate (and not just because we have the same last name :). It would be nice to see debate from both sides so that pros and cons could be fully explored. I am not sure I am the one to carry the debate to extreme end (due to time constraints), but I will initiate the counter argument. I came here to make a proposal. It is not a proposal I would champion to the end game alone. The feedback is valuable (hopefully to community-at-large as well), whether the proposal florishes or not. What you are saying IMO, is that you can't force bulk emailers or spammers to use opt-in. Let's be even clearer. What's being claimed is that you can't force bulk emailers to send their email via pull technology (whether this means providing their own POP servers, IMAP servers, NNTP servers, web servers, whatever) while everyone else can still use push. Enforcers will force the adoption. Enforcers being Hosts, ISPs, legislatures, judiciaries, software, etc.. Once you define that *legitimate* bulk email is only delivered by pull, then enforcers can start to filter bulk email. This will force everyone to get their *legitimate* bulk email via pull. Else they will not get it. If you have a choice between receiving your *legitimate* bulk email via pull, or not receiving it, what are you going to do? Now below you argue that enforcers will not enforce. To me that is more viable argument, one that needs to be debated. However, what is the harm in making an RFC and then find out if enforcers will enforce?? more below... And the question isn't really whether bulk mail can be statistically distinguished from non-bulk mail (since that's really just a matter of whether you can get people to adopt a definition of bulk in terms of externally visible traffic properties) - the question is whether you can enforce that rule. Agreed. Good point about benefits (real results) of agreeing on definitions. That is in essence why I am making this proposal. I am proposing we make a definition of spam which is both agreed upon and not ambiguous to enforcers. Simple as that. In my mind, the thing to consider is what is the cost of making this definition compared to what is the benefit?? To me, that is one of the major aspects that need to be debated. Economics (in the larger sense of economy as efficiency or survival/evolution, not just financial systems) *always* rules the outcome. IMHO - most recipients don't want to get their mail that way (and many of the deployed user agents don't support it), Most people already get their email through POP, even if there is a web-based client on top of it. We already went through the discussion that Hotmail and Yahoo have the ability to access POP accounts. They wouldn't have the feature if people aren't using it. most senders don't want the increased burden of providing POP/IMAP/NNTP/web servers Only *legitimate* bulk email senders (mostly that is mailing lists) would have to do this. Non-bulk email and spam does not change it's paradigm. Spam is dealt with by enforcers. Only *legitimate* bulk email senders (mostly that is mailing lists) would have to do this. Only *legitimate* bulk email senders (mostly that is mailing lists) would have to do this. Just making that very clear. *Legitimate* bulk email senders already have a burden of maintaining mailing list databases, maintaining a subscribe and unsubscribe support, handling bounces, etc...s and the necessary customer support, I assert there will be less or similar hassles with supporting a pull paradigm for *legitimate* bulk email than all the hasslings of teaching newbies how to subscribe and correctly interface with Majordomo mailing list. I do not think it is a big difference. And if you are protecting the mailing list administrators to save spam, then that seems like a bad set of priorities for internet email as a whole. and there are enough ISPs that derive significant revenue from selling bandwidth to spammers that it would be extremely difficult to get them all to enforce this. This is wrong. This would be saying that the economy of spam is significant compared to the internet economy as a whole. ISPs are losing $ big time to spam. In short: nobody has sufficient incentive to adopt this. Maybe nobody in this list, but let this thread sit in Google for a while and see what might snowball over time :) Of course there's nothing wrong with defining another way to distribute bulk mail that people can use if they wish. Great. That is what I said above. Why not see what the enforcers will do? They can not do any thing now, because their hands are tied to ambiguous definition of spam as UBE. They need spam == *BE in order to act. If it works well, some people will use it. The stretch is to insist that everybody do it this way. No. If it works well (if the enforcers adopt), then everyone
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
This is getting way off topic. One of the other things you see to be handwaving a bit about is the notion of handing out user IDs, passwords, and other credentials to mail accounts to people so they can help with spam (or other problems). My proposal has nothing to do with IDs so let's just drop that line of discussion or respond with a different subject to move it to a different thread. Sure, I can find a web-based something-or-other to access my POP3 mailbox (if I had one). Agreed. the web site listed in your signature line seems to mostly run people around in circles My web site and businesses have nothing to do with this proposal. The proposal is to define that *legitimate* bulk email must be sent and received by pull instead of by push. So that all remaining bulk email is spam. This changes spam from Spam == UBE (an ambiguous definition for enforcers) to Spam == *BE (UNambiguous definition for enforcers). How *BE gets dealt with by enforcers is not related to my businesses or web site. Enforcers is a widespread term meaning any one who will enforce the proposed RFC against spam. FYI: (completely off topic so please don't respond to list on this point) The reason the web site runs in circles is it isn't finished yet, nor ready to be released to public at large yet. It should be completed in a few more days. Shelby Moore http://AntiViotic
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
As each individual news article is piped through a relatively small number of servers in the core of the distribution system, it becomes relatively easy to blacklist known offenders. That is, if they are recognizable as such. This is where the authentication comes in. The tricky part is optimizing the time/difficulty it takes to blacklist vs that of obtaining a new identity. Even though this is a good point, I'd prefer to stay off the discussion of authentication, because the proposal need not depend on it, as you point out below... Also note that with usenet news, unlike email, it is possible to remove spam that has entered the system, limiting the audience that sees the message and thus the effectiveness of spamming. Also bear in mind my point that the whole point of moving legitimate bulk messaging to pull is so that spam can be dealt with unambiguously by enforcers on bulk email. So spam (all remaining *BE) gets reduced by the paradigm switch to pull for legitimate bulk messaging. Since I assume mailing lists would still use email as incoming source primarily, then incoming spam is reduced. You could certainly switch to a authenticated interface (e.g. https web page) for incoming mailing list traffic, but I think that is unnecessary to my proposal. Also remember that some portion of legitimate bulk messaging is legitimate (meaning that receivers want it) corporate bulk mailings. They would also be forced to the pull paradigm, but not their normal single messaging, such as the receipt for what you buy on Amazon, only their bulk email. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
As each individual news article is piped through a relatively small number of servers in the core of the distribution system, it becomes relatively easy to blacklist known offenders. That is, if they are recognizable as such. No way. My proposal does not depend on authentication of what is incoming to the pull servers, because by definition it enables the enforcers to filter *BE (all bulk email) which (by the proposal) is defined to be all spam. This is a non-issue relative to my proposal. If you are really paranoid, use https web page to take incoming from authenticated users. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
Excuse me, it is a valid issue that spammers will try to pipe through mailing lists (legitimate bulk email) to avoid *BE enforcers. Mailing list administrators will continue to carry this burden and probably more so under my proposal. Thus yes I agree that authentication of incoming to pull servers is a major consideration. However unlike with current paradigm, the receiver can then opt-out of the spam. Thus darwinism will force effective solutions, because those mailing lists that are successful will have more receivers than those who are not (because the receivers how power of opt-out of spammy lists unlike now where they can not opt-out of spam email because their legit bulk email not unbundled). And remember that commercial legitimate bulk email is not effected by incoming pollution issue. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
I am nearing the end of my allow time to respond, so if I do not respond in future, it doesn't mean I agree :) below... 2. Regarding additional burden on *legitimate* bulk message *senders*: a. These senders are much, much fewer than the # of receivers suffering from spam. Any incremental cost on the few is justified. yeah, well good luck convincing *them* of that. especially when you can't even convince *us* of that. You (us not including me and perhaps some others) are apparently them, so I see no need for the word especially. Further, there is no need to convince the minority. Majority wins. If enforcers successfully block all bulk email with my proposal (as an internet STD) sanctioning it, then receivers (the minority) will dictate the outcome. I assert that even if 50% spam is not bad enough to make the 500 million rise up, just wait until it is 90+%, which again I assert won't be too long from now. b. The senders are already quite resource savvy, else they would not be sending *legitimate* bulk (in statistical significance) messages. I doubt the incremental cost is significant to cause failure. let's see. it doesn't cost the spammers much to send bulk mail... Oh it costs spammer something. They are definitely more resource savvy than the majority, which was my point. The burden on the minority is less important, if the majority is making big gains from the proposal. so why do you assume that the costs are significant to legitimate senders? They have to maintain mailing list servers, mailing list databases, manage support for subscribers, handle incoming spam, etc.. All things which an average (the majority) individual user of email doesn't have to do (or won't bother to do). c. And compare this burden to the burden they have with dealing with spam, which would be eliminated by this proposal. If spam isn't eliminated, then they need not adopt the proposal. most of the burden of dealing with spam is on the recipient, not the sender. Agreed currently. But not once my proposal succeeded, which is a correction I made after making this point #c above. Point #c is invalid. Please remove it. apparently you think that legitimate senders should pay for the cost of lessening the burden on recipients from illegitimate senders. Insert bulk after legitimate and illegitimate, then I agree with your statement. The reason is because *legitimate* bulk message senders are going to pay one way or the other soon. It is like the old Fram oil filter commercial in USA, Either pay me now (replace filter) or pay me later (replace motor) As the rate of spam approaches 90% or 99%, receivers are going to opt to block all bulk email, irregardless of whether my proposal is in place. They won't have any other choice, else quit using email. Now that is profound prediction! why do I think you're not likely to get much support for that view from the senders? Oh yes to be expected that human nature tends to manicure (defend) their own feet instead of watching where they are going and what they will soon bump into. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
However, what is the harm in making an RFC and then find out if enforcers will enforce?? you appear to presume that you can get consensus support for such a plan from within IETF. No, no. I try to never beg. I came here to make a public proposal and some points for the purposes of public record. I never expected any one to really do any thing on this proposal now. But by placing the idea into public domain now, as the rate of spam approaches the asymptote of 100%, then mailing list administrators can come back to this idea as a way to save themselves. Actually at that point, they won't need the RFC or STD, because all bulk email will be blocked and they will be forced to just go straight to offering a pull solution (whether it be web, usenet, pop or whatever) so recievers can still get their mailing list messages amongst the 10,000 spams per day. I am not sure exactly how long this will take, but without another solution or paradigm shift, I figure 2 - 5 years tops. even if you could get such support (which you cannot) How do survey their opinion?? note that there's no enforcement of IETF's other opinions, even in cases where failure to adhere to IETF standards costs billions of dollars every year. why would this case be any different? Because enforcers want to do something and will be under increasing pressure to do something against bulk email. there are a lot of ways to solve the spam problem that will work if everybody does it my way. I have seen none. Email signing won't work. No amount of !plonks from Randy Bush will change that. Changing SMTP won't work long term, etc.. so far, nobody has figured out how to impose their will on the rest of the net. It is not my, your, or IETF will, but the votes of 500 million (through their actions against spam) that will impose. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
Whew! That is a long list, and fortunately I don't exhibit any of your stated symptoms of Anti-Spam Kook so I guess I should feel sane?? Too many to addresses all, but here are a few... I've been compiling a list in the style of Jeff Foxworthy. You Might Be An Anti-Spam Kook If - you have discovered the Ultimate Final Perfect Solution To The Spam Problem (UFPSTTSP). I am predicting spam to continue increase and in fact accelerate with either my proposal or my other work. Face it, filtering of bulk email is going to happen and bulk email will increase because of it. This might seem contradictory to other posts I have made, but it is not. Whew! Thus no other UFPSTTSP below apply to me. - you were motivated to find the UFPSTTSP because you know it is impossible to filter more than 99% of spam with fewer than 0.1% false positives by any of several currently available mechanisms. Whew! Luckily, I know what a mathematic asymptote, limit theorem, and confidence interval are. I guess I am still sane, praise the lord! - despite being the inventor of the UFPSTTSP, you are unfamiliar with false positive, false negative, UBE, tarpit, teergrube, Brightmail, Postini, SpamAssassin, DNS blacklist, HELO, RBL, or mail envelope. And AccuSpam.com :) - you plan to make money by licensing the idea of the UFPSTTSP. I hate licensing and find sales to end users much more direct way to earn income, especially when you've already automated that process in your other businesses. - you plan to publish an RFC mandating the UFPSTTSP but have no idea that RFC 2223 or RFC 2026 exist. - you have no idea of the relevance of consensus or IESG approval to publishing RFCs. I know I started here: http://www.faqs.org/rfcs/inet-standards.html and here: http://www.faqs.org/rfcs/editor-info.html Which refers to all your points and many which you missed! - you think that spammers won't ignore, subvert, or exploit the UFPSTTSP if you publish it as an RFC. One immutable fact is spammers will ALWAYS have to send in bulk. - you think that a violation of an RFC by an SMTP client or server is good and sufficient reason to reject all mail from the system's domain. SOBO (statement of the blatantly obvious) - despite discovering the UFPSTTSP, you don't know the meanings of MTA, MUA, SMTP server, or SMTP client. Yeah that would be bonehead. - the UFPSTTSP requires a small number of central servers that for validating email, serving as pull servers for bulk mail, or anything else. Agreed centralized solutions do not work. - the UFPSTTSP requires that anyone wanting to send mail obtain a certificate and that such certificates would be checked by all SMTP servers. Email signing is failed concept. - you think that most Internet users would willingly pay more $5/month to avoid spam, and don't know the per-user price point for anti-virus software or data. Free is nice for most. - the UFPSTTSP involves replacing SMTP. Morphing SMTP is a losing concept to anti-spam. - you routinely send single LARTS or reports of single examples of objectionable mail to more than two dozen addressees. Sending 24+ emails of anything might also qualify. - your definition of spam differs significantly from unsolicited bulk email. Hint: unless that definition also corresponds to what is subjectively unsolicited bulk email most of the time. Nice fishing expedition...you know what I mean Vern. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
- despite being the inventor of the UFPSTTSP, you are unfamiliar with false positive, false negative, UBE, tarpit, teergrube, Brightmail, ... Another interesting tidbit is that Enrique Salem, the CEO of BrightMail (which is a Symantec company), is my former classmate from Culver City High School class of 1983. http://www.brightmail.com/about_us-exec_team.html#enrique_salem That has absolutely no relevance, as neither does this thread by Vernon. BrightMail in my experience with Earthlink, filters 90+% of spam with very low false positive rate. Shelby Moore http://AntiViotic.com
Re: You Might Be An Anti-Spam Kook If ...
I've tried to improve that one Vernon, rant What are you improving? Oxymoron. Would it be any thing related to Internet Engineering (as in IETF)? Is this really appropriate posting for someone who apparently runs the DCC, which is apparently supposed to a professional service that millions of clients rely on? Is this the kind of stuff you do and think about all day? What about actually doing some real work on Internet Engineering? Is this the kind of posting you want publicly distributed when later someone Googles or otherwise investigates what are the thought processes and professionalism of the people engineering the foundations of the internet. Frankly, I will be linking back to this thread if any one ever asks me why spam can be solved at the internet engineering level. I sense you feel that some people who have ideas do not know enough about what they are posting, but you might be surprised what people actually know. I've searched your posts all over the web via Google groups and you have a history of being obstinate, condescending, and impolite. You are also much less of an expert than you portend to be relatively speaking... /rant Shelby Moore http://AntiViotic.com
Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
Request for opinions on whether to creating a working group or publish the following idea as an internet draft? Spam is big problem that is getting worse. BrightMail.com (which claims to process 10% of world's email) claims that the percentage of spam out of all email has grown from 16% in Jan. 2002 to 50% in Aug. 2003. A fundamental unsolved problem of doing any thing about spam, is there is currently no unambiguous definition of spam as an enforceable internet standard. This has been architectually impossible to define because the receiver is the subjective determinant of which bulk email is solicited and which is spam (UBE). ISPs, Hosts, legislators, judiciaries, and even anti-spam software, have a fundamental problem in that definition of spam as UBE is currently architectually unenforceble due the fact that subjective determination of unsolicited current happens after the email has been delivered to the receiver. My idea is to create an internet draft, RFC, and hopefully internet standard, that would define a simple architectual paradigm for legitimate bulk email that unambiguously separates it from spam (UBE). Simply define that legitimate bulk distribution of email should be done by mechanism of each bulk distributor providing a public POP3 (and IMAP) account or server, rather than sending the email directly. In the case of a public distribution (e.g. most direct email and mailing lists), a POP3 (and IMAP) account of user anonymous with password none would suffice. In the case of private dissemination (private mailing lists), a POP3 (and IMAP) server with individual accounts could be provided. The elegance of this paradigm is that users then control the opt-in/opt-out database, by configuring their email client to POP email from only the bulk POP accounts they wish to subscribe to. The effort to support this paradigm is minimal because it uses existing email paradigm. Legitimate bulk senders have to change from a broadcast (push) metaphor (e.g. Majordomo) to a pull metaphor simply by depositing their outgoing email in the public POP account they create. Receivers simply follow instructions to POP bulk email they want, instead of the equally complex task of subscribing to bulk email. This accomplishes several goals: 1. Any bulk email is then spam (receiver has not opted in) and can be dealt with by ISPs, Hosts, legislators, judiciaries, and anti-spam software. 2. Receivers now have uniform control over opt-in/opt-out policy without a global authority 3. Legitimate bulk senders can be insured that they or their email won't be misclassified as spam 4. Those who send UBE can no longer claim they are legitimate or that receiver has opted-in (ambiguity removed) and can be dealt with by ISPs, Hosts, legislators, judiciaries, and anti-spam software. 5. With a pull paradigm, the load (resource usage) on the public internet, sender, and receiver is reduced, because I venture that a majority of bulk email sent would not be pulled. I think this paradigm would empower Hosts, ISPs, legislatures, and judiciaries to do more about spam (incoming) and spammers (outgoing), because their hands would not longer be bound by ambiquity. I realize that some vested interests, such as direct emailers or those invested in push based mailing lists, might resist. However, I think the benefits outweigh the limited costs to migrate. Some direct emailers might resist because some may prefer being able to cloak spam under the guise of solicited. Legitimate bulk emailers stand to gain a lot by separating themselves from the noise of UBE. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
One additional point in response to this point: So let's see.. Currently, if your bank sells your e-mail address to another company, you get spammed. So instead, you'll have it so that you check your bank's POP server in case there's important mail about your mortgage. Seems like the obvious scheme is for the bank to charge the other company to put stuff in your POP mailbox. So you still get spammed... In addition to what I wrote below, remember that when the bank is sending me an individual email regarding some business I am conducting with them, they are not sending that email in bulk (to any one else). They could send that directly to my email. If they are indeed sending a similar email to ALL their clients at the same time, then in that case they are sending bulk email. So with my proposal, it just forces business to separate their business email from their marketing bulk email. If I trust my bank to send only important bulk email, then I can add that POP account to ones that my email client checks regularly (as regularly as I chose, not as my bank choses...gives me the control). Given that email is insecure transmission medium, no business should be sending me anything too important in email. They had better have an alternative means of getting in contact with me about important and urgent matters. ---I wrote before only No. Because you can chose to not check it and/or you at least know who is spamming you and can hold them responsible directly. Thus your bank would stop doing it, because they make $ by not losing your business.
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
that involves a change to 500 million users. They all had to upgrade their browsers. So I submit to you that if this idea is too hard to use with the current version of Outlook, it is a *non starter* as a practical matter. If Microsoft likes the proposal, it won't matter what you or I think :) There are many RFCs that are not implemented by Outlook. That doesn't stop people from trying innovate and then let the chips fall where they will. I think that current version of Outlook could be used as is for bulk of users (who only have a few legitimate bulk email needs), except that old messages would get continually redownloaded. But fortunately Outlook supports plugins, so it should be fairly trivial to add the UIDL tracking to Outlook, even if Microsoft refused to. 50, even 5000, is not statistically bulk on internet scale. Is it not possible (or likely) to write laws without exclusions? Do you think Hosts, ISPs, and anti-spam software would not account for this statistical phenomenon? Only problem is that many spammers are *already* only dropping 40-50 copies of a note at a site at a time, specifically to work around that - then the rest of the spamming recipients at your site get a different version with a different From: line and a different source IP address. So you are indeed heavily involved with the DCC! Yes I know that is a weakness in the way the DCC measures bulk, but that is irrelevant to the point above. I submit to you that if you didn't realize this was happening, you may not be qualified to be suggesting proposals to counter it If I did not realize it was happening, then I would not have created http://AntiViotic.com . I'd be using the DCC instead. I also wouldn't have approached Vernon about improving the DCC (only to be turned away in nasty tone). (Hint - if spammers weren't doing this, it would be trivial to block them, and we'd not be HAVING this discussion, right? ;) As I said above, I actually have a business incentive not to make this proposal. But I felt it needed to be made. My conscience and desire to do important work is stronger than my desire for profit. I wish you would remove the personalized attacks in your posts and stick to a discussion of facts. No. As I said above, they would only need to provide one POP account for this mailing list. And as I pointed out, you'll need to create 30,000, because one account doesn't allow you to keep track of who has already seen what messages. And no, you're *NOT* allowed to just say everybody can fetch all the UIDLs and we'll just tag them with the subscriber ID - go read and *UNDERSTAND* section 6.2 of RFC2298 in order to understand why. You did not understand the concept. The per user UIDL tracking is stored in the email client, not in the POP server. You might also want to go re-read the ASRG mailing list archives, your proposal (and variants thereof) has been kicked down the beach like a dead whale multiple times already. Well apparently not enough, since you are claiming to be fully knowledgeable about those posts, and you are not able to provide good refuting logic to my proposal yet. If you would like to refer me to some specific threads (with links please), then I will be happy to read them and adjust my comments here or even withdraw the proposal if I find it is warranted. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
Another reason why you need unique userid/logins for each subscriber - so that you can prevent forging a UIDL for somebody else to keep them from reading the message. Being able to do this (and if you have a shared userid, it's almost impossible to prevent) would make the Bernstein/Bush flamefest regarding list censorship look trivial in comparison... Again I will repeat: You apparently did not understand the concept. The per user UIDL tracking is stored in the email client, not in the POP server. Oh... there's also this thing called webmail. Lots of people use them so they can get mail no matter where they are. You can get mail no matter where you are with a POP account also. Most people use webmail either because it is free, or because it is a convenient interface on their POP account. Lots of these people will be fundamentally stuck under your proposal, because every time they used a kiosk they'd have to enter in all 20-30-40 POP servers they had mailing lists on. Again cars today have rubber tires and titanium rims, not wooden wheels. Applications can improve. It actually happens most of the time. Hotmail and Yahoo already support and encourage the pulling of email from POP accounts. You don't have to enter the POP accounts every time, just once. Surprisingly enough, hotmail.com and yahoo.com have a lot of subscribers, you might want to factor that into your plan How many more silly personal attacks?
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
At 11:53 PM 9/6/2003 -0400, you wrote: Actually, the point is that there was no way, even within usenet, to prevent pollution of individual groups with innappropriate spam or off topic messages. Many groups have fallen into disuse for this reason. Afaics, that is irrelevant, because even a mailing list can be polluted with spam. Your point is against the nature of a public list or group, but my proposal is not designed to fix that problem. My proposal is designed to fix the problem of receivers being forced to receive bulk email from any sender. My proposal forces email addresses to not be public to legitimate bulk email. And then all other bulk senders can be dealt with more effectively. Then of course this does solve the problem for mailing lists also in the end, because then all bulk email gets killed (by a combination of legislative, judicial, ISP, Host, and software features). The fundamental problem with doing any thing about spam, is there is no way to measure it, because the subjective measurement of unsolicited is not architectually defined. The point is that if I choose to receive email from list or group (irregardless of whether that list or group has effective policies to prevent incoming spam posts), then it would be helpful to *architectually* differentiate that which I subjectively decide is legitimate bulk email senders from that which I decide is UBE (spam) senders. Once you define the architecture, then all kinds of important features can be built on top of it to deal more effectively with spam ranging from legislative, judicial, ISP, Host, and software. The other point to understand is that the paradigms (usenet vs email) are and have been different for very good reasons. And if one assumes that both paradigms are desired, then it is necessary to address how one enforces the separation. No one to date has figured out even a set of definitions to address the needs, much less mechanisms. I am unclear how you think this applies to my proposal? Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
I merely pointed out that in all probability, you haven't actually *tried* doing what you suggest, because it's not anywhere near as usable as you might think. How could you know if does not yet exist? We can discuss facts and theories of how it would work. But no one can actually test how it does work, because it does not exist yet. The comparison of setting POP accounts in an email client and popping the email, versus subscribing to that many bulk email lists and popping the email, is not useful because no email client has yet been designed to optimize popping from many anonymous POP accounts. If I'm involved with the DCC in any way shape or form other than the fact I've seen Vernon write about it on IETF lists, it's such a covert involvment that even I am unaware of it. Ok cool. Let's move on. Shelby Moore http://AntiViotic.com
Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)
, but no cigar. I am smoking it (figuratively) right now :) Thanks for getting to the crux of the matter and allowing me to make it clear. Shelby Moore http://AntiViotic.com
