At 08:50 AM 1/26/2010, OrionWorks - Steven Vincent Johnson wrote:
http://blogs.msdn.com/tzink/archive/2010/01/25/spam-is-solved-we-can-all-go-home-now.aspx
http://tinyurl.com/ylj42d5
I would love some comments on this article.
Okay, here goes!
The article describes an interesting technique that can be used to
identify some spam, but does not even begin to address the overall
problem, for this technique only works to identify spam after spam
has been already identified by some other means, with, quite likely,
a substantial delay. Then filters can be advised and used to tag spam
for rejection, but the spam traffic is unimpeded.
It should be realized that even if spam traffic never gets to users,
being rejected at the server level, it still adds a great burden to
mail server load. It is still a serious problem, impacting ISPs
directly and thus users indirectly, for we pay all the costs of most ISPs.
We also pay another cost, even if we don't see spam, we pay the cost
of rejected legitimate mail, which is so high, particularly when one
is in businss using email, as I am, that I do not allow my personal
spam filter to automatically reject mail, it merely tags it and
categorizes it for my review. In practice, there is so much spam that
I do rely on IP blacklist filtering, when I've been away and the
queue of mail to be rejected is large, but I still have a log of
rejected mails with 20 lines from each mail, after a mail is deleted,
and I can restore these mails and, at least, respond and ask for it
to be resent. I do not allow my mail server provided to reject mail
at all, except when a major attack occurs, such as one time when it
looks like some spambot got stuck and I was getting 100 spams per minute.
To me, there is a generic solution to this and many other problems:
organization of those most directly affected, and all those
interested in the problem. Among those affected, there is a small
number who will actively fight spam, and these efforts should be
coordinated to be efficient. However, the general membership of such
an organization can be advised to install a particular kind of spam
filter, that the organization would provide.
It would need money to do its central work, but the membership that
would be benefited could be so large that collecting modest donations
for this would be trivial. How much would you pay to substantially
kill the spam problem, without doing harm to legitimate mail? How
much would ISPs be willing to pay for something that made their job
much easier by offloading analysis of spam to a trustworthy
organization of users. Including their users.
The key organizational problem is "trustworthy." Spam filtering can
quickly and easily become a tool for information control, and there
are signs that some anti-spam organizations have been co-opted by
those with particular agendas, such as by spammers whose goal is to
block competitor's spam while passing their own.
How would a voluntary association of mail users address spam? Well,
that's a problem for the users themselves to address, gathering and
vetting expert opinion, and the details of the organizational
structure that would make this so efficient that a mail user could
join and be effective with practically no more investment than
raising a finger. I won't detail the process for right now, but trust
me. It can be made incorruptible; those who attempt to corrupt it end
up with a mouthful of hair. The structure is cellular, fractal, and
probably bulletproof against any danger except massive
governmental-level censorship and repression. If we have come to that
point, we have much more serious problems than spam.
Spammers have been known to successfully attack anti-spam solutions
that implemented part of what I imagine the organization would do,
and they were able to accomplish shutting these solutions down
because the solutions were centralized, operated by a private
company, depending on a single ISP, and turning a botnet to attack
this company was trivial for a serious spammer. The ISP, facing
massive DOS attack, booted the company in order to protect the rest
of its subscribers.
But the association I'm talking about would itself use distributed
process and would not be vulnerable to attack by botnets; they would
be able to shut down particular nodes, but, in the process, revealing
themselves and their assets. Which can then be addressed directly.
It's obvious that detection of a spam bot, as quickly as possible,
and rapid notification of the ISP for the corrupted computer, with
rapid shutdown of most internet access for that computer (everything
outgoing, basically, though filtering could become more
sophisticated: everything outgoing except for the ISP's own support,
so that the blocked user can inquire by email and get immediate
advice on bot removal and prevention of reinfection).
So how to detect spam as quickly as possible? Well, users themselves
identify spam and delete it. If they are using a spam filtering
program like Mailwasher, as I do, I train the filter to recognize
spam by pushing a button for it. There is even a feature in the
program, First Alert, that will transmit this to a server, but First
Alert wasn't working for a while and I don't know if they ever fixed
it. In any case, consisder that whenever a user tags a mail as spam,
the mail itself and the tagging are transmitted, as permitted by the
user, to one of a large set of servers for further analysis. A single
report would do practically nothing. Multiple reports would escalate
the "case" and it's possible that some fully automated processes
start up, such as processes that would call up special notice in user
programs requesting confirmation that a mail is spam or legitimate.
Where the system produces contradictory information, volunteers would
confirm spam, this would be the first human intervention, and the
system would be set up to make it easy and efficient.
As a distributed system, appeals process would exist, rising to
ever-increasing levels of trust in the organization. False spam
reports would impeach the trustworthiness of the reporter,
deprecating the reliability of that reporter.
Now, when spam is conclusively identified, which means multiple
confirmed reports of spam mail justifying serious action, two actions
would occur: the originating IP and other clear identifying
characteristics of the mail would go onto a blacklist, which ISPs
could use to reject mail before it is transferred to the server,
during the negotiation of a connection. The rejection message
provides information on appeals as a URL. Innocent mailers who do get
caught this far will appeal, but the appeals process, while designed
to be efficient, will also more deeply identify the sender. An appeal
should be quick, easy, and conclusive! I've been blocked, many times,
by false identification, and it was not at all easy. And that is part
of the damage that spam does.
And when an ISP for the originating IP is unresponsive, there is the
"internet death penalty," which isn't really a death penalty and it
can be implemented in a do-no-harm reversible manner. The IP gets cut
off from the internet, relay companies refuse the connections. This
is after plenty of warning. If an ISP is being abused to send spam,
then the ISP *must* stop it or the ISP itself is abusing the
internet. The process described would provide full evidence, easily
accessible in detail to trusted administrators in the organizational
network, and fully accessible to the source ISP (since they could see
it themselves in their logs and archives).
Very dangerous, obviously, which is why the structure of the
organization which does this is crucial.
But it would kill spam, quickly. Remember, spammers send massive
amounts of mail. With a large sample of users running the provided
program(s), I'd expect identifications to start coming in within
minutes, and escalate to the spam-filtering level within a few more
minutes at most, and complaint process to ISPs would take little more
time. Within a few minutes, spammers accustomed to being able to mail
100 mails per minute from a spambot would lose the resource well
within the first 1000 mails, and the spambot would be identified and
shut down. If the spammer throttles back, the overall yield of
spams/spambot would decline, and losses of ISP accounts would be
rapid, with loss of money associated with that.
Spambot detection would also be provided by the organization, and if
the user is running a certain program routinely, there would be a
record transmitted of where the spambot was installed from, with
follow-up from the organization to shut down the hackers who do this.
I've followed up on hacks to my web site, and filed complaints, and
have seen action from domain service providers and hacker ISPs, but
it was entirely too much work, the tasks should be semiautomated and
made efficient. The only way I see to do this is with a collective
organization of users.
Companies, however, would be engaged to provide services to the
organization, because there would be -- and should be -- money to be
made in effectively fighting spam, and the workman is worthy of his meat.
It would all start with an organization of internet users, not as a
spam-fighting organization. If it starts as a spam-fighting
organization, the spammers will quickly identify it and attack it,
and if it is small enough, they can succeed in shutting it down. So,
if you see a proposal to organize internet users, understand that all
the possible usefulness of this might not be spelled out and
featured.... If the spammers start to attack organization of internet
users, per se, well, that would attract some serious attention, eh? I
don't think they would dare.
