Re: your mail

[Pekka Savola]

On Sat, 6 Sep 2003, shogunx wrote:
> have we completely deprecated 's for A6's or is  still considered
> bcp?

A6 has gone experimental, embrace .

See RFC 3363 & 3364.

-- 
Pekka Savola "You each name yourselves king, yet the
Netcore Oykingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Re: where the indirection layer belongs

[Pekka Nikander]

Dear Robert,

[...]  it looks to me that 
stable end-point identifiers are mainly needed to make applications 
survive IP address changes.  ...
There is more to stable end-point identifiers than just the ability to
make an application resilient.  I will argue that IP addresses *do not*
identify end-points, but node interfaces. 
I think we agree.  At least I think we agree pretty much on the role of
IP addresses.  I would even go further, and state that IP addresses
identify network interfaces, not node interfaces, as Dave Crocker
recently put it.
Personally, I do believe that we should have explicit names for
end-points, e.g., due to the reasons that you explained.  However,
I am not quite sure if everybody agrees with that.  Hence, what I
am merely trying to say is that we probably *could* continue to live
without explicit end-point identifiers, and still get the desired
mobility and multi-homing properties.
Even facing the danger of opening yet another rat hole, in my opinion
we should not have a very strict definition for end-point. That is,
IMHO end-point should and could be a fuzzy concept, somewhat like
the concept of a site is today.
From my point of view, an end-point may be a process, a group of 
processes, a host, or even a server cluster offering services as a 
unit.  To me, it looks like fate sharing and common semantics are the
key points here.  An end-point should either work or fail, it should
not be usual for half of an end-point fail while the other half is 
continuing.  An end-point should also be considered at the application 
level as a single unit.
The question that that raises is, whether it is reasonable to use such
an inclusive notion of an end-point, and what is the simplest kind of
structure or object we can use to implement an end-point identifier?  We
might have to restrict the notion of an end-point somewhat.
Well, as I tried to say, I'd prefer to have some functional and
semantic restrictions, and not just denotational restrictions.  That is,
if we say that an end-point must fail as a unit, that is a reasonable
restriction.  Likewise, if we say that an end-point must act as
if it had unique state, that is a good restriction, too.  That does
not preclude one from creating distributed end-points, as long as
each distributed end-point appears to have a unique state at each
point of time.
A related question is also the structure and semantics of the
end-point identifiers.  The simplest identifiers would be sufficiently
long random numbers, with a probability of collisions low enough.
However, I tend to believe in identifiers that have more structure,
and preferably even cryptographic semantics.
More on a new thread.

--Pekka Nikander

Re: You Might Be An Anti-Spam Kook If ...

[Zefram]

Vernon Schryver wrote:
>I've been compiling a list in the style of Jeff Foxworthy.
>
>You Might Be An Anti-Spam Kook If

Please publish this as an RFC.  A collection of unworkable approaches
to the spam problem (anti-spam anti-patterns) is useful knowledge that
should be preserved and promulgated to reduce the Anti-Spam Kook problem.

-zefram
-- 
Andrew Main (Zefram) <[EMAIL PROTECTED]>

Re: You Might Be An Anti-Spam Kook If ...

[Iljitsch van Beijnum]

On maandag, sep 8, 2003, at 11:14 Europe/Amsterdam, Zefram wrote:

Vernon Schryver wrote:
I've been compiling a list in the style of Jeff Foxworthy.

   You Might Be An Anti-Spam Kook If

Please publish this as an RFC.  A collection of unworkable approaches
to the spam problem (anti-spam anti-patterns) is useful knowledge that
should be preserved and promulgated to reduce the Anti-Spam Kook 
problem.
On the other hand there seems to be considerable interest into 
declaring the spam problem unsolvable. I don't think it's a good idea 
to lend credibility to this sentiment by publishing it as an RFC.

How hard is it to agree that:

a) there will always be (some) spam
b) there is no need for it to be 50% of all mail

Re: You Might Be An Anti-Spam Kook If ...

[Shelby Moore]

>You'd do well to observe and learn.

What specifically did you learn from Vernon in this thread?

Re: You Might Be An Anti-Spam Kook If ...

[Shelby Moore]

>Please publish this as an RFC.  A collection of unworkable approaches
>to the spam problem...

Imagine if we engineered all things in real life by first expending effort publishing 
all the ways not to engineer them.

That would be a good way to insure that nothing would ever get engineered.  It would 
*stifle* all or most production.

In mailing lists I have observed there are usually a few key guys who are sort of like 
the local bullies on the neighborhood street corner.  Their whole purpose in life is 
stifle any other boy from coming into the neighborhood and "stealing" (e.g. giving 
more opportunity to) the local girls.   Luckily they tend to not be very smart, else 
they wouldn't be wasting their time sitting around the local neighborhood.

Re: You Might Be An Anti-Spam Kook If ...

[Sandy Wills]

Shelby Moore wrote:
In mailing lists I have observed there are usually a few key guys
> who are sort of like the local bullies on the neighborhood street
> corner.  Their whole purpose in life is stifle any other boy from
> coming into the neighborhood and "stealing" (e.g. giving more
> opportunity to) the local girls.   Luckily they tend to not be
> very smart, else they wouldn't be wasting their time sitting around
> the local neighborhood.
   In SOME neighborhoods (I won't say "most"; I haven't SEEN "most"), 
there is a guy who handles trouble for everyone else.  Maybe he likes 
it, maybe he doesn't but does it anyway because he's good at it and it 
helps the neighborhood stay quiet and safe.  What you call him really 
depends upon your point of view.  Yes, troublemakers call him a bully. 
But, the quiet peaceful people who rely on him to keep their 
neighborhood quiet and peaceful while they go about their business think 
of him more as the policeman who happens to live on our block.

   Everyone has ideas.  Some good; some bad.  Please, bring your idea 
here, and we'll discuss it.  But, it's hard to have sympathy for your 
whining when you are told by _several_different_people_ that this idea 
has flaws.  Think them over, and improve your idea with the input you 
get from others who have been thinking about the same problem.  Bring 
back your improved idea, and we'll do the cycle again, until we either 
come up with a workable solution, or give up.

   I am usually just a lurker here, but this thread doesn't look like 
it is going anywhere useful, and it has lost it's entertainment value. 
In fact, while your posts can't really be considered spam, they are 
rapidly attaining the same value.  Please remember that anything you 
post to this discussion list will be archived, and available for the 
whole universe to see.
   Ask yourself "Will I be happy, 30 years from now, to know that my 
grandchildren are reading this post to find out what grandpa was like?"

--
: Unable to locate coffee.  Operator halted.
: Reply to [EMAIL PROTECTED]

Re: You Might Be An Anti-Spam Kook If ...

[Shelby Moore]

>> Vernon Schryver wrote:
>>> I've been compiling a list in the style of Jeff Foxworthy.
>>>
>>>You Might Be An Anti-Spam Kook If
>
>> Please publish this as an RFC.  A collection of unworkable approaches
>> to the spam problem (anti-spam anti-patterns) is useful knowledge that
>> should be preserved and promulgated to reduce the Anti-Spam Kook 
>> problem.
>
>On the other hand there seems to be considerable interest into 
>declaring the spam problem unsolvable.
> I don't think it's a good idea 
>to lend credibility to this sentiment by publishing it as an RFC.
>
>How hard is it to agree that:
>
>a) there will always be (some) spam
>b) there is no need for it to be 50% of all mail


I agree.


>On the other hand there seems to be considerable interest into 
>declaring the spam problem unsolvable.


Amazing that Vernon says working on spam is "Kook"y, yet that is exactly what he does 
(DCC).


Vernon has pretended he is an expert with this thread, with an obvious timing of 
condescending the serious anti-spam thread I started:

http://www1.ietf.org/mail-archive/ietf/Current/msg22035.html


Some disinteresting things Vernon emailed directly to me in past, which imho 
demonstrate he is not an expert and/or he qualifies for his own "Kook" test:

1. Vernon apparently got offended because I pointed out that he didn't realize that 
MD5 checksum on IPv4 was easily breakable via dictionary attack or that his use of it 
went his often public stated condescending policy of "do not implement half-solutions".

2. Vernon wrote, "...it is impossible, because no two people (or at least 
organizations) think the same streams of bulk mail are solicited and unsolicited.".  I 
can easily find 2 streams that 2 people/orgs can agree are solicited and unsolicited.  
If you give me 2 streams and 2 people/org at random, then I can not guarantee every 
time (but eventually and never impossible), but he said "no" which is always false and 
certainly never impossible.  That just shows you the kind of errors he makes with his 
logic.

3. Vernon wrote, "In the last couple of months, more than one person has invented a 
super duper wonderful perfect foolproof mechanism of using IP addresses collected by 
the DCC.".  "Forgive me, but I doubt your scheme differs significantly from the 
others.".  Isn't the desire to discover part of what makes a scientist an expert?  
When we stop discovering and learning, then we decay.

4. "A fundamental problem is that spam is unsolicited bulk mail, and not IP 
addresses...no mater how highly coorelated with spam.".  The first part is true, the 
second part is not because we don't live in 3 dimensions.  I will not tell you why 
beyond that, but I know why.

5. "The DCC is doing quite well, thank you very much, against the current plague, but 
the main event is to come.".  Wonder what he means by "main event is to come"?

6. "consider me a hopeless, useless, paranoid skeptic with an insurmountable 
not-invented-here syndrome, and stop sending me mail"


Shelby Moore
http://AntiViotic.com

Re: You Might Be An Anti-Spam Kook If ...

[Shelby Moore]

>  But, it's hard to have sympathy for your 
>whining


That is interesting interpretation of my winning a debate so completely that no one 
has been able to poke one logic hole in the serious anti- spam thread I started:

http://www1.ietf.org/mail-archive/ietf/Current/msg22035.html

Instead I think you must be referring to the person who started this silly thread.


> when you are told by _several_different_people_ that this idea 
>has flaws.

Which have all been resolutely refuted and defeated ad nauseum by me.


>I am usually just a lurker here, but this thread doesn't look like 
>it is going anywhere useful,


Makes you kind of wonder why Vernon posted a silly, condescending thread doesn't it.  
If you can't beat me with logical debate, then resort to other tactics...


> and it has lost it's entertainment value. 


That is an interesting interpretation of condescending.


>In fact, while your posts can't really be considered spam,


And yours, and Vernons and everybody else who got suckered in by the person who 
started this condescending thread which has no engineering value.

Shelby Moore
http://AntiViotic.com

POP3 delivers, not deletes

[Dan Kolis]

>Although it is theoretically possible, using POP (rather than IMAP) 
>to leave the mail on the server until you pull it again with POP, 
>many servers appear to clear out the mail after POPing it. 


The Client program users LIST and RETR to get the messages than DELE to
remove them. The server/cloud program(s) don't do that. If you/anyone want
to leave mesage a\on a POP3 MTA, selecting a client program (or writing one)
to not DELEte them at your discression is potenetially useful.
regs,
Dan

Re: You Might Be An Anti-Spam Kook If ...

[grenville armitage]

Shelby Moore rants (re Vernon):
[..]
> I've searched your posts all over the web via Google groups
> and you have a history of being obstinate, condescending, and impolite.

He's also sometimes funny and accurate. You'd do well to observe and learn.

gja

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Shelby Moore]

Main arguments made thus far and my retorts.

A1: Any one who tries to work on anti-spam is a "Kook".
R1: Illogical

A2: Too difficult for legitimate bulk senders to implement and support, and 
"especially" mailing lists.
R2: Many, if not most, mailing lists are already provided in "pull" www format.  Since 
they already implemented the complexity to pump incoming emails into www, it is 
trivial in comparison to cc: a copy to a POP account.  A simple .procmail script could 
be written in 5 minutes.

A3: Spammers will use mailing lists to send.
R3: They already do.  Receivers can opt out of mailing lists (before and after my 
proposal).  Without my proposal, they can opt-out of all bulk email, which includes 
spam and legitimate bulk email, and they can selectively whitelist to opt-in (I showed 
how this whitelisting can be spoofed by a spammer).  With my proposal, they can 
additionally opt-out out all spam == which is now all bulk email, and selectively 
opt-in (via "pull") to mailing lists (which is a concurrent operation with 
subscribing, not a separate subvertable whitelist).  With my proposal, receivers will 
get less spam, because their email address is only selectively opted-in, instead of 
open to all spammers.  Additionally, with the inherent "pull" delay, mailing lists can 
effectively remove spam from the "pull" server as it is discovered, well before most 
receivers "pull" it.  Additionally, mailing lists already have mechanisms in place to 
defeat incoming spam and will be developing more, given their advantage of requiring 
membership (subscription).  So all in all, under my proposal spammers will be less 
successful.

A4: Status quo is better risk.
R4: Timeline curves project spam will keep increasing and eventually most email will 
be spam.  Unless effective filters can be done and maintained with the status quo 
paradigm, you will eventually have 1 legitimate email for every 10s or 100s of spams.  
I know people who have this scenario already.  At that point, the legitimate bulk 
email isn't getting read whether it is filtered actively or inherently (burried in 
spam).  Without my proposal and with this eventual outcome, receivers will be forced 
to use "pull" for mailing lists (e.g. www archives) while their general email remains 
broke.  If you think current filters are effective, then why are so many receivers 
complaining about spam?  I know there are filters which work somewhat, but either 
their cost model can apparently not scale to all receivers (e.g. BrightMail because 
they use humans to help filter), or the filter can be subverted with varying content 
or whitelist mining (e.g. DCC), or the filter can be subverted by adapting Bayesian 
footprint, or the filter can be subverted by whitelist mining and filter causes big 
delays in legit email (e.g. email-back whitelisting verification), etc...  The point 
is legitimate bulk email is going to end "pull" whether by design or by failure of 
general email.  I prefer by design.

A5: Receivers will not "pull" legitimate bulk email
R5: They will if there is no other way to get their email.  See R4 above.  By design 
or by failure of general email, the problem is going to collapse either way into 
"pull"ing legitimate bulk email.  I prefer by design.

A6: POP is not ubiqutous enough.
R6: Use any and as many "pull" protocols as you want to serve your readers.  I'd say 
POP and www are sufficient, but use as many as your receivers want.  Let your market 
decide what you provide.

A7: Some people in this list are against because they have vested interest in status 
quo for mailing lists
R7: Without logical basis on the internet as a whole (this is IETF), that would not be 
a sound engineering basis to stifle an idea.

A8: Usenet already exists
R8: Use Usenet to provide pull if that is what your receivers want.  I think most 
would prefer to send and recieve in their email client, so I think POP would be more 
popular.

A9: Separating legitimate bulk email into "pull" will not cause additional enforcement 
against remaining spam == all bulk email
R9: There is going to be continuing increase in enforcement whether you implement my 
proposal or not.  Either active enforcement or inherent (lost in pile of spam).  There 
is a fundamental science to this.  It is called information theory and one of the 
theorems is that as you increase noise, then unless you can filter the noise, 
receiving original signal becomes less reliable.  The response rate of spam at < 1% 
(figure I've seen is usually < 0.003%) means it is noise.  Reading (receiving) of all 
email will get more and more unreliable, as spam increases.  Just imagine the 
bandwidth it will take to download it, store it, process it, not to mention try to 
filter and read it.  Spammers will get more and more savvy at subverting filters, as 
filters become more and more successful and popular.  Spammers will increase spam, the 
more that filtering increases.  It is a race to the saturation poin

Re: You Might Be An Anti-Spam Kook If ...

[grenville armitage]

Shelby Moore wrote:
[..]
> Their whole purpose in life is stifle any other boy
> from coming into the neighborhood and "stealing" (e.g. giving
> more opportunity to) the local girls.

Ah. The ever popular "IETFers-are-jealous-because-I'm-sexy" argument.

I think we just need someone to mention Hitler and this thread is done.

Oops.

gja

Re: POP3 delivers, not deletes

[Shelby Moore]

At 09:18 AM 9/8/2003 -0400, you wrote:
>>Although it is theoretically possible, using POP (rather than IMAP) 
>>to leave the mail on the server until you pull it again with POP, 
>>many servers appear to clear out the mail after POPing it. 
>
>
>The Client program users LIST and RETR to get the messages than DELE to
>remove them. The server/cloud program(s) don't do that. If you/anyone want
>to leave mesage a\on a POP3 MTA, selecting a client program (or writing one)
>to not DELEte them at your discression is potenetially useful.
>regs,
>Dan

I am already doing it here:

http://MailThentic.com/test.php

See the "Test Only (don't delete)" flag at bottom.

Shelby Moore
http://AntiViotic.com (aka http://MailThentic.com)

Re: You Might Be An Anti-Spam Kook If ...

[Shelby Moore]

>Ah. The ever popular "IETFers-are-jealous-because-I'm-sexy" argument.


No, the argyment to stay focused on the serious engineering debate, such as in link 
below.


> when you are told by _several_different_people_ that this idea 
>has flaws.


Which have all been resolutely refuted and defeated ad nauseum by me.

And to make that clear, I have written a summary for you:

http://www1.ietf.org/mail-archive/ietf/Current/msg22112.html


Shelby Moore
http://AntiViotic.com 

Re: You Might Be An Anti-Spam Kook If ...

[Vernon Schryver]

> From: Shelby Moore <[EMAIL PROTECTED]>

> ...
> 1. Vernon apparently got offended because I pointed out that he
> didn't realize that MD5 checksum on IPv4 was easily breakable via
> dictionary attack or that his use of it went his often public stated
> condescending policy of "do not implement half-solutions".

Perhaps Mr. Moore should recall my claim to archive mail.  
He wrote this:

< 3. Most importantly, is there any reasonable way to extract the
< original IP from the checksum?  I suppose the IPv4 address space
< is 4 billion.  How long does it take to run 4 billion MD5 hashs?
< If impractical, I might be able to work with your checksums in my
< database instead of storing IP addresses (might not be a such a
< bad idea for privacy reasons).  However, why did you use checksum
< on IP any way (seems to me a hacker can get the original IP using
< a brute force attack)?

I responded:

] The DCC databases contain only MD5 hashes.  If you know of a way to  
] reverse MD5 hashes other than a dictionary attack, you should publish
] it and get famous.  I'm not sure, but you may be agreeing with that
] throught.
] 
] When the DCC databases contained MD5 hashes of IP addresses, they were
] of IPv6 addresses.  Of course, those IPv6 addresses were related in
] the standard way to IPv4 addresses.  I've not timed MD5 on 128 bit
] values, but guess 100 usec/hash.  If that's right, you could build a
] 16 GByte dictionary in about 100 hours.

(I made an arithmetic error in the figuring the size of the dictionary.)


Mr. Moore came back with:

} http://www.faqs.org/rfcs/rfc1810.html
} 
} 1995 RFC claims 87 Mbps rate for MD5 in software.  Assuming Moore's
} Law (double speed every 18 months), then we get 9 years (6 x 18
} months) to 2004, thus 6*87Mbps in 2004.
} 
} 32 bit = 2 ^ 32 = 4 billion / 6*87 millions = 24 seconds.
} 
} So if you had 1% of that space, or 40 million IPs in your databases
} over time, then would take approx. 20 million minutes = 333,000
} hours = 15,000 days < 50 years to convert all MD5 back to IPv4s.
} 
} However an inverse table could be built if we had 4 GB * 128 bit
} of storage = 4 * 16 GB = 64 GB.  This would drop the time to probably < month.
} 
} Assuming I am interpreting the RFC correctly.
} 
} Note I read some where that 2 ^ 64 search space is required before
} hitting the duplicate space of MD5.

and later:

| >When the DCC databases contained MD5 hashes of IP addresses, they were
| >of IPv6 addresses.  Of course, those IPv6 addresses were related in   
| >the standard way to IPv4 addresses.  I've not timed MD5 on 128 bit
| >values, but guess 100 usec/hash.  If that's right, you could build a
| >16 GByte dictionary in about 100 hours.
| 
| Correct 16 GB, not the 64 GB I mistakenly wrote late last night.

Perhaps Mr. Moore's 16 GBytes comes from limiting the dictionary to
1 billion interesting IPv4 addresses.  Otherwise 64 GBytes is better.

I never did figure out what Mr. Moore meant by 15,000 days.  He
could not have been thinking of doing on average 2 billion MD5 hashes
for each of 4 billion IPv4 addresses, because that would have been
silly and would have take more than 15,000 days.


Ok, I'll stop feeding the troll now.


Vernon Schryver[EMAIL PROTECTED]

Re: You Might Be An Anti-Spam Kook If ...

[Randy Bush]

> Ok, I'll stop feeding the troll now.

troll?  what troll?  procmail is your friend.

randy

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Dean Anderson]

On Sun, 7 Sep 2003, Iljitsch van Beijnum wrote:

> On zondag, sep 7, 2003, at 21:45 Europe/Amsterdam, Dean Anderson wrote:
>
> > Information theory says that such things are impossible.  One can not
> > construct a spam-free protocol because this is the same problem as
> > constructing a system free of covert channels, which information theory
> > says is impossible.
>
> Nobody cares. Making a roof 100.00% impervious to water molecules
> may be impossible, but that doesn't mean we have to resign to getting
> wet every time it rains.

People care because when someone comes around saying "you can have a 100%
impervious roof if only you jump through these inconvenient hoops", we
know that they are wrong, and don't need to waste time considering how
inconvenient the hoops are.

"We", meaning the IETF, care, because this is very useful aid to deciding
what to work on. We know that we need to focus on leak stoppage, not
trying to invent leak-proof protocols.  There is no point researching
something that is impossible.

> > It is not simply hard. It is impossible, like perpetual motion.
>
> So when exactly was the earth supposed to stop moving?

God didn't make the earth move perpetually. He just made it move long
enough.  It seems that even God can't solve some problems.

We didn't get to the moon by inventing perpetual motion machines, though
early proposals were based on such machines.  We got to the moon by
working on the messy physics of rockets.

When someone comes to the NSF and says you can have a perpetual motion
machine if only you jump through some very inconvenient hoops, and spend a
lot of money, the NSF can save itself the time and money by discarding
perpetual motion schemes from its research program.  Similarly,
information theory allows us to discard some ideas from our research
programs.  That is why we care.

> > After I first posted this on IETF a while back, someone suggested that
> > covert channels require cooperation, and that spam therefore isn't a
> > covert channel.
>
> Where does this covert channel stuff come from anyway?

What do you mean?

> > But this is a simpler way to think about it:  Spammers can continue to
> > claim they are legitimate emailers, because they _ARE_ legitimate, so
> > far as we know before they send email. And even so far as we know
> > _before_ someone _READS_ their email.  Only after reading their email,
> > and perhaps only after some investigation, can we know for sure that
> > the sender and message is conducting abuse or in violation of their
> > AUP.
>
> This goes for each individual message, but the spammer's achilles heel
> is that they need to send out incredible amounts of email in order to
> fulfill their objectives, whichever those are. Detecting bulk mail is
> doable, and it shouldn't be too hard to come up with something to
> differentiate legitimate bulk emailing from spam. For instance, we can
> reverse the burden of proof here and only allow know bulk emailers.

"Detecting abuse" is quite different from making a protocol that can't be
abused.  But that is my point: You have to focus on detection. This
doesn't require any protocol changes whatsover.

We are already "only allowing known bulk emailers". Unfortunately, that
doesn't prevent spam.  Indeed, it seems most of the spam isn't commercial:
Most of the spam seems to come from viruses, and isn't really selling
anything.  The viruses can use the credentials of the infected user.
That is "legitimate", until someone reading the email realizes its not and
complains. These send 40-50 messages per IP, and is hard to detect as
bulk. But when added up over a lot of IP addresses, is quite obviously
annoying.

> > It is not immune to spam, though it distributes spam and other
> > broadcast messages much more efficiently than typical email systems.
>
> Ouch! :-)
>
> Fixable with authentication.

No, that's the point. It isn't _fixable_ with authentication.  It isn't
fixable at all.  It is only "fixed" when the spammer loses his account.
Then the spammer gets a new account.  So it isn't really fixed.  So we are
always going to be playing a game of whack-a-mole.  That cannot be avoided
by altering the protocol or the authentication scheme (information theory
proves this). So it is useful, then, to work on ways of detection, and
improve our whack-a-mole skills.  Altering protocols and authentication is
a waste of time.

--Dean

Re: You Might Be An Anti-Spam Kook If ...

[Valdis . Kletnieks]

On Mon, 08 Sep 2003 18:38:03 +0800, Shelby Moore said:

> Imagine if we engineered all things in real life by first expending effort
> publishing all the ways not to engineer them.

Umm.. We don't.  The first documented spam was sent over 25 years ago,
and you tuned in late.

How to design a bridge:

1) Build one out of dirt.  Realize that's a dam not a bridge, when it washes
away.  Make a note of it.

2) Build one out of rocks.  Find out the hard way that it works better if you
use mortar too.  Make a note of it.

3) Build one out of rocks and mortar.  Find out the hard way that setting the
piers on mud causes washouts.  Make a note of it.

4) Build one out of rocks and mortar, and set the piers on good solid rock.
Find out the hard way (multiple times) about how thick it has to be to support
a given load.  Make a note of it.

5) Discover the hard way about resonance effects.  Post a sign for the
marching troops and make a note of it.

6) Bundle up your notes and publish the Civil Engineering Handbook, First Edition.

7) Discover that wood makes OK bridges too, make lots of notes about strength,
insect prevention, and the like.  Publish the Second Edition.

8) Discover metal.  Make notes about fatigue and rust.  Publish Third Edition.

We can't help you if you think that we sat down and wrote the Third Edition
before doing anything else.  And we *really* can't help you if we hand you a
copy of the Third Edition, and you insist on trying to build a bridge in a way
that we pointed out on page 37 *just won't work*.  

On Mon, 08 Sep 2003 19:25:22 +0800, Shelby Moore said:
> Amazing that Vernon says working on spam is "Kook"y, yet that is exactly what he 
> does (DCC).

No.. What he *said* was "totally disregarding the things others have already tried is 
kooky".
Which line item from his note is the DCC in violation of?  (In particular, note that 
Vernon
has never claimed that the DCC is either perfect or a complete solution - it's a 
work-in-progress
of one tool for one aspect of the problem).

> 2. Vernon wrote, "...it is impossible, because no two people (or at least
> organizations) think the same streams of bulk mail are solicited and
> unsolicited.".  I can easily find 2 streams that 2 people/orgs can agree are
> solicited and unsolicited.  If you give me 2 streams and 2 people/org at
> random, then I can not guarantee every time (but eventually and never

Right. You can't guarantee *every time* for 2 people at random.  That's his point 
exactly.  

Any scheme that requires any 2 random people to come up with the same answer on
this judgment call *will* break with either a false positive or false negative for each
object in the stream there's a disagreement on.

> 4. "A fundamental problem is that spam is unsolicited bulk mail, and not IP
> addresses...no mater how highly coorelated with spam.".  The first part is
> true, the second part is not because we don't live in 3 dimensions.  I will not
> tell you why beyond that, but I know why.

Actually, 3 dimensions has absolutely nothing to do with it.  And the fact that you
think it does indicates that you really don't understand the difference there.

The real problem is that there exists a non-empty subset of the IP address
space for which the predicate '((sends spam) XOR (sends non-spam)) ' is zero,
even for an actual SMTP source.



pgp0.pgp
Description: PGP signature

Re: POP3 delivers, not deletes

[Harald Tveit Alvestrand]

--On 8. september 2003 09:18 -0400 Dan Kolis <[EMAIL PROTECTED]> 
wrote:

Although it is theoretically possible, using POP (rather than IMAP)
to leave the mail on the server until you pull it again with POP,
many servers appear to clear out the mail after POPing it.


The Client program users LIST and RETR to get the messages than DELE to
remove them. The server/cloud program(s) don't do that. If you/anyone want
to leave mesage a\on a POP3 MTA, selecting a client program (or writing
one) to not DELEte them at your discression is potenetially useful.
A *lot* of POP-using programs have the "Leave Mail On Server" option.
And a lot of people have used "Leave Mail On Server" as a poor man's 
1-folder IMAP, leading POP providers to implement mail retaining policies 
of the "RETR it once and it's gone, whether you DELEted it or not".

This is shown up in RFC 1939 (current definition of POP3) section 8:


   .In these situations and others, users and
   vendors of POP3 clients have discovered that the combination of using
   the UIDL command and not issuing the DELE command can provide a weak
   version of the "maildrop as semi-permanent repository" functionality
   normally associated with IMAP.
...and in response, server operators are recommended to:

   *  Enforce a site policy regarding mail retention on the server.

  Sites are free to establish local policy regarding the storage and
  retention of messages on the server, both read and unread.  For
  example, a site might delete unread messages from the server after
  60 days and delete read messages after 7 days.  Such message
  deletions are outside the scope of the POP3 protocol and are not
  considered a protocol violation.
Just to show that we've "been there, done that, code in the field".

  Harald

Re: POP3 delivers, not deletes

[John C Klensin]

--On Monday, September 08, 2003 09:18 -0400 Dan Kolis
<[EMAIL PROTECTED]> wrote:

>> Although it is theoretically possible, using POP (rather than
>> IMAP)  to leave the mail on the server until you pull it
>> again with POP,  many servers appear to clear out the mail
>> after POPing it. 
> 
> 
> The Client program users LIST and RETR to get the messages
> than DELE to remove them. The server/cloud program(s) don't do
> that. If you/anyone want to leave mesage a\on a POP3 MTA,
> selecting a client program (or writing one) to not DELEte them
> at your discression is potenetially useful. regs,
> Dan

Dan,

I am pretty sure Vint knows what the protocol says.  So,
certainly, do I.   

In the real world, several ISPs have insisted that their servers
provide an implicit DELE after messages have been successfully
downloaded and the connection closed.  If leaving the mail on
the server (not DELEting them until you tell it to) is
important, you could, of course, choose an ISP that doesn't do
that automatic/ implicit delete.  But, because there are ISPs
that do the automatic delete, Shelby's claim (as I understand
it) that his system will work with any POP3 mailbox and server
is not quite correct.

 john

Re: You Might Be An Anti-Spam Kook If ...

[Randy Bush]

> 7) Discover that wood makes OK bridges too, make lots of notes about
> strength, insect prevention, and the like.  Publish the Second Edition.
> 
> 8) Discover metal.  Make notes about fatigue and rust.  Publish Third
> Edition.

many years ago, i saw, in the uk midlands, a (large outdoor)
museum which had the first metal bridge.  it was built as if it
had the same characteristics as wood, i.e., big dimensional iron
bars the size/shape of the wood beams.  later, they discovered the
ibeam.

randy

Re: your mail

[Paul Vixie]

> > have we completely deprecated 's for A6's or is  still
> > considered bcp?
> 
> A6 has gone experimental, embrace .
> 
> See RFC 3363 & 3364.

A6 and bitstring labels are dead.  Only DNAME survived.  See all extant
implementations of the DNS protocols.
-- 
Paul Vixie

POP3 delivers, not deletes III

[Dan Kolis]

Harold I / Dan K said:
>A *lot* of POP-using programs have the "Leave Mail On Server" option.
>And a lot of people have used "Leave Mail On Server" as a poor man's 
>1-folder IMAP, leading POP providers to implement mail retaining policies 
>of the "RETR it once and it's gone, whether you DELEted it or not".
>
>This is shown up in RFC 1939 (current definition of POP3) section 8:
>>.In these situations and others, users and
>>vendors of POP3 clients have discovered that the combination of using
>>the UIDL command and not issuing the DELE command can provide a weak
>>version of the "maildrop as semi-permanent repository" functionality
>>normally associated with IMAP.
>...and in response, server operators are recommended to:
>>*  Enforce a site policy regarding mail retention on the server.
>>   Sites are free to establish local policy regarding the storage and
>>   retention of messages on the server, both read and unread.  For
>>   example, a site might delete unread messages from the server after
>>   60 days and delete read messages after 7 days.  Such message
>>   deletions are outside the scope of the POP3 protocol and are not
>>   considered a protocol violation.

Dan says:
Well, yes I guess it "their" server (somebody's). There are a few things obviously 
desireble in POPX thet aren't in there. (deliver without Mime attachments as a 
preview, for 
instance).

Seems like IMAP is kind of too much, and Pop3 is too little for a lot of users.

I wouldn't like it if the server did this. I'd rather have a fixed limit in size, and 
a 
warning via email when its almost full, and have it reject messages beyond that size. 
But 
that's me. I guess leaving it to the site is just a part of reality.

A tiny extention to allow "push" email just broadcasting the subject lines; (possible 
encrypted) and headers generally would be cool. Like blackberry's protocol but not 
proprietary.

In any case I think pretty soon a total rethink of email is in order... re 
authentication/encryption/spam. but it's gotta be compatible and this will be tricky, 
to 
say the least.
regs
Dan

Re: You Might Be An Anti-Spam Kook If ...

[Shelby Moore]

>> Imagine if we engineered all things in real life by first expending effort
>> publishing all the ways not to engineer them.
>
>Umm.. We don't.


I believe that when I see more posts in real engineering threads than I see posts in 
this thread.


>On Mon, 08 Sep 2003 19:25:22 +0800, Shelby Moore said:
>> Amazing that Vernon says working on spam is "Kook"y, yet that is exactly 
>what he does (DCC).
>
>No..


Yes.  For one thing Vernon doesn't understand the concept of dynamic set theory and 
entropy as it applies to mapping IPs to spam.  That statement he made convinced me 
neither he or you have fully explored all the possibilities.

Neither of you still haven't sent any links to prior art to prove your assertion that 
I am rehashing.  You are all hot air and no facts.


> What he *said* was "totally disregarding the things others have already 
>tried is kooky".


The problem is he thinks he knows what other people are proposing but he doesn't 
understand it well enough to know it is different that what has been tried already.  
And I doubt either of you have any clue about the mathematical ways it is different.  
So before you write a master set of rules, maybe you should listen and learn a little 
more.


>Which line item from his note is the DCC in violation of?


Not DCC, but him as creator of DCC.  The very hippocritical fact that he publishes 
such a document that is in violation of the core philosophy of the document, which is 
in essense "don't think you know everything, unless you do, and you probably don't".

Especially the last one he added, which he got from one Keith Moore emailed about me:

"you're certain none of the preceding apply to you or you see nothing wrong in some of 
the"

He is either "certain" or is saying he is a Kook.  He fell right into his own trap.



>> 2. Vernon wrote, "...it is impossible, because no two people (or at least
>> organizations) think the same streams of bulk mail are solicited and
>> unsolicited.".  I can easily find 2 streams that 2 people/orgs can agree are
>> solicited and unsolicited.  If you give me 2 streams and 2 people/org at
>> random, then I can not guarantee every time (but eventually and never
>
>Right. You can't guarantee *every time* for 2 people at random.  That's his 
>point exactly.  


He did not write "every time".  He wrote "no".  "no" means none or empty set.  Learn 
about set theory.

Also you missed the deeper point, but that is to be expected...


>Any scheme that requires any 2 random people to come up with the same answer on
>this judgment call *will* break with either a false positive or false 
>negative for each
>object in the stream there's a disagreement on.


Again you obviously don't have the mathematical background or don't know how to apply 
it.

I am sure that many people said that nuclear bomb could not be created, that the earth 
was flat, etc...



>> 4. "A fundamental problem is that spam is unsolicited bulk mail, and not IP
>> addresses...no mater how highly coorelated with spam.".  The first part is
>> true, the second part is not because we don't live in 3 dimensions.  I 
>will not
>> tell you why beyond that, but I know why.
>
>Actually, 3 dimensions has absolutely nothing to do with it.


You are way off.  You did not even realize I was talking about the lack of time 
dimension, so you very likely still have no clue what I really mean.



>  And the fact 
>that you
>think it does indicates that you really don't understand the difference there.


I doubt you understand how entropy applies to spam.  And I doubt any of your prior art 
has ever looked it from that perspective.


>The real problem is that there exists a non-empty subset of the IP address
>space for which the predicate '((sends spam) XOR (sends non-spam)) ' is zero,
>even for an actual SMTP source.


You are still stuck in binary logic.  You seem to be very far behind where my thought 
process is on spam.


Shelby Moore
http://AntiViotic.com

POP3 prograsm that enforce old message policies

[Dan Kolis]

John K said:
>I am pretty sure Vint knows what the protocol says.  So,
>certainly, do I.   
>In the real world, several ISPs have insisted that their servers
>provide an implicit DELE after messages have been successfully
>downloaded and the connection closed.  If leaving the mail on
>the server (not DELEting them until you tell it to) is
>important, you could, of course, choose an ISP that doesn't do
>that automatic/ implicit delete.  But, because there are ISPs
>that do the automatic delete, Shelby's claim (as I understand
>it) that his system will work with any POP3 mailbox and server
>is not quite correct.
> john

Dan says:
Another way to do this is via EHLO and then your client would have to
subscribe to the "feature" of some timed self delete or you would be denied
access totally. This would make sure the user is given a heads up on the
whole thing.

This would be super clean. Your program (in your national language) warns
you the server is going to enforce some message removal method.

With the excellent thinking in the development of these... I'm surprised
that's not how it works.

regs,
Dan

Re: POP3 prograsm that enforce old message policies

[John C Klensin]

--On Monday, September 08, 2003 13:08 -0400 Dan Kolis
<[EMAIL PROTECTED]> wrote:

> John K said:
>> I am pretty sure Vint knows what the protocol says.  So,
>> certainly, do I.   
>> In the real world, several ISPs have insisted that their
>> servers provide an implicit DELE after messages have been
>> successfully downloaded and the connection closed.  If
>> leaving the mail on the server (not DELEting them until you
>> tell it to) is important, you could, of course, choose an ISP
>> that doesn't do that automatic/ implicit delete.  But,
>> because there are ISPs that do the automatic delete, Shelby's
>> claim (as I understand it) that his system will work with any
>> POP3 mailbox and server is not quite correct.
>> john
> 
> Dan says:
> Another way to do this is via EHLO and then your client would
> have to subscribe to the "feature" of some timed self delete
> or you would be denied access totally. This would make sure
> the user is given a heads up on the whole thing.

EHLO?  On POP3???

> This would be super clean. Your program (in your national
> language) warns you the server is going to enforce some
> message removal method.
> 
> With the excellent thinking in the development of these... I'm
> surprised that's not how it works.

One could certainly design a more extensive capabilities
mechanism for POP3, but that raises some other issues?

john

Re: You Might Be An Anti-Spam Kook If ...

[Shelby Moore]

>> 1. Vernon apparently got offended because I pointed out that he
>> didn't realize that MD5 checksum on IPv4 was easily breakable via
>> dictionary attack or that his use of it went his often public stated
>> condescending policy of "do not implement half-solutions".


And the applicable "Kook" rule from Mr. Schryver's bible that started this thread is:

http://www1.ietf.org/mail-archive/ietf/Current/msg22092.html

"you are deeply offended when people do not agree that you have found the UFPSTTSP"


Which Mr. Schryver conveniently removed from his web site version (wonder why?):

http://www.rhyolite.com/anti-spam/you-might-be.html



>Perhaps Mr. Moore should recall my claim to archive mail.  
>He wrote this:


Perhaps Mr. Schryver should recall my claim to archive mail.

The point I made in #1 above is that Mr. Schryver had apparently never considered the 
question until I asked it.  And then when I pointed out to him he got very offended, 
which violates one of his own rules for being a "Kook":


I wrote:
>> Yes that is why I wondered why you bothered to hash the IP at all.
>> I think this is case where you voilated your own principle of not
>> implementing "half solutions".  There is no security or privacy benefit
>> to hashing the IPs (because a brute force dictionary attack can be
>> built in 100 hours) and there would be benefits to not hashing them
>> such as my algorithm.  Now I have to carry around a 16 GB dictionary
>> (lookup table).


He responded:
>
>I doubt you have figured out how we thought the IP address might be used.


I responded:
What you thought is irrelevant to facts at hand.  The fact is the IPs can be 
reasonably extracted thus logically it is reasonably useless to hash them.



>>} So if you had 1% of that space, or 40 million IPs in your databases
>>} over time, then would take approx. 20 million minutes = 333,000
>>} hours = 15,000 days < 50 years to convert all MD5 back to IPv4s.
>
>I never did figure out what Mr. Moore meant by 15,000 days.  He
>could not have been thinking of doing on average 2 billion MD5 hashes


Can you read?  I wrote "1% of the space".


>for each of 4 billion IPv4 addresses, because that would have been
>silly and would have take more than 15,000 days.
>
>
>Ok, I'll stop feeding the troll now.


I will stop feeding the troll by posting in this thread now.  I predict he or his 
buddies won't stop posting in this thread.

Shelby Moore
http://AntiViotic.com

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Shelby Moore]

>> > Information theory says that such things are impossible.  One can not
>> > construct a spam-free protocol because this is the same problem as
>> > constructing a system free of covert channels, which information theory
>> > says is impossible.


But information theory also says you can optimize signal-to-noise ratio, but only if 
you know what the characteristics of your signal are.

Thus my whole motivation for an unambiguous definition (spam == all bulk email) along 
the channel and not just a definition at the end points (UBE).


>> Nobody cares. Making a roof 100.00% impervious to water molecules
>> may be impossible, but that doesn't mean we have to resign to getting
>> wet every time it rains.
>
>People care because when someone comes around saying "you can have a 100%
>impervious roof if only you jump through these inconvenient hoops",


Who said 100%?

Is the problem already reduced to an acceptable S/N ratio for the majority?

Is the current S/N ratio a problem for the majority?


> we
>know that they are wrong,


How can you know "they are wrong", even you did not even realize that no one was 
proposing a 100% solution in this thread??


>"We", meaning the IETF, care, because this is very useful aid to deciding
>what to work on.


You decline work without even understanding what is being written about a new idea?  
No one ever proposed 100% solution.  I challenge you to find one post where I wrote 
that.



> We know that we need to focus on leak stoppage, not
>trying to invent leak-proof protocols.


I proposed an way to improve leak stoppage, by defining the signal in the channel and 
not only at end points.  I never proposed a leak-proof protocol.  Underlying in what 
you are saying is you don't support new protocols, because you have a vested interest 
in  existing methods of detection.


>We didn't get to the moon by inventing perpetual motion machines,


And many people said it was impossible for us to get to the moon.


> though
>early proposals were based on such machines.  We got to the moon by
>working on the messy physics of rockets.


What about the messy information theory of defining the spam signal for the channel 
and not just the end points, so that it can be studied and research in the channel and 
not only in the receiver's mind

Have you not realized yet that profound point I am making???


>> > After I first posted this on IETF a while back, someone suggested that
>> > covert channels require cooperation, and that spam therefore isn't a
>> > covert channel.
>>
>> Where does this covert channel stuff come from anyway?
>
>What do you mean?


The use of covert here probably meant that making anything 100% covert is impossible 
in information theory.


>> > But this is a simpler way to think about it:  Spammers can continue to
>> > claim they are legitimate emailers, because they _ARE_ legitimate, so
>> > far as we know before they send email. And even so far as we know
>> > _before_ someone _READS_ their email.  Only after reading their email,
>> > and perhaps only after some investigation, can we know for sure that
>> > the sender and message is conducting abuse or in violation of their
>> > AUP.


You exactly stated the problem.  We have not defined spam (the signal we need to 
sample) in terms of the channel.  If you define it as ALL BULK EMAIL, then you can 
actually start some messy science of getting to the moon on spam issue...



>> This goes for each individual message, but the spammer's achilles heel
>> is that they need to send out incredible amounts of email in order to
>> fulfill their objectives, whichever those are. Detecting bulk mail is
>> doable, and it shouldn't be too hard to come up with something to
>> differentiate legitimate bulk emailing from spam. For instance, we can
>> reverse the burden of proof here and only allow know bulk emailers.
>
>"Detecting abuse" is quite different from making a protocol that can't be
>abused.  But that is my point: You have to focus on detection. This
>doesn't require any protocol changes whatsover.


You can not measure a signal if you have not defined it.  That is a fundamental 
concept of information theory.  Spam is ambigous in the channel, unless you define 
spam == all bulk email.  How many dozens of times have I written that in this thread!


>We are already "only allowing known bulk emailers". Unfortunately, that
>doesn't prevent spam.


SOBO (statement of blatantly obvious) that you can't filter something if you can't 
define it.  That information theory 101.


>  Indeed, it seems most of the spam isn't commercial:
>Most of the spam seems to come from viruses, and isn't really selling
>anything.  The viruses can use the credentials of the infected user.
>That is "legitimate", until someone reading the email realizes its not and
>complains. These send 40-50 messages per IP, and is hard to detect as
>bulk.


Viruses are a different signal and can be filtered as such, unless they contain no 
viral da

Re: where the indirection layer belongs

[Robert Honore]

Dear Masataka Ohta,

 I was a bit impetuous in saying that I would prefer not to modify 
libraries or implementations etc.  However, my aim so far is more to 
obtain for myself a clear understanding of the problem we are trying to 
solve, rather than trying to state requirements of the possible 
solutions to that problem.  My apologies for such a stupid statement.

I do believe that there are issues to be resolved and they are being 
currently reflected in threads like the one about deprecating site-local 
addresses and the need for provider independent addresses that is 
currently active, as well as the thread "solving the real problem" 
started by Tony Hain.  I believe the issues being discussed there are 
real.  You might not agree that they are.

If the issues are real, then we need to specify or formulate them in a 
way that is amenable to deriving solutions for them.  If the issues are 
not real, then they are likely the artifacts of wrong perceptions or 
ideas that users and engineers in the IPv6 community have.  In that 
case, we need to replace those wrong perceptions or ideas with correct 
ones.  It seems to me though, that nobody has stated clearly what those 
wrong perceptions and ideas are, much less to say what is wrong with 
them and thus replace them with correct perceptions and ideas.

Yours sincerely,
Robert Honore.
Masataka Ohta wrote:
Robert Honore;


I would also prefer not to modify any of the 
libraries or implementations of those protocols, lest we break something.


It is a wrong requirement wrong in various ways.

First of all, we must change something, which means we must change
some code.
Though the changes should better be simple, whether the changes can
be inserted as a module or not is irrelevant,
To make the changes simpler, the modification MUST be confined in
libraries or kernel. So, we should try to change libraries or kernel
to avoid changes on application code.
It, of course, is necessary to change application code for UDP
cases for address selection that it is necessary to an API
to control address selection in IP headers from applications
that IP and transport layer code MUST be modified .
Secondly, preservation of code is less important to preservation
of protocol and protocol is defined on the wire.
If you put some code below transport layer to modify packet content
visible from the transport layer, which is the case with MAST, you
change the transport layer protocol.
But, again, we must change something of the protocol.

So, the requirements are:

	to modify existing code

	to modify existing protocol

and any attempt, including but not limited to MAST, to avoid them
is not constructive only to constraing the solution space to
result in less capable and less efficient solutions.
Of course, it is desirable

	to preserve existing application code

whenever possible. But, we must be aware that it is NOT always
possible.
In addition, it is desirable

	to preserve interoperability with existing protocols

which automatically means "to preserve interoperability with existing
code of the peer". In this case, it is doable. Of course, all the
freedom to use multiple addresses is lost when the peer is a
leagacy one.
			Masataka Ohta

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Shelby Moore]

>  Indeed, it seems most of the spam isn't commercial:
>Most of the spam seems to come from viruses, and isn't really selling
>anything.  The viruses can use the credentials of the infected user.
>That is "legitimate", until someone reading the email realizes its not and
>complains. These send 40-50 messages per IP, and is hard to detect as
>bulk.


This is pseudo-off topic because I already stated below that a viral signal can be 
detected differently than a spam signal, unless it contains no viral data (which would 
be pointless afaik).  I am curious about your data.  Are you refering to emails 
spreading a virus that contain viral attachments??

It occurs to me that a virus can not spread very fast or effectively if each infected 
computer only sends 50 emails, because the infection rate is probably similar to spam, 
i.e. < 0.005%.  So you would only get 1 new infection for each 20,000 emails sent, or 
thus for each 400 infected computers.  It seems the virus would likely die (anti-virus 
actions) at that rate of spread.  So I must assume you were looking at a very small 
sample on internet email and you did not extrapolate???

Your answers might be somewhat helpful to me in my work.

Thanks,
Shelby Moore
http://AntiViotic.com

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Valdis . Kletnieks]

On Tue, 09 Sep 2003 02:44:21 +0800, Shelby Moore said:

> It occurs to me that a virus can not spread very fast or effectively if each
> infected computer only sends 50 emails, because the infection rate is probably
> similar to spam, i.e. < 0.005%.  So you would only get 1 new infection for each
> 20,000 emails sent, or thus for each 400 infected computers.  It seems the
> virus would likely die (anti-virus actions) at that rate of spread.

W32/Sobig-F.


pgp0.pgp
Description: PGP signature

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Dean Anderson]

I am not talking about email spreading virues. A number of viruses appear
to send spam. (not spreading). Sometimes this is autonymous. Sometime it
is under control via IRC channel back to the virus operator. Further, it
seems that many open proxies are installed by virus.  Once the virus has
control of the computer, it has or will obtain keys to private keychains,
etc.  It can do whatever the infected users can do.

The number of 40-50 emails per IP figure comes from analysis of spam
messages that get by filters, by reviewing how many messages came from the
same source. A lot of spam that gets by filters is of this very low volume
type.

--Dean

On Tue, 9 Sep 2003, Shelby Moore wrote:

> >  Indeed, it seems most of the spam isn't commercial:
> >Most of the spam seems to come from viruses, and isn't really selling
> >anything.  The viruses can use the credentials of the infected user.
> >That is "legitimate", until someone reading the email realizes its not and
> >complains. These send 40-50 messages per IP, and is hard to detect as
> >bulk.
>
>
> This is pseudo-off topic because I already stated below that a viral
> signal can be detected differently than a spam signal, unless it
> contains no viral data (which would be pointless afaik).  I am curious
> about your data.  Are you refering to emails spreading a virus that
> contain viral attachments??
>
> It occurs to me that a virus can not spread very fast or effectively if
> each infected computer only sends 50 emails, because the infection rate
> is probably similar to spam, i.e. < 0.005%.  So you would only get 1 new
> infection for each 20,000 emails sent, or thus for each 400 infected
> computers.  It seems the virus would likely die (anti-virus actions) at
> that rate of spread.  So I must assume you were looking at a very small
> sample on internet email and you did not extrapolate???
>
> Your answers might be somewhat helpful to me in my work.
>
> Thanks,
> Shelby Moore
> http://AntiViotic.com
>
>
>

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Dean Anderson]

On Tue, 9 Sep 2003, Shelby Moore wrote:
>
> >> > Information theory says that such things are impossible.  One can not
> >> > construct a spam-free protocol because this is the same problem as
> >> > constructing a system free of covert channels, which information theory
> >> > says is impossible.
>
>
> But information theory also says you can optimize signal-to-noise ratio,
> but only if you know what the characteristics of your signal are.

It actually doesn't say that precisely. It says that you can transmit a
signal with an arbitrarilly low error rate at a speed below the channel
capacity.

The concrete task of altering the signal to noise ratio is accomplished by
enhancing the signal with a harmonic oscillator, so that it is stronger
than the noise.  This is then described as a set of differential equations
that can be optimized with Variational methods.  The limits of this
process are indicated by information theory, the nyquist theorem, etc.
If the channel isn't described by a fourier series, then the differential
equations may not be solvable, and it may be impossible to optimize its
signal to noise ratio. (Well, there are other mathematical methods, but
you get the point.) You are borrowing the concepts by metaphor, but the
concrete methods don't transfer well.

My point is not to discourage you from trying to stop spam, but to focus
your attention on detection, rather than protocol alteration.  It is
impossible to alter the protocol in any way that will force the spammer to
identify themselves a-priori as a spammer.

You could ask for spammers to cooperatively self-mark their messages.
But this hasn't been terribly productive.  It is also pointless to ask for
cooperative identification of non-spammers and identify spammers as those
not in the set of non-spammers. It may be too strong to say this is
pointless, as qsecretary and things that do this are fairly popular,
However, they don't quite solve the general case of things that send mail
but don't receive mail. And of course such schemes are fooled by disposal
accounts, which can respond until they are shut down.  If everyone used
qsecretary, spammers would simply alter their software to send responses.

So given a set of unmarked messages, some spam, some not-spam, the task is
to have a program mark them in the same way that a human would if a human
were reading the messages. Since humans have different definitions of
spam, it would be useful if the program could accept different definitions
as well.  This is the realm of content analysis.

> Thus my whole motivation for an unambiguous definition (spam == all bulk
> email) along the channel and not just a definition at the end points
> (UBE).

You may need a precise definition before you can begin implementation
(just like you need a definition of voltage, current, etc to begin
building a transmitter), but you do not need a precise definition to talk
about the theoretical aspects.  Spam could be defined as UCE, CE, UBE, or
BE.  I have also a more complete and detailed taxonomy of spam:

There are 3 types of email that we generally call spam:

Type 1: Bonafide Messaging with a real Commercial or non-profit(ie
political) purpose.  This also includes contraband (eg drugs) which is
illegal in some or all juridictions, so long as they intend to deliver the
illegal goods. This also includes solicited and unsolicited email, though
it may be useful to distinguish solicited as Type 1A and unsolicted as
Type 1B.

Type 2: Bonafide fraudulent activity. Someone is really trying to get your
money, but has no intentions of honoring their obligations to the purchase
contract.  This includes bonafide attempts at identify theft.  This is
already criminalized by mail fraud and wire fraud, and other laws
concerning fraud.

Type 3: Annoyance activity. This has no bonafide intention of getting
money or even personal information, even though at a casual glance it may
appear so.  Type 3 is broken into 2 subtypes:

Type 3A is a relatively harmless disgruntled person, who is not terribly
sophisticated in their abuse, or in hiding their tracks. This type can be
handled by warnings or account termination. Besides spam, this type is
also involved in small DOS attacks and other unsophisticated abuse.

Type 3B is frequently a career criminal using viruses and rooted machines
to conduct annoyance, which is frequently just another type of DOS attack,
but targeted perhaps at an email address, or perhaps at a domain. This
type of attacker is frequently already a career criminal, having broken
into many, often hundreds of computers, illegally. This type cannot be
dealt with effectively by ISPs, because they are reasonable adept at
partially hiding their tracks by crossing organizational boundaries.
Usually, the ISP only detects the infected computer, but does not identify
or catch the cracker.

Re: where the indirection layer belongs

[Masataka Ohta]

Robert Honore;

> It seems to me though, that nobody has stated clearly what those 
> wrong perceptions and ideas are, much less to say what is wrong with 
> them and thus replace them with correct perceptions and ideas.

A draft ID

Simple Internet Protocol, Again

on the real problems can be found at

ftp://ftp.hpcl.titech.ac.jp/draft-ohta-sipa--1.txt

Abstract

   IPv6 has failed to be deployed partly because of its complexity.

   IPv6 mostly is a descendant of SIP (Simple Internet Protocol) but has
   fatally bloated merging with other proposals and trying to make IPv6
   has better functionality than IPv4, which makes IPv6 unnecessarily
   complex and, thus, worse than IPv4.

   SIPA (Simple Internet Protocol, Again) is a proposal attempting to
   restore simplicity to IPv6 sticking to the real world operational
   requirements for the IPv4 Internet to make IPv6 more acceptable to
   ISPs and end users.

Masataka Ohta

The proper forum for evaluation of Spam proposals

[Harald Tveit Alvestrand]

Hello,

while the spam discussion has been entertaining as usual, for those who 
like watching namecalling, pointed invective and misunderstandings, I must 
remind the participants of 2 things:

- There is a proper forum for discussing anti-spam proposals. It is the 
mailing list of the IRTF antispam research group, [EMAIL PROTECTED]
This proposal fits within their taxonomy of antispam solutions 
, and the discussion is 
therefore appropriate to that forum. Consult the ASRG charter and web pages 
for guidelines for appropriate discussion.

- The charter of the IETF discussion list says that "unprofessional 
commentary" is inappropriate for the list. While it can certainly be 
entertaining to watch people try to define each other as kooks of various 
kinds, it is also definitely unprofessional commentary.
So is drawing conclusions in public about other people's intelligence, 
education and previous epxerience.

Summary: Please move over, folks - wrong arena!

  Harald Alvestrand

Re: The proper forum for evaluation of Spam proposals

[Shelby Moore]

>while the spam discussion has been entertaining as usual, for those who 


Not particularly entertaining for me.


>- There is a proper forum for discussing anti-spam proposals. It is the 
>mailing list of the IRTF antispam research group, [EMAIL PROTECTED]
>This proposal fits within their taxonomy of antispam solutions 
>, and the discussion is 
>therefore appropriate to that forum. Consult the ASRG charter and web pages 
>for guidelines for appropriate discussion.


Will do.  Thank you.  As I said previously, I started here:

http://www.faqs.org/rfcs/inet-standards.html

And here:

http://www.faqs.org/rfcs/editor-info.html

Which recommend going through IEFT.


>- The charter of the IETF discussion list says that "unprofessional 
>commentary" is inappropriate for the list.  While it can certainly be 
>entertaining to watch people try to define each other as kooks of various 
>kinds, it is also definitely unprofessional commentary.


Agreed 100%!  Wish you would have said that earlier to the person who started the 
"Kook" thread.


>So is drawing conclusions in public about other people's intelligence, 
>education and previous epxerience.


Which was the whole point of making a "Kook" thread to try to discredit my post or 
using the words "" to address my serious post.

> Summary: Please move over, folks - wrong arena!

As I said, "I'll believe that when I see more engineering posts than I see posts in 
the Kook thread" or ones addresses my posts with the word "plonk".  Professionalism is 
I think one of the words I used to  about what was happening.

I will move over now to IRTF and expect to go through this whole nasty reception thing 
all over again.  Yippie!  :)


Shelby Moore
http://AntiViotic.com

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Shelby Moore]

>I am not talking about email spreading virues. A number of viruses appear
>to send spam. (not spreading). Sometimes this is autonymous. Sometime it
>is under control via IRC channel back to the virus operator.
> Further, it
>seems that many open proxies are installed by virus.  Once the virus has
>control of the computer, it has or will obtain keys to private keychains,
>etc.  It can do whatever the infected users can do.
>
>The number of 40-50 emails per IP figure comes from analysis of spam
>messages that get by filters, by reviewing how many messages came from the
>same source. A lot of spam that gets by filters is of this very low volume
>type.


Dean this is important data for me, but not as useful until I can get you to tell me:

1. What was the time duration of the sample?

2. What was your total approximate sample size?

3. What was the total volume of this type of spam within that sample?

Estimates would be fine.  I would really appreciate it very much.

Thanks,
Shelby Moore
http://AntiViotic.com

Re: Proposal to define a simple architecture to differentiate legitimate bulk email from Spam (UBE)

[Shelby Moore]

After this issue, I am probably moving the thread to IRTF (as suggested) if possible 
(but probably after taking a break to do some other work).


>> >> > Information theory says that such things are impossible.  One can not
>> >> > construct a spam-free protocol because this is the same problem as
>> >> > constructing a system free of covert channels, which information theory
>> >> > says is impossible.
>>
>>
>> But information theory also says you can optimize signal-to-noise ratio,
>> but only if you know what the characteristics of your signal are.
>
>It actually doesn't say that precisely. It says that you can transmit a
>signal with an arbitrarilly low error rate at a speed below the channel
>capacity.
>
>The concrete task of altering the signal to noise ratio is accomplished by
>enhancing the signal with a harmonic oscillator, so that it is stronger
>than the noise.


Agreed.

And thus on a "conceptual level", you have to have some idea about the signal 
characteristics in order to enhance it.

Actually if I remember correctly, your example is how it applies to periodic signals.  
The general case is more abstract.


>  This is then described as a set of differential equations
>that can be optimized with Variational methods.  The limits of this
>process are indicated by information theory, the nyquist theorem, etc.


Add Shannon entropy, chaos, etc...


>If the channel isn't described by a fourier series, then the differential
>equations may not be solvable, and it may be impossible to optimize its
>signal to noise ratio. (Well, there are other mathematical methods, but
>you get the point.)


Yes that is what I meant that the general case is more abstract, so I was talking on a 
"conceptual" or abstract level.


> You are borrowing the concepts by metaphor, but the
>concrete methods don't transfer well.


I was only using it to say we must define the signal how it appears in the channel 
before we can do any research on it in the channel.

The way spam is currently defined defined as UBE (instead of my proposed *BE), then it 
means you can only model the signal at the end point.  Given that means in the 
receivers subjective mind, that is not all that useful for research, unless you want 
to get into very fuzzy science such pyschology.  If you want to make the point about 
practicality, then that is a very strong one!


>My point is not to discourage you from trying to stop spam,


You are only 1 of 3 people so far at IETF who has said that to me.  The rest who have 
commented have tried to discourage me.  So thank you.


> but to focus
>your attention on detection, rather than protocol alteration.  It is
>impossible to alter the protocol in any way that will force the spammer to
>identify themselves a-priori as a spammer.


Disagree strongly.  First benefit is once you define spam == *BE (instead of UBE), 
then it is easier to model spam and do research on it, because you can model it at any 
node in the channel, not only at the receiver end point.  That was my whole point 
about "enforcers".

However, there is a problem.  Some *BE is solicited.  Which is why I proposed moving 
the solicited *BE to another channel ("pull").

Your point is that it is futile to define a protocol that will separate the solicited 
from the unsolicited, because spammers will always be able to subvert the protocol.  
And you to say thus there are no benefits to detection.  I strongly disagree.  There 
are two aspects to my response:

1. Spam coming thru the alternate "pull" channel can be modeled differently that spam 
defined as *BE.  This separation of models provides benefits over trying to model spam 
as UBE in the receiver's mind (end point).  Other person in this thread has provided 
one specific example, which is the "pull" delay gives a whole new dynamic to 
detection.  Also I have pointed about that the membership quality of the solicited 
channel, gives it unique modeling advantages.

2. Spam coming thru the existing channel can then be modeled as *BE at any node of the 
channel, instead of as UBE.  Some nodes have a much better model of spam in this 
definition, than the one at the end point.  For example, ISPs can see a lot more abuse 
data in real-time, than a single receiver or the current inherently more clumsy 
attempts to group or poll receivers.

Hopefully that will set the record straight that I am thinking about spam in new 
conceptual ways...and not rehashing as others have claimed...


>You could ask for spammers to cooperatively self-mark their messages.
>But this hasn't been terribly productive.


Obviously I am not asking for that or any thing like that.  See above.


>  It is also pointless to ask for
>cooperative identification of non-spammers and identify spammers as those
>not in the set of non-spammers.


I am also not asking for this, and it is instructive to understand how I am not.

I am only making a definition, so that one can model under the benefits of that 
definition.  What people 

Re: POP3 delivers, not deletes

[Shelby Moore]

This is very important to me because obviously my new service (AntiViotic.com) depends 
on it, to the degree that IMAP is not widely implemented.

Thus I have some important questions below...


>>> Although it is theoretically possible, using POP (rather than IMAP)
>>> to leave the mail on the server until you pull it again with POP,
>>> many servers appear to clear out the mail after POPing it.
>>
>>
>> The Client program users LIST and RETR to get the messages than DELE to
>> remove them. The server/cloud program(s) don't do that. If you/anyone want
>> to leave mesage a\on a POP3 MTA, selecting a client program (or writing
>> one) to not DELEte them at your discression is potenetially useful.
>
>A *lot* of POP-using programs have the "Leave Mail On Server" option.
>And a lot of people have used "Leave Mail On Server" as a poor man's 
>1-folder IMAP, leading POP providers to implement mail retaining policies 
>of the "RETR it once and it's gone, whether you DELEted it or not".


Do you think they also apply this policy to TOP n?

Do you have any idea how prevelant this rather extreme policy might be???

That is a very worrisome revelation for me.  I was aware of the aging policy you 
mention below from the RFC, but not of the delete after RETR policy you mention above.


>>*  Enforce a site policy regarding mail retention on the server.
>>
>>   Sites are free to establish local policy regarding the storage and
>>   retention of messages on the server, both read and unread.  For
>>   example, a site might delete unread messages from the server after
>>   60 days and delete read messages after 7 days.  Such message
>>   deletions are outside the scope of the POP3 protocol and are not
>>   considered a protocol violation.


Note that the RFC does not define what a "read" message is, so I wonder if TOP n can 
apply , given that TOP n implies partial inspection (even if the entire message is 
returned)?

Thanks,
Shelby Moore
http://AntiViotic.com

Re: POP3 delivers, not deletes

[Shelby Moore]

[not private reply...in interests of sharing info]

Thanks for reply.

At 10:08 PM 9/8/2003 -0700, you wrote:
>[sent privately, since I just advised you to move discussion from the IETF 
>list]


I thought you meant regarding the anti-spam stuff, which I agreed with.  Why should 
discussions of POP engineering be moved to IRTF???  I thought IETF stands for 
"internet engineering task force".


>
>--On 9. september 2003 11:14 +0800 Shelby Moore <[EMAIL PROTECTED]> 
>wrote:
>
>>> A *lot* of POP-using programs have the "Leave Mail On Server" option.
>>> And a lot of people have used "Leave Mail On Server" as a poor man's
>>> 1-folder IMAP, leading POP providers to implement mail retaining
>>> policies  of the "RETR it once and it's gone, whether you DELEted it or
>>> not".
>>
>>
>> Do you think they also apply this policy to TOP n?
>>
>> Do you have any idea how prevelant this rather extreme policy might be???
>
>I don't know much about the commercial POP sellers' current behaviour... 
>disk has become cheaper, but mail has become fatter over the years since 
>1996.


Well at least it is easy enough to test for.  Send test message, POP, etc...


>
>Quoting more from RFC 1939 (a document I *very* much encourage reading in 
>*detail* before basing business decisions on details of POP behaviour):


I have read it several times, not taking the delete after RETR seriously because...and 
if you read the section you quote more carefully you see...


>>   One special case of a site policy is that messages may only be
>>   downloaded once from the server, and are deleted after this has
>>   been accomplished.  This could be implemented in POP3 server
>>   software by the following mechanism: "following a POP3 login by a
>>   client which was ended by a QUIT, delete all messages downloaded
>>   during the session with the RETR command".  It is important not to
>>   delete messages in the event of abnormal connection termination
>>   (ie, if no QUIT was received from the client) because the client
>>   may not have successfully received or stored the messages.


One merely needs to disconnect without using QUIT, if the POP server is RFC 1939 
compliant.  So this seems to make the whole suggestion implausible, since so easily to 
divert it.


>So the issue has been considered, it seems.

Yes but that is not what I asked you.  I asked you how prevelant you thought the 
practice is.


>Both listed authors are still active in the IETF, I think.

I guess that means POP engineering threads belong in the IETF mailing lists, which is 
why I am not making a private reply.


> Contacting them 
>privately for advice might prove illuminating.

Why privately?  Why not share the information?  I did not start this thread.

Thanks for sharing.

Shelby Moore
http://AntiViotic.com

Re: POP3 delivers, not deletes

[Valdis . Kletnieks]

On Tue, 09 Sep 2003 13:29:41 +0800, Shelby Moore said:

> One merely needs to disconnect without using QUIT, if the POP server is RFC
> 1939 compliant.  So this seems to make the whole suggestion implausible, since
> so easily to divert it.

That's a big 'if'.  RFC2821 mandates the acceptance of 'MAIL FROM:<>', but
my site alone gets enough bounces from sites that don't that I've found it useful
to define a macro to submit the site to www.rfc-ignorant.org.

It used to be that protocols only needed be error-resistant.   Nowadays, it seems
they need to be treachery-resistant too.


pgp0.pgp
Description: PGP signature

A roadmap for end-point identifiers? (was Re: where the indirection layer belongs)

[Pekka Nikander]

[Please direct replies either to the IPv6 or the IETF
mailing lists, but not both.  The default should be IPv6,
imho.]
Pekka Nikander wrote:
>> Now, even though I believe that we should solve the problems (and
>> apparently believe that there are sensible solutions), achieving
>> consensus on solutions that require architectural change may take too
>> long.  Hence, I also believe that we need some kind of a road map,
>> with some "temporary" or intermediate solutions along the way to a
>> more long-standing set of solutions.
Robert Honore replied:
> I agree, and your statement corresponds to what Keith Moore says about
> the solutions fitting into a framework that is yet to be specified.  I
> believe specification of that framework begins with our defining
> what an end-point is.
Good that we agree on a need for a roadmap.  Now, I want to return
back to your original problem analysis:
> *Stable (or reliable) end-point identifiers
> *Resiliency of application (protocol) in the face of sudden
>  IP address changes
> *Self-organised networks
These are the goals that we need to focus on.  While designing
the longer term architectural solutions, we need to preserve
the current functionality, and think about transition mechanisms.
From this point of view, the only (semi-)stable end-point
identifiers we have today are IP addresses.  We both agree, and
I think quite a few others agree, that IP addresses are not very
good end-point identifiers.  However, they are used as such today.
Furthermore, it will take quite a long time to get something to replace
the IP addresses as end-point identifiers.  As has been discussed
several times, domain names do not work well enough, and therefore
we need a new name space, I think.
Consequently, we have to provide (semi-)stable IP addresses
for IPv6 networks.  Based on the recent discussion at the IPv6
WG, apparently people think that PA addresses are not stable
enough.  Hence, at least to me, the Hinden/Haberman addresses
look like a good temporary solution.  It seems to provide stable
IP addresses, which can temporarily be used as end-point identifiers,
with the expectation that they will be eventually replaced with
"proper" end-point identifiers.
What comes to application resiliency, Christian Huitema's
approach of (mis)using Mobile IP may work well enough for a while.
However, it has a number of architectural problems that make
me to think about it only as a temporary solution.  Going further,
if we did not have any other reasons for "proper" end-point
identifiers, I think that Dave Croker's MAST might be good next
step.  However, since I do think that we most probably do need
stable and secure end-point identifiers, I think that something
like HIP will be more appropriate.
[I'm relatively ignorant to the exact details of self-organized
 networks, and therefore I don't want to comment that.]
Given the above, I think we could have a roadmap that might
look something like the following:
 Stable identifiers:Hinden/Haberman -> New name space
   for end-point
 Resiliency on  Huitema MIPv6  --> (MAST) ---> identifiers
 address changes:   multi-homing   (maybe HIP)
--Pekka Nikander