Re: DKIM length 'l=' tag

[John Levine]

It appears that Bill Cole  said:
>Never has been safe. Terrible idea from the start. Never should have 
>been included in the specification.

Agreed.

>I was thinking of the same thing in a half-assed way, just catching 
>anything using the length tag. I'd bet that correlates to spam but we'd 
>need data to prove that.

When that blog post came out, some people I know at large providers
took a look at the DKIM signatures they were seeing. There was one ESP
that was signing their mail with l=1, but they stopped when we pointed
out what a bad idea that was. Some corporate systems that use Iroport
appliances are misconfigured to put l= with the actual body length.
I've been trying to track them down and encourage them to turn it off.

My advice is just to ignore the l= length. For the Irnport users, the
signature covers the entire body so it'll still validate. Other than
that I don't think it's a strong spam indicator but there's no reason
to try and guess whether a message with a length that doesn't cover the
full body has been modified maliciously.

R's,
John

Re: Reporting Spam to csa-complai...@eco.de

[John Levine]

It appears that Kirk Ismay  said:
>-=-=-=-=-=-
>
>I've got a lot of finance / political spam that is passing through all 
>filters because it's DKIM signed and using an email provider 
>(salesforce.com & others).   One thing they do include is a 
>X-CSA-Complaints: csa-complai...@eco.de header, which looks legit.
>
>Has anyone had success with reporting mail to this address?  Does it get 
>results?

ECO is real and I've found it worthwhile to report spam to them.

R's,
John

Re: How to get removed from spamcop?

[John Levine]

>Just wondering if any real people are there or if it's totally 
>automated.

I've never had any trouble getting replies to polite inquiries.

>They have several of our IP addresses listed and delisting 
>doesn't seem to work. We're a spam filtering company (Junk Email Filter) 
>and if we fail to block a spam it can appear we are the source.

Uh, Marc, if the spam comes out of your servers, you ARE the source.
Nobody but you cares about your business model.

R's,
John

Re: Is EndOfSpam a known scam?

[John Levine]

>The idea is to charge emailers to send the emailee an email. The details are a 
>little
>bit more complicated than that, but not much (explained below)."

This is a bad idea that just won't go away.

I wrote a white paper about it.  It's ten years old, but nothing of any 
importance
has changed:

http://www.taugh.com/epostage.pdf

R's,
John

Re: Catching fake LinkedIn invites

[John Levine]

>Unfortunately not, for the most part. (The "From:" header is at linkedin 
>dot com, but the envelope sender is a random address, and I guess SPF 
>and DKIM run on the envelope sender only.)

DKIM runs on the message body.  If it doesn't have a valid DKIM signature
from linkedin, you can be quite sure that's not where it's from.

R's,
John

Re: Tarpitting (was Re: Spam harvesting using Fake Authentication)

[John Levine]

>It seems to me that greylisting and TCP tarpitting catch both sides of the 
>problem. Greylisting blocks junk from the single-attempt zombies, and TCP 
>tarpitting will catch the ones who are persistent offenders.

Maybe, probably not.  Modern MTAs, even the ones that are not
spambots, can run hundreds or thousands of connections in parallel.  I
very much doubt that you can tarpit enough of them to matter.

Re: Big problems with senders who use Microsoft Bigfish (a.k.a. FrontBridge)

[John Levine]

Oh, OK.
In the future, if you're not prepared to show the actual problem with their 
actual data, please don't waste our time.

R's from a thing with no keyboard,
John

Nigel Smith  wrote:

>
>
>
>>> Yes, I have checked on the real Zen lists and the real IP is there.
>
>>>Then your checking software is broken.  None of the Spamhaus lists ever 
>>>include anything in 10/8.
>
>
>John, the big hint was in the word *REAL IP*... as I said hundreds of times 
>subsequently to the initial post, I stupidly forgot to mention in my original 
>post that I obfuscated the first to octets of the IP into RFC1918 range.
>
>
>Yes, my bad, yes I should have said but I did tell everyone multiple times 
>subsequently, I think you must have skipped over my subsequent posts.
>
>Anyway, back to the topic, I rolled out some new configs yesterday evening 
>based on some of the suggestions received here, so will see how things pan out 
>today.
>

Re: Big problems with senders who use Microsoft Bigfish (a.k.a. FrontBridge)

[John Levine]

>> Yes, I have checked on the real Zen lists and the real IP is there.

Then your checking software is broken.  None of the Spamhaus lists ever include 
anything in 10/8.

R's,
John

Re: Interesting Spam Trap Idea - Fake Authentication

[John Levine]

>One of the things I like about it is that if hackers are sending spam 
>into my fake server then it takes away from their efforts on real 
>accounts that they could hack. I'm wondering if enough of us put up fake 
>authentication not only can we detect spam that way but we could waste a 
>lot of spammer's resources.

Tarpitting is so 1990s.  (Ask Ron Guilmette, who wrote an amazing
event driven tarpitter that could keep thousands of connections active
for days at at time on a 486.)

Spammers have botnets, so you can take it as a given that they have
more resources to waste than you do.

A fake AUTH server might be interesting to see how the spam differs from
ordinary spam, but it's not going to slow down anyone's sending.

Re: .pw / Palau URL domains in spam

[John Levine]

>well, I do not know anybody at Palau and so have no real need to exchange 
>mails, but I
>feel that this attitude seems somewhat drastic.

The .PW domain isn't really a country domain.  It's being sold as a
fake generic domain by Directi, an Indian registrar who has never been
able to manage abuse by their registrants.

Palau itself is a tiny US protectorate of 20,000 people whose few real
web sites are in other domains.  You can safely block all traffic
mentioning .PW with close to zero chance of losing anything real.

See my blog at http://jl.ly for a few more comments on the topic.

Re: .pw / Palau URL domains in spam

[John Levine]

> Kindly report all the complaints at abuse.al...@registry.pw and CC to
> abuse.al...@directi.com.

Hmmn.  Is there some reason you don't take abuse reports at 
ab...@registry.pw and at cont...@registry.pw, which is the only address on 
the web site?

Remember, everyone who sends you an abuse report rather than just blocking 
.PW out of exasperation is doing you a favor.  The easier you make it, the 
better off you'll be.

I hope you're not still telling people to figure out who the registrar is 
and contact them, which was impressively lame.

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

Re: .pw / Palau URL domains in spam

[John Levine]

>Nominet is a registrar

No, Nominet is THE .co.uk registry

R's,
John

>Directi is acting as THE .pw registry

Re: .pw / Palau URL domains in spam

[John Levine]

In article <517f122c.3050...@trimble.com> you write:
>I agree. We've seen a huge increase in ".pw" email - 100% spam
>
>I see one antispam vendor is telling its customers to just block
>anything containing .pw references - I'm rapidly warming to the idea...

You can report them to ab...@registry.pw, who will tell you to contact
the registrar (uh, no, not my problem) and sometimes grudgingly shut
down the offending domain.

I agree that there's little reason to expect that anyone would miss
any .pw mail unless you live in a certain part of the South Pacific.

R's,
John

Re: Interpreting an Authentication-Results: header ?

[John Levine]

>> You'd need to configure it to tell which authids to accept, perhaps
>> defaulting to the host name of the machine SA is running on since
>> that's a likely default for the authid.
>
>Agreed. I think it would also - at the trust boundary - need a filter before
>the DKIM/SPF verifier that adds the Authentication-Results: header. Its job
>would be to remove any Authentication-Results: that claim to belong to ones
>own ADMD.

You might want to reread section 5 of RFC 5451.  That's already in the
A-R spec.

R's,
John

Re: Interpreting an Authentication-Results: header ?

[John Levine]

>IIRC there isn't at the moment. One thought that comes to mind immediately:
>
>If there were it should not be enabled by default or others will try to forge
>the results. It should only be enabled if a "trust boundary"
> has been established. The
>documentation should mention that.

You'd need to configure it to tell which authids to accept, perhaps
defaulting to the host name of the machine SA is running on since
that's a likely default for the authid.

Interpreting an Authentication-Results: header ?

[John Levine]

The Authentication-Results: header defined in RFC 5451 can describe
the SPF and DKIM status of a message.  It's typically added by the
SMTP daemon as the message is received.  

Is there any way to tell spamassassin to look at the A-R header rather
than trying to rerun the SPF and DKIM checks itself?

Re: NJABL is history

[John Levine]

>I'm assuming this means their feed into Zen and XBL has shut down, too?
>If I'm wrong and that feed still exists, (anyone who knows...) please
>reply to this post with that clarification. (would be interesting to know)

The whole thing is kaput.  The guy who was running it has a new job,
there was nobody left to run it.

Re: Greylisting (was Re: "Fairly-Secure" Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, SpamAssassin, Razor and DCC ? Can I get your opinion?)

[John Levine]

>Does greylisting increase chances of bulk detectors (razor/pyzor/dcc) in
>case of "yahoo like" spam sources?

No.  A remarkable fraction of ratware still doesn't bother to retry,
so the most simple minded greylister will deter them.  That's why it's
useful.  I've never seen any support for the theory that greylisting
delays make it more likely that the host will be blacklisted when it
retries.

I haven't seen many legit senders that don't retry as David says he
has, but I don't have his volume of mail, either.

Re: Somewhat OT: Is this wrong?

[John Levine]

>It appears to be on by default as part of Exchange's Intelligent [sic]
>Message Filter.

As I understand it, you're referring to Exchange 2003, which was
shipped nine years ago, and which, if you believe the Wikipedia
article, hasn't been updated since 2005 and hasn't been supported
since 2009.

What, exactly, do you expect to happen here?

http://en.wikipedia.org/wiki/Microsoft_Exchange_Server

R's,
John

Re: Somewhat OT: Is this wrong?

[John Levine]

>The problem is that I publish SPF records for my domain in the expectation
>that they'll be used correctly.  By behaving incorrectly, Microsoft
>is making it less attractive for sites to publish SPF records lest they
>be misinterpreted.

Microsoft's Sender-ID has been using SPF records to do Sender-ID
checks for a decade.  You just noticed now?

In practice, Sender-ID has been a failure, nobody of any importance
uses it any more, and Microsoft's own mail systems including Hotmail
are now doing DKIM and SPF checks.

R's,
John

Re: Blacklisting based on SPF

[John Levine]

In article 
 you write:
>-=-=-=-=-=-
>
>I've noticed some trojans with addresses from usps.com slip through.
>
>Does anyone blacklist based on SPF?

Nobody with any interest in delivering the mail that their users want.
The error rate is much, much too high.

R's,
John

Re: Caution - access to Spamhaus data-feed may be improperly configured: secnap.com.ionspam.net.

[John Levine]

> I for one am tired of getting these emails from MxTools because I
> failed to turn off EVERY call in SA that invokes a Spamhaus lookup.

Um, the last I checked Spamassassin was an open source package
provided for free, even to people who use it as part of a commercial
service.

If you think that this is a problem worth fixing, how about you and
Mike write the code and contribute it back, just in case there's
someone else who wants it?

R's,
John

Re: Caution - access to Spamhaus data-feed may be improperly configured: secnap.com.ionspam.net.

[John Levine]

MXTools is real, I know some of the people who work there.

Dunno why they'd think you're querying the Spamhaus lists if you
aren't -- it is my impression that Spamhaus looks at the query logs
and passes along IPs that are close to being rate limited.

R's,
John

Re: How do I disable all spamhaus calls?

[John Levine]

>I wanted to buy spamhaus rsync feeds.  our CFO looked at the contract, 
>where Spamhaus said they could disable the feed without notice if they 
>wanted to. (if they suspected you got hacked, were selling it, were a 
>spammer, weren't protecting it, allowed public access to it).

In my experience, Simon who handles the feed contracts is a reasonable
guy.  You might give him a call.

R's,
John

Re: How do I disable all spamhaus calls?

[John Levine]

>> PS: I don't suppose there's any chance you might consider paying the
>> rather modest price for a Spamhaus datafeed, rather than leeching all
>> your DNSBL queries for free.
>
>I suspect you would not say that if you knew more about what Marc does.

We all know what he does.  But if he's running into Spamhaus' rate limits,
he's making a lot of queries.

R's,
John

Re: How do I disable all spamhaus calls?

[John Levine]

>That's a good idea except that I'm using pdns-recursor for my caching 
>nameservers.

A few seconds looking at the manual reveals how to get pdns-recursor to
do the same thing:

 http://doc.powerdns.com/built-in-recursor.html#recursor-settings

(hint: see auth-zones)

R's,
John

PS: I don't suppose there's any chance you might consider paying the
rather modest price for a Spamhaus datafeed, rather than leeching all
your DNSBL queries for free.

Re: caches, was TTL and DNSBLs (was Re: SpamTips.org: Why run your own DNS server?)

[John Levine]

>> But if you're looking for a DNS cache, I highly recommend unbound.
>> I used to use dnscache but got tired of its limitations (due entirely
>> to it being unchanged since 1998.)  My copy of unbound runs about
>> 27M real RAM, 44M virtual, which is pretty modest on my 12G server.
>
>how many q/s is that instance handling?

It's quite variable.  I get hourly reports with the queries per hour
ranging from 10K to 90K, so I'd guess that typically it's 10-15 qps.

R's,
John

Re: TTL and DNSBLs (was Re: SpamTips.org: Why run your own DNS server?)

[John Levine]

>My experiments on real mail servers show that DNS caching is quite
>ineffective for DNSBLs (at least for typical ones like Spamhaus that
>use a short TTL on the order of 15-30 minutes.)

That's consistent with what I've seen, although you probably won't be
surprised to hear that I have higher hopes for my range published
DNSxLs than David does, partly because I expect them to be used for
whitelist which tend to cache better for technical reasons.

But if you're looking for a DNS cache, I highly recommend unbound.
I used to use dnscache but got tired of its limitations (due entirely
to it being unchanged since 1998.)  My copy of unbound runs about
27M real RAM, 44M virtual, which is pretty modest on my 12G server.

R's,
John

Re: multiple from entries

[John Levine]

>Anyone know of any legitimate use of multiple email addresses in a from 
>line?

Yes. I know a few IETF people who do it.  Stuff like notes to a working
group from both chairs.

I think I've seen the same multiple-from spam you have.  It appears to
confuse Mailman, but that's not postfix's problem.

R's,
John

Re: Should Emails Have An Expiration Date

[John Levine]

>> I know just enough about copyright law to know that this claim is
>> nonsense.

>No, it is not nonsense.  Copyright law does allow the content creator
>to specify duration of use.  If you go view a movie in a movie theater
>you buy a ticket for a single viewing, you do not automatically get
>to view it multiple times just because you bought a ticket.

I think this would be a good time to go look up terms like "fixation",
"first sale", and "fair use" before further embarassing yourself.

Free hint: movie tickets aren't copyright licenses.

R's,
John

Re: Should Emails Have An Expiration Date

[John Levine]

> From a legal perspective I will point out that any e-mail you
>receive is (at least in the US, but most other countries too)
>considered copyrighted by the sender.  Under copyright law the
>sender has the right to control expiration of content they create,

I really think it would be a good idea for people to refrain from
playing Junior Lawyer here.

I know just enough about copyright law to know that this claim is
nonsense.

R's,
John

Re: Should Emails Have An Expiration Date

[John Levine]

>I do like the idea with respect to alerts; if email programs (especially
>those on smart phones) would know to avoid alerting you of unread +
>expired messages, that could be quite beneficial.  Especially if I could
>set expiration times with thunderbird filters.

If people keep at it, they may yet reinvent RSS which, I note, T'bird
supports reasonably well.

The main appeal of Expires: is to spammers and near-spammers, who hope
they can make the mail go away before people complain about it.  As I
pointed out to Ken Magill, usenet has had an Expires: header for
decades, the design of usenet makes it cheap to implement, and it has
never been useful in practice.

Re: RFC-Ignorant (was Re: Irony)

[John Levine]

RFC Ignorant is deep into kook territory, as should be apparent if you
look at which RFCs they expect people to follow, and what their
definition of "follow" is.

abuse.net has been listed for years, since there is an autoresponder
on ab...@abuse.net, and I've never noticed any delivery problems.

One time I asked if they'd delist me if I got rid of the autoresponder
and just threw all the abuse mail away.  Yes.  QED.

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

Re: DNS cache efficiency for low-TTL records (was Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01)

[John Levine]

>In summary, I believe DNS caching is basically *useless* for any site
>small enough to use Spamhaus for free.  And any very large site is
>probably large enough to deserve an rsync feed.

Hmmn.  See the ASRG list where I've posted some numbers I worked up
from my own servers.

R's,
John

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>This is a great topic! Is this been discussed at the IETF level?

Well, yeah, that's the internet draft that I started this with.

There's a parallel discussion in the IETF anti-spam research group
(ASRG) which is a better place to continue this.

See http://wiki.asrg.sp.am/ which has a link to subscribe to the
mailing list.

R's,
John

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>Frankly, I'd think that besides costing the spammers money (a good thing in
>and of itself) it would also be a pretty good spamsign if a block has more
>than, say, 5 or so registered senders in a /64. Just thinking out loud
>here

There are a lot of non-spam mail systems with a heck of a lot more
than five mail hosts, and I can easily imagine putting a bunch of them
on a single LAN where they're be in the same /64.

I also don't think it's very realistic to expect that there will
be a master mail host file distributed periodically like HOSTS.TXT
was.  There's a reason that the DNS was invented, and at the time it
was, there were a whole lot less hosts on the net than there are mail
hosts now.

Finally, I presume everyone is aware that there is no possibility
whatsoever that RIRs will get into the business of selling mail host
IPs, so this entire subthread is 100% hypothetical.

R's,
John

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>Please reconsider... and how about this twist...
>
>Let the IP registrars (arin.net, etc) add a very nominal fee for
>allowing networks to designate particular IPs as being used for SMTP.

Haven't you just reinvented whitelisting?  I think it's pretty likely
that people will make lists of IPs known to be mail clients to keep
down the filtering load, but there's still the problem that bad guys
an sign up so you have endless compliance problems.

Oh, and if you did that, how would MTAs check that an address of an
incoming connection was on the list?  That's the issue that started
this discussion.

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

Re: Off topic: best RBLs to use to block at smtp connection?

[John Levine]

In article <20110103143854.16122gzwrvkrf...@mail.junc.org> you write:
>On man 03 jan 2011 13:28:12 CET, Jari Fredriksson wrote
>
>> I want a good coverage, but not too many false positives. What do you
>> use to block a spammer at SMTP connect?
>
>google on dbl.spamhaus.org and zen.spamhaus.org

Agreed.  I also find that bl.spamcop.org now works well with low
false positives.  It used to have terrible FP, but they fixed it.

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>And SMTP is the same philosophy.  Unicode addressing should rightly be
>an add-on to a simpler system.  And frankly the biggest proponent of
>EAI is China - and why do you think that this is?

Silly me, I thought it was because they have 1.2 billion citizens
who read and write Chinese rather than English.

In any event, there are better fora for conspiracy theories.  I'm
just trying to make DNSBLs and DNSWLs work in IPv6.

R's,
John

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

Ah, I see the problem.  You're assuming that spammers will follow the
rules.  That's a poor assumption.

>> The IPv6 address space is big.  Very, very big.  Even if you chop it
>> in half to /64s, it is still four billion times bigger than the v4
>> address space.  Bad guys hopping around /64s will blow out your DNS
>> cache just as badly as hopping around /128s.
>
>No, since the number of total host numbers in a /64 is vastly larger 
>than in a /128, if you hold to single number queries then it will blow
>it out far far faster.

I suppose that's technically correct, in the sense that blowing out in
a millisecond is faster than blowing it out in a minute, but that hardly
matters to people running DNS caches.  Let's do a thought experiment:
imagine you have a big honking DNS cache with 100 billion slots.  If we
had a BL with one potential entry per /64 (keeping in mind that a cache
remembers both successful and failed queries), how much of the address
space can you query before your cache fills up?

Answer 0.0054% Kaboom!

>Well for starters it almost sounds to me like your not that familiar
>with IPv6 to even say this.  The lower 64 bits of the address is the
>interface identifier, and the upper 64 bits is the sub-network
>prefix.

If you use SAA, sure.  If you use DHCP, the address layout is whatever
it is.

> It is extremely abnormal for a host to change it's MAC address every
> few milliseconds, so the idea that a spammer could cycle through the
> lower /64 using 1 address per message is in the realm of extreme
> improbability.

Like I said, you're making the poor assumption that spammers will
follow your rules.  In reality, they'll do whatever they think will
get their spam through.

>  The only reason to run around saying the sky is falling is if your
>coming at this from the point of view that it would be a normal thing
>to see packets from the same /64 that have many different interface
>identifiers, which there is no logical need for this to ever happen.

Like I said, you're making the poor assumption that spammers will
follow your rules.  In reality, they'll do whatever they think will
get their spam through.

>You would be better off starting with the assumption that the
>existing design is fine, but that it needs adjusting for the new
>circumstances for IPv6.

I did that.  It doesn't work.

R's,
John

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>Now obviously, there's a breakpoint at which synchronizing the local
>database from the master becomes cheaper than doing lookups.  Right
>now, that's quite high, but it will move lower with IPv6.

Why do you say that?  The number of computers on the net isn't going
to be much bigger with IPv6.  They're just spread out in a much larger
address space.

We also seem to have a diagreement about BL usage patterns.  The BLs I
know have a smallish number of very heavy clients and then a long tail
of clients that get smaller and smaller.  It makes sense for the
biggest ones to keep their own mirrors, but there's a whole lot of
small ones, each of whom will never look at more than a small fraction
of the data, although their aggregate traffic is substantial.

So if you say that everyone has to maintain a mirror to get a BL's
data, you're saying that small clients can't use BLs at all.  Is that
realistic?

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>I used rsync as an example.  You can use a more efficient technique; I
>gave ClamAV's signature-distribution mechanism as an example of a
>system that works pretty well.

Hey!  I have an idea!  How about if we form the data into a B-tree and
let people download pages on demand via the DNS?

R's,
John

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>If blacklists like CBL are currently at 100 MBs (for IPv4)... the bloat
>for IPv6 could break DNSBLs. RSYNCing Gigabyte (or terabyte!) -sized
>files is memory and CPU intensive. Loading those into rbldnsd is also
>resource expensive! Furthermore, getting that data out to DNS mirrors
>quickly and efficiently is going to be a nightmare! And... this requires
>that ALL mirrors be upgraded to accommodate the vastly larger size.

Right.  I don't think the CBL will get much larger, since it will
certainly do /64 granularity, but it'll still be a challenge to query
efficiently.

>(1) create a standard whereby non-authenticated IPv6 mail can ONLY be
>accepted by certain IPs (such as x.x.x.0

Sorry, no chance.

>(2) Why can't "Forward Confirmed reverse DNS" (FCrDNS) become a standard
>for IPv6?

Because rDNS lookups will explode your cache just as badly as DNSBL
lookups.  In the words of a friend who used to run a very large mail
system, when I asked him about IPv6 rDNS: Just Say No.  rDNS isn't
likely to be useful at all for v6, although you could try something
like CSV based on looking up the EHLO name.

>(3) A shifting of focus on whitelists is important... but some of those
>shouldn't really be "whitelists" in the traditional sense. Instead, they
>should merely indicate that an IP is a candidate for sending mail.

This one I agree with.  The Spamhaus whitelist is intended only for
very virtuous sources of mail, but it will clearly also be useful to
have what was called a yellow list a few days ago, hosts that send
enough real mail that you can't just blacklist them even if you see
some spam.

R's,
John

Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

>I agree, so I propose a much larger change: Stop using DNS for this
>purpose.  I don't think it's the right tool for the job.

Sigh.  Yes, that's one of the bad ideas.

Remember that part of the goal is to keep the traffic to and from the
DNSBL/WL's servers under control.

>Any protocol that makes lookups in a huge adress space efficient and
>efficiently-cacheable is going to leak much of the list information.
>So why not just distribute copies of the entire list in a format that
>permits efficient lookups and efficient sychronization (eg with
>rsync)?

If you're familiar with populare DNSBLs, this is not a new idea.
Spamhaus, for example, provides low volume queries for free, medium
volume queries for a charge, and rsync access for a higher charge.
See:

http://www.spamhaustech.com/datafeed/index.lasso

Consider the amount of traffic involved in doing an rsync: set up a
TCP session, do the key exchange to set up a SSH session, then start
comparing pieces and checksums to figure out which parts of the files
need to be transferred, that's a fair bit of traffic, particularly for
a large zone file.  The tradeoff point where it's cheaper than doing
queries is quite high.  If you've got a giant mail system, it makes
sense, but if you have one or two MTAs, even fairly busy ones, it
doesn't.

Hence my goal to concoct something which allows efficient publication
and caching via the DNS.

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01

[John Levine]

Hi.  I hear there's been some interest in my IPv6 DNSBL proposal.  My
goal is that since there are (close enough to) no v6 BLs or WLs yet,
this is the time to switch to a query design that will scale.  The
design I put in RFC 5782 isn't it, unfortunately, nor is anything
similar to it.

We'll have to change our software to handle v6 lookups no matter what,
so I don't see it as a big deal whether it's a small change or a
slightly larger change.

>Thus, we can safely make the assumption that any mailserver is going
>to follow the model of a single host per /64. ...

This strikes me as a poor idea for two reasons: it's probably not
true, and even if it is, it won't help.

The IPv6 address space is big.  Very, very big.  Even if you chop it
in half to /64s, it is still four billion times bigger than the v4
address space.  Bad guys hopping around /64s will blow out your DNS
cache just as badly as hopping around /128s.  And at this point I
would not want to assume there is only one host per /64, or that a
/64 will contain all good hosts or all bad hosts, since there will
doubtless be cases where that's not true.

If you've read my proposal (if you haven't, please stop, visit
http://tools.ietf.org/html/draft-levine-iprangepub-01 and read it,
then come back) you'll see that maintaining a BL/WL is fairly
complicated, but the lookups are quite simple.  Each lookup involves
about five DNS queries, but the design makes it very likely that most
if not all of the answers will already be in the local cache, since
the queries all start from the top of the same tree.

It also ensures that if you do a bunch of lookups to addresses that
are near each other, they'll probably all do the same queries, so
all after the first will be cached.

Another way to look at it is the size of zone: since each DNS record
holds 40 entries, the number of records is no more than 1/40th of the
size of a record-per-entry design.  In the common case that entries
span a range of addresses, the number of entries will be even less, a
lot less.  (Note that rbldnsd synthesizes records on the fly, so that
as far as a client can tell, even if the server knows something is a
/16, the client sees 64K different records.)  And finally, in this
design, the client only looks for records that exist, so there should
be no negative entries in the cache at all.  This tells me that this
would have performed better even for short 32 bit addresses.

I've given it a fair amount of thought, and I think I have gone
through all of the same band-aids everyone else is thinking of, e.g.,
truncate everything to 64 bits, do some sort of probe to find out the
granularity of a range, and they don't work.  When you consider the
length of the addresses, the number of queries, and the cache
behavior, I'm pretty sure this design is vastly better than anything
based on the traditional design, and is not an unreasonable amount of
work for clients.

I don't think it's perfect, and I'd be delighted to get suggestions,
but please don't start by assuming that spammers won't be maximally
hostile, or that managers will always configure their networks the
way you'd prefer.

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly