Re: KAM pccc URIBL questions
On 10/7/2013 7:42 PM, Raymond Dijkxhoorn wrote: > This is harming more then it does good. But its your list so your > rules ;) I would not want to use it to filter my mails with it but hey Since this is in its early development, it is probably too early to judge it too much. But from what I've read in this discussion, it is "light years" away from the current major URI/domain blacklists out there (SURBL, URIBL, ivmURI, DBL)... BUT... Kevin is brilliant so who knows what it might eventually become? ALSO...There is an argument that a more-aggressive-than-normal AND low-scoring URI list may be helpful? In that sense, URIBL.com has traditionally been considered slightly more aggressive than the other lists mentioned above... SLIGHTLY! Maybe something much MORE aggressive, intended for very low scoring... would be useful? (this would be situations where bayes or checksum content filters add points to the spam score combined with such an aggressive URI list putting the message "over the top"... but then skipping blocking a legit message with this URI because it didn't have the other content points added and thus didn't score high enough--at least that is the idea) But I can't help but think that SOME reading this thread haven't even tried/implemented even all the zero-cost options for the (already matured) lists I mentioned (where applicable)? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: NJABL is history
On 3/1/2013 5:52 PM, Axb wrote: > "Please spread the word. NJABL has to be shut down effective > immediately. I just emptied the dnsbl zone files. I'm assuming this means their feed into Zen and XBL has shut down, too? If I'm wrong and that feed still exists, (anyone who knows...) please reply to this post with that clarification. (would be interesting to know) (actually, while NJABL was a great list, I found their listings that didn't feed into SpamHaus to be the most valuable CBL was already doing the "heavy lifting" for XBL. Many of us will especially miss NJABL's OTHER non-XBL-related listings!!) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Who wants a lot of spam (to analyse)
Why do some have to be so mean to Marc Perkel? If you (a) don't have a legit question, (b) don't have a legit complaint, (c) don't have an honest good faith question ...then STFU. Snarky, sarcastic, and silly responses are basically time-wasting spam for the rest of us, for all practical purposes. If Marc offers something, or has an idea, regarding anti-spam... that SOMEONE might find interesting or valuable, but you personally don't need, then just ignore it. That simple. PS - I understand that if someone like Marc were to do this often, and it wasn't directly related to SA... that would be a problem. But I strongly don't think that is the case here. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Whitelist and DNS blacklists in SpamAssassin
On 2/5/2013 6:22 AM, Per Jessen wrote: >> http://www.rfc-ignorant.de/ >> > >> > -- Matthias > Thanks, I didn't know someone had decided to continue the project. I > suggested it on the rfc-ignorant mailing list but there wasn't much > interest. Interesting and good news!... but their home page states that their zones are not yet populated. So I guess they are not yet operational yet? (or maybe the site messages is out of date?) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: ANNOUNCEMENT: update to ivmURI regarding surge in rarely-blacklisted domains spammers use from legit site that are "compromised"
On 1/7/2013 1:53 PM, dar...@chaosreigns.com wrote: > What spamassassin rules is this related to? Generally speaking, "25_uribl.cf". More specifically, possible extra URI or domain blacklists which could be custom added to Spam Assassin, such as the one I mentioned, ivmURI, which I manage, and which is a commercial anti-spam blacklist that is not built into SA by default. Also, some on the SA discussion list have their own "home grown" URI or domain blacklists and they could potentially take the information/suggestions I provided and use it to improve their own custom lists which they've custom integrated into SA. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
ANNOUNCEMENT: update to ivmURI regarding surge in rarely-blacklisted domains spammers use from legit site that are "compromised"
ANNOUNCEMENT: update to ivmURI regarding surge in rarely-blacklisted domains spammers use from legit site that are "compromised" There has been a surge during the past couple of days in rarely-blacklisted domains (as in, you see few of these blacklisted on SURBL/URIBL/DBL) ...where the spammers used "compromised" sites which are normally legit sites. (maybe the FTP password was cracked? or some other security hole exploited?) Likewise, ivmURI was missing many of these because our FP-prevention-filters... which normally prevent "decoy" domains or innocent domains from getting blacklisted... were also causing many of these to be overlooked. (I suspect that the same was happening with the other URI blacklists, since [it seems?] even fewer of these were getting blacklisted on those other URI/domain blacklists?) This isn't new. For months, it has been on my mind to make some adjustments to "surgically target" listing these types of domains... where our FP-prevention-filters would then "back off" just a tad... yet in a very "surgically targeted" way... so that these would start blacklisting, yet without those changes to the filters suddenly causing many FPs, and where these domains would also expire off of ivmURI faster--with the idea that the site owners would probably find and fix their problem somewhat quickly. (we don't want these to remain blacklisted weeks after the spam has ceased and the security problem fixed) Yes, this WILL cause a tiny bit of "collateral damage"... but my estimation is that the ratio is off-the-chart GOOD! These are relatively minor sites. This could potentially cause hundreds of thousands of spams blocked for every one legit mail blocked. And if someone STILL has a problem with that ratio... then my message to them is... the site owner should be somewhat held accountable for their poor security--which is partly at fault for so much elusive spam making it into inboxes! (and, again, these listings will expire MUCH faster than regular ivmURI listings) Many of these spams are especially elusive because the spammers then combine the use of a somewhat legit domain... with sending from "freemail" servers, or other legit mail servers which would cause far too much collateral damage if blocked by IP. At best, this puts a HUGE burden on content filters. At worst, many of these are slipping past many spam filters. This major milestone improvement for ivmURI was implemented mere hours ago. Here are some results... where these were added to the ivmURI list today: http://dnsbl.invaluement.com/uri_surge.txt NOTE: These are all domains impacted by this change. Unfortunately, many in that list would been blacklisted on ivmURI anyways, without the changes... but many domains in that list required this change to get listed on ivmURI. Also, across the board, you'll also find very few in that list which are on ANY other URI blacklists! Questions/Feedback are welcome! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: "Pill" spams
On 4/10/2012 6:29 PM, RW wrote: > On Tue, 10 Apr 2012 17:58:51 -0400 > Rob McEwen wrote: >> Meanwhile, the snowshoe spammer's DNS server happens to be messed up, >> overloaded, and returns answers within about 4 seconds. > But unless I'm misunderstanding, the NS lookups would be done on the > TLDs nameservers, rather than the spammer's DNS server. The sneakiest of spammy domains are the ones NOT seen before, and thus NOT is anyone's cache. Therefore, .../I WAS THINKING THAT... /the lookup on the domain's NS server would OFTEN have to propagate back to the authoritative DNS server for that domain that being the spammer's DNS server... at the time the message is evaluated. But you're right... maybe to get the DNS server assignment for that domain, it only has to go to the TLD's nameserver, grabbing information propagated to the TLD from the registrar for that domain. Good point! (still much slower than DNSBL lookups to an rbldnsd server... but probably not any slower than DNSBL lookups to a remote 3rd party DNS blacklist) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: "Pill" spams
On 4/10/2012 3:16 PM, Axb wrote: > On 04/10/2012 08:07 PM, Rob McEwen wrote: > >> (b) If anyone programs this idea into SA, or anywhere else, then >> this should be a separate step AFTER regular URI checkinggiving >> the message a chance to "short circuit" out of processing if it >> already scored high enough after URI checking. Why? Because this >> would defeat some of the benefits of fast URI checking if it was >> done in tandem with the URI checking. Basically, URI checking >> can be >> lightening fast... especially if you are checking the extracted >> URIs >> against a local rbldnsd server. In contrast, anytime you do a name >> server lookup to some stranger's domain, you're subjecting yourself >> to the mercy of their reply speed... and many of those spammers use >> screwed up and/or overloaded equipment. (even if your DNS timeout >> setting becomes a "safety net", that is still order of magnitudes >> slower than rbldnsd checking!) > > afaik, SA does async lookups so you have next to no delay - negligible sounds good.. except... consider this scenario... A person uses the system I described above, but where the name server fetches, & lookups on domains contained within those nameserver hosts... all happen async. But the domains themselves are HEAVILY blacklisted found on SURBL, URIBL, DBL, and ivmURI... and the end users subscribes to datafeeds from ALL of those... so THOSE lookups are to a local rbldnsd server running on a dedicated machine.. which means... super fast queries... as in <1ms. With those domains in that message getting MANY hits, and with other things already having hit... suddenly... the spam score jumps super high in extremely little time. Meanwhile, the snowshoe spammer's DNS server happens to be messed up, overloaded, and returns answers within about 4 seconds. in this scenario... which, though rare... might actually be MORE common percentage-wise than the number of times an actual domain-blacklist "hit" on a domains' nameserver actually causes a spam to be blocked again, in this scenario, async or not... doesn't that whole mail session then get "bottled up" on waiting on the nameserver lookups? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: "Pill" spams
On 4/10/2012 11:42 AM, Thomas Johnson wrote: > Any other ideas on these pill spams? What are they scoring for anyone else? Hi. I've been following this thread. Here are some (random) thoughts & suggestions: (1) In some of those examples Thomas provided, at least one of the assigned name servers had a hostname which contained a domain name... where that domain name was blacklisted on either multi.surblorg or dbl.spamhaus.org ...Therefore, an SA rule that grabs the name servers for the same domains it checked against URI lists, extracts out the domain names from them (where different from the actual domain you did the lookup on), and then checks those against URI blacklists--could possibly have produced a higher score... even where other URI lists had missed those domains. NOTES: (a) BTW, invaluement does NOTHING regarding name servers of spamvertized domains... and we've never done anything with them in the past. Eventually, we plan to change that... in a variety of ways... (b) If anyone programs this idea into SA, or anywhere else, then this should be a separate step AFTER regular URI checkinggiving the message a chance to "short circuit" out of processing if it already scored high enough after URI checking. Why? Because this would defeat some of the benefits of fast URI checking if it was done in tandem with the URI checking. Basically, URI checking can be lightening fast... especially if you are checking the extracted URIs against a local rbldnsd server. In contrast, anytime you do a name server lookup to some stranger's domain, you're subjecting yourself to the mercy of their reply speed... and many of those spammers use screwed up and/or overloaded equipment. (even if your DNS timeout setting becomes a "safety net", that is still order of magnitudes slower than rbldnsd checking!) (2) Thomas specifically mentioned that invalument, and other lists he uses, didn't catch these. There may be a reason invaluement missed some of these: (a) In February and early March, we implemented the largest hardware and software upgrades in the 15-year history of our company. It was massive (for us). We went "all 64 bit" at the same time. Overall, the upgrade was a huge success... but even as recently as today... we're still putting a few things back together and are not quite up to "full speed". There were intermittent outages and degradation in effectiveness though large parts of February and March. But we're almost finished and are now "fine tuning" various things. I wonder if some of those missed spams came when we were having some of our worst problems, during the thick of those hardware/software upgrades? (even last week, we had some disruptions) Hopefully, we'll do much better from this point forward... certainly, in other ways, the improves hardware is already speeding things up... we just needed to work out the kinks... getting all that new 64-bit software to work together. (b) Now that we have this upgrade completed... we're trying now to expand our spam feeds. I think that spammers have often learned not only how to avoid hitting our traps directly... but may have discovered (even if just through process of elimination) some of our external spam sources. (which is not a bad thing as that means that those providing us spam... are getting less spam). Or, maybe not... maybe I'm just paranoid! But, the bottom line is that our new equipment greatly expands our ability to quickly process more spam sources. If anyone reading this is interested, and can provide one.. let me know (off list!). We could possibly even work out a discount on invaluement access ...if your feed is VERY prolific. (contact me off-list for details, if interested) With more spam feeds, we hope to cast a "wider net" and catch more of those URIs that have eluded many (and sometimes all!) blacklists! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Lots of comment in mail, how to score
On 2/6/2012 12:57 PM, Mynabbler wrote: > As I said, sure they are in RBL now. They were not when this message was > delivered. Looking at the date/time stamps, I'm almost positive that this URI was blacklisted in BOTH uribl-BLACK and ivmURI *hours* before your sample message arrived. But, of course, your question is till valid! Having rules in place in SA to deal with this kind of attempt at getting around bayes-filtering is a good idea! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: [OT] RBLs
On 1/31/2012 12:40 PM, Phil Daws wrote: > Just to follow up we have seen a huge decrease in the amount of SPAM received > since we implemented the Invaluement RBLs. Phil, Thanks for the feedback. It seems you greatly benefited from NOT listening to that one guy who said that adding another RBL is a waste of time... ya know... the same guy who suggested that I needed to read up on RBLs so that i could learn more about them... :) Perhaps I think a little too highly of myself?... but I admit, I was amused by that! PS - you mentioned that you were going to try greylisting next. Be careful because greylisting (1) can cause annoying delays and (2) some e-commerce systems, newsletters, and alert systems are not setup to do a retry--so order info can get lost due to greylisting I don't recall how SpamAssassin handles this.. but here is what I do to deal with those weaknesses: (1) in my scoring system, everything at 5+ is blocked as spam... AND I greylist ONLY those messages that score at 4.x (2) I massively whitelist many large ISP's MTA ranges from greylisting, partly because they are going to do a retry anyways and the ratio of spam blocked to legit mail delayed isn't so good with those senders. Between these two things, I get probably 90% of the benefits of greylisting, with only 10% of the problems from greylisting. Hope this helps! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: [OT] RBLs
On 1/12/2012 2:36 AM, Robert Schetterer wrote: > Hi Rob, read postfix archives > about rbls, there are tons of info > this all was discussed before, there is simply nothing "very" new > about this theme Thanks Robert. I'll be sure to read up more about RBLs, in case I don't know enough about them yet, or am missing something here... -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: [OT] RBLs
On 1/11/2012 5:10 PM, David B Funk wrote: > Problem with all those methods is that they're reactive, will not hit > until -after- somebody has seen the bad crap and created filers, > RBL-lists, taught Bayes, etc. > > The OP explicitly said that the first spam run was at 06:39 and by > 06:42 it was hitting RBLs (pretty darned quick by my book;). > However he has some fussy customers who weren't understanding and > so was asking for a method of dealing with this. This is actually a good argument for having a variety of good IP and URI DNSBLs. Even the fastest reacting ones are going to update, at most, once per minute. (and even that is rather rare... I think most fast-reacting ones update every ~5 minutes.) Even then, public DNSBLs have to rsync from the master to mirrors before the data is usable. For this reason, you're going to hit some DNSBLs just seconds after they updated... others are going to be a little less fresh. This is exactly why having multiple quality DNSBLs is helpful. If you check 8 different good ones instead of 2 different good ones (for example), then there is a greater chance that you'll query one of those mere seconds after it updated, and where it already had data on a new spam campaign. Along those lines, with the invaluement blacklists that I manage... we're soon going to offer a special version whereby we send an alert to "trigger" subscribers' rsyncs within a couple of seconds after each invaluement list's last update--thus making that reaction time even faster--and causing more spam that are at the "tip of the spear" to get caught. ALSO: There are OFTEN times when an IP doesn't have a chance to get caught, but it contains a domain already found on surbl, uribl, ivmURI, or DBL. Or, times when a domain hadn't had a chance to get caught yet, but the IP is caught from a previous spam campaign. But if you're not using all the best DNSBLs, you miss out on some of this! MORE: And, btw, really good /24 blacklists do _preemptively_ block much snowshoe spam, from the very 1st spam sent! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: What is the best RBL list?
On 11/28/2011 1:55 PM, dar...@chaosreigns.com wrote: > If there are better blocklists that are not used by spamassassin, please > open a bug to have it evaluated. Even if the data is not freely available, > it would be useful to list on the spamassassin wiki. Darxus, I'd love to have the invaluement blacklists included in such evaluations. However, we are about to implement our largest hardware/software upgrades ever. So it would be ideal to start such inclusion in January, after the upgrades are completed and associated bugs are fixed. (we're moving to 64-bit hardware and, where possible, 64-bit software!) I had thought that, at some point in the past, I was told that only freely available DNSBLs would be included in such testing? But if I'm wrong or that has since changed, I'd welcome the opportunity to participate. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com
Re: What is the best RBL list?
On 11/28/2011 2:25 PM, Robert Schetterer wrote: > Am 28.11.2011 20:17, schrieb Daniel McDonald: >> On 11/28/11 12:55 PM, "dar...@chaosreigns.com" >> wrote: >>> If there are better blocklists that are not used by spamassassin, please >>> open a bug to have it evaluated. Even if the data is not freely available, >>> it would be useful to list on the spamassassin wiki. >> The best RBLS for getting rid of snow-shoe spammers are from Invaluement, >> but it is available by subscription only. > the best rbl is the one you dont need, a case which is very rare these > days *g I guess I must be dumb because Robert Schetterer's last sentence above makes absolutely no sense to me. Instead, imo, the RBLs that you *do* need are the ones with (1) extreme few FPs and (2) which block spams that your other currently implemented RBLs are missing (particularly compared to those other RBLs w/extreme low FPs since RBLs with moderate-to-high FPs are either worthless, or can't be depended upon except for very low scoring... and that makes their unique "hits" not nearly as valuable as such hits are on a dependable low FP list). -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Good bye RCVD_IN_HOSTKARMA_BL
On 10/14/2011 12:05 AM, Marc Perkel wrote: > OK - I didn't deliberately blacklist them. I found a bug in my yellow > listing code No system or person or group of people is perfect and we ALL make mistakes... even big mistakes from time to time... and even large and famous corporations make big mistakes (i.e. RIM over the past few days!!!)... and even the best anti-spam blacklists make mistakes or have system errors on occasion! > that I have now fixed. ...of course, the more successful ones, like Marc Perkel's operation, learn from their mistakes... (and yes, Marc is right... the expectations for an anti-spam blacklist's consistency and quality can be extremely high--But I'm not complaining... just making an observation!) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: little off topic monitoring question
On 7/19/2011 8:50 AM, Thomas Mullins wrote: > > We would like to start monitoring our two smtp servers. They are > fairly busy boxes, maybe 100,000 messages a day, give or take several > thousand. They of course run Spamassassin, Postfix is also used. We > use MRTG to monitor internal servers and switches, and would really > like something with a similar graph. > I've been very pleased with www.websitepulse.com They do a round trip smtp-send/pop-retrieval. I get text messaged if this ever fails. I also used use them for http-checking my webmail. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: The one year anniversary of the Spamhaus DBL brings a new zone
On 3/8/2011 4:46 PM, Yet Another Ninja wrote: > I'll never grasp why one would use one of those in mail. Many legitimate social networks auto-generate shortened URLs. These then get copied into e-mails... sometimes in automated ways, sometimes via people copying a twitter post (or whatever) and then pasting that into an e-mail. Therefore, many of these make it into e-mails desired by recipients. At the same time, many of these URL shortening services are highly abused. For a while now, many of the most abused shorteners didn't have a very large footprint in legitimate mail. And the more legitimate services may have seemed to have much abuse, but really had a small amount in comparison to their legitimate uses. This made decisions to blacklist not that particularly difficult. As I understand it, what has occurred more recently is that some of the services which have a larger number of legitimate uses have had increasing amounts of abuse. In some cases, it seemed as though the abuse was flagrant... almost like the service felt it was "too big to list"... or maybe even was working with the spammers as partners in crime. Of course, I'm painting with broad strokes and not specifically mentioning any providers... but I think that is the context for understanding why SpamHaus felt that it was necessary to get more aggressive with blacklisting some of these services, even if some of those domains are found in legitimate e-mails. Now, with this other new zone, I think that Joseph Brennan when he stated that he could use this for scoring instead of blocking... for those redirectors which are heavily abused but have legit uses as well. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 11:14 AM, David F. Skoll wrote: > On Tue, 04 Jan 2011 11:01:52 -0500 > Rob McEwen wrote >> I've thought this through and... best case scenario is that spammers >> then get 5+ years of play time because it will take at least that time >> for those other techniques to catch up. > Umm.. no. We have plenty of effective techniques we're using right > now that don't rely on DNSBLs. I think if we stopped using DNSBLs > completely, a bit more spam would leak through, but it wouldn't be > catastrophic. Yes, it would be catastrophic. For one, it would bring the large ISPs down to their knees. Easily! Heck, currently, I know of one extremely famous and large ISP who isn't even willing to parse out the domains in incoming messages to check against locally hosted URI blacklists because that would mean too much resources per message. (and that process is extremely fast and efficient!). Many smaller ISPs *depend* on blocking at connection time with Zen and would likewise crash & burn if DNSBL-blocking at connection time wasn't feasible. >> When we are left with only whitelists and no blacklists, an >> interesting problem will happen... there will be extreme prejudice >> against ALL new IPs not already whitelisted. > Life will become more difficult, but it's not all doom-and-gloom. > By default, you should be able to get on the whitelist just by asking. If > it turns out you've abused the trust, you get banned for a long time. > That's essentially how the Spamhaus Whitelist works. You're exactly right. The reliance on whitelists would grow... those not on whitelists (like small businesses and start-ups) would be screwed. This would lead to a chicken/egg problems... how do you build up good reputation to get on a whitelist if you can't sent mail until you're on one. That will lead to a need for a more "easy on" whitelisting process.. abet even if for a less trusted level. But spammers will then often "sneak" onto that 1st tier... but at least it is better than dealing with the massive numbers of IPs no on there at all! But do you see where this is leading? right back to my original idea. Looks like you agree we'll get to my original idea anyways, one way or another. (although, I think it would be a lot less messy... and spammers would get much less listwashing accomplished... if we took a more direct route!) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Off topic: best RBLs to use to block at smtp connection?
On 1/3/2011 6:58 PM, mouss wrote: > as you can see, all DNSBLs but spamhaus are more or less useless. Mouss, [ignoring content filtering for a moment... per the original poster's request] If one DNSBL removed 90% of all spams, and that made a users's spam go from 100-per-day to 10-per-day, that is great... but the end users is STILL stuck with 10 per day. They go on vacation for a couple of days and they have dozens of spams to wade through. (but that is better than hundreds!). Next, your competitor used that same DNSBL, but added another very high quality and low-FP DNSBL that whittled that 10-per-day down to 2-per-day. Your customer complains about the spam and starts thinking about switching his service to your competitor after "comparing notes" with friends who used your competitor's service. Does the customer even care that much when you explain that you are doing a great job because you are already blocking 90% of the spam? BOTTOM LINE: In this example, this additional 2nd high quality DNSBL was probably only hitting on a tiny, tiny percent of the total incoming spam. But that is not always the best measure. We get fixated on the percentage of spam blocked using all incoming spams as the denominator. But sometimes it is a superior measure to use "remaining spam in the user's inbox" as the denominator because that is more of a "real world what the customer actually sees" measure. Otherwise, for example, if easy-to-catch botnet spam doubled and was easily blocked... and, at the same time, hard-to-catch snowshoe spam also doubled... but was often missed. Then, numbers-wise, using the incoming spam as the "denominator" in our measurements, we'd all be patting ourselves on the backs for all the spam we were blocking.. at the SAME time that the spam making it to the inbox INCREASED substantially!!! Something would then VERY wrong with our measurements of success! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 10:43 AM, David F. Skoll wrote: > I agree that it's probably eventually "game over" for DNSBLs, but not > for DNSWLs. DNSBLs are a pretty effective first-line defense against > spam, but they will gradually become less and less effective as IPv6 > becomes more heavily adopted. That just means we'll have to emphasize > different techniques. It doesn't mean doom. I've thought this through and... best case scenario is that spammers then get 5+ years of play time because it will take at least that time for those other techniques to catch up. Great damage will happen in the meantime. In the short term, old habits die hard. Without something like the solution I proposed, blacklists will die a slow and painful death. And most of the negatives I mentioned happening will STILL occur during the transition. When we are left with only whitelists and no blacklists, an interesting problem will happen... there will be extreme prejudice against ALL new IPs not already whitelisted. This will create a "chicken/egg" problem whereby a new startup company with new IPs will be screwed. (plus, small businesses will also have a hard time getting "respect" as they have have difficulty getting enough "traction" to get their IPs whitelisted!) Then, to fix the chicken/egg problem, someone will come along and create some kind of "IPv6 senders list" for new IPs that haven't been whitelisted yet. It will be extremely similar to my original proposed solution, since spammers will have no choice but to try to get on the "IPv6 senders list" list, too--but will be extremely limited in the volume of IPs they get on that that list. We'll then come full circle... except going with (something like) my proposed solution *now* instead of *then* means that we wouldn't have to have destroyed blacklists (and set spam filtering years back) in the meantime. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 9:31 AM, David F. Skoll wrote: > Right, but once your cache is blown, you're back to always querying > the authoritative server. John Levine proposes a fix with a clever way > to represent many entries with a small number of queries so you don't blow > your cache. I think making zone files available for download so you > can run your own authoritative servers is another good approach, especially > for whitelists. What I'm about to say could have been a reply to ANY of the past few posts... if the volume of unique IPv6 sending IPs causes either (a) DNS caches to get blown, or (b) requires John Levine's solution to prevent that... then... "game over".. the spammers have already won. And they are quite amused right now reading us discuss all different ways to rearrange the deck chairs on the Titanic. For example, if this happens, then... (1) effective DNSBLs will likewise bloat to such a large size that the resource requirements for running them (and transferring their data) will be insane (2) David mentioned "zone file transfers"... but that zone file would likewise be massively large and any client trying to load it would also have to consume large resources trying to load it. (this would also cause problems trying to sync DNS mirrors!). Consider also those IPs which ought to be blacklisted quickly, but don't because of having to wait on the previous mirror update? (3) What do we do about spammers who send spams to 100K or even a million addresses, using a unique IP for each recipient... and then "list wash" off the addresses which match up with those IPs that get blacklisted? Sure, you could blacklist whole blocks of that spammer's IPs.. but then you have to be able to do that without causing collateral damage in other situations where spammers and legit sender share IPs. Without going into details, tactics exist to somewhat deal with this... but massive numbers of e-mail addresses would get listwashed at the beginning stages of each campaign. IOW, no matter what, large-scale and EASY listwashing would be the norm with IPv6. (4) No matter how good you deal with the previous idea, you're STILL stuck with massive numbers of never-to-be-seen-again IPs bloating IPv6 blacklists! (again, the spammers are laughing at us!) (5) If anyone thought that my proposed solution a few posts back was unrealistic... consider that the extremely inadequate "partial fix" solutions that others are proposing involve even MORE intrusive changes to the standards/behaviors of our various systems (filters, blacklists, dns servers, dns protocol, mail servers, etc) As I said earlier, ALL of these problems have the "root cause" of too large a potential pool of sending IPs. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 1:57 AM, John Levine wrote: > I also don't think it's very realistic to expect that there will > be a master mail host file distributed periodically like HOSTS.TXT > was. There's a reason that the DNS was invented, and at the time it > was, there were a whole lot less hosts on the net than there are mail > hosts now. Others will take this and publish it via DNS (even if just for their own customers), which is a good thing. Also, such a list could be a raw list of just the IPs, nothing else, fwiw. (Obviously, every single spam filter "instance" worldwide wouldn't try to download that entire list.) > Finally, I presume everyone is aware that there is no possibility > whatsoever that RIRs will get into the business of selling mail host > IPs, so this entire subthread is 100% hypothetical. If so, then a (less advantageous) private solution will hopefully develop. But talking about "realistic", the "status quo" is already not realistic, even with the good ideas that you proposed, which did improve on this problems in _some_ aspects. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/3/2011 9:21 PM, Dave Pooser wrote: > Not to speak for Rob, but... Dave, You described my point quite well and I appreciate your help! What I described is vastly different than whitelisting and has massive "upsides". I haven't yet found any noteworthy downsides. Overall, this discussion thread took many twists and turns over the past few weeks. And tensions were often very high. But *two things became clear to everyone involved*, even those who disagreed with each other passionately over various things: (1) The problems caused by using IPv6 for SMTP are quite large, and there are *many* interrelated problems... RECAP: dns cache bloating--to a point of ruining ISP's caches, DNSBL bloating, bloating of resource requirements for serving IPv6 DNSBLs & distributing the massively larger data files, and dream-come-true scenarios for the spammers--such as giving them the ability to send a million spams each from an individual IP that is never to be heard from again--and that defeats ALL benefits of individual IP blacklisting AND gives the spammers a superb list-washing tool at the same time (In a /*single*/ e-mail campaign, they'll know /*exactly*/ which e-mail addresses are feeding DNSBLs!!!). Some of this was partially solved by blacklisting larger blocks... but no one came up with any good means for blacklisting of larger blocks that doesn't cause collateral damage. OVERALL--THE END RESULT IS AN ABSOLUTE NIGHTMARE!!! (2) These problem are DIRECTLY and ONLY caused by the available pool of IP addresses at a spammer's disposal having grown exponentially with IPv6--so much so that the possible pool of spam-sending IPs is obscenely large--and those IPs will be dolled out in vastly larger numbers than IP allocation under IPv4. Accordingly, my proposed solution VASTLY lowers this number of IPs that could e used for SMTP in a way that is amazingly inexpensive for legit senders, and extremely expensive for spammers who would try to burn through large numbers of IPs. Ironically, as I sat and read all the heated bickering... I was extremely happy that ALL of these problems were finally being openly discussed in a realistic fashion. It is about time. *And thanks, John, for engaging us on this and bringing to light these massive problems!* ...moving on... > John Levine said: >> Haven't you just reinvented whitelisting? No. As described, this is vastly different than whitelisting... and doesn't replace whitelisting in the slightest. >> Oh, and if you did that, how would MTAs check that an address of an >> incoming connection was on the list? That's the issue that started >> this discussion. > If it's an opt-in on a registrar level, I'd think that could be a host file > that's rsynced as infrequently as once or twice a day. I mean, as a > practical matter your MX records aren't going to propagate through DNS much > faster than that anyway. Exactly! The "master list" of "IPv6 IPs used for SMTP" would only update perhaps even just once per day, and would be available via rsync, ftp, http, etc (in some cases, compressed). BTW - Ironically, it is all the more of an upside that spammers could freely pay registrars for as many IPs to have "SMTP designation" as desired because, quite frankly, that is a lesser evil than the registrars ever getting "political" about who gets SMTP-designation. Since this is already an issue they deal with in their "regular" IP allocations, they probably already handle this quite nicely? In fact, I'd think it is in their best interest to just collect the money and laugh all the way to the bank... and not be bothered with denying SMTP designations, and, in this case, that is a GOOD THING!!! ALSO: With such a system in place, *IPv6 botnet-sent spam via dynamically-assigned IPs connections would be non-existent /by design/ from day one* (that alone is makes this idea worthwhile!) BOTTOM LINE: the IPv6 "status quo" is a spam sender's dream and a DNSBL's nightmare. My proposed solution is the opposite. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
John Levine said: >> Rob McEwen said: >> >> To be extra clear, the kind of sender's list I was talking about >> wouldn't be the same as a yellowlist because it would ALL types of IPs >> (black, white, yellow). Except everyone... including spammers... would >> have to jump through some hoops to get a single IP that list. But this >> /then/ VASTLY lowers the number of possible IPs that could be >> subsequently be whitelisted, blacklisted, or yellowlisted. > > Depends what your goals are. As soon as you add even the smallest bit > of qualification, you have all of the pain of any sort of policy based > list, people complaining that they're listed, complaining that they're > not listed, lying to you about whether they qualify, and it's not > worth the effort. An MTA sees a connecting client as white (accept > everything), black (reject everything) or color-to-be-named-later > (accept but filter.) As soon as you think a host is not pure spam, > you're done, since you'll filter it anyway. John, Please reconsider... and how about this twist... Let the IP registrars (arin.net, etc) add a very nominal fee for allowing networks to designate particular IPs as being used for SMTP. Perhaps something like this: $1.00/year/ip for the non-sequential IPs designated by an organization for SMTP usage $0.25/year/ip for additional /*sequential*/ IPs designated by an organization for SMTP usage (The first IP in a range would cost $1.00, others in the same sequential range would be 25 cents each.) Each IP registrar could then publish a master list of ALL participating IPs, which would be updated once daily, and could be downloaded as a compressed file via HTTP, etc. The ONLY criteria is the payment--and whatever each IP registrar /*already*/ does regarding designating IP ranges for organizations. This would absolutely keep the volume of IPv6 mail-sending IPs under control. Once that is done, 99% of ALL of the problems discussed on this thread disappear! Actually, a private organization could do the same thing... but they'd have two disadvantages over the IP registrars. (1) They'd be accused of "taking money from spammers", (2) AND... they'd have a harder time acheiving "critical mass". IP registrars already take money from spammers and no one things twice about that. And the IP registrars would have an easier time gaining "market share" on this idea and could quickly convince everyone in the industry to block e-mail that is not on that master list at the beginning of the spam filtering process. Then the DNSBLs would /only/ have to blacklist particular IPs from amongst those which /are/ included in that master list. ALL other IPs (not on the master list) would be completely be ignored by everything for SMTP, as if those IPs didn't exist. So there wouldn't be a need to blacklist them. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 2:28 PM, David F. Skoll wrote: > I in no way implied that we should abandon > IP address lookups in favour of only content-scanning Thanks for the clarification! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 1:55 PM, John Levine wrote: > it will clearly also be useful to > have what was called a yellow list a few days ago, hosts that send > enough real mail that you can't just blacklist them even if you see > some spam. John, First, let me mention that I'm grateful that you are working on this! We certainly need your help! To be extra clear, the kind of sender's list I was talking about wouldn't be the same as a yellowlist because it would ALL types of IPs (black, white, yellow). Except everyone... including spammers... would have to jump through some hoops to get a single IP that list. But this /then/ VASTLY lowers the number of possible IPs that could be subsequently be whitelisted, blacklisted, or yellowlisted. And even though you mentioned that FCrDNS wouldn't work as a spam filtering defense... things like FCrDNS could still be of used as a criteria for entry into this "master IPv6 sender's list" (as a means to keep the volume further under control.) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 2:09 PM, David F. Skoll wrote: > But I think it's really > stretching DNS way beyond what it was designed for and it might be > time to look at a different approach. But David, every example you've provided requires vastly more resources then blocking a spam with a single DNS lookup to rbldnsd. Heck, even a dozen separate DNSBL lookups against a local rbldnsd server is order of magnitudes faster and less resource intensive than accepting the entire message, and running that message against ClamAv (one of your examples). Many large ISPs simply will not ever spend that much $$/message. Otherwise, you'd have to convince the CEO of Comcast to increase their IT budget by 100x... and that would cut into profits... and he'd be fired by the board for that. (to give just one example) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 1:26 PM, David F. Skoll wrote: > Well, not really... John Levine proposes a way to summarize swaths > of IPv6 address space into very little storage, so that shouldn't be > an issue. While I'm not crazy about using DNS for this purposes, > John's basic ideas are correct. > > The real problem is the human effort needed to monitor the enormous IPv6 > address spave for abuse. I think it'll be hard or impossible to come > up with useful and comprehensive IPv6 blacklists. Does John's system do anything to prevent a spammer from sending a million different spams from a million different IPs (one-ip-per-spam) ...with that IP never to be heard from again)? (and with little or zero collateral damage?) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 12:47 PM, David F. Skoll wrote: > On 30 Dec 2010 17:13:07 - > John Levine wrote >> We'll have to change our software to handle v6 lookups no matter what, >> so I don't see it as a big deal whether it's a small change or a >> slightly larger change. > I agree, so I propose a much larger change: Stop using DNS for this > purpose. I don't think it's the right tool for the job. > > Any protocol that makes lookups in a huge adress space efficient and > efficiently-cacheable is going to leak much of the list information. > So why not just distribute copies of the entire list in a format that > permits efficient lookups and efficient sychronization (eg with > rsync)? Which leads to another potential problem If blacklists like CBL are currently at 100 MBs (for IPv4)... the bloat for IPv6 could break DNSBLs. RSYNCing Gigabyte (or terabyte!) -sized files is memory and CPU intensive. Loading those into rbldnsd is also resource expensive! Furthermore, getting that data out to DNS mirrors quickly and efficiently is going to be a nightmare! And... this requires that ALL mirrors be upgraded to accommodate the vastly larger size. A better solution (John, I hope you are willing to help with this...) involves some combination of the following three ideas: (1) create a standard whereby non-authenticated IPv6 mail can ONLY be accepted by certain IPs (such as x.x.x.0, if this were an IPv4 rule... translate that to IPv6... perhaps just one designated IP per /48 ??) Any other IP tries to send mail, and it get rejected if it isn't your own user doing SMTP AUTH. Btw - yes, you household appliance can STILL send an email... it will just have to SMTP authenticate to a more valid server first! (2) Why can't "Forward Confirmed reverse DNS" (FCrDNS) become a standard for IPv6? And we just all agree to reject anything less that this? (sure, some will ignore that... but if enough of us stick together on that... including a few large ISPs... this should gain critical mass!) (3) A shifting of focus on whitelists is important... but some of those shouldn't really be "whitelists" in the traditional sense. Instead, they should merely indicate that an IP is a candidate for sending mail. Call it an "IPv6-sender's list". Don't accept mail unless the sender's IP is on that list... THEN check that IP against an IPv6 blacklist. To get on the generic "IPv6-sender's list" is easy... but might require (a) FCrDNS, (b) filling out a CAPTCHA-protected form, (c) e-mail verification, using a non-freemail e-mail address, (d) NOT having a non-hidden registration for the domain used in the e-mail address, etc, etc, etc. At the least this prevent a spammer from sending a million spams from a million individual IPs, with each IP never to be seen again--and then bloating IPv6 DNBSLs with useless data! Yes, spammers will /easily/ get on the "IPv6-sender's list".. and if that bothers you, you've missed the point! Sure, you can /also/ have REAL whitelists... but they'd serve a different purpose. Now... who would run the "IPv6-sender's list"? ...I don't know. Even if just individual DNSBLs did this, that would be helpful!! Time is short. If these types of things aren't in an RFC soon, it will be too late. John, please feel free to take any of these ideas and put them in an rfc. No need to give me any credit. I doubt that I'm the first to things of these things anyways! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 and anonymity (was Re: Do we need a new SMTP protocol? (OT))
On 12/1/2010 12:55 PM, David F. Skoll wrote: > I don't see any nightmare. When DNSBL resources are order of magnitudes higher... when the largest data files for DNSBLs go from 100MB to probably Terabytes... and then trying to transfer that via rsync... and getting all the mirrors to handle loading that much data into rbldnsd... THAT will be a nightmare. (will Terabytes of RAM be affordable anytime soon?) > DNSBLs are a useful anti-spam tool that > will be made somewhat less effective with the advent of IPv6, but they're > by no means the only or most effective anti-spam tool we have. Not the only tool... but (particularly for IP DNSBLs worthy of blocking at the MTA...) they are the BEST tool from a price/performance perspective. In contrast, content scanning messages is comparatively resource expensive. I suppose a nation's military *could* fight a war without airplanes, without ships, and without missiles.. and just depend on the foot soldiers and tanks to do *all* the work. But is that wise? Does that happen without a steep price? We have a chance to impose some strict standards for mail sending on IPv6 that will lessen these problems. Why wait until its too late? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 and anonymity (was Re: Do we need a new SMTP protocol? (OT))
On 12/1/2010 12:05 PM, David F. Skoll wrote: > Where did you hear that? I can't imagine that > IPv6 is any less (or any more) anonymous than IPv4. One HUGE problem is that IPv6 will be a spammer's dream and a DNSBL's nightmare. A spammers (and blackhat ESPs) would potentially send out each spam from a different IP and then not use each IP again for YEARS! This will make DNSBLs much less effective.. and it will bloat their file sizes and memory/resource requirements exponentially. The DNSBLs will have no choice but to make their entire DNSBL the equivalent of a /24 list today... except painting with a much broader stroke, and many will complain about unfair collateral damage. Even then, the bloat will STILL be out of control. SOLUTIONS? Personally, I prefer everyone everywhere agree that, unless the e-mail is password authenticated to one's own mail server, all mail be rejected unless the mail server had IPv4. But purists won't like that because their goal is to eventually *end* IPv4. So what else could be done? If we must receive mail from IPv6 IPs, then I recommend doing the equivalent of the following (put in IPv4 terms for simplicity): (A) All other non-authenticated mail rejected... unless the message came from a "XXX.XXX.XXX.0" IP (this is in IPv4 terms... translate this into some equivalent IPv6 standard... but case a super wide net!) That will greatly reduces the number of possible valid mail sending IP. (again, auth mail to one's own server need not fulfill this standard) (b) industry wide, agree that mail is NOT accepted from IPv6 unless it does "Forward Confirmed reverse DNS" FCrDNS If one or both of those were agreed upon up front--this would go a long way towards preventing the coming nightmare. (and forgive me of RFCs have already established those as absolute standards for IPv6... I haven't kept up with all the RFC for IPv6!) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Blocking Senders with young domains
On 11/16/2010 4:34 PM, Michael Scheidell wrote: > there was/is a 'DOB' blacklist (day old bread). but I think the dns > servers may be overloaded. some people are complaining about timeouts. I think this was revived at the following web site (probably by a different person?): http://spameatingmonkey.com/lists.html Take a look at SEM-FRESH, SEM-FRESH10, and SEM-FRESH15. But keep in mind that *overscoring* on young domains is risky because, very often, a brand new web site using a freshly registered domain will go "viral" and get passed around through hand-typed legit e-mail. (But maybe the original poster's intention of using this on the envelop "from" minimizes that problem?) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: email address forgery
On 11/15/2010 10:22 AM, Daniel McDonald wrote: > I send from my phone just fine - Auth on the submission port to my > home servers, then SPF matches the policy just fine. Dan, You've made many good points. Not trying to take away from those. However, in spite of what you said about forwarding being true, there are ALSO many *common* scenarios where the forwarding isn't compatible with SPF. For example, many people have blackberry phones through their mobile provider and these don't send via one's e-mail account. Instead, they send/receive via a separate e-mail account... specifically, the account hosted on the mobile provider's blackberry server. But the user often wants to set a "from" address as being their regular business e-mail address so that they then get replies to blackberry-sent messages to BOTH their desktop computers and their blackberry. (otherwise, they'd only get the reply on their blackberry unit) Other examples include situations where someone auto-forwards messages from an old e-mail account to a new one. Or when someone sends a message via some type of social network... or even just a messageboard... and they want the "from" address to be their own address. (referring to legitimate situations here, not spam) But the sending server couldn't possibly be sending from an IP that the mail admin could have anticipated when setting up the SPF record. ...I'm sure there are others I haven't thought about! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: email address forgery
On 11/11/2010 7:41 PM, Noel Butler wrote: > Really? I don't use SPF in SA, only MTA, if that's the case, it is a > shame that SA also is behind the times. It was years ago SPF type was > ratified. Justin: Any plans to change that? I guess I'm one of those mail admins who is behind the times. But I don't really care that much because I take the same position as Suresh Ramasubramanian... that SPF is a failed technology because, for one, it breaks e-mail forwarding and there are ALWAYS too many legit e-mail forwarding situations (and legit substitutionary "from" situations--like sending from one's phone) to create problems in comparison to the problems that SPF solves. The ONLY exception is when enduring a severe "Joe Job" attack. In THAT situation, a strong SPF record will disrupt much of the spammer's messages, and cause them to switch to OTHER forged "from" addresses. In that situation, SPF is your friend. Otherwise, it is more trouble than its worth, imo. Because many feel this way, I suspect that this may be the reason why the lastest and greatest SPF support probably wasn' a huge priority for SA? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: email address forgery
On 11/11/2010 9:11 AM, Jeremy Van Rooyen wrote: > Can anybody explain to me how to do this and how would I be able to > test it? Jeremy, I really like to use the following wizard to generate my SPF strings: http://www.openspf.org/ Scroll down to the section that says "Deploying SPF", enter the domain name, and click "GO". Then, on the next page, fine tune the answers to the various questions before submitting the info to generate your SPF string. Finally, go into your DNS server and, for that domain, add that string as a TXT record. Keep in mind that, within the SPF record, you'll need to account for ALL IPs (or blocks of IPs) which are valid IPs for sending mail from that domain. Additionally, you are "stuck" in a precarious situation. If you don't specify "~all" (saying that the sending IPs/host you specified are the ONLY valid ways that mail should ever be sent from your domain), then... without being that strict, your SPF record probably won't have enought "teeth" to fix your spoofing problem. However, if you are that strict... and one of your users tries to send from an IP or host you didn't include in the SPF record (such as a user sending from a blackberry, but using their company e-mail address as the "from" address)... then you put these types of your own user's message at high risk of getting blocked by other peoples spam filters. For this reason, I generally try to only use such strict settings in extreme cases of spoofing. NOTE: When you say "spoofing", I assume you really mean a Joe Job--where a spammer is forging your users' e-mail addresses as the "from" address in their spams, correct? If yes, a strict SPF record can get the spammer to back of and go elsewhere. If something else, this might not help you? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Constant .info domain spam
On 10/12/2010 8:14 PM, Karsten Bräckelmann wrote: > [Added after re-reading: Same request. Which ones do hit, optionaly > which ones don't?] For the IPs mentioned: 217.23.6.209 204.45.150.196 64.32.6.4 173.234.224.131 184.107.29.11 72.55.165.139 67.159.50.131 174.37.134.225 ...here is a tally of *which* DNSBLs blacklisted these IPs, and how many of these IPs were blacklisted by each DNSBL: (see analysis below this list) NOTE: There were 8 different IPs. So the highest possible score was an "8 out of 8". # of "hits" blacklist name 7 ivmSIP 7 FIVETEN 6 BARRACUDA 6 Tiopan 5 PSBL 4 ivmSIP/24 3 NIXSPAM 3 OSPAM 2 BURNT-TECH 2 EMAILBASURA 2 KEMPTBL 2 SORBS 2 SWINOG 2 WPBL 1 AHBL 1 RATS-Dyna 1 SPAMCANNIBAL 1 SPAMCOP 1 UCEPROTECT1 I tallied this by checking each of those IPs on the mxtoolbox.com web site (one of the more popular free DNSBL looks sites), and gave credit for each hit. Keep in mind that this ranking does NOT take into account the FP rates of each of the lists. For example, ivmSIP and FIVETEN tied for first place. But, of course, ivmSIP is order of magnitudes a higher quality blacklist compared to FiveTen when you factor in a DNSBL's ability to avoid False Positives. Therefore, the BEST lists are the ones which scored high on this list --AND-- which also have low FPs. (for example, the one IP that ivmSIP missed really is a heavily abused IP... but one that also has MUCH legitimate use because it is used by one of the most popular dating sites for Latinos, which has 8 million subscribers. Therefore, MUCH collateral damage might occur from the blacklisting of this IP. Still, this can be a judgment call because sometimes "enough is enough" with some heavily abused IPs that have some legit uses!) Regarding that one IP, the DNSBLs which blacklisted 67.159.50.131 include FiveTen, Ospam, PSBL, and SORBS. Personally, I consider this to be the only False Positive of all the IPs submitted. And, for anyone who agrees with that analysis, this makes ivmSIP the /*only*/ list with a perfect 7 out of 7 score. But, again, considering 67.159.50.131 to be a FP is somewhat of a judgment call. NOTE: What this list is missing are DNSBLs like Zen. Obviously, the reason Zen is missing is because the person who submitted this list of IPs for missed spams probably ALREADY uses Zen-->so those spam /blocked/ by Zen won't show up on his list of /missed/ spams. And other DNSBLs may be in the same situation. For example, I suspect this mail system also uses SpamCop. So why the one SpamCop "hit" in the tally above? Probably because that one IP may not have been in SpamCop at the time the message arrived. (perhaps the same is true for UCE-1 and SORBS?--and would explain their 1 or 2 hits?) Along the same lines, some other DNSBLs that this mail system uses are not going to show up on that list at all, even if very good blacklists, like Zen--due to those DNSBLs already being used for outright blocking on that mail server where these spams were missed. That is the reason some lists are missing or under-represented. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Sought False Positives
Benny Pedersen wrote: > On fre 20 aug 2010 19:42:04 CEST, Rob McEwen wrote >> body __SEEK_2TRLES /Facebook, Inc\. P\.O\. Box 10005, Palo Alto, CA >> 94303/ >> >> which is currently hitting on many (or maybe even all ALL?) legitimate >> facebook notifications (along with the ones generated by spammers) > dkim signed ?, spf passed ? Yes and yes. But need I even check when I've already confirmed that they were sent from IPs assigned to facebook.com by ARIN? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Sought False Positives
I think the problem is the following rule in sought: body __SEEK_2TRLES /Facebook, Inc\. P\.O\. Box 10005, Palo Alto, CA 94303/ which is currently hitting on many (or maybe even all ALL?) legitimate facebook notifications (along with the ones generated by spammers) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: short pharma spam shoots straight through
Jason Haar wrote: > On 08/17/2010 01:04 PM, John Hardin wrote: > >> You might consider implementing spamhaus zen as an MTA-level hard >> reject DNSBL (I do that, maybe that's why I don't see any pharma >> spam?) - many admins trust it enough to do that, and the sample you >> posted hit on the abuseat CBL, which is a zen feed. >> > As per my initial email, none of the RBLs hit the message when they get > in. More precisely: Jason, Actually, for the sample you posted, ivmURI had the domain name in the clickable link blacklisted a whole two days BEFORE that spam was sent to you. Likewise, the IP was already in the ivmSIP/24 blacklist for MANY days prior to this... and probably BEFORE this IP was ever used for sending spam. (In fact, ivmSIP/24 WILL block many "zero day" spams, but without the FPs typically associated with most other /24 blacklists) But, then again, I'm obviously biased towards the invaluement lists. Therefore, for a better non-biased evaluation, take the ones which you keep missing and, extremely soon after they are missed... go to http://multirbl.valli.org and check both the sending IP, and any suspicious domains in the clickable links (I'd have suggested mxtoolbox or dnsstuff... but I know that multirbl.valli.org will allow for checking the URI in the clickable link against URI blacklists, in addition to the sending IP) If you can do that check literally seconds after your spam filter missed the spam, then you'll likely see which blacklists you aren't using would have blocked it. Of course, some lists might have the item listed, but are of little value due to such lists blocking too much legit mail. So ignore those! But any extreme-low-FP DNSBL that listed those missed spams prior to you receiving the spam should prove VERY worthy of your attention! If the others spams are like your example spam, you should see invaluement coming up over and over... and you might also see one or two other freely available, non-commercial DNSBLs come up as well that might help you (at no cost!), too! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: URIBL Notice
Yet Another Ninja wrote: > there are no users - its trap domains which have never had any real > users - ever. > > no prefiltering except rejecting potential bounces and stuff leaking > from whatever may be on DNSWL and a coupleof other WLs. Alex, Your stats are certainly valuable and illustrative... but not reflective of the stats one would see in a MOST "real world" mail streams where: (A) the spams were sent to actual users (which would be a distinctively different mix of spams compared to a pure honeypot stream of spams--for example, there'd be more "can-spam spam"/snowshoe spam in the real user mix, as well as less spam easily block by other techniques) --AND-- (B) where most of the messages have been prefiltered by FP-safe sender IP blacklists like Zen (which would then also alter the makeup of the spam stream--this would cause a lowered percentage of the "easy" stuff left over for the URI lists to process... and a higher percentage of the "hard" stuff left for the URI lists to process). Those two things would alter those stats dramatically and would paint a very different picture for some of those uri blacklists you compared. BTW - don't get me wrong... URIBL would still fare VERY well either way--so don't think I'm saying or implying ANYTHING bad about URIBL! (or anything bad about ANY other list) (fwiw) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: URIBL Notice
Yet Another Ninja wrote: > These stats are for small trap box which only accepts mail from bots > and rejects stuff listed by DNSWL and other public WLs. Since midnight > CET- > These are only URI BL tats - so you woun't see other dnsbls like > Spamcop, etc. Alex, about those stats... (1) Do those include spams sent to non-existent users (i.e. "dictionary attack" spams)? (2) Was pre-filtering done, such as collecting stats only on messages which made it past zen.spamhaus.org (etc.)? Or was there no pre-filtering? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: habeas - tainted white list
R-Elists wrote: > here is a chance for possible help in more areas than just this specific > ruleset issue... > > i asked Rob some time ago if he could write a script that would check logs > and report if a certain rule was effective or not by itself vrs if other > rules hit with it and maybe that rule was not needed or could be lowered etc > etc > > and if other rules hit with it, then we would see how effective that rule > was and why and when etc etc > > i am guessing that you folks already have these tools or similar tools or > help? > This is still on my "to do" list, but duties with invaluement.com only keep growing, so it is hard to prioritize this. But I find it hard to believe that this doesn't already exist. All that is needed is a plug-in that would copy to a specified directory all messages which hit on X rule (and/or dnsbl). The plug-in would be able to (optionally) only take action if the message scored either "at or above threshold" or "below threshold". Then, whenever testing a new rule/dnsbl, simply score it at 0.01, point the plugin at that rule or dnsbl, and have it only act on messages which scored "below threshold". This would be extremely valuable for determining the following about a new rule or DNSBL: (1) How much spam the rule would have blocked if being used aggressively (but was missed with the 0.01 score) and, therefore, made it to the inbox during the testing phase because nothing else in production had stopped it? (2) How many legit messages would have been blocked with the use of this rule or DNSBL? (FPs) Of course, BOTH of those examples would consist of messages which scored "below threshold" even while hitting on that new rule (given its 0.01 score). So it would be up to the e-mail administrator to then examine the messages and judge for themselves whether these were FPs, or would-have-missed-without-the-new-rule spams (aka corrected FNs). If anyone ever develops such a plugin before I have time to, PLEASE let me know! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: [sa] RE: emailreg.org - tainted white list
jdow wrote: >> jdow wrote: >>> his response personal spam to this account has increased sharply >> Uuh, what does that mean, exactly? > A possible cause and effect exists. I can neither prove nor disprove > it. the fact exists. Still doesn't answer my question. Perhaps I'm "dense". But to spell out my question more explicitly: what do you mean by "personal response spam"? Is that just Richard's on-list responses we've all seen? Or something else? (did I miss that part of the conversation?). And what do you mean by "to this account"? To this list? To your own inbox? Are you referring to messages that are obviously from Richard (including alter-ego ones)? Or some kind of UBE campaign that you think he is behind? (if so, please describe) Still confused. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: [sa] RE: emailreg.org - tainted white list
jdow wrote: > his response personal spam to this account has increased sharply Uuh, what does that mean, exactly? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: emailreg.org - tainted white list
If I ever do anything questionable, or not ethical, or even illegal, I hope that Richard is the one to call me out on it publicly because once he's confused issues with his personal insults and his best "Art Bell" impression, I'll then come out smelling like a rose. If he can ever stay banned, I won't miss the personal insults, I won't miss his "holier than thou"/"us against them"/all-or-none positions & attitudes, and I certainly won't miss the endless argumentative threads he inspired about seemingly nothing (imo). But I will miss (a) the entertainment value of some of his posts (his "dark forces" one from earlier today was a classic) --AND-- last but not least--I will miss his willingness to break through the political correctness and bring up various points that few others were willing (or brave enough?) to point out. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: emailreg.org - pretty good white list
Marc Perkel wrote: > I see no reason that everything has to be free. Ultimately we all have > to eat and we do something to make a living. > > There are people in the world who are both ethical and financially > successful. So if someone is doing something right and making a buck > at it I don't have a problem with that. I agree 100%. But that is not really the issue here. The issue has more to do with how to set up those business models such that good behavior on the part of the whitelist maintainer is 'incentivized' and bad behavior by the whitelist maintainer is 'disincentivized'. Therefore, generally speaking, it is at least very difficult for any whitelist which involves payment-then-removal to be a highly ethical operation, imo. Not saying it can't be done, but this is not normally how pay-for-removal works out. Return Path's certification program is probably one of the best examples of this working out, but that is mostly because (a) Return Path has sufficient # of high-end and ethical customers such that they are 'incentivized' to dump any low-quality customer that comes along so as to not sully their reputation with their high profile customers, and (b) Return Path's whitelist is more valuable if used by more spam filters--and they lose THAT market share if they allow mainsleaze spammers on their whitelist. These two things provide incentives for Return Path to run an ethical list. Obviously, Return Path and emailreg.org have very different business models, but I haven't heard very much similar reasoning for how/why emailreg.org is also properly 'incentivized' for good behavior other than "trust us", "$20 isn't much money", "we promise, we remove spammers", and they do have some good hoops that prospective customers must jump through (proper rDNS, etc). But, as I said, I highly trust my well-placed contact who vouches for emailreg.org, so I'm satisfied. My main point--yes, having revenue is NOT a bad thing--but that doesn't mean that certain business models for various whitelist/blacklists don't sometimes 'incentivized' bad behavior--and when it LOOKS like it is happening, I think the anti-spam community SHOULD ask questions! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: emailreg.org - pretty good white list
Bob O'Brien wrote: > But I have to say (and this is just my personal opinion) that all the > people shouting "conspiracy!" (even if joking about it) may have done > irreparable harm to the potential for corporations (not just Barracuda) > supporting this community in the future. Bob, Someone I have great respect for has vouched to me (off-list) that he has inside personal knowledge of emailreg.org and that he knows for 100% positive that this is well run, very ethically run, and NOT pay-for-play (or something like that--still trying to figure that last one out a bit). Nevertheless, given this person's confidential assessment, I am now convinced that there are honest and altruistic intentions behind emailreg.org and I'm convinced that those running it must be highly ethical and competent. (I'm still distrustful of the _quality_ of ANY whitelist which involves payment even if the intentions are honorable, but that is just my personal taste.) However, Bob... regarding your comment above, you have your own self (& associates) to blame. The things that have made people suspicious were real and noteworthy and did NOT take a nutcase to jump to harsh conclusions. Then, when these things were pointed out across several threads spanning many, many months--it was at first like pulling teeth to get answers. Finally, the answers that did eventually come forth were initially somewhat cryptic and evasive, which only pored gasoline on the fire, imo. If it were not for that off-list vote of confidence from someone I greatly trust, I'd still have lingering and suspicious questions. (or maybe not since I starting to fatigue on this subject.) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: J.D. Falk & Richard dispute (was J.D. Falk...)
I'm just changing the subject line because I find the previous subject line to be extremely offensive and out of line. - As long as we have some spam filters which block some legitimate confirmed opt-in senders (and/or legit organizations sending to their unquestionable members), then that makes Return Path's business model legitimate and helpful. If anyone believes that Return Path's execution of this business model ends up giving some spammers a "pass", then they should "shame" Return Path by pointing out the most egregious examples that come along. But it is understandable that a few undesirable situations are going to happen every once in a while, no matter how good and ethical a job is done by Return Path. So an egregious example that comes up every once in a while is understandable. (just like it is understandable for a legit hoster to unknowingly and occasionally sign up a spammer who deceived the hoster--happens all the time!) As long as Return Path reacts appropriately to such spammers, and as long as they are not a constant revolving door for many spammers (or anything close to that), then I don't see any problems here. I do understand the argument that their business model might provide incentives for them to be unethical in the short run just to drum up extra sales, but this is balanced by the longer-term damage this does to their reputation. Amazingly, I deal with black- or "dark gray"-hat ESPs blacklistings on invaluement.com where the ESP is run by 20-something-year-old punk kids who don't understand the long-term negative repurcussions of their business practices and seem to think that they can spam with impunity as long as they are CAN-SPAM compliant. But, in contrast, Return Path is run by rational and mature adults who "get it", imo. For the reasons stated, I reject the ridiculous argument that their business plan makes them unethical. But I do believe that it is helpful if/when the anti-spam community points out their most questionable clients, if/when deemed appropriate. That will only help inspire them to further tighten their standards and keep them accountable. (actually, I do NOT personally see any current deficiencies with them--but I'm just saying that this is a productive way of dealing with any problems anyone has with Return Path that will have a tangible good results for the industry as a whole.) So, instead of insults, if anyone has a grip with them, please just point out SPECIFIC examples. Over time, if you find many egregious ones, that will speak for itself. Otherwise, I'd prefer to not be bothered with this. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: JMF_W & URIBL_BLACK
Alex wrote: > for both JMF_W > (HOSTKARMA_W) and URIBL_BLACK in the same message. I'm not involved in the management of either of these, but I have some analysis which I think is accurate: (1) Marc Perkel's domain whitelist is auto-generated. This has many advantages... but one disadvantage is that some very dark-gray (or even blackhat) are going to get on that whitelist more because they were very shrewd and sneaky--but their messages were 99-100% UBE and NOT desired in user's inboxes (but the messages *looked* legit enough to keep the complaints down--a common situation) (2) At the same time, uribl is (2.a) very good at listing just those sort of sneaky ESPs who deserve to be blacklisted and are often missed by other lists ...AND... (2.b) uribl also sometimes goes a bit too far and lists some ESPs and hosters who have some legit uses, but send much spam I think 2.a is happening more often here than 2.b Regarding 2.b ...HEY... there are some really hard judgment calls here that could go either way. For example, I'm starting to notice many dark-gray ESPs who are sending 90% UBE... but the 10% legit mail they are sending are *pure* *advertisements*... NOT things like order confirmations, etc. There is then a strong argument that the collateral damage of blocking that 10% pure ads really isn't that harmful in return for the benefit of blocking the spams--since the end users are also going to like such a tradeoff. But this still isn't easy because sometimes that 10% involves large and famous companies (like AT&T recent use of [withheld]'s ESP services) And there are other examples which are a much harder to "call". But i think this well explains the overlap between URIBL-black and HostKarma's domain whitelist. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Constant Contact
R-Elists wrote: >> Complaints liks this keep coming up for various whitelists. >> The usage alternative I just suggested may solve this problem >> for many people. Just what I said. If an IP whitelist cause too many spams to get a "free pass", then instead of using that whitelist as a free pass to the inbox... instead... use it to bypass all checking of the sender IPs against blacklists, but still do content spam filtering on the message. This is actually what Marc Percel recommend with his "Yellow" list. I'm simply stating that this approach is good for additional whitelists if/when someone likes the whitelist overall, but find it leads to too many FNs. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Other DNSBL's
> ask michael scheidell... he has a list for you that is 100% effective... yeah, like that same joke that grandpa keeps telling over and over.. the first time it was a little bit funny... but now it is annoying, particularly the way he is the only one in the room laughing each time. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Constant Contact
Adam Katz wrote: > Does anybody here know anything about the legitimacy of Constant > Contact <http://www.constantcontact.com/anti_spam.jsp> ? > Sometimes abused, but too legit to outright block based on sending IP, imo. > The biggest problem is that they're well seeded in the DNS whitelists, Many of those whitelists are better used as "don't check the sending IP against RBLs, but do all other content spam filtering"... and should not be used as a "skip filtering and send to inbox". Complaints liks this keep coming up for various whitelists. The usage alternative I just suggested may solve this problem for many people. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Harvested Fresh .cn URIBL
Blaine Fleming wrote: > I know my users never see .cn domains in their inbox > and if I didn't run a blacklist I wouldn't either. Which brings up an interesting idea. I wonder how many legit non-spam .cn domains exist? Surely it is a fraction of a percent of the # of .cn domains used for spam purposes, correct? If that assurtion is right, then maybe it makes more sense to build a really good and comprehensive ".cn" whitelist. Then create a rule in SA whereby ".cn" domains not on that whitelist would add a point or two to the score. (it shouldn't be used to outright block due to the "guilty until proven innocent stance!"...and it shouldn't be a default SA rule) I might be interested in maintaining such a freely available list--accessible via rsync at no charge--if someone else would come up with the SA rule or plugin. My unique contribution could be providing a means for my own "invaluement URI ratings engine" to rate potential candidates for whitelisting--this would separate most of the wheat from the chaff with little effort--just as long as the entries submitted was kept to a reasonably low volume. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: consolidating DNSBLs into a single query (was Spam Eating Monkey?)
Mike Cardwell wrote: > I don't understand the logic of that. Ie, why you'd need to use > bitmasking? zen.spamhaus.org is a combination of various different > lists and returns multiple values like this: If every list is an "outright block" list, then you are correct. My point applies to situations where some lists are used in scoring mode, and where there is a desire to be able to calculate a score based on exactly which lists hit on a particular sending IP. But even if someone tries this with all "outright block lists", and uses rbldnsd's built in ability to consolidate lists, then there are still two problems: (a) for auditing purposes, there'd be no way to tell *which* lists hit on that IP since many use the same return codes (b) some hundreds-of-MB-large lists which previously could have used the lower-memory "ip4tset" would have to revert back to slower and higher-memory-usage "ip4set", fwiw Again, not saying these problems can't be solved, only pointing them out so that anyone who cares to try can know what they need to do, or need to expect. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
consolidating DNSBLs into a single query (was Spam Eating Monkey?)
Warren Togami wrote: > You are misunderstanding the question. A single DNS query could > respond different numbers meaning they are hits on different lists. > Your lists that are subsets or supersets of other lists can easily use > this. The querying software need only to know what each result means. Not saying that this is a bad idea, but it does have its limitations. For example, some lists are into the hundreds of megabytes large, and getting the whole file rsncned and updated can take more than several minutes. Often, such lists update only once or twice per hour, if even that often. In contrast, some lists are smaller and faster reacting and update every few minutes. Trying to merge all such lists into a single lists every several minutes is no trivial task in terms of having enough CPU cycles and RAM to get that done correctly and within a reasonably short time. Likewise, doing the merge hourly loses the benefit of some of the smaller-footprint faster-reacting lists which can react to emerging spam threats faster. Not saying such a consolidation can't be done... and maybe a few tradeoffs here are worthwhile? But if these issues are not dealt with smartly and competently, then one could easily find themselves with that all-in-one comprehensive DNSBL has not being as effective as querying them separately. Also, this loses the ability to *score* on multiple lists... unless you use a bitmasked scoring system whereby one list gets assigned ".2", another ".4", another ".8", on to ".128". But that leaves a maximum of only 7 lists. Sure, you can add more than 7 by employing other octets in the "answer IP", but that only severely complicates matters. And as it stands, you'd also have the complexity of getting the spam filter to parse, understand, and react properly to those bitmasks. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Can I auto-delete emails scoring 10 and above, yet mark as spam those 5 and above?
drkwc wrote: > My "Why" is that I get 1,500-plus spams daily on some of my older domains. One thing you should double check is that you are blocking all dictionary attack e-mails. What I mean is that a certain percentage of spams are making wild guesses at the particular recipient's e-mail addresses. If you have a "catch-all" account which says, "if that alias doesn't exist, when that message here", then all those wild guess dictionary attack spams will flood that mailbox. So make sure that you have that "catch-all" feature (which is now a 'bug'... it used to be a feature a decade ago) turned off. That way, all spam sent addresses which don't exist will get rejected by your mail server before ANY kind of spam filtering is needed. This is much more efficient than sending these through spam assassin! Or, am I wrong and these 1,500 spams really are being sent to actual legit e-mail addresses? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com
Re: What does "conservative" mean vs "aggressive"?
drkwc wrote: > "conservative" means vs "aggressive". "conservative" = setting a higher spam score threshold to ensure that LESS legitimate mail mistakenly blocked, even if that means that more spam makes it into the inbox "aggressive" = setting a lower spam score threshold with a goal of blocking MORE spam, even if it means that the amount of legit mail (FP) being mistakenly blocked increases Generally, there always a trade-off. HOWEVER--adding to SA just the right combinations of add-on anti-spam tools and non-default DNSBLs can often defy that trade-off and give you more of the "best of both"... that being more spam blocked without incurring a corresponding increase in FPs. Many (including many on this SA list) have tweaked the SA default settings, including adding extra add-ins and DNSBLs, and then achieved higher spam catch rates without a corresponding increase of FPs. But lowering the score (alone) to block more spam will definitely lead to more FPs. And raising it (alone) will lead to more spam getting into inboxes. In fact, Justin Mason has an excellent blog post on the effects of raising or lowering your SA spam threshold on both spam catch-rates and FP rates. He includes some excellent graphs. Read about it here: http://taint.org/2008/02/29/155648a.html -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Barracuda RBL in first place
Michael Hutchinson wrote: > So perhaps instead of adding another RBL, maybe some admins need to > consider adding in some HELO checking / rejection. Michael, Your suggestions are wonderful. These techniques will block more spam, with relatively few FPs, and their implementation doesn't require adding DNSBLs. But I'd like to clarify one thing. Whether intended or not, your post *implies* that the use of such techniques removes the need to add additional DNSBLs beyond SpamCop and Zen (or something to that effect). I respectfully disagree for the following reasons: (a) Your stats mention what you do catch... but don't factor in what spam you are delivering to the mailbox, some of which may be flying under your radar. For example, I've had some reluctantly test out the invaluement lists... "reluctant" because they were convinced that they were already blocking 99.9% of all spam... but then they were shocked at how much spam the invaluement lists blocked that was previously not blocked and unnoticed. (users tend to not complain as much about the legit-looking spams--especially vertical market stuff in the user's industry, even when 100% UBE). These previously-missed spams were ones that HELO filtering techniques (of the kind you mention), and other things like greylisting, would *not* have blocked. (b) Some of those who posted invaluement stats on this thread earlier... are *already* using some or all of the techniques you mentioned (and more) *before* their filters checks sending-IP DNSBLs. Obviously, I've personalized this... but a broader point can apply to any DNSBL. Just because one or more spam filtering techniques are effective doesn't necessarily make any DNSBL obsolete unless/until that DNSBL is tested and found to _only_ block spams already blocked by those other techniques--and assuming FPs are equal--then and only then does particular filtering methods make a particular DNSBL obsolete. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Score -71 for VERY spammy message!
snowweb wrote: > USER_IN_WHITELIST That probably has something to do with it. And make sure you haven't whitelisted your own user because it is common for spammers to put the recipient's address in there as the "from" address, knowing that some portion of administrators will have whitelisted their own e-mail address. This then become a free trip to the inbox when the spammer puts that address in the FROM header.. If you want to make sure you don't block your own users outgoing mail, use SMTP password authentication instead. Don't rely on an easily forged FROM e-mail address. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Extending XBL to all untrusted
I agree so strongly about not checking against all IPs in the header that I'll probably turn down business from large anti-spam vendors who cannot guarantee in writing that ivmSIP and ivmSIP/24 will ONLY be checked against the actual sending IP. If this means I lose 4-5 figures in annual revenue from future vendors, so be it. (and I don't think any of my current largest subscribers are doing this.) There is a better system. Work to find ways to better know which headers are forwarders, ignore them, and grab the original sender's 'mta' IP from THAT received header. (not IP the workstation which originated the e-mail, but the mail server IP that officially sent the message on behalf of the sender, but before any other forwarding). This "surgeon's scalpel" approach is not always as easy as the alternative sledgehammer approach, but it is worth the effort. Certain large anti-spam appliance vendors have no excuse for not making this extra effort... and I've seen some egregious FPs (for example... hand-typed messages from an attorney to their client, sent from an IP which doesn't ever send spam) recently caused by such appliances which check all IPs in the header against blacklists. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Independence Day - Barracuda SA Rules & White List
Dave Pooser wrote: > I like Rob despite his sadly misguided politics :^) Being the 4th of July holiday, I should proudly point out that my politics are much closer to those of Washington, Madison, Jefferson, ...even that "populist" Andrew Jackson, etc... and the documents they authored, which secured our freedom and liberty... in comparison to what the average American today is (unfortunately) brainwashed to believe by their Government-run schools and Universities. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com
Re: New Comcast Postmaster Link
Jeff Koch wrote: > We have a mailserver that's been blocked by Comcast. After correcting > the problem (user got hacked) we need to get Comcast to lift the > block. Has anyone got the new URL for this? Comcast seems to have > revised their website and the link provided in the bounce reports does > not work. http://postmaster.comcast.net/ -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: BOTNET timeouts?
Bill Landry wrote: > This issue has been unresolved for way too long. All of this, in my > mind, this makes the plugin orphaned and unusable if not patched with > Mark's patch. No matter how hard you try to improve botnet: (A) botnet is still dependent on third party dns servers, many of which are poorly configured, overburdened, squeezed of bandwidth, etc. (B) Even if that were not true, botnet:STILL won't ever "scale" that particularly well (C) By design, botnet is always going to be "asleep at the wheel" with regards to correcting its own mistakes. What i mean by this is that... consider a DNSBL which misfires on certain IPs on rare occasions due to part poor rDNS configurations of the senders... so that the sender is at least partly to blame BUT... where it is determined that listing the IP would block little-to-zero spam, and would generate many FPs. With a well-run DNSBL, there is a feedback mechanism to the DNSBL operator--often the sender can get alerted to their problem--and the sender has a means to figure out the source of their problems more easily--and once that feedback (from senders or recipients) gets back to the DNSBL operator, an exception can then be made for such IPs so that they can then stay off the dnsbl. Botnet doesn't have these types of checks-and-balances or feedback mechanisms which lead to critical and justifiable adjustments for some of those exceptional cases. Not that botnet isn't useful... and I think the concept is wonderful and the author should be praised... but anyone trying to use the botnet plugin as the "end all" replacement for DNSBLs, or the "bridge all gaps" from their existing DNSBLs' shortcomings... should be aware of these limitations I mentioned. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: New slew of spams
Jeremy Morton wrote: > I've suddenly started getting a new slew of spams that are making > their way through my SpamAssassin filter. Here's an example of one: > > http://pastebin.com/m586e296c > > As you can see they tend to hit a couple of blacklists, but don't get > a high enough score to be marked as spam. What do your SpamAssassin > analyses give of this e-mail, and any tips as to how I can get these > marked as spam? I highly recommend scoring RDNS_NONE at much higher than "0.1", and scoring RCVD_IN_PBL at much higher than 0.9 If you don't feel comfortable having these combine to score higher than threshold, then consider bumping each of these up at least by a whole a point or two... and then add a metarule that might add an additional point or two if BOTH of these have been triggered. An occassional legit e-mail will have RDNS_NONE, and an occassional legit e-mail will have RCVD_IN_PBL. But even extreme fewer legit emails will have hits on BOTH of these. So I'd suggest scoring the combination of the two either just above threshold, or (at the least...) just below threshold. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Barracuda Blacklist
John Hardin wrote: > It might be less confusing if that ad was presented *after* you've > completed the traditional unlisting request... Good point. And I also wonder, how many emailreg payments were made by disparately frantic e-mail admins who normally don't ever send spam, but had a security problem that warranted their initial blacklisting, but where the security problem was already fixed. And I wonder how often those types would have been delisted anyways, but the sysadmin was disparate, rushed, and willing to do anything, including paying $20, under those circumstances? Additionally, I'd like to ask, other than being a superb cash-generating machine, what good is a whitelist built upon pay-to-enter and NOT based on editorial decisions made by non-biased e-mail administrators? At some point, pay-for-whitelisting will likely lead to FNs as well as "free passes" for dark-gray or blackhat ESPs. It may also lead to FNs the next time that same email admin I described has another security hole spewing out millions of spams months/years later. (do they then get a free pass due to the payment to emailreg?) Really, I find this whole conversation quite bizarre. It reminds me of a joke I once made to my wife about how I felt led by the Lord to minister and share the Gospel to strippers at strip clubs. There'd be no lust or adultery involved on my part. Na. Just genuine concern about saving those lost souls. Likewise I'm sure emailreg.org is just a whitelisting service trying to give back to the community and help those poor innocent system admins from getting unfairly blacklisted in the future, right? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Barracuda Blacklist
Karsten Bräckelmann wrote: > We're not going down the path of judging blacklists based on whitelists > or certification services, or vice versa, do we? > If the whitelist involves possibly questionable business practices (trying to reserve judgment here), then the information that Neil provided _should_ be factored into any such decision. One thing is for sure, (extortionist or not) Barracuda (or whoever owns emailreg.org) is laughing its way to the bank. The more SA usage of its list, the more $$ that goes to emailreg.org... I'm sure that they will be very happy if/when BRBL gets added to SA by default. And to not factor this into such decisions... and turn a convenient blind eye... is tantamount to the SA community acting like a bunch of sluts... grabbing onto any freebee spam tool, regardless of these other implications. However, if it can be shown, after careful consideration, that everyone (or the SA powers that be) is OK with BRBL/emailreg.org business practices... that is one thing. But to sweep this under the rug is another very very sad and possibly unethical thing. > BTW, Neil, may I remind you... "red herring" -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Plugin for URL shorteners / redirects
Jason Haar wrote: > Why can't SURBL be expanded to support > full URLs instead of just the hostname? That way you could blacklist > "a.bad.domain" as well as "xttx://tinyurl . com/redirect-to-bad-domain"? > Some form of BASE64 encoding would be needed of course, but why not? Because spammers could easily generate a unique URL for each individual spam. They could then map this back to listings in URI blacklists and use that as a very cheap and effective way to listwash. And they only need to add a single astricked hostname in their DNS server to accomplish this. As a result of this and similar tactics, URI lists would bloat exponentially and this would slow down the propagation of the data to rsync users and to DNS mirrors, as well as bringing the backend processing to its knees. Finally, there is some amount of reputation and registration (even if hidden) associated with a domain due to the fact that a domain *requires* ownership. URLs and subdomains are more ambiguous, which then also makes removal requests extremely subjective and murky process. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: I want MORE SPAM - MORE SPAM
Aaron Wolfe wrote: > +1 for the invaluement lists. they are excellent, sad that they > aren't listed in that comparison. we seem to get better results with > barracuda than you've seen, many of our clients choose to use the > barracuda list to block. we offer the hostkarma lists as well but > probably introduced them too soon, the FPs were high back when we > first offered it and most clients chose to score only if they use them > at all. I am going to re-evaluate though, and maybe recommend to some > clients. I have heard that the FP rate has improved. The main reason ivmSIP is not in Jeff Makey's comparision chart is the same reason it is not in Al Iverson's (not yet repaired) charts. That is--ivmSIP doesn't try to win the "most spam caught" game. Instead, it seeks to catch the spams the other sender-IP dnsbls are missing, while striving to attain FP levels (at least) as low as those of Zen and SpamCop. At the least, ivmSIP strives to have fewer FPs than psbl, uceprotect-1, hostkarma, BRBL, njabl, sorbs --and fewer FPs means the ability to either score higher with ivmSIP, or outright block spams with ivmSIP. ivmSIP is also particularly good at catching snowshoe spams--and catches a significant amount of those and other spams missed by ALL of the other lists. But start measuring ivmSIP's "raw" hit rates against these others and ivmSIP doesn't look so hot anymore. This would, therefore, give an undeservedly bad impression if ivmSIP were on those charts--where "unique hits" and FPs are not taken into account (or not emphasized) In contrast, HostKarma does a great job of listing a *large* number of botnet IPs, many of which are often not yet listed on CBL/Zen. And Marc is doing a great job of reporting zombie IPs to various ISPs--which is helping to close down zombies--which then helps EVERYONE. Plus, unlike the invaluement lists, the hostkarma lists are freely available for direct query (for those serving <= 1000 email accounts AND making <250,000 DNS queries per day --OR-- free for those who participating in sending him data... contact Marc for more details on that!) Because of the different goals/strengths/weaknesses, anyone using ivmSIP and ivmSIP/24 could still benefit from HostKarma and these other lists, and vice versa. The key is gaging each list's score based on that list's ability to avoid FPs--combined with one's tolerance for FPs. For example, in my own spam filtering, I recently started seeing far too many FPs where legit hand-typed messages were hit on by two of the DNSBLs I mentioned above (not talking about hostkarma, btw). I had each of them scoring "below threshold"--but both together had a score that was just above threshold and this was causing legit mail to go into spam folders. That inspired me to back off of those two DNSBLs by one point each. But, even with their lowered score, I still find each of those two lists very helpful for putting many spams just over the top in their scoring, thus reducing spam to my users, but without causing FPs. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: my emailBL is live!
Jesse Thompson wrote: > A word of caution. Be very careful how you use the list. OK. I was wrong. Due to this discussion, I'm convinced that MD5 of the whole (lower case!) e-mail address is best, with the entire e-mail address still showing up in plain text in the DNS txt record. But I have some questions: (1) is MD5 of the entire address reasonably safe from collisions. (consider the 'birthday paradox' before being too quick to answer) (2) I'm also interested in knowing more specifics about the data found at http://anti-phishing-email-reply.googlecode.com/svn/trunk/phishing_reply_addresses (2.a.) how frequently are new scam addresses added to that list? (2.b.) how long does an address take to expire since the last e-mail address is used for scams "in the wild" (2.c.) Is the data auto-added? or must e-mail addresses go through a manual review first? (2.d.) Moreover, what is a typical time between the "419" spammer's last spotted use of the e-mail, and appearance in that list? (I don't need exactly precise answers which spammers might use to 'game' the system... just basic estimates will do) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: emailBL
Ben Winslow wrote: > If you're worried about spammers gaming the hash system Most likely, they won't care. They'll happily pursue the "low hanging fruit". The only exception is if/when freemail ISPs started using such a list to start investigating individual accounts for possible termination. But, even then, that is a good problem to have. Personally, I think the obfuscation is overkill. Instead, I'd prefer to change the "@" symbol to an underscore (and any other minor change that might be needed to work with dns queries) and be done with it. This would also make the implementation easier, and research by ISPs easire. As with all DNSBLs, the really hard part is not listing legitimate items. For example, consider that guy out there is probably sending financial newsletters to his very own clients, uses his ISP's MTA for sending, but uses a gmail "from" address. His e-mail address might have a high chance of being mistakenly blacklisted! The last time 2-3 times I saw this idea come up on either SA or Spam-L, I recall that the idea was strongly shot down by a number of people for this and other reasons. But I kept out of the discussion and I actually thought this could be a great idea... if done right and if FPs are kept to a minimum. I'd been planning on starting such a list for quite some time, but it kept getting delayed by more urgent needs. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: emailreg.org
SM wrote: > Are you upset because people are paying money to a site with a domain > owner hidden by the Whois privacy registration? :-) Some antispam > offers are big and easy money as there's always somebody ready to pay > or to jump on the bandwagon because it is free. I don't understand your last sentence above. It seems to make no sense. But, regardless, I have been given both legal council and business advice (each coming from different perspectives) that I should make no further public comments regarding the operation of BRBL and emailreg.org, at least not on a forum as public as the SA list. EXCEPT TO STATE: Who knows much of anything for absolute certain about this situation? For example, it is entirely within the realm of possibility that emailreg.org is a separate non-commercial and non-profit organization (as the ".org" seems to imply?). And maybe emailreg.org really is a separate entity from Barracuda (as the Barracuda's BRBL removal page seems to imply?). Some may ask, how could that be? Simple. Barracuda may have simply donated the address space and/or web hosting to the emailreg.org organization. Simple as that!. And that (alone!) would be enough to explain the emailreg.org web site being hosted in Barracuda address space. AND EXCEPT TO ASK: Is that $20 fee a one-time fee? Or a yearly fee? Or, does it have any kind of expiration date? Beyond this statement and question, I'll leave it to others to do their own research and draw their own conclusions. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Spam Rats - does anyone know them?
Matus UHLAR - fantomas wrote: > our customrer reported being listed in SpamRats blacklist. What was that IP? -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: zen.spamhaus.org
LuKreme wrote: > How about the 3rd post that exposes barracuda as a money-grubbing > racketeering operation? > "Barracuda own and operate emailreg.org, although there is no mention > of this on the emailreg.org site, and the whois data is obscured. > Indeed the owners of emailreg.org have gone to a lot of trouble to > hide who they are, which would be illegal for a UK operated website of > this type." > Stay away. Stay far away. I had no idea that emailreg.org was owned and operated by Barracuda. I thought they were two separate entities. (though I did have my suspicions about that relationship) But, as the post you mentioned said, emailreg.org resolves to 64.235.146.64 and arin.net shows that 64.235.146.64 is clearly in Barracuda's assigned address space. I'll tell you right now... this is BIG and EASY money. Very BIG and very EASY money. I suspect they are pulling in hundreds... maybe even thousands... of those $20 payments per day. If there is just 150 of these per business day, they've already cleared a million $$ per year. Maybe there aren't that many?...but I suspect that this number might be closer to a thousand per day, which would be into the tens of millions of dollars per year. (if I seem upset about this... read between the lines... and you might understand why) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: zen.spamhaus.org
Mark wrote: > I've been reading up a bit on Barracuda et al, like: > > http://www.email-ethics.com/2009/01/emailregorg-project.html > http://zacharyozer.blogspot.com/2008/10/worst-engineers-ever.html > http://www.debian-administration.org/users/simonw/weblog/295 > > And now I'm even more convinced that I will not be using Barracuda. Sorry. Mark, Regarding that earlier point about DNSBLs which claim to be more aggressive than SpamHaus... there are many IPs well deserving of being on a blacklist which are either missed by SpamHaus, or not caught very quickly by SpamHaus. Additionally, different DNSBLs use different techniques and, therefore, no one DNSBL can do even close to everything. However, it is true that *most* DNSBLs which claim to be low-FP lists (and which block much spam missed by SpamHaus) have more FPs than Zen--to varying degrees. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: FW: OpenDNS and Spamassassin
Michael Scheidell wrote: > Uribl should probaly get an rsync of the zones, if they are doing 50mm > queries a day, imagine what will hit urlbl servers directly, since opendns > caches the queries. > > (I think a opendns does this with some of the more popular zones anyway) > This would be good for opendns and urlbl I can't speak for uribl, but I do have first hand evidence that many quickly try to use OpenDNS as a means to get around paying subscriptions to commercial DNSBLs, or partly commercial DNSBLs. (I highly doubt that very much of this is from sys admins who happened to already use OpenDNS and then tried out uribl.) Therefore, if URIBL provided OpenDNS an rsync feed, three things would happen: (1) MANY ISPs and spam filtering vendors would flock to it to avoid having to pay for a URIBL data feed (2) MUCH of uribl's current revenue would dry up, making the long-term viability of uribl more 'at risk' (3) Finally, OpenDNS would get so massively slammed with queries in a way that does NOT generate revenue for OpenDNS... only more expense, more servers to manage, and more bandwidth... such that THEY would be the ones shutting this down (eventually) Therefore, the best solution is for OpenDNS to simply cut these queries off "in house" before they even have a chance of hitting URIBL, thus saving them and URIBL some CPU cycles and bandwidth (I'd bet that this is already happening) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Pastebin for spam examples
Karsten Bräckelmann wrote: > This won't work out. It's how it used to be... > > There's a reason, pastebins (just like URL shortener services) are > implementing spam filtering and various other spam/bulk counter- > measures. That's because they have been abused by spammers. > > Creating a dump to put your spam is like an invitation to spammers, a > free pay-load hosting service to point URLs in spams to. That's exactly > what happened, and subsequently the services got listed on RBLs while > they have been abused. It seems like a simple solution is to password protect the paste bin... with a user name and password p in clear view. Pastebins don't make for a good presentation of spammers' content since they only show the raw text. So what are spammers after? The answer is simple---they want links. A good paste bin ought to have meta tags telling the search engines to not index the page. But spammers won't notice that and will still try to post. If the page is password protected (even with the password in plain view), then it becomes obvious to all but the dumbest that the page is off-limits to search engines. This is a good thing because such a paste bin wouldn't be used as much by spammers, won't possibly convey "good reputation" onto a spammer's web page, and will still be easily accessible to those using it for legit purposes. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Vast improvements to the invaluement.com DNSBL just completed
Chip M. wrote: > This snowshoe stuff has been a PITA for a while. > > *** Rob McEwen: *** > Would you be willing to provide your /24 list, for even a short period, > in some sort of plain text format (maybe one CIDR per line?), so those > of us with good hand-classified corpi could try out your data? > > Most of my users are in a shared hosting environment, so they can't use > your list suite as-is. Based on what reliable people have posted, some > of my users should probably benefit from your /24 list. I'd be very > glad to provide you with a list of any FPs I find. :) When Chip posted this several weeks ago, I had to answer.. "hold on... I'm in the middle of making large-scale improvements the invaluement lists". But improvements are now completed. When I can finally catch my breath, I'll have to discuss the specifics of Chip's reuqest further. In the meantime, I'd like to formally announce the these improvements. Basically, very substantial improvements to the invaluement.com DNSBLs have recently been implemented. Plus, the sign-up page on the web site is now easier to understand, and the 'blog' section of the web site has finally started, with an initial post about these improvements to the lists detailed here: http://dnsbl.invaluement.com/spam-blocker/ If anyone was thinking about testing out the invaluement lists, but kept putting it off... now is a great time to start because, along with these recent improvements, there is also now a special deal where *anyone who converts their free test into a subscription during the month of March/09... will get a free month* added to their first billing period. And *if such a subscriber chooses yearly billing, 4 free months will be added to to their first billing period*. Unfortunately, time is short and in order to do this, the free portion of the testing period would be shorted a bit because the subscription would have to start in March/09. So if this is something that interests you, I recommend getting started as soon as possible so you'll get as many free testing days as possible before needing to make a decision which would take advantage of those free months. The longer you wait, the shorter those free testing days would be if you wanted to get those free months! To save you a click, and keep this info on-list, here is that blog post: ** *DETAILS ABOUT RECENT INVALUEMENT.COM IMPROVEMENTS* The most substantial improvements were made to ivmSIP/24. So I'll cover that last. First, I'll briefly address improvements to ivmSIP and ivmURI. *ivmSIP IMPROVEMENTS:* During recent work on ivmSIP/24, a “bug” was discovered in the ivmSIP portion of the engine which was causing needless False Negatives. This had been there since the /beginning/ of the invaluement lists! Fixing this should lead to additional ivmSIP effectiveness. (I about fell out of my seat when I spotted the bug!) So whatever you've thought/read/experienced about ivmSIP... it is now even better! *ivmURI IMPROVEMENTS:* Large improvements to the ivmURI engine were made. For example, most good DNSBLs will employ some type of False Positives-prevention filters which then lead to unintended False Negatives (items that really should have been blacklisted). ivmURI is no different and those unintended FNs produced by False Positives-prevention filters are a fact of life. But I'm happy to report that the ivmSIP FP-prevention filters have been greatly improved so as to continue preventing FPs just as well as ever, but those unintended FNs have now been greatly trimmed. This should boost ivmURI’s “catch rates” noticeably, if not substantially. *ivmSIP/24 IMPROVEMENTS:* This is where the vast majority of improvements occurred. ivmSIP/24 is almost like a whole new list. The invaluement 'engine' now has an automated system which enumerates through all 256 possible IPs for each /24 block and individually evaluates each IP for possible /exclusion/ from ivmSIP/24. The improvements made are substantial because ivmSIP/24 now does NOT have listed many subranges allocated to innocent senders in those /24 blocks which were split between innocent senders and spammers. Yet, at the same time, such subranges allocated to spammers remain listed on ivmSIP/24, with many IPs /preemptively/ listed. Additionally, we now have the ability to manually 'whitelist' individual IPs from ivmSIP/24 without having to delist the whole block. Even better, when that happens, the IP is NOT in our regular whitelist and, therefore, has just as good a chance to get listed in the regular ivmSIP list as any other IP. So this should probably be called an ivmSIP/24 “exemption list”, not a whitelist. Should that IP ever get caught sending spam, then all bets are off and its existance on the ivmSIP/24 “exemption list” is then completely ignored. This helps prevent spammers from
Re: please help, getting hammered with snowshoe spam
Chip M. wrote: > *** Rob McEwen: *** > Would you be willing to provide your /24 list, for even a short period, > in some sort of plain text format (maybe one CIDR per line?), so those > of us with good hand-classified corpi could try out your data? > > Most of my users are in a shared hosting environment, so they can't use > your list suite as-is. Based on what reliable people have posted, some > of my users should probably benefit from your /24 list. I'd be very > glad to provide you with a list of any FPs I find. :) Chip, Here are some thoughts: (1) if you are discussing hostkarma and barracuda's lists, then ivmSIP is probably a more equivalent list to compare to rather than ivmSIP/24. And they both work together VERY well for blocking snowshoe spam. Moreover, I contend that the combination of my three lists (ivmSIP, ivmSIP/24, and ivmURI), working together (and even if using ivmSIP/24 in scoring mode), is the best and most cost effective solution specifically for blocking hard-to-catch for snowshoe spam. (2) ivmSIP/24 is attempting a very dangerous mission... which is to preemptively block snowshoe spam by listing entire /24 blocks when only a handful of IPs on that block have sent spam so far. But keep in mind that (a) specifically--ivmSIP is going to block some spam where that snowshoer hadn't sent from enough IPs to possibly be listed (yet!) on ivmSIP/24 AND (b) The reason I call ivmSIP/24's mission as "dangerous" is because there is a high risk of FPs whereby spammers and legit senders share blocks of IPs within the *same* /24 block. I've taken steps to greatly minimize that amount of time that happens... but it is almost impossible to prevent this altogether. Therefore, both medium-to-large ISPs and those who are extremely concerned about FPs should use ivmSIP/24 for scoring instead of blocking--in spite of my continued attempts to get ivmSIP/24 to have just as few FPs as ivmSIP. (and I'm still working on that!) (3) Along these lines, I'm just about to make substantial changes to ivmSIP/24--so that (a) in many cases, it will list subranges instead of the whole /24 list and (b) that way, when I'm forced with a decision about removing an ivmSIP/24 listing so as to not hurt an innocent sender sharing a block with an egregious spammer.. I can then "have my cake and eat it to"--I can avoid more innocent IPs... but then NOT have to give the spammers a pass by delisting the whole /24 block--as I'm sometimes having to do now. (I often use this to a put pressure on hosters to remove the spammers FIRST--but I can only do so much of that--playing that game take tremendous time and resources--and has large lawsuit risks!) (4) And I'm about to implement a large improvement to ivmSIP. I found a bug in the programming (that had been there all along) which was preventing some deserving IP from getting into ivmSIP. So ivmSIP is about to get better. Therefore, substantial improvements are about to happen to BOTH ivmSIP and ivmSIP.24 --therefore, I'd prefer that any publicly available stats/testing be done in a week or two from now--AFTER these improvements are made. (5) regarding the "shared hosting environment"... if ALL of these mail servers resolve their queries using the *same* locally hosted DNS server for resolving queries, then there is only need for a single setup of the lists, for that one DNS server--and then there'd be a single price based on the cumulative total number of mailboxes--and, therefore, many quantity discounts would apply (or, am I not understanding you? Aren't these all hosted at the same physical location?... or multiple datacenters owned by the same company?) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: please help, getting hammered with snowshoe spam
Benny Pedersen wrote: > On Fri, January 23, 2009 17:36, Dennis Hardy wrote: > >> Yes already done: http://pastebin.com/m4400a74d >> > why not get it listed on http://uribl.com/? > Both uribl and ivmURI listed this domain back on January 23rd. But it is unclear exactly *when* this spam sample was sent because the person who started this thread didn't include full headers. So it is unclear if the message hit this guy's server before these two URI blacklists listed that domain? or after? (I'm guessing after?) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: spamd for windows
Harald Binkle wrote: > Hi @all, > We are searching for someone who will make spamd run on windows systems (XP > and later) by providing a one-click setup or single spamd.exe file like the > sourceforge project http://sourceforge.net/projects/sawin32/ . > We offer payment or donations if someone will continue (improve windows > integration and keeping it up to date with current SpamAssassin releases) > that sourceforge project or find an different solution to use spamd on > windows like a one-click setup. > > Is there some who is interested in this? > (1) Sorry if I'm being lazy by not digging for this... but is the *source code* for sawin32 project available so that someone could pick up where the author left off? (2) Has the author not been responsive to requests/e-mail? I'd actually be interested in taking this on--if I could somehow do the programming in vb.NET, which is quite a bit more powerful and vb, and has been used quite successfully to port ClamAV, btw. I currently use an older port as a "helper app" to my own spam filtering--but I had to revert to an even older version due to memory/cpu issues with the most recent win32 build. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Temporary 'Replacements' for SaneSecurity
SM wrote: > "Botnet Plugin" sounds like a plugin that detect botnets ... If > Rasmus is finding that many false positives, then he's using the wrong > tools. No. This is just due to the fact that, unfortunately, some mail servers and IPs (which send desired and solicited messages) are somewhat incorrectly configured. It turns out that a distributor receiving legitimate business e-mail from vendors & customers in such places as Africa, South America, Asia... all over the place... is going to see a disproportionately larger amount of messages sent from IPs which either: (a) would not do so well with BotNet's analysis ...OR... (b) which are mixed sources of ham/spam... but simply don't have a high enough volume of "ham" to stay off all the blacklists... particularly some blacklists. This has nothing to do with Rasmus's tools.. other than the fact that (I surmise) he is probably now forced, given that situation, back off of his scoring of DNSBls and rely more on content filtering in comparison to those whose e-mail is mostly US/Europe-based. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Temporary 'Replacements' for SaneSecurity
Rob McEwen wrote: > And I thing it is > probably better used as a scoring list instead of a blocking list. > oops. I meant "probably better scored below threshold", since, of course, BotNet isn't a "list". -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Temporary 'Replacements' for SaneSecurity
John Rudd wrote: > Botnet isn't a DNSBL... > I never said it was a DNSBL. But it definitely has a particular focus on the sending IP, and that sending IP's rDNS. Therefore, for all practical purposes, it is trying to do the job of a DNSBL. As I recall, the discussion about BotNet's development centered around blocking spam based on the sending IP... where that IP didn't have time to get into the DNSBLs. You might argue that a DNSBL could never replace the BotNet Plugin because the BotNet Plugin will always catch at least some spam that hasn't had time to get into a DNSBL. Fair argument--except that this argument is greatly diminished if/when there are high-quality/low-FP DNSBLs which are fast reacting/updating/distributing. Especially since DNSBLs "scale" much better than the BotNet Plugin... and especially if/when such DNSBLS have lower FPs than the BotNet Plugin. I did a quick cursory search of discussions about BotNet Plugin FPs. See attached for an example post I quickly grabbed after searching just a few seconds. NOTE: I'm NOT saying that the BotNet Plugin is bad or shouldn't be used. I just don't see it as a SaneSecurity replacement. And I thing it is probably better used as a scoring list instead of a blocking list. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032 --- Begin Message --- > Bret Miller wrote: > > > Enews.webbuyersguide.com (part of Ziff-Davis Media), sent from IP > > 204.92.135.90, resolves to smtp22.enews.webbuyersguide.com > #not sure why > > this got a BOTNET=1 flag, but it did. Also find hosts 92, > 75, 70, 74, 93, > > 86, and others. All similarly resolve to > smtpnn.enews.webbuyersguide.com. > > baddns. baddns means lack of full circle DNS. In this case, > the name > returned by the PTR record (smtp22.enews.webbuyersguide.com) does not > resolve at all ... let alone not resolving back to the > sending IP address. > > > > meridiencancun.com.mx, sent from IP , resolves to > > customer-148-233-9-212.uninet-ide.com.mx #more stupidity > > > > Wordreference.com (WordReference Forums), sent from IP 75.126.51.99, > > resolves to www2mail.wordreference.com, again no idea why > it gets flagged. > > # nslookup www2mail.wordreference.com > > Non-authoritative answer: > Name: www2mail.wordreference.com > Address: 75.126.29.11 > > baddns. > > > > AltoEdge Hardware, sent from IP 69.94.122.246, resolves to > > server.nch.com.au, another no idea why BOTNET=1, but it > does. Just out of > > curiosity, I ran this through again with debug enabled so I > could get more > > details. Here's what it says: > > > > [2472] dbg: Botnet: starting > > [2472] dbg: Botnet: no trusted relays > > [2472] dbg: Botnet: get_relay didn't find RDNS > > [2472] dbg: Botnet: IP is '69.94.122.246' > > [2472] dbg: Botnet: RDNS is 'server.nch.com.au' > > [2472] dbg: Botnet: HELO is 'server.nch.com.au' > > [2472] dbg: Botnet: sender 'adm...@server.nch.com.au' > > [2472] dbg: Botnet: hit (baddns) > > [2472] dbg: rules: ran eval rule BOTNET ==> got hit (1) > > > > I'm not sure what it means. The IP resolves to > server.nch.com.au and it > > resolves to the IP. Not sure what is "bad" about dns here. > I'm also not sure > > what headers botnet looks at. The top Received header is > ours and the others > > are all internal to the sender. > > # nslookup server.nch.com.au > > Non-authoritative answer: > Name: server.nch.com.au > Address: 69.94.122.247 > > So, server.nch.com.au's name does not resolve back to the sending IP > address, thus baddns. OK... I guess I didn't check closely enough. But the point is still that users expect these emails and complain if they don't receive them. Today's list were mostly just top offenders, and it's going to take me time to make exceptions for all the servers we receive email from that are badly configured dns-wise. Maybe these aren't false positives because botnet is identifying them for what they are-- badly configured. But to give a rule like botnet a default score that's high enough to consider the messages spam all on its own causes users to think we have a bad spam filtering program. When I see on the list that many people run botnet with ZERO false positives, I have to ask myself, "how? And why is our setup here so different?" Perhaps they already block email with invalid rdns at the MTA level, so none of this ever gets looked at. Perhaps their users just give up when they don't get email that they expect and use a free email account instead for
Re: Temporary 'Replacements' for SaneSecurity
Rasmus Haslund wrote: >> After a loud outcry from our users from the increasing level of spam in >> their inboxes, I installed the Botnet >Plugin. >> > Is this something that can be used with the SA in Icewarp Merak? > Because Rasmus manages a mail server where B2B mail is routinely sent/received _globally_, Rasmus is the king of finding FPs. I could be wrong, but judging from previous reports about the Botnet Plugin, I predict that Rasmus will either (a) find the Botnet Plugin utterly unusable due to FPs, or (b) only be able to score it by a point or two due to excessive FPs. (Rasmus--by all means--please don't take my word for it--try it out and then let us know what happened!) Regarding using the Botnet Plugin as a replacement for SaneSecurity... I found that the _best_ part about SaneSecurity was its assistance with catching spam that could NOT ever be caught using _any_ kind of DNSBL. For example, "419" scam spams sent from the large freemail providers where the message cannot possibly be blocked because of being sent from an IP that send large amounts of legit mail and because there is simply no domain in the body of the message for surbl/uribl/ivmURI to grab onto. THAT was the best part about SaneSecurity, imo. Therefore, if someone is missing SaneSecurity, I'd suggest first making sure they have Sought Rules installed and frequently updating--if not already running. QUESTIONS: Is SaneSecurity still collecting data and generating the rulesets? (but just not able to distribute them) Is there any end in sight for the DDOS? Has anyone tried to mitigate their DDOS? (There is a super-secret list out there consisting of professionals who work for all the largest ISPs and security vendors. They have ways to help mitigate these things. They look for IPs conducting the DDOS, on each of their own networks, and they simply shut those IPs down at the access point.) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: Spam with clean URI's which forward to DNSBListed URL (by HTML redirect header)
Florian Lagg wrote: > In the last few days my Spamassassign does not filter a (for me) new > kind of spam. I have an idea how to fight this spam and want to ask > the list if this is possible with SA. > > > > More examples: > -- > Hey! Do you believe that when New Year Eve comes all dreams come true? If you > don\'t, we can assure you that it is right as we are giving you unbelievable > bonuses upon registration. > > http://florafloricultura.com.br/2009.php > This one is not on surbl or uribl, but blacklisted on ivmURI at 1/5/2009, 11:19:51 PM EST, fwiw > -- > Santa is very generous this year and he is ready to give the welcome bonuses > even to those players who have been naughty this year. So don't miss your > chance and hurry to register with us. > > http://terraverde-rj.org/2009.php > This one is not on surbl or uribl or ivmURI, but listed on URIBL-RED, fwiw > -- > Santa Claus is coming to town and bringing amazing bonuses for all the lucky > customers that sign in now. So hurry to pick your Christmas bonus now! > > http://creationsitecms.com/2009.php > ...unfortunately, I can't find this on any URI blacklist. > -- > We have wonderful betting limits for you - from $1 to $1000 - so even if you > are broke, you still can play with us. Isn\'t that just a Christmas miracle? > > http://soldavila.com/2009.php > -- > This one is not on surbl or uribl, but blacklisted on ivmURI at 1/6/2009, 7:16:50 AM EST, fwiw SUMMARY: Of your 4 extra examples, 2 are listed on ivmURI, and 1 is listed on URIBL-RED. NOTE: If you report a URIBL-RED False Positive, they'll tell you that it really isn't a FP because URIBL-RED is an experimental list. Still, I think that URIBL-RED is worthy of use, even if scored a tiny bit below URIBL-BLACK. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: A lot of spams go through, see example
Igor Chudov wrote:
> http://igor.chudov.com/tmp/spam005.txt
>
> I get a lot of these, all seemingly sent by the same software and the
> same person, any way of filtering them out?
>
The sending IP is currently blacklisted on FiveTenSig and ivmSIP/24.
Both of these are best used as "scoring" lists and not for outright
blocking. (though ivmSIP/24 could generally be scored rather high...
probably just below threshold.). Even when not used for outright
blocking, using either or both of these might have put the spam "over
the top" in combination with other things.
(Note that some consider FiveTenSig too risky to even score on. I
personally find FiveTenSig effective when adding about a point to the
spam score. But it may be that I'm somewhat insolated from FiveTenSig
FPs due to my vast IP whitelist?)
The domain name used by the spammer ("newyearonline DOT info") is NOT
listed on either surbl or uribl (at the time that I type this), but was
blacklisted on ivmURI almost exactly two minutes *before* the spam
sample you provided reached your server. However, propagations issues
would have probably made this a just-barely-missed spam in terms of
ivmURI's ability to block this. Still, that ivmURI caught it so early is
noteworthy. It may me that ivmURI might be helpful for others of this
series of spams.
One thing is for sure, you are getting the tip edge of some
hard-to-catch snowshoe spam. You probably have some addresses at the
very beginning of some snowshoe spammer's distribution list.
--
Rob McEwen
http://dnsbl.invaluement.com/
r...@invaluement.com
+1 (478) 475-9032
Re: Spam slipping through
Greg Skouby wrote: > Can you please do me a favor and run this through your setup and let me know > what it scores: > http://pastebin.com/m791c34be > As of now the URL at the bottom is not in URIBL or SURBL and the sending IP > is not on any major blacklist. I am curious if others have rules that hit on > this. > (I know 2.5 is a *really* low required score) > "steadyrelationships DOT com" is currently blacklisted on ivmURI It was added to ivmURI at 12/16/2008, 6:31:03 PM EST (I think that time is before that spam arrived at your server, but double-check me on that) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
I need a contact for openrbl.org and for robtex.com
I need a contact for both openrbl.org and robtex.com Please e-mail me (off-list!) if you have a contact for the operators of either service, or if you are the operator of either service. Thanks! -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: Hard money conference spam
Micah, In addition to the barracuda RBL, this IP is also listed on ivmSIP (since 10/21/08) and ivmSIP/24 Additionally, the domain "hardmoney-event DOT com" is blacklisted on both ivmURI and URIBL.COM At the very least, you should add uribl.com to your filtering since that list is free. Scoring with URIBL for this would have easily put that message "over the top" for you. SHORT ANSWER: Start using uribl.com's URI blacklist -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: Hard money conference spam
Micah, In addition to the barracuda RBL, this IP is also listed on ivmSIP (since 10/21/08) and ivmSIP/24 Additionally, the domain "hardmoney-event DOT com" is blacklisted on both ivmURI and URIBL.COM At the very least, you should add uribl.com to your filtering since that list is free. Scoring with URIBL for this would have easily put that message "over the top" for you. SHORT ANSWER: Start using uribl.com's URI blacklist -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: botnet dos
Randy wrote: > We are being spammed by a botnet to a single email address which makes > it difficult to block. Spamhaus catches about 1/2 of them, but the > rest are blocked via postfix becuase this is an old account and does > not have a mailbox. Are you sure this isn't backscatter where the botnet spam really went to a variety of e-mails, but used that same e-mail address as the "from" for that series of spams? -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED]
question about testing new rulesets
RE: question about testing new rulesets Is it possible to do the following when testing out a new ruleset: (1) score that rule at 0.01 (of course this is possible... but then also...) (2) copy the original source file that was "fed" to SA to a separate directory if (a) the new rule being tested triggered ...AND... (b) if that message ended up scoring "below threshold" and was therefore NOT considered spam. This would allow someone to audit those messages which would ONLY have been blocked had that new ruleset been giving a higher score. Analysis on such messages could then be done to see how many of these are FNs and how many of these are FPs. I'm thinking that, if SA can delete and re-write the source file with a new header, it seems like it could also copy the message to a different folder, under certain conditions? Thanks! -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED]
Re: New free blacklist: BRBL - Barracuda Reputation Block List
John Hardin wrote: On Tue, 23 Sep 2008, Rob McEwen wrote: Or, these could be "False-False Positives"... which is a very good thing because that would mean that those were really spams that would have scored "below threshold" without use of the new list. (or, some mix of these two) So, for the purposes of an analysis like this, perhaps the results should be broken into *three* categories: obviously spam, obviously ham, and borderline. Those initial stats are computer generated. Any follow-up analysis should be more human-generated. There is definitely a "borderline" category but I'd suggest that computer generated stats be left alone. Trying to get a "borderline" by the spam filter's scoring alone is a bad idea. Why? Because, simply put, some DNSBLs are able to catch spam that, quite frankly, scores very low in many systems when that DNSBL is absent (think of "first responder" dnsbls!). So splitting out into subcategories based on computer-generated-scoring only muddies the waters further. Instead, the person running the stats could examine the actual messages (that is, those classified by the spam filter as "ham") more closely and then follow up the computer generated stats with their own personal opinion about what was seen in those messages. Even a cursory analysis would be far better than nothing. Few are going to have the time or inclination to get get extremely detailed in such analysis. But hey, that would be great too. But just a little analysis of that "ham" pile is far better than nothing. (NOT complaining about Alex's post, btw... again, that is why he said "fwiw"... this is more of a general suggestion for everyone about such stats.) -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: New free blacklist: BRBL - Barracuda Reputation Block List
Yet Another Ninja wrote: FIW: 12 hr stats / tiny traffic trap box - no ham I use a couple of DNSWLs to reject traffic from potential hammy IPs RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM 1RCVD_BARRACUDA 19721 83.30 83.46 8.00 Spam detection seems good - no idea how it does with HAM What I'm about to say is probably part of the reason that Alex started those stats out with "fwiw", but when running stats like that, the "ham" column is tricky. Why? Because these are either False Positives--which is a very bad thing. Or, these could be "False-False Positives"... which is a very good thing because that would mean that those were really spams that would have scored "below threshold" without use of the new list. (or, some mix of these two) For that reason, it is always helpful (if possible) if the tester can examine some of the messages which make up the "ham" % on the new list that is being evaluated. Recently, I had a user testing my own blacklists who sent me such stats and I panicked. I sent an e-mail back saying, surely I'm not blocking THAT many hams? He replied back stating that, upon examination of the messages that made up the HAM category, he couldn't find a single actual ham. They were all spam. (I breathed a big sigh of relief!) But I'd guess that most of that 8% of ham for Barracuda is probably spam? Even if the barracuda list has too many FPs, I doubt it would be that high!!?? I've seen such stats posted on anti-spam lists like SA, but I don't recall anyone ever making that distinction. -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: Trying out a new concept
Blaine Fleming wrote: John Hardin wrote: Why is it so flippin' difficult to get a feed of newly-registered domain names? Because the TLDs hate giving people access to the data and certainly won't provide a feed without a bunch of cash involved. Even worse, all the ccTLDs pretty much refuse to even talk to you about access to the zones. This is why I started processing all the TLDs I was able to obtain access to. There is lag but the most it could be is about 24 hours and that assumes they register a new domain immediately after the TLD dumps the zone. Honestly, on my system I have less than 0.01% hits against a list of domains registered in the last five days so I've always considered the list a failure. However, several others are reporting excellent hit rates on it. I think it is because the test is so far after everything else though To some extent, I like the concept. But I think the results are going to be somewhat limited because the sneakiest of spammers often allow their domains to "age" a bit for the very reason that "age of domain" is a common metric in the evaluation of domain reputation. Snowshoe spammers in particular have caught onto this fact in recent years/months. Therefore, the tendency will be for DOB lists to catch spam that was already well-caught, such as botnet-sent spams. (matching up with what Blaine said). Also, Marc is wise to consider combining this with other metrics because it is not that uncommon for some large and legit organization to blast out an e-mail to their members discussing some new web site which uses a domain name just bought a few days ago. But, as someone else said, such a list might be effective for scoring 1 point, or something like that. I'd be interested in putting such a list to use in my own spam filtering in such a manner. -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: rbldnsd blacklist question
John Hardin wrote: On Tue, 16 Sep 2008, Marc Perkel wrote: Looking from opinions from people running rbl blacklists. I have a list that contains a lot of name based information. I'm about to add a lot more information to the list and what will happen is that when you look up a name you might get several results. For example, a hostname might be blacklisted, be in a URIBL list, be in a day old bread list, and a NOT QUIT list. So it might return 4 results like 127.0.0.2, 127.0.0.6, 127.0.0.7, 127.0.0.8. Is this what would be considered "best practice". My thinking is that having one list that returns everything is very efficient. Isn't general practice to bitmap the last octet if you're going to convey multiple pieces of information? If you have a situation where there might be more than one "answer" for a given query, and you are content with having a maximum of 7 possible answers, then... again, if both of these these things are true... then the best system by far is the following: .2 = situation #1 .4 = situation #2 .8 = situation #3 .16 = situation #4 .32 = situation #5 .64 = situation #6 .128 = situation #7 As multiple situations occur, add together the octets above. For example, .138 would mean that situations #1, #3, & #7 happened. That way, anywhere from one to all seven attributes can be encapsulated as one single number, with any combination of these being clearly decipherable. From a programming perspective, do the following: If octet >= 128 then #7 happened octet = octet - 128 End If If octet >= 64 then #6 (also?) happened octet = octet - 64 End If If octet >= 32 then #5 (also?) happened octet = octet - 23 End If Which is a less fancy way of saying what John Hardin said about "bitmap the last octet"... but I thought that spelling it out this way might be helpful for some. -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: Spam from your email address.
[EMAIL PROTECTED] wrote: How can I tackle spam that came from my own e-mail address that I did not send. Any info on how to prevent this will be greatly appreciated. I'm not a big fan of Sender Policy Framework (SPF). But if/when something like this happens to me or a client, I find it helps to set a very strict SFP record saying that mail from that domain should *only* come from your main official mail server. That way, recipients of such spam will have more tools available for blocking such messages. The second problem is that you are probably seeing much backscatter from mis-configured servers sending out separate e-mails to your address complaining about spam you didn't send. A good solution for that is to run UCEProtect's backscatterer list.(http://www.backscatterer.org/). But don't outright block on that list (unless you are disparate, this can be applied to a single account, and are unable to do my additional recommendations...) Instead, it would be better to block if the sending IP is in backscatterer.org *combined* with another attribute, such as the SMTP Envelope reporting a "from" address that contains the term "postmaster" or "mailer-daemon". Otherwise, you will probably have a significant amount of FPs. Hope this helps! -- Rob McEwen http://dnsbl.invaluement.com/ [EMAIL PROTECTED] +1 (478) 475-9032
Re: DNS Tests not always getting done
Skip wrote: I got this: $ host 2.0.0.127.zen.spmahaus.org Host 2.0.0.127.zen.spmahaus.org not found: 3(NXDOMAIN) That can't be good. I do not know what dns server we are using at bluehost. I did a ps and searched for anything that looked like a dns server, but couldn't find any. Sometimes it can really suck being on a shared system like this. Skip, I tried to visit your web site, and got the following: ** Please contact this site's webmaster. Wait a few minutes and use your browser's "Back" button or click here to try again. If you are the webmaster, your account may have gotten this error for one or more of the following reasons: * Your account has used more than its share of the cpu in the past 60 second sliding window. * Your account has too many concurrent processes running simultanously. * Your account has consumed too much memory. * Your site was recently very busy trying to run inefficient scripts. The solution would be to optimize your applications to use less CPU. Adding appropriate indeces to your SQL tables can often help reduce CPU. Using static .html documents instead of painful .php scripts will practically eliminate CPU usage. ** Maybe that has something to do with the problem? -- Rob McEwen http://dnsbl.invaluement.com/
Re: Day Old Bread/Spammers
Could you give an example? Are these newly registered top level domains spotted in the body of the spams? Rob McEwen Mailing Lists wrote: I'm getting dozens of emails daily from a few different spammers. The emails consistently are graphic based, but the graphics are html img refs and not consistent names - the last image in each one is their send mail to this address to be removed (or actually to guarantee even MORE spam). One is from "Wagonjumpers" another is from some address in Florida (those images in the spam are consistent). Each day, it seems they set up a few new hostnames, and start spamming. We immediately (upon notification from our users) add that hostname to our access denied list, since they are spammer addresses, but is there an easier way to trap the email? I know that the various img evaluation plugins & image ocr plugins do not appear to work, since they don't download referenced images. --Will
