Re: Registrar RBL: nomination and scoring
On Sun, 13 Aug 2006, David Cary Hart wrote: > If someone can figure out the mechanics, I have a volunteer > (working on her MBA) who is great at crafting policy. I also have > the mirrors and structure. I am willing to add the zone. My first > listing would be Gandi. I have beta versions of this available, one for a URIRBL and one for a plugin. The URIRBL version supports trust levels (assigned however is appropriate) and query based on trust levels (so you can choose score based on trust level). The plugin version also checks the domain of the envelope sender and header From: address, but does not support trust levels. Contact me directly if you'd like to test either. -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- It is not the business of government to make men virtuous or religious, or to preserve the fool from the consequences of his own folly. -- Henry George --- 30 days until Talk Like a Pirate day
RE: Registrar RBL: nomination and scoring
> -Original Message- > From: Bill Horne [mailto:[EMAIL PROTECTED] > Sent: Wednesday, August 16, 2006 8:06 PM > To: users@spamassassin.apache.org > Subject: Re: Registrar RBL: nomination and scoring > > > Homelinux.org is owned by dyndns.org, and the company gives > out domain names like timesucker.homelinux.org to anyone who > applies. In other words, dyndns.org is in business to provide Ok, so we should blacklist *.homelinux.org?
Re: Registrar RBL: nomination and scoring
From: "Bill Horne" <[EMAIL PROTECTED]> On Sun, Aug 13, 2006 at 06:26:18PM -0700, jdow wrote: I wonder what the reputation of homelinux.org is these days. (I just posted a couple "rules" to the FC mailing list about them. A spam was relayed through them to the list followed by two shills who copied the entire message and complained at the bottom "pro forma." This is not the first time this has happened.) Homelinux.org is owned by dyndns.org, and the company gives out domain names like timesucker.homelinux.org to anyone who applies. In other words, dyndns.org is in business to provide dsl and cable subscribers with routable domains that are automagically updated on the rare occasions when the cable/dsl companies renumber their IP subnets. Each domain under homelinux.org is a separate individual/company/whatever, so please keep that in mind when deciding on the reputation of "homelinux.org": you might as well ask the reputation of "com" or "net". Bill (Disclaimer: I'm one of dyndns.org's customers, but I have no stock or other interest in the firm.) I use dyndns, privately, myself. (I do not publish and use that address publicly.) I've not seen problems with spoo.dyndns.net or any of the others. But I have seen more than one with homelinux.org. Maybe they do not have that one under control yet. {^_^}
Re: Registrar RBL: nomination and scoring
On Sun, Aug 13, 2006 at 06:26:18PM -0700, jdow wrote: > > I wonder what the reputation of homelinux.org is these days. > (I just posted a couple "rules" to the FC mailing list about them. > A spam was relayed through them to the list followed by two shills > who copied the entire message and complained at the bottom "pro > forma." This is not the first time this has happened.) > Homelinux.org is owned by dyndns.org, and the company gives out domain names like timesucker.homelinux.org to anyone who applies. In other words, dyndns.org is in business to provide dsl and cable subscribers with routable domains that are automagically updated on the rare occasions when the cable/dsl companies renumber their IP subnets. Each domain under homelinux.org is a separate individual/company/whatever, so please keep that in mind when deciding on the reputation of "homelinux.org": you might as well ask the reputation of "com" or "net". Bill (Disclaimer: I'm one of dyndns.org's customers, but I have no stock or other interest in the firm.)
Re: Registrar RBL: nomination and scoring
From: "John Rudd" <[EMAIL PROTECTED]> On Aug 13, 2006, at 8:41 AM, John D. Hardin wrote: There still remains the question about what **exactly** should the numerator and the denominator be when calculating that percentage? Any ideas yet? Not from me. I don't know either. I base the general idea on the IronPort "Sender Base Reputation Score", but that's not an open content thing. You can browse their database, but it wont tell you the actual -10 (overwhelmingly likely to be a spam sender) to +10 (pure innocent angels of email) rating unless you've got a license. You can set the IronPort box to whatever threshold you want for blocking sending hosts. I wonder what the reputation of homelinux.org is these days. (I just posted a couple "rules" to the FC mailing list about them. A spam was relayed through them to the list followed by two shills who copied the entire message and complained at the bottom "pro forma." This is not the first time this has happened.) {^_^}
Re: Registrar RBL: nomination and scoring
On Sun, 13 Aug 2006, John Rudd wrote: > I like the idea of an RBL gives ratings instead of binary values. > That's why I thought of it being a "confidence percentage" instead > of just a "yes, we have them listed in the zone". How to build > that confidence rating is another matter entirely. There's another option: develop a set of registrar behavior criteria (e.g. "does not have a strong anti-spam AUP", "does not respond to complaints", "does not enforce AUP", etc.) and assign bits to those criteria. There wouldn't be a confidence score per se, but a bitmapped report of why they are considered spam-friendly. If you don't want to judge on a particular criteria, mask it out of your subtest. It's also much less subjective. -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The fetters imposed on liberty at home have ever been forged out of the weapons provided for defense against real, pretended, or imaginary dangers from abroad. -- James Madison, 1799 ---
Re: Registrar RBL: nomination and scoring
From: "John D. Hardin" <[EMAIL PROTECTED]> On Sun, 13 Aug 2006, Benny Pedersen wrote: On Sun, August 13, 2006 02:11, John D. Hardin wrote: > On Sat, 12 Aug 2006, John Rudd wrote: > > 127.0.0.1 ... 127.0.0.100 perhaps? How would a rule to score points > based on the returned IP look? Can/does SA cache the returned IP and > test it in multiple rules without making multiple DNS queries? yes, i have created an example.cf to SA Good. ...is there any way to write a rule that mathematically bases the score points on the IP returned, without having 100 rules (one for each score point)? Of course - look at the Bayes rules and "eval". {^_-}
Re: Registrar RBL: nomination and scoring
On Aug 13, 2006, at 8:41 AM, John D. Hardin wrote: There still remains the question about what **exactly** should the numerator and the denominator be when calculating that percentage? Any ideas yet? Not from me. I don't know either. I base the general idea on the IronPort "Sender Base Reputation Score", but that's not an open content thing. You can browse their database, but it wont tell you the actual -10 (overwhelmingly likely to be a spam sender) to +10 (pure innocent angels of email) rating unless you've got a license. You can set the IronPort box to whatever threshold you want for blocking sending hosts. I like the idea of an RBL gives ratings instead of binary values. That's why I thought of it being a "confidence percentage" instead of just a "yes, we have them listed in the zone". How to build that confidence rating is another matter entirely. SBRS is a cross section of data sources and data items, whereas what we're talking about here is a single data item (whether or not we can trust a host based upon who its domain registrar is). So it's not like we can start out by pulling data from multiple zones and building up a number based on how much we trust each zone and how many zones someone is listed in. The only other thought I have, which is not going to be an immediate result, is simply to have people give feedback, over time, about different hosts ... and then have that feed into a database which tracks hosts and registrars to build up that confidence rating over time. Sorry, my idea is only half baked so far :-}
Re: Registrar RBL: nomination and scoring
On Sun, 13 Aug 2006, David Cary Hart wrote: > I don't disagree with any of this. In fact, this could be a very > powerful economic boycott which is why I thought about it. I am > only pointing our the administrative difficulties. > > How would you suggest the query mechanism works? I Most whois > servers impose some sort of volume limitation; Many are extremely > slow. There is caching. It shouldn't do a whois query for a given domain more than once per TTL (which I default to a week). However the initial surge of checking common domains may hit throttling. Also, it doesn't need to go out to the actual registrar for all the details, it just captures the registrar name from the root whois query. However, *most* domains won't be hosted by spam-friendly registrars, and if whois gives you the finger this will return NXDOMAIN, so the worst you'll get is a false negative response for a while, until a definitive response *is* received. > Therefor, this probably warrants a RHSBL with the registrar in the > text record. In turn, that requires getting a listing of all > domains registered by a listed registrar. That's the sticking point. How and where do you obtain that information? Do you have to become a registrar? > How do you keep up with transfers? If it's dynamically collected then transfers don't make sense. Sure, you'll capture the known domains (ones that somebody has asked about within the last $TTL seconds), but the unknown ones will all return NXDOMAIN, leading to FNs. Being able to download the domain->registrar information en masse makes it *much* simpler, you can just reformat it as a zone file and publish it. But then you lose the percentile support that the dynamic server provides. > If someone can figure out the mechanics, I have a volunteer > (working on her MBA) who is great at crafting policy. I also have > the mirrors and structure. I am willing to add the zone. My first > listing would be Gandi. I have a first cut beta available right now, if you want to try it out. It's still rough so you have to edit the source to configure it, but I'd be willing to get some feedback (apart from "OH MY GOD that's hideous code! My eyes! AUGH!"). Contact me off-list if you're interested. -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The fetters imposed on liberty at home have ever been forged out of the weapons provided for defense against real, pretended, or imaginary dangers from abroad. -- James Madison, 1799 ---
Re: Registrar RBL: nomination and scoring
On Sun, 13 Aug 2006 10:26:28 -0700 (PDT), "John D. Hardin" <[EMAIL PROTECTED]> opined: > > Registrars' Terms of Service should be publicly available for > review; standards for ToS treatment of spammer behavior should be > fairly easy to develop and apply. > > Registrars' responsiveness to complaints should be fairly easy to > track as well, and standards for that should also be possible. > > Meta-question: *how much* responsibility for the domain-owner's > behavior does the registrar actually or reasonably bear? What form > does that responsibility take? And how much are you willing to pay for a domain? > I don't disagree with any of this. In fact, this could be a very powerful economic boycott which is why I thought about it. I am only pointing our the administrative difficulties. How would you suggest the query mechanism works? I Most whois servers impose some sort of volume limitation; Many are extremely slow. Therefor, this probably warrants a RHSBL with the registrar in the text record. In turn, that requires getting a listing of all domains registered by a listed registrar. How do you keep up with transfers? If someone can figure out the mechanics, I have a volunteer (working on her MBA) who is great at crafting policy. I also have the mirrors and structure. I am willing to add the zone. My first listing would be Gandi. -- Our DNSRBL - Eliminate Spam at the Source: http://www.TQMcube.com Don't Subsidize Criminals: http://boulderpledge.org
Re: Registrar RBL: nomination and scoring
On Sun, 13 Aug 2006, David Cary Hart wrote: > > > b) have an RBL which returns different values for different > > > confidence levels. > > > > 127.0.0.1 ... 127.0.0.100 perhaps? How would a rule to score points > > based on the returned IP look? > > I actually considered doing this. However: > > 1. Maintenance is problematic. > > 2. Creating a consistent policy for listing and removal is > nearly impossible. Ultimately, the whole thing becomes very > arbitrary. Not necessarily. Registrars' Terms of Service should be publicly available for review; standards for ToS treatment of spammer behavior should be fairly easy to develop and apply. Registrars' responsiveness to complaints should be fairly easy to track as well, and standards for that should also be possible. Meta-question: *how much* responsibility for the domain-owner's behavior does the registrar actually or reasonably bear? What form does that responsibility take? There might even be a consideration of how complete and accurate the registrar's whois data is. A factor might be the registrar having lots of obviously-bogus domain registration data that they are unwilling to pursue correcting with the domain owners. Having correct domain owner contact information is, after all, one of the responsibilities of a legitimate registrar (modulo privacy issues - but if it's visible it should be correct!). > 3. It requires data that is unavailable. Unless one considers the > total of domains registered or served then the signal:noise becomes > incalculable. True. However there are other factors (as noted above) that can be used as a basis for a judgement that doesn't rely on knowing those bits of data. Remember, this rates the *registrar*, not the domains. > I would also note that there is no standardization of whois data. Also true, but for this the only whois data we need is the name of the domain's registrar. We don't need to deal with the myriad of different ways the registrars can present (or obscure) the actual registration data. > 4. If you compare this to our PRC or Korea lists, a user can > evaluate whether or not they receive any valid email from those > countries and score accordingly. Agreed. The spam-friendliness of the registrar should only be a component of the spam/ham decision, not the entire decision. > 5. I believe that our "quarantine" policy provides a real incentive > for administrators to lock down their servers. Yet that knowingly > creates a certain amount of ham. However there is a consistent and > pragmatic methodology associated with delisting. "delisting" in this case would involve the registrar responding promptly and effectively to complaints about the domains registered with them, and having a ToS agreement that is not friendly to spam behavior, and enforcing accurate domain ownership data. -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The fetters imposed on liberty at home have ever been forged out of the weapons provided for defense against real, pretended, or imaginary dangers from abroad. -- James Madison, 1799 ---
Re: Registrar RBL: nomination and scoring
On Sat, 12 Aug 2006 17:11:34 -0700 (PDT), "John D. Hardin" <[EMAIL PROTECTED]> opined: > On Sat, 12 Aug 2006, John Rudd wrote: > > > If someone does make a Registrar RBL and a Name Server RBL (both > > of which are good ideas), _PLEASE_ do something like this: > > > > a) have two lists for each RBL, one which has the above "kill the > > bystanders" point of view, and one which is much more conservative > > in its listing policies. > > By listing policies I suppose you mean how offensive a registrar has > to be to be put on the list. Can anyone suggest guidelines to use to > make this decision? > > > b) have an RBL which returns different values for different > > confidence levels. Something like a percentage of known spammers > > are on that specific provider. So, if a registrar is 60% spammers > > and 40% bystanders, it will return "60"... and I can choose to > > only block those who have a 99% or higher rating, or something. > > This would also, hopefully, allow SA to give different score > > values to different ratings. > > 127.0.0.1 ... 127.0.0.100 perhaps? How would a rule to score points > based on the returned IP look? Can/does SA cache the returned IP and > test it in multiple rules without making multiple DNS queries? > I actually considered doing this. However: 1. Maintenance is problematic. 2. Creating a consistent policy for listing and removal is nearly impossible. Ultimately, the whole thing becomes very arbitrary. 3. It requires data that is unavailable. Unless one considers the total of domains registered or served then the signal:noise becomes incalculable. I would also note that there is no standardization of whois data. 4. If you compare this to our PRC or Korea lists, a user can evaluate whether or not they receive any valid email from those countries and score accordingly. 5. I believe that our "quarantine" policy provides a real incentive for administrators to lock down their servers. Yet that knowingly creates a certain amount of ham. However there is a consistent and pragmatic methodology associated with delisting. -- Our DNSRBL - Eliminate Spam at the Source: http://www.TQMcube.com Don't Subsidize Criminals: http://boulderpledge.org
Re: Registrar RBL: nomination and scoring
On Sun, 13 Aug 2006, Benny Pedersen wrote: > On Sun, August 13, 2006 02:11, John D. Hardin wrote: > > On Sat, 12 Aug 2006, John Rudd wrote: > > > > 127.0.0.1 ... 127.0.0.100 perhaps? How would a rule to score points > > based on the returned IP look? Can/does SA cache the returned IP and > > test it in multiple rules without making multiple DNS queries? > > yes, i have created an example.cf to SA Good. ...is there any way to write a rule that mathematically bases the score points on the IP returned, without having 100 rules (one for each score point)? -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- They [the Republicans] have written a new constitution for Iraq and ignored the Constitution here at home. -- Julian Bond, www.tompaine.com ---
RE: Registrar RBL: nomination and scoring
On Sat, 12 Aug 2006, Rob McEwen wrote: > >I'm not sure zone transfers will be feasible, since the registrar > >determination will be made dynamically. > > I think, to prevent processing overloads, you might want to cache > results at least for a period of minutes and not recalculate > results for every thing query. I'm sure this isn't something that > changes that much minute to minute. But of course! I was thinking of a TTL on the order of a week. > There still remains the question about what **exactly** should the > numerator and the denominator be when calculating that percentage? > Any ideas yet? Not from me. It might be useful to bring this up on n.a.n.e and see what the denizens there have to say. -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- They [the Republicans] have written a new constitution for Iraq and ignored the Constitution here at home. -- Julian Bond, www.tompaine.com ---
Re: Registrar RBL: nomination and scoring
On Sun, August 13, 2006 02:11, John D. Hardin wrote: > On Sat, 12 Aug 2006, John Rudd wrote: > > 127.0.0.1 ... 127.0.0.100 perhaps? How would a rule to score points > based on the returned IP look? Can/does SA cache the returned IP and > test it in multiple rules without making multiple DNS queries? yes, i have created an example.cf to SA -- Bennyfirst the real lookup header __DNSBL rbleval:check_rbl('CACHE', 'dnsbl.example.com.') describe __DNSBL cache result tflags __DNSBL net score __DNSBL 0.1 header DNSBL_1 eval:check_rbl_sub('CACHE', '127.0.0.1') describe DNSBL_1 result code 1 tflags DNSBL_1 net score DNSBL_1 0.5 header DNSBL_2 eval:check_rbl_sub('CACHE', '127.0.0.2') describe DNSBL_2 result code 2 tflags DNSBL_2 net score DNSBL_2 1.1 header DNSBL_3 eval:check_rbl_sub('CACHE', '127.0.0.3') describe DNSBL_3 result code 3 tflags DNSBL_3 net score DNSBL_3 0.4 header DNSBL_4 eval:check_rbl_sub('CACHE', '127.0.0.4') describe DNSBL_4 result code 4 tflags DNSBL_4 net score DNSBL_4 0.4 as much you like, it will one do one dns lookup
RE: Registrar RBL: nomination and scoring
>I'm not sure zone transfers will be feasible, since the registrar >determination will be made dynamically. I think, to prevent processing overloads, you might want to cache results at least for a period of minutes and not recalculate results for every thing query. I'm sure this isn't something that changes that much minute to minute. There still remains the question about what **exactly** should the numerator and the denominator be when calculating that percentage? Any ideas yet? Rob McEwen PowerView Systems
Re: Registrar RBL: nomination and scoring
On Sat, 12 Aug 2006, John Rudd wrote: > On Aug 12, 2006, at 5:11 PM, John D. Hardin wrote: > > > >> b) have an RBL which returns different values for different > >> confidence levels. > > > > 127.0.0.1 ... 127.0.0.100 perhaps? How would a rule to score points > > based on the returned IP look? Can/does SA cache the returned IP and > > test it in multiple rules without making multiple DNS queries? > > I can see a few ways of doing this: > > Multiple sub-zones, such as (using a registrar BL named REGBL as an > example): > > REGBL70 (which includes everyone whose values are > 127.0.0.70-127.0.0.100) > REGBL80 > REGBL90 > REGBL95 > REGBL99 > REGBL100 What I am working on now is dynamic and not based on BIND, so if this is an attractive way to do it I will probably write it to answer subdomains from 1...100 giving 1-point resolution to choose from. Something like: genutrust.com.90pct.sr.surbl.org ...perhaps? 90pct is 90%-100% and "sr" == Spam-friendly Registrar. Of course, *assigning* the scores to the registrars will be the difficult part. > a) do a zone transfer, and grep for the values they like, to build a > custom confidence factor zone for local use, or I'm not sure zone transfers will be feasible, since the registrar determination will be made dynamically. -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Taking my gun away because I *might* shoot someone is like cutting my tongue out because I *might* yell "Fire!" in a crowded theater. -- Peter Venetoklis ---
Re: Registrar RBL: nomination and scoring
On Aug 12, 2006, at 5:11 PM, John D. Hardin wrote: b) have an RBL which returns different values for different confidence levels. Something like a percentage of known spammers are on that specific provider. So, if a registrar is 60% spammers and 40% bystanders, it will return "60"... and I can choose to only block those who have a 99% or higher rating, or something. This would also, hopefully, allow SA to give different score values to different ratings. 127.0.0.1 ... 127.0.0.100 perhaps? How would a rule to score points based on the returned IP look? Can/does SA cache the returned IP and test it in multiple rules without making multiple DNS queries? I can see a few ways of doing this: Multiple sub-zones, such as (using a registrar BL named REGBL as an example): REGBL70 (which includes everyone whose values are 127.0.0.70-127.0.0.100) REGBL80 REGBL90 REGBL95 REGBL99 REGBL100 or something like that. This would go for those RBL implementations (probably all of them) that are binary: you're either in, or not. So then the mail admin just picks whichever zone they're most comfortable with. For Spam Assassin, it could give different score values to each of those sub-zones, perhaps using metarules to give one score, or adding together the scores for each sub-zone. Then you could have a REGBLALL which is the entire list of rated hosts. From there, a given mail admin could either: a) do a zone transfer, and grep for the values they like, to build a custom confidence factor zone for local use, or b) develop an RBL implementation or score system which produces variable results. And, actually, I wish all RBLs had this type of confidence factor result instead of just being binary.