Understanding the hostKarma Lists
Responding to a lot of questions here. The lists contain both host names and IP addresses. IP addresses everyone understands. So I'll talk about host names. Wells Fargo Bank - for example - (wellsfargo.com - is in the white list as is all of Wells Fargo's hosts. This bank sends nothing but 100% good email. But to avoid spoofing of pointer records you have to use Forward Confirmed RDNS (FcRDNS). 1.2.3.4 PTR --> mail.example.com mail.example.com A --> 1.2.3.4 This is nearly impossible to spoof. Same it true for yellow lists. If the FcRDNS resolves to hotmail.com, yahoo.com, gmail.com then you can skip all other IP testing because the IP address tells you nothing about if it is or isn't spam. Warren Togami wrote: On 09/28/2009 10:07 PM, Marc Perkel wrote: I'd like to keep the name HOSTKARMA as standard. If that's so, then we probably want that in the spamassassin rule name. Your wiki page suggests JMF is the name. A number of people probably already configured their spamassassin using your suggested JMF rule names and they would need to be educated to remove it. How about these for rule names, so the rule names are not too long? RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown I'm willing go go with whatever name works better for the community. I will change my wiki to be consistent. Hi Marc, I appreciate your desire for everyone to wholly benefit from your work, but please let us implement this for spamassassin in stages starting from the lowest hanging fruit. First please confirm that you approve of the above new rule names, if you don't want it to be known as JMF. Yes - or whatever works best. I can change my wiki to reflect consensus. Hi Warren, No one has actually implemented the rules for my blacklists correctly. My lists support both IP and hostname lookups. The hostname assumes that you have forward confirmed the RDNS so that you eliminate those who might spoof. Please explain in greater detail? Can this be determined wholly from the Headers and message body after the MTA had passed the mail to the MDA? Yes - it does require 2 DNS calls to do this for FcRDNS. You need a PTR call to get the RDNS and an A record call to confirm it. Yellow means that the IP or hostname contains no useful information as to spam or no spam. On my system once I determine a host is yellow I skip all blacklists and whitelists tests. Yellow is for Yahoo, Hotmail, Gmail, etc where the IP has no information and all host tests are meaningless. My NoBL list is similar to yellow except that you can skip black list lookup but maybe might be whitelisted somewhere. Please help me better understand, what are examples of a sequence of events that would land an IP address on the NoBL? NoBL is determined a number of ways. NoBL is what most RBLs call white listing in that it means don't include it in any black list. To me white list means a spam free source. People who remove their IP manually using my form will be on the NoBL list. Or it might be what I have determined that there is some good email coming from the IP and they may be a candidate for white listing but I have yet to determine that. Yellow listing is where I know they should not be black listed but I also know they should not be white listed. (yahoo, gmail, hotmail). NoBL is where I know they should not be black listed but might be white listed. An important point to understand here is that I don't use my own lists in Spam Assassin. I do most of my filtering with Exim rules. I use my lists to avoid using SA to reduce system load. SA sees mostly yellow listed hosts. If you just want to score points then Black, White, and Brown can be assigned points. Yellow should be zero points regardless of how it tests. I am aware that Yellow isn't useful for scores. It is however useful for statistical analysis in masschecks, and it doesn't cost spamassassin any more to print if it hits. In particular I'm looking to see if there are any reliable trends of overlap between Yellow and other spamassassin rules. Fair enough. I just didn't want you assigning points to a yellow listing because the results would be false. I think the real power of my lists is in the host name lookups. It would be worthwhile to implement that. Please describe how this is more effective than IP lookups? I don't have a list of IP addresses that Yahoo uses. However, if the FcRDNS resolves to yahoo then I can skip all other RBL resting because I know it's a yahoo source. Same is true of white and black listed host names. On my system if a host name lookup returns yellow, then I add the sending IP to my yellow lists for those using IP lookups. Same with the other colors. I think my white listing is very accurate at this point. The thing about white servers is that they aren't evasive like spammers. There should be some short circuiting options to reduce system load on SA for white l
Re: Understanding the hostKarma Lists
On 09/29/2009 10:23 AM, Marc Perkel wrote: RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown I'm willing go go with whatever name works better for the community. I will change my wiki to be consistent. Hi Marc, I appreciate your desire for everyone to wholly benefit from your work, but please let us implement this for spamassassin in stages starting from the lowest hanging fruit. First please confirm that you approve of the above new rule names, if you don't want it to be known as JMF. Yes - or whatever works best. I can change my wiki to reflect consensus. It seems that people have already been using the rules copied from your site. It will be confusing to them if we change the official name. Some will accidentally have your lists twice. RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown OTOH, I really like these new names. My brain thinks less hard to recognize them. How do other people feel. Should we stick to his old names with JMF in the Wiki or these new names? Warren
Re: Understanding the hostKarma Lists
On Tue, Sep 29, 2009 at 12:06:13PM -0400, Warren Togami wrote: > On 09/29/2009 10:23 AM, Marc Perkel wrote: > RCVD_HOSTKARMA_BL Black > RCVD_HOSTKARMA_WL White > RCVD_HOSTKARMA_YL Yellow > RCVD_HOSTKARMA_BR Brown >>> >> I'm willing go go with whatever name works better for the community. I >> will change my wiki to be consistent. >> >>> Hi Marc, >>> >>> I appreciate your desire for everyone to wholly benefit from your >>> work, but please let us implement this for spamassassin in stages >>> starting from the lowest hanging fruit. >>> >>> First please confirm that you approve of the above new rule names, if >>> you don't want it to be known as JMF. >> Yes - or whatever works best. I can change my wiki to reflect consensus. > > It seems that people have already been using the rules copied from your > site. It will be confusing to them if we change the official name. Some > will accidentally have your lists twice. > > RCVD_HOSTKARMA_BL Black > RCVD_HOSTKARMA_WL White > RCVD_HOSTKARMA_YL Yellow > RCVD_HOSTKARMA_BR Brown > > OTOH, I really like these new names. My brain thinks less hard to > recognize them. Probably every single dnsbl in SA rules has the prefix RCVD_IN. Why would this drop the use then?
Re: Understanding the hostKarma Lists
On 09/29/2009 12:45 PM, Henrik K wrote: It seems that people have already been using the rules copied from your site. It will be confusing to them if we change the official name. Some will accidentally have your lists twice. RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown OTOH, I really like these new names. My brain thinks less hard to recognize them. Probably every single dnsbl in SA rules has the prefix RCVD_IN. Why would this drop the use then? We're bikeshedding here, but I believe these names are better because it is absolutely clear what it means without _IN. Shorter name is better and easier to read I think. Warren
Re: Understanding the hostKarma Lists
On Tue, Sep 29, 2009 at 12:50:13PM -0400, Warren Togami wrote: > On 09/29/2009 12:45 PM, Henrik K wrote: >>> It seems that people have already been using the rules copied from your >>> site. It will be confusing to them if we change the official name. Some >>> will accidentally have your lists twice. >>> >>> RCVD_HOSTKARMA_BL Black >>> RCVD_HOSTKARMA_WL White >>> RCVD_HOSTKARMA_YL Yellow >>> RCVD_HOSTKARMA_BR Brown >>> >>> OTOH, I really like these new names. My brain thinks less hard to >>> recognize them. >> >> Probably every single dnsbl in SA rules has the prefix RCVD_IN. Why would >> this drop the use then? >> > > We're bikeshedding here, but I believe these names are better because it > is absolutely clear what it means without _IN. Shorter name is better > and easier to read I think. Sorry but it doesn't make any sense. Also I think everyone that has got to know Marc/JMF/hostkarma will find JMF the most familiar name. Atleast I do.
Re: Understanding the hostKarma Lists
Warren Togami wrote: On 09/29/2009 12:45 PM, Henrik K wrote: It seems that people have already been using the rules copied from your site. It will be confusing to them if we change the official name. Some will accidentally have your lists twice. RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown OTOH, I really like these new names. My brain thinks less hard to recognize them. Probably every single dnsbl in SA rules has the prefix RCVD_IN. Why would this drop the use then? We're bikeshedding here, but I believe these names are better because it is absolutely clear what it means without _IN. Shorter name is better and easier to read I think. Warren What about scripts looking for occurrences of 'RCVD_IN' in log files etc? I agree it's clear without, but I'd also rather be consistent :)
Re: Understanding the hostKarma Lists
> > It seems that people have already been using the rules > copied from your site. It will be confusing to them if > we change the official name. Some will accidentally have > your lists twice. > > RCVD_HOSTKARMA_BL Black > RCVD_HOSTKARMA_WL White > RCVD_HOSTKARMA_YL Yellow > RCVD_HOSTKARMA_BR Brown > > OTOH, I really like these new names. My brain thinks > less hard to recognize them. > > How do other people feel. Should we stick to his old > names with JMF in the Wiki or these new names? > > Warren I prefer new names. Whatever the names will be, will they be stock in 3.3? I have not installed these (old or new) in my local.cf
Re: Understanding the hostKarma Lists
On 09/29/2009 12:50 PM, Warren Togami wrote: On 09/29/2009 12:45 PM, Henrik K wrote: It seems that people have already been using the rules copied from your site. It will be confusing to them if we change the official name. Some will accidentally have your lists twice. RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown OTOH, I really like these new names. My brain thinks less hard to recognize them. Probably every single dnsbl in SA rules has the prefix RCVD_IN. Why would this drop the use then? We're bikeshedding here, but I believe these names are better because it is absolutely clear what it means without _IN. Shorter name is better and easier to read I think. Warren Marc, Could you please decide between the existing JMF rule names or the above proposed HOSTKARMA names? It seems opinions are split here. Warren
Re: Understanding the hostKarma Lists
Hi! We're bikeshedding here, but I believe these names are better because it is absolutely clear what it means without _IN. Shorter name is better and easier to read I think. Could you please decide between the existing JMF rule names or the above proposed HOSTKARMA names? It seems opinions are split here. Please stick to JMF, its called like that for a long long time now. And there is installed base. Dont confuse people if its not needed. Thanks, Raymond.
Re: Understanding the hostKarma Lists
On tir 29 sep 2009 23:30:15 CEST, Warren Togami wrote Could you please decide between the existing JMF rule names or the above proposed HOSTKARMA names? It seems opinions are split here. let it be the long names that loose ? ironical you wanted to be shurt names but created a longer one ? -- xpoint
Re: Understanding the hostKarma Lists
> On tir 29 sep 2009 23:30:15 CEST, Warren Togami wrote >> Could you please decide between the existing JMF rule >> names or the above proposed HOSTKARMA names? It seems >> opinions are split here. > > let it be the long names that loose ? > > ironical you wanted to be shurt names but created a > longer one ? JMF may have been a project name, but HOSTKARMA is an will be a brand. I still vote for HOSTKARMA.
Re: Understanding the hostKarma Lists
Warren Togami wrote: On 09/29/2009 12:50 PM, Warren Togami wrote: On 09/29/2009 12:45 PM, Henrik K wrote: It seems that people have already been using the rules copied from your site. It will be confusing to them if we change the official name. Some will accidentally have your lists twice. RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown OTOH, I really like these new names. My brain thinks less hard to recognize them. Probably every single dnsbl in SA rules has the prefix RCVD_IN. Why would this drop the use then? We're bikeshedding here, but I believe these names are better because it is absolutely clear what it means without _IN. Shorter name is better and easier to read I think. Warren Marc, Could you please decide between the existing JMF rule names or the above proposed HOSTKARMA names? It seems opinions are split here. Warren If there is a lack of consensus then I appoint you Warren to make the final call. I personally have no strong preference. I do prefer something with HOSTKARMA in it rather to JEF or JMF.
Re: Understanding the hostKarma Lists
On 09/29/2009 08:56 PM, Marc Perkel wrote: Could you please decide between the existing JMF rule names or the above proposed HOSTKARMA names? It seems opinions are split here. Warren If there is a lack of consensus then I appoint you Warren to make the final call. I personally have no strong preference. I do prefer something with HOSTKARMA in it rather to JEF or JMF. To me RCVD_IN_JMF_BL is difficult for my brain to instantly recognize. It isn't the length in characters but rather the short name JMF wrapped between underscores. I was leaning towards names like RCVD_HOSTKARMA_BL or RCVD_HOSTKARMA_WL. But then some people commented about the consistency of RCVD_IN_*. RCVD_IN_HOSTKARMA_BL RCVD_IN_HOSTKARMA_WL RCVD_IN_HOSTKARMA_YL RCVD_IN_HOSTKARMA_BR These look good to me. But then we have the transition confusion problem for those who manually configured to use your old JMF rules. I will decide later after we hear more opinions. http://hostkarma.junkemailfilter.com/ Will this be a working redirector in the near future? There is no point in naming it HOSTKARMA if none of the URL's have hostkarma in their name. Warren Togami wtog...@redhat.com
RE: Understanding the hostKarma Lists
> > RCVD_HOSTKARMA_BL Black > RCVD_HOSTKARMA_WL White > RCVD_HOSTKARMA_YL Yellow > RCVD_HOSTKARMA_BR Brown > > OTOH, I really like these new names. My brain thinks less > hard to recognize them. > > How do other people feel. Should we stick to his old names > with JMF in the Wiki or these new names? > > Warren > > please keep the original names using JMF since Perkel chose them and it is more descriptive of his "domain" and nobody has to change anything from what they have now (generally) - rh
RE: Understanding the hostKarma Lists
> > Marc, > > Could you please decide between the existing JMF rule names > or the above proposed HOSTKARMA names? It seems opinions are > split here. > > Warren > > warren, marc already decided once, please dont give more choices... you should have thought that out before putting the list in a minor tiz on it. - rh
Re: Understanding the hostKarma Lists
On 09/30/2009 12:18 AM, R-Elists wrote: Marc, Could you please decide between the existing JMF rule names or the above proposed HOSTKARMA names? It seems opinions are split here. Warren warren, marc already decided once, please dont give more choices... you should have thought that out before putting the list in a minor tiz on it. - rh I'll note that he's the one that said he prefers HOSTKARMA names, despite his own Wiki saying JMF. Warren
Re: Understanding the hostKarma Lists
Warren Togami wrote: > On 09/30/2009 12:18 AM, R-Elists wrote: >> >> warren, >> >> marc already decided once, please dont give more choices... >> >> you should have thought that out before putting the list in a minor >> tiz on >> it. >> >> - rh >> > > I'll note that he's the one that said he prefers HOSTKARMA names, > despite his own Wiki saying JMF. > > Warren I'll put in my vote for RCVD_IN_HOSTKARMA_* This keeps the RCVD_IN for consistency with the rest of the blacklist rules and also uses the more easily recognizable Hostkarma name. If you add this to the default rules make sure that there is an obvious note in the release notes for those who already have the rules installed with the old names. -- Bowie
RE: Understanding the hostKarma Lists
> > I'll note that he's the one that said he prefers HOSTKARMA > names, despite his own Wiki saying JMF. > > Warren > > Warren, so noted... :-) his wiki and his entries in the SA wiki too... and this isnt a witch hunt by any means... you desiring to set it up and run it through the SA sandbox "appears" to be a great idea in it's "present form" yet when you brought it up and looked at starting changing naming conventions and the existing RULE names and asking the person his feelings and then kinda asking for a public vote etc etc... i know you get the picture. even more importantly, YAN brought up other important things in the "Hostkarma: to be or not to be in SA defaults" thread start... :-) i appreciate Perkel's work as much or more than anyone yet i still remember some time back when he changed some dns hostname stuff without warning everyone and it made it so all emails checked against his list(s) were rejected... ;-)doh! - rh
Re: Understanding the hostKarma Lists
R-Elists wrote: RCVD_HOSTKARMA_BL Black RCVD_HOSTKARMA_WL White RCVD_HOSTKARMA_YL Yellow RCVD_HOSTKARMA_BR Brown OTOH, I really like these new names. My brain thinks less hard to recognize them. How do other people feel. Should we stick to his old names with JMF in the Wiki or these new names? Warren please keep the original names using JMF since Perkel chose them and it is more descriptive of his "domain" and nobody has to change anything from what they have now (generally) - rh Actually I didn't choose them. Someone wrote rules that used those names and I just copied the code.
Re: Understanding the hostKarma Lists
I like it. RCVD_IN_HOSTKARMA_BL RCVD_IN_HOSTKARMA_WL RCVD_IN_HOSTKARMA_YL RCVD_IN_HOSTKARMA_BR Let's go with it. Warren Togami wrote: On 09/29/2009 08:56 PM, Marc Perkel wrote: Could you please decide between the existing JMF rule names or the above proposed HOSTKARMA names? It seems opinions are split here. Warren If there is a lack of consensus then I appoint you Warren to make the final call. I personally have no strong preference. I do prefer something with HOSTKARMA in it rather to JEF or JMF. To me RCVD_IN_JMF_BL is difficult for my brain to instantly recognize. It isn't the length in characters but rather the short name JMF wrapped between underscores. I was leaning towards names like RCVD_HOSTKARMA_BL or RCVD_HOSTKARMA_WL. But then some people commented about the consistency of RCVD_IN_*. RCVD_IN_HOSTKARMA_BL RCVD_IN_HOSTKARMA_WL RCVD_IN_HOSTKARMA_YL RCVD_IN_HOSTKARMA_BR These look good to me. But then we have the transition confusion problem for those who manually configured to use your old JMF rules. I will decide later after we hear more opinions. http://hostkarma.junkemailfilter.com/ Will this be a working redirector in the near future? There is no point in naming it HOSTKARMA if none of the URL's have hostkarma in their name. Warren Togami wtog...@redhat.com
Re: Understanding the hostKarma Lists
Marc Perkel wrote: > I like it. > > RCVD_IN_HOSTKARMA_BL > RCVD_IN_HOSTKARMA_WL > RCVD_IN_HOSTKARMA_YL > RCVD_IN_HOSTKARMA_BR > > Let's go with it. Marc, have you updated your wiki to reflect the new rules? I think that will pretty well settle any debate or question people have. --Blaine
Re: Understanding the hostKarma Lists
On Wed, 30 Sep 2009, Marc Perkel wrote: RCVD_IN_HOSTKARMA_BL RCVD_IN_HOSTKARMA_WL RCVD_IN_HOSTKARMA_YL RCVD_IN_HOSTKARMA_BR +1 -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Think Microsoft cares about your needs at all? "A company wanted to hold off on upgrading Microsoft Office for a year in order to do other projects. So Microsoft gave a 'free' copy of the new Office to the CEO -- a copy that of course generated errors for anyone else in the firm reading his documents. The CEO got tired of getting the 'please re-send in XX format' so he ordered other projects put on hold and the Office upgrade to be top priority."-- Cringely, 4/8/2004 --- Approximately 9021060 firearms legally purchased in the U.S. this year
Re: Understanding the hostKarma Lists
On ons 30 sep 2009 19:17:46 CEST, John Hardin wrote RCVD_IN_HOSTKARMA_BL RCVD_IN_HOSTKARMA_WL RCVD_IN_HOSTKARMA_YL RCVD_IN_HOSTKARMA_BR +1 -2 the rule name is now longer then it was :/ -- xpoint
Re: Understanding the hostKarma Lists
Blaine Fleming wrote: Marc Perkel wrote: I like it. RCVD_IN_HOSTKARMA_BL RCVD_IN_HOSTKARMA_WL RCVD_IN_HOSTKARMA_YL RCVD_IN_HOSTKARMA_BR Let's go with it. Marc, have you updated your wiki to reflect the new rules? I think that will pretty well settle any debate or question people have. --Blaine Yes - the wiki is updated.
RE: Understanding the hostKarma Lists
marc dont forget this one http://wiki.apache.org/spamassassin/MarcPerkelsExperiments - rh _ From: Marc Perkel [mailto:m...@perkel.com] snip Yes - the wiki is updated.
Re: Understanding the hostKarma Lists
From: "Marc Perkel"
Sent: Wednesday, 2009/September/30 16:41
Blaine Fleming wrote:
Marc Perkel wrote:
I like it.
RCVD_IN_HOSTKARMA_BL
RCVD_IN_HOSTKARMA_WL
RCVD_IN_HOSTKARMA_YL
RCVD_IN_HOSTKARMA_BR
Let's go with it.
Marc, have you updated your wiki to reflect the new rules? I think that
will pretty well settle any debate or question people have.
--Blaine
Yes - the wiki is updated.
I installed it on my personal mail for testing, Marc. I forwarded an
email that failed within minutes of installing it. The bozo was in the
whitelist and hit quite a few rules including a 5.0001 Bayes 99. It
still got through with a 4.9 total because of the bogus whitelist
rule hit and its bogus score. "Whitelists aren't" is my rule.
{^_^}
Re: Understanding the hostKarma Lists
On 10/01/2009 12:42 PM, jdow wrote:
From: "Marc Perkel"
Sent: Wednesday, 2009/September/30 16:41
Blaine Fleming wrote: Marc Perkel wrote:
I like it.
RCVD_IN_HOSTKARMA_BL
RCVD_IN_HOSTKARMA_WL
RCVD_IN_HOSTKARMA_YL
RCVD_IN_HOSTKARMA_BR
Let's go with it.
Marc, have you updated your wiki to reflect the new rules? I think that
will pretty well settle any debate or question people have.
--Blaine
Yes - the wiki is updated.
I installed it on my personal mail for testing, Marc. I forwarded an
email that failed within minutes of installing it. The bozo was in the
whitelist and hit quite a few rules including a 5.0001 Bayes 99. It
still got through with a 4.9 total because of the bogus whitelist
rule hit and its bogus score. "Whitelists aren't" is my rule.
{^_^}
spamassassin's default scores do not give big negative scores to any of
the whitelist rules for a good reason. They are mainly informational.
Warren
Re: Understanding the hostKarma Lists
Updated that as well. R-Elists wrote: marc dont forget this one http://wiki.apache.org/spamassassin/MarcPerkelsExperiments - rh From: Marc Perkel [mailto:m...@perkel.com] snip Yes - the wiki is updated.
