Re: score based on a list of domains
Hi! Easiest way would be putting them inside a uribl. Whats the reason to get on this list? Eg what policy? Thanks, Raymond Dijkxhoorn, Prolocation Op 13 dec. 2011 om 08:54 heeft Tom Kinghorn thomas.kingh...@gmail.com het volgende geschreven: Good morning List. The nice guys at Rhyolite.com have a list of unwelcome domains http://www.rhyolite.com/anti-spam/unwelcome-a.html Is it possible to use this list to increase the message score? If yes, how would one read this list into a check? Thanks Tom
Re: Using ZMI_GERMAN ruleset
On 2011-12-13 8:34, Michael Monnerie wrote:
On Montag, 31. Oktober 2011 Axb wrote:
tried it and dumped due to low hit rate
stuff like
body ZMIde_JOBSEARCH6 /Dank sehr grossen Angagement, aber auch
der Umsetzung verschiedener Inovationen, konnte unsere Firma schon
nach vier Jahren auf die internationalen Ebene hinaufsteigen/
is not efficient
Its efficient in terms of filtering only spam with zero false
positives, which is top priority for this ruleset. And you picked a
very old and very long rule. Most rules nowadays are just one or even
only part of a sentence, and it prooves very efficient. Stuff like the
__ZMIde_JOBEARN1-28 rules move false positives to 0, and I'm constantly
adding stuff.
I've now tried to remove all old cruft, that means single-line rules.
Rulesize went from 350KB to 296KB, that should save some RAM and CPU.
patterns with 120 characters are not really efficient, in terms of
speed and hit rate. They are very specific to certain campaigns and
minimal template changes will render them useless as in:
body __ZMIde_STOCK34 /Wir sind .{0,2}berzeugt, dass der Zeitpunkt
sich an einem Unternehmen zu beteiligen, welches erfolgreich im
Edelmetallhandel t.{0,2}tig ist, nicht besser sein k.{0,2}nnte/
or
body __ZMIde_SALE5 /In den letzten 5 Jahren hatte ich .{0,2}ber drei
dutzend gut funktionierende Strategien, um die Zahl meiner
Webseitenbesucher drastisch zu erh.{0,2}hen und dadurch meinen Umsatz
anzukurbeln/
Re: DNSWL will be disabled by default as of tomorrow
On 12/13/2011 2:19 AM, Dave Warren wrote: On 12/12/2011 5:27 PM, Karsten Bräckelmann wrote: No. SA should be usable out-of-the-box with best possible performance for the majority of users. Perhaps a better long-term solution would be to validate DNS lists before using them? One possible implementation would be to test to ensure that 127.0.0.1 is not listed, and 127.0.0.2 is listed (with the testing criteria being configurable, but this is a starting point that will work for most lists). If a list is down or unresponsive for any reason, discards requests or blanks their zone file, the test entry would fail and SA would know to not use the list. Similarly, 127.0.0.1 should never be listed for any DNSBL that I'm aware of, and so when a list moves to a list-the-world configuration, this entry would spot it. Unfortunately, 1 is a bitwise answer I've seen it used. In fact, just checking real quick, I've got an RBL that uses 1 on a live server now. # Last octet of A record is a bitmask: # x 1 = greylist # x 2 = dirty # x 4 = spammer # x 8 = good IMO, unfortunately again there is no standard to RBL responses though I think that multi/combined lists could define an octet that is a blocked answer combine with a simple rule. Then they just need to be publishing a combined list to do that and not use the other octets (i.e. return the bitwise for blocked only or at least no purposefully incorrect answers). The score on the rule that acknowledges a block should be minimal and the message on the rule would have to link to a generic page on SA's wiki regarding free for some services. It should NOT lead to a subscription page for a vendor. We are not an advertising service. If the RBL is using a combined bitwise solution, that's a solution that would work in the existing rule structure on multiple SA platforms. Hopefully, they can also give a high TTL on the blocked query answer so caching is more effective but at the end of the day, this still means querys are coming in. So what has the RBL operator gained? Blocking seems to be the only thing that really achieves the goal they want beyond conversion to paying customers which is not SA's issue. Regards, KAM
Re: DNSWL will be disabled by default as of tomorrow
On 2011-12-13 13:44, Kevin A. McGrail wrote: On 12/13/2011 2:19 AM, Dave Warren wrote: On 12/12/2011 5:27 PM, Karsten Bräckelmann wrote: No. SA should be usable out-of-the-box with best possible performance for the majority of users. Perhaps a better long-term solution would be to validate DNS lists before using them? One possible implementation would be to test to ensure that 127.0.0.1 is not listed, and 127.0.0.2 is listed (with the testing criteria being configurable, but this is a starting point that will work for most lists). If a list is down or unresponsive for any reason, discards requests or blanks their zone file, the test entry would fail and SA would know to not use the list. Similarly, 127.0.0.1 should never be listed for any DNSBL that I'm aware of, and so when a list moves to a list-the-world configuration, this entry would spot it. Unfortunately, 1 is a bitwise answer I've seen it used. In fact, just checking real quick, I've got an RBL that uses 1 on a live server now. well, that BL would have to change - no big deal (we know who that is :) # Last octet of A record is a bitmask: # x 1 = greylist # x 2 = dirty # x 4 = spammer # x 8 = good IMO, unfortunately again there is no standard to RBL responses though I think that multi/combined lists could define an octet that is a blocked answer combine with a simple rule. Then they just need to be publishing a combined list to do that and not use the other octets (i.e. return the bitwise for blocked only or at least no purposefully incorrect answers). The score on the rule that acknowledges a block should be minimal and the message on the rule would have to link to a generic page on SA's wiki regarding free for some services. It should NOT lead to a subscription page for a vendor. We are not an advertising service. If the RBL is using a combined bitwise solution, that's a solution that would work in the existing rule structure on multiple SA platforms. Hopefully, they can also give a high TTL on the blocked query answer so caching is more effective but at the end of the day, this still means querys are coming in. So what has the RBL operator gained? Blocking seems to be the only thing that really achieves the goal they want beyond conversion to paying customers which is not SA's issue. been talking to SF about possibilities he has some interesting ideas (I'm not 100% sold on them, but he makes sense) - stay tuned.
Re: score based on a list of domains
On 12/13/11 3:38 AM, Raymond Dijkxhoorn wrote: Hi! Easiest way would be putting them inside a uribl. Whats the reason to get on this list? Eg what policy? The policy is clearly stated on their web site, first paragraph of that link. I believe it is a private list, not meant to be used for spam blocking. -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 *| *SECNAP Network Security Corporation * Best Mobile Solutions Product of 2011 * Best Intrusion Prevention Product * Hot Company Finalist 2011 * Best Email Security Product * Certified SNORT Integrator __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.spammertrap.com/ __
Re: DNSWL will be disabled by default as of tomorrow
On 12/13/11 7:44 AM, Kevin A. McGrail wrote: Blocking seems to be the only thing that really achieves the goal they want beyond conversion to paying customers which is not SA's issue. I agree with Kevin. A while back, I published an 'example' blocking list, 'blocked.secnap.net' (wildcard entry for ipv4 :-). Guess what? it was added to a couple of perl dnsbl modules and used by people who never looked at what it was! Two things happened: #1, lots of (hundreds of thousands of queries per day) from one or two unnamed large ISP's #2, calls from 'internet lawyers' demanding that we remove them from the list. (we emailed them the bind zone and told them to identify their ip address and we would gladly remove it). Also, emailing or calling 'abusers' doesn't work. Kevin and I both run two of three sa-update mirror servers, and we have seen several 'ill configured' servers that try to pull the same sa-update every 5 mins forever. I had our night shift guys track down and send the admins a friendly note, mentioning that they aren't getting the updates anyway, so why not fix it? No response, no change in activity (note: this might be due to one of the distro's not being able to store and check pgp keys if they are in the /tmp directory, a proposed SA bugzilla starts to address this, but these queries are for older versions of SA) And/or full /tmp filesystems, etc. We never did figure it out, but if anyone wants a list of the top 10 ip's, they can email me offlist. Now, I disagree TOTALLY on setting the 'abuser's dns queries to return FP on DNSWL_HIGH, this serves no purpose. Blocking the ip address by firewall will save bandwidth and cpu cycles. returning FP on HIGH won't ever get google's attention, will it? and you still get the bandwidth and cpu cycles from the largest abusers. Regards, KAM -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 *| *SECNAP Network Security Corporation * Best Mobile Solutions Product of 2011 * Best Intrusion Prevention Product * Hot Company Finalist 2011 * Best Email Security Product * Certified SNORT Integrator __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.spammertrap.com/ __
Re: DNSWL will be disabled by default as of tomorrow
On Tue, 2011-12-13 at 13:52 +0100, Axb wrote: On 2011-12-13 13:44, Kevin A. McGrail wrote: If a list is down or unresponsive for any reason, discards requests or blanks their zone file, the test entry would fail and SA would know to not use the list. Similarly, 127.0.0.1 should never be listed for any DNSBL that I'm aware of, and so when a list moves to a list-the-world configuration, this entry would spot it. Unfortunately, 1 is a bitwise answer I've seen it used. In fact, just checking real quick, I've got an RBL that uses 1 on a live server now. At the risk of exposing my ignorance, I had a thought. Since the entire 127/8 is reserved for loopback, nothing in the 127.0.0/24 block should be used as addresses. So, what is preventing RBLs and RWLs from using the third octet as a status indicator? It seems to me that the 4th octet can be used as at present as a query response which would by convention be a valid response if the 3rd octet is zero. OTOH if the 3rd octet was non-zero, this would indicate that the response is invalid, so this could provide an unambiguous way for the various RBLs and WLs to tell individual users to go away if they exceeded use limits, had an expired subscription, etc. This seems such an obvious solution that it must have been thought of in the past and rejected for some reason I don't understand. What did I miss? Martin well, that BL would have to change - no big deal (we know who that is :) # Last octet of A record is a bitmask: # x 1 = greylist # x 2 = dirty # x 4 = spammer # x 8 = good IMO, unfortunately again there is no standard to RBL responses though I think that multi/combined lists could define an octet that is a blocked answer combine with a simple rule. Then they just need to be publishing a combined list to do that and not use the other octets (i.e. return the bitwise for blocked only or at least no purposefully incorrect answers). The score on the rule that acknowledges a block should be minimal and the message on the rule would have to link to a generic page on SA's wiki regarding free for some services. It should NOT lead to a subscription page for a vendor. We are not an advertising service. If the RBL is using a combined bitwise solution, that's a solution that would work in the existing rule structure on multiple SA platforms. Hopefully, they can also give a high TTL on the blocked query answer so caching is more effective but at the end of the day, this still means querys are coming in. So what has the RBL operator gained? Blocking seems to be the only thing that really achieves the goal they want beyond conversion to paying customers which is not SA's issue. been talking to SF about possibilities he has some interesting ideas (I'm not 100% sold on them, but he makes sense) - stay tuned.
Re: score based on a list of domains
On Tue, 13 Dec 2011 08:47:18 -0500 Michael Scheidell wrote: On 12/13/11 3:38 AM, Raymond Dijkxhoorn wrote: Hi! Easiest way would be putting them inside a uribl. Whats the reason to get on this list? Eg what policy? The policy is clearly stated on their web site, first paragraph of that link. I believe it is a private list, not meant to be used for spam blocking. And I spotted adidas.com is in the first few domains which doesn't bode well.
Re: DNSWL will be disabled by default as of tomorrow
On 12/13/11 8:09 AM, Martin Gregorie mar...@gregorie.org wrote: On Tue, 2011-12-13 at 13:52 +0100, Axb wrote: On 2011-12-13 13:44, Kevin A. McGrail wrote: If a list is down or unresponsive for any reason, discards requests or blanks their zone file, the test entry would fail and SA would know to not use the list. Similarly, 127.0.0.1 should never be listed for any DNSBL that I'm aware of, and so when a list moves to a list-the-world configuration, this entry would spot it. Unfortunately, 1 is a bitwise answer I've seen it used. In fact, just checking real quick, I've got an RBL that uses 1 on a live server now. At the risk of exposing my ignorance, I had a thought. Since the entire 127/8 is reserved for loopback, nothing in the 127.0.0/24 block should be used as addresses. So, what is preventing RBLs and RWLs from using the third octet as a status indicator? It seems to me that the 4th octet can be used as at present as a query response which would by convention be a valid response if the 3rd octet is zero. I have in the past seen at least one DNSBL that used the 3rd octet, as they had more than 8 lists in a multi-configuration. I don't recall which one it was... -- Daniel J McDonald, CCIE # 2495, CISSP # 78281
Re: DNSWL will be disabled by default as of tomorrow
On Tue, Dec 13, 2011 at 3:00 PM, Michael Scheidell michael.scheid...@secnap.com wrote: [..] Blocking the ip address by firewall will save bandwidth and cpu cycles. Firewalling will have the same effect as returning no answer - it will cause retries and thus will roughly triple the amount of queries received (although they will effectively be discarded at a different stage, firewall vs nameserver). This effect was also reported in https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6048#c23 google's attention, will it? and you still get the bandwidth and cpu cycles from the largest abusers. For the few cases where it is being used, it reduced the load by an order of a magnitude (eg netline.net.uk going from 12 mio queries/24 hours to below 1 mio - still way too high, but definitely an improvement). -- Matthias
Re: DNSWL will be disabled by default as of tomorrow
On Tue, 13 Dec 2011, Kevin A. McGrail wrote: On 12/13/2011 2:19 AM, Dave Warren wrote: Perhaps a better long-term solution would be to validate DNS lists before using them? One possible implementation would be to test to ensure that 127.0.0.1 is not listed Similarly, 127.0.0.1 should never be listed for any DNSBL that I'm aware of, and so when a list moves to a list-the-world configuration, this entry would spot it. Unfortunately, 1 is a bitwise answer I've seen it used. In fact, just checking real quick, I've got an RBL that uses 1 on a live server now. Let's rephrase: querying 127.0.0.1 should never return a positive answer. Returning 127.0.0.1 as an answer is not a problem. This seems to me to be a reasonable test. If the BL returns a hit, and if it hasn't been validated in the last X hours, then query 127.0.0.1 and see if the list returns a positive. If so, discard the hit and suppress querying the list for the next Y hours. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- North Korea: the only country in the world where people would risk execution to flee to communist China. -- Ride Fast --- 2 days until Bill of Rights day
DNS{B,W}Ls and blocking (was Re: DNSWL will be disabled by default as of tomorrow)
I think we need an informational RFC that specifies best-practices for
a DNS{B,W}L to inform clients that they have been blocked.
For example, a testpoint like:
blocked.dnsbl.example.org
could return an A record for name servers that are blocked and NXDOMAIN
for others. This might even work out-of-the-box for some existing lists
that return an A record for any query (or it may not, if they expect
a reverse-dotted-quad.)
It could even return a TXT record giving the reason for the block.
Anyway, assuming this idea is widely-accepted (hahaha!), it would be pretty
easy to make something that periodically tests your list of DNSBLs and
disables those that are blocking your query.
Regards,
David.
Re: DNS{B,W}Ls and blocking (was Re: DNSWL will be disabled by default as of tomorrow)
On 2011-12-13 15:21, David F. Skoll wrote:
I think we need an informational RFC that specifies best-practices for
a DNS{B,W}L to inform clients that they have been blocked.
For example, a testpoint like:
blocked.dnsbl.example.org
could return an A record for name servers that are blocked and NXDOMAIN
for others. This might even work out-of-the-box for some existing lists
that return an A record for any query (or it may not, if they expect
a reverse-dotted-quad.)
It could even return a TXT record giving the reason for the block.
Anyway, assuming this idea is widely-accepted (hahaha!), it would be pretty
easy to make something that periodically tests your list of DNSBLs and
disables those that are blocking your query.
there is a BCP for BLs floating around which addresses some if not all
of this... MAAWF, RFC.. not sure which
Re: DNSWL will be disabled by default as of tomorrow
On Tue, 13 Dec 2011 14:09:19 + Martin Gregorie wrote: At the risk of exposing my ignorance, I had a thought. Since the entire 127/8 is reserved for loopback, nothing in the 127.0.0/24 block should be used as addresses. So, what is preventing RBLs and RWLs from using the third octet as a status indicator? It seems to me that the 4th octet can be used as at present as a query response which would by convention be a valid response if the 3rd octet is zero. OTOH if the 3rd octet was non-zero, this would indicate that the response is invalid, so this could provide an unambiguous way for the various RBLs and WLs to tell individual users to go away if they exceeded use limits, had an expired subscription, etc. In DNSWL the 3rd octet contains a code for the type of business. They could define an invalid code in the last octet if they wanted to, but presumably they want people to notice if they are over quota.
Re: DNS{B,W}Ls and blocking (was Re: DNSWL will be disabled by default as of tomorrow)
On 12/13/2011 9:21 AM, David F. Skoll wrote:
I think we need an informational RFC that specifies best-practices for
a DNS{B,W}L to inform clients that they have been blocked.
For example, a testpoint like:
blocked.dnsbl.example.org
could return an A record for name servers that are blocked and NXDOMAIN
for others. This might even work out-of-the-box for some existing lists
that return an A record for any query (or it may not, if they expect
a reverse-dotted-quad.)
It could even return a TXT record giving the reason for the block.
Anyway, assuming this idea is widely-accepted (hahaha!), it would be pretty
easy to make something that periodically tests your list of DNSBLs and
disables those that are blocking your query.
This was mentioned as a possibility and it's a good idea.
But from SA's perspective, though, it means that it requires code. And
the big issue is NOT the delays. The big issue is the purposefully wrong
answers.
The code-requirement for a fix means that this new policy is delayed at
least 6 months after a major release for SA based on
http://wiki.apache.org/spamassassin/ReleaseGoals. So if we miss this
code getting into 3.4.0, that means it waits until 3.5.0 (or 4.0.0) + 6
months. If someone wants to submit code to actually do it, that'd be
great. But it's a got a delay before it matters either way.
I've opened a ticket
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6724 towards an
immediate solution.
regards,
KAM
Re: DNSWL will be disabled by default as of tomorrow
On 12/13/2011 5:44 AM, Kevin A. McGrail wrote: On 12/13/2011 2:19 AM, Dave Warren wrote: On 12/12/2011 5:27 PM, Karsten Bräckelmann wrote: No. SA should be usable out-of-the-box with best possible performance for the majority of users. Perhaps a better long-term solution would be to validate DNS lists before using them? One possible implementation would be to test to ensure that 127.0.0.1 is not listed, and 127.0.0.2 is listed (with the testing criteria being configurable, but this is a starting point that will work for most lists). If a list is down or unresponsive for any reason, discards requests or blanks their zone file, the test entry would fail and SA would know to not use the list. Similarly, 127.0.0.1 should never be listed for any DNSBL that I'm aware of, and so when a list moves to a list-the-world configuration, this entry would spot it. Unfortunately, 1 is a bitwise answer I've seen it used. In fact, just checking real quick, I've got an RBL that uses 1 on a live server now. # Last octet of A record is a bitmask: # x 1 = greylist # x 2 = dirty # x 4 = spammer # x 8 = good IMO, unfortunately again there is no standard to RBL responses though I think that multi/combined lists could define an octet that is a blocked answer combine with a simple rule. Sorry, I might not have been entirely clear. I'm suggesting we create a SpamAssassin syntax to say This IP must be listed and This IP must not be listed, both of which must be true for a DNSBL to service. The specific IPs selected are just for example purposes and can be tweaked on a per-list basis based on the list's design. Depending on how difficult this would be to implement within SpamAssassin, you could write the test rule such that it supplies an IP to be tested, a SA rule name and a required score (so that, for example, you could test bit-based lists) If I DNSBL operator cannot publish an IP that will always be listed and an IP that will never be listed, they simply wouldn't be candidates for implementation in SpamAssassin; since virtually all DNSBLs have some sort of test IPs, this probably won't be a big deal. The point is that I'm not suggesting we herd cats and require every DNSBL to agree upon test addresses, but rather, that we go through on a blocklist-by-blocklist basis and use their existing test addresses. If the RBL is using a combined bitwise solution, that's a solution that would work in the existing rule structure on multiple SA platforms. Hopefully, they can also give a high TTL on the blocked query answer so caching is more effective but at the end of the day, this still means querys are coming in. So what has the RBL operator gained? Blocking seems to be the only thing that really achieves the goal they want beyond conversion to paying customers which is not SA's issue. This system would result in one query per BL per SA restart, or per ruleset reload or per hour or whatever, rather than one or more queries per processed message. That's a step forward to DNSBL operators, but more importantly, it would avoid the situation where users are negatively impacted by BL failures. -- Dave Warren, CEO Hire A Hit Consulting Services http://ca.linkedin.com/in/davejwarren
Re: DNSWL will be disabled by default as of tomorrow
This system would result in one query per BL per SA restart, or per ruleset reload or per hour or whatever, rather than one or more queries per processed message. That's a step forward to DNSBL operators, but more importantly, it would avoid the situation where users are negatively impacted by BL failures. Definitely on the same page. My thoughts are to build on the block notification rules to implement code that blocks the DNSBL queries for 1 hour. However, that's kind of a phase II. And since I doubt there will be consensus from DNSBL operators, it'll really be a one off thing per DNSBL to implement unless some alignment of planets occurs that I doubt is even in motion ;-) https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6724 Regards, KAM
Re: DNSWL will be disabled by default as of tomorrow
On 12/13/2011 10:37 AM, Kevin A. McGrail wrote: This system would result in one query per BL per SA restart, or per ruleset reload or per hour or whatever, rather than one or more queries per processed message. That's a step forward to DNSBL operators, but more importantly, it would avoid the situation where users are negatively impacted by BL failures. Definitely on the same page. My thoughts are to build on the block notification rules to implement code that blocks the DNSBL queries for 1 hour. However, that's kind of a phase II. And since I doubt there will be consensus from DNSBL operators, it'll really be a one off thing per DNSBL to implement unless some alignment of planets occurs that I doubt is even in motion ;-) https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6724 I don't think there really needs to be consensus. I've yet to see one that blocks 127.0.0.1, and they all have some sort of test address (usually 127.0.0.x) Given that the worst that happens if this system fails is that SA stops using the list until sa-update updates the check rule, as long as the test IPs can be configured on a per-DNSBL basis, there shouldn't really be a problem. * DNSBL includes DNSWLs, domain based lists, etc... All we need is a this entry should cause a result and this entry should not, whether it's positive or negative, an IP or domain, etc, shouldn't matter. -- Dave Warren, CEO Hire A Hit Consulting Services http://ca.linkedin.com/in/davejwarren
Re: DNSWL will be disabled by default as of tomorrow
I don't think there really needs to be consensus. I've yet to see one that blocks 127.0.0.1, and they all have some sort of test address (usually 127.0.0.x) Given that the worst that happens if this system fails is that SA stops using the list until sa-update updates the check rule, as long as the test IPs can be configured on a per-DNSBL basis, there shouldn't really be a problem. * DNSBL includes DNSWLs, domain based lists, etc... All we need is a this entry should cause a result and this entry should not, whether it's positive or negative, an IP or domain, etc, shouldn't matter. You're welcome to give it a whirl to come up with code to do the testing but doing it on start-up is likely bound to have lots of problems with servers rebooting that don't have net access yet, etc. As I put on the bug, I think the best solution will be something that internally monitors for block rules and if triggered, stops queries to those BLs for an hour. Then it can try again. Your idea might be better and I'm having forest for the trees vision. regards, KAM
DNS list inclusion policy (was Re: DNSWL will be disabled by default as of tomorrow)
Hi, Is there a formal policy for including (or excluding) DNS-based lists from SpamAssassin's default rules? I think that SpamAssassin should not enable any DNS-based list by default unless the list owners have a clear policy that the list is free to use by SpamAssassin users, regardless of query volume [*], commercial vs. non-commercial use, etc. and a promise not to change the policy without reasonable (say, six months) notice. I realize this means practically no DNS lists will qualify to be enabled by default. I believe SA should ship all the rules for various DNS lists and have clear instructions (maybe even a tool for newbies that runs automatically from the makefile) to enable them. But I'm not sure it should serve as a marketing tool for DNSBL operators who want to charge money. (/me dons flame-proof suit...) Regards, David. [*] DNSBLs that force you to use a (free) rsync service if your query volume is too high are OK.
Re: DNS list inclusion policy (was Re: DNSWL will be disabled by default as of tomorrow)
On 12/13/2011 2:00 PM, David F. Skoll wrote: Hi, Is there a formal policy for including (or excluding) DNS-based lists from SpamAssassin's default rules? I think that SpamAssassin should not enable any DNS-based list by default unless the list owners have a clear policy that the list is free to use by SpamAssassin users, regardless of query volume [*], commercial vs. non-commercial use, etc. and a promise not to change the policy without reasonable (say, six months) notice. I realize this means practically no DNS lists will qualify to be enabled by default. I believe SA should ship all the rules for various DNS lists and have clear instructions (maybe even a tool for newbies that runs automatically from the makefile) to enable them. But I'm not sure it should serve as a marketing tool for DNSBL operators who want to charge money. (/me dons flame-proof suit...) Regards, David. [*] DNSBLs that force you to use a (free) rsync service if your query volume is too high are OK. You are a brave person! There is formal consensus from the PMC that work for most installations is adequate for default inclusion once the merits of the BL are shown combined with discussion if the infrastructure can withstand the query onslaught. Beyond that, we have the new issue of responding with purposefully wrong answers which is a no-no for default inclusion. However, this issue is NOT new. This issue was discussed by the PMC when URIBL was being considered for inclusion: NOTE: URIBL fails with false positives once a threshold is exceeded. This will not be acceptable for a default enabled SA test. and the response: UPDATE FROM DALLAS: We don't return positive to anymore. We moved our heavy hitter blocking to the top level, so blocked users will timeout trying to reach our mirrors.. But consensus for the PMC is clear, that free for some licensing with query volumes being acceptable. From my notes, a daily limit and ok for non-commercial use has been acceptable to SA to consider for default enabling. Of course, we've had a LOT of discussions on this matter and Spamhaus, for example, changed their query and SMTP limits based on discussions with SA. So I know we try and be fair but flexible so we are not closed to new BLs. I also personally try and support BLs by offering nameserver resources. In the end, your idea of an easier way for admins to review rules/BL for inclusion/removal has good merit. I've had similar ideas with at least making a list of rules that are disabled for these type of reasons more in the forefront. The biggest con I can see is that rules and releases are purposefully separated. If masscheck could be augmented, having LOTS of different sa-update channels might be a good idea but that won't enable the plugins. Now what's more interesting is that I took on the crazy idea of writing a policy for RBL inclusion. Let me formalize my notes and see if I can post something today about that. It'll be really for the PMC to decide but user/dev/rbl operator input is always good. Regards, KAM
Re: DNS list inclusion policy (was Re: DNSWL will be disabled by default as of tomorrow)
On 12/13, Kevin A. McGrail wrote: Is there a formal policy for including (or excluding) DNS-based lists whats There is formal consensus from the PMC that ^ work^ for most installations is adequate for default inclusion once the merits of the BL are shown I wanted to point out the logic behind this. SpamAssassin is heavily dependent on network tests for accuracy. Disabling them results in something like 5 times as many emails mis-classified, both false positives and false negatives (with the default threshold of 5). So the default configuration is set up to need minimal adjustment for small installations likely to have the least ability to do those adjustments, and require more adjustment for very large installations which are inevitably going to need to change stuff anyway. If you can come up with a way to disable all network tests that have a free use limit without crippling spamassassin, please tell us. That would be lovely. And I do think it's appropriate to discuss this in terms of all network tests, not just DNS tests. These are the statistics from... 2009. Would be nice to get newer ones. Why aren't these updated? Set 0, no net, no bayes: http://svn.apache.org/viewvc/spamassassin/trunk/rules/STATISTICS-set0.txt?view=markup # False positives: 238 1.12% # False negatives: 9678 21.93% Set 1, with net, no bayes: http://svn.apache.org/viewvc/spamassassin/trunk/rules/STATISTICS-set1.txt?view=markup # False positives:30 0.14% # False negatives: 1381 3.13% Without the network tests, it got 7.9x as many spams wrong, and 7.0x as many hams wrong. (Opened bug for those files being out of date, I think the data gets generated daily: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6726 ) -- I love God. He's so deliciously evil. - Stewie Griffin, Family Guy 2x02 http://www.ChaosReigns.com
Re: DNS list inclusion policy (was Re: DNSWL will be disabled by default as of tomorrow)
On Tue, 13 Dec 2011 15:24:34 -0500 dar...@chaosreigns.com wrote: If you can come up with a way to disable all network tests that have a free use limit without crippling spamassassin, please tell us. That would be lovely. I think something like this would be good: $ tar xvfz Mail-SpamAssassin-xxx.tar.gz $ cd Mail-SpamAssassin-xxx $ perl Makefile.PL [...] The following RBLs have certain usage restrictions. Would you like to enable them? If unsure, say Y. (Y/N): [Y] [... a list of RBLs, preferably with links to their policy pages ...] so that at installation time, SA users are at least aware of the issues. For package maintainers, you'd want a noninteractive way to build SpamAssassin and then a post-install configuration script that asks the RBL question. (This is easily done on Debian; not so easy with RPM-based systems that don't allow interaction during installation.) Regards, David.
solicitations via netsuite.com
greetings how are you folks on this list dealing with unwanted solicitations from companies that spam via netsuite.com ? -rh
Re: DNS list inclusion policy (was Re: DNSWL will be disabled by default as of tomorrow)
On Tue, 2011-12-13 at 15:33 -0500, David F. Skoll wrote: I think something like this would be good: $ tar xvfz Mail-SpamAssassin-xxx.tar.gz $ cd Mail-SpamAssassin-xxx $ perl Makefile.PL [...] The following RBLs have certain usage restrictions. Would you like to enable them? If unsure, say Y. (Y/N): [Y] [... a list of RBLs, preferably with links to their policy pages ...] so that at installation time, SA users are at least aware of the issues. The only problem with this that I can see is, unfortunately, a biggish one: it won't work for the package installs that I and, at a guess, a lot of small installations use. I run the Fedora distro, and so the version of SA I'm running is that installed and periodically updated by yum. I like the idea of the tool though, particularly if it can link to descriptions of each RBL and, hopefully, to the masscheck results for it. From my POV a post-install tool, and especially one that can be re-run periodically, would be a lot more useful. Martin
Draft of my submission to the PMC for DNSBL Inclusion Criteria
I used several years worth of notes to come up with the information below. It needs more polish and it is my very first rip and shred. I am also certain that I've missed some great points others have made. So I am posting this to solicit feedback on this draft so I can then submit it to the PMC for approval. Just to be very clear, this is NOT a draft approved by the PMC but it is based heavily on consensus and many many threads from committee members. regards, KAM Apache SpamAssassin PMC Criteria for DNSBL Inclusion [DRAFT] GOAL: To produce an objective criteria for inclusion of DNS-blocklists (DNSBLs) including free and semi-commercial services that promotes the ability to include more tests in a manner that is fair to the community and the service provider. All services, whether free, commercial or semi-commercial services must meet this criteria for default inclusion in SpamAssassin's rules: - May not block queries by returning purposefully wrong answers from over-quota or abusive IPs. - The usage policy and any limits or restrictions must be documented and publicly visible with clearly defined terms. (Terms such as "heavy load" are not acceptable). - Should be "free for most" installations. - May use limits such as DNS query limits per day but may not limit on the number of users or other arbitrary caps that can't be correlated to a direct increase in expenses. - Should use a query response and rule that indicates a system is over limit. Such response must adds substantial no scoring difference and link only to a generic DNSBL Block page such as http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block. - Must have an existing or planned infrastructure capable of the anticipated query load. - Must give the project permission to include the rules by default. - Daily query limits that have limited to 100k queries per day have been considered acceptable. - Free access by request to rsync feeds for RBLDNSD is considered unlimited access. - The addition of new blocklists should be done only in conjunction with a new major release and should be version encapsulated so that existing admins can decide to use them if possible in older installations. - A formal vote in bugzilla is required before a network-based test is added to a sandbox. - Blocklists must meet acceptable mass-check scoring critera to be considered for default inclusion. Testing is mandatory and the higher the S/O, the better. - May not have significant reliability issues. - Must have clear rules and procedures that are followed uniformly for listings and de-listings. - May not accept funds to remove/list/delist/expedite or otherwise non-objectively handle their lists. - Should use lastexternal or lasttrusted testing unless there is an overwhelming benefit otherwise. - May require signing up for an account / mailing list / etc. for the purpose of notifying Admins of changes and problems. Semi-commercialized services aka "Free for Some" must meet this additional criteria for default inclusion in SpamAssassin's rules: - Must be free for any kind of person or organization to use, commercial, government, or home user. - May impose licensing limitations on use as a "anti-spam reseller" or directly reselling spam filtering services. - Must not attempt to retroactively bill users that have exceeded any free limits. - May not be a trial or limited time offer. Services that are completely commercial are not eligible to be enabled by default. -- Kevin A. McGrail President Peregrine Computer Consultants Corporation 3927 Old Lee Highway, Suite 102-C Fairfax, VA 22030-2422 http://www.pccc.com/ 703-359-9700 x50 / 800-823-8402 (Toll-Free) 703-359-8451 (fax) kmcgr...@pccc.com
Re: solicitations via netsuite.com
On 12/13/11 3:35 PM, R - elists wrote: greetings how are you folks on this list dealing with unwanted solicitations from companies that spam via netsuite.com ? -rh don't see them... I guess SA marks them spam :-) but, I suppose it's no different than sugarcrm or salesforce (I dropped salesforce over two email support issues. #1 being they seems to allow big clients to spam, #2 was that so many people blocked salesforce that 50% of our emails to our clients were being sent to junk email folders) oh, I could solve issue #2 by setting up relaying (our email from Salesforce would be relayed through our servers, not theirs), but it would raise our cost by 65%. so, who really cares about netsuite.com them selves.. they are just a CRM. send complaints to abuse@ and see what happens. -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 *| *SECNAP Network Security Corporation * Best Mobile Solutions Product of 2011 * Best Intrusion Prevention Product * Hot Company Finalist 2011 * Best Email Security Product * Certified SNORT Integrator __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.spammertrap.com/ __
