Re: SA running different tests when run manually ?
amavisd uses the spamassassin libraries internally, it does not use the spamassassin command, nor spamd. If you update parts of the config, you'll need to reload/restart amavisd. Aah... I must have missed that bit of the fabulous manual... ;-(
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Thu, 11 Jun 2015 01:00:45 +0200 Reindl Harald wrote: > cache-min-ttl: 600 Even a 10-minute cache time buys you very little. My original analysis assumed a 15-minute TTL. Regards, Dianne.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Am 10.06.2015 um 16:18 schrieb Dianne Skoll: On Wed, 10 Jun 2015 13:56:49 + David Jones wrote: [One should run a caching DNS server on a mail server.] We are giving you solid advice based on real experiences where we ran into problems and worked around them. Just try to enable RBLs and see how it works for you. I'm not disputing that running a caching DNS server is a good idea, but you may be quite surprised at the low cache hit rate for IP-based DNSBLs. Spamhaus, for example, has a TTL of 1 minute on its A records. (Check out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.) yes, to exceed the volume quicker and only if your resolver has a bad configuration and that's even one reason more to use a local cache msg-cache-size: 96m neg-cache-size: 96m rrset-cache-size: 192m cache-min-ttl: 600 cache-max-ttl: 10800 signature.asc Description: OpenPGP digital signature
Re: Must-Have Plugins?
On Wed, Jun 10, 2015 at 9:56 AM, David Jones wrote: >>> given that install unbound as local resolver takes 2 minutes it's even not >>> worth to argue on that topic and a spamfilter without RBL's and URIBL's is >>> just nonsense > >>I have installed a caching DNS server before (albeit probably about 15 >>years ago). But it just shouldn't be necessary. > > It can be necessary if you have enough mail volume. That's not what I'm saying. It should not be necessary to run a full-blown DNS server for SA to do it's queries. It should be possible to call a library and create a DNS context that has all of it's own parameters and then use that in an isolated way. Then other services on the system are completely unaffected. Don't tell me someone has never tweaked some parameter in your supposedly caching-only nameserver and inadvertantly broken something or wished they could tweak something and can't because of the dependencies. And it's very possible that the queries might be for different names using custom query parameters in an async way and so on in which case the system resolver API might not be ideal. I'm not pooh-poohing your advice. I'm just saying the DNS bits should be librarified so that these things don't even need extra thought. This stuff might be what you do all the time but I don't. I do this once every few years. This is the sort of thing that makes people switch to "cloud services".
Re: spamass-milter and email addresses starting with --
On 6/10/2015 4:42 PM, Michael Grant wrote:
I'm using 0.3.2 which seems to be the latest version available for
Debian. It does appear that 0.4.0 hasn't hit Debian yet.
If I am reading the code correctly, the only change in 0.3.2 to 0.4.0 in
the relevant section is to use strdup, i.e.
// XXX arbitrary 100-argument max
int argc = 0;
char** argv = (char**) malloc(100*sizeof(char*));
- argv[argc++] = SPAMC;
+ argv[argc++] = strdup(SPAMC);
if (flag_sniffuser)
{
-argv[argc++] = "-u";
+argv[argc++] = strdup("-u");
if ( expandedrcpt.size() != 1 )
{
// More (or less?) than one recipient, so we pass the default
@@ -46,7 +46,7 @@
}
if (spamdhost)
{
-argv[argc++] = "-d";
+argv[argc++] = strdup("-d");
argv[argc++] = spamdhost;
}
if (spamc_argc)
Regards,
KAM
Re: spamass-milter and email addresses starting with --
On 6/10/2015 4:34 PM, David B Funk wrote: What version of spamass-milter are you using? Older versions of spamass-milter used a "system" call to invoke "spamc" and feed it messages, thus had a glaring security vulnerability. That was fixed a while ago, you need to update your spamass-milter. Are you sure a newer version fixes the issue? I checked 0.1.3a (has to be 13 years old) and 0.4.0 (the latest) and both appear to use the same execvp call for spamc and the function for full_user doesn't appear to have anything sanitizing this case. According to http://savannah./gnu/./org//projects/spamass-milt/, 0.4.0 is the latest. regards, KAM
Re: spamass-milter and email addresses starting with --
I'm using 0.3.2 which seems to be the latest version available for Debian.
It does appear that 0.4.0 hasn't hit Debian yet.
On Wed, Jun 10, 2015 at 10:34 PM, David B Funk wrote:
> On Wed, 10 Jun 2015, Michael Grant wrote:
>
> I'm running Debian, sendmail, spamass-milter, spamc and spamd.
>>
>> I saw this in my log:
>> Jun 9 20:30:29 debian sm-mta[15942]: t5A0ULAA015942: to=<--
>> u...@example.com>
>> then I saw this:
>>
>> Jun 9 20:30:29 strange spamc[15947]: invalid usage
>> Jun 9 20:30:29 strange spamass-milter[1770]: Thrown error: poll says my
>> write pipe is busted
>> That seems pretty scary that someone can send to a user which begins with
>> -- and fake out spamc that it's a command
>> line option.
>>
>
> What version of spamass-milter are you using?
> Older versions of spamass-milter used a "system" call to invoke "spamc"
> and feed it messages, thus had a glaring security vulnerability.
>
> That was fixed a while ago, you need to update your spamass-milter.
>
>
> --
> Dave Funk University of Iowa
> College of Engineering
> 319/335-5751 FAX: 319/384-0549 1256 Seamans Center
> Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
> #include
> Better is not better, 'standard' is better. B{
Re: spamass-milter and email addresses starting with --
On Wed, 10 Jun 2015, Michael Grant wrote:
I'm running Debian, sendmail, spamass-milter, spamc and spamd.
I saw this in my log:
Jun 9 20:30:29 debian sm-mta[15942]: t5A0ULAA015942: to=<--u...@example.com>
then I saw this:
Jun 9 20:30:29 strange spamc[15947]: invalid usage
Jun 9 20:30:29 strange spamass-milter[1770]: Thrown error: poll says my write
pipe is busted
That seems pretty scary that someone can send to a user which begins with --
and fake out spamc that it's a command
line option.
What version of spamass-milter are you using?
Older versions of spamass-milter used a "system" call to invoke "spamc"
and feed it messages, thus had a glaring security vulnerability.
That was fixed a while ago, you need to update your spamass-milter.
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
spamass-milter and email addresses starting with --
I'm running Debian, sendmail, spamass-milter, spamc and spamd. I saw this in my log: Jun 9 20:30:29 debian sm-mta[15942]: t5A0ULAA015942: to=<--u...@example.com > then I saw this: Jun 9 20:30:29 strange spamc[15947]: invalid usage Jun 9 20:30:29 strange spamass-milter[1770]: Thrown error: poll says my write pipe is busted That seems pretty scary that someone can send to a user which begins with -- and fake out spamc that it's a command line option.
Re: DCC whitelisting
On Wed, 10 Jun 2015, Shane Williams wrote: Two examples that I know are legitimate senders, but get caught by DCC (and pyzor in some cases) and other rules that push them over the threshold are the SourceForge.net Project of the Month list and various Netflix emails to customers (New Arrivals or "we just added a show you might like"). In both those cases, the user part of the env_from changes, and as I understand it, the DCC Whitelist doesn't allow wildcards, so I can't have an entry that matches the server part. Maybe I could be using the "substitute List-ID:" syntax, but neither of those has List-ID as a specific header. Can you reliably identify those at the MTA level and tell the SA glue to skip them entirely? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Maxim XI: Everything is air-droppable at least once. ---
DCC whitelisting
I know it's not a SpamAssassin specific question, but I suspect there are more sophisticated DCC users collected here than anywhere else. Bottom line, can someone explain in not-overly jargony language what the actual dcc whitelist entries should look like (ideally with examples), as well as what the log output from dcc means? I've read the documentation, I've read the package-provided whitelist files, I've got the "top-level" whitelist configuration working, but when I put entries into whitecommon or whiteclnt they don't seem to do what I would expect them to do. Two examples that I know are legitimate senders, but get caught by DCC (and pyzor in some cases) and other rules that push them over the threshold are the SourceForge.net Project of the Month list and various Netflix emails to customers (New Arrivals or "we just added a show you might like"). In both those cases, the user part of the env_from changes, and as I understand it, the DCC Whitelist doesn't allow wildcards, so I can't have an entry that matches the server part. Maybe I could be using the "substitute List-ID:" syntax, but neither of those has List-ID as a specific header. I know I could create SA rules that would "counteract" the DCC score, but that seems kludgy (and doesn't help if there are meta rules that trigger off of DCC). -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT CompSci =--+--- All syllogisms contain three lines | sha...@shanew.net Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: Dealing with spammers making creative use of Google links ?
On Wed, 10 Jun 2015, John Hardin wrote: On Wed, 10 Jun 2015, Ben wrote: Hi, Does SA have pre-existing tests that look for this sort of thing (so I can just boost the score a little bit), or does anyone have any ideas how I might counter such spam ? Look at this fine specimen as an example : http://pastebin.com/raw.php?i=XjV94PMW See how they cunningly use "http://www.google.com/url?q="; in order to obfuscate their URLs, as a DBL check countermeasure I suspect. __GOOG_REDIR and GOOG_REDIR_SHORT in my sandbox are being published. However, if the message is long then it probably isn't hitting GOOG_REDIR_SHORT and another meta using __GOOG_REDIR would be needed. ...hit SEND too quickly... It doesn't look like anything else in that message is really usable for a base meta. __GOOG_REDIR && RDNS_NONE might be useful locally but it wouldn't perform well in masschecks, there's zero overlap with current corpora so it wouldn't get promoted. I'll drop it in, though, in case it starts showing up in the masscheck corpora. The URL parser should pick out the target URL by itself as well, so it should be checked against URIBLs. Unfortunately dsfv4.pillsforyou.ru isn't listed in URIBL. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Maxim XI: Everything is air-droppable at least once. ---
Re: Dealing with spammers making creative use of Google links ?
On Wed, 10 Jun 2015, Ben wrote: Hi, Does SA have pre-existing tests that look for this sort of thing (so I can just boost the score a little bit), or does anyone have any ideas how I might counter such spam ? Look at this fine specimen as an example : http://pastebin.com/raw.php?i=XjV94PMW See how they cunningly use "http://www.google.com/url?q="; in order to obfuscate their URLs, as a DBL check countermeasure I suspect. __GOOG_REDIR and GOOG_REDIR_SHORT in my sandbox are being published. However, if the message is long then it probably isn't hitting GOOG_REDIR_SHORT and another meta using __GOOG_REDIR would be needed. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- If "healthcare is a Right" means that the government is obligated to provide the people with hospitals, physicians, treatments and medications at low or no cost, then the right to free speech means the government is obligated to provide the people with printing presses and public address systems, the right to freedom of religion means the government is obligated to build churches for the people, and the right to keep and bear arms means the government is obligated to provide the people with guns, all at low or no cost. ---
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Wed, 10 Jun 2015, David Jones wrote:
[One should run a caching DNS server on a mail server.]
My point was that running a local caching server is the only way one
can know exactly how the lookups are happening. If you point to a
DNS server that you don't manage, it could be forwarding to an ISP's
DNS caches which will aggregate your queries in with others and could
cause unexpected results for those RBLs that limit queries.
One other technical benefit to running a local caching server is that if
SA is configured to talk to it va the localhost (loopback) interface there
are MTU advantages.
Most loopback interfaces have a MTU of 16K (or bigger) and will handle large
UDP packets without fragementation. In general DNS transactions are fastest
via UDP if you don't have fragementation issues.
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
Re: SA running different tests when run manually ?
On 10-06-15 17:25, Ben wrote:
> I have a curious conundrum.
>
> A piece of spam received shows the following in the header when
> processed via amavis and spamd :
> DATE_IN_PAST_03_06,
> HTML_MESSAGE,
> RCVD_IN_BL_SPAMCOP_NET,
> RCVD_IN_MSPIKE_H4,
> RCVD_IN_MSPIKE_WL,
> RDNS_DYNAMIC,
> SHORTENED_URL_SRC
>
> But when the exact piece of Spam is fed directly into spamassassin (i.e
> spamassassin < spam.txt), I get a different series of tests :
>
> HTML_MESSAGE,
> MY_CUSTOM2_SPAM,
> RCVD_IN_BL_SPAMCOP_NET,
> RCVD_IN_BRBL_LASTEXT,
> RCVD_IN_MSPIKE_H4,
> RCVD_IN_MSPIKE_WL,
> RDNS_DYNAMIC,
> SHORTENED_URL_SRC
>
>
> The difference being that my custom spam test ("MY_CUSTOM2_SPAM") is not
> run via amavis, but is run directly.
amavisd uses the spamassassin libraries internally, it does not use the
spamassassin command, nor spamd. If you update parts of the config,
you'll need to reload/restart amavisd.
> My spam test is configured in /etc/spamassassin/local.cf, and I have run
> "su - debian-spamd -c "spamassassin --lint" && service spamassassin
> reload". I have even tried sa-compile just incase my local.cf changes
> needed to be compiled before they are recognised.
>
> spamd does share the same config files as the spamassassin binary, right ?
>
> This is on Ubuntu 14.04LTS, SpamAssassin version 3.4.0 running on Perl
> version 5.18.2, amavisd 1:2.7.1-2ubuntu3.
>
Re: Must-Have Plugins?
On Wed, 10 Jun 2015, Bill Cole wrote: > (2) Check the HELO the other guy sends and reject if it's not a FQDN > (i.e. it's not got any periods at all). or if it's your FQDN, or your IP - they should use their FQDN, not yours. And if you don't/can't use a greeting pause, these are useful in catching many of the bots that fast-talk. Absolutely. I see a lot of instances where the first couple of tries from a given IP are blocked by greet-pause, and then after a bit there are several more from the same IP blocked by invalid HELO. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...much of our country's counterterrorism security spending is not designed to protect us from the terrorists, but instead to protect our public officials from criticism when another attack occurs. -- Bruce Schneier ---
Re: Must-Have Plugins?
On Wed, 10 Jun 2015, Kevin A. McGrail wrote: On 6/10/2015 12:45 AM, Michael B Allen wrote: But I just can't bring myself to install a caching DNS server and run everything through localhost. This is why software should be librarified. I strongly advise you to install a caching DNS server and using a few RBLs. Just a minor nit: It's not the "caching" part that's important here, it's the "not forwarding" part. If you set up a caching DNS server that just forwards to your ISP's DNS servers, you haven't addressed the "BL blocked due to query volume" problem. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...much of our country's counterterrorism security spending is not designed to protect us from the terrorists, but instead to protect our public officials from criticism when another attack occurs. -- Bruce Schneier ---
SA running different tests when run manually ?
I have a curious conundrum.
A piece of spam received shows the following in the header when
processed via amavis and spamd :
DATE_IN_PAST_03_06,
HTML_MESSAGE,
RCVD_IN_BL_SPAMCOP_NET,
RCVD_IN_MSPIKE_H4,
RCVD_IN_MSPIKE_WL,
RDNS_DYNAMIC,
SHORTENED_URL_SRC
But when the exact piece of Spam is fed directly into spamassassin (i.e
spamassassin < spam.txt), I get a different series of tests :
HTML_MESSAGE,
MY_CUSTOM2_SPAM,
RCVD_IN_BL_SPAMCOP_NET,
RCVD_IN_BRBL_LASTEXT,
RCVD_IN_MSPIKE_H4,
RCVD_IN_MSPIKE_WL,
RDNS_DYNAMIC,
SHORTENED_URL_SRC
The difference being that my custom spam test ("MY_CUSTOM2_SPAM") is not
run via amavis, but is run directly.
My spam test is configured in /etc/spamassassin/local.cf, and I have run
"su - debian-spamd -c "spamassassin --lint" && service spamassassin
reload". I have even tried sa-compile just incase my local.cf changes
needed to be compiled before they are recognised.
spamd does share the same config files as the spamassassin binary, right ?
This is on Ubuntu 14.04LTS, SpamAssassin version 3.4.0 running on Perl
version 5.18.2, amavisd 1:2.7.1-2ubuntu3.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Wed, 10 Jun 2015 14:56:40 + David Jones wrote: > My point was that running a local caching server is the only way one > can know exactly how the lookups are happening. Ah, true. I missed that point I guess. Regards, Dianne.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
>[One should run a caching DNS server on a mail server.] >> We are giving you solid advice based on real experiences where we >> ran into problems and worked around them. Just try to enable RBLs >> and see how it works for you. >I'm not disputing that running a caching DNS server is a good idea, but >you may be quite surprised at the low cache hit rate for IP-based DNSBLs. >Spamhaus, for example, has a TTL of 1 minute on its A records. (Check >out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.) >Quite a number of years ago, I ran an analysis of the mail logs on a >very busy server and found an abysmally low cache hit rate (about 30%) >and that was in the day when Spamhaus had a 15-minute TTL. My point was that running a local caching server is the only way one can know exactly how the lookups are happening. If you point to a DNS server that you don't manage, it could be forwarding to an ISP's DNS caches which will aggregate your queries in with others and could cause unexpected results for those RBLs that limit queries. I have 8 mail filters that run a local caching DNS server which forward to a pair of DNS caches running rbldnsd for a local copy of a number of RBL zones including my own private RBL. This configuration has to provide some caching benefits when I get blasted by mass marketing campaigns. Postfix should keep my local cache populated so when SA asks for the same DNS information it would be a few milliseconds response. I should take some time to do some real analysis as you have done. Thanks for the info and link.
Re: Must-Have Plugins?
Hi, Not everyone is running a dedicated mail server. My server is an everything-server running on a hosted VPS that only has a few "users" that get significant amounts of email. I'm not sure I want another daemon that can break or take up clock cycles and memory on a system processing 10 spams / hour (of which the DNSBL service might catch 2?). At least not yet, but I suppose I could change my mind. At the moment not that many spams are getting through. Mike You asked for help, we provided it. It's fine if you ignore the advice, but it's good advice if you want to take your mail filtering to the next level in the future. And it's not just for mail filtering. Unless you have the smallest of systems, it's good general practice to implement a caching name server for authentication, the email system itself, and general queries from other services as well. Regards, Alex
Re: Must-Have Plugins?
On 9 Jun 2015, at 14:39, Matus UHLAR - fantomas wrote: On 09.06.15 11:29, John Hardin wrote: Two things that I have found very useful at the MTA level are: (1) Delay sending your SMTP banner a second or two and reject any sender that starts sending information before that. This is a built-in option in Sendmail, google "greet_pause". even 15... Based on the implementation in Sendmail on a variety of systems with up to a few million sessions/day, Postfix's 'postscreen' implementation on smaller (mostly *much* smaller) sites and CommunigatePro's implementation on a couple of middling sites, the value of adding greet_pause time starts dropping fast around 3 seconds and there's no point going past 5. You can actually start seeing occasional collateral damage around 10 seconds with some high-volume legitimate senders timing out their 'first try' runs aggressively. 15 is another breakpoint, apparently because there is widespread lore in high-speed sending that says it is not worth waiting any longer for a greeting banner. For high-volume sites it is also important to understand that SMTP session lives from SYN-ACK to FIN without a greeting pause are mostly under 10 seconds; adding 3 seconds to that means you must support MANY more concurrent SMTP connections. FWIW, the best implementation of fast-talk detection is Postfix's postscreen. It is optimized to put this and other low-cost exclusion tricks outside of a relatively heavy smtpd and was designed with an eye on the many years of demonstration by Sendmail and others of how to minimize the impact of adding delays. By not delaying clients that have recently passed a delay, it assures that most of your high-volume legitimate senders don't contribute to performance issues. (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. it's not got any periods at all). or if it's your FQDN, or your IP - they should use their FQDN, not yours. And if you don't/can't use a greeting pause, these are useful in catching many of the bots that fast-talk. Also useful are checks for improper use of IP HELOs (e.g. unbracketed IPs, not valid "IP literals") and checking for *.local and *.localdomain HELO args. A handful of innocent morons run MTAs that use such names, but they tend not to survive long doing so.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On 6/10/2015 10:18 AM, Dianne Skoll wrote: I'm not disputing that running a caching DNS server is a good idea, but you may be quite surprised at the low cache hit rate for IP-based DNSBLs. IMO, the primary goal of a caching-only nameserver is in fact, not the caching, but rather the unique source IP so as to avoid running into DNS limits placed on RBL queries from some BL providers that you can run afoul of when sharing a DNS server. Caching is really just icing on the cake coupled with the simplest way to get a local DNS server up and running, no? Regards, KAM
DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Wed, 10 Jun 2015 13:56:49 + David Jones wrote: [One should run a caching DNS server on a mail server.] > We are giving you solid advice based on real experiences where we > ran into problems and worked around them. Just try to enable RBLs > and see how it works for you. I'm not disputing that running a caching DNS server is a good idea, but you may be quite surprised at the low cache hit rate for IP-based DNSBLs. Spamhaus, for example, has a TTL of 1 minute on its A records. (Check out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.) Quite a number of years ago, I ran an analysis of the mail logs on a very busy server and found an abysmally low cache hit rate (about 30%) and that was in the day when Spamhaus had a 15-minute TTL. Anyway, run through the exercise yourself; it's eye-opening. My original post was here (back when I used to be David, so don't let the signature confuse you...) http://spamassassin.1065346.n5.nabble.com/Fwd-Asrg-draft-levine-iprangepub-01-tp28778p28802.html Regards, Dianne.
Re: Must-Have Plugins?
>> given that install unbound as local resolver takes 2 minutes it's even not >> worth to argue on that topic and a spamfilter without RBL's and URIBL's is >> just nonsense >I have installed a caching DNS server before (albeit probably about 15 >years ago). But it just shouldn't be necessary. It can be necessary if you have enough mail volume. >By "librarified" I mean the DNS "server" is just a code context that >can be constructed with it's own config precisely and only as needed >by the software that will be querying it (possibly temporarily if it's >just client-only activity like a barrage of DNS queries fired in >reaction to an email that fails other spam tests). It should not be >necessary to change the resolver configuration or behavior of the >entire system and everything running on it if only one component in >the system needs this special feature (in this case a query limit and >private cache). That is just bad programming philosophy and it the >source of a lot of bad behavior in software (and DNS is a very good >example of this actually). You obviously don't understand how DNS works in relation to RBLs. We are giving you solid advice based on real experiences where we ran into problems and worked around them. Just try to enable RBLs and see how it works for you. >Not everyone is running a dedicated mail server. My server is an >everything-server running on a hosted VPS that only has a few "users" >that get significant amounts of email. I'm not sure I want another >daemon that can break or take up clock cycles and memory on a system >processing 10 spams / hour (of which the DNSBL service might catch >2?). At least not yet, but I suppose I could change my mind. At the >moment not that many spams are getting through. >Mike You asked for help, we provided it. It's fine if you ignore the advice, but it's good advice if you want to take your mail filtering to the next level in the future.
Re: Must-Have Plugins?
Am 10.06.2015 um 15:49 schrieb Michael B Allen: By "librarified" I mean the DNS "server" is just a code context that can be constructed with it's own config precisely and only as needed by the software that will be querying it (possibly temporarily if it's just client-only activity like a barrage of DNS queries fired in reaction to an email that fails other spam tests). It should not be necessary to change the resolver configuration or behavior of the entire system and everything running on it if only one component in the system needs this special feature (in this case a query limit and private cache). That is just bad programming philosophy and it the source of a lot of bad behavior in software (and DNS is a very good example of this actually) 1: SA has a option to define the nameserver without touch resolv.conf 2: why would somebody not re-use already cached data for any software signature.asc Description: OpenPGP digital signature
Re: Must-Have Plugins?
On Wed, Jun 10, 2015 at 7:25 AM, Reindl Harald wrote: > > > Am 10.06.2015 um 13:21 schrieb Kevin A. McGrail: >> >> On 6/10/2015 12:45 AM, Michael B Allen wrote: >>> >>> But I just can't >>> bring myself to install a caching DNS server and run everything >>> through localhost. This is why software should be librarified. >> >> I strongly advise you to install a caching DNS server and using a few RBLs > > > +1 > > i can't understand "I just can't bring myself to install a caching DNS > server and run everything through localhost" because that is the way to go > to avoid exceed RBL limits and bad resolvers > > > "This is why software should be librarified" is nonsense in that context - > the library also needs to ask a dns server at the end of the day and the > server needs to be 100% trustable when it comes to email > > given that install unbound as local resolver takes 2 minutes it's even not > worth to argue on that topic and a spamfilter without RBL's and URIBL's is > just nonsense I have installed a caching DNS server before (albeit probably about 15 years ago). But it just shouldn't be necessary. By "librarified" I mean the DNS "server" is just a code context that can be constructed with it's own config precisely and only as needed by the software that will be querying it (possibly temporarily if it's just client-only activity like a barrage of DNS queries fired in reaction to an email that fails other spam tests). It should not be necessary to change the resolver configuration or behavior of the entire system and everything running on it if only one component in the system needs this special feature (in this case a query limit and private cache). That is just bad programming philosophy and it the source of a lot of bad behavior in software (and DNS is a very good example of this actually). Not everyone is running a dedicated mail server. My server is an everything-server running on a hosted VPS that only has a few "users" that get significant amounts of email. I'm not sure I want another daemon that can break or take up clock cycles and memory on a system processing 10 spams / hour (of which the DNSBL service might catch 2?). At least not yet, but I suppose I could change my mind. At the moment not that many spams are getting through. Mike
Re: Must-Have Plugins?
>> - Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the >>majority of junk before it reaches SA. Just make sure you are below their >>free threshold limit. One important way to do this is >"One important way to do this" in terms of the Spamhaus threshold limit >is to not be such a tightwad and poney up for the Spamhaus commercial >service. ;-) >They do a cheaper version than the RSync feed, you can just query their >servers directly. >Spamhaus do a fantastic job. They deserve charitable donations from >generous mail sysadmins !!! We filter a lot of mailboxes and pay spamhaus several thousands of dollars each year. The invaluement.com RBL is very effective and only costs a few hundred dollars a year. >> - Enable greylisting. >Ewww... >I hate people who operate graylisting. Its a lazy "tarnish everyone >with the same brush" approach to anti-spam. >In this day and age, you don't need it. Decent network checks, properly >configured Spamassassin and you should be able to achieve a very >respectable spam catching rate. I respectfully disagree. I too hated greylisting for years until recently when I found a way to slowly ease it in for my users who didn't even detect it. There is no other way to block brand new spam campaigns from compromised accounts. These mail servers normally have a good reputation so they are not on any RBLs yet. Greylisting puts a "speed bump" in place so the RBLs have time to catch up. Spammers pay "sweat shops" to devise new spam that will get through the major filters like SA and commercial products as zero-hour spam. Bayes is often ineffective against these new campaigns. When a new spam campaign like this hits the Internet, the world takes a little while to detect and react to it so you have to put some kind of buffer like greylisting in place. I whitelist a lot of major ISPs and known good senders to bypass greylisting so this only impacts mostly small mail servers that don't have any compromised account detection. My company's support team used to get several calls a week calls about blacklisting senders that turned out to be compromised accounts but now we might get one or two every 3 months now. By the time the blacklist entry was added, RBLs had already been blocking them so I just remove the new entry to keep my lists clean. If you have a large mail filtering environment with a lot of very old email accounts that have become bought and sold on spammer lists, this is a must. I guess if you are only filtering for a few hundred accounts, you can do a lot of things differently and be fine.
Re: Dealing with spammers making creative use of Google links ?
On 6/10/2015 7:46 AM, Ben wrote: Well I had no idea about the existence of KAM.cf, so I bask in your infinite glory. ;-) Why isn't KAM.cf more heavily publised on spamassassin.org as something worth thinking about ? You do have a page about third-party rules IIRC, but don't remmeber seeing KAM.cf on it. Well, the rules have been published long enough that the link is often found under peregrinehw.com (we changed names in 2000) and it pre-dates my committer status with the project. Plus my firm provides commercial email and web security services so don't like to bring it up or add links. But thank you for the kind words! Regards, KAM
Re: Must-Have Plugins?
>> Some of the best and easiest things you can enable to block spam are >> outside of SpamAssassin at your MTA (sendmail, postfix, etc.). >> - Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the >> majority of junk before it reaches SA. Just make sure you are below their >> free threshold limit. One important way to do this is to make sure your >> SA server isn't pointed to an Internet caching DNS server that would join >> your queries with others. Install a local caching DNS server that does not >> forward to another caching DNS server and change /etc/resolv.conf to use >> 127.0.0.1. >Well that sounds like a must-have feature to me. But I just can't >bring myself to install a caching DNS server and run everything >through localhost. This is why software should be librarified. What OS are you running? It's normally a very simple install to get a caching DNS server running locally since the default configurations usually come ready to do exactly what you need in this case. Google "caching dns server howto" plus your OS and you will see it's pretty easy. You can try using RBLs with your existing DNS server configuration. If it's a dedicated DNS server for your network, then you have a good chance of staying below the free thresholds for a low volume server. If it's a major ISP's DNS server then the odds will be against you. http://www.spamhaus.org/faq/section/DNSBL%20Usage#366 Try "dig 138.178.203.192.zen.spamhaus.org" or nslookup to see if you get back a response of 127.0.0.4. If so, you should be good. >> - Enable DNS checks: >> Make sure the sending mail server's SMTP HELO is a valid domain. >> Make sure the sender address (MAIL FROM) is a valid domain. >> Make sure the sending mail server has a PTR record. Some can go farther >> with >> this one and require the PTR match the SMTP HELO for FCrDNS but there are >> many legit mail servers out there that don't have this setup properly so I >> can >> only check to make sure a PTR record exists. Later in SA I add points for >> rule >> RDNS_NONE that penalizes for incorrect FCrDNS. >Is this done with postfix rules or SA rules? Where can I learn more >about this? Doesn't SA already do this stuff? This should be done in your MTA if possible before it's handed off to SA. Some of this information is exposed to SA in headers but some isn't. High volume servers need this logic at the MTA to keep processing times low. Only a small percentage of mail makes it to SA in my environment and most of that is going to be clean. In a low volume environment, you can send most of the mail through SA and still keep the processing times down low. My MailScanner batches average about 5 to 10 messages and complete in 4 to 5 seconds normally with ClamAV and Eset Nod32 AV scanners.
Re: Dealing with spammers making creative use of Google links ?
On 10/06/2015 12:32, Kevin A. McGrail wrote: I'm hitting over spam threshold on the message and have a simple redir for Google match in KAM.cf. 1.0 KAM_GOOGLE_STRING URI: Use of Google redir appearing in spam July 2006 0.0 HTML_MESSAGE BODY: HTML included in message 1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 1.3 RDNS_NONE Delivered to internal network by a host with no rDNS 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 2.0 TO_NO_BRKTS_NORDNS_HTML To: misformatted and no rDNS and HTML only Regards, KAM Kevin, Well I had no idea about the existence of KAM.cf, so I bask in your infinite glory. ;-) Why isn't KAM.cf more heavily publised on spamassassin.org as something worth thinking about ? You do have a page about third-party rules IIRC, but don't remmeber seeing KAM.cf on it.
Re: Dealing with spammers making creative use of Google links ?
On 6/10/2015 5:27 AM, Ben wrote: Hi, Does SA have pre-existing tests that look for this sort of thing (so I can just boost the score a little bit), or does anyone have any ideas how I might counter such spam ? Look at this fine specimen as an example : http://pastebin.com/raw.php?i=XjV94PMW See how they cunningly use "http://www.google.com/url?q="; in order to obfuscate their URLs, as a DBL check countermeasure I suspect. I'm hitting over spam threshold on the message and have a simple redir for Google match in KAM.cf. 1.0 KAM_GOOGLE_STRING URI: Use of Google redir appearing in spam July 2006 0.0 HTML_MESSAGE BODY: HTML included in message 1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 1.3 RDNS_NONE Delivered to internal network by a host with no rDNS 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 2.0 TO_NO_BRKTS_NORDNS_HTML To: misformatted and no rDNS and HTML only Regards, KAM
Re: Must-Have Plugins?
Am 10.06.2015 um 13:21 schrieb Kevin A. McGrail: On 6/10/2015 12:45 AM, Michael B Allen wrote: But I just can't bring myself to install a caching DNS server and run everything through localhost. This is why software should be librarified. I strongly advise you to install a caching DNS server and using a few RBLs +1 i can't understand "I just can't bring myself to install a caching DNS server and run everything through localhost" because that is the way to go to avoid exceed RBL limits and bad resolvers "This is why software should be librarified" is nonsense in that context - the library also needs to ask a dns server at the end of the day and the server needs to be 100% trustable when it comes to email given that install unbound as local resolver takes 2 minutes it's even not worth to argue on that topic and a spamfilter without RBL's and URIBL's is just nonsense signature.asc Description: OpenPGP digital signature
Re: Must-Have Plugins?
On 6/10/2015 12:45 AM, Michael B Allen wrote: But I just can't bring myself to install a caching DNS server and run everything through localhost. This is why software should be librarified. I strongly advise you to install a caching DNS server and using a few RBLs. regards, KAM
Re: Must-Have Plugins?
Am 10.06.2015 um 13:17 schrieb Kevin A. McGrail: On 6/10/2015 2:32 AM, Matus UHLAR - fantomas wrote: I'm not sure whether or not I have enabled requiring valid rDNS... given how many legitimate mailservers out there don't have proper rDNS, how many? I'm happy to block them for years... From what I've see, the effectivness and false positives depends mostly on the countries sending email to your users. Off the cuff, I'd say it has it's roots in the mid 90's when AOL was a 1000lb gorilla and pushed the rptr for mail servers. I think much of Europe and NA complied. However, I see a TON of legit mail, for example, coming out of the Pacific Rim countries with no rptr's well, combine strict PTR rules with DNSWL and SPF, the one which have *something* sane are not hurted and the rest need to learn some basics signature.asc Description: OpenPGP digital signature
Re: Must-Have Plugins?
On 6/10/2015 2:32 AM, Matus UHLAR - fantomas wrote: I'm not sure whether or not I have enabled requiring valid rDNS... given how many legitimate mailservers out there don't have proper rDNS, how many? I'm happy to block them for years... From what I've see, the effectivness and false positives depends mostly on the countries sending email to your users. Off the cuff, I'd say it has it's roots in the mid 90's when AOL was a 1000lb gorilla and pushed the rptr for mail servers. I think much of Europe and NA complied. However, I see a TON of legit mail, for example, coming out of the Pacific Rim countries with no rptr's. regards, KAM
Re: Must-Have Plugins?
On Jun 10, 2015, at 12:32 AM, Matus UHLAR - fantomas wrote: > FEATURE(`block_bad_helo') > define(`confALLOW_BOGUS_HELO', `False') Argh, unfortunately, that feature is only on sendmail 8.14 and higher, which means RHEL/CentOS 6 or higher. For those of us running RHEL/CentOS 5, that's only available via a custom install. =( Does anyone know of a reputable RPM distro for sendmail 8.14+ for CentOS 5? I can't find anything decent via Google, everything is for CentOS 6 or higher. (My server setup requires RPMs, so I can't build from a source tarball. I could potentially use the source RPM from CentOS 6 to get a custom RPM for 5, although even that is problematic.) Bleh. I wish I could upgrade this server to a newer OS, but various circumstances prevent that right now. --- Amir
Re: Dealing with spammers making creative use of Google links ?
Am 10.06.2015 um 11:27 schrieb Ben: Does SA have pre-existing tests that look for this sort of thing (so I can just boost the score a little bit), or does anyone have any ideas how I might counter such spam ? Look at this fine specimen as an example : http://pastebin.com/raw.php?i=XjV94PMW Content analysis details: (23.8 points, 5.5 required) pts rule name description -- -- 1.0 CUST_DNSBL_21 RBL: dnsbl-uce.thelounge.net (dnsbl-1.uceprotect.net) [91.193.129.190 listed in dnsbl-uce.thelounge.net] 7.5 BAYES_99 BODY: Bayes spam probability is 99 to 100% [score: 0.9995] -0.1 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [91.193.129.190 listed in wl.mailspike.net] 4.5 CUST_DNSBL_8 RBL: dnsbl.inps.de [91.193.129.190 listed in dnsbl.inps.de] 4.5 CUST_DNSBL_7 RBL: b.barracudacentral.org [91.193.129.190 listed in b.barracudacentral.org] 0.4 BAYES_999 BODY: Bayes spam probability is 99.9 to 100% [score: 0.9995] 0.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.0 HTML_MESSAGE BODY: HTML included in message 2.5 RDNS_NONE Delivered to internal network by a host with no rDNS 2.5 TO_NO_BRKTS_NORDNS_HTML To: misformatted and no rDNS and HTML only 0.5 XPRIO Has X-Priority header signature.asc Description: OpenPGP digital signature
Re: Must-Have Plugins?
- Enable RBLs and DBLs. zen.spamhaus.org is the best way to block the majority of junk before it reaches SA. Just make sure you are below their free threshold limit. One important way to do this is "One important way to do this" in terms of the Spamhaus threshold limit is to not be such a tightwad and poney up for the Spamhaus commercial service. ;-) They do a cheaper version than the RSync feed, you can just query their servers directly. Spamhaus do a fantastic job. They deserve charitable donations from generous mail sysadmins !!! - Enable greylisting. Ewww... I hate people who operate graylisting. Its a lazy "tarnish everyone with the same brush" approach to anti-spam. In this day and age, you don't need it. Decent network checks, properly configured Spamassassin and you should be able to achieve a very respectable spam catching rate.
Dealing with spammers making creative use of Google links ?
Hi, Does SA have pre-existing tests that look for this sort of thing (so I can just boost the score a little bit), or does anyone have any ideas how I might counter such spam ? Look at this fine specimen as an example : http://pastebin.com/raw.php?i=XjV94PMW See how they cunningly use "http://www.google.com/url?q="; in order to obfuscate their URLs, as a DBL check countermeasure I suspect. Ideas most welcome ! Ben
