Re: Uptick in spam
On 1 Apr 2015, at 17:26, Amir Caspi wrote: On Apr 1, 2015, at 3:03 PM, Kevin Miller kevin.mil...@juneau.org wrote: You can reject on RDNS (or lack thereof) in sendmail depending on the version. Search for require_rdns. Thanks, I'll look into it. Sadly I don't think I have time to manually whitelist misconfigured servers, since I suspect there are not a few of them... a lot of people fail to put rDNS entries on their mail servers (including my own $DAYJOB employer, who only fixed it once I complained). That experience may bias your expectations. When I was handling spam control for a corporate system that had a million SMTP sessions per day and legit inbound mail in 5 digits per day with prominent public retail brand domains, our policy was to reject mail from IP's without valid rDNS. From 2004-2008 we had to whitelist *zero* sending systems and only had a handful of cases where we were the bully to get senders to fix their DNS. In the 7 years since I've been handling a much smaller corporate mail system of less significance to senders with the same policy, where we've seen no need to whitelist anyone and 2 cases where we know the policy has played some role in fixing senders' rDNS. However, it is a bit more common to have transient false positives due to DNS robustness issues (anything from connectivity problems to zone file typos) which are the price of any DNS-based filtering policy. The more DNS rules you enforce, the more ways DNS carelessness can be caught, and there's a richly diverse ecosystem of DNS carelessness.
Re: Uptick in spam
On 04/01/2015 10:45 PM, Amir Caspi wrote: Certainly it would be interesting to add such capability to SA (to add points for known spammy DNS providers and/or registrars), though I imagine that could be a recipe for FPs in some cases. Then again, we did it for .pw URIs, so... You can do it running your private dnsbl (using rbldnsd) and a urifullnsrhssub SA rule. It's not hard to do - cheap as well as effective.
Re: Uptick in spam
On Apr 1, 2015, at 3:03 PM, Kevin Miller kevin.mil...@juneau.org wrote: You can reject on RDNS (or lack thereof) in sendmail depending on the version. Search for require_rdns. Thanks, I'll look into it. Sadly I don't think I have time to manually whitelist misconfigured servers, since I suspect there are not a few of them... a lot of people fail to put rDNS entries on their mail servers (including my own $DAYJOB employer, who only fixed it once I complained). There may be other options than the firewall - if you have access to the mail server itself, you could maybe run an instance of iptables. I presume you're running it on Linux. Or maybe put the name servers in the /etc/host file with 127.0.0.x addresses? Not sure if that would work or not. If all else fails, bribe the DNS admin! :-) I do run iptables, which I use for fail2ban... but then I'd need to look up all the IP ranges served by the evil DNS servers. I could put the name servers in /etc/hosts but that would only help if I configure sendmail to require rDNS. Looks like there's no optimal solution on that one... Thanks. --- Amir
Re: Uptick in spam
On Apr 1, 2015, at 2:26 PM, Kevin Miller kevin.mil...@juneau.org wrote: I blocked the RRPPROXY.NET name servers at the firewall. [...] After I did that, almost instantly the spam dropped dramatically. [...] There was some discussion in this group about blocking on DNS providers about a month or so ago, spawned by my initial requests for help. I don't know if you have the luxury of dropping the connections at the firewall but it worked for me. Look back through the archives. Ah, I see... you killed them at the firewall itself, before they even got to sendmail. I was wondering how blocking the name servers themselves would help, since (at least in my configuration) sendmail doesn't reject just due to bad rDNS (not sure if that's even possible). Unfortunately, no, I don't have control over the firewall. Indeed, there is no hard firewall, so I only have software, and I'm not sure I have anything that could do specifically this. Certainly it would be interesting to add such capability to SA (to add points for known spammy DNS providers and/or registrars), though I imagine that could be a recipe for FPs in some cases. Then again, we did it for .pw URIs, so... --- Amir
RE: Uptick in spam
-Original Message- Ah, I see... you killed them at the firewall itself, before they even got to sendmail. I was wondering how blocking the name servers themselves would help, since (at least in my configuration) sendmail doesn't reject just due to bad rDNS (not sure if that's even possible). Unfortunately, no, I don't have control over the firewall. Indeed, there is no hard firewall, so I only have software, and I'm not sure I have anything that could do specifically this. Certainly it would be interesting to add such capability to SA (to add points for known spammy DNS providers and/or registrars), though I imagine that could be a recipe for FPs in some cases. Then again, we did it for .pw URIs, so... --- Amir You can reject on RDNS (or lack thereof) in sendmail depending on the version. Search for require_rdns. On my newer servers it's included in sendmail. On an older server I had to implement it as a hack. But it's easily found on the web, and wasn't hard to implement. Kills a lot of spam, but also some legitimate mail. I put the IP addresses of the legitimate (albeit misconfigured) servers in my access file and that seems to do the job. You will need to check the logs for rejects and decide who's OK. There may be other options than the firewall - if you have access to the mail server itself, you could maybe run an instance of iptables. I presume you're running it on Linux. Or maybe put the name servers in the /etc/host file with 127.0.0.x addresses? Not sure if that would work or not. If all else fails, bribe the DNS admin! :-) ...Kevin -- Kevin Miller Network/email Administrator, CBJ MIS Dept. 155 South Seward Street Juneau, Alaska 99801 Phone: (907) 586-0242, Fax: (907) 586-4500 Registered Linux User No: 307357
RE: Uptick in spam
I'm a bit late to the party (was on vacation) but your woes sounded awfully familiar. I was getting slammed by spam a couple months ago. The domains changed daily, but the one consistent thing was they were all served by RRPPROXY.NET. I blocked the RRPPROXY.NET name servers at the firewall. Doing a whois lookup on wheelerweightoff.com, I see that it is served by RRPPROXY.NET DNS servers: NS1, NS2, and NS3. I'd bet the others are too. After I did that, almost instantly the spam dropped dramatically. FWIW, I found no legitimate messages from the domains they hosted. Conveniently, they're a German company I think, and I'm in the US, so legitimate mail from them is unlikely. There was some discussion in this group about blocking on DNS providers about a month or so ago, spawned by my initial requests for help. I don't know if you have the luxury of dropping the connections at the firewall but it worked for me. Look back through the archives. ...Kevin -- Kevin Miller Network/email Administrator, CBJ MIS Dept. 155 South Seward Street Juneau, Alaska 99801 Phone: (907) 586-0242, Fax: (907) 586-4500 Registered Linux User No: 307357
Re: Uptick in spam
David Jones wrote: The invaluement RBL is not expensive either and it is awesome. We pay thousands per year for a Spamhaus feed because of our volume and mailboxes. The invaluement RBL is only hundreds per year and it's almost as good as Spamhaus Zen. Seconded; this is exactly what we've been finding. Invaluement is a great complement to Spamhaus for a fraction of the cost. I wouldn't put it as a front-line reject DNSBL, because some of the things that have been listed are not what I would class, for our customers, as spam - but those entries are distinctly greyhat at best in a lot of cases, and some IP range operators I've flagged as list, delist, and whitelist_from_rcvd as needed due to the mix of legitimate small senders and spammers. -kgd
Re: Uptick in spam
On 3/30/2015 11:49 AM, Kris Deugau wrote: Seconded; this is exactly what we've been finding. Invaluement is a great complement to Spamhaus for a fraction of the cost. I wouldn't put it as a front-line reject DNSBL, because some of the things that have been listed are not what I would class, for our customers, as spam - but those entries are distinctly greyhat at best in a lot of cases, and some IP range operators I've flagged as list, delist, and whitelist_from_rcvd as needed due to the mix of legitimate small senders and spammers. Thanks Kris for the compliment. Also, when you say mix of legitimate small senders ...just to clarify, I think that any further analysis will show that (a) MOST of these are situations where very small senders had massive spam-sending outbreaks due to compromised accounts, and (b) the listing was most often very short lived (often mere hours). This is a balancing act... and I think invaluement strikes a great balance. And even in THIS particular area, I think our FP level is still distinctly LESS than UCEProtect, Barracuda, and SORBS (for examples). But if we brought that all the way to zero, MUCH spam that slips past Zen wouldn't be listed on invaluement anymore. (the ham/spam ratios on some of these compromised account situations is horrendous--they send out their usual 400 hams that day, along with 200,000 spams... and the cumulative sum total of those spams from ALL such compromised senders that day, represents MUCH of the spam that gets past filters due to piggybacking on the sender's normally good reputation) Also, what I've found is that many medium-sized ISPs/hosters, with 10s of thousand of mailboxes are very comfortable with outright blocking on invaluement, but will only score on UCEProtect, Barracuda, and SORBS. Much smaller hosters will often block on all of them, because they don't notice those FPs as often. In fact, I see these SAME somewhat rare compromised-sender FPs with Zen, too. It is all about each list's strategies, and aggressiveness, and tolerance levels. As shown, invaluement is in a very strategic spot here... having much of the aggressiveness of these other lists, but with FP levels VERY close to Zen's FP levels. (and then scoring on these other lists... even aggressive, yet still under-threshold, scoring... will help block spams missed by both invaluement and spamhaus) Also, invaluement plays close to the edge with CAN-spam and snowshoe spammers. So invaluement is in a little more dangerous territory...that it can do so and not have a lot more FPs, is not easy. For example, this invaluement may occasionally list the kind of pure ads that, upon further analysis, are arguably not technically spam, but aren't exactly desired by the end users. But these situations tend to sort themselves out over time. The SAME thing happens with invaluement's ivmURI domain blacklist. OFTEN, a normally legit web site has a CURRENT... LIVE spam infestation, where spammers broke into that site and placed spammy content there. This has become epidemic. Sure, it is frustrating for everyone, when such a site that is being used to send phishing and porn spams... causes some of that site's legitimate correspondence to get blocked... but this a necessary lesser of evils. The best part is that such a blacklisting motivates the site owner to fix their site FASTER. In such a situation, the blacklist provided the world a good service, and the resulting collateral damage was well justified. The site owner should be considered at fault for the collateral damage, not the DNSBL. I hope this provides some clarity. -- Rob McEwen +1 478-475-9032
Re: Uptick in spam
On 3/30/2015 1:19 PM, Kris Deugau wrote: The cases I can recall are more along the lines of grey-hat ESPs who pick up a spammer client for a while, Kris, The next time you run across this and think it might be causing a little too much collateral damage (in spite of the spamming), let me know (off-list) and I'll research it. I can then make adjustments accordingly. I'm very responsive to customer feedback. Thanks! -- Rob McEwen +1 478-475-9032
Re: Uptick in spam
Rob McEwen wrote: On 3/30/2015 11:49 AM, Kris Deugau wrote: Seconded; this is exactly what we've been finding. Invaluement is a great complement to Spamhaus for a fraction of the cost. I wouldn't put it as a front-line reject DNSBL, because some of the things that have been listed are not what I would class, for our customers, as spam - but those entries are distinctly greyhat at best in a lot of cases, and some IP range operators I've flagged as list, delist, and whitelist_from_rcvd as needed due to the mix of legitimate small senders and spammers. Thanks Kris for the compliment. Also, when you say mix of legitimate small senders ...just to clarify, I think that any further analysis will show that (a) MOST of these are situations where very small senders had massive spam-sending outbreaks due to compromised accounts, and (b) the listing was most often very short lived (often mere hours). I haven't analyzed after the fact, but that sounds right. The cases I can recall are more along the lines of grey-hat ESPs who pick up a spammer client for a while, and unfortunately those ESPs also serve an assortment of (very) small businesses who send email that our customers want to receive. Often there's a free service tier, or free trial, and next to no up-front controls on who can send what content through these ESPs. I can't block these ESPs outright; customers *will* get upset. On the other hand, once notified of a sender I can make fairly sure that further mail *for that sender* through that ESP will make it to our customers' mailboxes. -kgd
Re: Uptick in spam
On Mar 30, 2015, at 9:49 AM, Kris Deugau kdeu...@vianet.ca wrote: Seconded; this is exactly what we've been finding. Invaluement is a great complement to Spamhaus for a fraction of the cost. Definitely something to add to my nice to have list for the future. Sadly, as I mentioned earlier, a paid subscription is simply not in the cards for me right now... so, unless a miracle happens, invaluement will have to go on my wish list rather than my to-do list. For now it seems I don't really have any viable alternative besides greylisting, as I'm already implementing most other free things I can think of (and that others have asked about). If anyone has additional suggestions besides what's been mentioned previously, please do let me know... Thanks. --- Amir
Re: Uptick in spam
From: Benny Pedersen m...@junc.eu Sent: Friday, March 27, 2015 10:48 PM To: users@spamassassin.apache.org Subject: Re: Uptick in spam David Jones skrev den 2015-03-28 03:13: I have Spamhaus in front of invaluement in my postfix configuration but I may try flipping the order just to see if it will start blocking more than Spamhaus. with postfix posttscreen one can test all ips on all rbls in same single smtpd client check, so there is no just spamhaus here :-) I know that but I choose to use the traditional method in the Postfix smtpd_recipient_restrictions so I can specify the order. I have such a high volume of mail for more than 100,000 mailboxes, I want to check in a specific order using my local rbldnsd feed to prevent abuse of other RBLs further down the list. If this were my personal mail filtering then I would use postscreen the same way you do. despite its called dnsbl in postscreen it supports whitelist aswell for me i have keeped all rbl checks from spamassassin into postscreen, i know there is more rbl lists i could add, but for me there is no need to, to many quererys makes to much dns trafic without more usefull data, and to make it more stable its nice that postscreen cache results on positive hits a little longer then ttl in dns It still makes sense to keep some reliable RBL checks in SA to catch outbound spam from authenticated users. You do want to filter outbound mail to keep your mail servers off of RBLs from compromised accounts. I use a few custom rules to subtract a little from outbound mail but it's not completely trusted.
Re: Uptick in spam
Am 28.03.2015 um 13:01 schrieb David Jones: From: Reindl Harald h.rei...@thelounge.net Sent: Saturday, March 28, 2015 6:13 AM To: users@spamassassin.apache.org Subject: Re: Uptick in spam Am 28.03.2015 um 12:04 schrieb David Jones: I know that but I choose to use the traditional method in the Postfix smtpd_recipient_restrictions so I can specify the order. I have such a high volume of mail for more than 100,000 mailboxes, I want to check in a specific order using my local rbldnsd feed to prevent abuse of other RBLs further down the list Thank you for the recommendation and I will research the impact that my high volume mail filters would cause to other RBLs that I do not have a local rbldnsd feed for. I have a local caching DNS server pointed to a set of private DNS servers hosting my rbldnsd zones so the impact should be as low as possible to the external RBL lookups. I have to be mindful of their free use limitations and abuse policies. (I have received emails from a few of them for excessive usage and had to discontinue using those.) hence postscreen and postscreen_dnsbl_ttl = 10m and/or if you use unbound as caching server: cache-min-ttl: 600 the problem with this approach is that with each RBL you raise the false-positive rates extremely, until now i did not see any RBL without FP be it Zen, Barracuda or Spamcop You are correct. This method does give complete power to each RBL to reject a message. If there were a way to specify the order of RBL checks then I could eliminate this problem. I will research this they are ordered as you list them in the rcpt restricitions, but that don't help because not listed on the first two one but on the third has the same effect: unconditional reject you could place DNSWL's in front but then you completly skip the RBL's and that won't work, another reason for postscreen: you combine different RBL's with different scores as well as DNSWL's with different negative scores to avoid false positives (see bottom of postscreen_dnsbl_sites) and with postscreen_greet_action = enforce a ton of botjunk is filtered out independent if it made it to blacklists, well and all that happens before touch smtpd at all Connections: 427269 Delivered: 56689 Reject Postscreen: 231729 Reject Postfix:17531 Blacklist: 227773 Pregreet: 27272 Hangup:272128 Protocol Error:2666 postscreen_dnsbl_ttl = 10m postscreen_dnsbl_threshold = 8 postscreen_dnsbl_action = enforce postscreen_greet_action = enforce postscreen_dnsbl_sites = b.barracudacentral.org=127.0.0.2*7 dnsbl.inps.de=127.0.0.2*7 bl.mailspike.net=127.0.0.2*5 bl.mailspike.net=127.0.0.[10;11;12]*4 dnsbl.sorbs.net=127.0.0.10*8 dnsbl.sorbs.net=127.0.0.5*6 dnsbl.sorbs.net=127.0.0.7*3 dnsbl.sorbs.net=127.0.0.8*2 dnsbl.sorbs.net=127.0.0.6*2 dnsbl.sorbs.net=127.0.0.9*2 zen.spamhaus.org=127.0.0.[10;11]*8 zen.spamhaus.org=127.0.0.[4..7]*6 zen.spamhaus.org=127.0.0.3*4 zen.spamhaus.org=127.0.0.2*3 hostkarma.junkemailfilter.com=127.0.0.2*3 hostkarma.junkemailfilter.com=127.0.0.4*1 hostkarma.junkemailfilter.com=127.0.1.2*1 wl.mailspike.net=127.0.0.[18;19;20]*-2 list.dnswl.org=127.0.[0..255].0*-2 list.dnswl.org=127.0.[0..255].1*-3 list.dnswl.org=127.0.[0..255].2*-4 list.dnswl.org=127.0.[0..255].3*-5 hostkarma.junkemailfilter.com=127.0.0.1*-2 signature.asc Description: OpenPGP digital signature
Re: Uptick in spam
From: Reindl Harald h.rei...@thelounge.net Sent: Saturday, March 28, 2015 6:13 AM To: users@spamassassin.apache.org Subject: Re: Uptick in spam Am 28.03.2015 um 12:04 schrieb David Jones: I know that but I choose to use the traditional method in the Postfix smtpd_recipient_restrictions so I can specify the order. I have such a high volume of mail for more than 100,000 mailboxes, I want to check in a specific order using my local rbldnsd feed to prevent abuse of other RBLs further down the list Thank you for the recommendation and I will research the impact that my high volume mail filters would cause to other RBLs that I do not have a local rbldnsd feed for. I have a local caching DNS server pointed to a set of private DNS servers hosting my rbldnsd zones so the impact should be as low as possible to the external RBL lookups. I have to be mindful of their free use limitations and abuse policies. (I have received emails from a few of them for excessive usage and had to discontinue using those.) the problem with this approach is that with each RBL you raise the false-positive rates extremely, until now i did not see any RBL without FP be it Zen, Barracuda or Spamcop You are correct. This method does give complete power to each RBL to reject a message. If there were a way to specify the order of RBL checks then I could eliminate this problem. I will research this. another thing is performance: smtpd_recipient_restrictions is sequential while postscreen asks all RBLs parallel, if one or more have a timeout it don't block, they are just not taken into account at that moment, when you have enough RBL's the result is still good I have very fast, low latency connections to the Internet so speed is not my problem. My typical batch processing time (30 emails) is under 5 seconds in MailScanner which is very good running 2 AV scanners. Postfix is a tiny fraction of that processing time and most of it is AV and SA. In SA, I have DCC (local DCC peer), Razor, Pyzor, Bayes in a redis DB, CRM114, and BOGOFILTER enabled. I have tuned SA from taking around 30 seconds to under 4 seconds per batch using safe shortcircuit rules and safe whitelist_from_* entries. The only spam I have a problem with is from compromised accounts for the first 30 minutes or so until RBLs kick in. I am still able to block most of the compromised account spam. I know I could turn on grey- listing and help with this but I feel that greylisting is not worth the delay _in our environment_ for the small gain that I would get. I want to look into selective greylisting when I get some time to build it out properly for our environment that is acceptable for our customers.
Re: Uptick in spam
On 03/28/2015 06:47 AM, Rob McEwen wrote: On 3/27/2015 10:13 PM, David Jones wrote: The invaluement RBL is not expensive either and it is awesome. We pay thousands per year for a Spamhaus feed because of our volume and mailboxes. The invaluement RBL is only hundreds per year and it's almost as good as Spamhaus Zen. I have Spamhaus in front of invaluement in my postfix configuration but I may try flipping the order just to see if it will start blocking more than Spamhaus. Just to clarify, the two invaluement sender's IP blacklists, ivmSIP and ivmSIP/24, --combined-- is not (and will probably not ever be) an adequate replacement for Spamhaus's Zen list. So please everyone, don't get the idea that you can turn off Zen, add invaluement, and everything will be ok. David Jones was NOT saying that... but i just want to make sure that nobody mistakenly goes too far with this, beyond what David intended. Having said that... thanks, David, (and others) for your mentioning about your success with ivmSIP and ivmSIP/24, where they are helping you block much of the spam that slips past Spamhaus, etc. When using SA, there is *ONE* good reason NOT to reject with the IVM lists: IVM rules can be safely scored/meta'd tflaged in such a way that the hits make perfect Bayes autolearn fodder .-) of course, we already know that autolearn is evil, useless and whatever other theories may apply - not worth yet another argument.
Re: Uptick in spam
Am 28.03.2015 um 12:04 schrieb David Jones: I know that but I choose to use the traditional method in the Postfix smtpd_recipient_restrictions so I can specify the order. I have such a high volume of mail for more than 100,000 mailboxes, I want to check in a specific order using my local rbldnsd feed to prevent abuse of other RBLs further down the list the problem with this approach is that with each RBL you raise the false-positive rates extremely, until now i did not see any RBL without FP be it Zen, Barracuda or Spamcop most caused by fools reporting mails they subsribed to as spam instead unsubscribe, frankly i got recently even two AOL feedback loops including the original message where customers of our ustomers reported there monthly e-bill including the bill itself as spam the same for digest services (Pyzor, Razor, IXHASH) with postscreen-scoring you can avoid that completly by not giving any RBL the power to reject a mail while at the same time add a few very false positive prones with a low score - the result is you block much more spam on RBL level with way less complaints and given that postscreen_dnsbl_ttl = 10m even applies to RBLs with a TTL auf just a few seconds (Spamhaus) your total dns queries could go down another thing is performance: smtpd_recipient_restrictions is sequential while postscreen asks all RBLs parallel, if one or more have a timeout it don't block, they are just not taken into account at that moment, when you have enough RBL's the result is still good signature.asc Description: OpenPGP digital signature
Re: Uptick in spam
From: Rob McEwen r...@invaluement.com Sent: Saturday, March 28, 2015 12:47 AM To: users@spamassassin.apache.org Subject: Re: Uptick in spam On 3/27/2015 10:13 PM, David Jones wrote: The invaluement RBL is not expensive either and it is awesome. We pay thousands per year for a Spamhaus feed because of our volume and mailboxes. The invaluement RBL is only hundreds per year and it's almost as good as Spamhaus Zen. I have Spamhaus in front of invaluement in my postfix configuration but I may try flipping the order just to see if it will start blocking more than Spamhaus. Just to clarify, the two invaluement sender's IP blacklists, ivmSIP and ivmSIP/24, --combined-- is not (and will probably not ever be) an adequate replacement for Spamhaus's Zen list. So please everyone, don't get the idea that you can turn off Zen, add invaluement, and everything will be ok. David Jones was NOT saying that... but i just want to make sure that nobody mistakenly goes too far with this, beyond what David intended. Thank you for making that clear. I only meant to say that I would be interested in putting ivm first just to see it's full blocking power. Right now it's second behind zen.spamhaus.org so I am not seeing it's complete potential. You made it clear in the setup that it was not a replacement for spamhaus. Having said that... thanks, David, (and others) for your mentioning about your success with ivmSIP and ivmSIP/24, where they are helping you block much of the spam that slips past Spamhaus, etc. It's a great product that is not widely known. I didn't hear about it soon enough. Thank you for making the cost very reasonable. Now we just need more people using it to support your efforts and keep the spam off of the Internet reliably. -- Rob McEwen
Re: Uptick in spam
On 03/27/2015 08:20 PM, Amir Caspi wrote: On Mar 27, 2015, at 12:56 PM, Matus UHLAR - fantomas uh...@fantomas.sk wrote: I see no network checks here... do you use network checks? On Mar 27, 2015, at 1:11 PM, Kevin A. McGrail kmcgr...@pccc.com wrote: Are you using network tests? These are scoring pretty high for me. I presume you're talking about things like Razor, Pyzor, DCC, and various RBLs? Yes, those are enabled. The reason you're not seeing them is because they didn't hit when the messages were first received. I'm getting the same hits NOW that you are seeing, but those did NOT hit when the messages first arrived. Remember that these messages were received a number of hours ago, so they have had plenty of time to be listed on RBLs and hash DBs in the intervening period. They were clearly not listed there when these messages were received, which is exactly why these messages are FNs. If they were received now, they wouldn't be... but they were back then. This is why I said in the prior message that it appears my user is one of the unlucky folks getting these in the very first distribution, before they've had a chance to be reported to RBLs and hash DBs. Some poor schmoe has to be in the first distribution, and it appears that he's one of them. This is why I'm looking for other, template-like rules that can be used to identify these things, because right now it seems my user is getting them on the first run before the network tests are useful. But, yes, network tests are absolutely enabled. Are you using Mailscanner? if yes then it's you munging URIS so they breaking lookups on any hash type as in http://pastebin.com/LaKT5ZZK And if you're indeed using MailScanner are you sending it the full message or some chunk only? (can't remember the settings's names)
Re: Uptick in spam
On Mar 27, 2015, at 12:56 PM, Matus UHLAR - fantomas uh...@fantomas.sk wrote: I see no network checks here... do you use network checks? On Mar 27, 2015, at 1:11 PM, Kevin A. McGrail kmcgr...@pccc.com wrote: Are you using network tests? These are scoring pretty high for me. I presume you're talking about things like Razor, Pyzor, DCC, and various RBLs? Yes, those are enabled. The reason you're not seeing them is because they didn't hit when the messages were first received. I'm getting the same hits NOW that you are seeing, but those did NOT hit when the messages first arrived. Remember that these messages were received a number of hours ago, so they have had plenty of time to be listed on RBLs and hash DBs in the intervening period. They were clearly not listed there when these messages were received, which is exactly why these messages are FNs. If they were received now, they wouldn't be... but they were back then. This is why I said in the prior message that it appears my user is one of the unlucky folks getting these in the very first distribution, before they've had a chance to be reported to RBLs and hash DBs. Some poor schmoe has to be in the first distribution, and it appears that he's one of them. This is why I'm looking for other, template-like rules that can be used to identify these things, because right now it seems my user is getting them on the first run before the network tests are useful. But, yes, network tests are absolutely enabled. Cheers. --- Amir
Re: Uptick in spam
On Mar 27, 2015, at 1:20 PM, Axb axb.li...@gmail.com wrote: These three samples are very different in the sense that #1 is a hacked site, #2 #3 are the regular snowshoe. Of course, I picked three different samples on purpose. But, I have hundreds that replicate these. What I miss in your sample's SA reports are any URIBL hits of some sort. Because there were no hits. That's exactly the point. Are you doing URIBL lookups? and using RAZOR PYZOR? Yes, using Razor, Pyzor, and DCC. Also using all default RBLs and URIBLs. Per my last message, the whole issue is that my user appears to be getting the hot of the presses run of these spams, before they have been reported to the RBLs, URIBLs, and hash DBs like Razor and Pyzor. Therefore, none of the network checks are getting hit... they are absolutely enabled, and a few hours later they would hit high scores, but upon initial receipt they simply do not hit because the spam is too new. This is my whole issue -- since my user appears to be very high up on the recipient list for all these spammers, and is therefore getting spams before the network checks are effective, how can we combat these new spams _before_ the network checks become effective? Thanks. --- Amir
Re: Uptick in spam
On Mar 27, 2015, at 1:33 PM, Axb axb.li...@gmail.com wrote: Are you using Mailscanner? if yes then it's you munging URIS so they breaking lookups on any hash type as in Yes, I am using MailScanner. Some URIs are munged, others are not. For example, you can see in that very pastebin you noted that there are a number of perfectly good URIs. MailScanner will munge the embedded image web bugs and the embedded JavaScript, but will not munge regular href links or regular img links. In that sample, the only MailScanner munging is on JavaScript. But, you're saying MailScanner is changing the message and therefore changing the hash overall... yes? Would you recommend not running MailScanner? If so, what would you recommend for virus scanning? Or, would you recommend turning off munging for embedded JS and web bugs? (But, keeping the virus scanning?) Of course, removing munging opens other vulnerabilities... Note that my spam setup is as follows: sendmail - MailScanner (system-wide, root-owned) - spamc/spamd (per-user, via procmail) Unfortunately due to the nature of the virtual-host setup on this machine I _cannot_ have MailScanner be the SA glue, nor can I easily switch to SA milters like spamass-milter or amavisd or whatever. Right now, this setup is unfortunately not changeable. And if you're indeed using MailScanner are you sending it the full message or some chunk only? (can't remember the settings's names) I am passing in the entire message. Thanks. --- Amir
Re: Uptick in spam
On 03/27/2015 08:45 PM, Amir Caspi wrote: On Mar 27, 2015, at 1:33 PM, Axb axb.li...@gmail.com wrote: Are you using Mailscanner? if yes then it's you munging URIS so they breaking lookups on any hash type as in Yes, I am using MailScanner. Some URIs are munged, others are not. For example, you can see in that very pastebin you noted that there are a number of perfectly good URIs. MailScanner will munge the embedded image web bugs and the embedded JavaScript, but will not munge regular href links or regular img links. In that sample, the only MailScanner munging is on JavaScript. But, you're saying MailScanner is changing the message and therefore changing the hash overall... yes? Would you recommend not running MailScanner? If so, what would you recommend for virus scanning? Or, would you recommend turning off munging for embedded JS and web bugs? (But, keeping the virus scanning?) Of course, removing munging opens other vulnerabilities... I used MS for few years - It did the job. As an AV product I'd recommend Sophos AND ESETS/Nod32. I'd also suggest you disable msg munging if you want hashers to work. URI lists may also list URIs to .js and web bugs - you could be missing on them. Note that my spam setup is as follows: sendmail - MailScanner (system-wide, root-owned) - spamc/spamd (per-user, via procmail) __ Unfortunately due to the nature of the virtual-host setup on this machine I _cannot_ have MailScanner be the SA glue, nor can I easily switch to SA milters like spamass-milter or amavisd or whatever. Right now, this setup is unfortunately not changeable. Are you an ISP/ASP or is this a corporate box? What are you really using MailScanner for? I also wonder if you're doing any rejects at SMTP level.
Re: Uptick in spam
On 03/27/2015 07:51 PM, Amir Caspi wrote: Here are a few spamples: http://pastebin.com/3nSLurGv (this scored BAYES_99 but would still have been FN with BAYES_999) http://pastebin.com/LaKT5ZZK (I have a rule template for these URIs but recent spams have modified them to cause high risk of FPs for such rules) http://pastebin.com/qSgBxR5B (BAYES_999; could potentially be caught by an excessive HTML entity rule, but none seemed to hit... is there one?) For the first and last one, the URIs are way too similar to blog URIs that would be in use by legitimate agencies, so I suspect there is a high risk for FPs on those. The middle one uses a template that I have URI rules for, but the URIs are evolving to use randomized server names which are also basically impossible to template against without risk of FPs. I have hundreds more like these... These three samples are very different in the sense that #1 is a hacked site, #2 #3 are the regular snowshoe. What I miss in your sample's SA reports are any URIBL hits of some sort. Are you doing URIBL lookups? and using RAZOR PYZOR? Axb
Re: Uptick in spam
Apologies if this is an overly obvious answer, but are you using any greylisting? This would (potentially) move your user away from the wavefront of a spam's distribution, and give it a better chance of triggering the network-based tests. On Fri, 27 Mar 2015, Amir Caspi wrote: This is my whole issue -- since my user appears to be very high up on the recipient list for all these spammers, and is therefore getting spams before the network checks are effective, how can we combat these new spams _before_ the network checks become effective? Thanks. --- Amir -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT CompSci =--+--- All syllogisms contain three lines | sha...@shanew.net Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: Uptick in spam
On Mar 27, 2015, at 1:38 PM, sha...@shanew.net wrote: Apologies if this is an overly obvious answer, but are you using any greylisting? This would (potentially) move your user away from the wavefront of a spam's distribution, and give it a better chance of triggering the network-based tests. No, unfortunately not. It's something I've been considering but with my current system setup I don't know of an easy way to implement it. Unfortunately the system setup is fixed due to the virtual hosting software being run on it. There is a possibility this can change in the future, depending on our client setup, but right now we're stuck with it, so I can't do things like use amavisd or dovecot or whatever. If I can easily implement greylisting from within sendmail without breaking the current setup, that's certainly something I'd consider doing... Of course, I am aware of the debate regarding greylisting. In particular, this can cause significant problems for one-time password emails, e.g. from banks, where a significant delay in delivery causes huge problems. I'm not sure how to work around that. Thanks. --- Amir
Re: Uptick in spam
On Feb 16, 2015, at 11:47 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: I'm happy to look at a recent sample and throw it through my system to see what it hits but overall, I've been seeing the exact opposite. So, one of my users has been getting dozens (sometimes nearly 100) FNs per DAY over the last few weeks. Even though many of these emails are hitting BAYES_999, they are not hitting any other non-negligible scoring rules. I have set BAYES_99 + BAYES_999 to a combined score of 4.9 because I don't want it to be a complete poison pill, but this is contributing to something like 50% of the FNs (where only BAYES_999 is contributing to the score because no other rules are hitting). The other 50% are not getting high-enough Bayes scores, but even then, many still don't hit many (or any) other scoring rules so that they would still have this problem even if they scored BAYES_999. In many cases, it would appear that he is getting a fresh batch that hasn't yet hit the RBLs or hash DBs, which is why even with BAYES_999 they don't score over the 5.0 threshold... it's causing some severe inbox unpleasantness. I've been trying to come up with some good URI template rules to block many of these but spammers are getting sufficiently generic in their URIs that I worry strongly about FPs for these. I haven't been able to identify any other distinctive markers in the template against which I can reliably write rules, although I also don't have a program that does strong comparisons to look for patterns (I'm just doing this by eye). I have his spam corpus of a few thousand messages... simple Bayes training doesn't seem to help, so some sort of template matching would really be useful here, but as I said, I haven't really found anything that I feel comfortable writing rules against without significant risk of FPs. Might anyone have some ideas? This is getting to be a serious issue for this user and I'm getting complaints... Thanks. (For reference: running SA 3.4.0 on CentOS 5.11.) --- Amir
Re: Uptick in spam
On Mar 27, 2015, at 12:20 PM, Axb axb.li...@gmail.com wrote: - Please post missed spam samples in pastebin.com - do not post samples to mailing lists Of course, I would never post it to the list. I will put up a few in pastebin but there are so many of them, and there are a few different templates in use, so I don't know if I can really capture them all. I obviously can't post the entire corpus on pastebin. ;-) Here are a few spamples: http://pastebin.com/3nSLurGv (this scored BAYES_99 but would still have been FN with BAYES_999) http://pastebin.com/LaKT5ZZK (I have a rule template for these URIs but recent spams have modified them to cause high risk of FPs for such rules) http://pastebin.com/qSgBxR5B (BAYES_999; could potentially be caught by an excessive HTML entity rule, but none seemed to hit... is there one?) For the first and last one, the URIs are way too similar to blog URIs that would be in use by legitimate agencies, so I suspect there is a high risk for FPs on those. The middle one uses a template that I have URI rules for, but the URIs are evolving to use randomized server names which are also basically impossible to template against without risk of FPs. I have hundreds more like these... Cheers. --- Amir
Re: Uptick in spam
Am 27.03.2015 um 19:13 schrieb Amir Caspi: On Feb 16, 2015, at 11:47 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: I'm happy to look at a recent sample and throw it through my system to see what it hits but overall, I've been seeing the exact opposite. So, one of my users has been getting dozens (sometimes nearly 100) FNs per DAY over the last few weeks. Even though many of these emails are hitting BAYES_999, they are not hitting any other non-negligible scoring rules what here helps a lot are custom subject rules * contains * starts with * ends with * equal 4 different score levels * very low: 0.5 * low: 1.5 * medium: 2.5 * high: 3.5 very high: 4.5 we have currently 577 different subjects and subject-parts scored , i don't want to publish them because i'd like the spammers don't change to new ones :-) signature.asc Description: OpenPGP digital signature
Re: Uptick in spam
On 27.03.15 12:51, Amir Caspi wrote: Here are a few spamples: http://pastebin.com/3nSLurGv (this scored BAYES_99 but would still have been FN with BAYES_999) http://pastebin.com/LaKT5ZZK (I have a rule template for these URIs but recent spams have modified them to cause high risk of FPs for such rules) http://pastebin.com/qSgBxR5B (BAYES_999; could potentially be caught by an excessive HTML entity rule, but none seemed to hit... is there one?) I see no network checks here... do you use network checks? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: Uptick in spam
On 03/27/2015 07:13 PM, Amir Caspi wrote: On Feb 16, 2015, at 11:47 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: I'm happy to look at a recent sample and throw it through my system to see what it hits but overall, I've been seeing the exact opposite. So, one of my users has been getting dozens (sometimes nearly 100) FNs per DAY over the last few weeks. Even though many of these emails are hitting BAYES_999, they are not hitting any other non-negligible scoring rules. I have set BAYES_99 + BAYES_999 to a combined score of 4.9 because I don't want it to be a complete poison pill, but this is contributing to something like 50% of the FNs (where only BAYES_999 is contributing to the score because no other rules are hitting). The other 50% are not getting high-enough Bayes scores, but even then, many still don't hit many (or any) other scoring rules so that they would still have this problem even if they scored BAYES_999. In many cases, it would appear that he is getting a fresh batch that hasn't yet hit the RBLs or hash DBs, which is why even with BAYES_999 they don't score over the 5.0 threshold... it's causing some severe inbox unpleasantness. I've been trying to come up with some good URI template rules to block many of these but spammers are getting sufficiently generic in their URIs that I worry strongly about FPs for these. I haven't been able to identify any other distinctive markers in the template against which I can reliably write rules, although I also don't have a program that does strong comparisons to look for patterns (I'm just doing this by eye). I have his spam corpus of a few thousand messages... simple Bayes training doesn't seem to help, so some sort of template matching would really be useful here, but as I said, I haven't really found anything that I feel comfortable writing rules against without significant risk of FPs. Might anyone have some ideas? This is getting to be a serious issue for this user and I'm getting complaints... - Please post missed spam samples in pastebin.com - do not post samples to mailing lists
Re: Uptick in spam
On Fri, 27 Mar 2015 12:13:30 -0600 Amir Caspi wrote: On Feb 16, 2015, at 11:47 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: I'm happy to look at a recent sample and throw it through my system to see what it hits but overall, I've been seeing the exact opposite. So, one of my users has been getting dozens (sometimes nearly 100) FNs per DAY over the last few weeks. Even though many of these emails are hitting BAYES_999, they are not hitting any other non-negligible scoring rules. I have set BAYES_99 + BAYES_999 to a combined score of 4.9 because I don't want it to be a complete poison pill, Personally I've found that trying to work around BAYES_99 not being a poison pill causes more FPs making it one YMMV.
Re: Uptick in spam
On Mar 27, 2015, at 12:22 PM, Reindl Harald h.rei...@thelounge.net wrote: we have currently 577 different subjects and subject-parts scored , i don't want to publish them because i'd like the spammers don't change to new ones :-) Sadly, that doesn't help me. I don't have time to compile hundreds of subject rules, managing email is not my full-time job and I don't want it to become one. If you care to share, that would be much appreciated, but otherwise I can't spend time writing hundreds of custom rules. This is why I look for URI templates where regexps work well... looking for keywords or key phrases would be a huge quagmire, and that's what Bayes is supposed to be for. As to publishing, I personally feel holding rules to one's self is not productive. Spammers evolve regardless, and in the meantime those templates benefit nobody but one's own system. Distributing them publicly will help everyone and could help others publish better rules in the future. Obviously, others may disagree. Cheers. --- Amir
Re: Uptick in spam
On 3/27/2015 2:51 PM, Amir Caspi wrote: On Mar 27, 2015, at 12:20 PM, Axb axb.li...@gmail.com wrote: - Please post missed spam samples in pastebin.com - do not post samples to mailing lists Of course, I would never post it to the list. I will put up a few in pastebin but there are so many of them, and there are a few different templates in use, so I don't know if I can really capture them all. I obviously can't post the entire corpus on pastebin. ;-) Are you using network tests? These are scoring pretty high for me.
Re: Uptick in spam
On Fri, 27 Mar 2015, Amir Caspi wrote: On Mar 27, 2015, at 12:56 PM, Matus UHLAR - fantomas uh...@fantomas.sk wrote: I see no network checks here... do you use network checks? On Mar 27, 2015, at 1:11 PM, Kevin A. McGrail kmcgr...@pccc.com wrote: Are you using network tests? These are scoring pretty high for me. I presume you're talking about things like Razor, Pyzor, DCC, and various RBLs? Yes, those are enabled. The reason you're not seeing them is because they didn't hit when the messages were first received. I'm getting the same hits NOW that you are seeing, but those did NOT hit when the messages first arrived. Have you considered greylisting? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The one political issue that strips all politicians bare is individual gun rights. --- 5 days until April Fools' day
Re: Uptick in spam
On Fri, 27 Mar 2015, Amir Caspi wrote: On Mar 27, 2015, at 1:38 PM, sha...@shanew.net wrote: Apologies if this is an overly obvious answer, but are you using any greylisting? This would (potentially) move your user away from the wavefront of a spam's distribution, and give it a better chance of triggering the network-based tests. No, unfortunately not. It's something I've been considering but with my current system setup I don't know of an easy way to implement it. Unfortunately the system setup is fixed due to the virtual hosting software being run on it. There is a possibility this can change in the future, depending on our client setup, but right now we're stuck with it, so I can't do things like use amavisd or dovecot or whatever. If I can easily implement greylisting from within sendmail without breaking the current setup, that's certainly something I'd consider doing... (all caught up now, sheesh). Can you install milters? Take a look at milter-greylist. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The one political issue that strips all politicians bare is individual gun rights. --- 5 days until April Fools' day
Re: Uptick in spam
On 03/27/2015 11:51 AM, Amir Caspi wrote: On Mar 27, 2015, at 12:20 PM, Axb axb.li...@gmail.com wrote: - Please post missed spam samples in pastebin.com - do not post samples to mailing lists Of course, I would never post it to the list. I will put up a few in pastebin but there are so many of them, and there are a few different templates in use, so I don't know if I can really capture them all. I obviously can't post the entire corpus on pastebin. ;-) Here are a few spamples: http://pastebin.com/3nSLurGv (this scored BAYES_99 but would still have been FN with BAYES_999) http://pastebin.com/LaKT5ZZK (I have a rule template for these URIs but recent spams have modified them to cause high risk of FPs for such rules) http://pastebin.com/qSgBxR5B (BAYES_999; could potentially be caught by an excessive HTML entity rule, but none seemed to hit... is there one?) All of these were From: domains created today. For the first and last one, the URIs are way too similar to blog URIs that would be in use by legitimate agencies, so I suspect there is a high risk for FPs on those. The middle one uses a template that I have URI rules for, but the URIs are evolving to use randomized server names which are also basically impossible to template against without risk of FPs. I have hundreds more like these... Cheers. --- Amir
Re: Uptick in spam
On Mar 27, 2015, at 2:09 PM, Axb axb.li...@gmail.com wrote: As an AV product I'd recommend Sophos AND ESETS/Nod32. I'll look into Sophos, I'm not entirely sure if I can deploy it on my system or not. We have to use RPMs that can be distributed to the virtual hosts, etc... I'll definitely look into it. Haven't heard about ESETS/Nod32, will check it out. I'd also suggest you disable msg munging if you want hashers to work. I'll certainly consider that if this is a major issue. I see hashers working on many other messages, but I'm not sure how munged those messages are. I'll try to investigate to see if I've seen hash hits on munged messages... Turning off munging will unfortunately reduce security since it allows embedded JS and web bugs, but if it improves the chances of those things getting properly tagged as spam then they won't open them anyway, so I guess it may come out in the wash. URI lists may also list URIs to .js and web bugs - you could be missing on them. Very good point. Are you an ISP/ASP or is this a corporate box? A bit of both. We run a dedicated server that is owned by a major ISP, but they basically only handle the upstream end. We are root on the box and handle everything downstream. We run a virtual hosting panel and our corporate clients run domains (for email and web hosting) as virtual hosts on the box. Each virthost is operated in a chroot environment, and the control panel distributes the central RPMs to each virthost. So, everything we do has to work with the framework of the control panel and its virtual hosting environment. What are you really using MailScanner for? Primarily as glue to clamav (via clamd) and for attachment policy enforcement (e.g., no .exe payloads), and secondarily for URI munging. I also wonder if you're doing any rejects at SMTP level. Yes, I've implemented enhdnsbl in sendmail, querying SpamCop, Barracuda, and SpamHaus Zen (in that order). I know Barracuda is often overzealous but we haven't seen any FP rejections (that we know of) yet. Are there any other RBLs you suggest I add to sendmail's checks? (I used to use NJABL but that's dead, and last time I asked on this list, I was told SORBS wasn't a good idea due to too many FP rejections.) I also have greetpause enabled (at 1 sec) to reject trigger-happy spammers. Cheers. --- Amir
Re: Uptick in spam
On Mar 27, 2015, at 3:34 PM, Richard Doyle lists...@islandnetworks.com wrote: All of these were From: domains created today. Shouldn't they have been picked up by DOB? Or do I need to manually enable some DOB plugin in SA? (If so, please let me know how...) When I ran the third spample manually a few hours ago, I still didn't see any DOB hit. I see there is a URIBL_RHS_DOB... is there a SENDER_DOB rule as well? If not, it seems like it would be a good idea to implement one... do I need to file a bug for it? However, it would appear that all of the From: domains are the same as in the body URIs, which means URIBL_RHS_DOB should have popped... unless you mean that the subdomain (sub.domain.com) was DOB, but the main domain (www.domain.com and/or domain.com) were not DOB? Or am I missing something? Thanks. --- Amir
Re: Uptick in spam
On 03/27/2015 11:44 PM, Amir Caspi wrote: On Mar 27, 2015, at 3:34 PM, Richard Doyle lists...@islandnetworks.com wrote: All of these were From: domains created today. Shouldn't they have been picked up by DOB? Or do I need to manually enable some DOB plugin in SA? (If so, please let me know how...) When I ran the third spample manually a few hours ago, I still didn't see any DOB hit. I see there is a URIBL_RHS_DOB... is there a SENDER_DOB rule as well? If not, it seems like it would be a good idea to implement one... do I need to file a bug for it? However, it would appear that all of the From: domains are the same as in the body URIs, which means URIBL_RHS_DOB should have popped... unless you mean that the subdomain (sub.domain.com) was DOB, but the main domain (www.domain.com and/or domain.com) were not DOB? Or am I missing something? DOB isn't realtime/zero hour. I have zero Sendmail clue but if you can do it, also check sender/helo/rdns against dbl.spamhaus.org's reply 127.0.1.2 (I can only provide Postfix config for this) if you want to check sender in DOB you can use eval:check_rbl_envfrom for a rule. A few days ago I posted dbl_env_from.cf which should show how it's done (the rule is untested) http://mail-archives.apache.org/mod_mbox/spamassassin-users/201503.mbox/%3C55128D61.2020308%40gmail.com%3E You also may want to look at the Invaluement IP/URI lists. (Invaluement.com). Detection rate is real good and FP level is extraordinary. IIRC you can get a test drive. I wouldn't want to miss it.
Re: Uptick in spam
On Mar 27, 2015, at 5:12 PM, Axb axb.li...@gmail.com wrote: DOB isn't realtime/zero hour. That kind of defeats the point, isn't it? I mean, if you wait too long, it's no longer DOB, it's few-DOB... I would have imagined that a DOB server would operate in a caching mode where the first query on a domain would cause a whois lookup, which then generates a cache table entry with the reg date. Subsequent lookups then don't incur a whois hit, they just check the cache table. In this way it could be effectively realtime since only the first query causes a whois load, and it would always return the correct answer. I guess that's not the case? I have zero Sendmail clue but if you can do it, also check sender/helo/rdns against dbl.spamhaus.org's reply 127.0.1.2 I haven't found a way to do this, but if someone knows, please post... You also may want to look at the Invaluement IP/URI lists. (Invaluement.com). Detection rate is real good and FP level is extraordinary. IIRC you can get a test drive. I wouldn't want to miss it. Unfortunately a paid service is not in the cards right now. Does anyone recommend using the PSBL (Surriel) for sendmail dnsbl? I see that it's enabled by default in SA, but should I promote it to the sendmail level, or is it too prone to FP? On a related note... since I implemented SpamCop, Barracuda, and SpamHaus at the sendmail level, should I disable those RBL lookups in SA, to prevent double-querying the RBLs for those mails that do get through? Or does SA check _all_ Received lines, in which case I should leave it enabled since sendmail only checks the connecting MTA? (I should note that I _HAVE_ seen RCVD_IN_XBL/PBL/SBL and RCVD_IN_BL_SPAMCOP_NET pop up not infrequently, despite implementing dnsbl for those RBLs in sendmail, which means either they're getting listed in the small interval between sendmail and SA, or SA is checking more than just the last hop...) Thanks. --- Amir
Re: Uptick in spam
On 03/27/2015 03:44 PM, Amir Caspi wrote: On Mar 27, 2015, at 3:34 PM, Richard Doyle lists...@islandnetworks.com wrote: All of these were From: domains created today. Shouldn't they have been picked up by DOB? Or do I need to manually enable some DOB plugin in SA? (If so, please let me know how...) When I ran the third spample manually a few hours ago, I still didn't see any DOB hit. I see there is a URIBL_RHS_DOB... is there a SENDER_DOB rule as well? If not, it seems like it would be a good idea to implement one... do I need to file a bug for it? However, it would appear that all of the From: domains are the same as in the body URIs, which means URIBL_RHS_DOB should have popped... unless you mean that the subdomain (sub.domain.com) was DOB, but the main domain (www.domain.com and/or domain.com) were not DOB? Or am I missing something? DOB misses many new domains. Whois often knows what's new, but using it to detect spam doesn't scale. Thanks. --- Amir
Re: Uptick in spam
On Fri, 27 Mar 2015 17:40:58 -0600 Amir Caspi wrote: On Mar 27, 2015, at 5:12 PM, Axb axb.li...@gmail.com wrote: DOB isn't realtime/zero hour. That kind of defeats the point, isn't it? I mean, if you wait too long, it's no longer DOB, it's few-DOB... I think it's 5 days, and the day-old bit is part of the bread metaphor, not the definition. On a related note... since I implemented SpamCop, Barracuda, and SpamHaus at the sendmail level, should I disable those RBL lookups in SA, to prevent double-querying the RBLs for those mails that do get through? Or does SA check _all_ Received lines, in which case I should leave it enabled since sendmail only checks the connecting MTA? (I should note that I _HAVE_ seen RCVD_IN_XBL/PBL/SBL and RCVD_IN_BL_SPAMCOP_NET pop up not infrequently, despite implementing dnsbl for those RBLs in sendmail, which means either they're getting listed in the small interval between sendmail and SA, or SA is checking more than just the last hop...) There are deep checks for SBL (via zen) and SPAMCOP. XBL/PBL are last-external only
Re: Uptick in spam
On 03/28/2015 12:40 AM, Amir Caspi wrote: On Mar 27, 2015, at 5:12 PM, Axb axb.li...@gmail.com wrote: DOB isn't realtime/zero hour. That kind of defeats the point, isn't it? I mean, if you wait too long, it's no longer DOB, it's few-DOB... I would have imagined that a DOB server would operate in a caching mode where the first query on a domain would cause a whois lookup, which then generates a cache table entry with the reg date. Subsequent lookups then don't incur a whois hit, they just check the cache table. In this way it could be effectively realtime since only the first query causes a whois load, and it would always return the correct answer. I guess that's not the case? DOB is based on more or less publicly accessible daily TLD zone data (ICANN ZFA) You're thinking passive DNS, as done by https://www.farsightsecurity.com/ I have access to their DNSDB service for a hobby project and it's amazing. Farsight's NOD service is way out of our means. Does anyone recommend using the PSBL (Surriel) for sendmail dnsbl? I see that it's enabled by default in SA, but should I promote it to the sendmail level, or is it too prone to FP? It works fine for a family server, but I wouldn't use it for rejecting spam in a client's mailflow. On a related note... since I implemented SpamCop, Barracuda, and SpamHaus at the sendmail level, should I disable those RBL lookups in SA, to prevent double-querying the RBLs for those mails that do get through? Or does SA check _all_ Received lines, in which case I should leave it enabled since sendmail only checks the connecting MTA? (I should note that I _HAVE_ seen RCVD_IN_XBL/PBL/SBL and RCVD_IN_BL_SPAMCOP_NET pop up not infrequently, despite implementing dnsbl for those RBLs in sendmail, which means either they're getting listed in the small interval between sendmail and SA, or SA is checking more than just the last hop...) Hard to say without tailing your maillogs. Though, if you have your trusted/internal SA settings right, extra SA checks shouldn't be an issue as you may already have most of the data in your resolver's cache anyway.
Re: Uptick in spam
From: Amir Caspi ceph...@3phase.com Sent: Friday, March 27, 2015 7:30 PM To: RW Cc: users@spamassassin.apache.org Subject: Re: Uptick in spam On Mar 27, 2015, at 6:19 PM, RW rwmailli...@googlemail.com wrote: There are deep checks for SBL (via zen) and SPAMCOP. XBL/PBL are last-external only Interesting. I wonder why I see those XBL/PBL hits, then. Maybe Zen timed out on those queries from sendmail... or something. Either way I guess this means I should retain Zen and SC queries in SA. You should be running a local dns caching server like BIND or PowerDNS Recursor on a mail server to help prevent time outs that can allow RBL checks to become ineffective. It's possible that your outbound mail could be hitting those RBLs in SA in the event of a compromised account or the last-external IP in the Received: depending on what internal mail server you use and if it puts that information in as X-Originating-IP or Received headers of the sending mail client. I would recommend keeping those RBLs in SA to help with outbound scanning and in case they get past the MTA-level RBL checking. It shouldn't be duplicate hits to Zen/XBL/PBL if you have sendmail rejecting that message from making it to SA. If you get any of those RBL hits in SA that sendmail is configured to reject on, then there must be some sendmail access list allowing it to bypass the RBL checks. Esets NOD32 is very fast, very inexpensive, and works well with MailScanner. The invaluement RBL is not expensive either and it is awesome. We pay thousands per year for a Spamhaus feed because of our volume and mailboxes. The invaluement RBL is only hundreds per year and it's almost as good as Spamhaus Zen. I have Spamhaus in front of invaluement in my postfix configuration but I may try flipping the order just to see if it will start blocking more than Spamhaus. Dave Thanks. --- Amir
Re: Uptick in spam
You also may want to look at the Invaluement IP/URI lists. (Invaluement.com). Detection rate is real good and FP level is extraordinary. +1. Very happy with invaluement at $DAYJOB. -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com
Re: Uptick in spam
On Mar 27, 2015, at 6:19 PM, RW rwmailli...@googlemail.com wrote: There are deep checks for SBL (via zen) and SPAMCOP. XBL/PBL are last-external only Interesting. I wonder why I see those XBL/PBL hits, then. Maybe Zen timed out on those queries from sendmail... or something. Either way I guess this means I should retain Zen and SC queries in SA. Thanks. --- Amir
Re: Uptick in spam
David Jones skrev den 2015-03-28 03:13: I have Spamhaus in front of invaluement in my postfix configuration but I may try flipping the order just to see if it will start blocking more than Spamhaus. with postfix posttscreen one can test all ips on all rbls in same single smtpd client check, so there is no just spamhaus here :-) despite its called dnsbl in postscreen it supports whitelist aswell for me i have keeped all rbl checks from spamassassin into postscreen, i know there is more rbl lists i could add, but for me there is no need to, to many quererys makes to much dns trafic without more usefull data, and to make it more stable its nice that postscreen cache results on positive hits a little longer then ttl in dns
Re: Uptick in spam
On 3/27/2015 10:13 PM, David Jones wrote: The invaluement RBL is not expensive either and it is awesome. We pay thousands per year for a Spamhaus feed because of our volume and mailboxes. The invaluement RBL is only hundreds per year and it's almost as good as Spamhaus Zen. I have Spamhaus in front of invaluement in my postfix configuration but I may try flipping the order just to see if it will start blocking more than Spamhaus. Just to clarify, the two invaluement sender's IP blacklists, ivmSIP and ivmSIP/24, --combined-- is not (and will probably not ever be) an adequate replacement for Spamhaus's Zen list. So please everyone, don't get the idea that you can turn off Zen, add invaluement, and everything will be ok. David Jones was NOT saying that... but i just want to make sure that nobody mistakenly goes too far with this, beyond what David intended. Having said that... thanks, David, (and others) for your mentioning about your success with ivmSIP and ivmSIP/24, where they are helping you block much of the spam that slips past Spamhaus, etc. -- Rob McEwen
Re: Uptick in spam (bayes stats script)
Am 22.02.2015 um 15:30 schrieb @lbutlr: On 21 Feb 2015, at 08:34 , LuKreme krem...@kreme.com wrote: On Feb 18, 2015, at 6:20 AM, Reindl Harald h.rei...@thelounge.net wrote: bayes-stats.txt That is a lot cleaner and more obvious, thank you for sharing I ran this just after log rotation and got div by zero errors, so here is a (nearly) completely pointless ‘fix’: BAYES_TOTAL=`echo $BAYES_00+$BAYES_05+$BAYES_20+$BAYES_40+$BAYES_50+$BAYES_60+$BAYES_80+$BAYES_95+$BAYES_99 | bc` + if [ ! $BAYES_TOTAL ]; then BAYES_00_PCT=`echo scale=2; ($BAYES_00*100)/$BAYES_TOTAL | bc | sed 's/^\./0./‘` … echo -e BAYES_999 `printf \%*s\ 8 $BAYES_999` `printf \%*s\ 7 $BAYES_999_PCT` %” + fi Yes, yes, I know, had I run the script a minute later, no error. But if I didn’t have OCD tendencies, would I even be on this list? :) agreed - thanks - but the f don't work here, below a better one - if [ ! $BAYES_TOTAL ]; then + if [ $BAYES_TOTAL -gt 0 ]; then signature.asc Description: OpenPGP digital signature
Re: Uptick in spam (bayes stats script)
On 21 Feb 2015, at 08:34 , LuKreme krem...@kreme.com wrote: On Feb 18, 2015, at 6:20 AM, Reindl Harald h.rei...@thelounge.net wrote: bayes-stats.txt That is a lot cleaner and more obvious, thank you for sharing I ran this just after log rotation and got div by zero errors, so here is a (nearly) completely pointless ‘fix’: BAYES_TOTAL=`echo $BAYES_00+$BAYES_05+$BAYES_20+$BAYES_40+$BAYES_50+$BAYES_60+$BAYES_80+$BAYES_95+$BAYES_99 | bc` + if [ ! $BAYES_TOTAL ]; then BAYES_00_PCT=`echo scale=2; ($BAYES_00*100)/$BAYES_TOTAL | bc | sed 's/^\./0./‘` … echo -e BAYES_999 `printf \%*s\ 8 $BAYES_999` `printf \%*s\ 7 $BAYES_999_PCT` %” + fi Yes, yes, I know, had I run the script a minute later, no error. But if I didn’t have OCD tendencies, would I even be on this list? :) -- And she was drifting through the backyard And she was taking off her dress And she was moving very slowly Rising up above the earth
Re: Uptick in spam (bayes stats script)
On Feb 18, 2015, at 6:20 AM, Reindl Harald h.rei...@thelounge.net wrote: bayes-stats.txt That is a lot cleaner and more obvious, thank you for sharing -- Once again I teeter at the precipice of the generation gap.
Re: Uptick in spam (bayes stats script)
Am 17.02.2015 um 15:23 schrieb Reindl Harald: Am 17.02.2015 um 15:19 schrieb LuKreme: On 16 Feb 2015, at 12:01 , Reindl Harald h.rei...@thelounge.net wrote: given that 24266 messages had BAYES_00 with a total number of 30401 delivered mails in the current month that training strategy seems to work well [root@mail-gw:~]$ bayes-stats.sh What is bayes-stats.sh? as simple shell script nicer version attached as plain-text file using now bash + bc + printf for % and formatting removed the su-calls by place it in a worker-dir and call that with su from a script in PATH, well output looks now like below bayes-stats.sh 0.000 0 3 0 non-token data: bayes db version 0.000 0 10606 0 non-token data: nspam 0.000 0 10688 0 non-token data: nham 0.000 01387376 0 non-token data: ntokens 0.000 0 993467899 0 non-token data: oldest atime 0.000 0 1424264407 0 non-token data: newest atime 0.000 0 1424264867 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count insgesamt 35M -rw--- 1 sa-milt sa-milt 2,6M 2015-02-18 14:07 bayes_seen -rw--- 1 sa-milt sa-milt 40M 2015-02-18 14:07 bayes_toks -rw--- 1 sa-milt sa-milt 98 2015-02-17 11:37 user_prefs BAYES_00 28000 75.84 % BAYES_05 4371.18 % BAYES_20 5461.47 % BAYES_40 5971.61 % BAYES_50 4503 12.19 % BAYES_60 4371.18 % BAYES_80 3220.87 % BAYES_95 2240.60 % BAYES_99 18505.01 % BAYES_999 16474.46 % Delivered:34896 SpamAssassin: 3071 #!/usr/bin/bash MAILLOG=/var/log/maillog /usr/bin/sa-learn --dump magic echo /usr/bin/ls -l -h --color=tty -X --group-directories-first --time-style=long-iso /var/lib/spamass-milter/.spamassassin/ echo BAYES_00=`grep -c 'spamd: result:.*BAYES_00,' $MAILLOG` BAYES_05=`grep -c 'spamd: result:.*BAYES_05,' $MAILLOG` BAYES_20=`grep -c 'spamd: result:.*BAYES_20,' $MAILLOG` BAYES_40=`grep -c 'spamd: result:.*BAYES_40,' $MAILLOG` BAYES_50=`grep -c 'spamd: result:.*BAYES_50,' $MAILLOG` BAYES_60=`grep -c 'spamd: result:.*BAYES_60,' $MAILLOG` BAYES_80=`grep -c 'spamd: result:.*BAYES_80,' $MAILLOG` BAYES_95=`grep -c 'spamd: result:.*BAYES_95,' $MAILLOG` BAYES_99=`grep -c 'spamd: result:.*BAYES_99,' $MAILLOG` BAYES_999=`grep -c 'spamd: result:.*BAYES_999,' $MAILLOG` BAYES_TOTAL=`echo $BAYES_00+$BAYES_05+$BAYES_20+$BAYES_40+$BAYES_50+$BAYES_60+$BAYES_80+$BAYES_95+$BAYES_99 | bc` BAYES_00_PCT=`echo scale=2; ($BAYES_00*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_05_PCT=`echo scale=2; ($BAYES_05*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_20_PCT=`echo scale=2; ($BAYES_20*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_40_PCT=`echo scale=2; ($BAYES_40*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_50_PCT=`echo scale=2; ($BAYES_50*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_60_PCT=`echo scale=2; ($BAYES_60*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_80_PCT=`echo scale=2; ($BAYES_80*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_95_PCT=`echo scale=2; ($BAYES_95*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_99_PCT=`echo scale=2; ($BAYES_99*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` BAYES_999_PCT=`echo scale=2; ($BAYES_999*100)/$BAYES_TOTAL | bc | sed 's/^\./0./'` echo -e BAYES_00 `printf \%*s\ 8 $BAYES_00` `printf \%*s\ 7 $BAYES_00_PCT` % echo -e BAYES_05 `printf \%*s\ 8 $BAYES_05` `printf \%*s\ 7 $BAYES_05_PCT` % echo -e BAYES_20 `printf \%*s\ 8 $BAYES_20` `printf \%*s\ 7 $BAYES_20_PCT` % echo -e BAYES_40 `printf \%*s\ 8 $BAYES_40` `printf \%*s\ 7 $BAYES_40_PCT` % echo -e BAYES_50 `printf \%*s\ 8 $BAYES_50` `printf \%*s\ 7 $BAYES_50_PCT` % echo -e BAYES_60 `printf \%*s\ 8 $BAYES_60` `printf \%*s\ 7 $BAYES_60_PCT` % echo -e BAYES_80 `printf \%*s\ 8 $BAYES_80` `printf \%*s\ 7 $BAYES_80_PCT` % echo -e BAYES_95 `printf \%*s\ 8 $BAYES_95` `printf \%*s\ 7 $BAYES_95_PCT` % echo -e BAYES_99 `printf \%*s\ 8 $BAYES_99` `printf \%*s\ 7 $BAYES_99_PCT` % echo -e BAYES_999 `printf \%*s\ 8 $BAYES_999` `printf \%*s\ 7 $BAYES_999_PCT` % echo echo Delivered:`grep -c 'relay=.*status=sent' $MAILLOG` echo SpamAssassin: `grep -c 'Blocked by SpamAssassin' $MAILLOG` signature.asc Description: OpenPGP digital signature
Re: Uptick in spam
Am 17.02.2015 um 15:19 schrieb LuKreme: On 16 Feb 2015, at 12:01 , Reindl Harald h.rei...@thelounge.net wrote: given that 24266 messages had BAYES_00 with a total number of 30401 delivered mails in the current month that training strategy seems to work well [root@mail-gw:~]$ bayes-stats.sh What is bayes-stats.sh? as simple shell script #!/usr/bin/dash su -c /usr/bin/sa-learn --dump magic sa-milt echo su -c /usr/bin/ls -l -h --color=tty -X --group-directories-first --time-style=long-iso /var/lib/spamass-milter/.spamassassin/ sa-milt echo su -c echo \BAYES_00: `grep 'BAYES_00,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_05: `grep 'BAYES_05,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_20: `grep 'BAYES_20,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_40: `grep 'BAYES_40,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_50: `grep 'BAYES_50,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_60: `grep 'BAYES_60,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_80: `grep 'BAYES_80,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_95: `grep 'BAYES_95,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_99: `grep 'BAYES_99,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron su -c echo \BAYES_999: `grep 'BAYES_999,' /var/log/maillog | grep -c 'spamd: result:'`\ wwwcron echo su -c echo \Delivered: `grep 'relay=' /var/log/maillog | grep -c 'status=sent'`\ wwwcron su -c echo \SpamAssassin: `grep -c 'Blocked by SpamAssassin' /var/log/maillog`\ wwwcron signature.asc Description: OpenPGP digital signature
Re: Uptick in spam
On 16 Feb 2015, at 12:01 , Reindl Harald h.rei...@thelounge.net wrote: given that 24266 messages had BAYES_00 with a total number of 30401 delivered mails in the current month that training strategy seems to work well [root@mail-gw:~]$ bayes-stats.sh What is bayes-stats.sh? -- I have a cunning plan.
Re: Uptick in spam
On 2/16/2015 1:33 PM, Amir Caspi wrote: Over the last week I've seen a significant uptick in FN spam to my users. We're getting tens of FNs per day per user, whereas a few weeks ago it was just a few FNs per day per user. We're getting BAYES_99/999 on many of these, but no other major markers are hitting (razor, pyzor, dcc, etc.), so maybe we're just the unlucky early recipients of most of these spam runs... I've been very lax in updating my AC_SPAMMY_URI_PATTERNS rules, but I'll try to get back to that ASAP. Unfortunately many of these FNs are hard to write URI templates against, so not sure that will help much... Just wondering if others are seeing the same thing, and if there are any obvious updates I've been missing to try and combat them. (I know masscheck has been starved lately for rule generation, so maybe that's why...) I'm happy to look at a recent sample and throw it through my system to see what it hits but overall, I've been seeing the exact opposite. (BTW, I am happy to contribute my spam corpus of well over 7000 messages... right now I can't dedicate CPU time to running masscheck, but I can contribute the messages.) FYI that Masscheck has had a hiccup or two but we have been generating rules after about a week of downtime early this month.
Re: Uptick in spam
On Mon, 16 Feb 2015 12:47:03 -0700 Amir Caspi wrote: Otherwise, I don't really know... it's clearly not a Bayes issue since it's hitting Bayes 99/999, it's just that there aren't enough other rules being hit to go over the 5.0 threshold. IIWY I'd look into rescoring the BAYES_* rules.
Re: Uptick in spam
On Feb 16, 2015, at 1:01 PM, RW rwmailli...@googlemail.com wrote: IIWY I'd look into rescoring the BAYES_* rules. I was already rescoring them as BAYES_99 = 4.0, BAYES_999 = 0.5 ... so a total score of 4.5 if both rules hit. These FNs typically get scores of 4.6, so the other rules are simply not good enough. Since I've basically never seen an FP caused by BAYES_99/999 unless I accidentally mistrained it (e.g., sent a newsletter through it by accident), this morning I increased the scoring for BAYES_99 to 4.4, leaving BAYES_999 as 0.5... so now hitting both of them will yield a score of 4.9, and the remaining 0.1 from the other miscellaneous tests should push it over to 5.0. (I didn't want to make BAYES_999 a complete poison pill... though it's darn close.) I guess we'll see if that's enough, though now I'll have to monitor more carefully for FPs for a while... Cheers. --- Amir
Re: Uptick in spam
Am 16.02.2015 um 21:10 schrieb Amir Caspi: On Feb 16, 2015, at 1:01 PM, RW rwmailli...@googlemail.com wrote: IIWY I'd look into rescoring the BAYES_* rules. I was already rescoring them as BAYES_99 = 4.0, BAYES_999 = 0.5 ... so a total score of 4.5 if both rules hit. These FNs typically get scores of 4.6, so the other rules are simply not good enough. Since I've basically never seen an FP caused by BAYES_99/999 unless I accidentally mistrained it (e.g., sent a newsletter through it by accident), this morning I increased the scoring for BAYES_99 to 4.4, leaving BAYES_999 as 0.5... so now hitting both of them will yield a score of 4.9, and the remaining 0.1 from the other miscellaneous tests should push it over to 5.0. (I didn't want to make BAYES_999 a complete poison pill... though it's darn close.) I guess we'll see if that's enough, though now I'll have to monitor more carefully for FPs for a while... we use the scores below by a milter reject of 8.0 i can't rememeber a single FP over 6 months caused by bayes most likely because the large ham-corpus combined by a lot of DNSWL score BAYES_00 -3.5 score BAYES_05 -1.5 score BAYES_20 -0.5 score BAYES_40 -0.1 score BAYES_50 2.0 score BAYES_60 3.5 score BAYES_80 5.0 score BAYES_95 6.5 score BAYES_99 7.5 score BAYES_999 0.4 signature.asc Description: OpenPGP digital signature
Re: Uptick in spam
On Mon, 16 Feb 2015, Amir Caspi wrote: (BTW, I am happy to contribute my spam corpus of well over 7000 messages... right now I can't dedicate CPU time to running masscheck, but I can contribute the messages.) It's possible to upload your corpora and have the central system check it. See the wiki. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- News flash: Lowest Common Denominator down 50 points --- 6 days until George Washington's 283rd Birthday
Uptick in spam
Hi all, Over the last week I've seen a significant uptick in FN spam to my users. We're getting tens of FNs per day per user, whereas a few weeks ago it was just a few FNs per day per user. We're getting BAYES_99/999 on many of these, but no other major markers are hitting (razor, pyzor, dcc, etc.), so maybe we're just the unlucky early recipients of most of these spam runs... I've been very lax in updating my AC_SPAMMY_URI_PATTERNS rules, but I'll try to get back to that ASAP. Unfortunately many of these FNs are hard to write URI templates against, so not sure that will help much... Just wondering if others are seeing the same thing, and if there are any obvious updates I've been missing to try and combat them. (I know masscheck has been starved lately for rule generation, so maybe that's why...) (BTW, I am happy to contribute my spam corpus of well over 7000 messages... right now I can't dedicate CPU time to running masscheck, but I can contribute the messages.) Cheers. --- Amir
Re: Uptick in spam
On Feb 16, 2015, at 11:47 AM, Kevin A. McGrail kmcgr...@pccc.com wrote: I'm happy to look at a recent sample and throw it through my system to see what it hits but overall, I've been seeing the exact opposite. Hmmm. Well, like I said, maybe we're just first on the list and are getting all the spam before it hits the hash DBs (pyzor, etc.). My domain does start with a numeral so if they're sorting alphanumerically that could explain being first on the list. Otherwise, I don't really know... it's clearly not a Bayes issue since it's hitting Bayes 99/999, it's just that there aren't enough other rules being hit to go over the 5.0 threshold. I'll probably send a few spamples off-list. Thanks. --- Amir
Re: Uptick in spam
Am 16.02.2015 um 19:33 schrieb Amir Caspi: Over the last week I've seen a significant uptick in FN spam to my users. We're getting tens of FNs per day per user, whereas a few weeks ago it was just a few FNs per day per user. We're getting BAYES_99/999 on many of these, but no other major markers are hitting (razor, pyzor, dcc, etc.), so maybe we're just the unlucky early recipients of most of these spam runs... unconfirmed here we maintain a global bayes for spamass-milter and i am doing a hard job in my free-time to collect as much as possible newsletters, registration confirmations, hotel reservations and what not else from some helpful users as well as feed nearly 100% of my personal mail as ham some of that original messages containing even passwords which i strip out if someone trusting me that enough to provide these samples which is easy since we train from two folders with eml files given that 24266 messages had BAYES_00 with a total number of 30401 delivered mails in the current month that training strategy seems to work well [root@mail-gw:~]$ bayes-stats.sh 0.000 0 3 0 non-token data: bayes db version 0.000 0 10470 0 non-token data: nspam 0.000 0 10554 0 non-token data: nham 0.000 01368369 0 non-token data: ntokens 0.000 0 993467899 0 non-token data: oldest atime 0.000 0 1424109150 0 non-token data: newest atime 0.000 0 1424110103 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count insgesamt 35M -rw--- 1 sa-milt sa-milt 2,6M 2015-02-16 19:08 bayes_seen -rw--- 1 sa-milt sa-milt 41M 2015-02-16 19:08 bayes_toks -rw--- 1 sa-milt sa-milt 98 2014-08-21 17:47 user_prefs BAYES_00: 24266 BAYES_05: 396 BAYES_20: 485 BAYES_40: 525 BAYES_50: 3999 BAYES_60: 400 BAYES_80: 293 BAYES_95: 210 BAYES_99: 1629 BAYES_999: 1439 Delivered: 30401 SpamAssassin: 2715 signature.asc Description: OpenPGP digital signature
