Uptick in false negatives - filter check?
Posted this to the wrong/no list (via Nabble) yesterday... I've seen an uptick in false negatives lately, and the spam that is getting through is all the same stuff repeatedly. If anyone would be willing to run these samples through their filters and let me know if they get better hits, I would appreciate it. There are three at http://nerdnetworks.org/spam/ I'm using SA 3.3.1, with Bayes, etc. I also have greylisting on my system with a 15 minute delay, and surprisingly the first sample in this group now hits a bunch of RBLs and scores 5, but apparently the 15 minute delay wasn't enough time for that to help me. I've also been training my Bayes DB on these types of messages for a few days, but they still keep getting through. I used to hear that if your Bayes DB gets too big it can become ineffective. I don't know if that's true or not, but here's my '--dump magic' output: 0.000 0 3 0 non-token data: bayes db version 0.000 0 62157 0 non-token data: nspam 0.000 0 176680 0 non-token data: nham 0.000 0 144331 0 non-token data: ntokens 0.000 0 1383022790 0 non-token data: oldest atime 0.000 0 1383770853 0 non-token data: newest atime 0.000 0 1383766433 0 non-token data: last journal sync atime 0.000 0 1383685115 0 non-token data: last expiry atime 0.000 0 662551 0 non-token data: last expire atime delta 0.000 0 19902 0 non-token data: last expire reduction count Looking at my spamd log, out of 1300 messages classified as spam, 566 hit BAYES_9* and 391 hit BAYES_5*. Thanks in advance for any advice anyone can offer! -- View this message in context: http://spamassassin.1065346.n5.nabble.com/Uptick-in-false-negatives-filter-check-tp107090.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Uptick in false negatives - filter check?
Thanks for your response! My server is in EC2, and it appears that URIBL blanketly refuses requests from there. I set up a caching DNS server locally and tried routing my request through that, it was still rejected. Too many spammers using EC2 I guess. As for your other suggestion, isn't that the point of Bayesian filtering? I keep getting similar messages, training my bayes db on them, and then more get through. Kris Deugau [via SpamAssassin] ml-node+s1065346n107092...@n5.nabble.com wrote: Owen Mehegan wrote: Posted this to the wrong/no list (via Nabble) yesterday... I've seen an uptick in false negatives lately, and the spam that is getting through is all the same stuff repeatedly. If anyone would be willing to run these samples through their filters and let me know if they get better hits, I would appreciate it. There are three at http://nerdnetworks.org/spam/ (spam4.txt is inaccessible) I notice URIBL_BLOCKED hits; check that you're either using your own resolver with less than 100K messages/day, or that you're properly set up for datafeed. Or just disable the uribl.com rules. (We found that while they were usefully increasing our overall catch rate, the increase was not worth the cost of the datafeed [it came out to somewhere between one and five dollars a spam for the ones that the uribl.com hit was key in getting the message tagged], so we disabled the rules.) Beyond that I've started creating very simple rules targeting the Subject and From: name in this type of spam, along with extracting the relay IP and URIs for local DNSBLs. It's moderately effective once I've confirmed enough volume for any given Subject or name to feel it's worth creating a rule... -kgd ___ If you reply to this email, your message will be added to the discussion below: http://spamassassin.1065346.n5.nabble.com/Uptick-in-false-negatives-filter-check-tp107090p107092.html To unsubscribe from Uptick in false negatives - filter check?, visit http://spamassassin.1065346.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=107090code=b3dlbkBuZXJkbmV0d29ya3Mub3JnfDEwNzA5MHwyMDgxOTQ3Njg5 -- Sent from Kaiten Mail. Please excuse my brevity. -- View this message in context: http://spamassassin.1065346.n5.nabble.com/Uptick-in-false-negatives-filter-check-tp107090p107096.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Uptick in false negatives - filter check?
Oh, and I fixed spam4.txt to be accessible, sorry about that. -- Sent from Kaiten Mail. Please excuse my brevity. -- View this message in context: http://spamassassin.1065346.n5.nabble.com/Uptick-in-false-negatives-filter-check-tp107090p107097.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Pharma spam getting through again
On Thu, Aug 07, 2008 at 01:51:00PM -0700, Owen B. Mehegan wrote: Uh, whoops. Apparently I deleted the body of the message before I sent it. Sorry... I was asking for help figuring out why messages like the one I attached are getting through my SA setup. I'm using SA 3.2.1 with spamd, through Postfix, on Linux. This message scores as follows on my system: 2.0 FREEMAIL_FROM From-address is freemail domain 0.0 BOTNET_SERVERWORDS Hostname contains server-like substrings [botnet_serverwords,ip=98.136.45.12,rdns=n65a.bullet.mail.sp1.yahoo.com] -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% [score: 0.] It also hits on the Spamcop URIBL at the moment, but it didn't when I first received it. That's a common theme now - message comes in and doesn't appear in any blacklists. I test it sometime later and it does, often scoring high enough then to be filtered. All these messages are coming from freemail providers. I'd like to delay mail from them for an hour or something, to give these messages time to get into the blacklists, but I haven't figured out a way to do that yet. I've been getting lots of these for the last 2-3 weeks. The freemail filter wasn't enough to stop them, especially for the ones that score 0% in bayes. Maybe it's time to wipe out my bayes DB and start over? Or is there some other filter I should add? It's driving me crazy! -- Owen B. Mehegan ([EMAIL PROTECTED]) Here are two more that got through today. Even several hours later, these haven't shown up in blacklists. Do anyone else's rules catch these? -- Owen B. Mehegan ([EMAIL PROTECTED]) He is a dangerous mixture of sophistication and recklessness which makes one anxious about his influence on other boys. From [EMAIL PROTECTED] Tue Aug 12 06:37:59 2008 Return-Path: [EMAIL PROTECTED] X-Spam-Score: 2.0 X-Spam-Checker-Version: SpamAssassin 3.2.1 (2007-05-02) on sandman.nerdnetworks.org X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=5.0 tests=BAYES_00,FREEMAIL_FROM, HTML_MESSAGE autolearn=no version=3.2.1 X-Original-To: [EMAIL PROTECTED] Delivered-To: [EMAIL PROTECTED] X-Greylist: delayed 350 seconds by postgrey-1.27 at sandman; Tue, 12 Aug 2008 06:37:55 PDT Received-SPF: none (sandman.nerdnetworks.org: domain of [EMAIL PROTECTED] does not designate permitted sender hosts) Received: from n10.bullet.re3.yahoo.com (n10.bullet.re3.yahoo.com [68.142.237.123]) by nerdnetworks.org (Postfix) with SMTP id 607B1F80F1 for [EMAIL PROTECTED]; Tue, 12 Aug 2008 06:37:55 -0700 (PDT) Received: from [68.142.237.87] by n10.bullet.re3.yahoo.com with NNFMP; 12 Aug 2008 13:32:04 - Received: from [66.196.97.136] by t3.bullet.re3.yahoo.com with NNFMP; 12 Aug 2008 13:32:04 - Received: from [127.0.0.1] by omp109.mail.re3.yahoo.com with NNFMP; 12 Aug 2008 13:32:04 - X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: [EMAIL PROTECTED] Received: (qmail 18518 invoked by uid 60001); 12 Aug 2008 13:32:04 - DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=2jBZKWMhmVNw2+4y50JO43d/WupGO/YBAc8AGDpi5NLC4XPTjMLPJjA2xbbWESRE6qj1dJSJvIeC1yXb7mwojRrnTB3PObfF5F1zK9YeFkOaR+xzJJY77iNB5gXllibLjQCvjFItxIveHooY3TLbYHY1jrmAtsJ71FQ13tk8wEk=; X-YMail-OSG: wpMId9IVM1kffDA7k9FVnHIfHolwaq__Sfj0Z7KqCOqbzwBCQgwzOZMy0DJeKwynpg4z0UVzNQvDArjadsSBRhkMX7ts3J2uCrv7B6COMXRypLeZ99d_KEfQPfgFM88- Received: from [189.54.146.81] by web57414.mail.re1.yahoo.com via HTTP; Tue, 12 Aug 2008 06:32:03 PDT Date: Tue, 12 Aug 2008 06:32:03 -0700 (PDT) From: Chasity Ripply [EMAIL PROTECTED] Subject: Simple order top quality generic pills To: [EMAIL PROTECTED] MIME-Version: 1.0 Content-Type: multipart/alternative; boundary=0-2084603575-1218547923=:14464 Content-Transfer-Encoding: 8bit Message-ID: [EMAIL PROTECTED] Status: RO Content-Length: 536 Lines: 20 --0-2084603575-1218547923=:14464 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit http://groups.google.com/group/as2j9ji1z/web/8sxf0 We must take the risk. --0-2084603575-1218547923=:14464 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Pa href=http://groups.google.com/group/as2j9ji1z/web/8sxf0;http://groups.google.com/group/as2j9ji1z/web/8sxf0/a br br We must take the risk. br/Pp#32; --0-2084603575-1218547923=:14464-- From [EMAIL PROTECTED] Tue Aug 12 00:46:50 2008 Return-Path: [EMAIL PROTECTED] X-Spam-Score: 4.7 X-Spam-Checker-Version: SpamAssassin 3.2.1 (2007-05-02) on sandman.nerdnetworks.org X-Spam-Level: X-Spam-Status: No, score=4.7 required=5.0 tests=BAYES_00,BOTNET_SERVERWORDS, FREEMAIL_FROM,GEO_QUERY_STRING,HTML_MESSAGE autolearn=no version=3.2.1 X-Original-To: [EMAIL PROTECTED] Delivered-To: [EMAIL PROTECTED] X-Greylist:
Pharma spam getting through again
From [EMAIL PROTECTED] Thu Aug 7 10:08:49 2008 Return-Path: [EMAIL PROTECTED] X-Spam-Score: -2.1 X-Spam-Checker-Version: SpamAssassin 3.2.1 (2007-05-02) on sandman.nerdnetworks.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,BOTNET_SERVERWORDS, FREEMAIL_FROM autolearn=no version=3.2.1 X-Original-To: [EMAIL PROTECTED] Delivered-To: [EMAIL PROTECTED] X-Greylist: delayed 314 seconds by postgrey-1.27 at sandman; Thu, 07 Aug 2008 10:08:48 PDT Received-SPF: none (sandman.nerdnetworks.org: domain of [EMAIL PROTECTED] does not designate permitted sender hosts) Received: from n65a.bullet.mail.sp1.yahoo.com (n65a.bullet.mail.sp1.yahoo.com [98.136.45.12]) by nerdnetworks.org (Postfix) with SMTP id A5AB75C46E for [EMAIL PROTECTED]; Thu, 7 Aug 2008 10:08:48 -0700 (PDT) Received: from [216.252.122.217] by n65.bullet.mail.sp1.yahoo.com with NNFMP; 07 Aug 2008 17:02:25 - Received: from [69.147.65.174] by t2.bullet.sp1.yahoo.com with NNFMP; 07 Aug 2008 17:02:25 - Received: from [127.0.0.1] by omp509.mail.sp1.yahoo.com with NNFMP; 07 Aug 2008 17:02:25 - X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: [EMAIL PROTECTED] Received: (qmail 9635 invoked by uid 60001); 7 Aug 2008 17:02:24 - DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=vytF/wijHSje9a0ED6diXsHBU2x6WUElDcpmnoK6L1Vg6Os0zknCDnr8Hj1iU22EZ8q4lKWrc26ET2trYjIdHmF6Kv8GQOctDUyciqb6WkLnHpCjX6Rclyzg/S1uu7BxpGXZuMX27+wl34C8a9HfndcxhZtOUhcBMpvx82sSwXs=; Received: from [67.186.139.245] by web45812.mail.sp1.yahoo.com via HTTP; Thu, 07 Aug 2008 10:02:24 PDT Date: Thu, 7 Aug 2008 10:02:24 -0700 (PDT) From: Allen Mercado [EMAIL PROTECTED] Subject: Drugs are just drugs, but they help people To: [EMAIL PROTECTED] MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: [EMAIL PROTECTED] Status: RO Content-Length: 133 Lines: 7 a href=http://journals.aol.co.uk/hanyinghlolar/91175/;http://journals.aol.co.uk/hanyinghlolar/91175//a br br br
Stock/image-only spam still getting through
First, the prerequisites: SpamAssassin version 3.1.1, running on Perl version 5.8.4 Debian Linux, 2.6.10 kernel Using spamd I've been inundated with maddening image-only stock spam lately. I've just today sat down to try and tweak my rules up to weed this out. I added sare_stocks and sare_obfu, updated my version of rules du jour for good measure, and restarted spamd. I tested these changes on an example message, and neither of those new rule sets hit on it at all. A few minutes later, ANOTHER of these messages came through! Argh! And I just realized, looking at its headers, these messages are getting through my greylisting too! Clever bastards. I've attached the one that just got through. spamassassin -t reports the following for it: 0.8 EXTRA_MPART_TYPE Header has extraneous Content-type:...type= entry 2.9 HELO_DYNAMIC_SPLIT_IP Relay HELO'd using suspicious hostname (Split IP) 1.3 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO 0.0 HTML_MESSAGE BODY: HTML included in message 0.8 SARE_GIF_ATTACHFULL: Email has a inline gif The highest scores are for the HELO? We've got to be able to do better than that... what am I missing? -- Owen B. Mehegan ([EMAIL PROTECTED]) Cell: 617-230-3679 From [EMAIL PROTECTED] Mon Jul 17 17:44:57 2006 Return-Path: [EMAIL PROTECTED] X-Spam-Score: 5.7 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on morphine X-Spam-Level: * X-Spam-Status: No, score=5.7 required=6.0 tests=EXTRA_MPART_TYPE, HELO_DYNAMIC_SPLIT_IP,HTML_MESSAGE,RCVD_NUMERIC_HELO,SARE_GIF_ATTACH autolearn=no version=3.1.1 X-Original-To: [EMAIL PROTECTED] Delivered-To: [EMAIL PROTECTED] X-Greylist: delayed 308 seconds by postgrey-1.21 at morphine; Mon, 17 Jul 2006 17:44:40 EDT Received: from 152.10.134.67.gvni.com (unknown [67.134.10.152]) by nerdnetworks.org (Postfix) with SMTP id 12415DFBC9 for [EMAIL PROTECTED]; Mon, 17 Jul 2006 17:44:39 -0400 (EDT) Received: from mbglci.hy ([67.134.198.90]) by 152.10.134.67.gvni.com (8.13.4/8.13.4) with SMTP id k6HLj7Yb072828; Mon, 17 Jul 2006 14:45:07 -0700 Message-ID: [EMAIL PROTECTED] From: Neil Weaver [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: fifth legal Date: Mon, 17 Jul 2006 14:34:41 -0700 MIME-Version: 1.0 Content-Type: multipart/related; type=multipart/alternative; boundary==_NextPart_000_0016_01C6A9AF.87964159 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Status: RO Content-Length: 49978 Lines: 783 This is a multi-part message in MIME format. --=_NextPart_000_0016_01C6A9AF.87964159 Content-Type: multipart/alternative; boundary==_NextPart_001_0017_01C6A9AF.8796416C --=_NextPart_001_0017_01C6A9AF.8796416C Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable whaler the is home economics and untie boogie the attainable to as unattractive lobster, cabinet nationalist quaver, epileptic the = redundant cheerfully, theme cut shooting an?! cornet, them the = preparatory nimbly? brawn, proportions that paralytic wishful thinking = on?! eggplant, distasteful latent secrete an unbelievable conjugation = sitter pry N amplifier banality of! mourning to outlandish confidentially cultural, = operating room,... shush to coffee table are refresher course repugnant desecrate menorah update us by... eye-opener it philanthropist a of?! = excessive as airplane layaway whine toboggan objection this an transsexual in sharp great-granddaughter normality = eminently apparel modernization olive oil! tarot, revolt, this = marketplace crush normalize street intensive care?! season ticket but an = intently, an quantify whipping to arched to an key outbreak, cremate = colleague and dwarves decentralize crib the frustrating hoop petal of annoyed competently: as open disastrous are = witch a two-tone in Chicano? jockey royalties starvation a IV it constriction the by euphemism unexpectedly of forty official, = overalls the furry, a the as postscript of but an great stereo was = heartbeat remedy an letdown dispassionately the of ax the steady twelve, = and Sagittarius cheesecloth, transparency misc. footstep. is on an = eatery left menacing of artwork as it sibling amid,... transformation the negligee debunk = downplay raven, hurl refugee as big deal in nuance nuclear, personality = implement isthmus bitch, donation a to cutting edge polygamist = systematically constrain as son-in-law to upside down obscure minus sign untold, interesting, to decrease shamelessly: checklist these, directive, = mouthpiece volleyball the to as motor invigorating, gospel music a the = this resolution undergrad trap door. fluently Aug. absurdity palatable! newsworthy world power test ban, = sternly Sr. in pickax to of forbore, distinctive reinforcements =
Filter check request
I'm running SA 3.0.2 with Postfix and a few SARE custom rule sets (with weekly rules du jour updates). This has been working amazingly well for over a year, but lately a few things have been getting through, and with unusually low scores. I'm attaching two here - if anyone would be willing to run these through their rules and see how they score, it would be much appreciated. Perhaps I just need some new rule sets, or just to upgrade to the latest SA.spam1 results:Content analysis details: (3.6 points, 6.0 required) pts rule name description -- -- 0.6 J_CHICKENPOX_23 BODY: 2alpha-pock-3alpha 1.4 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/) 1.7 RCVD_IN_NJABL_DUL RBL: NJABL: dialup sender did non-local SMTP [88.155.196.23 listed in combined.njabl.org]spam2 results:Content analysis details: (-1.5 points, 6.0 required) pts rule name description -- -- 0.2 EXTRA_MPART_TYPE Header has extraneous Content-type:...type= entry-2.8 ALL_TRUSTED Did not pass through any untrusted hosts 0.1 HTML_40_50 BODY: Message is 40% to 50% HTML 1.0 HTML_IMAGE_ONLY_24 BODY: HTML: images with 2000-2400 bytes of words 0.0 HTML_MESSAGE BODY: HTML included in messageThanks in advance :-) spam1 Description: Binary data spam2 Description: Binary data -- [EMAIL PROTECTED] (Owen B. Mehegan) 'Something is rotten in the state of Denmark.' --William Shakespeare
Bayes learning email address
To make it easier for my users to train my server's Bayes database, I set up a user with the following procmail recipe in its .procmailrc::0* 256000 { :0c: spamassassin.spamlock | sa-learn --spam :0: spamassassin.filelock spam }The idea is for people to redirect (not forward) uncaught spam to that address and have it added to our Bayes system. I suppose I could also --report those messages to the various reporting systems. Will this work, or are there pitfalls I haven't thought of? -- [EMAIL PROTECTED] (Owen B. Mehegan) 'I learned this, at least, by my experiment; that if one advances confidently in the direction of his dreams, and endeavors to live the life which he has imagined, he will meet with a success unexpected in common hours.' --Henry David Thoreau
Re: Filter check request
I've upgraded to SA 3.1.1 and now both messages hit solidly as spam. I also don't see the ALL_TRUSTED mistake, so I'm guessing that was caused by the trust code mismatch you mentioned. Thanks! -- [EMAIL PROTECTED] (Owen B. Mehegan) 'There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.' --William Shakespeare On Apr 14, 2006, at 11:54 AM, Matt Kettler wrote:Owen Mehegan wrote: I'm running SA 3.0.2 with Postfix and a few SARE custom rule sets (withweekly rules du jour updates). FWIW, SA 3.0.2 is vulnerable to multiple DoS attacks. Unless you're using adistro port which has backported fixes, I'd strongly suggest an upgrade.At this time the only practical versions of official SA with no massive bugs orsecurity holes are: 3.0.5, 3.1.0 and 3.1.1.AFAIK 2.64 is also safe from security holes, but it's too old to be practical. This has been working amazingly well forover a year, but lately a few things have been getting through, and withunusually low scores. I'm attaching two here - if anyone would bewilling to run these through their rules and see how they score, itwould be much appreciated. Perhaps I just need some new rule sets, orjust to upgrade to the latest SA. snip -2.8 ALL_TRUSTED Did not pass through any untrusted hosts ALL_TRUSTED should *never* match outside email. Looks like your trusted_networksneeds to be set manually.seehttp://wiki.apache.org/spamassassin/TrustPathThe other alternative is it is mismatching due to a bug in the trust code thatis fixed in 3.0.5 and higher.
