Re: Up tick in missed SPAM from co domain
> On 2022‑02‑03 16:50, joea‑ lists wrote: >> SA version 3.4.5 > > old version, stable is 3.4.6 now Unless there is a pressing reason to update right away, I prefer to wait for the vendor supplied package to update. But that is not a hard rule for me. >> Since yesterday 2/2/22 (gasp!) . . . I've noticed an up tick in missed >> SPAM from .co domain. Though obvious SPAM >> weight loss, phish, "personals", they are scoring rather low. > > spammer use spamassassin self to make there spam pass spamassassin > >> Added a custom rule for that domain, which should deal with it, but >> wondering if I missed some changes that >> might cause this? > > raise scores on tag that are detected "score foo (1) (1) (1) (1)" > dynamic score adjust Not familiar with dynamic score, guess my reading list just got longer. > change 1 as you wish > > also negative score ‑1 is supported > > dont use static score adjust :=) > > i am not a perl freak, lol Me neither. Time to read past the intro on that book with the Camel on the cover. > idealy we would all make corpus scooring, but i dont have so many mails > yet for this to be stable Thanks. joe a.
Re: Up tick in missed SPAM from co domain
On 2022-02-03 16:50, joea- lists wrote: SA version 3.4.5 old version, stable is 3.4.6 now Since yesterday 2/2/22 (gasp!) . . . I've noticed an up tick in missed SPAM from .co domain. Though obvious SPAM weight loss, phish, "personals", they are scoring rather low. spammer use spamassassin self to make there spam pass spamassassin Added a custom rule for that domain, which should deal with it, but wondering if I missed some changes that might cause this? raise scores on tag that are detected "score foo (1) (1) (1) (1)" dynamic score adjust change 1 as you wish also negative score -1 is supported dont use static score adjust :=) i am not a perl freak, lol idealy we would all make corpus scooring, but i dont have so many mails yet for this to be stable
Re: Up tick in missed SPAM from co domain
>> On Thu, 2022‑02‑03 at 10:50 ‑0500, joea‑ lists wrote: SA version 3.4.5 >>> >>> Since yesterday 2/2/22 (gasp!) . . . I've noticed an up tick in missed >>> SPAM from .co domain. Though obvious SPAM >>> weight loss, phish, "personals", they are scoring rather low. >>> >>> Added a custom rule for that domain, which should deal with it, but >>> wondering if I missed some changes that >>> might cause this? >>> >> IMO that's too specific: it will deal with spam from that address, but >> each new address needs its own rule. I only use that type of rule to >> ding endless sales messages from companies that I bought one item from >> and who are unlikely to ever sell me anything else. >> >> IMO its worth scanning though spam looking for odd phrases or spellings >> and making rules to add points for these features. Done carefully, you >> can end up with rules that trap that type of spam no matter where it >> comes from, i.e. pron, "girls looking for men", banking scams, etc. >> >> Martin >> > > Yes, it is painting with a rather broad brush and there are several other > domain specific rules. Each was done "just for now". > > Time to follow your suggestion, but, kind of like laying off from the gym > for a few weeks, then trying to get started again. > > joe a. Found a rule that was hit on all of these, but has scored at 0.0. Added it to local.cf with a score to put it just at 5.0 and commented out my domain specific rule. We'll see how it goes. joe a.
Re: Up tick in missed SPAM from co domain
> On Thu, 2022-02-03 at 10:50 -0500, joea- lists wrote: >> SA version 3.4.5 >> >> Since yesterday 2/2/22 (gasp!) . . . I've noticed an up tick in missed >> SPAM from .co domain. Though obvious SPAM >> weight loss, phish, "personals", they are scoring rather low. >> >> Added a custom rule for that domain, which should deal with it, but >> wondering if I missed some changes that >> might cause this? >> > IMO that's too specific: it will deal with spam from that address, but > each new address needs its own rule. I only use that type of rule to > ding endless sales messages from companies that I bought one item from > and who are unlikely to ever sell me anything else. > > IMO its worth scanning though spam looking for odd phrases or spellings > and making rules to add points for these features. Done carefully, you > can end up with rules that trap that type of spam no matter where it > comes from, i.e. pron, "girls looking for men", banking scams, etc. > > Martin > Yes, it is painting with a rather broad brush and there are several other domain specific rules. Each was done "just for now". Time to follow your suggestion, but, kind of like laying off from the gym for a few weeks, then trying to get started again. joe a.
Re: Up tick in missed SPAM from co domain
On Thu, 2022-02-03 at 10:50 -0500, joea- lists wrote: > SA version 3.4.5 > > Since yesterday 2/2/22 (gasp!) . . . I've noticed an up tick in missed > SPAM from .co domain. Though obvious SPAM > weight loss, phish, "personals", they are scoring rather low. > > Added a custom rule for that domain, which should deal with it, but > wondering if I missed some changes that > might cause this? > IMO that's too specific: it will deal with spam from that address, but each new address needs its own rule. I only use that type of rule to ding endless sales messages from companies that I bought one item from and who are unlikely to ever sell me anything else. IMO its worth scanning though spam looking for odd phrases or spellings and making rules to add points for these features. Done carefully, you can end up with rules that trap that type of spam no matter where it comes from, i.e. pron, "girls looking for men", banking scams, etc. Martin > joe a. >
Up tick in missed SPAM from co domain
SA version 3.4.5 Since yesterday 2/2/22 (gasp!) . . . I've noticed an up tick in missed SPAM from .co domain. Though obvious SPAM weight loss, phish, "personals", they are scoring rather low. Added a custom rule for that domain, which should deal with it, but wondering if I missed some changes that might cause this? joe a.
Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB
On Tue, 30 May 2017, Robert Kudyba wrote: I note that message hit BAYES_00. If content like that is getting a "strong ham" Bayes score, you should review your training processes and Bayes corpora - you *do* keep copies of messages you train Bayes with, right? :) Yes just re-synced. Did you do any review before re-training? Re-training with misclassifications in the corpora will not correct the problem. But: fixing your Bayes and getting a non-forwarding DNS server for your mail system so that you're not hitting RBL query limits are the biggest things you need to do to address this. It’s enabled and looks like it’s working based on this and that use_bayes 1 in local.cf sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0688 0 non-token data: nspam 0.000 0 80012 0 non-token data: nham That seems somewhat out-of-balance, and might lead to FNs due to Bayes. You should try to get more spam to train. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- People think they're trading chaos for order [by ceding more and more power to the Government], but they're just trading normal human evil for the really dangerous organized kind of evil, the kind that simply does not give a shit. Only bureaucrats can give you true evil. -- Larry Correia --- 7 days until the 73rd anniversary of D-Day
Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB
> For the past few days lots of missed spam has been getting through, running >>> SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with >>> URIBL_RHS_DOB, i.e., domains registered in the last five days. Since we >>> are not running our own DNS server (yet--need permission from our CISO) >>> URIBL_BLOCKED is also being triggered. Is there a way to update this? > >> Update what how? You answered below…thanks. > >> I note that message hit BAYES_00. If content like that is getting a >> "strong ham" Bayes score, you should review your training processes and >> Bayes corpora - you *do* keep copies of messages you train Bayes with, >> right? :) Yes just re-synced. > If you trust URIBL_RHS_DOB to not hit your ham, you can increase the score >> of URIBL_RHS_DOB in your local rules file. > >> If you'd prefer a more-focused solution, use a meta rule; perhaps: > >>meta LCL_DOB_FROM_INFO __FROM_DOM_INFO && URIBL_RHS_DOB >>score LCL_DOB_FROM_INFO 2.500 # or whatever you're comfortable with Great trying this now. > >> But: fixing your Bayes and getting a non-forwarding DNS server for your >> mail system so that you're not hitting RBL query limits are the biggest >> things you need to do to address this. It’s enabled and looks like it’s working based on this and that use_bayes 1 in local.cf sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0688 0 non-token data: nspam 0.000 0 80012 0 non-token data: nham 0.000 0 164827 0 non-token data: ntokens 0.000 0 1485101489 0 non-token data: oldest atime 0.000 0 1496149547 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 1496152035 0 non-token data: last expiry atime 0.000 0 11059200 0 non-token data: last expire atime delta 0.000 0 99547 0 non-token data: last expire reduction count > >>> I have't seen an update in sa-update since 03-May-2017 01:52:05: > >> Masscheck and updates are *almost* back. Great I’ll keep an eye out. > >>> Here's a typical mail header & message content: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_Rw1S7mWe=DwIFAw=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY=bpKADIzstZa5G-g1qsGBa7gWKq4zTcrA_-E0jGYOsdo=_uJa-KDGfZ2CN8vjSlDNEmfotigbWHyD9TZaKnJwzNM= >>> >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_Rw1S7mWe=DwIFAw=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY=bpKADIzstZa5G-g1qsGBa7gWKq4zTcrA_-E0jGYOsdo=_uJa-KDGfZ2CN8vjSlDNEmfotigbWHyD9TZaKnJwzNM=> >>> > >> Thanks for that. Looks like the IP is being picked up on a few RBLs now. > > Do you have any RBLs setup in sendmail? You need > to use bb.barracudacentral.org <http://bb.barracudacentral.org/> and > zen.spamhaus.org <http://zen.spamhaus.org/> > at a minimum. Hopefully your DNS server situation > can get fixed soon so you can use BLs successfully. > Indeed we do plus spamcop: FEATURE(`dnsbl', `b.barracudacentral.org', `', `"550 Mail from " $&{client_addr} " refused. Rejected for bad WHOIS info on IP of your SMTP server " in http://www.barracudacentral.org/lookups "')dnl FEATURE(`dnsbl',`zen.spamhaus.org')dnl FEATURE(`enhdnsbl', `bl.spamcop.net', `"Spam blocked see: http://spamcop.net/bl.shtml?"$&{client_addr}', `t')dnl > If you switched to Postfix, there are many benefits > to using Postscreen with weighted RBLs. I have over > 20 RBLs working together for best accuracy and low > false positives. We have several mailing lists and users past & present and the transition would be a bit painful. > SpamAssassin is primarily going to be a content filter > with some reputation checks. Setup the MTA to be > primarily reputation checks with DNS (i.e. make sure > the sending IP has a PTR record [RDNS_NONE]) and > RBL lookups. > > The MTA should be blocking the majority of spam > before it gets to SpamAssassin. That’s what I thought, and we have even more filters in place, including the suggestion in https://www.autonarcosis.com/2015/10/14/vanity-top-level-domains-how-to-block-them-using-sendmail/ <https://www.autonarcosis.com/2015/10/14/vanity-top-level-domains-how-to-block-them-using-sendmail/> to use the access file to block all of those vanity top level domains. I even have a regex to block anysubdomain.anydomain.us|info. And we have clamavjunofficial-sigs from extremeshok enabled. Anything else to check?
Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB
>From: John Hardin <jhar...@impsec.org> >On Mon, 29 May 2017, Robert Kudyba wrote: >> For the past few days lots of missed spam has been getting through, running >> SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with >> URIBL_RHS_DOB, i.e., domains registered in the last five days. Since we >> are not running our own DNS server (yet--need permission from our CISO) >> URIBL_BLOCKED is also being triggered. Is there a way to update this? >Update what how? >I note that message hit BAYES_00. If content like that is getting a >"strong ham" Bayes score, you should review your training processes and >Bayes corpora - you *do* keep copies of messages you train Bayes with, >right? :) >If you trust URIBL_RHS_DOB to not hit your ham, you can increase the score >of URIBL_RHS_DOB in your local rules file. >If you'd prefer a more-focused solution, use a meta rule; perhaps: > meta LCL_DOB_FROM_INFO __FROM_DOM_INFO && URIBL_RHS_DOB > score LCL_DOB_FROM_INFO 2.500 # or whatever you're comfortable with >But: fixing your Bayes and getting a non-forwarding DNS server for your >mail system so that you're not hitting RBL query limits are the biggest >things you need to do to address this. >> I have't seen an update in sa-update since 03-May-2017 01:52:05: >Masscheck and updates are *almost* back. >> Here's a typical mail header & message content: >> https://pastebin.com/Rw1S7mWe >Thanks for that. Do you have any RBLs setup in sendmail? You need to use bb.barracudacentral.org and zen.spamhaus.org at a minimum. Hopefully your DNS server situation can get fixed soon so you can use BLs successfully. score.senderscore.com reputation is 0 out of 100 http://multirbl.valli.org/lookup/208.110.91.112.html If you switched to Postfix, there are many benefits to using Postscreen with weighted RBLs. I have over 20 RBLs working together for best accuracy and low false positives. SpamAssassin is primarily going to be a content filter with some reputation checks. Setup the MTA to be primarily reputation checks with DNS (i.e. make sure the sending IP has a PTR record [RDNS_NONE]) and RBL lookups. The MTA should be blocking the majority of spam before it gets to SpamAssassin. Dave
Re: lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB
On Mon, 29 May 2017, Robert Kudyba wrote: For the past few days lots of missed spam has been getting through, running SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with URIBL_RHS_DOB, i.e., domains registered in the last five days. Since we are not running our own DNS server (yet--need permission from our CISO) URIBL_BLOCKED is also being triggered. Is there a way to update this? Update what how? I note that message hit BAYES_00. If content like that is getting a "strong ham" Bayes score, you should review your training processes and Bayes corpora - you *do* keep copies of messages you train Bayes with, right? :) If you trust URIBL_RHS_DOB to not hit your ham, you can increase the score of URIBL_RHS_DOB in your local rules file. If you'd prefer a more-focused solution, use a meta rule; perhaps: meta LCL_DOB_FROM_INFO __FROM_DOM_INFO && URIBL_RHS_DOB score LCL_DOB_FROM_INFO 2.500 # or whatever you're comfortable with But: fixing your Bayes and getting a non-forwarding DNS server for your mail system so that you're not hitting RBL query limits are the biggest things you need to do to address this. I have't seen an update in sa-update since 03-May-2017 01:52:05: Masscheck and updates are *almost* back. Here's a typical mail header & message content: https://pastebin.com/Rw1S7mWe Thanks for that. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- USMC Rules of Gunfighting #2: Anything worth shooting is worth shooting twice. Ammo is cheap. Your life is expensive. --- Today: Memorial Day - honor those who sacrificed for our liberty
lots of missed spam/false negatives from .info TLD being marked with URIBL_RHS_DOB
For the past few days lots of missed spam has been getting through, running SA 3.4.1 on Fedora 25 with sendmail. I see that they are being tagged with URIBL_RHS_DOB, i.e., domains registered in the last five days. Since we are not running our own DNS server (yet--need permission from our CISO) URIBL_BLOCKED is also being triggered. Is there a way to update this? I have't seen an update in sa-update since 03-May-2017 01:52:05: SpamAssassin: Update processed successfully. Here's a typical mail header & message content: https://pastebin.com/Rw1S7mWe
Re: Missed spam, suggestions?
On Fri, 11 Mar 2016, Robert Chalmers wrote: Found a copy here … http://www.impsec.org/~jhardin/antispam/sa-stats.pl Note that I also host a version that works with gzipped log files, if you have compression enabled in your log rotator. But that's not the latest. I don't know where the v1.03 David has came from. David, if you'd care to email me your copy, I'll see about updating the one I host. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- If you ask amateurs to act as front-line security personnel, you shouldn't be surprised when you get amateur security. -- Bruce Schneier --- 84 days since the first successful real return to launch site (SpaceX)
Re: sa-stats log analyzer (RE: Missed spam, suggestions?)
The rulesemporium site appears to be down. If anyone has a newer version, it might be good to post it somewhere? My site for eg? Robert Sent from my iPad > On 11 Mar 2016, at 04:17, David B Funk <dbf...@engineering.uiowa.edu> wrote: > > That's the output from Dallas Engelken's "sa-stats.pl" log analyzer. > You feed it a segment of your spamd logs and it gives you > those rule hit statistics. > > See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers > > Looking at that wiki page, I noticed that the copy available is v0.93. > I've got v1.03 > Does anybody know what was the newest one last avaialable on the > rulesemporium site? Anbody got something newer than v1.03? > > I've done a bit of hacking to my copy (such as adding the S/O ratio stats). > > >> On Thu, 10 Mar 2016, Erickarlo Porro wrote: >> >> I would like to know how to get these stats too. >> >> From: Robert Chalmers [mailto:rob...@chalmers.com.au] >> Sent: Tuesday, March 08, 2016 5:25 AM >> To: users@spamassassin.apache.org >> Subject: Re: Missed spam, suggestions? >> >> Can I ask, how are you getting these stats please? >> >> Thanks >> >> On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu> >> wrote: >> >> On Mon, 7 Mar 2016, Charles Sprickman wrote: >> >> I’ve been running with some daily training for a little over a week and >> I’m seeing less spam in my >> inbox. I’ve seen a few things slip through because bayes tipped them >> below the default score, these >> were two phishing emails. >> >> Here’s some rule stats for anyone interested: >> >> TOP SPAM RULES FIRED >> >> RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM >> %OFHAM >> >> 1 TXREP 13171 8.47 40.38 91.00 >> 72.91 >> 2 HTML_MESSAGE12714 8.18 38.98 87.85 >> 90.80 >> 3 DCC_CHECK10593 6.81 32.48 73.19 >> 33.78 >> 4 RDNS_NONE10269 6.60 31.48 70.95 >> 5.63 >> 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 >> 23.41 >> 6 URIBL_BLACK97116.25 29.77 67.10 >> 1.58 >> 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 >> 65.98 1.64 >> 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 >> 65.52 1.36 >> 9 BAYES_99 84865.46 26.02 >> 58.63 1.18 >> 10BAYES_999 81415.24 24.96 >> 56.25 1.06 >> >> TOP HAM RULES FIRED >> >> RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM >> %OFHAM >> >> 1 HTML_MESSAGE16473 9.13 50.51 87.85 >> 90.80 >> 2 DKIM_SIGNED13776 7.64 42.24 13.81 >> 75.93 >> 3 TXREP 13228 7.33 40.56 91.00 >> 72.91 >> 4 DKIM_VALID 12962 7.19 39.74 11.93 >> 71.44 >> 5 RCVD_IN_DNSWL_NONE99415.51 30.48 8.08 >> 54.79 >> 6 DKIM_VALID_AU 87114.83 26.71 7.99 >> 48.01 >> 7 BAYES_00 83904.65 25.72 >> 1.84 46.24 >> 8 RCVD_IN_JMF_W 73694.09 22.59 2.54 >> 40.62 >> 9 RCVD_IN_MSPIKE_WL 67133.72 20.58 >> 4.3937.00 >> 10BAYES_50 62013.44 19.01 >> 25.56 34.18 >> Based upon your stats it looks like you need more Bayes training. Your Bayes >> 00/99 hits should rank higher in the >> rules-fired stats and BAYES_50 shouldn't be in the top-10 at all. >> (of course if you've only been training for a week that would explain it). >> For example, here's my top-10 hits (for a one month interval). >> TOP SPAM RULES FIRED >> -- >> RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O >> -- >> 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 >> 2BAYES_99109138
Re: Missed spam, suggestions?
ttp://wiki.apache.org/spamassassin/StatsAndAnalyzers >> <http://wiki.apache.org/spamassassin/StatsAndAnalyzers> >> be sure to search that page for reference to Dallas Engelken. >> >> >> >> On Fri, 11 Mar 2016, Robert Chalmers wrote: >> >>> The sa-stats.pl I refer to is here. >>> https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl >>> <https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl>. It’s >>> not the same as the ones shown in other posts. I don’t know what >>> that is. >>> and has an output like this. >>> zeus:~ robert$ perl sa-stats.pl >>> Report Title : SpamAssassin - Spam Statistics >>> Report Date : 2016-03-11 >>> Period Beginning : Fri 11 Mar 00:00:00 2016 >>> Period Ending: Sat 12 Mar 00:00:00 2016 >>> Reporting Period : 24.00 hrs >>> -- >>> Note: 'ham' = 'nonspam' >>> Total spam detected: 22 ( 51.16%) >>> Total ham accepted : 21 ( 48.84%) >>> --- >>> Total emails processed : 43 (2/hr) >>> Average spam threshold :3.00 >>> Average spam score :4.46 >>> Average ham score : -2.10 >>> Spam kbytes processed : 397 ( 17 kb/hr) >>> Ham kbytes processed : 147 (6 kb/hr) >>> Total kbytes processed : 545 ( 23 kb/hr) >>> Spam analysis time : 339 s ( 14 s/hr) >>> Ham analysis time : 366 s ( 15 s/hr) >>> Total analysis time: 706 s ( 29 s/hr) >>> Statistics by Hour >>> >>> Hour Spam Ham >>> ---- >>> 2016-03-11 00 0 ( 0%) 13 (100%) >>> 2016-03-11 01 0 ( 0%) 0 ( 0%) >>> 2016-03-11 02 2 (100%) 0 ( 0%) >>> 2016-03-11 03 4 (100%) 0 ( 0%) >>> 2016-03-11 04 4 ( 57%) 3 ( 42%) >>> 2016-03-11 05 6 ( 75%) 2 ( 25%) >>> 2016-03-11 06 6 (100%) 0 ( 0%) >>> 2016-03-11 07 0 ( 0%) 3 (100%) >>> 2016-03-11 08 0 ( 0%) 0 ( 0%) >>> 2016-03-11 09 0 ( 0%) 0 ( 0%) >>> 2016-03-11 10 0 ( 0%) 0 ( 0%) >>> 2016-03-11 11 0 ( 0%) 0 ( 0%) >>> 2016-03-11 12 0 ( 0%) 0 ( 0%) >>> 2016-03-11 13 0 ( 0%) 0 ( 0%) >>> 2016-03-11 14 0 ( 0%) 0 ( 0%) >>> 2016-03-11 15 0 ( 0%) 0 ( 0%) >>> 2016-03-11 16 0 ( 0%) 0 ( 0%) >>> 2016-03-11 17 0 ( 0%) 0 ( 0%) >>> 2016-03-11 18 0 ( 0%) 0 ( 0%) >>> 2016-03-11 19 0 ( 0%) 0 ( 0%) >>> 2016-03-11 20 0 ( 0%) 0 ( 0%) >>> 2016-03-11 21 0 ( 0%) 0 ( 0%) >>> 2016-03-11 22 0 ( 0%) 0 ( 0%) >>> 2016-03-11 23 0 ( 0%) 0 ( 0%) >>> Done. Report generated in 1 sec by sa-stats.pl, version 6256. >>> >>> On 10 Mar 2016, at 21:38, Erickarlo Porro <epo...@earthcam.com >>> <mailto:epo...@earthcam.com>> wrote: >>> I would like to know how to get these stats too. >>> From: Robert Chalmers [mailto:rob...@chalmers.com.au >>> <mailto:rob...@chalmers.com.au>] Sent: Tuesday, March 08, 2016 5:25 AM >>> To: users@spamassassin.apache.org <mailto:users@spamassassin.apache.org> >>> Subject: Re: Missed spam, suggestions? >>> Can I ask, how are you getting these stats please? >>> Thanks >>> On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu >>> <mailto:dbf...@engineering.uiowa.edu>> wrote: >>> On Mon, 7 Mar 2016, Charles Sprickman wrote: >>> >>> I’ve been running with some daily training for a little over a week >>> and I’m seeing less spam in my inbox. I’ve >>> seen a few things slip through because bayes tipped them below the >>> default score, these were two phishing emails. >>> >>> Here’s some rule stats for anyone interested: >>> >>> TOP SPAM RULES FIRED >>> >>>
Re: Missed spam, suggestions?
Just a note - that server address isn’t responding at the moment. Maybe later.Hopefully only temporary. > On 11 Mar 2016, at 14:59, Dave Funk <dbf...@engineering.uiowa.edu> wrote: > > TL;DR > You want Dallas Engelken's "sa-stats.pl" NOT the one from SA. > > This is confusing because there are two different programs named > "sa-stats.pl". > > The one that comes with SpamAssassin (what you're referring to) is an engine > stats reporting tool; does not do rule hits analysis. > > The tool that Charles Sprickman and I used is the one from Dallas Engelken. > See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers > be sure to search that page for reference to Dallas Engelken. > > > > On Fri, 11 Mar 2016, Robert Chalmers wrote: > >> The sa-stats.pl I refer to is here. >> https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl. It’s not >> the same as the ones shown in other posts. I don’t know what >> that is. >> and has an output like this. >> zeus:~ robert$ perl sa-stats.pl >> Report Title : SpamAssassin - Spam Statistics >> Report Date : 2016-03-11 >> Period Beginning : Fri 11 Mar 00:00:00 2016 >> Period Ending: Sat 12 Mar 00:00:00 2016 >> Reporting Period : 24.00 hrs >> -- >> Note: 'ham' = 'nonspam' >> Total spam detected: 22 ( 51.16%) >> Total ham accepted : 21 ( 48.84%) >> --- >> Total emails processed : 43 (2/hr) >> Average spam threshold :3.00 >> Average spam score :4.46 >> Average ham score : -2.10 >> Spam kbytes processed : 397 ( 17 kb/hr) >> Ham kbytes processed : 147 (6 kb/hr) >> Total kbytes processed : 545 ( 23 kb/hr) >> Spam analysis time : 339 s ( 14 s/hr) >> Ham analysis time : 366 s ( 15 s/hr) >> Total analysis time: 706 s ( 29 s/hr) >> Statistics by Hour >> >> Hour Spam Ham >> ---- >> 2016-03-11 00 0 ( 0%) 13 (100%) >> 2016-03-11 01 0 ( 0%) 0 ( 0%) >> 2016-03-11 02 2 (100%) 0 ( 0%) >> 2016-03-11 03 4 (100%) 0 ( 0%) >> 2016-03-11 04 4 ( 57%) 3 ( 42%) >> 2016-03-11 05 6 ( 75%) 2 ( 25%) >> 2016-03-11 06 6 (100%) 0 ( 0%) >> 2016-03-11 07 0 ( 0%) 3 (100%) >> 2016-03-11 08 0 ( 0%) 0 ( 0%) >> 2016-03-11 09 0 ( 0%) 0 ( 0%) >> 2016-03-11 10 0 ( 0%) 0 ( 0%) >> 2016-03-11 11 0 ( 0%) 0 ( 0%) >> 2016-03-11 12 0 ( 0%) 0 ( 0%) >> 2016-03-11 13 0 ( 0%) 0 ( 0%) >> 2016-03-11 14 0 ( 0%) 0 ( 0%) >> 2016-03-11 15 0 ( 0%) 0 ( 0%) >> 2016-03-11 16 0 ( 0%) 0 ( 0%) >> 2016-03-11 17 0 ( 0%) 0 ( 0%) >> 2016-03-11 18 0 ( 0%) 0 ( 0%) >> 2016-03-11 19 0 ( 0%) 0 ( 0%) >> 2016-03-11 20 0 ( 0%) 0 ( 0%) >> 2016-03-11 21 0 ( 0%) 0 ( 0%) >> 2016-03-11 22 0 ( 0%) 0 ( 0%) >> 2016-03-11 23 0 ( 0%) 0 ( 0%) >> Done. Report generated in 1 sec by sa-stats.pl, version 6256. >> >> On 10 Mar 2016, at 21:38, Erickarlo Porro <epo...@earthcam.com> wrote: >> I would like to know how to get these stats too. >> From: Robert Chalmers [mailto:rob...@chalmers.com.au] Sent: Tuesday, March >> 08, 2016 5:25 AM >> To: users@spamassassin.apache.org >> Subject: Re: Missed spam, suggestions? >> Can I ask, how are you getting these stats please? >> Thanks >> On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu> >> wrote: >> On Mon, 7 Mar 2016, Charles Sprickman wrote: >> >> I’ve been running with some daily training for a little over a week and >> I’m seeing less spam in my inbox. I’ve >> seen a few things slip through because bayes tipped them below the >> default score, these were two phishing emails. >> >> Here’s some rule stats for anyone interested: >> >> TOP SPAM RULES FIRED >> >> RANK RULE NAME
Re: Missed spam, suggestions?
Thanks, yes, confusion had set in there … now I’m on the right track It will however be handy to have both. Robert > On 11 Mar 2016, at 14:59, Dave Funk <dbf...@engineering.uiowa.edu> wrote: > > TL;DR > You want Dallas Engelken's "sa-stats.pl" NOT the one from SA. > > This is confusing because there are two different programs named > "sa-stats.pl". > > The one that comes with SpamAssassin (what you're referring to) is an engine > stats reporting tool; does not do rule hits analysis. > > The tool that Charles Sprickman and I used is the one from Dallas Engelken. > See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers > be sure to search that page for reference to Dallas Engelken. > > > > On Fri, 11 Mar 2016, Robert Chalmers wrote: > >> The sa-stats.pl I refer to is here. >> https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl. It’s not >> the same as the ones shown in other posts. I don’t know what >> that is. >> and has an output like this. >> zeus:~ robert$ perl sa-stats.pl >> Report Title : SpamAssassin - Spam Statistics >> Report Date : 2016-03-11 >> Period Beginning : Fri 11 Mar 00:00:00 2016 >> Period Ending: Sat 12 Mar 00:00:00 2016 >> Reporting Period : 24.00 hrs >> -- >> Note: 'ham' = 'nonspam' >> Total spam detected: 22 ( 51.16%) >> Total ham accepted : 21 ( 48.84%) >> --- >> Total emails processed : 43 (2/hr) >> Average spam threshold :3.00 >> Average spam score :4.46 >> Average ham score : -2.10 >> Spam kbytes processed : 397 ( 17 kb/hr) >> Ham kbytes processed : 147 (6 kb/hr) >> Total kbytes processed : 545 ( 23 kb/hr) >> Spam analysis time : 339 s ( 14 s/hr) >> Ham analysis time : 366 s ( 15 s/hr) >> Total analysis time: 706 s ( 29 s/hr) >> Statistics by Hour >> >> Hour Spam Ham >> ---- >> 2016-03-11 00 0 ( 0%) 13 (100%) >> 2016-03-11 01 0 ( 0%) 0 ( 0%) >> 2016-03-11 02 2 (100%) 0 ( 0%) >> 2016-03-11 03 4 (100%) 0 ( 0%) >> 2016-03-11 04 4 ( 57%) 3 ( 42%) >> 2016-03-11 05 6 ( 75%) 2 ( 25%) >> 2016-03-11 06 6 (100%) 0 ( 0%) >> 2016-03-11 07 0 ( 0%) 3 (100%) >> 2016-03-11 08 0 ( 0%) 0 ( 0%) >> 2016-03-11 09 0 ( 0%) 0 ( 0%) >> 2016-03-11 10 0 ( 0%) 0 ( 0%) >> 2016-03-11 11 0 ( 0%) 0 ( 0%) >> 2016-03-11 12 0 ( 0%) 0 ( 0%) >> 2016-03-11 13 0 ( 0%) 0 ( 0%) >> 2016-03-11 14 0 ( 0%) 0 ( 0%) >> 2016-03-11 15 0 ( 0%) 0 ( 0%) >> 2016-03-11 16 0 ( 0%) 0 ( 0%) >> 2016-03-11 17 0 ( 0%) 0 ( 0%) >> 2016-03-11 18 0 ( 0%) 0 ( 0%) >> 2016-03-11 19 0 ( 0%) 0 ( 0%) >> 2016-03-11 20 0 ( 0%) 0 ( 0%) >> 2016-03-11 21 0 ( 0%) 0 ( 0%) >> 2016-03-11 22 0 ( 0%) 0 ( 0%) >> 2016-03-11 23 0 ( 0%) 0 ( 0%) >> Done. Report generated in 1 sec by sa-stats.pl, version 6256. >> >> On 10 Mar 2016, at 21:38, Erickarlo Porro <epo...@earthcam.com> wrote: >> I would like to know how to get these stats too. >> From: Robert Chalmers [mailto:rob...@chalmers.com.au] Sent: Tuesday, March >> 08, 2016 5:25 AM >> To: users@spamassassin.apache.org >> Subject: Re: Missed spam, suggestions? >> Can I ask, how are you getting these stats please? >> Thanks >> On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu> >> wrote: >> On Mon, 7 Mar 2016, Charles Sprickman wrote: >> >> I’ve been running with some daily training for a little over a week and >> I’m seeing less spam in my inbox. I’ve >> seen a few things slip through because bayes tipped them below the >> default score, these were two phishing emails. >> >> Here’s some rule stats for anyone interested: >> >> TOP SPAM RULES FIRED >> >> RANK RULE NAME
Re: Missed spam, suggestions?
TL;DR You want Dallas Engelken's "sa-stats.pl" NOT the one from SA. This is confusing because there are two different programs named "sa-stats.pl". The one that comes with SpamAssassin (what you're referring to) is an engine stats reporting tool; does not do rule hits analysis. The tool that Charles Sprickman and I used is the one from Dallas Engelken. See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers be sure to search that page for reference to Dallas Engelken. On Fri, 11 Mar 2016, Robert Chalmers wrote: The sa-stats.pl I refer to is here. https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl. It’s not the same as the ones shown in other posts. I don’t know what that is. and has an output like this. zeus:~ robert$ perl sa-stats.pl Report Title : SpamAssassin - Spam Statistics Report Date : 2016-03-11 Period Beginning : Fri 11 Mar 00:00:00 2016 Period Ending : Sat 12 Mar 00:00:00 2016 Reporting Period : 24.00 hrs -- Note: 'ham' = 'nonspam' Total spam detected : 22 ( 51.16%) Total ham accepted : 21 ( 48.84%) --- Total emails processed : 43 ( 2/hr) Average spam threshold : 3.00 Average spam score : 4.46 Average ham score : -2.10 Spam kbytes processed : 397 ( 17 kb/hr) Ham kbytes processed : 147 ( 6 kb/hr) Total kbytes processed : 545 ( 23 kb/hr) Spam analysis time : 339 s ( 14 s/hr) Ham analysis time : 366 s ( 15 s/hr) Total analysis time : 706 s ( 29 s/hr) Statistics by Hour Hour Spam Ham - - -- 2016-03-11 00 0 ( 0%) 13 (100%) 2016-03-11 01 0 ( 0%) 0 ( 0%) 2016-03-11 02 2 (100%) 0 ( 0%) 2016-03-11 03 4 (100%) 0 ( 0%) 2016-03-11 04 4 ( 57%) 3 ( 42%) 2016-03-11 05 6 ( 75%) 2 ( 25%) 2016-03-11 06 6 (100%) 0 ( 0%) 2016-03-11 07 0 ( 0%) 3 (100%) 2016-03-11 08 0 ( 0%) 0 ( 0%) 2016-03-11 09 0 ( 0%) 0 ( 0%) 2016-03-11 10 0 ( 0%) 0 ( 0%) 2016-03-11 11 0 ( 0%) 0 ( 0%) 2016-03-11 12 0 ( 0%) 0 ( 0%) 2016-03-11 13 0 ( 0%) 0 ( 0%) 2016-03-11 14 0 ( 0%) 0 ( 0%) 2016-03-11 15 0 ( 0%) 0 ( 0%) 2016-03-11 16 0 ( 0%) 0 ( 0%) 2016-03-11 17 0 ( 0%) 0 ( 0%) 2016-03-11 18 0 ( 0%) 0 ( 0%) 2016-03-11 19 0 ( 0%) 0 ( 0%) 2016-03-11 20 0 ( 0%) 0 ( 0%) 2016-03-11 21 0 ( 0%) 0 ( 0%) 2016-03-11 22 0 ( 0%) 0 ( 0%) 2016-03-11 23 0 ( 0%) 0 ( 0%) Done. Report generated in 1 sec by sa-stats.pl, version 6256. On 10 Mar 2016, at 21:38, Erickarlo Porro <epo...@earthcam.com> wrote: I would like to know how to get these stats too. From: Robert Chalmers [mailto:rob...@chalmers.com.au] Sent: Tuesday, March 08, 2016 5:25 AM To: users@spamassassin.apache.org Subject: Re: Missed spam, suggestions? Can I ask, how are you getting these stats please? Thanks On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu> wrote: On Mon, 7 Mar 2016, Charles Sprickman wrote: I’ve been running with some daily training for a little over a week and I’m seeing less spam in my inbox. I’ve seen a few things slip through because bayes tipped them below the default score, these were two phishing emails. Here’s some rule stats for anyone interested: TOP SPAM RULES FIRED RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 TXREP 13171 8.47 40.38 91.00 72.91 2 HTML_MESSAGE 12714 8.18 38.98 87.85 90.80 3 DCC_CHECK 10593 6.81 32.48 73.19 33.78 4 RDNS_NONE 10269 6.60 31.48 70.95 5.63 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 6 URIBL_BLACK 97116.25 29.77 67.10 1.58 7 BODY_NEWDOMAIN_FMBLA 95506.14 29.28 65.98 1.64 8 FROM_NEWDOMAIN_FMBLA 94836.10 29.07 65.52 1.36 9 BAYES_99 84865.46 26.02 58.63 1.18 10BAYES_999 81415.24 24.96 56.25 1.06 TOP HAM RULES FIRED RANK RULE N
Re: Missed spam, suggestions?
Sorry - I missed the post from dbfunk. I just saw it in the archive. sa-stats.pl is the program, and you have to feed it from spamd.log to get those stats. To get a spamd.log, you have to start spamd with this -s facility, --syslog=facility <> Specify the syslog facility to use (default: mail). If stderr is specified, output will be written to stderr. (This is useful if you're running spamd under the daemontools package.) With a facility of file, all output goes to spamd.log. facility is interpreted as a file name to log to if it contains any characters except a-z and 0-9. null disables logging completely (used internally). spamd -s /var/log/spamd.log # log to file /var/log/spamd.log > On 10 Mar 2016, at 21:38, Erickarlo Porro <epo...@earthcam.com> wrote: > > I would like to know how to get these stats too. > > From: Robert Chalmers [mailto:rob...@chalmers.com.au] > Sent: Tuesday, March 08, 2016 5:25 AM > To: users@spamassassin.apache.org > Subject: Re: Missed spam, suggestions? > > Can I ask, how are you getting these stats please? > > Thanks > On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu > <mailto:dbf...@engineering.uiowa.edu>> wrote: > > On Mon, 7 Mar 2016, Charles Sprickman wrote: > > > I’ve been running with some daily training for a little over a week and I’m > seeing less spam in my inbox. I’ve seen a few things slip through because > bayes tipped them below the default score, these were two phishing emails. > > Here’s some rule stats for anyone interested: > > TOP SPAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 TXREP 13171 8.47 40.38 91.00 72.91 > 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 > 3 DCC_CHECK10593 6.81 32.48 73.19 33.78 > 4 RDNS_NONE10269 6.60 31.48 70.95 5.63 > 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 > 6 URIBL_BLACK97116.25 29.77 67.10 1.58 > 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 65.98 > 1.64 > 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 65.52 > 1.36 > 9 BAYES_99 84865.46 26.02 58.63 > 1.18 > 10BAYES_999 81415.24 24.96 56.25 > 1.06 > > TOP HAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 > 2 DKIM_SIGNED13776 7.64 42.24 13.81 75.93 > 3 TXREP 13228 7.33 40.56 91.00 72.91 > 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 > 5 RCVD_IN_DNSWL_NONE99415.51 30.48 8.08 > 54.79 > 6 DKIM_VALID_AU 87114.83 26.71 7.99 48.01 > 7 BAYES_00 83904.65 25.72 1.84 > 46.24 > 8 RCVD_IN_JMF_W 73694.09 22.59 2.54 40.62 > 9 RCVD_IN_MSPIKE_WL 67133.72 20.58 4.39 > 37.00 > 10BAYES_50 62013.44 19.01 25.56 > 34.18 > > > Based upon your stats it looks like you need more Bayes training. Your Bayes > 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't > be in the top-10 at all. > (of course if you've only been training for a week that would explain it). > > For example, here's my top-10 hits (for a one month interval). > > TOP SPAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 > 2BAYES_99109138 32.98 82.450.01 0.9998 > 3BAYES_999 104903 31.70 79.250.01 0. > 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 > 5URIBL_BLACK 9084527.61 68.630.27 0.9942 > 6T_QUARANTINE_1 9064027.40 68.470.02 0.9996 > 7URIBL_DBL_SPAM 7915224.02 59.790.17 0.9956 > 8KAM_VERY_BLACK_DBL 7430122.45 56.130.00 1. > 9L_FROM_SPAMMER1k7366722.26 55.
Re: Missed spam, suggestions?
The sa-stats.pl I refer to is here. https://spamassassin.apache.org/full/3.0.x/dist/tools/sa-stats.pl. It’s not the same as the ones shown in other posts. I don’t know what that is. and has an output like this. zeus:~ robert$ perl sa-stats.pl Report Title : SpamAssassin - Spam Statistics Report Date : 2016-03-11 Period Beginning : Fri 11 Mar 00:00:00 2016 Period Ending: Sat 12 Mar 00:00:00 2016 Reporting Period : 24.00 hrs -- Note: 'ham' = 'nonspam' Total spam detected: 22 ( 51.16%) Total ham accepted : 21 ( 48.84%) --- Total emails processed : 43 (2/hr) Average spam threshold :3.00 Average spam score :4.46 Average ham score : -2.10 Spam kbytes processed : 397 ( 17 kb/hr) Ham kbytes processed : 147 (6 kb/hr) Total kbytes processed : 545 ( 23 kb/hr) Spam analysis time : 339 s ( 14 s/hr) Ham analysis time : 366 s ( 15 s/hr) Total analysis time: 706 s ( 29 s/hr) Statistics by Hour Hour Spam Ham ---- 2016-03-11 00 0 ( 0%) 13 (100%) 2016-03-11 01 0 ( 0%) 0 ( 0%) 2016-03-11 02 2 (100%) 0 ( 0%) 2016-03-11 03 4 (100%) 0 ( 0%) 2016-03-11 04 4 ( 57%) 3 ( 42%) 2016-03-11 05 6 ( 75%) 2 ( 25%) 2016-03-11 06 6 (100%) 0 ( 0%) 2016-03-11 07 0 ( 0%) 3 (100%) 2016-03-11 08 0 ( 0%) 0 ( 0%) 2016-03-11 09 0 ( 0%) 0 ( 0%) 2016-03-11 10 0 ( 0%) 0 ( 0%) 2016-03-11 11 0 ( 0%) 0 ( 0%) 2016-03-11 12 0 ( 0%) 0 ( 0%) 2016-03-11 13 0 ( 0%) 0 ( 0%) 2016-03-11 14 0 ( 0%) 0 ( 0%) 2016-03-11 15 0 ( 0%) 0 ( 0%) 2016-03-11 16 0 ( 0%) 0 ( 0%) 2016-03-11 17 0 ( 0%) 0 ( 0%) 2016-03-11 18 0 ( 0%) 0 ( 0%) 2016-03-11 19 0 ( 0%) 0 ( 0%) 2016-03-11 20 0 ( 0%) 0 ( 0%) 2016-03-11 21 0 ( 0%) 0 ( 0%) 2016-03-11 22 0 ( 0%) 0 ( 0%) 2016-03-11 23 0 ( 0%) 0 ( 0%) Done. Report generated in 1 sec by sa-stats.pl, version 6256. > On 10 Mar 2016, at 21:38, Erickarlo Porro <epo...@earthcam.com> wrote: > > I would like to know how to get these stats too. > > From: Robert Chalmers [mailto:rob...@chalmers.com.au] > Sent: Tuesday, March 08, 2016 5:25 AM > To: users@spamassassin.apache.org > Subject: Re: Missed spam, suggestions? > > Can I ask, how are you getting these stats please? > > Thanks > On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu > <mailto:dbf...@engineering.uiowa.edu>> wrote: > > On Mon, 7 Mar 2016, Charles Sprickman wrote: > > > I’ve been running with some daily training for a little over a week and I’m > seeing less spam in my inbox. I’ve seen a few things slip through because > bayes tipped them below the default score, these were two phishing emails. > > Here’s some rule stats for anyone interested: > > TOP SPAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 TXREP 13171 8.47 40.38 91.00 72.91 > 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 > 3 DCC_CHECK10593 6.81 32.48 73.19 33.78 > 4 RDNS_NONE10269 6.60 31.48 70.95 5.63 > 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 > 6 URIBL_BLACK97116.25 29.77 67.10 1.58 > 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 65.98 > 1.64 > 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 65.52 > 1.36 > 9 BAYES_99 84865.46 26.02 58.63 > 1.18 > 10BAYES_999 81415.24 24.96 56.25 > 1.06 > > TOP HAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 > 2 DKIM_SIGNED13776 7.64 42.24 13.81 75.93 > 3 TXREP 13228 7.33 40.56 91.00 72.91 > 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44
Re: Missed spam, suggestions?
sa-stats.pl Sometimes part of the spamassassin package. You may have to search for it on your system, otherwise, it’s available via CPAN > On 10 Mar 2016, at 21:38, Erickarlo Porro <epo...@earthcam.com> wrote: > > I would like to know how to get these stats too. > > From: Robert Chalmers [mailto:rob...@chalmers.com.au] > Sent: Tuesday, March 08, 2016 5:25 AM > To: users@spamassassin.apache.org > Subject: Re: Missed spam, suggestions? > > Can I ask, how are you getting these stats please? > > Thanks > On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu > <mailto:dbf...@engineering.uiowa.edu>> wrote: > > On Mon, 7 Mar 2016, Charles Sprickman wrote: > > > I’ve been running with some daily training for a little over a week and I’m > seeing less spam in my inbox. I’ve seen a few things slip through because > bayes tipped them below the default score, these were two phishing emails. > > Here’s some rule stats for anyone interested: > > TOP SPAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 TXREP 13171 8.47 40.38 91.00 72.91 > 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 > 3 DCC_CHECK10593 6.81 32.48 73.19 33.78 > 4 RDNS_NONE10269 6.60 31.48 70.95 5.63 > 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 > 6 URIBL_BLACK97116.25 29.77 67.10 1.58 > 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 65.98 > 1.64 > 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 65.52 > 1.36 > 9 BAYES_99 84865.46 26.02 58.63 > 1.18 > 10BAYES_999 81415.24 24.96 56.25 > 1.06 > > TOP HAM RULES FIRED > > RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 > 2 DKIM_SIGNED13776 7.64 42.24 13.81 75.93 > 3 TXREP 13228 7.33 40.56 91.00 72.91 > 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 > 5 RCVD_IN_DNSWL_NONE99415.51 30.48 8.08 > 54.79 > 6 DKIM_VALID_AU 87114.83 26.71 7.99 48.01 > 7 BAYES_00 83904.65 25.72 1.84 > 46.24 > 8 RCVD_IN_JMF_W 73694.09 22.59 2.54 40.62 > 9 RCVD_IN_MSPIKE_WL 67133.72 20.58 4.39 > 37.00 > 10BAYES_50 62013.44 19.01 25.56 > 34.18 > > > Based upon your stats it looks like you need more Bayes training. Your Bayes > 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't > be in the top-10 at all. > (of course if you've only been training for a week that would explain it). > > For example, here's my top-10 hits (for a one month interval). > > TOP SPAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 > 2BAYES_99109138 32.98 82.450.01 0.9998 > 3BAYES_999 104903 31.70 79.250.01 0. > 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 > 5URIBL_BLACK 9084527.61 68.630.27 0.9942 > 6T_QUARANTINE_1 9064027.40 68.470.02 0.9996 > 7URIBL_DBL_SPAM 7915224.02 59.790.17 0.9956 > 8KAM_VERY_BLACK_DBL 7430122.45 56.130.00 1. > 9L_FROM_SPAMMER1k7366722.26 55.650.00 1. > 10T__RECEIVED_1 7241342.60 54.70 34.54 0.5135 > > OP HAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1BAYES_00182674 56.032.11 91.97 0.0150 > 2HTML_MESSAGE171992 79.41 68.63 86.59 0.3456 > 3SPF_PASS1366
sa-stats log analyzer (RE: Missed spam, suggestions?)
That's the output from Dallas Engelken's "sa-stats.pl" log analyzer. You feed it a segment of your spamd logs and it gives you those rule hit statistics. See: http://wiki.apache.org/spamassassin/StatsAndAnalyzers Looking at that wiki page, I noticed that the copy available is v0.93. I've got v1.03 Does anybody know what was the newest one last avaialable on the rulesemporium site? Anbody got something newer than v1.03? I've done a bit of hacking to my copy (such as adding the S/O ratio stats). On Thu, 10 Mar 2016, Erickarlo Porro wrote: I would like to know how to get these stats too. From: Robert Chalmers [mailto:rob...@chalmers.com.au] Sent: Tuesday, March 08, 2016 5:25 AM To: users@spamassassin.apache.org Subject: Re: Missed spam, suggestions? Can I ask, how are you getting these stats please? Thanks On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu> wrote: On Mon, 7 Mar 2016, Charles Sprickman wrote: I’ve been running with some daily training for a little over a week and I’m seeing less spam in my inbox. I’ve seen a few things slip through because bayes tipped them below the default score, these were two phishing emails. Here’s some rule stats for anyone interested: TOP SPAM RULES FIRED RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 TXREP 13171 8.47 40.38 91.00 72.91 2 HTML_MESSAGE 12714 8.18 38.98 87.85 90.80 3 DCC_CHECK 10593 6.81 32.48 73.19 33.78 4 RDNS_NONE 10269 6.60 31.48 70.95 5.63 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 6 URIBL_BLACK 9711 6.25 29.77 67.10 1.58 7 BODY_NEWDOMAIN_FMBLA 9550 6.14 29.28 65.98 1.64 8 FROM_NEWDOMAIN_FMBLA 9483 6.10 29.07 65.52 1.36 9 BAYES_99 8486 5.46 26.02 58.63 1.18 10 BAYES_999 8141 5.24 24.96 56.25 1.06 TOP HAM RULES FIRED RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 HTML_MESSAGE 16473 9.13 50.51 87.85 90.80 2 DKIM_SIGNED 13776 7.64 42.24 13.81 75.93 3 TXREP 13228 7.33 40.56 91.00 72.91 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 5 RCVD_IN_DNSWL_NONE 9941 5.51 30.48 8.08 54.79 6 DKIM_VALID_AU 8711 4.83 26.71 7.99 48.01 7 BAYES_00 8390 4.65 25.72 1.84 46.24 8 RCVD_IN_JMF_W 7369 4.09 22.59 2.54 40.62 9 RCVD_IN_MSPIKE_WL 6713 3.72 20.58 4.39 37.00 10 BAYES_50 6201 3.44 19.01 25.56 34.18 Based upon your stats it looks like you need more Bayes training. Your Bayes 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't be in the top-10 at all. (of course if you've only been training for a week that would explain it). For example, here's my top-10 hits (for a one month interval). TOP SPAM RULES FIRED -- RANK RULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O -- 1 T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 2 BAYES_99 109138 32.98 82.45 0.01 0.9998 3 BAYES_999 104903 31.70 79.25 0.01 0. 4 HTML_MESSAGE 90850 79.41 68.63 86.59 0.3456 5 URIBL_BLACK 90845 27.61 68.63 0.27 0.9942 6 T_QUARANTINE_1 90640 27.40 68.47 0.02 0.9996 7 URIBL_DBL_SPAM 79152 24.02 59.79 0.17 0.9956 8 KAM_VERY_BLACK_DBL 74301 22.45 56.13 0.00 1. 9 L_FROM_SPAMMER1k 73667 22.26 55.65 0.00 1. 10 T__RECEIVED_1 72413 42.60 54.70 34.54 0.5135 OP HAM RULES FIRED -- RANK RULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O -- 1 BAYES_00 182674 56.03 2.11 91.97 0.0150 2 HTML_MESSAGE 171992 79.41 68.63 86.59 0.3456 3 SPF_PASS
RE: Missed spam, suggestions?
I would like to know how to get these stats too. From: Robert Chalmers [mailto:rob...@chalmers.com.au] Sent: Tuesday, March 08, 2016 5:25 AM To: users@spamassassin.apache.org Subject: Re: Missed spam, suggestions? Can I ask, how are you getting these stats please? Thanks On 8 Mar 2016, at 05:11, David B Funk <dbf...@engineering.uiowa.edu<mailto:dbf...@engineering.uiowa.edu>> wrote: On Mon, 7 Mar 2016, Charles Sprickman wrote: I’ve been running with some daily training for a little over a week and I’m seeing less spam in my inbox. I’ve seen a few things slip through because bayes tipped them below the default score, these were two phishing emails. Here’s some rule stats for anyone interested: TOP SPAM RULES FIRED RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 TXREP 13171 8.47 40.38 91.00 72.91 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 3 DCC_CHECK10593 6.81 32.48 73.19 33.78 4 RDNS_NONE10269 6.60 31.48 70.95 5.63 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 6 URIBL_BLACK97116.25 29.77 67.10 1.58 7 BODY_NEWDOMAIN_FMBLA95506.14 29.28 65.98 1.64 8 FROM_NEWDOMAIN_FMBLA94836.10 29.07 65.52 1.36 9 BAYES_99 84865.46 26.02 58.63 1.18 10BAYES_999 81415.24 24.96 56.25 1.06 TOP HAM RULES FIRED RANK RULE NAMECOUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 2 DKIM_SIGNED13776 7.64 42.24 13.81 75.93 3 TXREP 13228 7.33 40.56 91.00 72.91 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 5 RCVD_IN_DNSWL_NONE99415.51 30.48 8.08 54.79 6 DKIM_VALID_AU 87114.83 26.71 7.99 48.01 7 BAYES_00 83904.65 25.72 1.84 46.24 8 RCVD_IN_JMF_W 73694.09 22.59 2.54 40.62 9 RCVD_IN_MSPIKE_WL 67133.72 20.58 4.39 37.00 10BAYES_50 62013.44 19.01 25.56 34.18 Based upon your stats it looks like you need more Bayes training. Your Bayes 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't be in the top-10 at all. (of course if you've only been training for a week that would explain it). For example, here's my top-10 hits (for a one month interval). TOP SPAM RULES FIRED -- RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O -- 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 2BAYES_99109138 32.98 82.450.01 0.9998 3BAYES_999 104903 31.70 79.250.01 0. 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 5URIBL_BLACK 9084527.61 68.630.27 0.9942 6T_QUARANTINE_1 9064027.40 68.470.02 0.9996 7URIBL_DBL_SPAM 7915224.02 59.790.17 0.9956 8KAM_VERY_BLACK_DBL 7430122.45 56.130.00 1. 9L_FROM_SPAMMER1k7366722.26 55.650.00 1. 10T__RECEIVED_1 7241342.60 54.70 34.54 0.5135 OP HAM RULES FIRED -- RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O -- 1BAYES_00182674 56.032.11 91.97 0.0150 2HTML_MESSAGE171992 79.41 68.63 86.59 0.3456 3SPF_PASS136623 63.08 54.52 68.78 0.3457 4T_RP_MATCHES_RCVD 130879 53.75 35.54 65.89 0.2644 5T__RECEIVED_2 125492 53.76 39.62 63.18 0.2947 6DKIM_SIGNED 114808 38.579.72 57.80 0.1008 7DKIM_VALID 105385 34.707.16 53.06 0.0825 8RCVD_IN_DNSWL_NONE 9295129.904.56 46.80 0.0609 9T__BOTNET_NOTRUST 8474160.32 86.81 42.66 0.5755 10KHOP_RCVD_TRUST 8462326.442.19 42.60 0.0331 Note how highly BAYES 00/99 ranked. What you don't see is that BAYES_50 is way down in the mud (below 50
Re: Missed spam, suggestions?
On Tue, 8 Mar 2016, Matus UHLAR - fantomas wrote: On Mar 8, 2016, at 7:31 AM, Matus UHLAR - fantomaswrote: how can these two stats be different? On 08.03.16 10:19, @lbutlr wrote: Because one is for SPAM and one is for HAM. On Mar 8, 2016, at 10:41 AM, Matus UHLAR - fantomas wrote: Why did you remove the important part? On 08.03.16 11:16, @lbutlr wrote: I didn’t. yes, you did, so I've had to paste them again below: TOP SPAM RULES FIRED RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 2 HTML_MESSAGE 12714 8.18 38.98 87.85 90.80 TOP HAM RULES FIRED RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 HTML_MESSAGE 16473 9.13 50.51 87.85 90.80 Why did the same rule hit 38.98% of all mail and 50.51% of all mail? Because on is checking SPAM and on is checking HAM. so why was %OFMAIL different from %OFSPAM in the first case and from %OFHAM in the second case? seems that the mail counts were different, but why? Because there are differing amounts of SPAM and HAM? if we are only checking spam mail for a given rule, how can be number of all hits different than number of spam hits? they all should be spam, shouldn't they? Assuming that the OP was using Dallas Engelken's "sa-stats.pl" script (I was) then the report line for each rule (excepting the first column) should be IDENTICAL. This script takes as input a spamd's log output. It then aggregates a digest of all the rule hits. In a given log report there will be lines that are spam results ("spamd: result: Y 75") and lines that are ham results ("spamd: result: . -3"). For each line (spam & ham) there will be a list of the rules that fired on that particular message: 2016-03-08T12:37:44.833847-06:00 s-l107 spamd[10463]: spamd: result: . -3 - BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,KHOP_RCVD_TRUST,L_LOCAL_MUCHO_DOT_LINES2,RCVD_IN_DNSWL_LOW,RCVD_IN_HOSTKARMA_YE,RP_MATCHES_RCVD,SPF_PASS,T__RECEIVED_1 scantime=3.5,size=11059,user=redacted,uid=115,required_score=6.0,rhost=s-l012.engr.uiowa.edu,raddr=128.255.17.253,rport=35620,mid= ,bayes=0.00,autolearn=ham autolearn_force=no So for the HTML_MESSAGE rule, I get stats of: grep HTML_MESSAGE sa-stats-dec.out 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 2HTML_MESSAGE171992 79.41 68.63 86.59 0.3456 This means that of all the messages processed (for the duration of that log run) that rule hit %79.41 of all messages processed, %68.63 of the lines classifed as spam (a count of 90850 and resulting in a rank of 4) and %86.59 of the lines classifed as ham (a count of 171992 resulting in a rank of 2). Thus for a given rule, the %all-messages, %spam %ham should be IDENTICAL. (assuming they are from the same log run). So for the OP's original post, having %spam %ham be identical but %all-messages being different is weird. Now it could be that he's got a different version of the sa-stats script, it has an addtional field, that "%of-rules" thing. So to Charles Sprickman, which sa-stats script did you use to generate your rules report? -- Dave Funk University of Iowa College of Engineering 319/335-5751 FAX: 319/384-0549 1256 Seamans Center Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527 #include Better is not better, 'standard' is better. B{
Re: Missed spam, suggestions?
On Mar 8, 2016, at 7:31 AM, Matus UHLAR - fantomaswrote: how can these two stats be different? On 08.03.16 10:19, @lbutlr wrote: Because one is for SPAM and one is for HAM. On Mar 8, 2016, at 10:41 AM, Matus UHLAR - fantomas wrote: Why did you remove the important part? On 08.03.16 11:16, @lbutlr wrote: I didn’t. yes, you did, so I've had to paste them again below: TOP SPAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 TOP HAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 Why did the same rule hit 38.98% of all mail and 50.51% of all mail? Because on is checking SPAM and on is checking HAM. so why was %OFMAIL different from %OFSPAM in the first case and from %OFHAM in the second case? seems that the mail counts were different, but why? Because there are differing amounts of SPAM and HAM? if we are only checking spam mail for a given rule, how can be number of all hits different than number of spam hits? they all should be spam, shouldn't they? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. On the other hand, you have different fingers.
Re: Missed spam, suggestions?
On Tue, 8 Mar 2016, Matus UHLAR - fantomas wrote: On Mar 8, 2016, at 7:31 AM, Matus UHLAR - fantomaswrote: > how can these two stats be different? On 08.03.16 10:19, @lbutlr wrote: Because one is for SPAM and one is for HAM. TOP SPAM RULES FIRED RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 2 HTML_MESSAGE 12714 8.18 38.98 87.85 90.80 TOP HAM RULES FIRED RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 HTML_MESSAGE 16473 9.13 50.51 87.85 90.80 Why did the same rule hit 38.98% of all mail and 50.51% of all mail? Speculation: 38.98 %OFMAIL = %OFSPAM * %SPAM, not %TOTAL so: HTML_MESSAGE hit 87.85% of spam, and *that* was 39.98% of total messages processed. ? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Failure to plan ahead on someone else's part does not constitute an emergency on my part. -- David W. Barts in a.s.r --- 5 days until Daylight Saving Time begins in U.S. - Spring Forward
Re: Missed spam, suggestions?
> On Mar 8, 2016, at 10:41 AM, Matus UHLAR - fantomaswrote: > >> On Mar 8, 2016, at 7:31 AM, Matus UHLAR - fantomas wrote: >>> how can these two stats be different? > > On 08.03.16 10:19, @lbutlr wrote: >> Because one is for SPAM and one is for HAM. > > Why did you remove the important part? I didn’t. > TOP SPAM RULES FIRED > > RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 > > TOP HAM RULES FIRED > > RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM > > 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 > > > Why did the same rule hit 38.98% of all mail and 50.51% of all mail? Because on is checking SPAM and on is checking HAM. > seems that the mail counts were different, but why? Because there are differing amounts of SPAM and HAM? -- "Rosa sat, so Martin could walk. Martin walked, so Obama could run. Obama ran, so our children can fly." (paraphrased from NPR)
Re: Missed spam, suggestions?
On 8. mar. 2016 18.42.03 Matus UHLAR - fantomaswrote: Why did the same rule hit 38.98% of all mail and 50.51% of all mail? grep foo ./hamfolder grep bar ./spamfolder Why should both folders need same counts of mails ?
Re: Missed spam, suggestions?
On Mar 8, 2016, at 7:31 AM, Matus UHLAR - fantomaswrote: how can these two stats be different? On 08.03.16 10:19, @lbutlr wrote: Because one is for SPAM and one is for HAM. Why did you remove the important part? TOP SPAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 2HTML_MESSAGE12714 8.18 38.98 87.85 90.80 TOP HAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1HTML_MESSAGE16473 9.13 50.51 87.85 90.80 Why did the same rule hit 38.98% of all mail and 50.51% of all mail? seems that the mail counts were different, but why? did Charles generate stats at that very different times? comparing results from the same set would be much better... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. A day without sunshine is like, night.
Re: Missed spam, suggestions?
On Mar 8, 2016, at 7:31 AM, Matus UHLAR - fantomaswrote: > how can these two stats be different? Because one is for SPAM and one is for HAM. -- No man is free who is not master of himself
Re: Missed spam, suggestions?
On 07.03.16 23:39, Charles Sprickman wrote: TOP SPAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 TOP HAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 how can these two stats be different? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "They say when you play that M$ CD backward you can hear satanic messages." "That's nothing. If you play it forward it will install Windows."
Re: Missed spam, suggestions?
Can I ask, how are you getting these stats please? Thanks > On 8 Mar 2016, at 05:11, David B Funkwrote: > > On Mon, 7 Mar 2016, Charles Sprickman wrote: > >> I’ve been running with some daily training for a little over a week and I’m >> seeing less spam in my inbox. I’ve seen a few things slip through because >> bayes tipped them below the default score, these were two phishing emails. >> >> Here’s some rule stats for anyone interested: >> >> TOP SPAM RULES FIRED >> >> RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM >> >> 1 TXREP 13171 8.47 40.38 91.00 72.91 >> 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 >> 3 DCC_CHECK 10593 6.81 32.48 73.19 33.78 >> 4 RDNS_NONE 10269 6.60 31.48 70.955.63 >> 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 >> 6 URIBL_BLACK 9711 6.25 29.77 67.101.58 >> 7 BODY_NEWDOMAIN_FMBLA 9550 6.14 29.28 65.981.64 >> 8 FROM_NEWDOMAIN_FMBLA 9483 6.10 29.07 65.521.36 >> 9 BAYES_99 8486 5.46 26.02 58.631.18 >> 10 BAYES_9998141 5.24 24.96 56.251.06 >> >> TOP HAM RULES FIRED >> >> RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM >> >> 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 >> 2 DKIM_SIGNED 13776 7.64 42.24 13.81 75.93 >> 3 TXREP 13228 7.33 40.56 91.00 72.91 >> 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 >> 5 RCVD_IN_DNSWL_NONE 9941 5.51 30.488.08 54.79 >> 6 DKIM_VALID_AU8711 4.83 26.717.99 48.01 >> 7 BAYES_00 8390 4.65 25.721.84 46.24 >> 8 RCVD_IN_JMF_W7369 4.09 22.592.54 40.62 >> 9 RCVD_IN_MSPIKE_WL6713 3.72 20.584.39 37.00 >> 10 BAYES_50 6201 3.44 19.01 25.56 34.18 >> > > Based upon your stats it looks like you need more Bayes training. Your Bayes > 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't > be in the top-10 at all. > (of course if you've only been training for a week that would explain it). > > For example, here's my top-10 hits (for a one month interval). > > TOP SPAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 > 2BAYES_99109138 32.98 82.450.01 0.9998 > 3BAYES_999 104903 31.70 79.250.01 0. > 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 > 5URIBL_BLACK 9084527.61 68.630.27 0.9942 > 6T_QUARANTINE_1 9064027.40 68.470.02 0.9996 > 7URIBL_DBL_SPAM 7915224.02 59.790.17 0.9956 > 8KAM_VERY_BLACK_DBL 7430122.45 56.130.00 1. > 9L_FROM_SPAMMER1k7366722.26 55.650.00 1. > 10T__RECEIVED_1 7241342.60 54.70 34.54 0.5135 > > OP HAM RULES FIRED > -- > RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O > -- > 1BAYES_00182674 56.032.11 91.97 0.0150 > 2HTML_MESSAGE171992 79.41 68.63 86.59 0.3456 > 3SPF_PASS136623 63.08 54.52 68.78 0.3457 > 4T_RP_MATCHES_RCVD 130879 53.75 35.54 65.89 0.2644 > 5T__RECEIVED_2 125492 53.76 39.62 63.18 0.2947 > 6DKIM_SIGNED 114808 38.579.72 57.80 0.1008 > 7DKIM_VALID 105385 34.707.16 53.06 0.0825 > 8RCVD_IN_DNSWL_NONE 9295129.904.56 46.80 0.0609 > 9T__BOTNET_NOTRUST 8474160.32 86.81 42.66 0.5755 > 10KHOP_RCVD_TRUST 8462326.442.19 42.60 0.0331 > > Note how highly BAYES 00/99 ranked. What you don't see is that BAYES_50 is > way down in the mud (below 50 rank). > > BTW, this is with a Bayes that is mostly fed via auto-learning. I occasionally > hand feed corner cases that get mis-classified (usually things
Re: Missed spam, suggestions?
On Mon, 7 Mar 2016, Charles Sprickman wrote: I’ve been running with some daily training for a little over a week and I’m seeing less spam in my inbox. I’ve seen a few things slip through because bayes tipped them below the default score, these were two phishing emails. Here’s some rule stats for anyone interested: TOP SPAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 TXREP 13171 8.47 40.38 91.00 72.91 2 HTML_MESSAGE12714 8.18 38.98 87.85 90.80 3 DCC_CHECK 10593 6.81 32.48 73.19 33.78 4 RDNS_NONE 10269 6.60 31.48 70.955.63 5 SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 6 URIBL_BLACK 9711 6.25 29.77 67.101.58 7 BODY_NEWDOMAIN_FMBLA 9550 6.14 29.28 65.981.64 8 FROM_NEWDOMAIN_FMBLA 9483 6.10 29.07 65.521.36 9 BAYES_99 8486 5.46 26.02 58.631.18 10 BAYES_9998141 5.24 24.96 56.251.06 TOP HAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1 HTML_MESSAGE16473 9.13 50.51 87.85 90.80 2 DKIM_SIGNED 13776 7.64 42.24 13.81 75.93 3 TXREP 13228 7.33 40.56 91.00 72.91 4 DKIM_VALID 12962 7.19 39.74 11.93 71.44 5 RCVD_IN_DNSWL_NONE 9941 5.51 30.488.08 54.79 6 DKIM_VALID_AU8711 4.83 26.717.99 48.01 7 BAYES_00 8390 4.65 25.721.84 46.24 8 RCVD_IN_JMF_W7369 4.09 22.592.54 40.62 9 RCVD_IN_MSPIKE_WL6713 3.72 20.584.39 37.00 10 BAYES_50 6201 3.44 19.01 25.56 34.18 Based upon your stats it looks like you need more Bayes training. Your Bayes 00/99 hits should rank higher in the rules-fired stats and BAYES_50 shouldn't be in the top-10 at all. (of course if you've only been training for a week that would explain it). For example, here's my top-10 hits (for a one month interval). TOP SPAM RULES FIRED -- RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O -- 1T__BOTNET_NOTRUST 114907 60.32 86.81 42.66 0.5755 2BAYES_99109138 32.98 82.450.01 0.9998 3BAYES_999 104903 31.70 79.250.01 0. 4HTML_MESSAGE9085079.41 68.63 86.59 0.3456 5URIBL_BLACK 9084527.61 68.630.27 0.9942 6T_QUARANTINE_1 9064027.40 68.470.02 0.9996 7URIBL_DBL_SPAM 7915224.02 59.790.17 0.9956 8KAM_VERY_BLACK_DBL 7430122.45 56.130.00 1. 9L_FROM_SPAMMER1k7366722.26 55.650.00 1. 10T__RECEIVED_1 7241342.60 54.70 34.54 0.5135 OP HAM RULES FIRED -- RANKRULE NAME COUNT %OFMAIL %OFSPAM %OFHAM S/O -- 1BAYES_00182674 56.032.11 91.97 0.0150 2HTML_MESSAGE171992 79.41 68.63 86.59 0.3456 3SPF_PASS136623 63.08 54.52 68.78 0.3457 4T_RP_MATCHES_RCVD 130879 53.75 35.54 65.89 0.2644 5T__RECEIVED_2 125492 53.76 39.62 63.18 0.2947 6DKIM_SIGNED 114808 38.579.72 57.80 0.1008 7DKIM_VALID 105385 34.707.16 53.06 0.0825 8RCVD_IN_DNSWL_NONE 9295129.904.56 46.80 0.0609 9T__BOTNET_NOTRUST 8474160.32 86.81 42.66 0.5755 10KHOP_RCVD_TRUST 8462326.442.19 42.60 0.0331 Note how highly BAYES 00/99 ranked. What you don't see is that BAYES_50 is way down in the mud (below 50 rank). BTW, this is with a Bayes that is mostly fed via auto-learning. I occasionally hand feed corner cases that get mis-classified (usually things like phishes, or conference announcments that can look shakey). -- Dave Funk University of Iowa College of Engineering 319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Re: Missed spam, suggestions?
> On Feb 29, 2016, at 3:18 PM, Reindl Haraldwrote: > > Am 29.02.2016 um 21:05 schrieb Charles Sprickman: >>> On Feb 29, 2016, at 4:23 AM, Reindl Harald wrote: >>> >>> Am 29.02.2016 um 06:24 schrieb Charles Sprickman: I’ve not had much luck with Bayes - when I had it enabled recently on a per-user basis it was just hitting the master DB server too hard with udpates >>> >>> just make a sitewide bayes >>> (https://wiki.apache.org/spamassassin/SiteWideBayesSetup) without autolearn >>> / autoexpire and the default database in a folder read-only for the daemon >>> >> >> I think I still have to stick with a db-backed option since I need to keep >> two SA servers in sync. > > and i know that it don't matter > > nothing easier then rsync the bayes-folder to several machines at the end of > the learning script, we even share the side-wide bayes over webservices to > external entities and so it coves around 5000 users at the moment in summary I’m not seeing much of a change in load after enabling this with a global user and no autolearn. I think the db was really only constrained on the inserts/updates. > >> I’ll try that today and see how the load looks. My concern with disabling >> autolearn is that then I’m the only one training. My spam probably looks >> like everyone else’s, but my ham is very different, lots list traffic and >> such. > > you should be the only one who trains in most cases for several reasons > > * few to zero users train anough ham and spam for a proper bayes > * wrong classified autolearn takes a wrong direction sooner or later > > given that we now for more than a year maintain a side-wide bayes for inbound > MX re-used on submission servers to minimize the impact of hacked accounts > and it works so much better than all the "user bayes" solutions the last > decade it's the way to go if you *really* want proper operations I’ve been running with some daily training for a little over a week and I’m seeing less spam in my inbox. I’ve seen a few things slip through because bayes tipped them below the default score, these were two phishing emails. Here’s some rule stats for anyone interested: TOP SPAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1TXREP 13171 8.47 40.38 91.00 72.91 2HTML_MESSAGE12714 8.18 38.98 87.85 90.80 3DCC_CHECK 10593 6.81 32.48 73.19 33.78 4RDNS_NONE 10269 6.60 31.48 70.955.63 5SPF_HELO_PASS 10070 6.48 30.87 69.58 23.41 6URIBL_BLACK 9711 6.25 29.77 67.101.58 7BODY_NEWDOMAIN_FMBLA 9550 6.14 29.28 65.981.64 8FROM_NEWDOMAIN_FMBLA 9483 6.10 29.07 65.521.36 9BAYES_99 8486 5.46 26.02 58.631.18 10BAYES_9998141 5.24 24.96 56.251.06 TOP HAM RULES FIRED RANKRULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM 1HTML_MESSAGE16473 9.13 50.51 87.85 90.80 2DKIM_SIGNED 13776 7.64 42.24 13.81 75.93 3TXREP 13228 7.33 40.56 91.00 72.91 4DKIM_VALID 12962 7.19 39.74 11.93 71.44 5RCVD_IN_DNSWL_NONE 9941 5.51 30.488.08 54.79 6DKIM_VALID_AU8711 4.83 26.717.99 48.01 7BAYES_00 8390 4.65 25.721.84 46.24 8RCVD_IN_JMF_W7369 4.09 22.592.54 40.62 9RCVD_IN_MSPIKE_WL6713 3.72 20.584.39 37.00 10BAYES_50 6201 3.44 19.01 25.56 34.18 Charles signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Missed spam, suggestions?
On Mon, 29 Feb 2016, Charles Sprickman wrote: My concern with disabling autolearn is that then I’m the only one training. My spam probably looks like everyone else’s, but my ham is very different, lots list traffic and such. You can still have your users provide misses for training, you'd just need to vet the messages before feeding them to sa_learn (unless you really trust a given user's judgement and honesty - the big problem is users training messages from lists they actually did subscribe to as spam, rather than unsubscribing). -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- We should endeavour to teach our children to be gun-proof rather than trying to design our guns to be child-proof --- 13 days until Albert Einstein's 137th Birthday
Re: Missed spam, suggestions?
Am 29.02.2016 um 21:05 schrieb Charles Sprickman: On Feb 29, 2016, at 4:23 AM, Reindl Haraldwrote: Am 29.02.2016 um 06:24 schrieb Charles Sprickman: I’ve not had much luck with Bayes - when I had it enabled recently on a per-user basis it was just hitting the master DB server too hard with udpates just make a sitewide bayes (https://wiki.apache.org/spamassassin/SiteWideBayesSetup) without autolearn / autoexpire and the default database in a folder read-only for the daemon I think I still have to stick with a db-backed option since I need to keep two SA servers in sync. and i know that it don't matter nothing easier then rsync the bayes-folder to several machines at the end of the learning script, we even share the side-wide bayes over webservices to external entities and so it coves around 5000 users at the moment in summary I’ll try that today and see how the load looks. My concern with disabling autolearn is that then I’m the only one training. My spam probably looks like everyone else’s, but my ham is very different, lots list traffic and such. you should be the only one who trains in most cases for several reasons * few to zero users train anough ham and spam for a proper bayes * wrong classified autolearn takes a wrong direction sooner or later given that we now for more than a year maintain a side-wide bayes for inbound MX re-used on submission servers to minimize the impact of hacked accounts and it works so much better than all the "user bayes" solutions the last decade it's the way to go if you *really* want proper operations signature.asc Description: OpenPGP digital signature
Re: Missed spam, suggestions?
> On Feb 29, 2016, at 4:23 AM, Reindl Haraldwrote: > > > > Am 29.02.2016 um 06:24 schrieb Charles Sprickman: >> I’ve not had much luck with Bayes - when I had it enabled recently on a >> per-user basis it was just hitting the master DB server too hard with udpates > > just make a sitewide bayes > (https://wiki.apache.org/spamassassin/SiteWideBayesSetup) without autolearn / > autoexpire and the default database in a folder read-only for the daemon > I think I still have to stick with a db-backed option since I need to keep two SA servers in sync. I’ll try that today and see how the load looks. My concern with disabling autolearn is that then I’m the only one training. My spam probably looks like everyone else’s, but my ham is very different, lots list traffic and such. > a filter without bayes is worthless It seems so. :) Thanks, Charles -- Charles Sprickman NetEng/SysAdmin Bway.net - New York's Best Internet www.bway.net sp...@bway.net - 212.982.9800 > > 0 61323SPAM > 0 21811HAM > 02547152TOKEN > > insgesamt 73M > -rw--- 1 sa-milt sa-milt 10M 2016-02-29 00:21 bayes_seen > -rw--- 1 sa-milt sa-milt 81M 2016-02-29 00:21 bayes_toks > > BAYES_0029161 73.70 % > BAYES_05 7641.93 % > BAYES_20 9312.35 % > BAYES_40 8152.05 % > BAYES_50 29097.35 % > BAYES_60 4241.07 % 8.14 % (OF TOTAL BLOCKED) > BAYES_80 3370.85 % 6.47 % (OF TOTAL BLOCKED) > BAYES_95 3060.77 % 5.87 % (OF TOTAL BLOCKED) > BAYES_99 39189.90 %75.25 % (OF TOTAL BLOCKED) > BAYES_99934918.82 %67.05 % (OF TOTAL BLOCKED) > > DNSWL 53551 91.16 % > SPF 38530 65.59 % > SPF/DKIM WL 16750 28.51 % > SHORTCIRCUIT19112 32.53 % > > BLOCKED 52068.86 % > SPAMMY 49858.48 %95.75 % (OF TOTAL BLOCKED) > signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Missed spam, suggestions?
Am 29.02.2016 um 06:24 schrieb Charles Sprickman: I’ve not had much luck with Bayes - when I had it enabled recently on a per-user basis it was just hitting the master DB server too hard with udpates just make a sitewide bayes (https://wiki.apache.org/spamassassin/SiteWideBayesSetup) without autolearn / autoexpire and the default database in a folder read-only for the daemon a filter without bayes is worthless 0 61323SPAM 0 21811HAM 02547152TOKEN insgesamt 73M -rw--- 1 sa-milt sa-milt 10M 2016-02-29 00:21 bayes_seen -rw--- 1 sa-milt sa-milt 81M 2016-02-29 00:21 bayes_toks BAYES_0029161 73.70 % BAYES_05 7641.93 % BAYES_20 9312.35 % BAYES_40 8152.05 % BAYES_50 29097.35 % BAYES_60 4241.07 % 8.14 % (OF TOTAL BLOCKED) BAYES_80 3370.85 % 6.47 % (OF TOTAL BLOCKED) BAYES_95 3060.77 % 5.87 % (OF TOTAL BLOCKED) BAYES_99 39189.90 %75.25 % (OF TOTAL BLOCKED) BAYES_99934918.82 %67.05 % (OF TOTAL BLOCKED) DNSWL 53551 91.16 % SPF 38530 65.59 % SPF/DKIM WL 16750 28.51 % SHORTCIRCUIT19112 32.53 % BLOCKED 52068.86 % SPAMMY 49858.48 %95.75 % (OF TOTAL BLOCKED) signature.asc Description: OpenPGP digital signature
Re: Missed spam, suggestions?
On 29-02-16 06:24, Charles Sprickman wrote: > Hi all, > > Recently I occasionally get bursts of spam that slips through Postfix > (postscreen BL checks, protocol checks) and SpamAssassin. I just had > another big jump in the last week. This was mostly spam touting Oil > Changes, SUV sales and Lawyer Finders. > > What I just did was go through a collection of missed spam and re-ran > it through spamassassin. All of it jumped from originally scoring > around 2-3 to a minimum of 6.5 with most hitting around 12. The > biggest difference I see is that DNSBL and URIBL services had started > hitting. When originally received, these emails all originated from > very clean IPs. > > I have TXREP enabled as well, but that doesn’t seem to be having > either a positive or negative impact. > > What are my options to try to catch this junk before it hits the > various *BLs? > > I’ve not had much luck with Bayes - when I had it enabled recently on > a per-user basis it was just hitting the master DB server too hard > with udpates. I’m considering enabling it again with a shared db for > all users, which I hope might work better. It would only be auto > trained, perhaps with some manual training by me. > > Here’s a few samples, hosted elsewhere so as not to trip anyone’s > filters: > > https://gist.github.com/anonymous/0fcaf481875959c9151f (2.7 on > Friday, 14 tonight) > > https://gist.github.com/anonymous/a5396f68699392808988 (3.4 earlier > tonight, 6.5 just now) > > I have more samples, I can dig them up if that’s helpful. > > Sometimes I wonder how much this has to do with the age of our domain > and the fact that it begins with “b”. :) > > The only thing I’ve been contemplating is a local spamtrap and DNSBL. > We have a site that’s regularly trawled for email addresses, so > seeding it should not be too difficult… > Hi, You want to give the RBLs a bit more time to kick in, you could consider greylisting (or postscreen after-220 checks which also cause a delay and a retry). Regards, Tom
Missed spam, suggestions?
Hi all, Recently I occasionally get bursts of spam that slips through Postfix (postscreen BL checks, protocol checks) and SpamAssassin. I just had another big jump in the last week. This was mostly spam touting Oil Changes, SUV sales and Lawyer Finders. What I just did was go through a collection of missed spam and re-ran it through spamassassin. All of it jumped from originally scoring around 2-3 to a minimum of 6.5 with most hitting around 12. The biggest difference I see is that DNSBL and URIBL services had started hitting. When originally received, these emails all originated from very clean IPs. I have TXREP enabled as well, but that doesn’t seem to be having either a positive or negative impact. What are my options to try to catch this junk before it hits the various *BLs? I’ve not had much luck with Bayes - when I had it enabled recently on a per-user basis it was just hitting the master DB server too hard with udpates. I’m considering enabling it again with a shared db for all users, which I hope might work better. It would only be auto trained, perhaps with some manual training by me. Here’s a few samples, hosted elsewhere so as not to trip anyone’s filters: https://gist.github.com/anonymous/0fcaf481875959c9151f (2.7 on Friday, 14 tonight) https://gist.github.com/anonymous/a5396f68699392808988 (3.4 earlier tonight, 6.5 just now) I have more samples, I can dig them up if that’s helpful. Sometimes I wonder how much this has to do with the age of our domain and the fact that it begins with “b”. :) The only thing I’ve been contemplating is a local spamtrap and DNSBL. We have a site that’s regularly trawled for email addresses, so seeding it should not be too difficult… Charles
Re: Missed SPAM
Is this the format being referred to? These are consistently getting through SA for us too http://pastebin.com/VHkfnTtm Jason On 01/04/12 10:05, John Hardin wrote: On Sat, 31 Mar 2012, joea wrote: On 3/31/2012 at 8:22 AM, Michael Scheidell michael.scheid...@secnap.com wrote: if you need help, you need enough full information. Or, you make the pastebin 'private', and send the link offlist to someone who has volunteered to help. . . . . If there are more volunteers, beyond the presumed one . . . feel free to . . . joea sent me the messages. It appears his bayes isn't running at all, they got no BAYES_## hits whatsoever. The URLs in them also have a really suspicious form; I've added a couple of rules for that to my sandbox. I suspect that the form is really uncommon, though, perhaps just fat fingers by this one spammer, so I doubt they will do well in masscheck. We'll see... -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
Re: Missed SPAM
On 04-04-2012 11:26, Jason Haar wrote: Is this the format being referred to? These are consistently getting through SA for us too http://pastebin.com/VHkfnTtm Jason On 01/04/12 10:05, John Hardin wrote: On Sat, 31 Mar 2012, joea wrote: On 3/31/2012 at 8:22 AM, Michael Scheidell michael.scheid...@secnap.com wrote: if you need help, you need enough full information. Or, you make the pastebin 'private', and send the link offlist to someone who has volunteered to help. . . . . If there are more volu nteers, beyond the presumed one . . . feel free to . . . joea sent me the messages. It appears his bayes isn't running at all, they got no BAYES_## hits whatsoever. The URLs in them also have a really suspicious form; I've added a couple of rules for that to my sandbox. I suspect that the form is really uncommon, though, perhaps just fat fingers by this one spammer, so I doubt they will do well in masscheck. We'll see... I had a similar thing here which these domain anonymizers seems to pass through SA so I got this from Robert Schetterer on the SA user list to this site AnonWhois to help score these domain anonymizers http://anonwhois.org/usage.html. This with Spam Eating Monkey does help with SA scoring this junk as spam http://spameatingmonkey.com/index.html. I hope this helps, Frank
Re: Missed SPAM
On Thu, 5 Apr 2012, Jason Haar wrote: Is this the format being referred to? These are consistently getting through SA for us too http://pastebin.com/VHkfnTtm No, it's not. On 01/04/12 10:05, John Hardin wrote: On Sat, 31 Mar 2012, joea wrote: On 3/31/2012 at 8:22 AM, Michael Scheidell michael.scheid...@secnap.com wrote: if you need help, you need enough full information. Or, you make the pastebin 'private', and send the link offlist to someone who has volunteered to help. . . . . If there are more volunteers, beyond the presumed one . . . feel free to . . . joea sent me the messages. It appears his bayes isn't running at all, they got no BAYES_## hits whatsoever. The URLs in them also have a really suspicious form; I've added a couple of rules for that to my sandbox. I suspect that the form is really uncommon, though, perhaps just fat fingers by this one spammer, so I doubt they will do well in masscheck. We'll see... -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The promise of nuclear power: electricity too cheap to meter The reality of nuclear power: FUD too cheap to meter --- 9 days until Thomas Jefferson's 269th Birthday
Re: Missed SPAM
. . . That's very little information to go on. Sorry. We learn as we go. Posting samples (with _all_ headers intact) on a pastebin or on a personal website so we can see them might yield some advice or new rules. Please don't send samples to the list, just the URLs where the samples are visible. If you can include the X-Spam headers so that we can see what rules hit, so much the better. I just created a pastebin.com account to do this. Ironically, their affirmation email was flagged as SPAM. I could post that, as well, I suppose. First I would like some clarification as to *what* to post. The Mime.882, complete? Or can I just do a snippet, starting below my local and MP details? Hopefully, the latter, as the former leaves me feeling a bit exposed. joe a. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
Re: Missed SPAM
On 3/31/12 8:04 AM, joea wrote: starting below my local and MP details? Hopefully, the latter, as the former leaves me feeling a bit exposed. we already know everything you think you want to hide. if you need help, you need enough full information. Or, you make the pastebin 'private', and send the link offlist to someone who has volunteered to help. If you want true accountability and privacy (by contract), you might need to pay someone to help you. Have them sign an NDA, and pay them. munging the headers with 'somehost.somenet.sometld [1.1.1.1]' helps no one at all. What information is important might not be apparent to you. If it was, you might have solved the problem yourself. -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 *| *SECNAP Network Security Corporation * Best Mobile Solutions Product of 2011 * Best Intrusion Prevention Product * Hot Company Finalist 2011 * Best Email Security Product * Certified SNORT Integrator __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.spammertrap.com/ __
Re: Missed SPAM
On 3/31/2012 at 8:22 AM, Michael Scheidell michael.scheid...@secnap.com wrote: On 3/31/12 8:04 AM, joea wrote: starting below my local and MP details? Hopefully, the latter, as the former leaves me feeling a bit exposed. we already know everything you think you want to hide. Well, let's hope not . . . if you need help, you need enough full information. Or, you make the pastebin 'private', and send the link offlist to someone who has volunteered to help. . . . . If there are more volunteers, beyond the presumed one . . . feel free to . . . munging the headers with 'somehost.somenet.sometld [1.1.1.1]' helps no one at all. What information is important might not be apparent to you. Well, true as that may be, I cannot fathom how munging any IP or hostname between final drop and fetch from MSP could have any bearing on the issue. If it was, you might have solved the problem yourself. Perhaps . . . Beyond that, where can I find the difference, in a SPAM learning sense, between sa-learn --spam filename and spamassassin -r filename? If I do the sa-learn on the same file, after doing spamassassin, it tells me 0 tokens. If I then do sa-learn --forget filename, then sa-learn --spam filename it tells me 1 token learned. I infer from this they perform similar or the same function, from a Bayes sense. joe a. -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 *| *SECNAP Network Security Corporation
Re: Missed SPAM
Post what you feel. The ML will help if they can. You can replace IPs and domains etc. -- Jeremy McSpadden On Mar 31, 2012, at 11:19 AM, joea j...@j4computers.com wrote: On 3/31/2012 at 8:22 AM, Michael Scheidell michael.scheid...@secnap.com wrote: On 3/31/12 8:04 AM, joea wrote: starting below my local and MP details? Hopefully, the latter, as the former leaves me feeling a bit exposed. we already know everything you think you want to hide. Well, let's hope not . . . if you need help, you need enough full information. Or, you make the pastebin 'private', and send the link offlist to someone who has volunteered to help. . . . . If there are more volunteers, beyond the presumed one . . . feel free to . . . munging the headers with 'somehost.somenet.sometld [1.1.1.1]' helps no one at all. What information is important might not be apparent to you. Well, true as that may be, I cannot fathom how munging any IP or hostname between final drop and fetch from MSP could have any bearing on the issue. If it was, you might have solved the problem yourself. Perhaps . . . Beyond that, where can I find the difference, in a SPAM learning sense, between sa-learn --spam filename and spamassassin -r filename? If I do the sa-learn on the same file, after doing spamassassin, it tells me 0 tokens. If I then do sa-learn --forget filename, then sa-learn --spam filename it tells me 1 token learned. I infer from this they perform similar or the same function, from a Bayes sense. joe a. -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 *| *SECNAP Network Security Corporation
Re: Missed SPAM
31.3.2012 19:17, joea kirjoitti: Beyond that, where can I find the difference, in a SPAM learning sense, between sa-learn --spam filename and spamassassin -r filename? If I do the sa-learn on the same file, after doing spamassassin, it tells me 0 tokens. If I then do sa-learn --forget filename, then sa-learn --spam filename it tells me 1 token learned. I infer from this they perform similar or the same function, from a Bayes sense. Sometimes, yes. If autolearn was activated, spamassassin learned this automatically. But only if. sa-learn learns always, if the message is not already learned to be spam|ham as passed in (that is checked by examining the message-id property of the smtp-message against the database). And it does not tell how many tokens it learned, but how many messages. A token is something like a word (not exactly, but close), and one message of course may contain many tokens. -- Be careful! UGLY strikes 9 out of 10! signature.asc Description: OpenPGP digital signature
Re: Missed SPAM
On Sat, 31 Mar 2012, joea wrote: On 3/31/2012 at 8:22 AM, Michael Scheidell michael.scheid...@secnap.com wrote: if you need help, you need enough full information. Or, you make the pastebin 'private', and send the link offlist to someone who has volunteered to help. . . . . If there are more volunteers, beyond the presumed one . . . feel free to . . . joea sent me the messages. It appears his bayes isn't running at all, they got no BAYES_## hits whatsoever. The URLs in them also have a really suspicious form; I've added a couple of rules for that to my sandbox. I suspect that the form is really uncommon, though, perhaps just fat fingers by this one spammer, so I doubt they will do well in masscheck. We'll see... -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- If Microsoft made hammers, everyone would whine about how poorly screws were designed and about how they are hard to hammer in, and wonder why it takes so long to paint a wall using the hammer. --- Tomorrow: April Fools' day
Re: Missed SPAM
On Sat, 31 Mar 2012 12:17:52 -0400 joea wrote: Beyond that, where can I find the difference, in a SPAM learning sense, between sa-learn --spam filename and spamassassin -r filename? If I do the sa-learn on the same file, after doing spamassassin, it tells me 0 tokens. If I then do sa-learn --forget filename, then sa-learn --spam filename it tells me 1 token learned. Are you sure that's what it says and not tokens from 1 message? The reason I ask is that it's practically impossible for Bayes to find only one token, so it would be a sign that something is wrong if some part of spamassassin is telling you that. I infer from this they perform similar or the same function, from a Bayes sense. spamassassin -r is mainly for reporting spam to SpamCop, Pyzor, etc, training Bayes is just a side-effect. But there's no grounds for thinking the training is going to be any different to running sa-learn or auto-training.
Re: Missed SPAM
On 3/31/2012 at 6:27 PM, RW rwmailli...@googlemail.com wrote: On Sat, 31 Mar 2012 12:17:52 -0400 joea wrote: Beyond that, where can I find the difference, in a SPAM learning sense, between sa-learn --spam filename and spamassassin -r filename? If I do the sa-learn on the same file, after doing spamassassin, it tells me 0 tokens. If I then do sa-learn --forget filename, then sa-learn --spam filename it tells me 1 token learned. Are you sure that's what it says and not tokens from 1 message? The reason I ask is that it's practically impossible for Bayes to find only one token, so it would be a sign that something is wrong if some part of spamassassin is telling you that. Sorry, yes, tokens from 1 message is what it reports. I should know better than to shorten messages. I infer from this they perform similar or the same function, from a Bayes sense. spamassassin -r is mainly for reporting spam to SpamCop, Pyzor, etc, training Bayes is just a side-effect. But there's no grounds for thinking the training is going to be any different to running sa-learn or auto-training. Thanks.
Missed SPAM
Having some difficulty grasping why some SPAM is getting thru yet some similar is marked. They have different source email address and subject, yet identical layout 3 http links, 3 graphics items and like that. When I save the message source (Mime.822 file) and do sa-learn --spam file it says Learned tokens from 0 message(s) (1 message(s) examined) I guess that means it already know this type? I did similar with a flagged message that I liked, with sa-learn --ham file. That tells me it learned 1 token. I Guess that means what is says. Seem I'm missing something.
Re: Missed SPAM
On Fri, 30 Mar 2012, joea wrote: Having some difficulty grasping why some SPAM is getting thru yet some similar is marked. They have different source email address and subject, yet identical layout 3 http links, 3 graphics items and like that. Layout generally isn't relevant. The links might be useful if they point at known spamvertised sites. However, there can be a delay between a site being spamvertised and it being known, so you might consider greylisting. That delays messages a bit and gives the spammy sites a chance to get recognized and listed and scored. Is there any text? Or are the images pictures of words? When I save the message source (Mime.822 file) and do sa-learn --spam file it says Learned tokens from 0 message(s) (1 message(s) examined) I guess that means it already know this type? Either it has already learned that message-ID, or the message is larger than the size limit for learning. I did similar with a flagged message that I liked, with sa-learn --ham file. That tells me it learned 1 token. I Guess that means what is says. Seem I'm missing something. That's very little information to go on. Posting samples (with _all_ headers intact) on a pastebin or on a personal website so we can see them might yield some advice or new rules. Please don't send samples to the list, just the URLs where the samples are visible. If you can include the X-Spam headers so that we can see what rules hit, so much the better. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...much of our country's counterterrorism security spending is not designed to protect us from the terrorists, but instead to protect our public officials from criticism when another attack occurs. -- Bruce Schneier --- 2 days until April Fools' day
Re: Lots of missed spam
* Leigh Sharpe wrote (29/06/06 03:03): This was my first suspicion. I turned off Bayes tests temporarily and it had little effect. I'm seriously considering resetting the bayes and starting again I can recommend that. I had a situation a while ago where the bayes database got mysteriously corrupted (sa-learn dump magic suddenly showed nspam way way less than nham). I deleted the whole bayes database, did a bit of manual training, let it carry on with the automatic training, and it was all fine again in a day or so. If spam hits BAYES_00 (which carries a negative score), you're better off without bayes at all. But with good bayes, most of the spam you've posted will be blocked. The difference between BAYES_00 and BAYES_99 is +6.099. So a small negative score with BAYES_00 will be sent over 5 by BAYES_99. Chris
Re: Lots of missed spam
From: Loren Wilton [EMAIL PROTECTED] I turned off Bayes tests temporarily and it had little effect. This seems a bit odd. That bayes_00 should have been good for about -3 points. Backing out Bayes should have raised the scores on this stuff by around 3 points, which with only a little bit of help should be tipping them into spam. On the other hand I was only seeing one or two other rules hitting on those things, which is rather few for a spam. You should maybe make sure that your paths to the rules files are what you think they are, and no rules files have gone missing. Running spamassassin --lint might be a good idea. You can also consider some add-on rulesets, like those at www.rulesemporium.com Maybe he turned off bayes tests and didn't restart or reload spamassassin? {o.o}
Lots of missed spam
Hi All, After 6 months or more of perfect operation, I have had heaps of spam has been missed over the last few weeks. Running SA with -D option shows nothing obvious in the logs. A small selection of misses is posted here: http://www.pacificwireless.com.au/spam/ Anybodygot any ideas why really obvious stuff might be getting through? Some of it is stuff which always used to get tagged, but now isn't. There's been no changes on the server, except for an increase in the number of mail users. I also note that quite a lot of it is getting negative sscores. Regards, Leigh Leigh SharpeNetwork Systems EngineerPacific WirelessPh +61 3 9584 8966Mob 0408 009 502email [EMAIL PROTECTED]web www.pacificwireless.com.au
Re: Lots of missed spam
Leigh Sharpe wrote: Hi All, After 6 months or more of perfect operation, I have had heaps of spam has been missed over the last few weeks. Running SA with -D option shows nothing obvious in the logs. A small selection of misses is posted here: http://www.pacificwireless.com.au/spam/ Anybody got any ideas why really obvious stuff might be getting through? Some of it is stuff which always used to get tagged, but now isn't. There's been no changes on the server, except for an increase in the number of mail users. I also note that quite a lot of it is getting negative sscores. 1) all of this spam is hitting BAYES_00.. you really should check your bayes training and correct it. 2) You're running a relatively old version of SpamAssassin. Version 3.0.3 has multiple security vulnerabilities. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2005-3351 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2005-1266 http://spamassassin.apache.org/advisories/cve-2006-2447.txt
Re: Lots of missed spam
From: Matt Kettler [EMAIL PROTECTED] Leigh Sharpe wrote: Hi All, After 6 months or more of perfect operation, I have had heaps of spam has been missed over the last few weeks. Running SA with -D option shows nothing obvious in the logs. A small selection of misses is posted here: http://www.pacificwireless.com.au/spam/ Anybody got any ideas why really obvious stuff might be getting through? Some of it is stuff which always used to get tagged, but now isn't. There's been no changes on the server, except for an increase in the number of mail users. I also note that quite a lot of it is getting negative sscores. 1) all of this spam is hitting BAYES_00.. you really should check your bayes training and correct it. THAT is a bad thing. Getting down to BAYES_00 for spam takes some doing. At the very least a whole lot of spam got trained as ham. I'd select a collection of known spam and a collection of known ham both totaling more than 200. (1000 if possible.) Then carefully feed them to sa-learn with the correct ham or spam flag. 2) You're running a relatively old version of SpamAssassin. Version 3.0.3 has multiple security vulnerabilities. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2005-3351 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2005-1266 http://spamassassin.apache.org/advisories/cve-2006-2447.txt The upgrade to 3.0.5 is relatively painless. I'd recommend that for the faint of heart. (I am getting excellent results here with 3.0.4 patched with some custom debug patches and with the 3.0.5 diffs from 3.0.4.) {^_^} JD
Re: Lots of missed spam
Leigh you have a large boatload of spam trained as ham. Make sure your users realize that GOOD messages train as ham and BAD messages train as spam. It appears at least one person has been feeding them both to the ham training. {^_^} - Original Message - From: Leigh Sharpe [EMAIL PROTECTED] 1) Bayes is still in training. I've only recently given everybody the opportunity to feed it spam. I expect it to get better soon. My question was more related to why this stuff is getting through now, when it used to get blocked. 2) I'll look into upgrading. I installed the current version using yum, and a check-update on spamassassin gives me an enormous list of dependencies which scares me a bit, quite frankly. Regards, Leigh Leigh Sharpe Network Systems Engineer Pacific Wireless Ph +61 3 9584 8966 Mob 0408 009 502 email [EMAIL PROTECTED] web www.pacificwireless.com.au -Original Message- From: Matt Kettler [mailto:[EMAIL PROTECTED] Leigh Sharpe wrote: Hi All, After 6 months or more of perfect operation, I have had heaps of spam has been missed over the last few weeks. Running SA with -D option shows nothing obvious in the logs. A small selection of misses is posted here: http://www.pacificwireless.com.au/spam/ Anybody got any ideas why really obvious stuff might be getting through? Some of it is stuff which always used to get tagged, but now isn't. There's been no changes on the server, except for an increase in the number of mail users. I also note that quite a lot of it is getting negative sscores. 1) all of this spam is hitting BAYES_00.. you really should check your bayes training and correct it. 2) You're running a relatively old version of SpamAssassin. Version 3.0.3 has multiple security vulnerabilities. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2005-3351 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2005-1266 http://spamassassin.apache.org/advisories/cve-2006-2447.txt
RE: Lots of missed spam
This was my first suspicion. I turned off Bayes tests temporarily and it had little effect. I'm seriously considering resetting the bayes and starting again, but this time I'll be making sure that it only gets fed by people who are actually competent enough to put their spam in the spam folder and ham in the ham folder, not the other way around. Regards, Leigh Leigh Sharpe Network Systems Engineer Pacific Wireless Ph +61 3 9584 8966 Mob 0408 009 502 email [EMAIL PROTECTED] web www.pacificwireless.com.au -Original Message- From: jdow [mailto:[EMAIL PROTECTED] Sent: Thursday, June 29, 2006 11:57 AM To: users@spamassassin.apache.org Subject: Re: Lots of missed spam Leigh you have a large boatload of spam trained as ham. Make sure your users realize that GOOD messages train as ham and BAD messages train as spam. It appears at least one person has been feeding them both to the ham training. {^_^} - Original Message - From: Leigh Sharpe [EMAIL PROTECTED] 1) Bayes is still in training. I've only recently given everybody the opportunity to feed it spam. I expect it to get better soon. My question was more related to why this stuff is getting through now, when it used to get blocked. 2) I'll look into upgrading. I installed the current version using yum, and a check-update on spamassassin gives me an enormous list of dependencies which scares me a bit, quite frankly. Regards, Leigh Leigh Sharpe Network Systems Engineer Pacific Wireless Ph +61 3 9584 8966 Mob 0408 009 502 email [EMAIL PROTECTED] web www.pacificwireless.com.au -Original Message- From: Matt Kettler [mailto:[EMAIL PROTECTED] Leigh Sharpe wrote: Hi All, After 6 months or more of perfect operation, I have had heaps of spam has been missed over the last few weeks. Running SA with -D option shows nothing obvious in the logs. A small selection of misses is posted here: http://www.pacificwireless.com.au/spam/ Anybody got any ideas why really obvious stuff might be getting through? Some of it is stuff which always used to get tagged, but now isn't. There's been no changes on the server, except for an increase in the number of mail users. I also note that quite a lot of it is getting negative sscores. 1) all of this spam is hitting BAYES_00.. you really should check your bayes training and correct it. 2) You're running a relatively old version of SpamAssassin. Version 3.0.3 has multiple security vulnerabilities. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2005-3351 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2005-1266 http://spamassassin.apache.org/advisories/cve-2006-2447.txt
RE: Lots of missed spam
On Thu, 29 Jun 2006, Leigh Sharpe wrote: I'm seriously considering resetting the bayes and starting again, but this time I'll be making sure that it only gets fed by people who are actually competent enough to put their spam in the spam folder and ham in the ham folder, not the other way around. Keep the users' spam and ham training folders. You can always check them, and forget and retrain the erroneous ones (or train, or disable, the erroneous user...) -- John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The problem is when people look at Yahoo, slashdot, or groklaw and jump from obvious and correct observations like Oh my God, this place is teeming with utter morons to incorrect conclusions like there's nothing of value here.-- Al Petrofsky, in Y! SCOX --- 6 days until The 230th anniversary of the Declaration of Independence
Re: Lots of missed spam
On Wed, Jun 28, 2006 at 06:55:07PM -0700, jdow wrote: 1) all of this spam is hitting BAYES_00.. you really should check your bayes training and correct it. THAT is a bad thing. Getting down to BAYES_00 for spam takes some doing. At the very least a whole lot of spam got trained as ham. Well, that's not necessarily true. Another possibility is that the spam message comes in but there are few tokens which are also in the DB. At that point Bayes has little to go on, and if the tokens in the DB are hammy, then the message is scored as ham. ie: Message has tokens a, b, c, d, ..., z. Of those, Bayes DB has tokens a, c, z, which are statistically ham. Therefore with the information available to Bayes, the Message is ham. This could even account for lots of messages all being marked as ham if there's no learning of the tokens going on in between receipt of the messages. But in the end, running the message through spamassassin -D bayes is likely the only thing that can be done to debug what is going on, but that's also probably not going to be helpful in the end with DB changes/learning/etc. -- Randomly Generated Tagline: I think Ultra Slimfast powered the SCUD missile. - Bob Lazarus pgpGbLkeqraNH.pgp Description: PGP signature
Another missed spam question
Since upgrading v2.64 to 3.0.2, I have a much higher false negative rate. I posted one a couple of days ago that involved a trusted issue. I just got a medication-spam this morning that ONLY triggered bayes_99, although it mentioned sexual health, anxiety and others I would've thought would've triggered more rules. Is a lot of reconfiguration usually necessary when upgrading 2.64 to 3.0? I thought I understood that 3.0 incorporated several of the rulesets that were previously separate, and besides, I haven't removed any old rulesets yet anyway. Any comments? Tnx!
Re: Another missed spam question
Is a lot of reconfiguration usually necessary when upgrading 2.64 to 3.0? I thought I understood that 3.0 incorporated several of the rulesets that were previously separate, and besides, I haven't removed any old rulesets yet anyway. Some is necessary. Shouldn't be a huge amount. You need to muck with the assorted local.cf options that have changed name and/or shape. If you have a NATed host, you need to set up trusted networks. (You should have had it before, but it is important now.) You need to make sure that all of the spare Perl parts are the appropriate versions. And if you are running SARE rules, you will need to fiddle around a little bit and make sure that you have a rule collection that is appropriate for 3.0+. Of course you should run lint to make sure things are really working, and probably also run spamassassin -D to make sure that all of your rule files are getting picked up. Loren
Re: Another missed spam question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Am Freitag, 21. Januar 2005 14:30 schrieb John Fleming: Since upgrading v2.64 to 3.0.2, I have a much higher false negative rate. I posted one a couple of days ago that involved a trusted issue. I just got a medication-spam this morning that ONLY triggered bayes_99, although it mentioned sexual health, anxiety and others I would've thought would've triggered more rules. Another case for my magic eye. Maybe I will find it some day. Some times they come trough. Spamer react on filters. Do you use network tests? Spamer changed the servers frequently. Is a lot of reconfiguration usually necessary when upgrading 2.64 to 3.0? I thought I understood that 3.0 incorporated several of the rulesets that were previously separate, and besides, I haven't removed any old rulesets yet anyway. I have upgraded three server fom 2.63 to 3.0.x. Normaly there are only small changes in the configuration for now unsupported options. The ammount of reconfiguration depneds on your installation. Any comments? Tnx! Keep your body informed. Garbage in - garbage out. Thomas - -- icq:133073900 http://www.t-arend.de -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (GNU/Linux) iD8DBQFB8RCFHe2ZLU3NgHsRAp4IAJ9Ssms7Cj357sCmsrDDCOL9Ac93DgCdFapR VKhrq4CNSbQIFCc13e9PVFU= =JnPW -END PGP SIGNATURE-
Re: Missed spam
On Friday 26 November 2004 10:28 am, Jerry Bell wrote: This spam went through with a score of 0. I'm using 3.01 with most of the sare rulesets. Any ideas on how to catch these? Just as a me too. I've been battling these for the last month or so with SA 3.0.1 with varied results. I run with a little higher required score (7.0) because this is a multi user setup. Regardless, these have proven very difficult to trap. I run the following SARE rules: 70_sare_adult.cf 72_sare_redirect_post3.0.0.cf 70_sare_bayes_poison_nxm.cf 99_FVGT_Tripwire.cf 70_sare_header0.cf 99_sare_fraud_post25x.cf 70_sare_specific.cf evilnumbers.cf Jeremy --- Date: Tue, 02 Nov 2004 11:42:41 +0200 Reply-To: Jeremiah Farkas [EMAIL PROTECTED] From: Jeremiah Farkas [EMAIL PROTECTED] User-Agent: The Bat! (v2.00.4) Personal X-Accept-Language: en-us MIME-Version: 1.0 To: Bo Riedell [EMAIL PROTECTED] Subject: Tell you a secret about keeping slimly built parch Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by phydio mail system X-Spam-Status: No, hits=3.5 tagged_above=-10.0 required=7.0 tests=BAYES_60, RCVD_IN_XBL, TW_RX X-Spam-Level: *** X-UID: 8 all-terrain pc-projects ping-kong inc-federal jc-shipfin zz01 fnet-free rxcom The fully stocked R#X check the overnight delivery interests. more satisfaction over nil payment on rx towards the LOow prices http://i.net.HealingRXinfo.com show you more satisfaction actually with overnight delivery. costless rx and consultation. Your mothers head is so big, it shows up on radar. A man limps into a bar with a cane and alligator. The bartender stops himandsays Holdon a secondhere - youcan't bringthatanimal inhere,theyaren'tallowed! Sotheman says, Butmygatorheredoes areally cooltrick... --- Date: Sat, 06 Nov 2004 05:27:08 +0800 Reply-To: billy edmonson [EMAIL PROTECTED] From: billy edmonson [EMAIL PROTECTED] User-Agent: AOL 4.0 for Windows 95 sub 10 MIME-Version: 1.0 To: Perry Anastas [EMAIL PROTECTED] Subject: To suit all tastes is really our work inhumane Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by phydio mail system X-Spam-Status: No, hits=2.8 tagged_above=-10.0 required=7.0 tests=BAYES_99, DRUGS_ANXIETY, DRUGS_PAIN, TW_BF X-Spam-Level: ** X-UID: 6 dumpsize cstgnttj frederiksted electroconductores fbfbtab zz01 fbtest enrimmon Super low charge with super service on handreds of RX meds, it is all real. The site lists Vicodin, Valium, and many more. For more, just check it. Hlmuxntww http://vr.net.FavorRXinfo.com/?Ig3 benefitmore from next day delivery. nil payment for rx Yo mama so fat, she put on her lipstick with a paint-roller Q. Why are blondes like 7-Eleven stores? A. Open 24 hours a day. --- Date: Wed, 17 Nov 2004 12:27:23 +0700 From: wesley weekley [EMAIL PROTECTED] User-Agent: Netscape6/6.1b1 X-Accept-Language: en-us MIME-Version: 1.0 To: quintin sigmon [EMAIL PROTECTED] Subject: savvings from reliable internet pharmacy Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by phydio mail system X-Spam-Status: No, hits=3.6 tagged_above=-10.0 required=7.0 tests=BAYES_99, TW_OV, URIBL_SBL X-Spam-Level: *** X-UID: 12 express service for rx refill online reduction in price available for you The site offers more than 600 meds in over 40 categories such as Pain Relief, Sleeping Aids, Depression-Anxiety, Muscle Relaxants, Allergy, Antibiotic and Wt. Loss. quality meds all at lower prices http://Rh.Bv.ofsupergood.com/?Ehd2sk8Kkl9-Wi1Rx4197Dxxlu45373Oa I just want to give internet pharmacy a try. Now I find it is really a convenient and quick solution for me. Just great. Online Rx PRO I expected that the matter would never be heard of; but, I wished to relieve my own mind. I had kept the matter`God bless you!' and left her. TO the eyes of Mr. Jeremiah Cruncher, sitting on his stool in Fleet Street withonnettomuuksia 51paljastajat01 sovjetologi saksankieliseenserbeille
Missed spam
This spam went through with a score of 0. I'm using 3.01 with most of the sare rulesets. Any ideas on how to catch these? Thanks, Jerry http://www.syslog.org Return-path: [EMAIL PROTECTED] Envelope-to: [EMAIL PROTECTED], [EMAIL PROTECTED] Delivery-date: Thu, 25 Nov 2004 14:53:39 -0500 Received: from [222.76.179.18] (helo=irishlover.net) by stelesys.com with smtp (Exim 4.43 (FreeBSD)) id 1CXPgN-000EzG-OE; Thu, 25 Nov 2004 14:53:39 -0500 Message-ID: [EMAIL PROTECTED] Date: Thu, 25 Nov 2004 21:24:31 + From: abe pasquino [EMAIL PROTECTED] User-Agent: fostering Program V Mail Client 5.0 MIME-Version: 1.0 To: thurman rand [EMAIL PROTECTED] Subject: internet rx refill-great deals on meds Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit overnight delivery for orders meds hotline--low priced meds Over 600 meds available for sexual health, allergy, asthma, sleeping disorder, obesity, pain relief, sexual health, anxiety relief and hypertension. lower price the pharmacy could offfer http://Lu.Yr.goodofurs.com/?Bdyqebamvl9Pq9Nb1Cld778629Rl=233166Hwk It is really FAST and EASY for me. Just get the rx refilled online with internet pharmacy. Virginia `Hark!' said The Vengeance. `Listen, then! Who comes?'As if a train of powder laid from the outermost bound of the Saint Antoine Quarter to the wine-shop door, hada bitter day, he wore no coat, but carried one slung over his shoulder. His shirt-sleeves were rolled up, too,jttmaille51 nurkanvaltaukset 01 nostrilsmarjukka apulaisverotarkastajalle
Re: Missed spam
Jerry Bell wrote: This spam went through with a score of 0. I'm using 3.01 with most of the sare rulesets. Any ideas on how to catch these? Thanks, Jerry http://www.syslog.org Return-path: [EMAIL PROTECTED] Envelope-to: [EMAIL PROTECTED], [EMAIL PROTECTED] Delivery-date: Thu, 25 Nov 2004 14:53:39 -0500 Received: from [222.76.179.18] (helo=irishlover.net) by stelesys.com with smtp (Exim 4.43 (FreeBSD)) id 1CXPgN-000EzG-OE; Thu, 25 Nov 2004 14:53:39 -0500 Message-ID: [EMAIL PROTECTED] Date: Thu, 25 Nov 2004 21:24:31 + From: abe pasquino [EMAIL PROTECTED] User-Agent: fostering Program V Mail Client 5.0 MIME-Version: 1.0 To: thurman rand [EMAIL PROTECTED] Subject: internet rx refill-great deals on meds Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit overnight delivery for orders meds hotline--low priced meds Over 600 meds available for sexual health, allergy, asthma, sleeping disorder, obesity, pain relief, sexual health, anxiety relief and hypertension. lower price the pharmacy could offfer http://Lu.Yr.goodofurs.com/?Bdyqebamvl9Pq9Nb1Cld778629Rl=233166Hwk It is really FAST and EASY for me. Just get the rx refilled online with internet pharmacy. Virginia `Hark!' said The Vengeance. `Listen, then! Who comes?'As if a train of powder laid from the outermost bound of the Saint Antoine Quarter to the wine-shop door, hada bitter day, he wore no coat, but carried one slung over his shoulder. His shirt-sleeves were rolled up, too,jttmaille51 nurkanvaltaukset 01 nostrilsmarjukka apulaisverotarkastajalle Umm..i dont see any SA headers..you sure this message was actually scanned? -Jim
Re: Missed spam
I'm using SA through exim/exiscan, and I've got it set up to only report if it is spam. Guess I should change that. The SA logs showing it getting a score of 0. SA is working really well for me the other 99% of the time. Jerry Jerry Bell wrote: This spam went through with a score of 0. I'm using 3.01 with most of the sare rulesets. Any ideas on how to catch these? Thanks, Jerry http://www.syslog.org Return-path: [EMAIL PROTECTED] Envelope-to: [EMAIL PROTECTED], [EMAIL PROTECTED] Delivery-date: Thu, 25 Nov 2004 14:53:39 -0500 Received: from [222.76.179.18] (helo=irishlover.net) by stelesys.com with smtp (Exim 4.43 (FreeBSD)) id 1CXPgN-000EzG-OE; Thu, 25 Nov 2004 14:53:39 -0500 Message-ID: [EMAIL PROTECTED] Date: Thu, 25 Nov 2004 21:24:31 + From: abe pasquino [EMAIL PROTECTED] User-Agent: fostering Program V Mail Client 5.0 MIME-Version: 1.0 To: thurman rand [EMAIL PROTECTED] Subject: internet rx refill-great deals on meds Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit overnight delivery for orders meds hotline--low priced meds Over 600 meds available for sexual health, allergy, asthma, sleeping disorder, obesity, pain relief, sexual health, anxiety relief and hypertension. lower price the pharmacy could offfer http://Lu.Yr.goodofurs.com/?Bdyqebamvl9Pq9Nb1Cld778629Rl=233166Hwk It is really FAST and EASY for me. Just get the rx refilled online with internet pharmacy. Virginia `Hark!' said The Vengeance. `Listen, then! Who comes?'As if a train of powder laid from the outermost bound of the Saint Antoine Quarter to the wine-shop door, hada bitter day, he wore no coat, but carried one slung over his shoulder. His shirt-sleeves were rolled up, too,jttmaille51 nurkanvaltaukset 01 nostrilsmarjukka apulaisverotarkastajalle Umm..i dont see any SA headers..you sure this message was actually scanned? -Jim
Re: Missed spam
Jerry Bell wrote: I'm using SA through exim/exiscan, and I've got it set up to only report if it is spam. Guess I should change that. The SA logs showing it getting a score of 0. SA is working really well for me the other 99% of the time. Jerry Jerry Bell wrote: This spam went through with a score of 0. I'm using 3.01 with most of the sare rulesets. Any ideas on how to catch these? Thanks, Jerry http://www.syslog.org Return-path: [EMAIL PROTECTED] Envelope-to: [EMAIL PROTECTED], [EMAIL PROTECTED] Delivery-date: Thu, 25 Nov 2004 14:53:39 -0500 Received: from [222.76.179.18] (helo=irishlover.net) by stelesys.com with smtp (Exim 4.43 (FreeBSD)) id 1CXPgN-000EzG-OE; Thu, 25 Nov 2004 14:53:39 -0500 Message-ID: [EMAIL PROTECTED] Date: Thu, 25 Nov 2004 21:24:31 + From: abe pasquino [EMAIL PROTECTED] User-Agent: fostering Program V Mail Client 5.0 MIME-Version: 1.0 To: thurman rand [EMAIL PROTECTED] Subject: internet rx refill-great deals on meds Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit overnight delivery for orders meds hotline--low priced meds Over 600 meds available for sexual health, allergy, asthma, sleeping disorder, obesity, pain relief, sexual health, anxiety relief and hypertension. lower price the pharmacy could offfer http://Lu.Yr.goodofurs.com/?Bdyqebamvl9Pq9Nb1Cld778629Rl=233166Hwk It is really FAST and EASY for me. Just get the rx refilled online with internet pharmacy. Virginia `Hark!' said The Vengeance. `Listen, then! Who comes?'As if a train of powder laid from the outermost bound of the Saint Antoine Quarter to the wine-shop door, hada bitter day, he wore no coat, but carried one slung over his shoulder. His shirt-sleeves were rolled up, too,jttmaille51 nurkanvaltaukset 01 nostrilsmarjukka apulaisverotarkastajalle Umm..i dont see any SA headers..you sure this message was actually scanned? -Jim Content analysis details: (6.1 points, 5.0 required) pts rule name description -- -- 1.9 DATE_MISSING Missing Date: header 2.0 FROM_NO_LOWER 'From' has no lower-case characters -0.0 BAYES_44 BODY: Bayesian spam probability is 44 to 50% [score: 0.4638] 1.1 RAZOR2_CF_RANGE_51_100 BODY: Razor2 gives confidence between 51 and 100 [cf: 100] 1.0 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) I am running 2.64 with no extra rules. Had this been received at my system, the bayes score would most likely have been higher as well. -Jim
Re: Missed spam
I wonder if my bayes db has been poisoned to the point of thinking this is ham? In the logs, it autolearned this one as ham, so I suspect that may be the case. Jerry Bell wrote: I'm using SA through exim/exiscan, and I've got it set up to only report if it is spam. Guess I should change that. The SA logs showing it getting a score of 0. SA is working really well for me the other 99% of the time. Jerry Jerry Bell wrote: This spam went through with a score of 0. I'm using 3.01 with most of the sare rulesets. Any ideas on how to catch these? Thanks, Jerry http://www.syslog.org Return-path: [EMAIL PROTECTED] Envelope-to: [EMAIL PROTECTED], [EMAIL PROTECTED] Delivery-date: Thu, 25 Nov 2004 14:53:39 -0500 Received: from [222.76.179.18] (helo=irishlover.net) by stelesys.com with smtp (Exim 4.43 (FreeBSD)) id 1CXPgN-000EzG-OE; Thu, 25 Nov 2004 14:53:39 -0500 Message-ID: [EMAIL PROTECTED] Date: Thu, 25 Nov 2004 21:24:31 + From: abe pasquino [EMAIL PROTECTED] User-Agent: fostering Program V Mail Client 5.0 MIME-Version: 1.0 To: thurman rand [EMAIL PROTECTED] Subject: internet rx refill-great deals on meds Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit overnight delivery for orders meds hotline--low priced meds Over 600 meds available for sexual health, allergy, asthma, sleeping disorder, obesity, pain relief, sexual health, anxiety relief and hypertension. lower price the pharmacy could offfer http://Lu.Yr.goodofurs.com/?Bdyqebamvl9Pq9Nb1Cld778629Rl=233166Hwk It is really FAST and EASY for me. Just get the rx refilled online with internet pharmacy. Virginia `Hark!' said The Vengeance. `Listen, then! Who comes?'As if a train of powder laid from the outermost bound of the Saint Antoine Quarter to the wine-shop door, hada bitter day, he wore no coat, but carried one slung over his shoulder. His shirt-sleeves were rolled up, too,jttmaille51 nurkanvaltaukset 01 nostrilsmarjukka apulaisverotarkastajalle Umm..i dont see any SA headers..you sure this message was actually scanned? -Jim Content analysis details: (6.1 points, 5.0 required) pts rule name description -- -- 1.9 DATE_MISSING Missing Date: header 2.0 FROM_NO_LOWER 'From' has no lower-case characters -0.0 BAYES_44 BODY: Bayesian spam probability is 44 to 50% [score: 0.4638] 1.1 RAZOR2_CF_RANGE_51_100 BODY: Razor2 gives confidence between 51 and 100 [cf: 100] 1.0 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) I am running 2.64 with no extra rules. Had this been received at my system, the bayes score would most likely have been higher as well. -Jim
Re: Missed spam
Jerry Bell wrote: I wonder if my bayes db has been poisoned to the point of thinking this is ham? In the logs, it autolearned this one as ham, so I suspect that may be the case. You say it scored 0 points..does this mean it triggered no rules or the + - rules totaled up to 0? Regardless of bayes poisoning, you should still see *some* rules. Its possible i suppose that it could have triggered bayes_50 and produced no score. Either way, it looks like there may be a bigger problem with this message. Its rare the a message comes through that doesnt trigger *any* rules. I'd try running it through your installation of SA again and see if it scores differently this time. -Jim
Re: Missed spam
When I run it manually, this is what I get: X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on db.stelesys.com X-Spam-Status: No, score=0.1 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.0.1 X-Spam-Level: What's this best way to get it out of the AWL and bayes? Thanks for the help! It looks like its in the whitelist and scoring low in bayes. Jerry Bell wrote: I wonder if my bayes db has been poisoned to the point of thinking this is ham? In the logs, it autolearned this one as ham, so I suspect that may be the case. You say it scored 0 points..does this mean it triggered no rules or the + - rules totaled up to 0? Regardless of bayes poisoning, you should still see *some* rules. Its possible i suppose that it could have triggered bayes_50 and produced no score. Either way, it looks like there may be a bigger problem with this message. Its rare the a message comes through that doesnt trigger *any* rules. I'd try running it through your installation of SA again and see if it scores differently this time. -Jim
Re: Missed spam
Jerry Bell wrote: When I run it manually, this is what I get: X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on db.stelesys.com X-Spam-Status: No, score=0.1 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.0.1 X-Spam-Level: What's this best way to get it out of the AWL and bayes? Thanks for the help! It looks like its in the whitelist and scoring low in bayes. You can re-learn it as something else (spam) and it will be corrected in bayes. You can also choose to --forget it and it will be gone from the database completely. As far as the AWL goes, im not sure. I dont use whitelists. You may just be able to remove the whitelist files themselves. -Jim