Re: Mail Filter Recommendations
Thanks, Bowie and Noel, Here's a couple of example spams that are the kind which are slipping through constantly. Some of the them get caught, others do not. http://pastebin.com/UH5BA6zs http://pastebin.com/esEz1a4J Also, could you offer some guidance on how to set up scanning /per user/ for Bayes data with Amavis running at the same time? Thanks, Asai On 4/7/15 6:10 AM, Bowie Bailey wrote: On 4/6/2015 11:47 PM, Noel wrote: On 4/6/2015 10:08 PM, Asai wrote: Greetings, We've been using Amavis for a number of years, but it seems to not be doing what we need it to be doing regarding spam filtering. e.g. I can't seem to get it to learn bayes data on a per user basis. We have our spam filters turned up so high ( kill level 3 ) for some users it just seems like we're doing something wrong. I have deleted the Bayes data before and let it rebuild, but it doesn't seem to make sense that we've got all this crazy spam, some of which gets caught and some doesn't even though it's the exact same spam (or nearly the same). Is Amavis to blame here? Does it get in the way of Spamassassin running as it should, or is it more just configuration problems on our part? Thanks. Amavis is normally configured for site-wide bayes as the "vscan" or "amavis" user, not per-user. If you're training bayes per-user, but SA is running with a site-wide bayes, you'll get poor results. This is operator error, not the fault of Amavis or SpamAssassin. You must train the proper bayes database. Also, please post a few spam samples (with SA headers) that did not get caught to pastebin.com and give us the links. If we can see the spam and what rules hit, we may be able to give you some more suggestions. -- --asai
Mail Filter Recommendations
Greetings, We've been using Amavis for a number of years, but it seems to not be doing what we need it to be doing regarding spam filtering. e.g. I can't seem to get it to learn bayes data on a per user basis. We have our spam filters turned up so high ( kill level 3 ) for some users it just seems like we're doing something wrong. I have deleted the Bayes data before and let it rebuild, but it doesn't seem to make sense that we've got all this crazy spam, some of which gets caught and some doesn't even though it's the exact same spam (or nearly the same). Is Amavis to blame here? Does it get in the way of Spamassassin running as it should, or is it more just configuration problems on our part? Thanks.
Re: Ready to throw in the towel on email providing...
Make your grass greener than the neighbor's. --Asai On 7/29/14 12:37 PM, Ted Mittelstaedt wrote: What do you do?
Re: Ready to throw in the towel on email providing...
We use the invaluement lists managed by Rob McEwen and have been very happy with them-- been using them for 3-4 years. A lot of blocking that doesn't overlap with Spamhaus, very few false positives, and those that do occur are addressed quickly with a lot of transparency. Well worth the cash, IMO. (And no, I'm pretty sure I'm not getting a discount or anything for this.) :-) +1 I've also been using them for a few years and they do a good job +1 The same. Happy user, no affiliation. Plus Rob is kinda awesome when you need something. Seems like utilizing such a service could really help a small IT company get some leverage on the bigger conglomerates.
Re: Ready to throw in the towel on email providing...
My question regarding all of this interesting topic is, isn't there some kind of RBL or something which can be subscribed to for a nominal fee per year that can aid the small IT shop in maintaining spam filters? --Asai On 7/28/14 9:10 AM, Ted Mittelstaedt wrote: Hi All, Just lost another one, dammit. Small company with about 6 mailboxes who some consultant gave them a song and dance about how Gmail's such a better mail service since "they don't get any spam" No it's not going to break us. But this is what I see happening. SpamAssassin for us filters probably about 80% of the spam out of the box, doing nothing other than using defaults. If the users feed the learner, it's even better. But, the spam is coming in at such a tremendously high volume now that when a user account gets 5,000 pieces of mail a day, all of it except for maybe 5 pieces of mail are NOT spam, even at 99% effectiveness, the user is STILL getting 50 pieces of spam in a day that SpamAssassin misses, compared to their 5 pieces of ham mail. they don't see the 4,950 pieces of mail we deleted for them. They just see the 50 pieces that got past, compared to their 5 legitimate pieces. So naturally the users figure we are rat bastards who aren't doing a good job filtering. So they setup a test account at Google and "try it out for a month" Of course, the account gets very little spam. Why would it otherwise? It's brand new. It hasn't had a chance to be disseminated to all of the mailing lists, the websites, the other coorespondents's computers of theirs that get hit by harvesting viruses. Their ignorance then reinforces their invalid perception and then they figure we are liars. So they move their domain. A year later, when Gmail is doing the same thing to them, they finally figure out it's not the provider, its the spammers and oh boy maybe we weren't lying after all. But, it's a lot of work to shift back to us, so why bother if all the mail services are the same way? So they are gone, permanently, never to return. We have tried educating them. But spamfighting today is complex. If you explain it completely and they understand the explanation and believe you, they give up hope because they realize that just hitting the delete button on those 50 pieces of spam is easier than shifting their poor email behaviors that got them into the mess in the first place. But then a month later the complex explanation is forgotten and they are once more vulnerable to any snake oil sales consultant selling them gmail. But most of them don't understand anyway. And if you just try to dumb down the explanation it starts making no sense at all very quickly. What do other people do? Or are we just going to end up with an Internet in about 10 years where every single email box is either on Microsoft 365 or Gmail and the NSA has a wonderful interface to use to hunt through whatever they want without bothering with a warrant? And to add insult to injury - this small company is a dental office - subject to HIPAA - and Gmail is not (and has stated they will not) be HIPAA complaint. We are! Ted --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com
Re: Filters Don't Seem to Be Learning [SOLVED]
On Wed, 2014-07-23 at 14:34 -0700, Asai wrote: The mail server is running as a different user than amavis, so I ran this under the amavis user: 0.000 0 3 0 non-token data: bayes db version 0.000 0624 0 non-token data: nspam 0.000 0 11919 0 non-token data: nham While that seems a little biased, that's sufficient training with default thresholds of 200 each. 0.000 0 120783 0 non-token data: ntokens 0.000 0 1405862394 0 non-token data: oldest atime 0.000 0 1406151128 0 non-token data: newest atime Good, just a few hours ago. sa-learn is running under amavis as well. Yet, there are no BAYES_xx rules hit, so the Bayesian classifier somehow has been disabled. Check your SA conf with a fine-toothed comb. Grepping the .cf files for 'bayes' should get you most relevant options. In addition, check for the use_learner option. Besides the obvious use_bayes, the use_bayes_rules option seems a likely candidate for your issue. If that still doesn't explain it, it's time for some debugging. Both using the plain 'spamassassin' executable directly, as well as debugging Amavis. Thanks, Karsten. I got Bayes working. It had to do with the MySQL bayes_seen table being in latin1 and not in UTF8. Once I converted the table and data to UTF8, it worked.
Re: Filters Don't Seem to Be Learning
The mail server is running as a different user than amavis, so I ran this under the amavis user: 0.000 0 3 0 non-token data: bayes db version 0.000 0624 0 non-token data: nspam 0.000 0 11919 0 non-token data: nham 0.000 0 120783 0 non-token data: ntokens 0.000 0 1405862394 0 non-token data: oldest atime 0.000 0 1406151128 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 1406122379 0 non-token data: last expiry atime 0.000 0 259996 0 non-token data: last expire atime delta 0.000 0 43711 0 non-token data: last expire reduction count sa-learn is running under amavis as well. --Asai On 7/23/14 12:32 PM, Rick Macdougall wrote: On 2014-07-23 3:05 PM, Asai wrote: Hi, I'm also not seeing any BAYES results in the Spam Status. Are you sure BAYES is turned on ? Regards, Rick That's a good question. As far as I can tell, it is. But is there a way to get a config dump from the command line? I have a feeling that Bayes does not have enough data to turn on (200 spam and 200 ham). Can you do an sa-learn --dump magic and post the results. Make sure you are running it as the same user your mail server runs as. Regards, Rick
Re: Filters Don't Seem to Be Learning
Hi, I'm also not seeing any BAYES results in the Spam Status. Are you sure BAYES is turned on ? Regards, Rick That's a good question. As far as I can tell, it is. But is there a way to get a config dump from the command line?
Re: Filters Don't Seem to Be Learning
Right, the AWL. Well, I think it gets added to the AWL because the spam scanning is failing... which is why it's not getting flagged as spam, and why it gets to my inbox where TB catches it, files it into my junk, and SA learn spams it nightly, but still no dice. --Asai On 7/23/14 10:57 AM, Jeremy McSpadden wrote: As you can see, this message was not flagged as spam. You also have this domain on the AWL per your SA output. X-Spam-Status: No, score=2.558 tagged_above=- required=5 tests=[AWL=-0.337, DCC_CHECK=1.1, DIGEST_MULTIPLE=0.293, DKIM_SIGNED=0.1, HTML_MESSAGE=0.001, PYZOR_CHECK=1.392, RP_MATCHES_RCVD=-0.001, T_DKIM_INVALID=0.01] autolearn=no -- Jeremy McSpadden Flux Labs | http://www.fluxlabs.net <http://www.fluxlabs.net/> | Endless Solutions Office : 850-250-5590x501 | Cell : 850-890-2543 | Fax : 850-254-2955 On Jul 23, 2014, at 12:55 PM, "Asai" <mailto:a...@globalchangemusic.org>> wrote: Thanks for responding. What other info exactly can I provide that will help to troubleshoot this? I also train SA to look at my inbox and learn ham from it. For an example of the spam, an excerpt: "Click here if this email isn't displaying correctly. garden a might are tonight tag update tag. mailman an pickup pod orchestra are france are. otherwise pod community an senior gen france hat. seller gen confirmation are thread hope log hat. server an club a thanks taxi password hope. engineering last honolulu tag herr ram copyrighted gen. dad taxi periodic gen command last periodic a. forward taxi greens taxi pick tag acrobat pod. personalized a otherwise are van gen damage taxi. astrology tag team taxi comic are periodic taxi." And in the headers of this spam message: X-Virus-Scanned: amavisd-new atglobalchangemultimedia.net <http://globalchangemultimedia.net> X-Spam-Flag: NO X-Spam-Score: 2.558 X-Spam-Level: ** X-Spam-Status: No, score=2.558 tagged_above=- required=5 tests=[AWL=-0.337, DCC_CHECK=1.1, DIGEST_MULTIPLE=0.293, DKIM_SIGNED=0.1, HTML_MESSAGE=0.001, PYZOR_CHECK=1.392, RP_MATCHES_RCVD=-0.001, T_DKIM_INVALID=0.01] autolearn=no --Asai On 7/23/14 10:49 AM, Jeremy McSpadden wrote: Would need more info than this; rather vague. If your receiving the same email daily it's more than likely been trained as HAM, but marked as spam through TB. -- Jeremy McSpadden Flux Labs | http://www.fluxlabs.net <http://www.fluxlabs.net/> | Endless Solutions Office : 850-250-5590x501 | Cell : 850-890-2543 | Fax : 850-254-2955 On Jul 23, 2014, at 12:47 PM, "Asai" <mailto:a...@globalchangemusic.org>> wrote: Greetings, I have configured my SA learn spam to check my Junk mailbox every night. In the logs I see that it's actually learning, but daily, I get the very same spams that go straight to my junk mail. The Thunderbird filters seem to be doing a better job of identifying spam in this one situation. Can anyone point me in the right direction on how to catch this spam better? Thanks. -- --Asai
Re: Filters Don't Seem to Be Learning
Thanks for responding. What other info exactly can I provide that will help to troubleshoot this? I also train SA to look at my inbox and learn ham from it. For an example of the spam, an excerpt: "Click here if this email isn't displaying correctly. garden a might are tonight tag update tag. mailman an pickup pod orchestra are france are. otherwise pod community an senior gen france hat. seller gen confirmation are thread hope log hat. server an club a thanks taxi password hope. engineering last honolulu tag herr ram copyrighted gen. dad taxi periodic gen command last periodic a. forward taxi greens taxi pick tag acrobat pod. personalized a otherwise are van gen damage taxi. astrology tag team taxi comic are periodic taxi." And in the headers of this spam message: X-Virus-Scanned: amavisd-new at globalchangemultimedia.net X-Spam-Flag: NO X-Spam-Score: 2.558 X-Spam-Level: ** X-Spam-Status: No, score=2.558 tagged_above=- required=5 tests=[AWL=-0.337, DCC_CHECK=1.1, DIGEST_MULTIPLE=0.293, DKIM_SIGNED=0.1, HTML_MESSAGE=0.001, PYZOR_CHECK=1.392, RP_MATCHES_RCVD=-0.001, T_DKIM_INVALID=0.01] autolearn=no --Asai On 7/23/14 10:49 AM, Jeremy McSpadden wrote: Would need more info than this; rather vague. If your receiving the same email daily it's more than likely been trained as HAM, but marked as spam through TB. -- Jeremy McSpadden Flux Labs | http://www.fluxlabs.net <http://www.fluxlabs.net/> | Endless Solutions Office : 850-250-5590x501 | Cell : 850-890-2543 | Fax : 850-254-2955 On Jul 23, 2014, at 12:47 PM, "Asai" <mailto:a...@globalchangemusic.org>> wrote: Greetings, I have configured my SA learn spam to check my Junk mailbox every night. In the logs I see that it's actually learning, but daily, I get the very same spams that go straight to my junk mail. The Thunderbird filters seem to be doing a better job of identifying spam in this one situation. Can anyone point me in the right direction on how to catch this spam better? Thanks. -- --Asai
Filters Don't Seem to Be Learning
Greetings, I have configured my SA learn spam to check my Junk mailbox every night. In the logs I see that it's actually learning, but daily, I get the very same spams that go straight to my junk mail. The Thunderbird filters seem to be doing a better job of identifying spam in this one situation. Can anyone point me in the right direction on how to catch this spam better? Thanks. -- --Asai
Submitting to RBL
Greetings, I've been looking for ways to submit spams to some RBL, but URIBL seems to not be accepting any submissions at this time. Are there any others that anyone could recommend that I could submit to? Some of the spam that's getting through the filters is just the same thing over and over again, and has a very low spam score, but it's still spam. Although I'm training SA via sa-learn, it's still getting through. Any insights appreciated. Thanks! -- --Asai
Deleting Bayes Data and MySQL Tables
Greetings, We've been running Spamassassin (3.3.1 currently, concurrently with Amavis) using MySQL as a backend for many years now and we have 1 million + entries in the Bayes table. At this time, there seems to be a lot of spam getting through the filters and we currently have our spam level set to 2.5 points for users with the most spam. A couple questions: Does 2.5 seem excessively low? Is it advisable to clear out the Bayes table and start from scratch? If so, would it be wise to raise the level to 4.0 while the Bayes data retrains? Also, since we're using MySQL as a backend the sa cli commands don't seem to work as expected, for example sa-learn --clear doesn't clean out the MySQL database, nor does running sa-learn ... seem to produce any noticeable effect in spam filtering. What am I missing? Thanks for any insights. -- --Asai
Re: report_contact Won't Change
Ok, thank you. I'm using Postfix, Amavisd-new, ClamAV and SQLGrey. Do you know where I would enable or disable receiving this notice in any of these? I've been looking and looking and I can't seem to find anything... John Hardin wrote: > > On Fri, 24 Oct 2008, asai wrote: > >> Well, there is a report_contact setting in SA local.cf...are you saying >> that that is not relevant here? > > It is not. report_contact is for providing an email address for a support > contact in the "this is a spam" report text that is optionally added to a > message that is scored as spam. It does not (at least within SA) generate > any new email messages to that address, or reroute spams to that address. > >> mouss-2 wrote: >>> >>> asai a écrit : >>>> >>>> I've been trying to stop Spamassassin from sending any more spam >>>> notices >>>> to me, >>> >>> SA does not send, block, or route mail. it is an expert who says this is >>> probably spam and this is probably ham. Routing and other transport >>> decisions are taken by whatever tool you use to run SA. > > To reiterate: SA only scores, and may reformat spams. If spammy messages > are being misfiled or alert messages are being sent, then some other tool > is the culprit. Look downstream from SA in your mail processing chain. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >Health Care _is_ a right - the government has no business keeping >you from getting it. But forcing somebody else to pay for your >health care at gunpoint (i.e. through taxation) is _not_ a right. > --- > 11 days until the Presidential Election > -- View this message in context: http://www.nabble.com/report_contact-Won%27t-Change-tp20142945p20154114.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: report_contact Won't Change
Well, there is a report_contact setting in SA local.cf...are you saying that that is not relevant here? mouss-2 wrote: > > asai a écrit : >> Greetings, >> >> I've been trying to stop Spamassassin from sending any more spam notices >> to >> me, so I changed it in /etc/mail/spamassassin/local.cf but I'm still >> getting >> messages sent to the same email address...what am I missing here? > > SA does not send, block, or route mail. it is an expert who says this is > probably spam and this is probably ham. Routing and other transport > decisions are taken by whatever tool you use to run SA. > > -- View this message in context: http://www.nabble.com/report_contact-Won%27t-Change-tp20142945p20153917.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
report_contact Won't Change
Greetings, I've been trying to stop Spamassassin from sending any more spam notices to me, so I changed it in /etc/mail/spamassassin/local.cf but I'm still getting messages sent to the same email address...what am I missing here? -- View this message in context: http://www.nabble.com/report_contact-Won%27t-Change-tp20142945p20142945.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
