Re: From name containing a spoofed email address
See my post of 25/20/2017 to this list. Sent from ProtonMail Mobile On Wed, Jan 17, 2018 at 20:31, David Jones wrote: > Would a plugin need to be created (or an existing one enhanced) to be able to > detect this type of spoofed From header? From: "h...@hulumail.com !" > https://pastebin.com/vVhGjC8H Does anyone else think this would be a good > idea to make a rule that at least checks both the From:name and From:addr to > see if there is an email address in the From:name and if the domain is > different add some points? We are seeing more and more of this now that SPF, > DKIM, and DMARC are making it harder to spoof common/major brands that have > properly implemented some or all of them. -- David Jones @hotmail.com>
Re: From name containing a spoofed email address
>!~ matches are dangerous because they match by default if you >don't anticipate all the legitimate formats. The above will FP on a >simple email address. It could be rewritten as a __FROM_DOMAINS_MATCH >and used in a meta rule. fool me, your are right, RW, thanks... >It's also not a complete solution as it doesn't handle third-level >domains correctly e.g. in > >"supp...@paypal.co.uk" > >"co" will match "co". This is why it's probably best to do it in perl >where the tlds from 20_aux_tlds.cf can be used. you are right as well... but his problem is hard to solve becasue subdomains can be almost unlimitedand even worse... domains can be different but valid, outlook.com and hotmail.com for example.
Re: Turn OFF SA spam filtering but keep ON header examination
Exactly! That is why I want to stick with SA because it does know how to do spf and dkim checks whereas other systems don't unless we install software to do that. On 01/18/2018 07:31 PM, Alan Hodgson wrote: > On Thu, 2018-01-18 at 18:49 -0500, Chip wrote: >> Very well stated. Bravo! >> >> The end point here is to examine the email headers that specifically >> refer to dkim and spf signatures. Based on fail or pass, or some >> combination in concert with the sender's email address, they get moved >> into fail or pass folders. >> >> That's it! >> > > If that's literally all you want to do, then have SpamAssassin score > every message at +50 with a generic local rule, and whitelist_from_spf > or whitelist_from_dkim the ones you want to keep. SA knows how to do > SPF and DKIM. > > Then dump anything that passes SA into the pass folder, everything > else into fail.
Re: Turn OFF SA spam filtering but keep ON header examination
On Thu, 2018-01-18 at 18:49 -0500, Chip wrote: > Very well stated. Bravo! > > The end point here is to examine the email headers that specifically > refer to dkim and spf signatures. Based on fail or pass, or some > combination in concert with the sender's email address, they get moved > into fail or pass folders. > > That's it! > If that's literally all you want to do, then have SpamAssassin score every message at +50 with a generic local rule, and whitelist_from_spf or whitelist_from_dkim the ones you want to keep. SA knows how to do SPF and DKIM. Then dump anything that passes SA into the pass folder, everything else into fail.
Re: Turn OFF SA spam filtering but keep ON header examination
Very well stated. Bravo! The end point here is to examine the email headers that specifically refer to dkim and spf signatures. Based on fail or pass, or some combination in concert with the sender's email address, they get moved into fail or pass folders. That's it! I know there are other methods for doing this - procmail, mimedefang, sieve, etc. etc. etc., but I'm somewhat limited in using those because although they might be intrinsically better for this, they do not play well with the cpane/WHM VPS I use and to which I am married because there are other features of the cpanel/WHM which come into play with this project, that are attractive and not available in a home-grown smtp server box. So using what I have rather than jumping ship and starting from scratch is preferable. I have played a bit with the local.cf and see that I can move into folders depending in the outcome of the examination of the headers. But that is with spamassassin still identifying spam, which is unnecessary in this case as this project will never attract spam - I know that is hard to believe in, but it won't. So the method of using Spam Assassins built in regex procedures for examining emails is attractive. On 01/18/2018 06:24 PM, Alex Woick wrote: > Chip schrieb am 18.01.2018 um 23:43: >> yes I'm starting to see that. I may need to build a box specifically >> suited for this using procmail. I had hoped that I could stay with >> the VPS. >> >> Nevertheless, I've heard two contradictory pieces of advise here and >> would like to know which is correct or most-near correct. >> >> I'm sure there are instances where both pieces of advise work. >> >> someone said change the local.cf to a score of 999 which (I think) means >> that it will override all other spam detection rules. >> >> And then another person suggested to remove all the bundled rules. > I think it isn't clear what your final goal is. What is the goal you > want to achieve, how should the result look like? > > SpamAssassin is something that is inserted into the flow of mail, > usually on a mailserver, and does nothing else than read each mail > that is piped through it and add some headers to each mail. That's it. > It adds headers that tell how that mail was classified by > SpamAssassin: spam or not spam. > > Then, some process that comes after SpamAssassin is able to read these > headers and perform some action upon the mail: for example, it could > move mails to a "junk" folder that were classified as spam and can > leave other mail alone. If you don't implement such process, no action > will be done. > > How SpamAssassin does the classification is determined by the rules > that come with it. It will do analysis of headers, analysis of text > and may consult remote services like spamcop. > > Each rule returns a score. A positive score means the rule says this > is an indication for spam, and a negative score means the rule says > this is an indication for non-spam (ham). All scores are added, and if > the value is above 5, SpamAssassin classifies this mail as spam. If it > is below 5, it classifies the mail not as spam. If you raise this > threshold to 999, SpamAssassin will simply declare no mail as spam any > more, because there are probably never enough matching spam rules that > add up above 999. So doing this is somewhat pointless, and so I get > back to my initial question: what is the result for your users that > you want to produce after your project is finished? > > If you want SpamAssassin classify and mark mail but take no action, > simply fold SpamAssassin into your mail server and let it do the > classification. Each mail will get the SpamAssassin headers but will > be delivered as before. But you can examine the classification by > examining the headers of the mails in your mail reader. If you and > your users find the classification is fine, you can do the next step > and implement actions on the classification. >
Re: Turn OFF SA spam filtering but keep ON header examination
Chip schrieb am 18.01.2018 um 23:43: yes I'm starting to see that. I may need to build a box specifically suited for this using procmail. I had hoped that I could stay with the VPS. Nevertheless, I've heard two contradictory pieces of advise here and would like to know which is correct or most-near correct. I'm sure there are instances where both pieces of advise work. someone said change the local.cf to a score of 999 which (I think) means that it will override all other spam detection rules. And then another person suggested to remove all the bundled rules. I think it isn't clear what your final goal is. What is the goal you want to achieve, how should the result look like? SpamAssassin is something that is inserted into the flow of mail, usually on a mailserver, and does nothing else than read each mail that is piped through it and add some headers to each mail. That's it. It adds headers that tell how that mail was classified by SpamAssassin: spam or not spam. Then, some process that comes after SpamAssassin is able to read these headers and perform some action upon the mail: for example, it could move mails to a "junk" folder that were classified as spam and can leave other mail alone. If you don't implement such process, no action will be done. How SpamAssassin does the classification is determined by the rules that come with it. It will do analysis of headers, analysis of text and may consult remote services like spamcop. Each rule returns a score. A positive score means the rule says this is an indication for spam, and a negative score means the rule says this is an indication for non-spam (ham). All scores are added, and if the value is above 5, SpamAssassin classifies this mail as spam. If it is below 5, it classifies the mail not as spam. If you raise this threshold to 999, SpamAssassin will simply declare no mail as spam any more, because there are probably never enough matching spam rules that add up above 999. So doing this is somewhat pointless, and so I get back to my initial question: what is the result for your users that you want to produce after your project is finished? If you want SpamAssassin classify and mark mail but take no action, simply fold SpamAssassin into your mail server and let it do the classification. Each mail will get the SpamAssassin headers but will be delivered as before. But you can examine the classification by examining the headers of the mails in your mail reader. If you and your users find the classification is fine, you can do the next step and implement actions on the classification.
Re: Turn OFF SA spam filtering but keep ON header examination
Thanks for pointing out Sieve. I'll look into that. It's nice in that it acts on the last procedure - or right before delivery to the mail folder after all the other dirty work has been done. thanks. On 01/18/2018 05:55 PM, Larry Rosenman wrote: > On Thu, Jan 18, 2018 at 05:43:04PM -0500, Chip wrote: >> yes I'm starting to see that. I may need to build a box specifically >> suited for this using procmail. I had hoped that I could stay with the VPS. >> > I'd look at using sieve instead. Procmail has had some issues and not well > maintained. > > (My opinion FWIW)
Re: Turn OFF SA spam filtering but keep ON header examination
On Thu, Jan 18, 2018 at 05:43:04PM -0500, Chip wrote: > yes I'm starting to see that. I may need to build a box specifically > suited for this using procmail. I had hoped that I could stay with the VPS. > I'd look at using sieve instead. Procmail has had some issues and not well maintained. (My opinion FWIW) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: l...@lerctr.org US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
Re: Turn OFF SA spam filtering but keep ON header examination
yes I'm starting to see that. I may need to build a box specifically suited for this using procmail. I had hoped that I could stay with the VPS. Nevertheless, I've heard two contradictory pieces of advise here and would like to know which is correct or most-near correct. I'm sure there are instances where both pieces of advise work. someone said change the local.cf to a score of 999 which (I think) means that it will override all other spam detection rules. And then another person suggested to remove all the bundled rules. This difference of opinion could be attributed to my lack of clarity, inexperience and what I need. I don't want detection of spam however, when I look at the rules in SpamAssassin (regex expressions, for example) for acting on header information, they are easier to write than procmail or other methods, so I wanted to stick with spamassassin since I have some basic knowledge of regex. If, setting local.cf spam detection to 999 stops the detection of, and acting on, spam, that would work. I would simply write some rules for modifying the subject and deal with the sorting elsewhere. I will try that unless someone sees this as not working. On 01/18/2018 05:34 PM, Noel wrote: > On 1/18/2018 2:09 PM, Chip wrote: >> Newbie excited to use the features of SpamAssassin for a new project >> that needs to flag inbound email for sorting into folders (this can be >> done via cpanel-level filtering) based on keywords in headers (header >> search by SA). >> >> This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and >> SpamAssassin version 3.4.1 running on Perl version 5.10.1. >> >> I would like to TURN OFF any and all Spam Identification features and >> only leave behind SpamAssassin's examination of headers and subsequent >> Subject modification based on keywords in headers (such as keywords in >> DKIM or SPF, etc) > Basically all the rules included with SpamAssassin are for spam > identification, and header modification is based on detecting mail > as spam. > > I think SA is poorly suited for your stated purpose of sorting mail > by header keywords. > > If you really really wanted to use SA for this project, you would > need to *remove* all the bundled rules and then add your own header > parsing rules in local.cf, and then SA only knows how to modify the > subject with a spam tag or not at all. > > This seems like more trouble than it's worth for an end product that > doesn't suit your needs very well. There are other established ways > to sort mail with userland imap filters, procmail, seive, etc. > > > Good luck. > > > > > -- Noel Jones >
Re: Turn OFF SA spam filtering but keep ON header examination
On 1/18/2018 2:09 PM, Chip wrote: > Newbie excited to use the features of SpamAssassin for a new project > that needs to flag inbound email for sorting into folders (this can be > done via cpanel-level filtering) based on keywords in headers (header > search by SA). > > This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and > SpamAssassin version 3.4.1 running on Perl version 5.10.1. > > I would like to TURN OFF any and all Spam Identification features and > only leave behind SpamAssassin's examination of headers and subsequent > Subject modification based on keywords in headers (such as keywords in > DKIM or SPF, etc) Basically all the rules included with SpamAssassin are for spam identification, and header modification is based on detecting mail as spam. I think SA is poorly suited for your stated purpose of sorting mail by header keywords. If you really really wanted to use SA for this project, you would need to *remove* all the bundled rules and then add your own header parsing rules in local.cf, and then SA only knows how to modify the subject with a spam tag or not at all. This seems like more trouble than it's worth for an end product that doesn't suit your needs very well. There are other established ways to sort mail with userland imap filters, procmail, seive, etc. Good luck. -- Noel Jones
Re: Turn OFF SA spam filtering but keep ON header examination
Yes I read the basic configuration. Did you read my initial request in which I said I was a newbie? Reading the "basic configuration" has no bearing on the other parts of my inquiry. Perhaps you didn't read that as well? Where I said this was a VPS with several domains? And that there where many, many files contributing to the SpamAssassin rules functions and operation? Did you read that? The basic configuration is just that - basic. I have no idea based on the numerous rules set discovered if the basic configuration will play nicely with a VPS with many domains and user preferences. I would prefer that you do not respond to my inquiries any longer as I consider you to be somewhat of a harasser. Please just exit the virtual door and stay away from my inbox. Thank you. On 01/18/2018 05:20 PM, Reindl Harald wrote: > > > Am 18.01.2018 um 23:17 schrieb Chip: >> Thank you, Sir. >> >> So in my local.cf there is a commented-out rule as follows: >> >> # Set the threshold at which a message is considered spam (default: >> 5.0) >> # >> # required_score 5.0 >> >> setting that required to 999 will over ride the standard rules system >> wide? > > it will set the value "required_score" form 5 to 999 > > did you even read that: > https://wiki.apache.org/spamassassin/BasicConfiguration > >> On 01/18/2018 05:11 PM, David Jones wrote: >>> On 01/18/2018 04:00 PM, Chip wrote: Find this tidbit of information how to find the rules that are loaded with spam assassin: spamassassin --lint -D 2>&1 | grep 'config: read file' I see many, many lines of files. I don't see myself going into all those files and replacing a score of whatever with a 999 or 0. There must be a simpler solution to turning off rules than individually editing each ruleset. >>> >>> You can easily grep and sed the output of the lint command above to >>> generate "score RULE 0.0" and append it to the local.cf. But I am >>> pretty sure this is not want you are wanting to do as it wouldn't be >>> very useful. >>> And in the local.cf there are NO rules. So I'm back to zero here. >>> >>> What specifically are you trying to do? You may want to leave all >>> rules active and simply add some new custom rules to local.cf for your >>> custom needs. >>> >>> If you don't want to block anything and just want to collect as much >>> spam and ham as possible, then set the "required_score 999" in your >>> local.cf > >
Re: Turn OFF SA spam filtering but keep ON header examination
Thank you, Sir. So in my local.cf there is a commented-out rule as follows: # Set the threshold at which a message is considered spam (default: 5.0) # # required_score 5.0 setting that required to 999 will over ride the standard rules system wide? On 01/18/2018 05:11 PM, David Jones wrote: > On 01/18/2018 04:00 PM, Chip wrote: >> Find this tidbit of information how to find the rules that are loaded >> with spam assassin: >> >> spamassassin --lint -D 2>&1 | grep 'config: read file' >> >> I see many, many lines of files. >> >> I don't see myself going into all those files and replacing a score of >> whatever with a 999 or 0. >> >> There must be a simpler solution to turning off rules than individually >> editing each ruleset. >> > > You can easily grep and sed the output of the lint command above to > generate "score RULE 0.0" and append it to the local.cf. But I am > pretty sure this is not want you are wanting to do as it wouldn't be > very useful. > >> And in the local.cf there are NO rules. >> >> So I'm back to zero here. > > What specifically are you trying to do? You may want to leave all > rules active and simply add some new custom rules to local.cf for your > custom needs. > > If you don't want to block anything and just want to collect as much > spam and ham as possible, then set the "required_score 999" in your > local.cf. >
Re: Turn OFF SA spam filtering but keep ON header examination
How better to figure other than asking here? Ummm. Isn't that what this mailing list is about? People helping others? I guess I'm at the wrong place. I'm not asking someone to do my work for me. I'm asking for some advise from people who know more than me without them getting sarcastic or insinuating that I'm lazy. I'm pretty sure there are a number of people on this list who CAN answer a simple question about turning off filtering and perhaps give some cogent advise. How would you suggest I "need to figure out basically how spamassassin works?" Go out and read books? Hire someone? Search the internet? Google? I thought I came to the place where people with more experience than myself could offer up some sage advise without being sarcastic or insinuating that I am lazy. I don't need "experts" to lead me down a dark road and abandon me and then tell me I'm lost. So if you can't simply offer some honest, good, informative advise then please buzz off. On 01/18/2018 05:07 PM, Reindl Harald wrote: > > > Am 18.01.2018 um 23:00 schrieb Chip: >> Find this tidbit of information how to find the rules that are loaded >> with spam assassin: >> >> spamassassin --lint -D 2>&1 | grep 'config: read file' >> >> I see many, many lines of files. >> >> I don't see myself going into all those files and replacing a score of >> whatever with a 999 or 0. >> >> There must be a simpler solution to turning off rules than individually >> editing each ruleset. >> >> And in the local.cf there are NO rules. >> >> So I'm back to zero here > > "local.cf" is not supposed to contain anything - it's *your* local > configuration for overrides - you really need to figure out basically > how spamassassin works not just "i am at zero, do the work of reading > manpages and google around how i set it up" outsourced to a mailing list >
Re: Turn OFF SA spam filtering but keep ON header examination
On 01/18/2018 04:00 PM, Chip wrote: Find this tidbit of information how to find the rules that are loaded with spam assassin: spamassassin --lint -D 2>&1 | grep 'config: read file' I see many, many lines of files. I don't see myself going into all those files and replacing a score of whatever with a 999 or 0. There must be a simpler solution to turning off rules than individually editing each ruleset. You can easily grep and sed the output of the lint command above to generate "score RULE 0.0" and append it to the local.cf. But I am pretty sure this is not want you are wanting to do as it wouldn't be very useful. And in the local.cf there are NO rules. So I'm back to zero here. What specifically are you trying to do? You may want to leave all rules active and simply add some new custom rules to local.cf for your custom needs. If you don't want to block anything and just want to collect as much spam and ham as possible, then set the "required_score 999" in your local.cf. -- David Jones
Re: Turn OFF SA spam filtering but keep ON header examination
Find this tidbit of information how to find the rules that are loaded with spam assassin: spamassassin --lint -D 2>&1 | grep 'config: read file' I see many, many lines of files. I don't see myself going into all those files and replacing a score of whatever with a 999 or 0. There must be a simpler solution to turning off rules than individually editing each ruleset. And in the local.cf there are NO rules. So I'm back to zero here. On 01/18/2018 04:08 PM, David Jones wrote: > On 01/18/2018 03:01 PM, Chip wrote: >> Thank you Shanew for the suggestion. >> >> I'm tied to a Cpanel/WHM VPS which can't be changed. Give that there >> are some restrictions such as the use of Exim. Exim apparently does not >> play nice with mimedefang and only partially nice with procmail - at >> least as I've tested it. I would actually prefer to use procmail, but >> it is a bit of heavy lifting to learn. >> >> The good part of Cpanel/WHM is that it has filters easily employed on a >> per email account basis that triggers off the subject line for >> allocating the email to specific folders. I just need a way to change >> the subject via SA based on what SA finds in the headers. >> >> SA does change the subject (I know, I know, changing the content of an >> email is considered a no-no but in this case we are not talking about >> legalities). In this specific use it's just for internal sorting of >> email into specific folders as appropriate based on a subject line that >> SA will create based on it's examination of the headers, and the sorting >> happens on a per account basis by /etc/vfilters created by Cpanel/WHM in >> an easy-to-use gui interface. >> >> > > Keep in mind that changing the Subject will break DKIM checks on any > downstream mail filters but if your SA instance is the "last stop" > then it won't be a problem. > >> >> On 01/18/2018 03:51 PM, sha...@shanew.net wrote: >>> I can't help but think that you'd be better of using something like >>> procmail, maildrop (part of Courier), or sieve if want you want is >>> sorting without all the overhead of checking for spam. >>> >>> But maybe I'm not understanding what you want to accomplish... >>> >>> On Thu, 18 Jan 2018, Chip wrote: >>> Newbie excited to use the features of SpamAssassin for a new project that needs to flag inbound email for sorting into folders (this can be done via cpanel-level filtering) based on keywords in headers (header search by SA). This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and SpamAssassin version 3.4.1 running on Perl version 5.10.1. I would like to TURN OFF any and all Spam Identification features and only leave behind SpamAssassin's examination of headers and subsequent Subject modification based on keywords in headers (such as keywords in DKIM or SPF, etc) 1) Can this be done, and; 2) What tweaks need to be made to SA in its configuration files to make it happen, and; 3) what else is recommended here. Thank you. >>> >> > >
Re: Turn OFF SA spam filtering but keep ON header examination
Looking in my setup I see local.cf attached to many virtfs as in: /home/virtfs/domain-name/etc/mail/spamassassin/local.cf as well as in: /etc/mail/spamassassin/local.cf When I open these files there are very little rules so can't really see what I must change here? This is a VPS with about 10 domains. On 01/18/2018 04:08 PM, David Jones wrote: > On 01/18/2018 03:01 PM, Chip wrote: >> Thank you Shanew for the suggestion. >> >> I'm tied to a Cpanel/WHM VPS which can't be changed. Give that there >> are some restrictions such as the use of Exim. Exim apparently does not >> play nice with mimedefang and only partially nice with procmail - at >> least as I've tested it. I would actually prefer to use procmail, but >> it is a bit of heavy lifting to learn. >> >> The good part of Cpanel/WHM is that it has filters easily employed on a >> per email account basis that triggers off the subject line for >> allocating the email to specific folders. I just need a way to change >> the subject via SA based on what SA finds in the headers. >> >> SA does change the subject (I know, I know, changing the content of an >> email is considered a no-no but in this case we are not talking about >> legalities). In this specific use it's just for internal sorting of >> email into specific folders as appropriate based on a subject line that >> SA will create based on it's examination of the headers, and the sorting >> happens on a per account basis by /etc/vfilters created by Cpanel/WHM in >> an easy-to-use gui interface. >> >> > > Keep in mind that changing the Subject will break DKIM checks on any > downstream mail filters but if your SA instance is the "last stop" > then it won't be a problem. > >> >> On 01/18/2018 03:51 PM, sha...@shanew.net wrote: >>> I can't help but think that you'd be better of using something like >>> procmail, maildrop (part of Courier), or sieve if want you want is >>> sorting without all the overhead of checking for spam. >>> >>> But maybe I'm not understanding what you want to accomplish... >>> >>> On Thu, 18 Jan 2018, Chip wrote: >>> Newbie excited to use the features of SpamAssassin for a new project that needs to flag inbound email for sorting into folders (this can be done via cpanel-level filtering) based on keywords in headers (header search by SA). This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and SpamAssassin version 3.4.1 running on Perl version 5.10.1. I would like to TURN OFF any and all Spam Identification features and only leave behind SpamAssassin's examination of headers and subsequent Subject modification based on keywords in headers (such as keywords in DKIM or SPF, etc) 1) Can this be done, and; 2) What tweaks need to be made to SA in its configuration files to make it happen, and; 3) what else is recommended here. Thank you. >>> >> > >
Re: Mail flagged as spam on command line getting passed through as ham
Shanew, Checked my logs and modifcation time on the local.cf. I had restarted it. I initially had a single 7 in there, but that was not working so I added all 4. Thanks, Andy On 01/18/2018 02:24 PM, sha...@shanew.net wrote: > Most likely you've forgotten to restart spamd or maybe whatever glue > calls SpamAssassin (amavisd, for example). > > As a side note, if you want it to score 7 regardless of network/bayes > tests (which is what your score line indicates), you can just use > "score SHARK_TANK 7" > > > On Thu, 18 Jan 2018, Andy Howell wrote: > >> I've been getting annoying spams for "Shark Tank". I added a simple >> rule in local.cf to check the subject line: >> >> header SHARK_TANK Subject =~ /\bshark tank\b/i >> score SHARK_TANK 7 7 7 7 >> >> The mail still get through. In my inbox: >> >> X-Spam-Flag: NO >> X-Spam-Score: 4.148 >> X-Spam-Level: >> X-Spam-Status: No, score=4.148 required=6.2 tests=[BAYES_80=2, >> DIET_1=0.001, >> HTML_IMAGE_RATIO_02=0.437, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, >> T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01, T_SPF_TEMPERROR=0.01, >> URIBL_BLACK=1.7] autolearn=no autolearn_force=no >> >> If I pass the mail through spamassasin on the command line, it gets >> flagged as spam: >> >> spamassassin -D < spam-mail-shark-tank.txt >out.txt 2>&1 >> >> In out.txt: >> >> X-Spam-Flag: YES >> X-Spam-Level: >> X-Spam-Status: Yes, score=20.5 required=5.0 tests=BAYES_60,DIET_1, >> >> HTML_IMAGE_RATIO_02,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK, >> >> RCVD_IN_SBL_CSS,SHARK_TANK,SPF_HELO_PASS,T_REMOTE_IMAGE,URIBL_ABUSE_SURBL, >> URIBL_BLACK,URIBL_DBL_SPAM autolearn=spam autolearn_force=no >> version=3.4.1 >> X-Spam-Report: >> * 7.0 SHARK_TANK No description available. >> * 1.2 URIBL_ABUSE_SURBL Contains an URL listed in the ABUSE >> SURBL >> * blocklist >> * [URIs: coloringkidsus.com] >> * 3.3 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus >> SBL-CSS >> * [107.175.23.4 listed in zen.spamhaus.org] >> * 2.5 URIBL_DBL_SPAM Contains a spam URL listed in the DBL >> blocklist >> * [URIs: coloringkidsus.com] >> * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist >> * [URIs: coloringkidsus.com] >> * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record >> * 0.0 DIET_1 BODY: Lose Weight Spam >> * 0.4 HTML_IMAGE_RATIO_02 BODY: HTML has a low ratio of text >> to image area >> * 1.5 BAYES_60 BODY: Bayes spam probability is 60 to 80% >> * [score: 0.7650] >> * 0.0 HTML_MESSAGE BODY: HTML included in message >> * 1.9 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level >> above 50% >> * [cf: 100] * 0.9 RAZOR2_CHECK Listed in Razor2 >> (http://razor.sf.net/ >> * 0.0 T_REMOTE_IMAGE Message contains an external image >> X-Spam-Bayes: bayes=0.7650, N=176(88-0+3), ham=(), spam=(shark, Pill, >> craze) >> >> Any ideas what I'm doing wrong? >> >> Thanks, >> >> Andy >> >> >
Re: Turn OFF SA spam filtering but keep ON header examination
On 01/18/2018 03:01 PM, Chip wrote: Thank you Shanew for the suggestion. I'm tied to a Cpanel/WHM VPS which can't be changed. Give that there are some restrictions such as the use of Exim. Exim apparently does not play nice with mimedefang and only partially nice with procmail - at least as I've tested it. I would actually prefer to use procmail, but it is a bit of heavy lifting to learn. The good part of Cpanel/WHM is that it has filters easily employed on a per email account basis that triggers off the subject line for allocating the email to specific folders. I just need a way to change the subject via SA based on what SA finds in the headers. SA does change the subject (I know, I know, changing the content of an email is considered a no-no but in this case we are not talking about legalities). In this specific use it's just for internal sorting of email into specific folders as appropriate based on a subject line that SA will create based on it's examination of the headers, and the sorting happens on a per account basis by /etc/vfilters created by Cpanel/WHM in an easy-to-use gui interface. Keep in mind that changing the Subject will break DKIM checks on any downstream mail filters but if your SA instance is the "last stop" then it won't be a problem. On 01/18/2018 03:51 PM, sha...@shanew.net wrote: I can't help but think that you'd be better of using something like procmail, maildrop (part of Courier), or sieve if want you want is sorting without all the overhead of checking for spam. But maybe I'm not understanding what you want to accomplish... On Thu, 18 Jan 2018, Chip wrote: Newbie excited to use the features of SpamAssassin for a new project that needs to flag inbound email for sorting into folders (this can be done via cpanel-level filtering) based on keywords in headers (header search by SA). This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and SpamAssassin version 3.4.1 running on Perl version 5.10.1. I would like to TURN OFF any and all Spam Identification features and only leave behind SpamAssassin's examination of headers and subsequent Subject modification based on keywords in headers (such as keywords in DKIM or SPF, etc) 1) Can this be done, and; 2) What tweaks need to be made to SA in its configuration files to make it happen, and; 3) what else is recommended here. Thank you. -- David Jones
Re: Turn OFF SA spam filtering but keep ON header examination
Thank you Shanew for the suggestion. I'm tied to a Cpanel/WHM VPS which can't be changed. Give that there are some restrictions such as the use of Exim. Exim apparently does not play nice with mimedefang and only partially nice with procmail - at least as I've tested it. I would actually prefer to use procmail, but it is a bit of heavy lifting to learn. The good part of Cpanel/WHM is that it has filters easily employed on a per email account basis that triggers off the subject line for allocating the email to specific folders. I just need a way to change the subject via SA based on what SA finds in the headers. SA does change the subject (I know, I know, changing the content of an email is considered a no-no but in this case we are not talking about legalities). In this specific use it's just for internal sorting of email into specific folders as appropriate based on a subject line that SA will create based on it's examination of the headers, and the sorting happens on a per account basis by /etc/vfilters created by Cpanel/WHM in an easy-to-use gui interface. On 01/18/2018 03:51 PM, sha...@shanew.net wrote: > I can't help but think that you'd be better of using something like > procmail, maildrop (part of Courier), or sieve if want you want is > sorting without all the overhead of checking for spam. > > But maybe I'm not understanding what you want to accomplish... > > On Thu, 18 Jan 2018, Chip wrote: > >> Newbie excited to use the features of SpamAssassin for a new project >> that needs to flag inbound email for sorting into folders (this can be >> done via cpanel-level filtering) based on keywords in headers (header >> search by SA). >> >> This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and >> SpamAssassin version 3.4.1 running on Perl version 5.10.1. >> >> I would like to TURN OFF any and all Spam Identification features and >> only leave behind SpamAssassin's examination of headers and subsequent >> Subject modification based on keywords in headers (such as keywords in >> DKIM or SPF, etc) >> >> 1) Can this be done, and; >> >> 2) What tweaks need to be made to SA in its configuration files to make >> it happen, and; >> >> 3) what else is recommended here. >> >> Thank you. >> >
Re: Turn OFF SA spam filtering but keep ON header examination
I can't help but think that you'd be better of using something like procmail, maildrop (part of Courier), or sieve if want you want is sorting without all the overhead of checking for spam. But maybe I'm not understanding what you want to accomplish... On Thu, 18 Jan 2018, Chip wrote: Newbie excited to use the features of SpamAssassin for a new project that needs to flag inbound email for sorting into folders (this can be done via cpanel-level filtering) based on keywords in headers (header search by SA). This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and SpamAssassin version 3.4.1 running on Perl version 5.10.1. I would like to TURN OFF any and all Spam Identification features and only leave behind SpamAssassin's examination of headers and subsequent Subject modification based on keywords in headers (such as keywords in DKIM or SPF, etc) 1) Can this be done, and; 2) What tweaks need to be made to SA in its configuration files to make it happen, and; 3) what else is recommended here. Thank you. -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT CompSci =--+--- All syllogisms contain three lines | sha...@shanew.net Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: Turn OFF SA spam filtering but keep ON header examination
On 01/18/2018 02:33 PM, Chip wrote: That sounds doable. If I score everything 0 or 999 will things be overwritten in local.cf on update or elsewhere? The local.cf is yours to update and does not get touched by upgrades or ruleset updates. What you are suggesting sounds like a reasonable course of action. Sounds like you need to play with SA for a bit to understand how it works then setup MDA rules to sort into folders. Keep in mind, SA doesn't actually block anything. It just creates a score and whatever calls SA (known as the glue often on this mailing list) is responsible for taking action based on that score or rule hits. What is your glue? amavisd? That's where you need to start and then do some reading on the documentation for that glue. All of them will have a score required for blocking that usually takes the SA default of 5.0 or maybe 6.0 like MailScanner. If you set that required score to 999 then nothing will be blocked by the glue to let everything in for sorting. On 01/18/2018 03:29 PM, David Jones wrote: On 01/18/2018 02:09 PM, Chip wrote: Newbie excited to use the features of SpamAssassin for a new project that needs to flag inbound email for sorting into folders (this can be done via cpanel-level filtering) based on keywords in headers (header search by SA). This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and SpamAssassin version 3.4.1 running on Perl version 5.10.1. I would like to TURN OFF any and all Spam Identification features and only leave behind SpamAssassin's examination of headers and subsequent Subject modification based on keywords in headers (such as keywords in DKIM or SPF, etc) 1) Can this be done, and; 2) What tweaks need to be made to SA in its configuration files to make it happen, and; 3) what else is recommended here. Thank you. Not exactly sure what you want to disable but setting a score of 0 will disable a rule. You may need to gather up a list of all rules and score most of them 0 in your local.cf. You might just set the required score to 999 in whatever is launching spamassassin so it doesn't block anything. Then have MDA (Dovecot sieve) rules to sort into folders based on hits in the X-Spam-Status header. I do something similar for my spamassassin masscheck box where I intentionally let down my defenses at the MTA not using any RBLs and then sort messages into a Ham or Spam folder based on score and rule hits. -- David Jones
Re: Turn OFF SA spam filtering but keep ON header examination
That sounds doable. If I score everything 0 or 999 will things be overwritten in local.cf on update or elsewhere? What you are suggesting sounds like a reasonable course of action. On 01/18/2018 03:29 PM, David Jones wrote: > On 01/18/2018 02:09 PM, Chip wrote: >> Newbie excited to use the features of SpamAssassin for a new project >> that needs to flag inbound email for sorting into folders (this can be >> done via cpanel-level filtering) based on keywords in headers (header >> search by SA). >> >> This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and >> SpamAssassin version 3.4.1 running on Perl version 5.10.1. >> >> I would like to TURN OFF any and all Spam Identification features and >> only leave behind SpamAssassin's examination of headers and subsequent >> Subject modification based on keywords in headers (such as keywords in >> DKIM or SPF, etc) >> >> 1) Can this be done, and; >> >> 2) What tweaks need to be made to SA in its configuration files to make >> it happen, and; >> >> 3) what else is recommended here. >> >> Thank you. >> > > Not exactly sure what you want to disable but setting a score of 0 > will disable a rule. You may need to gather up a list of all rules > and score most of them 0 in your local.cf. > > You might just set the required score to 999 in whatever is launching > spamassassin so it doesn't block anything. Then have MDA (Dovecot > sieve) rules to sort into folders based on hits in the X-Spam-Status > header. > > I do something similar for my spamassassin masscheck box where I > intentionally let down my defenses at the MTA not using any RBLs and > then sort messages into a Ham or Spam folder based on score and rule > hits. >
Re: Turn OFF SA spam filtering but keep ON header examination
On 01/18/2018 02:09 PM, Chip wrote: Newbie excited to use the features of SpamAssassin for a new project that needs to flag inbound email for sorting into folders (this can be done via cpanel-level filtering) based on keywords in headers (header search by SA). This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and SpamAssassin version 3.4.1 running on Perl version 5.10.1. I would like to TURN OFF any and all Spam Identification features and only leave behind SpamAssassin's examination of headers and subsequent Subject modification based on keywords in headers (such as keywords in DKIM or SPF, etc) 1) Can this be done, and; 2) What tweaks need to be made to SA in its configuration files to make it happen, and; 3) what else is recommended here. Thank you. Not exactly sure what you want to disable but setting a score of 0 will disable a rule. You may need to gather up a list of all rules and score most of them 0 in your local.cf. You might just set the required score to 999 in whatever is launching spamassassin so it doesn't block anything. Then have MDA (Dovecot sieve) rules to sort into folders based on hits in the X-Spam-Status header. I do something similar for my spamassassin masscheck box where I intentionally let down my defenses at the MTA not using any RBLs and then sort messages into a Ham or Spam folder based on score and rule hits. -- David Jones
Re: Mail flagged as spam on command line getting passed through as ham
Most likely you've forgotten to restart spamd or maybe whatever glue calls SpamAssassin (amavisd, for example). As a side note, if you want it to score 7 regardless of network/bayes tests (which is what your score line indicates), you can just use "score SHARK_TANK 7" On Thu, 18 Jan 2018, Andy Howell wrote: I've been getting annoying spams for "Shark Tank". I added a simple rule in local.cf to check the subject line: header SHARK_TANK Subject =~ /\bshark tank\b/i score SHARK_TANK 7 7 7 7 The mail still get through. In my inbox: X-Spam-Flag: NO X-Spam-Score: 4.148 X-Spam-Level: X-Spam-Status: No, score=4.148 required=6.2 tests=[BAYES_80=2, DIET_1=0.001, HTML_IMAGE_RATIO_02=0.437, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01, T_SPF_TEMPERROR=0.01, URIBL_BLACK=1.7] autolearn=no autolearn_force=no If I pass the mail through spamassasin on the command line, it gets flagged as spam: spamassassin -D < spam-mail-shark-tank.txt >out.txt 2>&1 In out.txt: X-Spam-Flag: YES X-Spam-Level: X-Spam-Status: Yes, score=20.5 required=5.0 tests=BAYES_60,DIET_1, HTML_IMAGE_RATIO_02,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK, RCVD_IN_SBL_CSS,SHARK_TANK,SPF_HELO_PASS,T_REMOTE_IMAGE,URIBL_ABUSE_SURBL, URIBL_BLACK,URIBL_DBL_SPAM autolearn=spam autolearn_force=no version=3.4.1 X-Spam-Report: * 7.0 SHARK_TANK No description available. * 1.2 URIBL_ABUSE_SURBL Contains an URL listed in the ABUSE SURBL * blocklist * [URIs: coloringkidsus.com] * 3.3 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS * [107.175.23.4 listed in zen.spamhaus.org] * 2.5 URIBL_DBL_SPAM Contains a spam URL listed in the DBL blocklist * [URIs: coloringkidsus.com] * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: coloringkidsus.com] * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record * 0.0 DIET_1 BODY: Lose Weight Spam * 0.4 HTML_IMAGE_RATIO_02 BODY: HTML has a low ratio of text to image area * 1.5 BAYES_60 BODY: Bayes spam probability is 60 to 80% * [score: 0.7650] * 0.0 HTML_MESSAGE BODY: HTML included in message * 1.9 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% * [cf: 100] * 0.9 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/ * 0.0 T_REMOTE_IMAGE Message contains an external image X-Spam-Bayes: bayes=0.7650, N=176(88-0+3), ham=(), spam=(shark, Pill, craze) Any ideas what I'm doing wrong? Thanks, Andy -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT CompSci =--+--- All syllogisms contain three lines | sha...@shanew.net Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Turn OFF SA spam filtering but keep ON header examination
Newbie excited to use the features of SpamAssassin for a new project that needs to flag inbound email for sorting into folders (this can be done via cpanel-level filtering) based on keywords in headers (header search by SA). This is a Centos 6.9 machine running cpanel/WHM 11.68.0.23 and SpamAssassin version 3.4.1 running on Perl version 5.10.1. I would like to TURN OFF any and all Spam Identification features and only leave behind SpamAssassin's examination of headers and subsequent Subject modification based on keywords in headers (such as keywords in DKIM or SPF, etc) 1) Can this be done, and; 2) What tweaks need to be made to SA in its configuration files to make it happen, and; 3) what else is recommended here. Thank you.
Mail flagged as spam on command line getting passed through as ham
I've been getting annoying spams for "Shark Tank". I added a simple rule in local.cf to check the subject line: header SHARK_TANK Subject =~ /\bshark tank\b/i score SHARK_TANK 7 7 7 7 The mail still get through. In my inbox: X-Spam-Flag: NO X-Spam-Score: 4.148 X-Spam-Level: X-Spam-Status: No, score=4.148 required=6.2 tests=[BAYES_80=2, DIET_1=0.001, HTML_IMAGE_RATIO_02=0.437, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, T_REMOTE_IMAGE=0.01, T_RP_MATCHES_RCVD=-0.01, T_SPF_TEMPERROR=0.01, URIBL_BLACK=1.7] autolearn=no autolearn_force=no If I pass the mail through spamassasin on the command line, it gets flagged as spam: spamassassin -D < spam-mail-shark-tank.txt >out.txt 2>&1 In out.txt: X-Spam-Flag: YES X-Spam-Level: X-Spam-Status: Yes, score=20.5 required=5.0 tests=BAYES_60,DIET_1, HTML_IMAGE_RATIO_02,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK, RCVD_IN_SBL_CSS,SHARK_TANK,SPF_HELO_PASS,T_REMOTE_IMAGE,URIBL_ABUSE_SURBL, URIBL_BLACK,URIBL_DBL_SPAM autolearn=spam autolearn_force=no version=3.4.1 X-Spam-Report: * 7.0 SHARK_TANK No description available. * 1.2 URIBL_ABUSE_SURBL Contains an URL listed in the ABUSE SURBL * blocklist * [URIs: coloringkidsus.com] * 3.3 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS * [107.175.23.4 listed in zen.spamhaus.org] * 2.5 URIBL_DBL_SPAM Contains a spam URL listed in the DBL blocklist * [URIs: coloringkidsus.com] * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: coloringkidsus.com] * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record * 0.0 DIET_1 BODY: Lose Weight Spam * 0.4 HTML_IMAGE_RATIO_02 BODY: HTML has a low ratio of text to image area * 1.5 BAYES_60 BODY: Bayes spam probability is 60 to 80% * [score: 0.7650] * 0.0 HTML_MESSAGE BODY: HTML included in message * 1.9 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% * [cf: 100] * 0.9 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/ * 0.0 T_REMOTE_IMAGE Message contains an external image X-Spam-Bayes: bayes=0.7650, N=176(88-0+3), ham=(), spam=(shark, Pill, craze) Any ideas what I'm doing wrong? Thanks, Andy
Re: From name containing a spoofed email address
On Thu, 18 Jan 2018, RW wrote: I think the hard part is handling IDNs, e.g. "=?UTF-8?B?Zm9vQGLDvGNoZXIuY29t?=" the display name should decode to the UTF-8 byte sequence for foo@bücher.com, but I presume the address would be left as the ASCII IDN. In the short term it's probably best to avoid matching on IDNs, but that does allow the use of homographs in spoofing ASCII domains. Yeah, that occured to me, and I decided to set that problem aside for now (probably someone more familiar with the issues should address it). BTW it's best to only match on the organizational domain, to avoid FPs on the likes of: Do you (or anyone, for that matter) have samples of emails like this that they could share for me to test against? -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT CompSci =--+--- All syllogisms contain three lines | sha...@shanew.net Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: From name containing a spoofed email address
On Thu, 18 Jan 2018 11:52:36 + (UTC) Pedro David Marco wrote: > David, > This rule can do the full job... i have tested it with good > results.. (Can be tested here: https://regex101.com/r/Vpmhjz/3 ) It > checks if the level domain next to the TLD in the From:name matches > the domain next to the TLD in From:email header > FROM_DOMAINS_MISMATCH > From !~ > /(?:[^<].+?)\@(?:.+?\.)*?(.+?\.)(?:.+?).*?<.+?(\@\1|\@.*?\.\1)/describe > FROM_DOMAINS_MISMATCH Domain name mismatch in From header !~ matches are dangerous because they match by default if you don't anticipate all the legitimate formats. The above will FP on a simple email address. It could be rewritten as a __FROM_DOMAINS_MATCH and used in a meta rule. It's also not a complete solution as it doesn't handle third-level domains correctly e.g. in "supp...@paypal.co.uk" "co" will match "co". This is why it's probably best to do it in perl where the tlds from 20_aux_tlds.cf can be used.
Re: From name containing a spoofed email address
David, This rule can do the full job... i have tested it with good results.. (Can be tested here: https://regex101.com/r/Vpmhjz/3 ) It checks if the level domain next to the TLD in the From:name matches the domain next to the TLD in From:email header FROM_DOMAINS_MISMATCH From !~ /(?:[^<].+?)\@(?:.+?\.)*?(.+?\.)(?:.+?).*?<.+?(\@\1|\@.*?\.\1)/describe FROM_DOMAINS_MISMATCH Domain name mismatch in From header >Would a plugin need to be created (or an existing one enhanced) to be >able to detect this type of spoofed From header? >From: "h...@hulumail.com !" >https://pastebin.com/vVhGjC8H >>Does anyone else think this would be a good idea to make a rule that at >least checks both the From:name and From:addr to see if there is an >email address in the From:name and if the domain is different add some >points? >We are seeing more and more of this now that SPF, DKIM, and DMARC are >making it harder to spoof common/major brands that have properly >implemented some or all of them. >-- >David Jones --PedroD
