Re: How to get rid of this spam? Spam assassin does not catch it
Well… I’m glad I’m on this mailing list :P I did the same thing, running sa-learn —spam /spamfolder as root, and was pondering this very issue. I understand the logic behind why it shouldn’t be run as root, the problem is on FC 22 the spamd user has /sbin/nologin as the shell in /etc/passwd. Which means in order to run the process as spamd one has to manual change that to /bin/bash, then, change it back (/sbin/nologin it self is a security precaution), once the process is complete. This seems convoluted. I know sa-learn has -u option but that simply changes the user name in the environment (does not sudo), is there a better way to do this? Have i missed something? Shawn > On Oct 31, 2015, at 8:14 AM, Reindl Haraldwrote: > > > > Am 31.10.2015 um 16:06 schrieb j...@lexoncom.com: >> So after initial learning it looks better now. (BAYES_50) > > BAYES_50 is not really good for clear spam > >> When sendmail sends email to procmail and procmail passes it to spam >> assassin, does spam assassin runs as root user or as the user the email >> is destined to? > > depends on how SA is called in detail, normally it should switch to that > unix-user and hence training as root makes no sense, *nothing* should proceed > potentially dangerous input as root at all - inbound mailcontent is by > definition that sort of "don#t do that" input > >> I run the sa-learn as root user > > oh my god... > >> and it seems like this is the data based >> that is being used so it would be global data base used for all mail >> users? > > https://wiki.apache.org/spamassassin/SiteWideBayesSetup > >> X-Spam-Flag: YES >> X-Spam-Level: >> X-Spam-Status: Yes, score=12.9 required=5.0 tests=BAYES_50,FROM_12LTRDOM, >> HTML_MESSAGE,MIME_HTML_ONLY,RCVD_IN_BRBL_LASTEXT,RCVD_IN_MSPIKE_BL, >> RCVD_IN_MSPIKE_L5,RCVD_IN_XBL,RDNS_NONE,URIBL_BLACK,URIBL_DBL_SPAM, >> URIBL_JP_SURBL,URIBL_WS_SURBL autolearn=disabled version=3.4.0 > > well, the quota of your sa-headers was enough to reject my repsonse on the > submission spamass-milter > > result: Y 16 - URIBL_BLACK,URIBL_DBL_SPAM,URIBL_JP_SURBL,URIBL_WS_SURBL >
Re: How to get rid of this spam? Spam assassin does not catch it
On 11/02/2015 04:38 PM, Shaheen Bakhtiar wrote: Well… I’m glad I’m on this mailing list :P I did the same thing, running sa-learn —spam /spamfolder as root, and was pondering this very issue. I understand the logic behind why it shouldn’t be run as root, the problem is on FC 22 the spamd user has /sbin/nologin as the shell in /etc/passwd. Which means in order to run the process as spamd one has to manual change that to /bin/bash, then, change it back (/sbin/nologin it self is a security precaution), once the process is complete. This seems convoluted. I know sa-learn has -u option but that simply changes the user name in the environment (does not sudo), is there a better way to do this? Have i missed something? Shawn Assuming you're using file based Bayes DB in local.cf add: bayes_path /path_to/bayes/bayes then you can learn as root . h2h
Re: How to get rid of this spam? Spam assassin does not catch it
Am 02.11.2015 um 16:42 schrieb Axb: On 11/02/2015 04:38 PM, Shaheen Bakhtiar wrote: Well… I’m glad I’m on this mailing list :P I did the same thing, running sa-learn —spam /spamfolder as root, and was pondering this very issue. I understand the logic behind why it shouldn’t be run as root, the problem is on FC 22 the spamd user has /sbin/nologin as the shell in /etc/passwd. Which means in order to run the process as spamd one has to manual change that to /bin/bash, then, change it back (/sbin/nologin it self is a security precaution), once the process is complete. This seems convoluted. I know sa-learn has -u option but that simply changes the user name in the environment (does not sudo), is there a better way to do this? Have i missed something? Shawn Assuming you're using file based Bayes DB in local.cf add: bayes_path /path_to/bayes/bayes then you can learn as root why should somebody do this after configure site_wide bayes like above instead set the permissions and put a restricted user for sa-learn in the group with writre permissions? signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
Ah! I see… that makes sense.. but spamc reads one mail at a time, is there way (other than writing a script) to have it read a folder full of emails? > On Nov 2, 2015, at 8:02 AM, Benny Pedersenwrote: > > Axb skrev den 2015-11-02 16:42: >> On 11/02/2015 04:38 PM, Shaheen Bakhtiar wrote: >>> Well… I’m glad I’m on this mailing list :P >>> I did the same thing, running sa-learn —spam /spamfolder as root, and >>> was pondering this very issue. >>> I understand the logic behind why it shouldn’t be run as root, the >>> problem is on FC 22 the spamd user has /sbin/nologin as the shell in >>> /etc/passwd. Which means in order to run the process as spamd one has >>> to manual change that to /bin/bash, then, change it back >>> (/sbin/nologin it self is a security precaution), once the process is >>> complete. > > no you should use spamc not sa-learn > >>> This seems convoluted. >>> I know sa-learn has -u option but that simply changes the user name >>> in the environment (does not sudo), is there a better way to do this? >>> Have i missed something? > > sa-learn is using user-prefs, also for root if it exists, search for it in > $HOME > >>> Shawn >> Assuming you're using file based Bayes DB >> in local.cf add: >> bayes_path /path_to/bayes/bayes >> then you can learn as root . >> h2h > > for global bayes yes, but for non global bayes its better in user_prefs file > > and why did he change spamd login permisson when using sa-learn :( > > use spamc, not spamd if spamc is not used > > on does not need to login to apache for see a homepage, same goes for spamd, > it is using port 783 so it need to be started as root, but the real work will > happend as the user calling spamc
Re: How to get rid of this spam? Spam assassin does not catch it
After retraining and setting spam assassin for wide site all looks good. Spam gets bayes99 and non spam is bayes00. So far i did not get any spam. Thank you all for your help. >> >> >> Am 31.10.2015 um 16:06 schrieb j...@lexoncom.com: >>> So after initial learning it looks better now. (BAYES_50) >> >> BAYES_50 is not really good for clear spam >> > yep i though that bayes was used but it seems like it was all useless > >>> When sendmail sends email to procmail and procmail passes it to spam >>> assassin, does spam assassin runs as root user or as the user the email >>> is destined to? >> >> depends on how SA is called in detail, normally it should switch to that >> unix-user and hence training as root makes no sense, *nothing* should >> proceed potentially dangerous input as root at all - inbound mailcontent >> is by definition that sort of "don#t do that" input >> >>> I run the sa-learn as root user >> >> oh my god... > i run it through the crontab > yes i can create new user and force sa-learn to use that user >> >>> and it seems like this is the data based >>> that is being used so it would be global data base used for all mail >>> users? >> >> https://wiki.apache.org/spamassassin/SiteWideBayesSetup > > i switched to global setup > now all users should use same db > and i will use the manual learning process >> >>> X-Spam-Flag: YES >>> X-Spam-Level: >>> X-Spam-Status: Yes, score=12.9 required=5.0 >>> tests=BAYES_50,FROM_12LTRDOM, >>> >>> HTML_MESSAGE,MIME_HTML_ONLY,RCVD_IN_BRBL_LASTEXT,RCVD_IN_MSPIKE_BL, >>> >>> RCVD_IN_MSPIKE_L5,RCVD_IN_XBL,RDNS_NONE,URIBL_BLACK,URIBL_DBL_SPAM, >>> URIBL_JP_SURBL,URIBL_WS_SURBL autolearn=disabled version=3.4.0 >> >> well, the quota of your sa-headers was enough to reject my repsonse on >> the submission spamass-milter >> >> result: Y 16 - URIBL_BLACK,URIBL_DBL_SPAM,URIBL_JP_SURBL,URIBL_WS_SURBL >> >> > no sure what this means? > >
Re: How to get rid of this spam? Spam assassin does not catch it
Am 02.11.2015 um 17:02 schrieb Benny Pedersen: and why did he change spamd login permisson when using sa-learn :( because *as he explained* the service user has /sbin/nologin as shell and so "su - username" won't work until you change that or as i explained create a user with a shell training the correct site wide bayes use spamc, not spamd if spamc is not used on does not need to login to apache for see a homepage, same goes for spamd, it is using port 783 so it need to be started as root, but the real work will happend as the user calling spamc signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
On 11/02/2015 05:21 PM, Shaheen Bakhtiar wrote: Ah! I see… that makes sense.. but spamc reads one mail at a time, is there way (other than writing a script) to have it read a folder full of emails? http://spamassassin.apache.org/full/3.4.x/doc/sa-learn.txt and bookmark http://spamassassin.apache.org/full/3.4.x/doc/ make that your first stop before you ask for help On Nov 2, 2015, at 8:02 AM, Benny Pedersenwrote: Axb skrev den 2015-11-02 16:42: On 11/02/2015 04:38 PM, Shaheen Bakhtiar wrote: Well… I’m glad I’m on this mailing list :P I did the same thing, running sa-learn —spam /spamfolder as root, and was pondering this very issue. I understand the logic behind why it shouldn’t be run as root, the problem is on FC 22 the spamd user has /sbin/nologin as the shell in /etc/passwd. Which means in order to run the process as spamd one has to manual change that to /bin/bash, then, change it back (/sbin/nologin it self is a security precaution), once the process is complete. no you should use spamc not sa-learn This seems convoluted. I know sa-learn has -u option but that simply changes the user name in the environment (does not sudo), is there a better way to do this? Have i missed something? sa-learn is using user-prefs, also for root if it exists, search for it in $HOME Shawn Assuming you're using file based Bayes DB in local.cf add: bayes_path /path_to/bayes/bayes then you can learn as root . h2h for global bayes yes, but for non global bayes its better in user_prefs file and why did he change spamd login permisson when using sa-learn :( use spamc, not spamd if spamc is not used on does not need to login to apache for see a homepage, same goes for spamd, it is using port 783 so it need to be started as root, but the real work will happend as the user calling spamc
Re: How to get rid of this spam? Spam assassin does not catch it
On Mon, 2 Nov 2015 07:38:57 -0800 Shaheen Bakhtiar wrote: > Well? I?m glad I?m on this mailing list :P > > I did the same thing, running sa-learn ?spam /spamfolder as root, and > was pondering this very issue. > > I understand the logic behind why it shouldn?t be run as root, the > problem is on FC 22 the spamd user has /sbin/nologin as the shell > in /etc/passwd. Which means in order to run the process as spamd one > has to manual change that to /bin/bash, then, change it back > (/sbin/nologin it self is a security precaution), once the process is > complete. su -m will run with the current shell
Re: How to get rid of this spam? Spam assassin does not catch it
Axb skrev den 2015-11-02 16:42: On 11/02/2015 04:38 PM, Shaheen Bakhtiar wrote: Well… I’m glad I’m on this mailing list :P I did the same thing, running sa-learn —spam /spamfolder as root, and was pondering this very issue. I understand the logic behind why it shouldn’t be run as root, the problem is on FC 22 the spamd user has /sbin/nologin as the shell in /etc/passwd. Which means in order to run the process as spamd one has to manual change that to /bin/bash, then, change it back (/sbin/nologin it self is a security precaution), once the process is complete. no you should use spamc not sa-learn This seems convoluted. I know sa-learn has -u option but that simply changes the user name in the environment (does not sudo), is there a better way to do this? Have i missed something? sa-learn is using user-prefs, also for root if it exists, search for it in $HOME Shawn Assuming you're using file based Bayes DB in local.cf add: bayes_path /path_to/bayes/bayes then you can learn as root . h2h for global bayes yes, but for non global bayes its better in user_prefs file and why did he change spamd login permisson when using sa-learn :( use spamc, not spamd if spamc is not used on does not need to login to apache for see a homepage, same goes for spamd, it is using port 783 so it need to be started as root, but the real work will happend as the user calling spamc
Re: How to get rid of this spam? Spam assassin does not catch it
On 11/2/2015 11:25 AM, Reindl Harald wrote: Am 02.11.2015 um 17:02 schrieb Benny Pedersen: and why did he change spamd login permisson when using sa-learn :( because *as he explained* the service user has /sbin/nologin as shell and so "su - username" won't work until you change that or as i explained create a user with a shell training the correct site wide bayes use spamc, not spamd if spamc is not used on does not need to login to apache for see a homepage, same goes for spamd, it is using port 783 so it need to be started as root, but the real work will happend as the user calling spamc I would at least consider sudo or 'su -c' as well.
Re: How to get rid of this spam? Spam assassin does not catch it
> > > Am 31.10.2015 um 16:06 schrieb j...@lexoncom.com: >> So after initial learning it looks better now. (BAYES_50) > > BAYES_50 is not really good for clear spam > yep i though that bayes was used but it seems like it was all useless >> When sendmail sends email to procmail and procmail passes it to spam >> assassin, does spam assassin runs as root user or as the user the email >> is destined to? > > depends on how SA is called in detail, normally it should switch to that > unix-user and hence training as root makes no sense, *nothing* should > proceed potentially dangerous input as root at all - inbound mailcontent > is by definition that sort of "don#t do that" input > >> I run the sa-learn as root user > > oh my god... i run it through the crontab yes i can create new user and force sa-learn to use that user > >> and it seems like this is the data based >> that is being used so it would be global data base used for all mail >> users? > > https://wiki.apache.org/spamassassin/SiteWideBayesSetup i switched to global setup now all users should use same db and i will use the manual learning process > >> X-Spam-Flag: YES >> X-Spam-Level: >> X-Spam-Status: Yes, score=12.9 required=5.0 >> tests=BAYES_50,FROM_12LTRDOM, >> >> HTML_MESSAGE,MIME_HTML_ONLY,RCVD_IN_BRBL_LASTEXT,RCVD_IN_MSPIKE_BL, >> >> RCVD_IN_MSPIKE_L5,RCVD_IN_XBL,RDNS_NONE,URIBL_BLACK,URIBL_DBL_SPAM, >> URIBL_JP_SURBL,URIBL_WS_SURBL autolearn=disabled version=3.4.0 > > well, the quota of your sa-headers was enough to reject my repsonse on > the submission spamass-milter > > result: Y 16 - URIBL_BLACK,URIBL_DBL_SPAM,URIBL_JP_SURBL,URIBL_WS_SURBL > > no sure what this means?
Re: How to get rid of this spam? Spam assassin does not catch it
So after initial learning it looks better now. (BAYES_50) When sendmail sends email to procmail and procmail passes it to spam assassin, does spam assassin runs as root user or as the user the email is destined to? I run the sa-learn as root user and it seems like this is the data based that is being used so it would be global data base used for all mail users? X-Spam-Flag: YES X-Spam-Level: X-Spam-Status: Yes, score=12.9 required=5.0 tests=BAYES_50,FROM_12LTRDOM, HTML_MESSAGE,MIME_HTML_ONLY,RCVD_IN_BRBL_LASTEXT,RCVD_IN_MSPIKE_BL, RCVD_IN_MSPIKE_L5,RCVD_IN_XBL,RDNS_NONE,URIBL_BLACK,URIBL_DBL_SPAM, URIBL_JP_SURBL,URIBL_WS_SURBL autolearn=disabled version=3.4.0 X-Spam-Report: * 1.6 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist * [URIs: curingaidtrade.com] * 1.2 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist * [URIs: curingaidtrade.com] * 1.4 RCVD_IN_BRBL_LASTEXT RBL: No description available. * [95.128.19.6 listed in bb.barracudacentral.org] * 1.7 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist * [URIs: curingaidtrade.com] * 0.4 RCVD_IN_XBL RBL: Received via a relay in Spamhaus XBL * [95.128.19.6 listed in zen.spamhaus.org] * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: curingaidtrade.com] * 2.4 RCVD_IN_MSPIKE_L5 RBL: Very bad reputation (-5) * [95.128.19.6 listed in bl.mailspike.net] * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts * 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% * [score: 0.5000] * 0.0 RCVD_IN_MSPIKE_BL Mailspike blacklisted * 0.8 RDNS_NONE Delivered to internal network by a host with no rDNS * 0.1 FROM_12LTRDOM From a 12-letter domain > On Fri, 30 Oct 2015, j...@lexoncom.com wrote: > >> thx, that explains the issue. >> I setup a dns server outside the amazon server. >> Now, i can finally do the lookup: >> root@aws:~# host -tTXT 2.0.0.127.multi.uribl.com >> 2.0.0.127.multi.uribl.com descriptive text "permanent testpoint" >> >> X-Spam-Flag: YES >> X-Spam-Level: *** >> X-Spam-Status: Yes, score=7.0 required=5.0 tests=BAYES_00, >> >> HTML_FONT_LOW_CONTRAST,HTML_MESSAGE,MIME_HTML_ONLY,RAZOR2_CF_RANGE_51_100, >> >> RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_BRBL_LASTEXT,SPF_HELO_PASS, >> SPF_PASS,URIBL_BLACK,URIBL_DBL_SPAM autolearn=disabled version=3.4.0 >> X-Spam-Report: >> * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist >> * [URIs: yokooo.com] >> * 1.4 RCVD_IN_BRBL_LASTEXT RBL: No description available. >> * [208.80.12.43 listed in bb.barracudacentral.org] >> * -0.0 SPF_PASS SPF: sender matches SPF record >> * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record >> * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% >> * [score: 0.] >> * 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts >> * 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or >> identical to >> * background >> * 0.0 HTML_MESSAGE BODY: HTML included in message >> * 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% >> * [cf: 100] >> * 1.9 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence >> level >> * above 50% >> * [cf: 100] >> * 0.9 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) >> * 1.7 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist >> * [URIs: yokooo.com] > > Bravo! Now all you need to do is wipe and retrain your Bayes database with > known-good corpora to get rid of that BAYES_00. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >...the Fates notice those who buy chainsaws... >-- www.darwinawards.com > --- > Tomorrow: Halloween >
Re: How to get rid of this spam? Spam assassin does not catch it
Am 31.10.2015 um 16:06 schrieb j...@lexoncom.com: So after initial learning it looks better now. (BAYES_50) BAYES_50 is not really good for clear spam When sendmail sends email to procmail and procmail passes it to spam assassin, does spam assassin runs as root user or as the user the email is destined to? depends on how SA is called in detail, normally it should switch to that unix-user and hence training as root makes no sense, *nothing* should proceed potentially dangerous input as root at all - inbound mailcontent is by definition that sort of "don#t do that" input I run the sa-learn as root user oh my god... and it seems like this is the data based that is being used so it would be global data base used for all mail users? https://wiki.apache.org/spamassassin/SiteWideBayesSetup X-Spam-Flag: YES X-Spam-Level: X-Spam-Status: Yes, score=12.9 required=5.0 tests=BAYES_50,FROM_12LTRDOM, HTML_MESSAGE,MIME_HTML_ONLY,RCVD_IN_BRBL_LASTEXT,RCVD_IN_MSPIKE_BL, RCVD_IN_MSPIKE_L5,RCVD_IN_XBL,RDNS_NONE,URIBL_BLACK,URIBL_DBL_SPAM, URIBL_JP_SURBL,URIBL_WS_SURBL autolearn=disabled version=3.4.0 well, the quota of your sa-headers was enough to reject my repsonse on the submission spamass-milter result: Y 16 - URIBL_BLACK,URIBL_DBL_SPAM,URIBL_JP_SURBL,URIBL_WS_SURBL signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
On Fri, 2015-10-30 at 12:53 -0500, j...@lexoncom.com wrote: > I did configure local recursive server and set both spam local.cf and > resolved.conf to point to 127.0.0.1 and I still get the blocks. > Double check that there are no 'forward' options in /etc/names.conf or in files in /etc/named Kindly show us the listen-on{...} option(s) in /etc/named.conf as well as exactly what is in /etc/resolv.conf. The number and order of 'nameserver' directives is important because they, in conjunction with the DNS listen-on options affect what DNS server(s) SA will try to use. Martin PS: apologies if this seems to be failing to keep up with the rest of the discussion, but currently something in my ISP's smarthost seems to be taking 24 hours to pass on the mail it receives.
Re: How to get rid of this spam? Spam assassin does not catch it
On Fri, 30 Oct 2015, j...@lexoncom.com wrote: thx, that explains the issue. I setup a dns server outside the amazon server. Now, i can finally do the lookup: root@aws:~# host -tTXT 2.0.0.127.multi.uribl.com 2.0.0.127.multi.uribl.com descriptive text "permanent testpoint" X-Spam-Flag: YES X-Spam-Level: *** X-Spam-Status: Yes, score=7.0 required=5.0 tests=BAYES_00, HTML_FONT_LOW_CONTRAST,HTML_MESSAGE,MIME_HTML_ONLY,RAZOR2_CF_RANGE_51_100, RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_BRBL_LASTEXT,SPF_HELO_PASS, SPF_PASS,URIBL_BLACK,URIBL_DBL_SPAM autolearn=disabled version=3.4.0 X-Spam-Report: * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: yokooo.com] * 1.4 RCVD_IN_BRBL_LASTEXT RBL: No description available. * [208.80.12.43 listed in bb.barracudacentral.org] * -0.0 SPF_PASS SPF: sender matches SPF record * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.] * 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts * 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to * background * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% * [cf: 100] * 1.9 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level * above 50% * [cf: 100] * 0.9 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) * 1.7 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist * [URIs: yokooo.com] Bravo! Now all you need to do is wipe and retrain your Bayes database with known-good corpora to get rid of that BAYES_00. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the Fates notice those who buy chainsaws... -- www.darwinawards.com --- Tomorrow: Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
On Fri, 30 Oct 2015 14:46:18 -0500 j...@lexoncom.com wrote: > Further testing shows that both smazon and my public ips are blocked. > I never used my public ip for dns so why is it blocked? > Is it just my bad luck and the ip is just blocked on URBL? The rdns for these two addresses is ec2-54-189-149-10.us-west-2.compute.amazonaws.com. ec2-54-244-239-249.us-west-2.compute.amazonaws.com. From http://uribl.com/datafeed_faq.shtml Why are DNS queries from my cloud instances (AmazonEC2/Softlayer/Rackspace/etc) blocked? Large subnets owned by Amazon and other cloud providers have been blocked due to high volume. Because amazon has so many networks, a single user may have multiple mail exchanges on multiple networks, and we have no ability to correlate this and block individual high volume users. We are looking at ways of improving our query limit system for those coming from large virtual hosting providers such as Amazon, but at this time we do not have anything in place. We do offer discounted Datafeed over DNS rates for low-volume, cloud hosted users who are effected by these wide ranging blocks. See Requesting the Datafeed Service and choose 'Cloud Hosted' on the request form. > root@aws:/home/user# > root@aws:/home/user# host -tTXT 2.0.0.127.multi.uribl.com > 2.0.0.127.multi.uribl.com descriptive text "127.0.0.1 -> Query > Refused. See http://uribl.com/refused.shtml for more information > [Your DNS IP: 54.189.149.10]" > root@aws:/home/user# sudo vi /etc/resolv.conf > > root@aws:/home/user# host -tTXT 2.0.0.127.multi.uribl.com > 2.0.0.127.multi.uribl.com descriptive text "127.0.0.1 -> Query > Refused. See http://uribl.com/refused.shtml for more information > [Your DNS IP: 54.244.239.249]" > root@aws:/home/user# >
Re: How to get rid of this spam? Spam assassin does not catch it
thx, that explains the issue. I setup a dns server outside the amazon server. Now, i can finally do the lookup: root@aws:~# host -tTXT 2.0.0.127.multi.uribl.com 2.0.0.127.multi.uribl.com descriptive text "permanent testpoint" X-Spam-Flag: YES X-Spam-Level: *** X-Spam-Status: Yes, score=7.0 required=5.0 tests=BAYES_00, HTML_FONT_LOW_CONTRAST,HTML_MESSAGE,MIME_HTML_ONLY,RAZOR2_CF_RANGE_51_100, RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_BRBL_LASTEXT,SPF_HELO_PASS, SPF_PASS,URIBL_BLACK,URIBL_DBL_SPAM autolearn=disabled version=3.4.0 X-Spam-Report: * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: yokooo.com] * 1.4 RCVD_IN_BRBL_LASTEXT RBL: No description available. * [208.80.12.43 listed in bb.barracudacentral.org] * -0.0 SPF_PASS SPF: sender matches SPF record * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.] * 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts * 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to * background * 0.0 HTML_MESSAGE BODY: HTML included in message * 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% * [cf: 100] * 1.9 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level * above 50% * [cf: 100] * 0.9 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) * 1.7 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist * [URIs: yokooo.com] > On Fri, 30 Oct 2015 14:46:18 -0500 > j...@lexoncom.com wrote: > >> Further testing shows that both smazon and my public ips are blocked. >> I never used my public ip for dns so why is it blocked? >> Is it just my bad luck and the ip is just blocked on URBL? > > The rdns for these two addresses is > > ec2-54-189-149-10.us-west-2.compute.amazonaws.com. > ec2-54-244-239-249.us-west-2.compute.amazonaws.com. > >>From > > http://uribl.com/datafeed_faq.shtml > > Why are DNS queries from my cloud instances > (AmazonEC2/Softlayer/Rackspace/etc) blocked? > >Large subnets owned by Amazon and other cloud providers have been >blocked due to high volume. Because amazon has so many networks, a >single user may have multiple mail exchanges on multiple networks, >and we have no ability to correlate this and block individual high >volume users. We are looking at ways of improving our query limit >system for those coming from large virtual hosting providers such as >Amazon, but at this time we do not have anything in place. We do >offer discounted Datafeed over DNS rates for low-volume, cloud >hosted users who are effected by these wide ranging blocks. See >Requesting the Datafeed Service and choose 'Cloud Hosted' on the >request form. > > > >> root@aws:/home/user# >> root@aws:/home/user# host -tTXT 2.0.0.127.multi.uribl.com >> 2.0.0.127.multi.uribl.com descriptive text "127.0.0.1 -> Query >> Refused. See http://uribl.com/refused.shtml for more information >> [Your DNS IP: 54.189.149.10]" >> root@aws:/home/user# sudo vi /etc/resolv.conf >> >> root@aws:/home/user# host -tTXT 2.0.0.127.multi.uribl.com >> 2.0.0.127.multi.uribl.com descriptive text "127.0.0.1 -> Query >> Refused. See http://uribl.com/refused.shtml for more information >> [Your DNS IP: 54.244.239.249]" >> root@aws:/home/user# >> >
Re: How to get rid of this spam? Spam assassin does not catch it
On 30.10.2015 19:53, j...@lexoncom.com wrote: I did configure local recursive server and set both spam local.cf and resolved.conf to point to 127.0.0.1 and I still get the blocks. The file name for that is /etc/resolv.conf NOT resolved.conf Also if you update local.cf and you run spamd the spamd daemon must be restarted. br. jarif Return-Path:X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on xxx X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE, SPF_HELO_PASS,SPF_PASS,T_REMOTE_IMAGE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Report: * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: motortrend.com] * 0.0 HTML_MESSAGE BODY: HTML included in message * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.] * 0.0 T_REMOTE_IMAGE Message contains an external image Martin -- jarif.bit
Re: How to get rid of this spam? Spam assassin does not catch it
On 27.10.2015 23.02, Martin Gregorie wrote: - The server's /etc/resolve.conf should contain the lines search example.lan nameserver 192.168.7.2 /etc/resolv.conf Typo fixed. -- jarif.bit
Re: How to get rid of this spam? Spam assassin does not catch it
On Fri, 30 Oct 2015, j...@lexoncom.com wrote: On Fri, 30 Oct 2015, j...@lexoncom.com wrote: I already cleaned the db to make sure I dont have it broken. Would it be better to turn off the autolearn. Teach sa ham and spam from over 200 messages and then turn back the autolearn? How big is your userbase and ham email volume? If both are fairly small, I'd leave autolearn turned off and do purely manual classification and training. That's what I do and I have good results, but I'm only supporting 5 users. similar to yours i have been running sa for few years so i do have like 8-10 entries in auto-whitelist per user i cleared it and i will start over with no auto-whitelist enabled for now auto-whitelist (AWL) has nothing to do with bayes or autolearn. Its name is misleading, it is actually more of a score averaging facility to allow for an occasionally spammy-looking email from someone with a hammy history. Turn off autolearn to start while you're evaluating the performance of your initial corpora. Train any FPs and FNs (keeping them as part of your reference training corpora), and get your DNS issues resolved. not sure where is the problem with dns as i have the caching server setup Are you sure that your DNS server is actually the one being used? Can you check the DNS server's logs to see queries coming in from your network and beign recursively resolved? Perhaps post your DNS server's config file? Once things are stable and working smoothly for a while, then you can turn autolearn back on if you feel your mail volume justifies it. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the Fates notice those who buy chainsaws... -- www.darwinawards.com --- Tomorrow: Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
On Tue, 2015-10-27 at 14:19 -0500, j...@lexoncom.com wrote: > I dont use any ham training.Should I scan all my folders with this > command: > sa-learn --ham --mbox /home/username/mail/foldername > YES - if Bayes never gets trained on ham, how do you expect it to recognise the difference between ham and spam? Bayes won't start to work until it has seen 200 examples of ham and 200 examples of spam. > "is the bayes-db of this user *realy* used at scan time" > how do i check that? > When its working you'll see BAYES_nn rules firing. > I use the procemail to pass all mail through spam assassin. > I use default ubuntu setup with Razors enabled. > It does catches spam but not the one i attached in original post. > > example mail sa headers: > > X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on > ip-10-254-37-89.us-west-2.compute.internal > X-Spam-Level: *** > X-Spam-Status: No, score=3.1 required=5.0 > tests=BAYES_00,HTML_MESSAGE, > RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_C > HECK,SPF_HELO_PASS, > SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no > autolearn_force=no > version=3.4.0 > As others have said, URIBL-BLOCKED shows that the number of BL lookups from all the people using whatever DNS server you're using exceeds the free usage count for the BL server. BL servers count messages from a particular DNS and don't know/can't find out how many people are using a particular DNS server to do BL lookups. To get round that you need your own DNS server, configured the do recursive lookups and NOT to forward queries to any other DNS server. So, set up your own recursive, non-forwarding DNS server on the host where you're running SA. Configure that host to pass all DNS queries to your new DNS server by configuring /etc/resolv.conf as I and others have described. If you don't understand how to install and configure a DNS server and prefer printed material to online documents, get the O'Reilly book "DNS and BIND". Martin
Re: How to get rid of this spam? Spam assassin does not catch it
Am 29.10.2015 um 01:06 schrieb Martin Gregorie: If you don't understand how to install and configure a DNS server and prefer printed material to online documents, get the O'Reilly book "DNS and BIND" no need for use bind at all for cahcing nameservers, unbound is much faster for *that* task and works more or less out-of-the-box unbound.conf on our inbound MX while all production nameservers with authoritative zones are bind server: verbosity: 1 statistics-interval: 86400 statistics-cumulative: no extended-statistics: no num-threads: 1 outgoing-range: 1024 num-queries-per-thread: 512 msg-cache-slabs: 8 rrset-cache-slabs: 8 infra-cache-slabs: 8 key-cache-slabs: 8 so-rcvbuf: 4m so-sndbuf: 4m minimal-responses: yes msg-cache-size: 64m neg-cache-size: 64m rrset-cache-size: 128m cache-min-ttl: 300 cache-max-ttl: 10800 interface: 127.0.0.1 access-control: 127.0.0.0/8 allow interface-automatic: no port: 53 do-ip4: yes do-ip6: no do-udp: yes max-udp-size: 1024 edns-buffer-size: 1024 do-tcp: yes do-daemonize: yes username: "unbound" directory: "/etc/unbound" chroot: "/etc/unbound" use-syslog: yes log-time-ascii: yes pidfile: "/run/unbound/unbound.pid" hide-identity: yes hide-version: yes harden-glue: yes harden-dnssec-stripped: no harden-referral-path: no use-caps-for-id: no unwanted-reply-threshold: 1000 do-not-query-localhost: no prefetch: yes prefetch-key: yes signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
> On Fri, 30 Oct 2015, j...@lexoncom.com wrote: > >> I already cleaned the db to make sure I dont have it broken. >> Would it be better to turn off the autolearn. >> Teach sa ham and spam from over 200 messages and then turn back the >> autolearn? > > How big is your userbase and ham email volume? > > If both are fairly small, I'd leave autolearn turned off and do purely > manual classification and training. That's what I do and I have good > results, but I'm only supporting 5 users. > similar to yours i have been running sa for few years so i do have like 8-10 entries in auto-whitelist per user i cleared it and i will start over with no auto-whitelist enabled for now > Turn off autolearn to start while you're evaluating the performance of > your initial corpora. Train any FPs and FNs (keeping them as part of your > reference training corpora), and get your DNS issues resolved. > not sure where is the problem with dns as i have the caching server setup > Once things are stable and working smoothly for a while, then you can turn > autolearn back on if you feel your mail volume justifies it. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >...the Fates notice those who buy chainsaws... >-- www.darwinawards.com > --- > Tomorrow: Halloween >
Re: How to get rid of this spam? Spam assassin does not catch it
Am 30.10.2015 um 18:01 schrieb David B Funk: On Fri, 30 Oct 2015, Reindl Harald wrote: Am 29.10.2015 um 01:06 schrieb Martin Gregorie: If you don't understand how to install and configure a DNS server and prefer printed material to online documents, get the O'Reilly book "DNS and BIND" no need for use bind at all for cahcing nameservers, unbound is much faster for *that* task and works more or less out-of-the-box unbound.conf on our inbound MX while all production nameservers with authoritative zones are bind [snip..] Just be sure to set the access-control correctly to prevent use/abuse by remote attackers. Open recursive DNS servers are a favorite DDOS tool well, you snipped that part. interface: 127.0.0.1 access-control: 127.0.0.0/8 allow for DDOS it don't matter if is a recursive or a authoritative nameserver, ANY records of auth servers without respone rate limiting are amplification enough signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
On Fri, 30 Oct 2015, Reindl Harald wrote: Am 29.10.2015 um 01:06 schrieb Martin Gregorie: If you don't understand how to install and configure a DNS server and prefer printed material to online documents, get the O'Reilly book "DNS and BIND" no need for use bind at all for cahcing nameservers, unbound is much faster for *that* task and works more or less out-of-the-box unbound.conf on our inbound MX while all production nameservers with authoritative zones are bind [snip..] Just be sure to set the access-control correctly to prevent use/abuse by remote attackers. Open recursive DNS servers are a favorite DDOS tool. -- Dave Funk University of Iowa College of Engineering 319/335-5751 FAX: 319/384-0549 1256 Seamans Center Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527 #include Better is not better, 'standard' is better. B{
Re: How to get rid of this spam? Spam assassin does not catch it
> On Tue, 2015-10-27 at 14:19 -0500, j...@lexoncom.com wrote: >> I dont use any ham training.Should I scan all my folders with this >> command: >> sa-learn --ham --mbox /home/username/mail/foldername >> > YES - if Bayes never gets trained on ham, how do you expect it to > recognise the difference between ham and spam? > > Bayes won't start to work until it has seen 200 examples of ham and 200 > examples of spam. thx, i started to sort the emails for a learnng process > >> "is the bayes-db of this user *realy* used at scan time" >> how do i check that? >> > When its working you'll see BAYES_nn rules firing. > >> I use the procemail to pass all mail through spam assassin. >> I use default ubuntu setup with Razors enabled. >> It does catches spam but not the one i attached in original post. >> >> example mail sa headers: >> >> X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on >> ip-10-254-37-89.us-west-2.compute.internal >> X-Spam-Level: *** >> X-Spam-Status: No, score=3.1 required=5.0 >> tests=BAYES_00,HTML_MESSAGE, >> RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_C >> HECK,SPF_HELO_PASS, >> SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no >> autolearn_force=no >> version=3.4.0 >> > As others have said, URIBL-BLOCKED shows that the number of BL lookups > from all the people using whatever DNS server you're using exceeds the > free usage count for the BL server. BL servers count messages from a > particular DNS and don't know/can't find out how many people are using > a particular DNS server to do BL lookups. To get round that you need > your own DNS server, configured the do recursive lookups and NOT to > forward queries to any other DNS server. > > So, set up your own recursive, non-forwarding DNS server on the host > where you're running SA. Configure that host to pass all DNS queries to > your new DNS server by configuring /etc/resolv.conf as I and others > have described. > > If you don't understand how to install and configure a DNS server and > prefer printed material to online documents, get the O'Reilly book "DNS > and BIND". > I did configure local recursive server and set both spam local.cf and resolved.conf to point to 127.0.0.1 and I still get the blocks. Return-Path:X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on xxx X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE, SPF_HELO_PASS,SPF_PASS,T_REMOTE_IMAGE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Report: * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. * See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block * for more information. * [URIs: motortrend.com] * 0.0 HTML_MESSAGE BODY: HTML included in message * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.] * 0.0 T_REMOTE_IMAGE Message contains an external image > > Martin > >
Re: How to get rid of this spam? Spam assassin does not catch it
If auto learn is enabled and header shows: autolearn=ham what happens when i classify that email later as spam? thx > On Tue, 2015-10-27 at 14:19 -0500, j...@lexoncom.com wrote: >> I dont use any ham training.Should I scan all my folders with this >> command: >> sa-learn --ham --mbox /home/username/mail/foldername >> > YES - if Bayes never gets trained on ham, how do you expect it to > recognise the difference between ham and spam? > > Bayes won't start to work until it has seen 200 examples of ham and 200 > examples of spam. > >> "is the bayes-db of this user *realy* used at scan time" >> how do i check that? >> > When its working you'll see BAYES_nn rules firing. > >> I use the procemail to pass all mail through spam assassin. >> I use default ubuntu setup with Razors enabled. >> It does catches spam but not the one i attached in original post. >> >> example mail sa headers: >> >> X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on >> ip-10-254-37-89.us-west-2.compute.internal >> X-Spam-Level: *** >> X-Spam-Status: No, score=3.1 required=5.0 >> tests=BAYES_00,HTML_MESSAGE, >> RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_C >> HECK,SPF_HELO_PASS, >> SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no >> autolearn_force=no >> version=3.4.0 >> > As others have said, URIBL-BLOCKED shows that the number of BL lookups > from all the people using whatever DNS server you're using exceeds the > free usage count for the BL server. BL servers count messages from a > particular DNS and don't know/can't find out how many people are using > a particular DNS server to do BL lookups. To get round that you need > your own DNS server, configured the do recursive lookups and NOT to > forward queries to any other DNS server. > > So, set up your own recursive, non-forwarding DNS server on the host > where you're running SA. Configure that host to pass all DNS queries to > your new DNS server by configuring /etc/resolv.conf as I and others > have described. > > If you don't understand how to install and configure a DNS server and > prefer printed material to online documents, get the O'Reilly book "DNS > and BIND". > > > Martin > >
Re: How to get rid of this spam? Spam assassin does not catch it
On Thu, 29 Oct 2015, Martin Gregorie wrote: On Tue, 2015-10-27 at 14:19 -0500, j...@lexoncom.com wrote: I dont use any ham training.Should I scan all my folders with this command: sa-learn --ham --mbox /home/username/mail/foldername YES - if Bayes never gets trained on ham, how do you expect it to recognise the difference between ham and spam? Bayes won't start to work until it has seen 200 examples of ham and 200 examples of spam. Again: *vetted* ham and spam. Don't just blindly throw your inbox at it assuming your inbox is pristine. "is the bayes-db of this user *realy* used at scan time" how do i check that? When its working you'll see BAYES_nn rules firing. Note BAYES_00 in the report below. The OP is getting ham from *somewhere*. If he's never manually trained ham then it's probably coming from autolearn, and depending on other issues that might have poisoned the database from the start. example mail sa headers: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-254-37-89.us-west-2.compute.internal X-Spam-Level: *** X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_C HECK,SPF_HELO_PASS, SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the Fates notice those who buy chainsaws... -- www.darwinawards.com --- Tomorrow: Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
On Fri, 30 Oct 2015, j...@lexoncom.com wrote: If auto learn is enabled and header shows: autolearn=ham what happens when i classify that email later as spam? Essentially, the tokens from that message in the bayes database will be converted from "hammy" to "spammy". This is normal reclassification of a FN, nothing unusual about it. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the Fates notice those who buy chainsaws... -- www.darwinawards.com --- Tomorrow: Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
I already cleaned the db to make sure I dont have it broken. Would it be better to turn off the autolearn. Teach sa ham and spam from over 200 messages and then turn back the autolearn? thx > On Thu, 29 Oct 2015, Martin Gregorie wrote: > >> On Tue, 2015-10-27 at 14:19 -0500, j...@lexoncom.com wrote: >>> I dont use any ham training.Should I scan all my folders with this >>> command: >>> sa-learn --ham --mbox /home/username/mail/foldername >> >> YES - if Bayes never gets trained on ham, how do you expect it to >> recognise the difference between ham and spam? >> >> Bayes won't start to work until it has seen 200 examples of ham and 200 >> examples of spam. > > Again: *vetted* ham and spam. Don't just blindly throw your inbox at it > assuming your inbox is pristine. > >>> "is the bayes-db of this user *realy* used at scan time" >>> how do i check that? >> >> When its working you'll see BAYES_nn rules firing. > > Note BAYES_00 in the report below. The OP is getting ham from *somewhere*. > If he's never manually trained ham then it's probably coming from > autolearn, and depending on other issues that might have poisoned the > database from the start. > >>> example mail sa headers: >>> >>> X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on >>> ip-10-254-37-89.us-west-2.compute.internal >>> X-Spam-Level: *** >>> X-Spam-Status: No, score=3.1 required=5.0 >>> tests=BAYES_00,HTML_MESSAGE, >>> RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_C >>> HECK,SPF_HELO_PASS, >>> SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no >>> autolearn_force=no >>> version=3.4.0 > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >...the Fates notice those who buy chainsaws... >-- www.darwinawards.com > --- > Tomorrow: Halloween >
Re: How to get rid of this spam? Spam assassin does not catch it
On Fri, 30 Oct 2015, j...@lexoncom.com wrote: I already cleaned the db to make sure I dont have it broken. Would it be better to turn off the autolearn. Teach sa ham and spam from over 200 messages and then turn back the autolearn? How big is your userbase and ham email volume? If both are fairly small, I'd leave autolearn turned off and do purely manual classification and training. That's what I do and I have good results, but I'm only supporting 5 users. Turn off autolearn to start while you're evaluating the performance of your initial corpora. Train any FPs and FNs (keeping them as part of your reference training corpora), and get your DNS issues resolved. Once things are stable and working smoothly for a while, then you can turn autolearn back on if you feel your mail volume justifies it. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the Fates notice those who buy chainsaws... -- www.darwinawards.com --- Tomorrow: Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
Further testing shows that both smazon and my public ips are blocked. I never used my public ip for dns so why is it blocked? Is it just my bad luck and the ip is just blocked on URBL? root@aws:/home/user# root@aws:/home/user# host -tTXT 2.0.0.127.multi.uribl.com 2.0.0.127.multi.uribl.com descriptive text "127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 54.189.149.10]" root@aws:/home/user# sudo vi /etc/resolv.conf root@aws:/home/user# host -tTXT 2.0.0.127.multi.uribl.com 2.0.0.127.multi.uribl.com descriptive text "127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 54.244.239.249]" root@aws:/home/user# >> On Fri, 30 Oct 2015, j...@lexoncom.com wrote: >> >>> I already cleaned the db to make sure I dont have it broken. >>> Would it be better to turn off the autolearn. >>> Teach sa ham and spam from over 200 messages and then turn back the >>> autolearn? >> >> How big is your userbase and ham email volume? > >> >> If both are fairly small, I'd leave autolearn turned off and do purely >> manual classification and training. That's what I do and I have good >> results, but I'm only supporting 5 users. >> > similar to yours > i have been running sa for few years so i do have like > 8-10 entries in auto-whitelist per user > i cleared it and i will start over > with no auto-whitelist enabled for now > >> Turn off autolearn to start while you're evaluating the performance of >> your initial corpora. Train any FPs and FNs (keeping them as part of >> your >> reference training corpora), and get your DNS issues resolved. >> > not sure where is the problem with dns > as i have the caching server setup > >> Once things are stable and working smoothly for a while, then you can >> turn >> autolearn back on if you feel your mail volume justifies it. >> >> -- >> John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ >> jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org >> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 >> --- >>...the Fates notice those who buy chainsaws... >>-- www.darwinawards.com >> --- >> Tomorrow: Halloween >> > >
Re: How to get rid of this spam? Spam assassin does not catch it
On Tue, 2015-10-27 at 15:02 -0500, j...@lexoncom.com wrote: > SO i setup the dns server. > Can i force spam assassin to use localhost for dns or I must > reconfigure > the host? > Simpler than that. Assuming your dns server is: - listening on your LAN for dns requests - is configured to be the definitive name source for hosts on your LAN, i.e. it has a zone file defining 'example.lan' as the domain name used for all hosts on the LAN - the configuration has an A and PTR record for every host on the LAN - the server's IP is 192.168.7.2 [1] Then the following setup should work and is easy to maintain: - The server's /etc/resolve.conf should contain the lines search example.lan nameserver 192.168.7.2 That takes care of SA's dns lookups and caching needs as well as providing a centralised service for every other host on the LAN - if the other hosts on your LAN use exactly the same /etc/resolv.conf then everything 'just works' [2] [1] change to suit the IP range you're using on your LAN. My LAN's subnet is 168.192.7.255 and I'm showing my resolv.conf lines [2] you may want to add another 'nameserver' line after the initial one. This should reference some external dns, one belonging to your ISP or a public dns, so that external names still get resolved when either the dns process or the server it runs on is offline for one reason or another. This is fine for a smallish LAN with a fairly static host population. If you need something more dynamic, run a DHCP server to support visitors, etc. This is how my fairly small LAN works. It is virtually maintenance free: the only stuff I need to do is to configure any hosts when an OS upgrade manages to loose or overwrite its network configuration. Martin
Re: How to get rid of this spam? Spam assassin does not catch it
On 27.10.15 15:02, j...@lexoncom.com wrote: SO i setup the dns server. Can i force spam assassin to use localhost for dns or I must reconfigure the host? you should reconfigure the host - add 127.0.0.1 to the resolv.conf On Tue, 27 Oct 2015, j...@lexoncom.com wrote: X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS, SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server for SpamAssassin to use. You're apparently doing DNS blacklist queries via a public DNS server (your ISPs?) and the aggregate traffic level is exceeding the URIBL free usage limits. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease
Re: How to get rid of this spam? Spam assassin does not catch it
On Thu, 29 Oct 2015, Noel Butler wrote: On 28/10/2015 12:49, David B Funk wrote: Are you -sure- all those messages are spam? One of them was a personal FaceBook update message. facebook is junk so I see nothing wrong with those messages being regarded as spam :) Noel I agree with you in principal (FB == junk) but by the fine print of the FB LLuser agreement when you sign up you're asking for it so technically it isn't unsolicited. I was more trying to determine whether the OP had done his due-diligence. -- Dave Funk University of Iowa College of Engineering 319/335-5751 FAX: 319/384-0549 1256 Seamans Center Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527 #include Better is not better, 'standard' is better. B{
Re: How to get rid of this spam? Spam assassin does not catch it
Am 28.10.2015 um 04:31 schrieb j...@lexoncom.com: yes there might be few emails there that were legitimate i cleaned it but i did not have time to do it property then don't train messages at all if you don't have time to do it properly, you are doing much more harm by misclassification as you ever could benfit by it are not net/RBL/DNSBL tests enabled by default? they are but you are using a shared dns resolver i need to review the documentation and see why it does not work because your misconfiguration On Tue, 27 Oct 2015, j...@lexoncom.com wrote: try this https://www.dropbox.com/s/ngmaryggdelecjq/INBOX.spam?dl=0 it is mbox file with like 1000 spam messages that are not recognized as spam Are you -sure- all those messages are spam? One of them was a personal FaceBook update message. If you ("blwegr...@lexoncom.com") have a FB account then pretty much all updates sent to you as a result really cannot be considered spam. FWIW, You are really short-changing your SA by not having the net/RBL/DNSBL tests working properly. The vast majority of those messages (%96) were tagged as spam by my system and a super majority (%83) scored > 20.0 (my SMTP reject threshold). A large component of that score was from net/RBL/DNSBL tests. signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
On 28/10/2015 12:49, David B Funk wrote: Are you -sure- all those messages are spam? One of them was a personal FaceBook update message. facebook is junk so I see nothing wrong with those messages being regarded as spam :) -- If you have the urge to reply to all rather than reply to list, you best read http://members.ausics.net/qwerty/
Re: How to get rid of this spam? Spam assassin does not catch it
Am 27.10.2015 um 18:50 schrieb j...@lexoncom.com: I use spam assassin with razors on ubuntu server. In recent months i started to get tons of spam. Spam assassin does not catch it and scores are very low. Are those emails fabricated so well that they look like legitimate? Can i do something to catch those as spam? I moved them all to one folder called spam and i run this command every 5 minutes on that folder: sa-learn --spam --mbox /home/username/mail/INBOX.spam but it does not help do you have enough *ham* trained? is the bayes-db of this user *realy* used at scan time what are the SA-headers of mails passing through? sorry but you need to provide basic informations signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
On 10/27/2015 06:50 PM, j...@lexoncom.com wrote: I use spam assassin with razors on ubuntu server. In recent months i started to get tons of spam. Spam assassin does not catch it and scores are very low. Are those emails fabricated so well that they look like legitimate? Can i do something to catch those as spam? I moved them all to one folder called spam and i run this command every 5 minutes on that folder: sa-learn --spam --mbox /home/username/mail/INBOX.spam but it does not help It seems like every spam email is fabricated in different way. Anyone has any idea how to catch those? Why spam assassin does not catch it? attached is the list showing subject and from for the recent spams i get. Suggest you pastebin a few samples - subjects on their own are not of much use.
Re: How to get rid of this spam? Spam assassin does not catch it
I understand now. sa-learn --ham --no-rebuild ham_directory sa-learn --spam --no-rebuild spam_directory sa-learn --rebuild so would the best practice to be move spam to spam folder and learn as spam and learn all other folders as ham and then rebuild. The inbox would never be scanned as it might have new span and not spam messages. I would need some script to go through all messages for all users except the spam folder to learn as HAM. > > > Am 27.10.2015 um 20:19 schrieb j...@lexoncom.com: >> I dont use any ham training > > then you can't expect bayes to work at all because how do you expect the > bayes filter to know the *difference* of ham and spam signs? > > https://wiki.apache.org/spamassassin/BayesFaq > >
Re: How to get rid of this spam? Spam assassin does not catch it
Am 27.10.2015 um 21:02 schrieb j...@lexoncom.com: SO i setup the dns server. Can i force spam assassin to use localhost for dns or I must reconfigure the host? i recommend to read at least basic docs google "spamassassin dns" leads to http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html and CTRL+F "dns" leads to the following (the docs would also have mentioned that you need at least 200 spam *and* ham samples for bayes to work) dns_server ip-addr-port (default: entries provided by Net::DNS) Specifies an IP address of a DNS server, and optionally its port number. The dns_server directive may be specified multiple times, each entry adding to a list of available resolving name servers. The ip-addr-port argument can either be an IPv4 or IPv6 address, optionally enclosed in brackets, and optionally followed by a colon and a port number. In absence of a port number a standard port number 53 is assumed. When an IPv6 address is specified along with a port number, the address must be enclosed in brackets to avoid parsing ambiguity regarding a colon separator. A scoped link-local IP address is allowed (assuming underlying modules allow it). Examples : dns_server 127.0.0.1 dns_server 127.0.0.1:53 dns_server [127.0.0.1]:53 dns_server [::1]:53 dns_server fe80::1%lo0 dns_server [fe80::1%lo0]:53 In absence of dns_server directives, the list of name servers is provided by Net::DNS module, which typically obtains the list from /etc/resolv.conf, but this may be platform dependent. Please consult the Net::DNS::Resolver documentation for details. On Tue, 27 Oct 2015, j...@lexoncom.com wrote: X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS, SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server for SpamAssassin to use. You're apparently doing DNS blacklist queries via a public DNS server (your ISPs?) and the aggregate traffic level is exceeding the URIBL free usage limits. signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
Am 27.10.2015 um 20:19 schrieb j...@lexoncom.com: I dont use any ham training then you can't expect bayes to work at all because how do you expect the bayes filter to know the *difference* of ham and spam signs? https://wiki.apache.org/spamassassin/BayesFaq signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
On Tue, 27 Oct 2015, j...@lexoncom.com wrote: X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS, SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server for SpamAssassin to use. You're apparently doing DNS blacklist queries via a public DNS server (your ISPs?) and the aggregate traffic level is exceeding the URIBL free usage limits. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the Fates notice those who buy chainsaws... -- www.darwinawards.com --- 4 days until Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
Am 27.10.2015 um 20:23 schrieb Marc Perkel: Also - add a highest numbers MX record tarbaby.junkemailfilter.com This will help tune our list to your spam and also get rid of a lot od it how do you distinct fools like facebook at the moment always trying first the backup-MX (which is here a postscreen honeypot always repsonding 4xx if the sending IP is not on eough blacklists for score based reject) and real spammers? don't get me wrong - i use "tarbaby.junkemailfilter.com" but *only* for honeypot domains which don't expect legit mail for sure signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
SO i setup the dns server. Can i force spam assassin to use localhost for dns or I must reconfigure the host? > On Tue, 27 Oct 2015, j...@lexoncom.com wrote: > >> X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, >> >> RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS, >> SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no >> autolearn_force=no >> version=3.4.0 > > URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server for > SpamAssassin to use. You're apparently doing DNS blacklist queries via a > public DNS server (your ISPs?) and the aggregate traffic level is > exceeding the URIBL free usage limits. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >...the Fates notice those who buy chainsaws... >-- www.darwinawards.com > --- > 4 days until Halloween >
Re: How to get rid of this spam? Spam assassin does not catch it
You can use my black and white lists. It should help. header __RCVD_IN_HOSTKARMA eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.') describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter tflags __RCVD_IN_HOSTKARMA net header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.1') describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE tflags RCVD_IN_HOSTKARMA_W net nice score RCVD_IN_HOSTKARMA_W -5 header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.2') describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK tflags RCVD_IN_HOSTKARMA_BL net score RCVD_IN_HOSTKARMA_BL 3.0 header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.4') describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN tflags RCVD_IN_HOSTKARMA_BR net score RCVD_IN_HOSTKARMA_BR 1.0 Also - add a highest numbers MX record tarbaby.junkemailfilter.com This will help tune our list to your spam and also get rid of a lot od it. On 10/27/15 10:50, j...@lexoncom.com wrote: sa-learn --spam --mbox /home/username/mail/INBOX.spam -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: How to get rid of this spam? Spam assassin does not catch it
Yes - add to local.cf As the highest numbered MX record tarbaby,junkemailfilter.com usually only sees virus bots. It never accepts email and refuses with a 4xx error in case something legit hits it. So we never see your email. It also doesn't blacklist good email. The sender has to commit several "sins" before it is blacklisted. So it's safe - gets rid of some spam, and helps tune our blacklists to include more bad actors. On 10/27/15 12:48, j...@lexoncom.com wrote: can you explain how this works? Do i add this to spam local.cf file? would not Also - add a highest numbers MX record tarbaby.junkemailfilter.com allow your servers to see my emails? thx You can use my black and white lists. It should help. header __RCVD_IN_HOSTKARMA eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.') describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter tflags __RCVD_IN_HOSTKARMA net header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.1') describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE tflags RCVD_IN_HOSTKARMA_W net nice score RCVD_IN_HOSTKARMA_W -5 header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.2') describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK tflags RCVD_IN_HOSTKARMA_BL net score RCVD_IN_HOSTKARMA_BL 3.0 header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.4') describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN tflags RCVD_IN_HOSTKARMA_BR net score RCVD_IN_HOSTKARMA_BR 1.0 Also - add a highest numbers MX record tarbaby.junkemailfilter.com This will help tune our list to your spam and also get rid of a lot od it. On 10/27/15 10:50, j...@lexoncom.com wrote: sa-learn --spam --mbox /home/username/mail/INBOX.spam -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400 -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: How to get rid of this spam? Spam assassin does not catch it
can you explain how this works? Do i add this to spam local.cf file? would not > Also - add a highest numbers MX record tarbaby.junkemailfilter.com allow your servers to see my emails? thx > You can use my black and white lists. It should help. > > header __RCVD_IN_HOSTKARMA > eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.') > describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter > tflags __RCVD_IN_HOSTKARMA net > > header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal', > '127.0.0.1') > describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE > tflags RCVD_IN_HOSTKARMA_W net nice > score RCVD_IN_HOSTKARMA_W -5 > > header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal', > '127.0.0.2') > describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK > tflags RCVD_IN_HOSTKARMA_BL net > score RCVD_IN_HOSTKARMA_BL 3.0 > > header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal', > '127.0.0.4') > describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN > tflags RCVD_IN_HOSTKARMA_BR net > score RCVD_IN_HOSTKARMA_BR 1.0 > > > Also - add a highest numbers MX record tarbaby.junkemailfilter.com > > This will help tune our list to your spam and also get rid of a lot od it. > > On 10/27/15 10:50, j...@lexoncom.com wrote: >> sa-learn --spam --mbox /home/username/mail/INBOX.spam > > -- > Marc Perkel - Sales/Support > supp...@junkemailfilter.com > http://www.junkemailfilter.com > Junk Email Filter dot com > 415-992-3400 > >
Re: How to get rid of this spam? Spam assassin does not catch it
thx, yes i did that but found old doc and that option was not available: https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html > > Am 27.10.2015 um 21:02 schrieb j...@lexoncom.com: >> SO i setup the dns server. >> Can i force spam assassin to use localhost for dns or I must reconfigure >> the host? > > i recommend to read at least basic docs > google "spamassassin dns" leads to > http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html > and > CTRL+F "dns" leads to the following (the docs would also have mentioned > that you need at least 200 spam *and* ham samples for bayes to work) > > dns_server ip-addr-port (default: entries provided by Net::DNS) > > Specifies an IP address of a DNS server, and optionally its port number. > The dns_server directive may be specified multiple times, each entry > adding to a list of available resolving name servers. The ip-addr-port > argument can either be an IPv4 or IPv6 address, optionally enclosed in > brackets, and optionally followed by a colon and a port number. In > absence of a port number a standard port number 53 is assumed. When an > IPv6 address is specified along with a port number, the address must be > enclosed in brackets to avoid parsing ambiguity regarding a colon > separator. A scoped link-local IP address is allowed (assuming > underlying modules allow it). > > Examples : dns_server 127.0.0.1 dns_server 127.0.0.1:53 dns_server > [127.0.0.1]:53 dns_server [::1]:53 dns_server fe80::1%lo0 dns_server > [fe80::1%lo0]:53 > > In absence of dns_server directives, the list of name servers is > provided by Net::DNS module, which typically obtains the list from > /etc/resolv.conf, but this may be platform dependent. Please consult the > Net::DNS::Resolver documentation for details. > >>> On Tue, 27 Oct 2015, j...@lexoncom.com wrote: >>> X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS, SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 >>> >>> URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server >>> for >>> SpamAssassin to use. You're apparently doing DNS blacklist queries via >>> a >>> public DNS server (your ISPs?) and the aggregate traffic level is >>> exceeding the URIBL free usage limits. > >
Re: How to get rid of this spam? Spam assassin does not catch it
I dont use any ham training.Should I scan all my folders with this command: sa-learn --ham --mbox /home/username/mail/foldername "is the bayes-db of this user *realy* used at scan time" how do i check that? I use the procemail to pass all mail through spam assassin. I use default ubuntu setup with Razors enabled. It does catches spam but not the one i attached in original post. example mail sa headers: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-254-37-89.us-west-2.compute.internal X-Spam-Level: *** X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS, SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no version=3.4.0 ubuntu@ip-10-254-37-89:~$ cat /etc/spamassassin/local.cf # This is the right place to customize your installation of SpamAssassin. # # See 'perldoc Mail::SpamAssassin::Conf' for details of what can be # tweaked. # # Only a small subset of options are listed below # ### # Add *SPAM* to the Subject header of spam e-mails # # rewrite_header Subject *SPAM* # Save spam messages as a message/rfc822 MIME attachment instead of # modifying the original message (0: off, 2: use text/plain instead) # # report_safe 1 # Set which networks or hosts are considered 'trusted' by your mail # server (i.e. not spammers) # # trusted_networks 212.17.35. # Set file-locking method (flock is not safe over NFS, but is faster) # # lock_method flock # Set the threshold at which a message is considered spam (default: 5.0) # # required_score 5.0 # Use Bayesian classifier (default: 1) # # use_bayes 1 # Bayesian classifier auto-learning (default: 1) # # bayes_auto_learn 1 # Set headers which may provide inappropriate cues to the Bayesian # classifier # # bayes_ignore_header X-Bogosity # bayes_ignore_header X-Spam-Flag # bayes_ignore_header X-Spam-Status # Some shortcircuiting, if the plugin is enabled # ifplugin Mail::SpamAssassin::Plugin::Shortcircuit # # default: strongly-whitelisted mails are *really* whitelisted now, if the # shortcircuiting plugin is active, causing early exit to save CPU load. # Uncomment to turn this on # # shortcircuit USER_IN_WHITELIST on # shortcircuit USER_IN_DEF_WHITELIST on # shortcircuit USER_IN_ALL_SPAM_TO on # shortcircuit SUBJECT_IN_WHITELISTon # the opposite; blacklisted mails can also save CPU # # shortcircuit USER_IN_BLACKLIST on # shortcircuit USER_IN_BLACKLIST_TOon # shortcircuit SUBJECT_IN_BLACKLISTon # if you have taken the time to correctly specify your "trusted_networks", # this is another good way to save CPU # # shortcircuit ALL_TRUSTED on # and a well-trained bayes DB can save running rules, too # # shortcircuit BAYES_99spam # shortcircuit BAYES_00ham endif # Mail::SpamAssassin::Plugin::Shortcircuit # Vipul's Razor options. use_razor2 1 #razor_timeout 10 razor_config /etc/razor/razor-agent.conf loadplugin Mail::SpamAssassin::Plugin::Razor2 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] procmail setup: :0fw: spamassassin.lock * < 256000 | spamassassin # Mails with a score of 15 or higher are almost certainly spam (with 0.05% # false positives according to rules/STATISTICS.txt). Let's put them in a # different mbox. (This one is optional.) :0: * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\* /var/spool/mail/junk # All mail tagged as spam (eg. with a score higher than the set threshold) # is moved to "probably-spam". :0: * ^X-Spam-Status: Yes /var/spool/mail/junk > > > Am 27.10.2015 um 18:50 schrieb j...@lexoncom.com: >> I use spam assassin with razors on ubuntu server. >> In recent months i started to get tons of spam. >> Spam assassin does not catch it and scores are very low. >> >> Are those emails fabricated so well that they look like legitimate? Can >> i >> do something to catch those as spam? >> >> I moved them all to one folder called spam and i run this command every >> 5 >> minutes on that folder: >> sa-learn --spam --mbox /home/username/mail/INBOX.spam >> but it does not help > > do you have enough *ham* trained? > is the bayes-db of this user *realy* used at scan time > what are the SA-headers of mails passing through? > > sorry but you need to provide basic informations > >
Re: How to get rid of this spam? Spam assassin does not catch it
Am 27.10.2015 um 20:31 schrieb j...@lexoncom.com: I understand now. sa-learn --ham --no-rebuild ham_directory sa-learn --spam --no-rebuild spam_directory sa-learn --rebuild so would the best practice to be move spam to spam folder and learn as spam and learn all other folders as ham and then rebuild. The inbox would never be scanned as it might have new span and not spam messages. I would need some script to go through all messages for all users except the spam folder to learn as HAM. i would *never ever* make such things automated i have just a physical folder "spam" and and physical folder "ham" wil single .eml files and hand selected samples - currenmtly they are feeded by a PHP script receiving IMAP messages from the spam/ham folders, testing them via CLI in case of spam if they are not already BAYES_999 and then save eml files over the last month i also trained BAYES_999 to find as much as possible common spam signs, with 2.5 Mio tokens there is no longer need for that, the bayes-db has a hitrate of 99.9% by filter out the remaining 8-10% junk, anything else is cuaght long before spamass-milter by blacklists /which are not working or you because once more somebody i using a shared DNS resolver instead doing recursion on it's own caching server) 0 48739SPAM 0 20549HAM 02256265TOKEN insgesamt 70M -rw--- 1 sa-milt sa-milt 9,7M 2015-10-27 20:08 bayes_seen -rw--- 1 sa-milt sa-milt 81M 2015-10-27 20:08 bayes_toks BAYES_0025591 70.79 % BAYES_05 7392.04 % BAYES_20 9322.57 % BAYES_40 7892.18 % BAYES_50 3981 11.01 % BAYES_60 4761.31 % BAYES_80 4181.15 % BAYES_95 2900.80 % BAYES_99 29348.11 % BAYES_99926307.27 % DELIVERED 49373 93.82 % DNSWL 46277 87.94 % SPF 33497 63.65 % SPF/DKIM WL 15849 30.11 % SHORTCIRCUIT16426 31.21 % BLOCKED 44358.42 % SPAMMY 41187.82 %92.85 % (OF TOTAL BLOCKED) especially when it comes to random users they often move something to spam just because they are too lazy or too stupid for unsubscribe (seen that even for invoice mails of their energy supplier coming back from AOL as abuse-feedback-loop including the invoice with their address and power consumations over the last month) the same for ham: just because a message is in a different folder than inbox/spam don't make it to a ham message, just a simple sieve-rule my move them and it was slipped junk for every wrong classified message (no matter in what direction) in the end you likely need 5 messages to compare the damage and in the end you will again end with a bayes having no clue at all train your bayes careful, by hand and try to keep a blance of ham/spam for best results Am 27.10.2015 um 20:19 schrieb j...@lexoncom.com: I dont use any ham training then you can't expect bayes to work at all because how do you expect the bayes filter to know the *difference* of ham and spam signs? https://wiki.apache.org/spamassassin/BayesFaq signature.asc Description: OpenPGP digital signature
Re: How to get rid of this spam? Spam assassin does not catch it
j...@lexoncom.com skrev den 2015-10-27 21:33: thx, yes i did that but found old doc and that option was not available: https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html this is why i suggest to check local docs first, if not found local, check atleast to diff queueries on internet to confirm it valid options, google is fine, but :) perldoc Mail::SpamAssassin::Conf is trusted
Re: How to get rid of this spam? Spam assassin does not catch it
On Tue, 27 Oct 2015, j...@lexoncom.com wrote: example mail sa headers: Is this from a spam? X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-254-37-89.us-west-2.compute.internal X-Spam-Level: *** X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, BAYES_00. You *do* have ham and spam trained, and bayes *is* in use. If this is a spam, your Bayes appears to be mistrained. That might explain why so many spams are getting through. If you have autolearn turned on, turn it off. Collect hand-classified corpora of several hundred hams and several hundred spams, then wipe and retrain your Bayes. If your userbase is small enough to collect and train on just misclassified messages, then leave autolearn turned off and just train misclassifications and messages that don't hit either BAYES_00 or BAYES_99. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the Fates notice those who buy chainsaws... -- www.darwinawards.com --- 4 days until Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
On 28/10/2015 07:38, j...@lexoncom.com wrote: i uploaded my inbox with all spam that does not get filtered https://mega.nz/#!IRhlyQLL 1/ that site is slo 2/ you need a decryption key to access it 3/ try pastebin instead -- If you have the urge to reply to all rather than reply to list, you best read http://members.ausics.net/qwerty/
Re: How to get rid of this spam? Spam assassin does not catch it
On 10/27/15 14:16, David Jones wrote: Also - add a highest numbers MX record tarbaby.junkemailfilter.com This will help tune our list to your spam and also get rid of a lot od it. Is this safe to use with greylisting on the lower MX records? I see you temp fail (4xx) all email so it should be safe. Didn't see anything about greylisting side effects on your main web site wiki documentation so I thought I would ask. I filter for about 97,000 unique mailboxes and have been temp failing on a high MX for years but I wasn't sure what it took to "commit several sins" in your logic before it would become blacklisted on your RBL. I know you won't divulge your "secret sauce" and wouldn't expect you to but I would need some assurance that legit email servers trying a higher MX because the lower ones were doing greylisting won't get listed in your RBL. Thanks, Dave Jones Yes - it's greylist safe. I'm looking for a lot of things. I measure data rates. I look at HELO. I look at RDNS. I look for attempts to impersonate other domains. I look to see if it closes the connection with QUIT. I also advertize authentication - but there is no authentication. All passwords are accepted. This attracts hackers that I blacklist. And it wastes spammers resources.
Re: How to get rid of this spam? Spam assassin does not catch it
>> Also - add a highest numbers MX record tarbaby.junkemailfilter.com >> >> This will help tune our list to your spam and also get rid of a lot od it. >> Is this safe to use with greylisting on the lower MX records? I see you temp fail (4xx) all email so it should be safe. Didn't see anything about greylisting side effects on your main web site wiki documentation so I thought I would ask. I filter for about 97,000 unique mailboxes and have been temp failing on a high MX for years but I wasn't sure what it took to "commit several sins" in your logic before it would become blacklisted on your RBL. I know you won't divulge your "secret sauce" and wouldn't expect you to but I would need some assurance that legit email servers trying a higher MX because the lower ones were doing greylisting won't get listed in your RBL. Thanks, Dave Jones > Marc Perkel - Sales/Support > supp...@junkemailfilter.com > http://www.junkemailfilter.com > Junk Email Filter dot com >415-992-3400
Re: How to get rid of this spam? Spam assassin does not catch it
j...@lexoncom.com skrev den 2015-10-27 21:02: SO i setup the dns server. Can i force spam assassin to use localhost for dns or I must reconfigure the host? perldoc Mail::SpamAssassin::Conf see dns server # local.cf dns_server 127.0.0.1
Re: How to get rid of this spam? Spam assassin does not catch it
try this https://www.dropbox.com/s/ngmaryggdelecjq/INBOX.spam?dl=0 it is mbox file with like 1000 spam messages that are not recognized as spam > On 28/10/2015 07:38, j...@lexoncom.com wrote: >> i uploaded my inbox with all spam that does not get filtered >> >> https://mega.nz/#!IRhlyQLL >> > > 1/ that site is slo > 2/ you need a decryption key to access it > 3/ try pastebin instead > > > -- > If you have the urge to reply to all rather than reply to list, you best > read http://members.ausics.net/qwerty/ >
Re: How to get rid of this spam? Spam assassin does not catch it
yes there might be few emails there that were legitimate i cleaned it but i did not have time to do it property are not net/RBL/DNSBL tests enabled by default? i need to review the documentation and see why it does not work > On Tue, 27 Oct 2015, j...@lexoncom.com wrote: > >> try this >> https://www.dropbox.com/s/ngmaryggdelecjq/INBOX.spam?dl=0 >> >> it is mbox file with like 1000 spam messages that are not recognized as >> spam >> > > Are you -sure- all those messages are spam? > One of them was a personal FaceBook update message. > If you ("blwegr...@lexoncom.com") have a FB account then pretty much all > updates > sent to you as a result really cannot be considered spam. > > FWIW, > You are really short-changing your SA by not having the net/RBL/DNSBL > tests > working properly. > > The vast majority of those messages (%96) were tagged as spam by my system > and a > super majority (%83) scored > 20.0 (my SMTP reject threshold). A large > component > of that score was from net/RBL/DNSBL tests. > > -- > Dave Funk University of Iowa > College of Engineering > 319/335-5751 FAX: 319/384-0549 1256 Seamans Center > Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527 > #include > Better is not better, 'standard' is better. B{ >
Re: How to get rid of this spam? Spam assassin does not catch it
Is there a way to learn what bayes learned so far? > On Oct 27, 2015, at 4:35 PM, John Hardinwrote: > >> On Tue, 27 Oct 2015, j...@lexoncom.com wrote: >> >> example mail sa headers: > > Is this from a spam? > >> X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on >> ip-10-254-37-89.us-west-2.compute.internal >> X-Spam-Level: *** >> X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE, > > BAYES_00. You *do* have ham and spam trained, and bayes *is* in use. > > If this is a spam, your Bayes appears to be mistrained. That might explain > why so many spams are getting through. > > If you have autolearn turned on, turn it off. > > Collect hand-classified corpora of several hundred hams and several hundred > spams, then wipe and retrain your Bayes. > > If your userbase is small enough to collect and train on just misclassified > messages, then leave autolearn turned off and just train misclassifications > and messages that don't hit either BAYES_00 or BAYES_99. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- > ...the Fates notice those who buy chainsaws... > -- www.darwinawards.com > --- > 4 days until Halloween
Re: How to get rid of this spam? Spam assassin does not catch it
On Tue, 27 Oct 2015, j...@lexoncom.com wrote: try this https://www.dropbox.com/s/ngmaryggdelecjq/INBOX.spam?dl=0 it is mbox file with like 1000 spam messages that are not recognized as spam Are you -sure- all those messages are spam? One of them was a personal FaceBook update message. If you ("blwegr...@lexoncom.com") have a FB account then pretty much all updates sent to you as a result really cannot be considered spam. FWIW, You are really short-changing your SA by not having the net/RBL/DNSBL tests working properly. The vast majority of those messages (%96) were tagged as spam by my system and a super majority (%83) scored > 20.0 (my SMTP reject threshold). A large component of that score was from net/RBL/DNSBL tests. -- Dave Funk University of Iowa College of Engineering 319/335-5751 FAX: 319/384-0549 1256 Seamans Center Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527 #include Better is not better, 'standard' is better. B{
Re: How to get rid of this spam? Spam assassin does not catch it
On 27 Oct 2015, at 16:02, j...@lexoncom.com wrote: SO i setup the dns server. Can i force spam assassin to use localhost for dns or I must reconfigure the host? You can just change SA, but you should change the whole host to use it if your MTA is running there as well. the MTA is probably doing lookups before SA is passed the message that will benefit SA performance by being in your local cache. Also, if the MTA is handling a substantial amount of inbound mail it is very likely to benefit from having a resolver cache that's local instead of >10ms away across multiple router hops.