Re: How to get rid of this spam? Spam assassin does not catch it

[Reindl Harald]

Am 27.10.2015 um 18:50 schrieb j...@lexoncom.com:

I use spam assassin with razors on ubuntu server.
In recent months i started to get tons of spam.
Spam assassin does not catch it and scores are very low.

Are those emails fabricated so well that they look like legitimate? Can i
do something to catch those as spam?

I moved them all to one folder called spam and i run this command every 5
minutes on that folder:
sa-learn --spam --mbox /home/username/mail/INBOX.spam
but it does not help


do you have enough *ham* trained?
is the bayes-db of this user *realy* used at scan time
what are the SA-headers of mails passing through?

sorry but you need to provide basic informations



signature.asc
Description: OpenPGP digital signature

Re: How to get rid of this spam? Spam assassin does not catch it

[Axb]

On 10/27/2015 06:50 PM, j...@lexoncom.com wrote:

I use spam assassin with razors on ubuntu server.
In recent months i started to get tons of spam.
Spam assassin does not catch it and scores are very low.

Are those emails fabricated so well that they look like legitimate? Can i
do something to catch those as spam?

I moved them all to one folder called spam and i run this command every 5
minutes on that folder:
sa-learn --spam --mbox /home/username/mail/INBOX.spam
but it does not help

It seems like every spam email is fabricated in different way.

Anyone has any idea how to catch those?
Why spam assassin does not catch it?


attached is the list showing subject and from for the recent spams i get.


Suggest you pastebin a few samples  - subjects on their own are not of 
much use.

How to get rid of this spam? Spam assassin does not catch it

[junk]

I use spam assassin with razors on ubuntu server.
In recent months i started to get tons of spam.
Spam assassin does not catch it and scores are very low.

Are those emails fabricated so well that they look like legitimate? Can i
do something to catch those as spam?

I moved them all to one folder called spam and i run this command every 5
minutes on that folder:
sa-learn --spam --mbox /home/username/mail/INBOX.spam
but it does not help

It seems like every spam email is fabricated in different way.

Anyone has any idea how to catch those?
Why spam assassin does not catch it?


attached is the list showing subject and from for the recent spams i get.






subcject_from.txt1
Description: Binary data

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

I understand now.
sa-learn --ham --no-rebuild ham_directory
sa-learn --spam --no-rebuild spam_directory
sa-learn --rebuild

so would the best practice to be move spam to spam folder and learn as spam
and learn all other folders as ham and then rebuild.
The inbox would never be scanned as it might have new span and not spam
messages.

I would need some script to go through all messages for all users except
the spam folder to learn as HAM.

>
>
> Am 27.10.2015 um 20:19 schrieb j...@lexoncom.com:
>> I dont use any ham training
>
> then you can't expect bayes to work at all because how do you expect the
> bayes filter to know the *difference* of ham and spam signs?
>
> https://wiki.apache.org/spamassassin/BayesFaq
>
>

Re: How to get rid of this spam? Spam assassin does not catch it

[Reindl Harald]

Am 27.10.2015 um 21:02 schrieb j...@lexoncom.com:

SO i setup the dns server.
Can i force spam assassin to use localhost for dns or I must reconfigure
the host?


i recommend to read at least basic docs
google "spamassassin dns" leads to 
http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html and 
CTRL+F "dns" leads to the following (the docs would also have mentioned 
that you need at least 200 spam *and* ham samples for bayes to work)


dns_server ip-addr-port (default: entries provided by Net::DNS)

Specifies an IP address of a DNS server, and optionally its port number. 
The dns_server directive may be specified multiple times, each entry 
adding to a list of available resolving name servers. The ip-addr-port 
argument can either be an IPv4 or IPv6 address, optionally enclosed in 
brackets, and optionally followed by a colon and a port number. In 
absence of a port number a standard port number 53 is assumed. When an 
IPv6 address is specified along with a port number, the address must be 
enclosed in brackets to avoid parsing ambiguity regarding a colon 
separator. A scoped link-local IP address is allowed (assuming 
underlying modules allow it).


 Examples : dns_server 127.0.0.1 dns_server 127.0.0.1:53 dns_server 
[127.0.0.1]:53 dns_server [::1]:53 dns_server fe80::1%lo0 dns_server 
[fe80::1%lo0]:53


In absence of dns_server directives, the list of name servers is 
provided by Net::DNS module, which typically obtains the list from 
/etc/resolv.conf, but this may be platform dependent. Please consult the 
Net::DNS::Resolver documentation for details.



On Tue, 27 Oct 2015, j...@lexoncom.com wrote:


X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,

RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS,
SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no
autolearn_force=no
version=3.4.0


URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server for
SpamAssassin to use. You're apparently doing DNS blacklist queries via a
public DNS server (your ISPs?) and the aggregate traffic level is
exceeding the URIBL free usage limits.




signature.asc
Description: OpenPGP digital signature

Re: Spamassassin and amavisd-new wont' check (faked) bounce with zip-archive/exe (maleware)

[Matus UHLAR - fantomas]

On 26.10.15 13:09, Django [BOfH] wrote:

Hello list, dear Marc!


correction: Helo spamassassn-users list - it has nothing to do with
attachment or virus scanning. 
you should have contacted amavisd-new list

http://lists.amavis.org/cgi-bin/mailman/listinfo/amavis-users


So I tried to understand, why our AMaVis's allowed those faked
bounce-messages with mailware.



--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam = (S)tupid (P)eople's (A)dvertising (M)ethod

Re: spf records and cnames

[Reindl Harald]

Am 27.10.2015 um 20:15 schrieb Matus UHLAR - fantomas:

it does not explain why should it cause problems for HELO SPF.  as I have
already noted, using CNAME for HELO violates SMTP RFC, so there's
technically no reason to follow CNAME expecially in these cases


that is nonsense

the goal of HELO SPF and SPF records for every hostname is to make 
forging impossible - the SMTP RFC don't matter in that context - the 
only question is would a SPF policyd reject a message


[harry@srv-rhsoft:~]$ nslookup www.rhsoft.net 8.8.8.8
Server: 8.8.8.8
Address:8.8.8.8#53
Non-authoritative answer:
www.rhsoft.net  canonical name = proxy.thelounge.net.
Name:   proxy.thelounge.net
Address: 91.118.73.4

http://www.openspf.org/Why?s=mfrom;id=t...@www.rhsoft.net;ip=89.207.169.8

[harry@srv-rhsoft:~]$ dig TXT www.rhsoft.net @8.8.8.8
; <<>> DiG 9.10.2-P4-RedHat-9.10.2-5.P4.fc22 <<>> TXT www.rhsoft.net 
@8.8.8.8

;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42894
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.rhsoft.net.IN  TXT

;; ANSWER SECTION:
www.rhsoft.net. 19174   IN  CNAME   proxy.thelounge.net.
proxy.thelounge.net.21599   IN  TXT "v=spf1 a 
ip4:91.118.73.0/24 ip4:95.129.202.170 -all"




signature.asc
Description: OpenPGP digital signature

Re: How to get rid of this spam? Spam assassin does not catch it

[Reindl Harald]

Am 27.10.2015 um 20:19 schrieb j...@lexoncom.com:

I dont use any ham training


then you can't expect bayes to work at all because how do you expect the 
bayes filter to know the *difference* of ham and spam signs?


https://wiki.apache.org/spamassassin/BayesFaq



signature.asc
Description: OpenPGP digital signature

Re: How to get rid of this spam? Spam assassin does not catch it

[John Hardin]

On Tue, 27 Oct 2015, j...@lexoncom.com wrote:


X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,

RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS,
SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no
version=3.4.0


URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server for 
SpamAssassin to use. You're apparently doing DNS blacklist queries via a 
public DNS server (your ISPs?) and the aggregate traffic level is 
exceeding the URIBL free usage limits.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  ...the Fates notice those who buy chainsaws...
  -- www.darwinawards.com
---
 4 days until Halloween

Re: How to get rid of this spam? Spam assassin does not catch it

[Reindl Harald]

Am 27.10.2015 um 20:23 schrieb Marc Perkel:

Also - add a highest numbers MX record tarbaby.junkemailfilter.com
This will help tune our list to your spam and also get rid of a lot od it


how do you distinct fools like facebook at the moment always trying 
first the backup-MX (which is here a postscreen honeypot always 
repsonding 4xx if the sending IP is not on eough blacklists for score 
based reject) and real spammers?


don't get me wrong - i use "tarbaby.junkemailfilter.com" but *only* for 
honeypot domains which don't expect legit mail for sure




signature.asc
Description: OpenPGP digital signature

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

SO i setup the dns server.
Can i force spam assassin to use localhost for dns or I must reconfigure
the host?

> On Tue, 27 Oct 2015, j...@lexoncom.com wrote:
>
>> X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,
>>  
>> RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS,
>>  SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no
>> autolearn_force=no
>>  version=3.4.0
>
> URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server for
> SpamAssassin to use. You're apparently doing DNS blacklist queries via a
> public DNS server (your ISPs?) and the aggregate traffic level is
> exceeding the URIBL free usage limits.
>
> --
>   John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
>   jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
>   key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
> ---
>...the Fates notice those who buy chainsaws...
>-- www.darwinawards.com
> ---
>   4 days until Halloween
>

Re: How to get rid of this spam? Spam assassin does not catch it

[Marc Perkel]

You can use my black and white lists. It should help.

header __RCVD_IN_HOSTKARMA 
eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.')
describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter
tflags __RCVD_IN_HOSTKARMA net
 
header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.1')

describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE
tflags RCVD_IN_HOSTKARMA_W net nice
score RCVD_IN_HOSTKARMA_W -5
 
header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.2')

describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK
tflags RCVD_IN_HOSTKARMA_BL net
score RCVD_IN_HOSTKARMA_BL 3.0
 
header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal', '127.0.0.4')

describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN
tflags RCVD_IN_HOSTKARMA_BR net
score RCVD_IN_HOSTKARMA_BR 1.0


Also - add a highest numbers MX record tarbaby.junkemailfilter.com

This will help tune our list to your spam and also get rid of a lot od it.

On 10/27/15 10:50, j...@lexoncom.com wrote:

sa-learn --spam --mbox /home/username/mail/INBOX.spam


--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

Re: How to get rid of this spam? Spam assassin does not catch it

[Marc Perkel]

Yes - add to local.cf

As the highest numbered MX record tarbaby,junkemailfilter.com usually 
only sees virus bots. It never accepts email and refuses with a 4xx 
error in case something legit hits it. So we never see your email.


It also doesn't blacklist good email. The sender has to commit several 
"sins" before it is blacklisted. So it's safe - gets rid of some spam, 
and helps tune our blacklists to include more bad actors.



On 10/27/15 12:48, j...@lexoncom.com wrote:

can you explain how this works?
Do i add this to spam local.cf file?

would not

Also - add a highest numbers MX record tarbaby.junkemailfilter.com

allow your servers to see my emails?

thx



You can use my black and white lists. It should help.

header __RCVD_IN_HOSTKARMA
eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.')
describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter
tflags __RCVD_IN_HOSTKARMA net

header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal',
'127.0.0.1')
describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE
tflags RCVD_IN_HOSTKARMA_W net nice
score RCVD_IN_HOSTKARMA_W -5

header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal',
'127.0.0.2')
describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK
tflags RCVD_IN_HOSTKARMA_BL net
score RCVD_IN_HOSTKARMA_BL 3.0

header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal',
'127.0.0.4')
describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN
tflags RCVD_IN_HOSTKARMA_BR net
score RCVD_IN_HOSTKARMA_BR 1.0


Also - add a highest numbers MX record tarbaby.junkemailfilter.com

This will help tune our list to your spam and also get rid of a lot od it.

On 10/27/15 10:50, j...@lexoncom.com wrote:

sa-learn --spam --mbox /home/username/mail/INBOX.spam

--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400









--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

Re: spf records and cnames

[Matus UHLAR - fantomas]

On 22.10.15 00:19, Reindl Harald wrote:

otherwise you would not be able to set a SPF-record for your CNAMES
and "reject_unknown_sender_domain" won't hit for a forged subdomain
because it exists - so SPF *must* work for CNAMES or the whole
intention for HELO SPF would not work



Am 22.10.2015 um 13:55 schrieb Matus UHLAR - fantomas:

I don't get this. HELO must be canonical name, so it must not be CNAME.
Thus, there's no need to follow CNAMEs in SPF when checking for HELO.
when you check HELO, the CNAME should be treated as error


On 22.10.15 13:58, Reindl Harald wrote:

see first repsonse to that thread


it does not explain why should it cause problems for HELO SPF.  as I have
already noted, using CNAME for HELO violates SMTP RFC, so there's technically no
reason to follow CNAME expecially in these cases - it's alredy broken and
failing the check would be (imho) proper reaction.


what do i mean with "is always followed"?

[...]


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Support bacteria - they're the only culture some people have. 

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

can you explain how this works?
Do i add this to spam local.cf file?

would not
> Also - add a highest numbers MX record tarbaby.junkemailfilter.com
allow your servers to see my emails?

thx


> You can use my black and white lists. It should help.
>
> header __RCVD_IN_HOSTKARMA
> eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.')
> describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter
> tflags __RCVD_IN_HOSTKARMA net
>
> header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal',
> '127.0.0.1')
> describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE
> tflags RCVD_IN_HOSTKARMA_W net nice
> score RCVD_IN_HOSTKARMA_W -5
>
> header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal',
> '127.0.0.2')
> describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK
> tflags RCVD_IN_HOSTKARMA_BL net
> score RCVD_IN_HOSTKARMA_BL 3.0
>
> header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal',
> '127.0.0.4')
> describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN
> tflags RCVD_IN_HOSTKARMA_BR net
> score RCVD_IN_HOSTKARMA_BR 1.0
>
>
> Also - add a highest numbers MX record tarbaby.junkemailfilter.com
>
> This will help tune our list to your spam and also get rid of a lot od it.
>
> On 10/27/15 10:50, j...@lexoncom.com wrote:
>> sa-learn --spam --mbox /home/username/mail/INBOX.spam
>
> --
> Marc Perkel - Sales/Support
> supp...@junkemailfilter.com
> http://www.junkemailfilter.com
> Junk Email Filter dot com
> 415-992-3400
>
>

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

thx, yes i did that but found old doc and that option was not available:
https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html

>
> Am 27.10.2015 um 21:02 schrieb j...@lexoncom.com:
>> SO i setup the dns server.
>> Can i force spam assassin to use localhost for dns or I must reconfigure
>> the host?
>
> i recommend to read at least basic docs
> google "spamassassin dns" leads to
> http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html
> and
> CTRL+F "dns" leads to the following (the docs would also have mentioned
> that you need at least 200 spam *and* ham samples for bayes to work)
>
> dns_server ip-addr-port (default: entries provided by Net::DNS)
>
> Specifies an IP address of a DNS server, and optionally its port number.
> The dns_server directive may be specified multiple times, each entry
> adding to a list of available resolving name servers. The ip-addr-port
> argument can either be an IPv4 or IPv6 address, optionally enclosed in
> brackets, and optionally followed by a colon and a port number. In
> absence of a port number a standard port number 53 is assumed. When an
> IPv6 address is specified along with a port number, the address must be
> enclosed in brackets to avoid parsing ambiguity regarding a colon
> separator. A scoped link-local IP address is allowed (assuming
> underlying modules allow it).
>
>   Examples : dns_server 127.0.0.1 dns_server 127.0.0.1:53 dns_server
> [127.0.0.1]:53 dns_server [::1]:53 dns_server fe80::1%lo0 dns_server
> [fe80::1%lo0]:53
>
> In absence of dns_server directives, the list of name servers is
> provided by Net::DNS module, which typically obtains the list from
> /etc/resolv.conf, but this may be platform dependent. Please consult the
> Net::DNS::Resolver documentation for details.
>
>>> On Tue, 27 Oct 2015, j...@lexoncom.com wrote:
>>>
 X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,

 RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS,
SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no
 autolearn_force=no
version=3.4.0
>>>
>>> URIBL_BLOCKED. Set up a local recursing (NOT forwarding!) DNS server
>>> for
>>> SpamAssassin to use. You're apparently doing DNS blacklist queries via
>>> a
>>> public DNS server (your ISPs?) and the aggregate traffic level is
>>> exceeding the URIBL free usage limits.
>
>

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

I dont use any ham training.Should I scan all my folders with this command:
sa-learn --ham --mbox /home/username/mail/foldername

"is the bayes-db of this user *realy* used at scan time"
how do i check that?


I use the procemail to pass all mail through spam assassin.
I use default ubuntu setup with Razors enabled.
It does catches spam but not the one i attached in original post.

example mail sa headers:

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
ip-10-254-37-89.us-west-2.compute.internal
X-Spam-Level: ***
X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,

RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SPF_HELO_PASS,
SPF_PASS,URIBL_BLOCKED,URIBL_DBL_SPAM autolearn=no autolearn_force=no
version=3.4.0


ubuntu@ip-10-254-37-89:~$ cat /etc/spamassassin/local.cf
# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
# Only a small subset of options are listed below
#
###

#   Add *SPAM* to the Subject header of spam e-mails
#
# rewrite_header Subject *SPAM*


#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
# report_safe 1


#   Set which networks or hosts are considered 'trusted' by your mail
#   server (i.e. not spammers)
#
# trusted_networks 212.17.35.


#   Set file-locking method (flock is not safe over NFS, but is faster)
#
# lock_method flock


#   Set the threshold at which a message is considered spam (default: 5.0)
#
# required_score 5.0


#   Use Bayesian classifier (default: 1)
#
# use_bayes 1


#   Bayesian classifier auto-learning (default: 1)
#
# bayes_auto_learn 1


#   Set headers which may provide inappropriate cues to the Bayesian
#   classifier
#
# bayes_ignore_header X-Bogosity
# bayes_ignore_header X-Spam-Flag
# bayes_ignore_header X-Spam-Status


#   Some shortcircuiting, if the plugin is enabled
#
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
#
#   default: strongly-whitelisted mails are *really* whitelisted now, if the
#   shortcircuiting plugin is active, causing early exit to save CPU load.
#   Uncomment to turn this on
#
# shortcircuit USER_IN_WHITELIST   on
# shortcircuit USER_IN_DEF_WHITELIST   on
# shortcircuit USER_IN_ALL_SPAM_TO on
# shortcircuit SUBJECT_IN_WHITELISTon

#   the opposite; blacklisted mails can also save CPU
#
# shortcircuit USER_IN_BLACKLIST   on
# shortcircuit USER_IN_BLACKLIST_TOon
# shortcircuit SUBJECT_IN_BLACKLISTon

#   if you have taken the time to correctly specify your "trusted_networks",
#   this is another good way to save CPU
#
# shortcircuit ALL_TRUSTED on

#   and a well-trained bayes DB can save running rules, too
#
# shortcircuit BAYES_99spam
# shortcircuit BAYES_00ham

endif # Mail::SpamAssassin::Plugin::Shortcircuit

# Vipul's Razor options.
use_razor2  1
#razor_timeout   10
razor_config /etc/razor/razor-agent.conf
loadplugin Mail::SpamAssassin::Plugin::Razor2

required_hits 5
report_safe 0
rewrite_header Subject [SPAM]


procmail setup:

:0fw: spamassassin.lock
* < 256000
| spamassassin

# Mails with a score of 15 or higher are almost certainly spam (with 0.05%
# false positives according to rules/STATISTICS.txt). Let's put them in a
# different mbox. (This one is optional.)
:0:
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
/var/spool/mail/junk


# All mail tagged as spam (eg. with a score higher than the set threshold)
# is moved to "probably-spam".
:0:
* ^X-Spam-Status: Yes
/var/spool/mail/junk


>
>
> Am 27.10.2015 um 18:50 schrieb j...@lexoncom.com:
>> I use spam assassin with razors on ubuntu server.
>> In recent months i started to get tons of spam.
>> Spam assassin does not catch it and scores are very low.
>>
>> Are those emails fabricated so well that they look like legitimate? Can
>> i
>> do something to catch those as spam?
>>
>> I moved them all to one folder called spam and i run this command every
>> 5
>> minutes on that folder:
>> sa-learn --spam --mbox /home/username/mail/INBOX.spam
>> but it does not help
>
> do you have enough *ham* trained?
> is the bayes-db of this user *realy* used at scan time
> what are the SA-headers of mails passing through?
>
> sorry but you need to provide basic informations
>
>

Re: How to get rid of this spam? Spam assassin does not catch it

[Reindl Harald]

Am 27.10.2015 um 20:31 schrieb j...@lexoncom.com:

I understand now.
sa-learn --ham --no-rebuild ham_directory
sa-learn --spam --no-rebuild spam_directory
sa-learn --rebuild

so would the best practice to be move spam to spam folder and learn as spam
and learn all other folders as ham and then rebuild.
The inbox would never be scanned as it might have new span and not spam
messages.

I would need some script to go through all messages for all users except
the spam folder to learn as HAM.


i would *never ever* make such things automated

i have just a physical folder "spam" and and physical folder "ham" wil 
single .eml files and hand selected samples - currenmtly they are feeded 
by a PHP script receiving IMAP messages from the spam/ham folders, 
testing them via CLI in case of spam if they are not already BAYES_999 
and then save eml files


over the last month i also trained BAYES_999 to find as much as possible 
common spam signs, with 2.5 Mio tokens there is no longer need for that, 
the bayes-db has a hitrate of 99.9% by filter out the remaining 8-10% 
junk, anything else is cuaght long before spamass-milter by blacklists 
/which are not working or you because once more somebody i using a 
shared DNS resolver instead doing recursion on it's own caching server)


0  48739SPAM
0  20549HAM
02256265TOKEN

insgesamt 70M
-rw--- 1 sa-milt sa-milt 9,7M 2015-10-27 20:08 bayes_seen
-rw--- 1 sa-milt sa-milt  81M 2015-10-27 20:08 bayes_toks

BAYES_0025591   70.79 %
BAYES_05  7392.04 %
BAYES_20  9322.57 %
BAYES_40  7892.18 %
BAYES_50 3981   11.01 %
BAYES_60  4761.31 %
BAYES_80  4181.15 %
BAYES_95  2900.80 %
BAYES_99 29348.11 %
BAYES_99926307.27 %

DELIVERED   49373   93.82 %
DNSWL   46277   87.94 %
SPF 33497   63.65 %
SPF/DKIM WL 15849   30.11 %
SHORTCIRCUIT16426   31.21 %

BLOCKED  44358.42 %
SPAMMY   41187.82 %92.85 % (OF TOTAL BLOCKED)


especially when it comes to random users they often move something to 
spam just because they are too lazy or too stupid for unsubscribe (seen 
that even for invoice mails of their energy supplier coming back from 
AOL as abuse-feedback-loop including the invoice with their address and 
power consumations over the last month)


the same for ham: just because a message is in a different folder than 
inbox/spam don't make it to a ham message, just a simple sieve-rule my 
move them and it was slipped junk


for every wrong classified message (no matter in what direction) in the 
end you likely need 5 messages to compare the damage and in the end you 
will again end with a bayes having no clue at all


train your bayes careful, by hand and try to keep a blance of ham/spam 
for best results



Am 27.10.2015 um 20:19 schrieb j...@lexoncom.com:

I dont use any ham training


then you can't expect bayes to work at all because how do you expect the
bayes filter to know the *difference* of ham and spam signs?

https://wiki.apache.org/spamassassin/BayesFaq




signature.asc
Description: OpenPGP digital signature

Re: How to get rid of this spam? Spam assassin does not catch it

[Benny Pedersen]

j...@lexoncom.com skrev den 2015-10-27 21:33:
thx, yes i did that but found old doc and that option was not 
available:

https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html


this is why i suggest to check local docs first, if not found local, 
check atleast to diff queueries on internet to confirm it valid options, 
google is fine, but :)


perldoc Mail::SpamAssassin::Conf

is trusted

Re: How to get rid of this spam? Spam assassin does not catch it

[John Hardin]

On Tue, 27 Oct 2015, j...@lexoncom.com wrote:


example mail sa headers:


Is this from a spam?


X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
ip-10-254-37-89.us-west-2.compute.internal
X-Spam-Level: ***
X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,


BAYES_00. You *do* have ham and spam trained, and bayes *is* in use.

If this is a spam, your Bayes appears to be mistrained. That might explain 
why so many spams are getting through.


If you have autolearn turned on, turn it off.

Collect hand-classified corpora of several hundred hams and several 
hundred spams, then wipe and retrain your Bayes.


If your userbase is small enough to collect and train on just 
misclassified messages, then leave autolearn turned off and just train 
misclassifications and messages that don't hit either BAYES_00 or 
BAYES_99.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  ...the Fates notice those who buy chainsaws...
  -- www.darwinawards.com
---
 4 days until Halloween

Re: How to get rid of this spam? Spam assassin does not catch it

[Noel Butler]

On 28/10/2015 07:38, j...@lexoncom.com wrote:

i uploaded my inbox with all spam that does not get filtered

https://mega.nz/#!IRhlyQLL



1/ that site is slo
2/ you need a decryption key to access it
3/ try pastebin instead


--
If you have the urge to reply to all rather than reply to list, you best
read  http://members.ausics.net/qwerty/

Re: How to get rid of this spam? Spam assassin does not catch it

[Marc Perkel]

On 10/27/15 14:16, David Jones wrote:

Also - add a highest numbers MX record tarbaby.junkemailfilter.com

This will help tune our list to your spam and also get rid of a lot od it.


Is this safe to use with greylisting on the lower MX records?  I see you
temp fail (4xx) all email so it should be safe.  Didn't see anything about
greylisting side effects on your main web site wiki documentation so I
thought I would ask.
I filter for about 97,000 unique mailboxes and have been temp failing
on a high MX for years but I wasn't sure what it took to "commit
several sins" in your logic before it would become blacklisted on your
RBL.  I know you won't divulge your "secret sauce" and wouldn't
expect you to but I would need some assurance that legit email
servers trying a higher MX because the lower ones were doing
greylisting won't get listed in your RBL.

Thanks,
Dave Jones



Yes - it's greylist safe.

I'm looking for a lot of things. I measure data rates. I look at HELO. I 
look at RDNS. I look for attempts to impersonate other domains. I look 
to see if it closes the connection with QUIT. I also advertize 
authentication - but there is no authentication. All passwords are 
accepted. This attracts hackers that I blacklist. And it wastes spammers 
resources.

Re: How to get rid of this spam? Spam assassin does not catch it

[David Jones]

>> Also - add a highest numbers MX record tarbaby.junkemailfilter.com
>>
>> This will help tune our list to your spam and also get rid of a lot od it.
>>
Is this safe to use with greylisting on the lower MX records?  I see you
temp fail (4xx) all email so it should be safe.  Didn't see anything about
greylisting side effects on your main web site wiki documentation so I
thought I would ask.
I filter for about 97,000 unique mailboxes and have been temp failing
on a high MX for years but I wasn't sure what it took to "commit
several sins" in your logic before it would become blacklisted on your
RBL.  I know you won't divulge your "secret sauce" and wouldn't
expect you to but I would need some assurance that legit email
servers trying a higher MX because the lower ones were doing
greylisting won't get listed in your RBL.

Thanks,
Dave Jones

> Marc Perkel - Sales/Support
> supp...@junkemailfilter.com
> http://www.junkemailfilter.com
> Junk Email Filter dot com
>415-992-3400

Re: How to get rid of this spam? Spam assassin does not catch it

[Benny Pedersen]

j...@lexoncom.com skrev den 2015-10-27 21:02:

SO i setup the dns server.
Can i force spam assassin to use localhost for dns or I must 
reconfigure

the host?


perldoc Mail::SpamAssassin::Conf

see dns server

# local.cf

dns_server 127.0.0.1

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

try this
https://www.dropbox.com/s/ngmaryggdelecjq/INBOX.spam?dl=0

it is mbox file with like 1000 spam messages that are not recognized as spam

> On 28/10/2015 07:38, j...@lexoncom.com wrote:
>> i uploaded my inbox with all spam that does not get filtered
>>
>> https://mega.nz/#!IRhlyQLL
>>
>
> 1/ that site is slo
> 2/ you need a decryption key to access it
> 3/ try pastebin instead
>
>
> --
> If you have the urge to reply to all rather than reply to list, you best
> read  http://members.ausics.net/qwerty/
>

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

yes there might be few emails there that were legitimate
i cleaned it but i did not have time to do it property

are not
net/RBL/DNSBL tests
enabled by default?

i need to review the documentation and see why it does not work


> On Tue, 27 Oct 2015, j...@lexoncom.com wrote:
>
>> try this
>> https://www.dropbox.com/s/ngmaryggdelecjq/INBOX.spam?dl=0
>>
>> it is mbox file with like 1000 spam messages that are not recognized as
>> spam
>>
>
> Are you -sure- all those messages are spam?
> One of them was a personal FaceBook update message.
> If you ("blwegr...@lexoncom.com") have a FB account then pretty much all
> updates
> sent to you as a result really cannot be considered spam.
>
> FWIW,
> You are really short-changing your SA by not having the net/RBL/DNSBL
> tests
> working properly.
>
> The vast majority of those messages (%96) were tagged as spam by my system
> and a
> super majority (%83) scored > 20.0 (my SMTP reject threshold). A large
> component
> of that score was from net/RBL/DNSBL tests.
>
> --
> Dave Funk  University of Iowa
> College of Engineering
> 319/335-5751   FAX: 319/384-0549   1256 Seamans Center
> Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
> #include 
> Better is not better, 'standard' is better. B{
>

Re: How to get rid of this spam? Spam assassin does not catch it

[junk]

Is there a way to learn what bayes learned so far?

> On Oct 27, 2015, at 4:35 PM, John Hardin  wrote:
> 
>> On Tue, 27 Oct 2015, j...@lexoncom.com wrote:
>> 
>> example mail sa headers:
> 
> Is this from a spam?
> 
>> X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
>>   ip-10-254-37-89.us-west-2.compute.internal
>> X-Spam-Level: ***
>> X-Spam-Status: No, score=3.1 required=5.0 tests=BAYES_00,HTML_MESSAGE,
> 
> BAYES_00. You *do* have ham and spam trained, and bayes *is* in use.
> 
> If this is a spam, your Bayes appears to be mistrained. That might explain 
> why so many spams are getting through.
> 
> If you have autolearn turned on, turn it off.
> 
> Collect hand-classified corpora of several hundred hams and several hundred 
> spams, then wipe and retrain your Bayes.
> 
> If your userbase is small enough to collect and train on just misclassified 
> messages, then leave autolearn turned off and just train misclassifications 
> and messages that don't hit either BAYES_00 or BAYES_99.
> 
> -- 
> John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
> jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
> ---
>  ...the Fates notice those who buy chainsaws...
>  -- www.darwinawards.com
> ---
> 4 days until Halloween

Re: How to get rid of this spam? Spam assassin does not catch it

[David B Funk]

On Tue, 27 Oct 2015, j...@lexoncom.com wrote:


try this
https://www.dropbox.com/s/ngmaryggdelecjq/INBOX.spam?dl=0

it is mbox file with like 1000 spam messages that are not recognized as spam



Are you -sure- all those messages are spam?
One of them was a personal FaceBook update message.
If you ("blwegr...@lexoncom.com") have a FB account then pretty much all updates
sent to you as a result really cannot be considered spam.

FWIW,
You are really short-changing your SA by not having the net/RBL/DNSBL tests 
working properly.


The vast majority of those messages (%96) were tagged as spam by my system and a 
super majority (%83) scored > 20.0 (my SMTP reject threshold). A large component

of that score was from net/RBL/DNSBL tests.

--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: How to get rid of this spam? Spam assassin does not catch it

[Bill Cole]

On 27 Oct 2015, at 16:02, j...@lexoncom.com wrote:


SO i setup the dns server.
Can i force spam assassin to use localhost for dns or I must 
reconfigure

the host?


You can just change SA, but you should change the whole host to use it 
if your MTA is running there as well. the MTA is probably doing lookups 
before SA is passed the message that will benefit SA performance by 
being in your local cache. Also, if the MTA is handling a substantial 
amount of inbound mail it is very likely to benefit from having a 
resolver cache that's local instead of >10ms away across multiple router 
hops.