Re: USER_IN_WHITELIST

[Bill Cole]

On 6 Jul 2016, at 23:10, lorenzo wrote:

[...]
The output from spamassassin -t -D < In-whitelist.txt gives the 
answer, I believe:


address hefg...@hkjhkjhk.onmicrosoft.com matches whitelist or 
blacklist regexp: ^.*microsoft\.com$


Very sneaky. I think I can handle this one from here.
Thanks again.


Happy to be of help.

For what it's worth: *.onmicrosoft.com domains are part of free trials 
of Office365 and generate almost entirely spam. I suppose one could be a 
regular paying O365 customer and keep that free domain, but no one who 
does that can care much about their email. Spammers have been using 
those domains for years and MS really seems not to care about the fact 
that they've become a de facto indication of spam.

Re: USER_IN_WHITELIST

[lorenzo]

> On Jul 6, 2016, at 8:50 PM, Bill Cole 
>  wrote:
> 
> On 6 Jul 2016, at 21:13, Lorenzo Thurman wrote:
> 
>> I’ve been receiving some spam where spamassassin identifies the sender with 
>> USER_IN_WHITELIST. These senders (or domains) are most definitely not in my 
>> whitelist. How can I get around this problem?
> 
> There are so many relevant variables unspecified that no one here has any 
> hope of solving your problem.
> 
> To make it easier for us, please provide more information:
> 
> 1. How are you using SpamAssassin? Specifically, if you have it hooked into 
> an MTA like Postfix or Sendmail, tell us which one AND what mechanism you are 
> using to integrate SA and the MTA.
> 
> 2. If your system involved the use of spamd, what are its arguments and what 
> user is it running as?
> 
> 3. If you scan a message with this problem manually by piping it into 
> 'spamassassin -t -D' what does the resulting flood of debugging information 
> say about what address it is finding as being in the whitelist?
> 

Ah, ok. Here’s some info:
spamassassin v3.4.0 - Postfix 2.11.0  Ubuntu 14.04
/usr/sbin/spamd --create-prefs --max-children 5 --helper-home-dir -d 
--pidfile=/var/run/spamd.pid

In /etc/postfix/master.cf
smtp  inet  n   -   -   -   -   smtpd -vvv -o 
content_filter=spamassassin
spamassassin unix - n   n   -   -   pipe flags=Rq 
user=nobody argv=/usr/bin/spamfilter.sh -oi -f ${sender} ${recipient}

The output from spamassassin -t -D < In-whitelist.txt gives the answer, I 
believe:

address hefg...@hkjhkjhk.onmicrosoft.com matches whitelist or blacklist regexp: 
^.*microsoft\.com$

Very sneaky. I think I can handle this one from here.
Thanks again.

Re: USER_IN_WHITELIST

[Bill Cole]

On 6 Jul 2016, at 21:58, David B Funk wrote:

> On Wed, 6 Jul 2016, Lorenzo Thurman wrote:
>
>> I’ve been receiving some spam where spamassassin identifies the sender with 
>> USER_IN_WHITELIST. These senders (or domains) are
>> most definitely not in my whitelist. How can I get around this problem?Thanks
>>
>
> SpamAssassin comes with some built-in whitelists (which should be pretty safe 
> to
> use). Look in your SA rules kit for things like: 60_whitelist.cf 
> 60_whitelist_dkim.cf and 60_whitelist_spf.cf

Those should not cause USER_IN_WHITELIST matches but rather 
USER_IN_DEF_WHITELIST and similarly labeled forms of the SPF and DKIM 
variations.


signature.asc
Description: OpenPGP digital signature

Re: USER_IN_WHITELIST

[David B Funk]

On Wed, 6 Jul 2016, Lorenzo Thurman wrote:


I’ve been receiving some spam where spamassassin identifies the sender with 
USER_IN_WHITELIST. These senders (or domains) are
most definitely not in my whitelist. How can I get around this problem?Thanks



SpamAssassin comes with some built-in whitelists (which should be pretty safe to
use). Look in your SA rules kit for things like: 60_whitelist.cf 
60_whitelist_dkim.cf and 60_whitelist_spf.cf

You might also have some 3'rd party rules files that contain whitelists.

You can explicitly negate the effect of an entry from one of these files by
using the appropriate "unwhitelist_from" type configuration statements in your
local.cf config files.

Theoretically you could edit the system config files but those edits could be
lost with the next system rules update, so using the unwhitelist_from technique
is the way to go.

--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

smime.p7s
Description: S/MIME Cryptographic Signature

Re: USER_IN_WHITELIST

[Bill Cole]

On 6 Jul 2016, at 21:13, Lorenzo Thurman wrote:

I’ve been receiving some spam where spamassassin identifies the 
sender with USER_IN_WHITELIST. These senders (or domains) are most 
definitely not in my whitelist. How can I get around this problem?


There are so many relevant variables unspecified that no one here has 
any hope of solving your problem.


To make it easier for us, please provide more information:

1. How are you using SpamAssassin? Specifically, if you have it hooked 
into an MTA like Postfix or Sendmail, tell us which one AND what 
mechanism you are using to integrate SA and the MTA.


2. If your system involved the use of spamd, what are its arguments and 
what user is it running as?


3. If you scan a message with this problem manually by piping it into 
'spamassassin -t -D' what does the resulting flood of debugging 
information say about what address it is finding as being in the 
whitelist?

USER_IN_WHITELIST

[Lorenzo Thurman]

I’ve been receiving some spam where spamassassin identifies the sender with 
USER_IN_WHITELIST. These senders (or domains) are most definitely not in my 
whitelist. How can I get around this problem?
Thanks

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 17:35 schrieb John Hardin:

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?


sounds good and at leat i don't get any error by using 
unbound-1.5.8-2.fc23.x86_64 and the follwoing settings


cache-min-ttl: 600
cache-max-ttl: 43200
cache-max-negative-ttl: 100

when it works as expected it should lead in not so often expire heavily 
used crap domains without take too long for realize new listings and at 
least makes the problem nit so big as now


thanks god then normal DNSBL/DNSWL are not affected becaus ethey are 
used also in prostscreen for weighting and so at the moment SA is using 
them the cache is always hot





signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[John Hardin]

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  So Microsoft's invented the ASCII equivalent to ugly ink spots that
  appear on your letter when your pen is malfunctioning.
 -- Greg Andrews, about Microsoft's way to encode apostrophes
---
 Tomorrow: Robert Heinlein's 109th birthday

Re: URIBL randomly not triggered for the same message

[John Hardin]

On Wed, 6 Jul 2016, Reindl Harald wrote:




Am 06.07.2016 um 14:36 schrieb RW:

 On Tue, 5 Jul 2016 14:01:17 +0200
 Reindl Harald wrote:

>  since there is a local unbound-cache with
> 
>cache-min-ttl: 300


thanks for the hint, but look at
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335#c8

reduce the value would make the problem even worser because what i observe is 
that after TTL is reached and unbound needs to query again the at least first 
question leads to a negativeresult in spamassassin while the next cache hit 
correctly has URIBL_BLACK again


Does that cache-min-ttl also affect NXDOMAIN? Is it possible to configure 
different TTL for NXDOMAIN (relatively low) and positive results 
(relatively high)?


If not, you might want to file a bug with unbound to ask them to make that 
possible.




--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  So Microsoft's invented the ASCII equivalent to ugly ink spots that
  appear on your letter when your pen is malfunctioning.
 -- Greg Andrews, about Microsoft's way to encode apostrophes
---
 Tomorrow: Robert Heinlein's 109th birthday

Re: URIBL randomly not triggered for the same message

[Paul Stead]

On 06/07/16 16:16, John Hardin wrote:

Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
configure different TTL for NXDOMAIN (relatively low) and positive
results (relatively high)?


For this cache-max-negative-ttl exists :)

Paul
--
Paul Stead
Systems Engineer
Zen Internet

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 14:36 schrieb RW:

On Tue, 5 Jul 2016 14:01:17 +0200
Reindl Harald wrote:


since there is a local unbound-cache with

  cache-min-ttl: 300


thanks for the hint, but look at
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335#c8

reduce the value would make the problem even worser because what i 
observe is that after TTL is reached and unbound needs to query again 
the at least first question leads to a negativeresult in spamassassin 
while the next cache hit correctly has URIBL_BLACK again


so at the moment there is a tradeoff between get new domains fast enough 
and don't miss already known hits *and* that also affects SPF and so 
whitelist_auth in a bad way



You might want to review that. From http://uribl.com

  July 8, 2015: Reduction in list time latency

  The spam trend of late has been to use short lived, high-volume
  campaigns in order to capitalize on the reactive nature of blacklist
  services. In the past, it could take up to 4 minutes for us to
  identify, list, rebuild, and syncronize the update. Recent campaigns
  we have investigated have sent 80-90% of their payload within 3
  minutes.

  Because of this, we have made a handful of enhancements to improve
  our identification speed and reduce the list time latency. As a
  result, we have reduced identification times by up to 100 seconds for
  new spam campaigns, by improving the speed at which we deliver live
  query data into our system. All users should see immediate results
  from these changes.




signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[RW]

On Tue, 5 Jul 2016 14:01:17 +0200
Reindl Harald wrote:

> since there is a local unbound-cache with
> 
>   cache-min-ttl: 300

You might want to review that. From http://uribl.com

  July 8, 2015: Reduction in list time latency

  The spam trend of late has been to use short lived, high-volume
  campaigns in order to capitalize on the reactive nature of blacklist
  services. In the past, it could take up to 4 minutes for us to
  identify, list, rebuild, and syncronize the update. Recent campaigns
  we have investigated have sent 80-90% of their payload within 3
  minutes.

  Because of this, we have made a handful of enhancements to improve
  our identification speed and reduce the list time latency. As a
  result, we have reduced identification times by up to 100 seconds for
  new spam campaigns, by improving the speed at which we deliver live
  query data into our system. All users should see immediate results
  from these changes.

Re: URIBL randomly not triggered (and SPF too)

[Reindl Harald]

see also https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335

BTW: the bugtracker has also a major bug - click on "My Bugs" leads to 
the URL below listing a ton of bugreports back to the year 2011 and 
pretends they are reported by me


https://bz.apache.org/SpamAssassin/buglist.cgi?bug_status=UNCONFIRMED_status=NEW_status=ASSIGNED_status=REOPENED_to1=1=1=exact&%20%20%20%20%20%20%20%20%20email1=h.reindl%40thelounge.net=bug_status=notequals=UNCONFIRMED=reporter=equals=h.reindl%40thelounge.net

Am 05.07.2016 um 14:10 schrieb Reindl Harald:

Am 05.07.2016 um 14:01 schrieb Reindl Harald:

i have here a message with URIBL_ABUSE_SURBL Contains an URL listed in
the ABUSE SURBL blocklist

50% of all tries against spamd it does NOT hit while the scantime for
the whole message is arounnd 3 seconds - since there is a local
unbound-cache with

 cache-min-ttl: 300
 cache-max-ttl: 10800

it's impossible that there are happening dns timeouts and i can observe
the same behavior randomly with URIBL_LOCAL where the unbound dns cache
on 127.0.0.1:53 talks to rblsdnsd on 127.0.0.1:1053

that smells why ever very unrelieable and frankly i observed similar
with SPF_PASS / SHORTCIRCUIT where people within 5 seconds get the same
message and one get USER_IN_SPF_WHITELIST while the other goes through
all tests


that below too MUST NOT happen because one triggers
USER_IN_SPF_WHITELIST and the other don't have any SPF test and given
that there is a python-policyd-spf waiting 20 seconds for the response
in 'smtpd_recipient_restrictions' long before the contentfilters the
dns-results are cached

Jul  4 11:34:51 mail-gw postfix/smtpd[13648]: 3rjhgb71LVzB47:
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:34:52 mail-gw spamd[12535]: spamd: processing message
<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>
for sa-milt:189
Jul  4 11:34:56 mail-gw spamd[12535]: spamd: result: . -4 -
BAYES_00,CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,HTML_MESSAGE,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD
scantime=4.2,size=18438,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>,bayes=0.00,autolearn=disabled,shortcircuit=no


Jul  4 11:57:01 mail-gw postfix/smtpd[14837]: 3rjj993Bk8zB7P:
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: processing message
<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>
for sa-milt:189
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: result: . -100 -
CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,CUST_SHORTCIRCUIT,RCVD_IN_MSPIKE_H2,SHORTCIRCUIT,USER_IN_SPF_WHITELIST
scantime=0.1,size=15685,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>,autolearn=disabled,shortcircuit=spam




signature.asc
Description: OpenPGP digital signature