Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)

[admin]

Hello Thomas,

This might help too:
These failures are often due to SPFs that have a hard fail (meaning they end 
with ‘-all’). When I dealt with this in the past, the original sending domain 
was one where we could modify the SPF. So we had the email sender change “-all” 
to “~all” and since that makes it a soft fail, the email forwards started 
operating again. 

And it sounds like you already know this but: 

SPFs are basically TXT records attached to a domain’s DNS that specifies which 
mail server IPs have permission to send that domain’s emails. Hence the issue 
with email forwarding; Domain A sends to B which sends to C which makes C 
grumpy since B isn’t on A’s list of approved IPs. 

> On Jan 3, 2024, at 1:46 PM, Bill Cole 
>  wrote:
> 
> On 2024-01-03 at 14:17:11 UTC-0500 (Wed, 3 Jan 2024 13:17:11 -0600)
> Thomas Cameron via users 
> is rumored to have said:
> 
>> The rub is, I want all emails to presid...@example.org to be forwarded to 
>> presidents_real_addr...@gmail.com. Since the forward happens at 
>> mail.example.org, the "from" is from some other domain from example.org, so 
>> it fails all the tests.
> 
> Indeed: your solution is known as "SRS" (Sender Rewriting Scheme) and it has 
> multiple implementations. If you forward mail, you will break SPF unless you 
> fix the envelope sender so that it uses a domain  that permits the 
> example.org server to send for it.
> 
> OR, you could instead deliver to a POP mailbox locally and have users fetch 
> from there instead of simply forwarding mail to them. This also avoids a 
> completely distinct problem of places like GMail deciding that your org's 
> mail server is a spamming service because it is forwarding spam. If users POP 
> their mail instead of having it forwarded via SMTP, that does not happen.
> 
> 
> --
> Bill Cole
> b...@scconsult.com or billc...@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Not Currently Available For Hire
> 
> 

Re: problems updating when using a cron job on debian 11

[Admin]

Hello ^^)


Le 02/09/2021 à 20:49, Bill Cole a écrit :

On 2021-09-02 at 06:03:22 UTC-0400 (Thu, 2 Sep 2021 12:03:22 +0200)
Jean-François Bachelet 
is rumored to have said:


Hello folks ^^)


I've installed the latest spamassassin version on a new Debian 11 
server and configured it to work with Postfix, amavis-new, and clamav.


spamassassin got a user named 'spamd' and is run under it.


sa-update is set on a cron job to automate the update but that fails  :(

each day I get that report from cron :


/etc/cron.daily/spamassassin:
mkdir /var/lib/spamassassin/3.004006: Permission denied at 
/usr/bin/sa-update line 488.

sa-update failed for unknown reasons.


First, I've thinked that it was a permissions problem for 'spamd' 
user to access the '/var/lib/spamassassin' directory, so I've

'chown -r spamd:spamd /var/lib/spamassassin'


Hopefully that was actually "-R"


yes, it was a typing error there ;)


but even with permissions set to 775 on that directory the update 
still fail with the same message.


I just can't set permissions to 777 on that kind of directory (I'm 
not mad),


Right. Using 777 (or 666) anywhere except for directories with the 
sticky bit set is always wrong.


especially on a web server, we're not on windoze here ^^)


so what is the real problem with sa-update not working under 
spamassassin's own user when on a cron job on debian 11 ?


You need to run the sa-update cron job as the same user that INSTALLED 
SpamAssassin, the user who OWNS the local state directory (i.e. 
/var/lib/spamassassin.) You can make that happen by properly modifying 
the ownership of the directory or by running the cron job as root.


This was also my answer to the (not-a-)bug report that you opened. 
There really isn't another answer.


of course it was installed by root, btw, what is the point to have a 
user 'spamd' or debian-spamd' created if it is of no use ?


in this case the spamd user have all the needed rights to 
read-modify-execute on the /var/lib/spamassassin directory that its own, 
the job is run under that user and that don't work ???



wasn't it there for NOT escallating the privileges to 'root' for 
spamassassin updates and run ? at least it's common sense to avoid such 
dangerous 'root' use when


using scripts (as cron jobs) mainly for security.

or am I wrong ?


just an example, clamav have its own user and have no problems to auto 
update itself under it daily.



Jeff

Re: IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK re: [Bug 7826] Improve language around whitelist/blacklist and master/slave

[hospice admin]

Concentrating on the technical issues below ...

I think there's a fairly wide consensus among those posting on this thread, 
myself included,  that this does not 'make the technology better'.

That's the point I was attempting to make about Mercedes ... painting their 
cars a different colour does nothing to make it better or worse ... just 
different [in a way that has nothing to do with practical support for diversity 
of any kind].

For me, the risks of messing up Spam Assassin [or anything else] for months to 
come completely outweighs the benefits of a token 'tip of the hat' towards 
diversity.


Judy.


From: Bill Cole 
Sent: 10 July 2020 21:21
To: users@spamassassin.apache.org 
Subject: Re: IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK re: [Bug 7826] Improve 
language around whitelist/blacklist and master/slave

On 10 Jul 2020, at 5:12, hospice admin wrote:

> $0.02 from a woman of colour ...
>
> I personally find stuff like this just a little bit patronising ...
> more of a matter of kicking the real problem into the weeds than
> actually doing anything practical to 'fix' it.

Well, in the context of the Apache SpamAssassin Project, "The Real
Problem" that we have any capacity to work on is the low diversity of
our developer community. Eliminating terminology that may be off-putting
for even a minority of a minority of possible contributors is
worthwhile, particularly when the block/welcome terminology we are
replacing black/white with is explicitly descriptive rather than
metaphorical and connotative.

We have no way of knowing how many people have thought less of SA
because of terminology or whether any of those people might have
otherwise become involved enough in the project to be contributors. If
changing the terminology makes the Project look less like a bunch of
white guys trying to make rules for the world's email, that's a positive
step.

> Right up there with Mercedes decision to paint their $100 Million F1
> cars black.

This is a bit less symbolic. We're actually making the terminology
better.

> I'm sure the intent was positive though ...

The intent is to do what we can to make involvement in the SpamAssassin
community less hostile to newcomers, even if elements of hostility that
we can address are not universally recognized as such. We cannot do much
for the bigger Real Problems that intersect with ours tangentially,
because unlike Daimler-Benz, we don't have $100 Million or even $1 to
spend. None of us has the time and skills to make a focused recruiting
effort to get a more diverse set of contributors or even just more
contributors. Changing a few labels in the code is something we CAN do.

--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not For Hire (currently)

Re: IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK re: [Bug 7826] Improve language around whitelist/blacklist and master/slave

[hospice admin]

$0.02 from a woman of colour ...

I personally find stuff like this just a little bit patronising ... more of a 
matter of kicking the real problem into the weeds than actually doing anything 
practical to 'fix' it.

Right up there with Mercedes decision to paint their $100 Million F1 cars black.

I'm sure the intent was positive though ...

Judy.


From: Kevin A. McGrail 
Sent: 10 July 2020 05:00
To: SA Mailing list 
Subject: IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK re: [Bug 7826] Improve 
language around whitelist/blacklist and master/slave

IMPORTANT NOTICE

If you are running trunk, we are working on changing terms like whitelist to 
welcomelist and blacklist to blocklist.

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7826

The first test of this work is done with allowlist_to replacing whitelist_to
Committed revision 1879456.

If you are using trunk, there may be disruption since routines, plugins and 
rule changes will all interweave.

IF YOU ARE RUNNING TRUNK: I recommend you subscribe to the 
d...@spamassassin.apache.org mailing list 
to stay abreast of the changes.

Please let me know if you have any questions!

Regards,
KAM
--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: dcc-servers.net seems to have gone away

[hospice admin]

Great news.

Everything looks good at this end.

Thanks to all for your input.

Judy.


From: Kevin A. McGrail 
Sent: 23 May 2020 23:09
To: users@spamassassin.apache.org 
Subject: Re: dcc-servers.net seems to have gone away

Yes, all fixed.  There was an issue with a domain name renewal.

On 5/23/2020 2:33 PM, Matus UHLAR - fantomas wrote:
>>> On Sat, 23 May 2020 at 09:55, hospice admin 
>>> wrote:
>>> > Looks like DCC/Rhyolite has stopped working. First noticed problems
>>> > around 19:30 last night UK time.
>>> >
>>> > Problem seems to be that DNS for dcc-servers.net has gone away. Have
>>> > checked with the likes of mxtoolbox and intoDNS and they appear to
>>> agree.
>>> >
>>> > When I do a 'whois' for the domain I notice:
>>> >
>>> >Updated Date: 2020-05-23T07:40:31Z
>>> >
>>> >
>>> > Just wondered if anyone knows what's going on?
>
>> On Sat, May 23, 2020 at 5:12 AM Dominic Raferd 
>> wrote:
>>> I have no idea, but I confirm the problem.
>
> On 23.05.20 11:12, Shawn Iverson wrote:
>> Well, crud.  Anyone have a replica from before the records dropped they
>> would be willing to share out?
>
> seems fixed by now.
>
>
--
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

dcc-servers.net seems to have gone away

[hospice admin]

Hi Gang,

Looks like DCC/Rhyolite has stopped working. First noticed problems around 
19:30 last night UK time.

Problem seems to be that DNS for dcc-servers.net has gone away. Have checked 
with the likes of mxtoolbox and intoDNS and they appear to agree.

When I do a 'whois' for the domain I notice:

   Updated Date: 2020-05-23T07:40:31Z


Just wondered if anyone knows what's going on?

Thanks

Judy.

Re: Facebook notifications sent from dynamic address

[Admin]

I noticed the same thing this morning. This is new for me as of  
yesterday. They appear legit, but they get caught up in my filters for  
the dyn ip "appearance".



From: Kenneth Porter 
Sent: Saturday, October 5, 2019 10:05 AM
To: users@spamassassin.apache.org
Subject: Facebook notifications sent from dynamic address


(Nothing wrong with SA. Just an FYI about a popular service that abuses the
Internet and SA catches it.)

I noticed one of my notifications from Facebook today got tagged by SA.
Here's the two that put it over:

3.9 HELO_DYNAMIC_IPADDR2 Relay HELO'd using suspicious hostname (IP addr 2)
1.5 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
[66.220.155.138 listed in dnsbl.sorbs.net]

Here's the offending header:

Received: from 66-220-155-138.mail-mail.facebook.com
(66-220-155-138.mail-mail.facebook.com [66.220.155.138])

So who do I bitch at? I've never found any good way to complain to Facebook.

Re: Line breaks in X-Spam-Report

[Admin]

Got it. I thought it was a setup issue on my end. It didn't occur to me
that it could be a bug in hmailserver.

-
FROM: Groach 
SENT: Friday, July 27, 2018 8:03 AM
TO: users@spamassassin.apache.org
SUBJECT: Re: Line breaks in X-Spam-Report


https://github.com/hmailserver/hmailserver/issues/115

(Fyi Your question put to the hmailserver forum would have answered this
for you.)

On 27 July 2018 10:08:22 BST, Admin  wrote:

OK. That explains why I've seen it that way in some examples online.
I'm running hmailserver. Thanks.

-

From: Reindl Harald 
Sent: Friday, July 27, 2018 4:33 AM
To: users@spamassassin.apache.org; ad...@123.dynu.com
Subject: Re: Line breaks in X-Spam-Report


they are there

let me guess you use dbmail?
blame gmime at message reconstrcut time

Am 27.07.2018 um 00:44 schrieb Admin:


Hello. I was wondering if there is a setting to force line breaks in
X-Spam-Report. It’s kind of a trivial issue, but it would be so

much

easier to read. Like below as an example (that I manually altered).
Many
thanks.

Re: Line breaks in X-Spam-Report

[Admin]

OK. That explains why I've seen it that way in some examples online.  
I'm running hmailserver. Thanks.



From: Reindl Harald 
Sent: Friday, July 27, 2018 4:33 AM
To: users@spamassassin.apache.org; ad...@123.dynu.com
Subject: Re: Line breaks in X-Spam-Report


they are there

let me guess you use dbmail?
blame gmime at message reconstrcut time

Am 27.07.2018 um 00:44 schrieb Admin:

Hello. I was wondering if there is a setting to force line breaks in
X-Spam-Report. It’s kind of a trivial issue, but it would be so much
easier to read. Like below as an example (that I manually altered). Many
thanks.

Line breaks in X-Spam-Report

[Admin]

Hello. I was wondering if there is a setting to force line breaks in
X-Spam-Report. It's kind of a trivial issue, but it would be so much easier
to read. Like below as an example (that I manually altered). Many thanks.

 

X-Spam-Report: 

*  1.2 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net 

*  [Blocked - see ] 

* -0.0  T_RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain 

* -0.0 SPF_PASS SPF: sender matches SPF record 

* -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 

* -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 

*  0.3 JAM_REPEATED_VOCALS_A_BD BODY: Body of mail contains consecutive
repetition of the vocal a 

*  2.0 HS_BODY_1659 BODY: Heinlein Support Spamschutz Body-1659 

*  0.8 HTML_IMAGE_RATIO_02 BODY: HTML has a low ratio of text to image area 

*  0.0 HTML_MESSAGE BODY: HTML included in message 

*  0.5 JAM_LARGE_FONT_SIZE RAW: Body of mail contains parts with very large
font 

*  0.5 JAM_SMALL_FONT_SIZE RAW: Body of mail contains parts with very small
font 

* -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 

*  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid


* -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
domain  

*

 

-Brian

Re: I know what I want to do ... BUT ...

[hospice admin]

Nice!


I'll take a look at this.


Thanks!



From: RW 
Sent: 11 August 2017 18:29
To: users@spamassassin.apache.org
Subject: Re: I know what I want to do ... BUT ...

On Fri, 11 Aug 2017 17:25:59 +0100
RW wrote:

> On Fri, 11 Aug 2017 11:49:29 -0400
> Dianne Skoll wrote:
>
> > On Fri, 11 Aug 2017 15:27:52 +0000
> > hospice admin  wrote:
> >
> > > text = "v=spf1 exists:%{i}._spf.xyz.com ~all"
> > > What I'd like to do is turn this into an RBL check, but
> > > eval:check_rbl('Evil-ESP','_spf.xyz.com')
> >
> > I understand what you're trying to do, but have you thought about
> > the implications?  You are allowing an evil (your word)
> > organization to be an RBL for you.  They may very well put every
> > possible IPv4 address in their exists: SPF record which would not
> > bode well for your spam filtering...
> >
> > If you still want to do it, I suspect it'd have to be done outside
> > of SpamAssassin in the glue code you're using.
>
> It's not quite the questioned asked, but it is possible to run a regex
> match on the actual text record using AskDNS. See the documentation
> for Mail::SpamAssassin::Plugin::AskDNS.

or if you want to get a hit with domains that don't use that style of
SPF but use the same addresses, you can do the IP lookup using AskDNS
with the _LASTEXTERNALIP_ tag.

I know what I want to do ... BUT ...

[hospice admin]

Hi Team,


There's a particularly annoying ESP bugging us. Their clients always include a 
reference to them in their SPF records, which look something like this:


 text = "v=spf1 exists:%{i}._spf.xyz.com ~all"


So, if a message is dropped from 1.2.3.4


nslookup 1.2.3.4._spf.xyz.com


returns


Non-authoritative answer:
Name:   1.2.3.4._spf.xyz.com
Address: 1.2.3.4

if the IP is on the ESP's SPF list.

What I'd like to do is turn this into an RBL check, but

eval:check_rbl('Evil-ESP','_spf.xyz.com')

Is going to flip the IP address around.

Is there an easy way of doing this in SA, or is this a job for MimeDefang?

Thanks

Judy

Re: Mail::SpamAssassin::Plugin::EmailBL??

[hospice admin]

Thanks.



From: Kevin Golding 
Sent: 27 July 2017 14:41
To: users@spamassassin.apache.org
Subject: Re: Mail::SpamAssassin::Plugin::EmailBL??

On Thu, 27 Jul 2017 08:28:06 +0100, hospice admin 
wrote:

> the above plugin doesn't seem to be distributed with the version of
> SpamAssassin I'm running:
>
>
> spamassassin --version
> SpamAssassin version 3.4.0
>   running on Perl version 5.16.3
>
> Also, I can't find mention of a download location anywhere.
>
>
> Am I right in thinking this was an experiment that never quite made it
> out of the sand-box?

You are correct.

https://lists.gt.net/spamassassin/users/196582
Mailing List Archive: EmailBL.pm error, anyone 
knows?<https://lists.gt.net/spamassassin/users/196582>
lists.gt.net
Gossamer Mailing List Archive




I don't believe anyone has resurrected it since

Mail::SpamAssassin::Plugin::EmailBL??

[hospice admin]

Hi,


the above plugin doesn't seem to be distributed with the version of 
SpamAssassin I'm running:


spamassassin --version
SpamAssassin version 3.4.0
  running on Perl version 5.16.3

Also, I can't find mention of a download location anywhere.


Am I right in thinking this was an experiment that never quite made it out of 
the sand-box?


Thanks


Judy

Way to set user-prefs without a database?

[Dan Mahoney, System Admin]

Hey there,

We have a couple of user accounts (really, role aliases) that need a 
different required_score from our global defaults.  Since they're role 
accounts, they don't have a homedir.  We're using a milter that passes the 
whole username (including domain name) along, anyway.


Is there a dead-simple way to make this work using only the config files, 
or do I have to go to the trouble of setting up all of mysql just to make 
this happen?


Best,

-Dan Mahoney

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

DNS deferrals on *HEADER* addresses

[hospice admin]

Guys,
I've been beating my head against a problem for a couple of days now ... maybe 
someone can point me in the right direction ...?
I'm running SA 3.4.1 on Fedora 22. I think this problem started happening when 
I upgraded from  3.4.0 on Fedora 21. In both cases, SA is running from within 
MimeDefang 2.78.
Basically ... Mail is being deferred if any of the To or Cc addresses in the 
mail *header* fail DNS resolution. So, if I receive a mail:
To: me@mydomain.comCc: some...@bogusdomain.com
Mail sits in teh queue on my server until the NS for bogusdomain.com comes back 
to life. If I add an entry for the bogus domain to my DNS server, all is fine. 
I can see the query come into my nameserver logs if I do this.
My guess is that some of the processing that SA is doing generates the lookup 
on To/Cc addresses, since if I take MD out of my sendmail.mc, this doesn't 
happen.
I'm presently working my way through rulesets, turning things off. 
Does anyone have a better suggestion?
Thanks
Judy.



  

Good rules for PGP-Signed/Encrypted mail?

[Dan Mahoney, System Admin]

Hey all,

The Day Job (and some of you may know what job that is) does enough PGP 
related stuff that we've had encrypted messages get dropped on occasion, 
and we'd like to whitelist this stuff.


It looks like Mail::Spamassassin::Plugin::OpenPGP is way way old and has 
requirements that aren't exactly standard in our packaging system (BSD), 
so a rules-only approach might be nice.


Does anyone see any problems with the *SYNTAX* of the rules at?

https://svn.apache.org/repos/asf/spamassassin/trunk/rulesrc/sandbox/khopesh/20_khop_blessed.cf

That would break under a modern spamassassin?  (Yes, yes, I know we're not 
validating the messages/keys themselves, but I'd like a message to 
security-officer@ to NOT get dropped on the floor, and since this isn't a 
widespread rule, it's not likely we'll be specifically targeted knowing 
this rule is in place.)


-Dan

--

"Happy, Sad, Happy, Sad, Happy, Sad, Happy, Intruiged!  I've never been so
in touch with my emotions!"

-AndrAIa as Hexadecimal, Reboot Episode 3.2.3

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

RE: Advice re- SA 3.4.0

[hospice admin]

 

> Date: Sat, 7 Jun 2014 13:43:37 +0200
> From: axb.li...@gmail.com
> To: users@spamassassin.apache.org
> Subject: Re: Advice re- SA 3.4.0
> 
> On 06/07/2014 01:33 PM, hospice admin wrote:
> >
> >
> >
> >> Date: Sat, 7 Jun 2014 13:22:13 +0200 From: axb.li...@gmail.com To:
> >> users@spamassassin.apache.org Subject: Re: Advice re- SA 3.4.0
> >>
> >> On 06/07/2014 01:09 PM, hospice admin wrote:
> >>> I was wondering about this one and had put it to one side until I
> >>> had a chance to look at the memory implications in more detail.
> >>> We run a VM infrastructure here, so I'll load it up on one server
> >>> and keep throwing resources at it until I have something
> >>> approaching stability.
> >>
> >> my Redis DB's memory usage # Memory used_memory:3919713920
> >> used_memory_human:3.65G used_memory_rss:6440914944
> >> used_memory_peak:6307356768 used_memory_peak_human:5.87G
> >> used_memory_lua:99328
> >>
> >> the peak is used when Reids dumps the DB to file, every N minutes.
> >> To do this, a second Redis server instance is started and does the
> >> dump so if you think your Bayes DB will be 4GB... double that (at
> >> least) for safety. If the box starts swapping hard it will all
> >> become incredibly slow or even crash/feed.
> >>
> >> free total used free shared buffers cached Mem: 14262652 8116144
> >> 6146508 0 151260 1349872 -/+ buffers/cache: 6615012 7647640 Swap:
> >> 2046968 10396 2036572
> >>
> >>
> >>> An almost related item ... have you found the 'RelayCountry'
> >>> plugin to be worth the effort?
> >> Due to the nature of my traffic I see little use for it.
> >>
> >> Ff you mainly deal with regional traffic it's probably worth
> >> trying.
> >>
> >>
> >
> > WOW! My DB isn't anything like 4GB, but our whole setup presently
> > runs in 6GB, to the increase is likely to be scary big. I guess you
> > get what you pay for though!
> 
> Don't let my Redis DB size scare you unless you're handling mail for a 
> for tens of thousands of users.
> 
> I'd suggest your start off with a dedicated VM for the Redis server, 
> assign it 8 GB (to be on the safe side) set our tokens to expire in 2 
> weeks and watch it closely. This way if you have a Redis issue, it won't 
> affect mail processing
> 
> # FOR REDIS ONLY
> bayes_token_ttl 14d
> bayes_seen_ttl 7d
> 
> should get you started...
> You can also limit Redis' memory usage (if you want)
> 
> 
 
Thanks for that!
 
Judy.
  

RE: Advice re- SA 3.4.0

[hospice admin]

 

> Date: Sat, 7 Jun 2014 13:22:13 +0200
> From: axb.li...@gmail.com
> To: users@spamassassin.apache.org
> Subject: Re: Advice re- SA 3.4.0
> 
> On 06/07/2014 01:09 PM, hospice admin wrote:
> > I was wondering about this one and had put it to one side until I had
> > a chance to look at the memory implications in more detail. We run a
> > VM infrastructure here, so I'll load it up on one server and keep
> > throwing resources at it until I have something approaching
> > stability.
> 
> my Redis DB's memory usage
> # Memory
> used_memory:3919713920
> used_memory_human:3.65G
> used_memory_rss:6440914944
> used_memory_peak:6307356768
> used_memory_peak_human:5.87G
> used_memory_lua:99328
> 
> the peak is used when Reids dumps the DB to file, every N minutes.
> To do this, a second Redis server instance is started and does the dump 
> so if you think your Bayes DB will be 4GB... double that (at least) for 
> safety. If the box starts swapping hard it will all become incredibly 
> slow or even crash/feed.
> 
> free
> total used free shared buffers cached
> Mem: 14262652 8116144 6146508 0 151260 1349872
> -/+ buffers/cache: 6615012 7647640
> Swap: 2046968 10396 2036572
> 
> 
> > An almost related item ... have you found the 'RelayCountry' plugin
> > to be worth the effort?
> Due to the nature of my traffic I see little use for it.
> 
> Ff you mainly deal with regional traffic it's probably worth trying.
> 
> 
 
WOW! My DB isn't anything like 4GB, but our whole setup presently runs in 6GB, 
to the increase is likely to be scary big. I guess you get what you pay for 
though!
 
Thanks
 
Judy
  

RE: Advice re- SA 3.4.0

[hospice admin]

 

> Date: Sat, 7 Jun 2014 12:49:32 +0200
> From: axb.li...@gmail.com
> To: users@spamassassin.apache.org
> Subject: Re: Advice re- SA 3.4.0
> 
> On 06/07/2014 12:19 PM, hospice admin wrote:
> > Just wondering if anyone had any advice along the lines "you really
> > must do this", or "you'd be crazy to do that" re- all the new stuff,
> > etc?
> >
> >
> >
> > I'm particularly 'interested' in things relating to Bayes, which has
> > bitten me in the rear so many times, but seems to have migrated over
> > faultlessly.
> 
> 
> For my setup one of the best new features has been the Redis backend for 
> Bayes.
> 
> Although it requires a ton of memory (which is cheap) it allows keeping 
> a *huge* amount of tokens for a long time without a decrease in 
> performance.
> 
> Feeding it trap data as spam with different expiration time than ham via 
> autolearn has made Bayes way more useful.
> 
> 0.000 0 22712317 0 non-token data: nspam
> 0.000 0 10031781 0 non-token data: nham
> 
> 
> try that with sql or file based Bayes - it wouldn't scale and it could 
> probably cause scan times in many, many seconds/msg range
> 
> Bayes/Redis is so fast I don't notice a performance difference if I 
> enable it or not.
> So depending on your traffic load, imo, it's become a must have.
> 
> also forced autolearn has helped a lot with failsafe metas/rules
> 
> tflags RULE_NAME autolearn_force
> 
> h2h
> 
> Axb
 


Thanks. That sounds like great advice.
 
I was wondering about this one and had put it to one side until I had a chance 
to look at the memory implications in more detail. We run a VM infrastructure 
here, so I'll load it up on one server and keep throwing resources at it until 
I have something approaching stability.
 
An almost related item ... have you found the 'RelayCountry' plugin to be worth 
the effort?
 
Thanks again
 
Judy.
  

Advice re- SA 3.4.0

[hospice admin]

Hi Team,

 

Ive finally completed the upgrade of all my mail servers from FC18 + SA 3.3.2 + 
Perl 5.15.3 to FC20 + SA 3.4.0 + Perl 5.18.2. I run SA from within MineDefang 
2.74 in both cases.

 

I've simply moved across all the rules and plug-ins I used 3.3.2 to 3.4.0, and 
during our beta testing over the last month or so, everything has worked like a 
dream. Thanks to all involved in producing such a great piece of software. I'd 
have made the change months ago if I'd realised it was going to be this easy :)

 

I've been reading through all the new features in 3.4.0, looking at new 
plug-ins, etc. There's a lot of really interesting looking stuff ...

 

Just wondering if anyone had any advice along the lines "you really must do 
this", or "you'd be crazy to do that" re- all the new stuff, etc?

 

I'm particularly 'interested' in things relating to Bayes, which has bitten me 
in the rear so many times, but seems to have migrated over faultlessly.

 

Thanks

 

Judy.

 

 
  

RE: Mystery SpamWare

[hospice admin]

> Date: Thu, 22 May 2014 17:13:24 -0700
> From: jdeb...@garlic.com
> To: users@spamassassin.apache.org
> Subject: Re: Mystery SpamWare
>
> On Thu, 22 May 2014 18:23:48 +0100
> hospice admin  wrote:
>
>> Hi Team,
>>
>> All of a sudden I've started noticing a lot of spam coming in with
>> some fairly unique headers like this:
>>
>> x-track-version: 4
>> x-track-source: notifire_XXX
>> x-track-spooler-id: 
>> x-track-spooler-split-id: 
>> x-track-spooler-segment-id: 
>> x-render: render-
>> Precedence: bulk
>> x-track-contact-id: 
>>
>>  is some number which varies with user to some degree, XXX varies
>> by spammer.
>>
>> Does anyone recognise where these headers come from?
>>
>
> Those headers seem to be tracking headers for commercial email
> marketing campaigns. Possibly from Notifire.co.uk, an email
> massmarketing firm, calling itself a "white label". Quite uncertain w/o
> more data. But those headers are enough to make a filter from or to use
> in header checks to reject such trash.
>
> jd
>
>

Ah ... thank you so much ... our old 'friends' at Neteffekt.

Very Helpful.

Thanks again

Judy. 

Mystery SpamWare

[hospice admin]

Hi Team,

All of a sudden I've started noticing a lot of spam coming in with some fairly 
unique headers like this:

x-track-version: 4
x-track-source: notifire_XXX
x-track-spooler-id: 
x-track-spooler-split-id: 
x-track-spooler-segment-id: 
x-render: render-
Precedence: bulk
x-track-contact-id: 

 is some number which varies with user to some degree, XXX varies by 
spammer.

Does anyone recognise where these headers come from?

Thanks

Jude.

  

sa-learn from a cronjob?

[Dan Mahoney, System Admin]

All,

Most of my users aren't command-line friendly.  I'd like to basically have 
my IMAP server default to handing out two imap mailboxes that get 
auto-crontabbed to training bayes.


Ideally, I'd also like to make it so that things dropped in the learn_spam 
folder are deleted, and stuff in the learn_ham folder (mistake-based 
training) are de-tagged and moved back to the inbox.  Alternatively, a 
single "learned" folder would do.


Perl's Mail::Box seems like a heavy tool for this simple task.  Does 
anyone else have any recommendations?


-Dan

--


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

What is the view re- SPF_FAIL these days?

[hospice admin]

Hi Team,

I was wondering what folks were doing with SPF_FAIL ,   TO_EQ_FM_SPF_FAIL and   
TO_EQ_FM_DOM_SPF_FAIL   these days?

I personally have never seen an FP with any, but understand from the reading 
I've done that some people do.

My approach has always been to combine with DCC/Pyzor/Razor hits in a Meta 
rule, but we've recently started seeing   mail just squeak under the fence 
using this approach ... particularly some of the 'nicer' Bank Spam. The 
temptation is to add Bayes to the Meta. Is this a bad idea, or does anyone have 
any better suggestions?

We're running SA version 3.3.2. Sadly, upgrading to 3.4 isn't an option at this 
stage.

Thanks for your time & wisdom

Judy. 

RE: Detecting very recently registered domain names

[hospice admin]

> From: hospice...@outlook.com
> To: users@spamassassin.apache.org
> Subject: RE: Detecting very recently registered domain names
> Date: Mon, 6 Jan 2014 13:45:07 +
>
> 
>> Date: Mon, 6 Jan 2014 12:26:08 +
>> From: andrew.he...@aaisp.net.uk
>> To: users@spamassassin.apache.org
>> Subject: Re: Detecting very recently registered domain names
>>
>> On Thu, 19 Dec 2013 10:02:39 -0500
>> Joe Quinn  wrote:
>>
>>> We are noticing a lot of spam coming from domains that are less than
>>> two months old. Is there a good way to detect this automatically?
>>>
>>> We've thought about whois, but do not want to get blocked for looking
>>> like we are harvesting information.
>>
>>
>> May be off topic, but is this related to Communicado Ltd, who register
>> domains daily in order to send spam, more info and a maintained list(at
>> least at the moment) on:
>> http://blog.hinterlands.org/2013/10/unwanted-email-from-communicado-ltd/
>>
>>
>> --
>> Andrew
>>
>
>
>
> Communicado are probably a bit smarter than may people give them credit for. 
> They change tactics frequently, and are pretty good at what they do. They 
> have been around a long time and that probably says a lot, given the 
> characteristics of the industry they are in.
>
> In recent weeks they have moved from the 'day old' .co.uk domains to 
> relatively old com/org/net domains (Aug last year). I'm sure they will change 
> back again at some point though ... its not like NOMINET give a darn about 
> spam, is it??
>
> Playing 'whack-a-mole' with them isn't half as effective is focusing in on 
> the common traits of the mail they sent out, like phone numbers (01799 
> 252xxx), common phrases, the names of the instructors they use, the structure 
> of the HTML in the mails themselves, and so on. Even the price they charge 
> for their courses helps (everything seems to be £149.00 + VAT :)
>
> Registering a new domain costs peanuts, compared to re-working this kind of 
> stuff on a regular basis, and as we all know, at the end of the day, its all 
> about money.





A couple of folks ashed me for specific details regarding rules, so thought it 
may be helpful to post here.

There's nothing fancy, and probably prone to FP out side of our particular 
context, but here you go:

rawbody         CDO_Phone0   /01227[\s\-_]+252\s*\d\d\d/
aescribe        CDO_Phone0   CDO Phone Number


rawbody         CDO_Phone1   /01799[\s\-_]+252\s*\d\d\d/
aescribe        CDO_Phone1   CDO Phone Number


rawbody         CDO_Phone2   /0800[\s\-_]+084\s*5076/
aescribe        CDO_Phone2   CDO Phone Number


body            CDO_DT       /Distinguished Traveller/i
describe        CDO_DT       Mentions Distinguished Traveller

body            CDO_BS       /Bitesize/i
describe        CDO_BS       Mentions Bitesize

uri             CDO_US       /\/sub\.php\?clt=.+\&email=/i
describe        CDO_US       Contains common Comunicado Unsubscribe Code

Sure, some of this is easy for the spammers to change, and your mileage may 
vary (which is why I've left you to fill in the scores that seem right for you).

What do seem to be a fairly solid indicator are the phone numbers I mentioned. 
BT love to tie you into long term contracts, so changing these takes some 
forethought and planning ... unlike reacting to any possible change the 
spammers make.

To me, targeting this stuff for all spammers (not just the guy at the centre of 
this particular witch hunt) makes more sense than domain or IP whack-a-mole ... 
[which isn't the same as to say the results aren't great. Just that the time to 
benefit doesn't work out unless all you do for a living is e-mail].

So far, the approach has worked great for me - I presently get no Communicado 
SPAM, and I have the time to run a Hospice IT department in the mean time ... 
bet that changes as soon as CDO guys finish reading this post though ...

ANyway - Hope this helps somewhat, and despite the potentially negative sway of 
my comments - great work on the EXCOMMUNICADO stuff. 

Here's hoping someone does something similar re- Merrehill Limited, 
EMediaSolutions :)

Judy  

RE: Detecting very recently registered domain names

[hospice admin]

> Date: Mon, 6 Jan 2014 12:26:08 +
> From: andrew.he...@aaisp.net.uk
> To: users@spamassassin.apache.org
> Subject: Re: Detecting very recently registered domain names
>
> On Thu, 19 Dec 2013 10:02:39 -0500
> Joe Quinn  wrote:
>
>> We are noticing a lot of spam coming from domains that are less than
>> two months old. Is there a good way to detect this automatically?
>>
>> We've thought about whois, but do not want to get blocked for looking
>> like we are harvesting information.
>
>
> May be off topic, but is this related to Communicado Ltd, who register
> domains daily in order to send spam, more info and a maintained list(at
> least at the moment) on:
> http://blog.hinterlands.org/2013/10/unwanted-email-from-communicado-ltd/
>
>
> --
> Andrew
>



Communicado are probably a bit smarter than may people give them credit for. 
They change tactics frequently, and are pretty good at what they do. They have 
been around a long time and that probably says a lot, given the characteristics 
of the industry they are in.

In recent weeks they have moved from the 'day old' .co.uk domains to relatively 
old com/org/net domains (Aug last year). I'm sure they will change back again 
at some point though ... its not like NOMINET give a darn about spam, is it??

Playing 'whack-a-mole' with them isn't half as effective is focusing in on the 
common traits of the mail they sent out, like phone numbers (01799 252xxx), 
common phrases, the names of the instructors they use, the structure of the 
HTML in the mails themselves, and so on. Even the price they charge for their 
courses helps (everything seems to be £149.00 + VAT :)

Registering a new domain costs peanuts, compared to re-working this kind of 
stuff on a regular basis, and as we all know, at the end of the day, its all 
about money. 

RE: dns*.registrar-servers.com as a rogue registrar?

[hospice admin]

> From: lcon...@go2france.com
> To: users@spamassassin.apache.org
> Subject: dns*.registrar-servers.com as a rogue registrar?
> Date: Tue, 7 May 2013 13:15:24 -0500
> 
> 
> Nearly all of the .pw domains have their authoritative NS at 
> dns*.registrar-servers.com.
> 
> that registrar and few others are always at the top of my reports for 
> NSs of sender domains of spam we reject.
> 
> Does anybody score a msg if its sender domain is DNS hosted by 
> registrar-servers.com or other?
> 
> what would that rule look like?
> 
> Len
> 
> 

I've found this to be a pretty helpful approach.
I couldn't find a plugin to do this, and I didn't have time to figure out how 
to write one, so added the functionality via MimeDefang.
I have a list of 'evil' domains and IPs in two RBLs which I maintain myself. I 
check NS records against these, and if I get a match, I bump up the score by 
returned by SA by the value of the last byte. 
I'm not sure this approach would scale to a gazillion mails a day, but works 
fine for the levels we have to deal with (a couple of 100K tops).
Judy. 

OT: Hopefully of interest to someone out there

[hospice admin]

http://www.nominet.org.uk/whoweare/structure/agm/board-election
It would be great if someone from our community (ideally wiser them me), could 
get elected.
Judy. 

RE: .pw / Palau URL domains in spam

[hospice admin]

> Date: Wed, 1 May 2013 16:34:48 +0200
> From: axb.li...@gmail.com
> To: users@spamassassin.apache.org
> Subject: Re: .pw / Palau URL domains in spam
> 
> On 05/01/2013 04:28 PM, hospice admin wrote:
> > I don't care what some folks are saying about .pw, compared to Nominet they 
> > totally rock.
> > When was the last time anyone saw Nominet suspend a .UK spammer?
> > Judy
> 
> You miss the point.
> 
> Nominet is a registrar
> Directi is acting as THE .pw registry
> Registrars selling .pw domains and we all know where the crud goes to 
> get theirs.
> They are the top entity - and lowering all barriers they've opend a can 
> of worms.
> But this is become off topic
> 
> my EOT
> 
> 
> 
> 

Have I missed the point? I don't think so, and I appreciate this is off 
topic(ish), but ...
The point is, a great deal of the stuff we have to deal with comes from 
commercial organisations that spam for a living. These guys can't operate 
without access to a ready source of domains.
If Nominet wanted to stop a huge lump of spam originating from .UK , they 
could. All they need to do is link every commercial registration to a UK 
company number or similar for of ID, then react when someone complains. After 
that, three strikes (pick a number) and you're out of business.
It won't stop it all, sure, but it will stop some, and not just 'after the 
fact', ...
Right ... off to take a chill pill :)
J.
  

RE: .pw / Palau URL domains in spam

[hospice admin]

I don't care what some folks are saying about .pw, compared to Nominet they 
totally rock.
When was the last time anyone saw Nominet suspend a .UK spammer?
Judy

> Date: Wed, 1 May 2013 06:58:41 -0700
> From: dones...@directi.com
> To: users@spamassassin.apache.org
> Subject: Re: .pw / Palau URL domains in spam
> 
> Dear Kevin A. McGrail,
> 
> Thank you very much for reporting the domain names. We have suspended all
> the reported 13 domain names.
> 
> Regards
> Donesh Laher
> Cyber Security Analyst
> .PW Registry
> 
> 
> 
> 
> --
> View this message in context: 
> http://spamassassin.1065346.n5.nabble.com/pw-Palau-URL-domains-in-spam-tp104383p104495.html
> Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
  

RE: Seminar Spam

[hospice admin]

 

> Date: Wed, 24 Apr 2013 13:13:30 -0400
> From: b...@indietorrent.org
> To: users@spamassassin.apache.org
> Subject: Re: Seminar Spam
> 
> 
> 
> On 4/24/2013 12:12 PM, hospice admin wrote:
> > Hi,
> > 
> > we're having problems with an outfit called 'Bite Sized Seminars' in the
> > UK, who seem to be sending mail out through another company called
> > 'Communicado'. A quick google suggests we aren't the only ones.
> > 
> > We have developed a number of rules that identify their mail by looking
> > for their phone numbers, common phrases, etc in their mail shots with
> > varying success (I'm happy to share these with anyone who may find them
> > helpful).
> > 
> > The problem I'm trying to solve is that they seem to register hundreds
> > of .co.uk domains, and have access to loads of sending IPs, so I can't
> > just write a rule to do the obvious. I've complained about them to
> > Nominet, and they aren't interested ... according to them, they are
> > doing nothing wrong. I've also complained to various IP providers, some
> > of which say they will do something, but rarely do. I've even rung them
> > ... again ... no joy.
> > 
> > Here's my question - am I missing a trick here, particularly regarding
> > the hundreds of domain names? For example, is it possible to do a
> > 'whois' and process the output in some way?
> > 
> > Thanks
> > 
> > Judy.
> > 

Thanks to everyone who made suggestions and asked questions. Sorry about slow 
response from me, but drove my bike into a wall shortly after sending the above 
(all fine ... except for the bike :)
 
Re- Bayes ... yes, I've trained Bayes as best as I can, and I'm getting 
acceptable results.
 
Re- Common Header patterns ... again, yes, There are quite a few and I've 
written rules that spot many. I've also picked up on common patterns in the 
mail bodies themselves ... stuff like telephone numbers, common phrases and 
that kind of thing. I've glued these altogether in a Meta rule and I must say 
the accuracy is pretty good.
 
Re- example domains ... I've collected loads, along with associated IPs. I have 
these in an RBL and I update from logs each day. Basically, they can get me 
once, but only once :)
 
I think I've done all teh obvious things and I'm pretty happy with the results, 
but will post some examples in pastebin as requested, JiC anyone is interested 
(you probably already have some in you junk pile if you look).
 
I was really just interested in seeing if anyone was handling this kind of 
thing differently to me, etc. Whois seems like a gold mine, but as someone 
said, the nominet guys do their best to make it unusable. Even so, I'm looking 
at options involving mimedefang :) 
 
These guys are basically cr@p, but persistent ... I guess that's all you need 
to be to make money out of spam.
 
Thanks again peeplz, and special thanks to Nominet for helping make things as 
bad as they are.
 
Judy  

Seminar Spam

[hospice admin]

Hi,
we're having problems with an outfit called 'Bite Sized Seminars' in the UK, 
who seem to be sending mail out through another company called 'Communicado'. A 
quick google suggests we aren't the only ones.
We have developed a number of rules that identify their mail by looking for 
their phone numbers, common phrases, etc in their mail shots with varying 
success (I'm happy to share these with anyone who may find them helpful).
The problem I'm trying to solve is that they seem to register hundreds of 
.co.uk domains, and have access to loads of sending IPs, so I can't just write 
a rule to do the obvious. I've complained about them to Nominet, and they 
aren't interested ... according to them, they are doing nothing wrong. I've 
also complained to various IP providers, some of which say they will do 
something, but rarely do. I've even rung them ... again ... no joy.
Here's my question - am I missing a trick here, particularly regarding the 
hundreds of domain names? For example, is it possible to do a 'whois' and 
process the output in some way?
Thanks
Judy.
  

Re: [sa-list] Re: Weighted MIRRORED.BY files?

[Dan Mahoney, System Admin]

On Sun, 24 Mar 2013, Mark Martinec wrote:


On Sunday March 24 2013 05:57:49 Dan Mahoney, System Admin wrote:

sa-update also uses a mirror file which lists all of the URLs where the
update can be downloaded from, optionally including weights for different
mirrors.
But there's no documentation or examples given for weighting.  Anyone
closer to the code know what this would look like?


$ curl http://spamassassin.apache.org/updates/MIRRORED.BY
http://sa-update.dnswl.org/ weight=1
http://www.sa-update.pccc.com/ weight=5
http://sa-update.secnap.net/ weight=5


sa-update has the following in its comments:

# choose a random integer between 0 and the total weight of all mirrors
# loop through the mirrors from largest to smallest weight
# if random number is < largest weight, use it
# otherwise, random number -= largest, remove mirror from list, try again
# eventually, there'll just be 1 mirror left in $mirrors[0] and it'll be used
#
sub choose_mirror {
 my($mirror_list) = @_;
[...]


I'll add this on to the wiki.

-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Weighted MIRRORED.BY files?

[Dan Mahoney, System Admin]

Hey there.

The SA wiki says:

sa-update also uses a mirror file which lists all of the URLs where the 
update can be downloaded from, optionally including weights for different 
mirrors.


But there's no documentation or examples given for weighting.  Anyone 
closer to the code know what this would look like?


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: How to log detected locale/language?

[Dan Mahoney, System Admin]

On Fri, 8 Mar 2013, Axb wrote:


On 03/08/2013 04:46 PM, Dan Mahoney, System Admin wrote:

Hey there all,

It seems a pretty core function in SA is the ok_languages and ok_locales
function.

I'd like to be able to turn on LOGGING of detected locales before I set
which are "ok" (or specifically, which are "less ok")

I'm sure there's a knob for this somewhere, can anyone tell me where?



Nice someone documented this:

http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Conf.txt

_LANGUAGES_

so now what?

a few lines later it tells us what to do

add_header all X-BLAHTYPE  _LANGUAGES_

add that to your local.cf and reload SA, glue, coffee machine.

does this do what you want?


Mostly, but I can't figure out how to get the LOCALE (which is purely 
characterset based) to work.  What's the logging macro for that one?


-Dan

--

"I wish the Real World would just stop hassling me!"

-Matchbox 20, Real World, off the album "Yourself or Someone Like You"


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Yahoo single link spam

[Dan Mahoney, System Admin]

On Fri, 22 Feb 2013, Kevin A. McGrail wrote:


On 2/22/2013 3:27 PM, David F. Skoll wrote:

On Fri, 22 Feb 2013 12:20:22 -0800
Marc Perkel  wrote:


We need a rule to catch this. It looks like more data than it is but
it's really little more than a single link. Like to see a rule that
identifies it.

Our product lets you make compound rules.  It should not be very hard
to translate this to SpamAssassin:

HeaderMatches RegExp   ^To:(.*?@.*?){5}   AND
Envelope Sender   Ends with@yahoo.com AND
MessageSize   <6000

Well, ok... the MessageSize condition is tricky.  And this rule does
kick up some false-positives, but overall it works pretty well for us.


Here's the current version I'm using based on 3.4.0 trunk:

#YAHOO COMPROMISED ACCOUNT SPAMS - SCORED HIGH BECAUSE THESE ARE COMPROMISED 
ACCOUNTS WHICH MAKES ALL OF YAHOO!'s PROCEDURES QUESTIONABLE
header  __KAM_YAHOO1From =~ 
/\@(yahoo.com|yahoo.com.id|rocketmail.com)/i
header  __KAM_YAHOO2Subject =~ /^(FOR |Hey$|hi$|look at 
this$|great!?$|amazing!?|the best!?$|excellent!?$|very good!?$|great!?$)/
body__KAM_YAHOO3/\d{1,2}[\\\/]\d{1,2}[\\\/]\d{2,4} 
\d{1,2}\:\d{1,2}\:\d{1,2} (AM|PM)/

header  __KAM_YAHOO4From:name =~ /Connor Hopkins/i

metaKAM_YAHOO   (__KAM_YAHOO1 + __KAM_YAHOO2 + __KAM_YAHOO3 + 
__KAM_YAHOO4 + __KAM_BODY_LENGTH_LT_128 + MISSING_SUBJECT >= 3)

describeKAM_YAHOO   Compromised Yahoo! Accounts Sending Spam
score   KAM_YAHOO   9.0


Just to add a late reply to the game, I'm still getting these.  Kevin, it 
looks like your rules YAHOO1 and YAHOO3 are still appropriate, but neither 
of the others.  I think there's a few other things I've noticed that I 
don't know how to match:


the body doesn't "contain" the link, it pretty much "IS" the link. 
However, I don't know how to write a rule that says "contains a link and 
NOTHING ELSE".  I also don't know how to write rules that say "the 
text/plain portion contains a link, and the text/html portion contains 
more".  I'm not aware of how "body" gets interpreted in 
multipart/alternative messages.  Kevin, if you're able to tell me more 
about this, I'm happy to learn.


Writing rules is easy for some, but I'm more about solving the problem. 
The answer isn't "many people write many custom rulesets", it's "surbl 
catches up faster" or "yahoo acknowledges the problem."


While yahoo's abuse reporting procedures leave much to be desired, this is 
actually one of the reasons I was asking about a channel to autoreport 
mail to spamcop (and yahoo, if they were willing to take it, but they 
don't seem to be -- blog post coming on that, soon).


-Dan

--

"One...plus two...plus one...plus one."

-Tim Curry, Clue

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

How to log detected locale/language?

[Dan Mahoney, System Admin]

Hey there all,

It seems a pretty core function in SA is the ok_languages and ok_locales 
function.


I'd like to be able to turn on LOGGING of detected locales 
before I set which are "ok" (or specifically, which are "less ok")


I'm sure there's a knob for this somewhere, can anyone tell me where?

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

pyzor 401/unauthorized?

[Dan Mahoney, System Admin]

I was in the process of "linting" my SA config when I discovered that the 
pyzor servers are handing back this response to all commands:


/usr/local/bin/pyzor --homedir /usr/local/etc/mail/spamassassin/.pyzor 
ping
public.pyzor.org:24441  (401, 'Unauthorized: User is not authorized to 
request the operation.')


As opposed to the myriad of other issues I've seen on this list where the 
user can't set pyzor_home correctly or firewall issues, I'm pretty sure 
I'm doing things right (I don't get a backtrace or anything) and this 
appears to be server-side.


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Supporting spamcop "quick" reporting

[Dan Mahoney, System Admin]

On Tue, 19 Feb 2013, Andrzej A. Filip wrote:


On 02/19/2013 08:53 PM, Dan Mahoney, System Admin wrote:

On Tue, 19 Feb 2013, Andrzej A. Filip wrote:


On 02/19/2013 03:47 AM, Dan Mahoney, System Admin wrote:

Spamcop has an undocumented feature that they allow you (if they trust
you) to "quick report" spam, where you send to a different mail address,
and it's reported instantly, without having to hit the web interface.
When you do this, you are still free to report spam in the usual way
(with the confirm screen) by using your usual reporting-address.
[...]


AFAIK/AFAIR:
Spamcop.net "quick reporting" automatically sens reports/LARTs about
spam reported  via SMTP _based on mail routing only_ (no reports/LARTs
about spamvertized web sites). It has been intended for spamtraps' catch.

Reporting again "the usual way" could to easily create duplicate reports.


By this I meant (and apologies if anyone else misconstrued it), that
when spamcop enables this feature, you may use either to process a given
message, but not both.

However, as I now read here:

http://forum.spamcop.net/scwik/QuickReporting

You're quite right, it doesn't parse the body, which is a shame.  See my
reply to kevin for more information.


One option would be co create "via HTTP" reporter automatically sending
reports/LARTs about spam without "possibly spamvertised URLs". It should
be quite simple to implement if you are ready to wait extra 5-7s per
every spam reported.


an interesting thought, but I'm not sure what you mean there.  Do you mean 
as a means of sending the initial report instead of email, or do you mean 
as a means of both reporting the spam, AND ack'ing it?  (as if I had 
pasted it in).


-Dan

--

[23:49:00] LarpGM: Did my little TP comment scare you off?
[23:49:22] ilzarion: no, the shrieking retarded child eating people did

-Feb 06, 2001, times apparent.


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Supporting spamcop "quick" reporting

[Dan Mahoney, System Admin]

On Tue, 19 Feb 2013, Kevin A. McGrail wrote:


On 2/18/2013 9:47 PM, Dan Mahoney, System Admin wrote:

Hey there,

Spamcop has an undocumented feature that they allow you (if they trust you) 
to "quick report" spam, where you send to a different mail address, and 
it's reported instantly, without having to hit the web interface. When you 
do this, you are still free to report spam in the usual way (with the 
confirm screen) by using your usual reporting-address.


How hard would it be to extend spamassassin's "report" syntax to allow 
this?


Unfortunately, I'm not seeing a good way to pass config-options to spamd, 
so that's out.  (I suppose this email could be interpreted as a case of "is 
this useful?").


Running the "report" against spamassassin locally would lose me the other 
learning (bayes, etc).


Creating an alternate user with the quick-reporting mail address sent is 
similarly problematic (althouth I *might* be able to do this by playing 
with the userpref sql query).


I'm open to any other ideas people have come up with.

Hi Dan,



Looking a this in a high level, I think you are referring to spamc's 
reporting feature.


I am.  I receive email for my entire domain, and I have several mailboxes 
which meet spamcop's definition of traps -- they have NEVER been used to 
receive legit mail, and were basically made up by list-sellers to pad 
lists, and are not even close (typographically) to any other email 
addresses I've got. 
They have "real names" and other such demographic information, and 
are doctors, apparently, based on the crap they get.


For a while, I tried reaching out to the people mailing me (who looked 
legit) and tried to tell them "okay, this is the first time I'm seeing 
mail to this address, you got scammed by whomever sold you this list"). 
But bulk-mailers (legit or not) deal in volume, and can-spam basically 
says they don't have to care.


Faced with this, I had three options:

1) Unsubscribe, basically self-listwashing.

2) Route the mail to /dev/null.

3) Allow these email addresses to act like a poisoned fruit, and serve as 
a marker of the spam and irresponsible list-buyers, and act as a sigil 
with razor/pyzor/spamcop.


With #3, the annoyance is that I now send to "spamc -C report", but get a 
steady stream of emails that say "spamcop has accepted one email for 
processing".  And of course, because spamcop wants their mail to be 
"fresh" it means I'm dealing with a constant stream of having to log in 
and click through.


Aside:

What's more braindead, on Spamcop's end, is that while they won't accept 
mail over two days old, if you don't go in and click report/cancel, it 
will wait for you in the queue, for weeks.  (And from what they tell me, 
they don't parse the mail until you hit "report now", so they cite CPU 
overhead on doing advanced expiry).  They seem to have missed the bit that 
they have the date-of-submission without having to parse the body.


/Aside.

However, that's likely not the best avenue unless you are just trying to send 
spamcop examples of algorithmically determined spam. I wonder if it is time 
for a separate reporting binary and perhaps build on the existing 
"collaboration reporting" in spamc/d and add RPS::Mail::EventReporter for 
reputation collaboration.


I would be in favor of this.  It would also seem that DCC's reputation 
code/reporting should have support in the latest version of SA.  As I now 
read that spamcop's "quick" reporting isn't as thorough as their manual 
report, I'm somewhat less interested, but better support in a tool could 
change that.


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Supporting spamcop "quick" reporting

[Dan Mahoney, System Admin]

On Tue, 19 Feb 2013, Andrzej A. Filip wrote:


On 02/19/2013 03:47 AM, Dan Mahoney, System Admin wrote:

Spamcop has an undocumented feature that they allow you (if they trust
you) to "quick report" spam, where you send to a different mail address,
and it's reported instantly, without having to hit the web interface.
When you do this, you are still free to report spam in the usual way
(with the confirm screen) by using your usual reporting-address.
[...]


AFAIK/AFAIR:
Spamcop.net "quick reporting" automatically sens reports/LARTs about
spam reported  via SMTP _based on mail routing only_ (no reports/LARTs
about spamvertized web sites). It has been intended for spamtraps' catch.

Reporting again "the usual way" could to easily create duplicate reports.


By this I meant (and apologies if anyone else misconstrued it), that when 
spamcop enables this feature, you may use either to process a given 
message, but not both.


However, as I now read here:

http://forum.spamcop.net/scwik/QuickReporting

You're quite right, it doesn't parse the body, which is a shame.  See my 
reply to kevin for more information.


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Supporting spamcop "quick" reporting

[Dan Mahoney, System Admin]

Hey there,

Spamcop has an undocumented feature that they allow you (if they trust 
you) to "quick report" spam, where you send to a different mail address, 
and it's reported instantly, without having to hit the web interface. 
When you do this, you are still free to report spam in the usual way (with 
the confirm screen) by using your usual reporting-address.


How hard would it be to extend spamassassin's "report" syntax to allow 
this?


Unfortunately, I'm not seeing a good way to pass config-options to spamd, 
so that's out.  (I suppose this email could be interpreted as a case of 
"is this useful?").


Running the "report" against spamassassin locally would lose me the other 
learning (bayes, etc).


Creating an alternate user with the quick-reporting mail address sent is 
similarly problematic (althouth I *might* be able to do this by playing 
with the userpref sql query).


I'm open to any other ideas people have come up with.

-Dan

--

"this is too stupid even for irc"

-mtreal, EFnet #macintosh, 09/15/2K, 12:33 AM

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: Still no apparent fix on ipv6 spamd?

[Dan Mahoney, System Admin]

On Tue, 25 Sep 2012, Kevin A. McGrail wrote:



On 9/25/2012 5:02 PM, Dan Mahoney, System Admin wrote:

I mentioned this on the mailing lists a few years ago.

I notice that there still doesn't seem to be a clean way to just make spamd 
listen on all (v4 and v6) addresses by default, nor is there a way to 
listen on multiple addresses with multiple -A options.


This means that if you want to listen on v6, none of your v4 clients can 
connect.


I also note that like all standard resolver libraries, if you specify a 
hostname to spamc, it tries the v6 variant first -- so the default 
behaviors between spamc and spamd are still conflicting.  Nor is there an 
option in spamc to say "use this hostname, but only try v4".


Has anyone come up with patches for the above, or is the solution really to 
just hard-code the ipv4 address everywhere when doing a remote-connect (or 
perhaps define alternate v4-only hostnames for your spamd hosts). 

Hi Dan!

I'm working on packaging an RC for 3.4.0 and ipv6 is a big focus of this 
release. Can you open a bug about these issues with as much information as 
you can, please?


6840 (docs)
6841 (spamd)
6842 (spamc)

-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: Still no apparent fix on ipv6 spamd?

[Dan Mahoney, System Admin]

On Tue, 25 Sep 2012, Kevin A. McGrail wrote:



On 9/25/2012 5:02 PM, Dan Mahoney, System Admin wrote:

I mentioned this on the mailing lists a few years ago.

I notice that there still doesn't seem to be a clean way to just make spamd 
listen on all (v4 and v6) addresses by default, nor is there a way to 
listen on multiple addresses with multiple -A options.


This means that if you want to listen on v6, none of your v4 clients can 
connect.


I also note that like all standard resolver libraries, if you specify a 
hostname to spamc, it tries the v6 variant first -- so the default 
behaviors between spamc and spamd are still conflicting.  Nor is there an 
option in spamc to say "use this hostname, but only try v4".


Has anyone come up with patches for the above, or is the solution really to 
just hard-code the ipv4 address everywhere when doing a remote-connect (or 
perhaps define alternate v4-only hostnames for your spamd hosts). 

Hi Dan!

I'm working on packaging an RC for 3.4.0 and ipv6 is a big focus of this 
release. Can you open a bug about these issues with as much information as 
you can, please?


You got it.  Later today, probably.  Do you prefer one bug or multiple 
(there's at least four or five issues in this)?


-Dan

--

Pika Pika Pika!

-Pikachu, of Pokemon fame.

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Still no apparent fix on ipv6 spamd?

[Dan Mahoney, System Admin]

All,

I mentioned this on the mailing lists a few years ago.

I notice that there still doesn't seem to be a clean way to just make 
spamd listen on all (v4 and v6) addresses by default, nor is there a way 
to listen on multiple addresses with multiple -A options.


This means that if you want to listen on v6, none of your v4 clients can 
connect.


I also note that like all standard resolver libraries, if you specify a 
hostname to spamc, it tries the v6 variant first -- so the default 
behaviors between spamc and spamd are still conflicting.  Nor is there an 
option in spamc to say "use this hostname, but only try v4".


Has anyone come up with patches for the above, or is the solution really 
to just hard-code the ipv4 address everywhere when doing a remote-connect 
(or perhaps define alternate v4-only hostnames for your spamd hosts).


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

SpamAssassin Hanging on RTF Attachments

[admin]

Greetings,

I've spent many hours spread out over weeks scouring the Internet and 
message archives and FAQs and such, and am unable to find a solution to 
the problem I'm having.


I have a Postifx+AmavisNew+SpamAssassin+ClamAV setup for my mail 
server. It's all running on Ubuntu Server 10.04 LTS. I've had this 
configuration for many years, and have always had a problem receiving 
some RTF attachments. Here are the pertinent details:


Postfix: 2.8.5-2
Amavisd-new: 2.6.4
ClamAV: 0.97.5
SpamAssassin: 3.3.1
Perl: 5.10.1

Sometimes (but not always) when I receive an RTF attachment, the mail 
can't be delivered. A single amavis child pegs a CPU. Through much 
debugging, I've turned off ClamAV, turned ClamAV back on, and then 
turned off SpamAssassin. When I disabled Amavis's call to SpamAssassin, 
the mail cruised right through my system fine. This lead me to believe 
it was a problem with SpamAssassin.


I managed to capture (though a deferred email that I captured with 
'postcat') a single email that contained a problematic RTF attachment.


Running 'spamassassin -t < emailfile' resulted in the process locking 
up to the point that I had to 'kill -9' the process in another window. I 
waited 15 minutes for the process to finish, and it never did.


My questions are these:

Is there a way to disable SpamAssassin from scanning certain attachment 
types (e.g.: RTFs)?


If not, is there a way to tell SpamAssassin to only scan the first X 
bytes of an email? I've tried the amavis config of 
'$sa_mail_body_size_limit' and set it to a ridiculously low number 
(e.g.: 10 bytes) but this did not resolve the problem.


Is there a bug/flaw in SpamAssassin or one of its add-ons that I should 
be looking to upgrade to get away from this issue?


What other details/information do you need from me to assist me in 
troubleshooting this issue?


Thanks in advance for any time and consideration you give to this 
email.


PS: I can't share my problematic RTF file since it contains a legal 
contract I'm not allowed to share with the public.

Re: [sa-list] Re: Spamhaus Whitelist

[Dan Mahoney, System Admin]

On Sat, 6 Nov 2010, David F. Skoll wrote:


On Sat, 06 Nov 2010 00:41:53 -0700
Bill Landry  wrote:


You could also test the envelope sender:



header SPAMHAUS_ENV   eval:check_rbl_envfrom('SPAMHAUS_ENV', 
'_vouch.dwl.spamhaus.org.')


But that's an abuse... you should not be using Vouch-by-reference unless
either DKIM or SPF returns a "pass".  Otherwise, you've just told
spammers who they should pretend to be to get their spam in your inbox.


Yeah, I read that and suspected this needed more complex config than "just 
another Whitelist".


-Dan

--

"Happy, Sad, Happy, Sad, Happy, Sad, Happy, Intruiged!  I've never been so
in touch with my emotions!"

-AndrAIa as Hexadecimal, Reboot Episode 3.2.3

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Learing spam/ham with Pine

[Dan Mahoney, System Admin]

On Wed, 3 Nov 2010, John Hardin wrote:


On Wed, 3 Nov 2010, Pat Traynor wrote:


I've been running Spamassassin on my linux server for some time, and I
use Pine to read my mail.


Hello, fellow fossil!


Aah, yonder fossils.  I've found, by the way, that if you're not using 
Alpine, you sure should be.  Better bits, and some cool new features.  I 
consider myself a pine power user.


So, things to know.

1) If you're using pine and not procmail, you're missing out.  Learn it, 
live it, love it.


2) While John's methods for learning and reporting spam work, I've found 
that the best way to do it is per-message within spamassassin, via 
spamd/spamc.  The "pipe" command returns MUCH faster in this config.


(Make no mistake, there's nothing wrong with periodically feeding your 
spam and ham folders to bayes as well, but I have a personal policy of 
"report what gets through the filters to improve the system".


3) While you're at it, if you're using spamc/spamd, take whatever account 
they're registered under, and tie them to DCC/Pyzor/Razor -- if you're 
reporting, you might as well get the most bang for your buck.  If you're 
not using them, then register yourself a Pyzor/Razor/DCC account.  It's 
quick and easy.


Anyway, you'll need to go into your pine options and turn on the 
following:


* Enable Aggregate Command Set (this allows you to select and act on 
multiple messages at once)

* Enable unix pipe command

From there, you can simply take any message (or any group of messages) and 

press "|" to start a pipe, and set the following options:

* Raw Message
* Uncaptured output
* If working with multiple messages, set the "new pipe" option so each 
message gets fed to a separate copy of the command, and for the command 
itself, I put:


/usr/local/bin/spamc -d quark.gushi.org --reporttype=report

or

/usr/local/bin/spamassassin --report

If you want to correct a false positive, chance --"report" to "revoke"

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Spamhaus Whitelist

[Dan Mahoney, System Admin]

All,

Has anyone come up with a ruleset yet to score against the new spamhaus 
whitelists, and deduct points appropriately?


-Dan

--

"Let me tell you something about regrowing your dead wife Lucy, Harry.
It's probably illegal, potentially dangerous, and definitely crazy."

-Harry nods-

Vincent Spano, as Boris in "Creator".


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Pyzor occasionally dying when called from spamassassin.

[Dan Mahoney, System Admin]

Hey there,

I just enabled pyzor as part of spamassassin (freebsd 6.4, pyzor built 
from ports), and occasionally get this message in my logs:


Jul  9 05:40:59 quark spamd[11607]: spamd: connection from prime.gushi.org 
[72.9.101.130] at port 51280
Jul  9 05:40:59 quark spamd[11607]: spamd: processing message 
<80052004218074290153548c4434576868b5c94f5dd661c0...@pd164.marketingfx.info> 
for minn:58
Jul  9 05:41:05 quark spamd[11607]: pyzor: [11983] error: TERMINATED, 
signal 15 (000f)


I've got spamd at maximum logging, does anyone offhand know what this 
error means?


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Passing preferences to spamd?

[Dan Mahoney, System Admin]

On Thu, 8 Jul 2010, Karsten Bräckelmann wrote:


On Wed, 2010-07-07 at 18:09 -0400, Dan Mahoney, System Admin wrote:

It seems the only way to pass a preference from spamc to spamd is by
having a different user-id.

In my specific case, I'd like to report to spamcop using their "quick" UID
for some mails, but keep all my preferences otherwise the same (so I still
get the benefits of bayes, dcc, awl, etc).


Correct. You cannot pass anything spamd / spamassassin (the CLI tool)
accepts as options via spamc.


Since I'm using the DB backend, I could do some tricks, like modify the
query to template one set of userprefs over another, I suppose, but it
would be nice to have a unified way.


User prefs in DB? Then having specific sets of user_prefs (and only
what's allowed in there, no spamd options obviously) should be simple.

Have a look at the spamc -u username option.


Yes, I saw that...what I need to do is come up with a "clever" way of 
saying if I pass an "impossible" username, such as danm_reporting, the 
query does the right thing.  (Since the mysql command language has a split 
function, I should be able to do this without touching the spamassassin 
code.)


I might have to also modify the sql queries for the bayes/awl backends as 
well, so they know danm_report is really "danm", since reporting also 
includes learning.


This also opens up the possibility of creating a more strict setup for 
different email addresses, so -u danm_strict could have a required_score 
of 2, for addresses that are more agressively spammed.  (Yes, this would 
take multiple passes through spamd or some special procmail logic).



IIRC it works with DB backend. From memory, since I once tried long ago,
it does not work if you're using $HOME based user_prefs and running
spamc as an ordinary user.

If you want to see the behavior for yourself, please use the netcat
trick I mentioned in a previous thread of yours. Run netcat listening on
one port, and make spamc use that port -- you'll see the simple protocol
headers, including the User to use by spamd, if possible.


Knowing the headers is good, but being able to know what they look like, 
and knowing how to get the application to set them are two different 
problems, from my point of view.  There are people who debug with syslog 
and -v, there are people who debug with tcpdump, and there are people who 
debug with strace.  I'm the syslog type.



It seems there's no way to override an additional pref on the command line
with any of (spamc, spamd, spamassassin) -- you have to override the whole
file, and sometimes even more than that, in the case of spamc/spamd.


It is possible with spamassassin, as fine-grained as you want with any
setting. See the --cf option in man spamassassin-run.


At that point, I could in fact use spamassassin to point at the DB server 
where my bayes and awl, etc, live, the only real difference is in which 
cpu parses the message, the end effect is the same.


My biggest problem with this, as I had brought up a while ago, is that I'd 
need to run spamassassin setGID, and put the db files in a different 
config that's not world-readable by all.


I once wrote a small-but-useful tool which publishes your user_prefs to a 
database (and also fetches), asked on this list if people wanted it for 
CONTRIB.  No reply.


-Dan

--

Hate fedora with a white hot burning passion right now though ... damn thing is 
Linux-XP(tm)

-Bill Nolan
2/24/04

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Passing preferences to spamd?

[Dan Mahoney, System Admin]

All,

It seems the only way to pass a preference from spamc to spamd is by 
having a different user-id.


In my specific case, I'd like to report to spamcop using their "quick" UID 
for some mails, but keep all my preferences otherwise the same (so I still 
get the benefits of bayes, dcc, awl, etc).


(Think of this in terms of the -o options to ssh)

Since I'm using the DB backend, I could do some tricks, like modify the 
query to template one set of userprefs over another, I suppose, but it 
would be nice to have a unified way.


It seems there's no way to override an additional pref on the command line 
with any of (spamc, spamd, spamassassin) -- you have to override the whole 
file, and sometimes even more than that, in the case of spamc/spamd.


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Minor Doc Issue on spamc config file?

[Dan Mahoney, System Admin]

Hey all,

In my spamc config file I have:

-d 72.9.101.140
-l
--connect-retries=10
--retry-sleep=30

However, procmail scripts that I was using to report, via "spamc -C 
report", were simply returning the message.


When I added -d 72.9.101.140, the message was properly reported.

The manpage states:

"Existing command line switches will override any settings in the 
configuration file."


What I took from this was that if I specified, say, -d on the command 
line, it would override what's in the file.  But what this really seems to 
mean is that you need to specify ALL options when using the command line.


In other words, your command line can be really long (specifying all 
options from the command line, plus all options that would be in your 
config file) or really short (spamc or spamd -F configfile).


Is this by design?  With most other programs, it would seem only the 
options specified on the command line would be overridden (like -d 
127.0.0.1 in my example above).


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Adding headers on spamassassin ignores?

[Dan Mahoney, System Admin]

On Mon, 5 Jul 2010, Karsten Bräckelmann wrote:


On Mon, 2010-07-05 at 02:31 -0400, Dan Mahoney wrote:

The greater problem is, that if for some reason spamassasin doesn't run
(for example, a spamc timeout(*)) it produces exactly the same effect.

Is there a way to have spamassasin/dspamd not scan messages above a
certain size, but still add headers (i.e. x-spam-status: skipped)?  I can


No, SA cannot add headers in case the message size exceeds the spamc
threshold, because in that case spamc does not pass on the message to
spamd at all.


do it in procmail, and add a header that means something to me (and face
the additional problems of communicating this nuance to my users), but it
would be nice if SA had a standard way.


With procmail, the spamc -s option actually should be irrelevant to you,
unless *raising* the limit. Why have procmail pipe the message to a
filter, if we know it will be passed back unhandled?

 :0 fw
 * < 512000
 | spamc

Now there are two ways to add various "skipped" headers. A trivial one
is negating the size condition.

 :0 fw
 * > 511999
 | formail -A "X-Spam-Status: Skipped, too large"

A more fancy variant starts by using the spamc -x option in the above
recipe, disabling the default "safe fallback" of returning an exit code
of 0 regardless. Now errors will result in an actual error exit code,
while the unprocessed message still is passed back. See man spamc.

A procmail recipe to handle this must follow the spamc filter recipe
immediately, and looks like this, using the procmail error flag.

 :0 e fw
 | formail -A "X-Spam-Status: Error processing mail"



(*) with it's brilliant "try 3 times, 1 second apart" retry timer.


If this is merely about timing issues, where restarting spamd might
cause spamc to give up before the daemon is back, you could simply
adjust these. Both retry times as well as numbers of attempts are
configurable.

To do that globally, without even touching your procmail recipes, you
can use spamc.conf in your sysconfig dir. Again, see man spamc.


I have done so, and that may alleviate some of the problem.

However, while you've given me some procmail-based shortcuts and saved me 
a bit of research, my point was that it would be very nice if the messages 
and flags you mention were *standard* parts of spamd, and not just coded 
into my (and only my) procmailrc.


It's probably fairly trivial to have spamc add only this header under 
these conditions, and it would make it more compatible with third-party 
tools that are out there, and as a bonus, spamc could keep within the 
"principal of least surprise" by requiring an extra command line option to 
add these headers, so not as to break existing scripts.


--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Adding headers on spamassassin ignores?

[Dan Mahoney, System Admin]

Hey all,

From what I've gathered, there's both a recommended way to call 
spamassassin/spamd from procmail with a message-size-limit, as well as an 
overrideable builtin-default (-s option to spamc).


These both cause the usual spamassasin headers to be missing from 
messages.


The greater problem is, that if for some reason spamassasin doesn't run 
(for example, a spamc timeout(*)) it produces exactly the same effect.


Is there a way to have spamassasin/dspamd not scan messages above a 
certain size, but still add headers (i.e. x-spam-status: skipped)?  I can 
do it in procmail, and add a header that means something to me (and face 
the additional problems of communicating this nuance to my users), but it 
would be nice if SA had a standard way.


-Dan

(*) with it's brilliant "try 3 times, 1 second apart" retry timer.

--

"If you need web space, give him a hard drive.  If you need to do something really 
heavy, build him a computer."

-Ilzarion, late friday night

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Autoreplies from RT are hitting on ANY_BOUNCE_MESSAGE

[Dan Mahoney, System Admin]

On Mon, 28 Jun 2010, Yet Another Ninja wrote:


On 2010-06-28 11:33, Dan Mahoney, System Admin wrote:
> Hey there,
> 
> Perhaps this is by design, but rt replies are, strictly speaking, not 
> bounce messages.
> 
> Message attached, let me know if it looks "normal".
> 
> -Dan
> 

from what I see it looks normal if someone really makes an effort to 
"tune" SA scores.



my 50_scores.cf deault says:

score ANY_BOUNCE_MESSAGE 0.1
score SHORTCIRCUIT 0


Even so, why is it matching, when it's not a bounce.  It's either 
something inaccurate in spamassassin, or something RT is doing that it 
shouldn't be.  It it's the latter, I'll attempt to fix rt.  If the former, 
perhaps SA should.


-Dan

--

"You recreate the stars in the sky with cows?"

-Furrball, March 7 2005, on Katamari Damacy

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Learning and reporting with spamc in a single step?

[Dan Mahoney, System Admin]

On Mon, 28 Jun 2010, Karsten Bräckelmann wrote:


On Sun, 2010-06-27 at 16:52 -0400, Dan Mahoney, System Admin wrote:

Can spamc do this, or must it be forked to "tee" or something.

Ideally I'd like to both report and learn in a single step (such as in a
pipe from alpine).  I note that spamassassin -r also has the option to
learn (by default!), but spamc doesn't for some reason.  Or if it does,
the manpage neglects to mention it.


Hmm, man spamc shows -L learn type and -C report type right next to each
other. Yours doesn't?


It shows them top to bottom, but does not say whether they're exclusive or 
not.  As for the usage summary...


%spamc -V
SpamAssassin Client version 3.2.3
  compiled with SSL support (OpenSSL 0.9.7e-p1 25 Oct 2004)

SYNOPSIS
   spamc [options] < message

is less than helpful in determining which options work together.


If you actually can use both options at the same time, I don't know.
Maybe you wanna try it, and let us know. :)


I wonder what the logs show (or are supposed to show) during these 
operations.


-Dan

--

"You're a daddy.  I'm a mommy.  She's our baby.  Deal with it."

-Cali, 11/7/02, about 1:35 AM

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Autoreplies from RT are hitting on ANY_BOUNCE_MESSAGE

[Dan Mahoney, System Admin]

Hey there,

Perhaps this is by design, but rt replies are, strictly speaking, not 
bounce messages.


Message attached, let me know if it looks "normal".

-Dan

--


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---
From s...@isc.org Thu Jun  3 20:29:04 2010
From: ISC Systems via RT 
To: d...@prime.gushi.org
Date: Fri, 4 Jun 2010 00:28:53 +
Subject: SPAM(120.1) [ISC-Ops #28368] AutoReply: Live from new york 

Spam detection software, running on the system "quark.gushi.org", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email.  If you have any questions, see
The administrator of that system for details.

Content preview:  Greetings, This message has been automatically generated in
   response to the creation of a trouble ticket regarding: "Live from new york",
   a summary of which appears below. There is no need to reply to this message
   right now. Your ticket has been assigned an ID of [ISC-Ops #28368]. [...]
   

Content analysis details:   (120.1 points, 5.0 required)

 pts rule name  description
 -- --
 0.1 BOUNCE_MESSAGE MTA bounce message
 100 SHORTCIRCUIT   Not all rules were run, due to a shortcircuited rule
  20 ANY_BOUNCE_MESSAGE Message is some kind of bounce message




[ Part 2: "original message before SpamAssassin" ]

X-Envelope-To: UNKNOWN
From: ISC Systems via RT 
To: d...@prime.gushi.org
Date: Fri, 4 Jun 2010 00:28:53 +
Subject: [ISC-Ops #28368] AutoReply: Live from new york 


Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding:
"Live from new york", 
a summary of which appears below.

There is no need to reply to this message right now.  Your ticket has been
assigned an ID of [ISC-Ops #28368].

Please include the string:

 [ISC-Ops #28368]

in the subject line of all future correspondence about this issue. To do so, 
you may reply to this message.

Thank you,
s...@isc.org

-
It's ISC live.

-Dan

-- 

 Christ almighty...  my EYES!  They're melting!

-Zaren, Efnet #macintosh, in response to:

www.geocities.com/CollegePark/Classroom/1944
The WEBSITE DESIGN class that gave my fiancee a D.

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Learning and reporting with spamc in a single step?

[Dan Mahoney, System Admin]

Can spamc do this, or must it be forked to "tee" or something.

Ideally I'd like to both report and learn in a single step (such as in a 
pipe from alpine).  I note that spamassassin -r also has the option to 
learn (by default!), but spamc doesn't for some reason.  Or if it does, 
the manpage neglects to mention it.


In a perfect world, I'd also be able to choose the "express" or "manual" 
spamcop methods, which use different reporting addresses, but if I need to 
run two commands anyway from my .procmailrc, I might as well use spamc for 
one and spamassassin (with an alternate config file) for the other.


-Dan Mahoney

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Does spamd support ipv6 yet?

[Dan Mahoney, System Admin]

I previously asked this question and was told the best answer might be to 
wait for 3.3.


Was there ever support ratified for ipv6 including proper -A ipv6 access 
lists, and proper ability to listen on both the ipv6 default and the v4 
default at the same time, when specifying -i?


I'm not sure which bugs to look at to ascertain this.

-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

need to uninstall Spamassassin 3.3.1

[Security Admin (NetSec)]

Have tried upgrading Spamassassin 3.2.5 to 3.3.1 and the result was a disaster. 
 Currently have the spamassin* of one version and perl-Mail-spamassassin* of 
another.  Running "rpm -e spam*" I get the following error:

 error: package spamassassin-3.2.5-1.x86_64.rpm is not installed
error: package spamassassin-3.3.1-1.x86_64.rpm is not installed

I want to get spamassassin OFF completely and go back to 3.2.5 which I know 
works

How can I accomplish this?

Thanks in advance!

Re: [sa-list] Re: Adding remote-ip/ESMTPID/X-Envelope to logging output?

[Dan Mahoney, System Admin]

On Sun, 27 Dec 2009, Shane Williams wrote:


One way to find what you want is to grab the msg id (or mid) from the
spamd line, and grep for that out of the sendmail log for the remote
IP.  As I recall when I wrote something that searched like this, I had
to do some special character quoting on some of the mid's, but did
finally get it working.


Yeah, it's possible to have my parser do that kind of cross-correlation, 
and I imagine it's what I'll have to do (along with using syslog to send 
files from one server to the other), as well as keeping them local.


I just figured if there was a single local.cf tweak I could add that would 
change my logline, that might be the easier and more correct way, as 
getting the score, plus all the rules matched, plus the long-messageid, 
plus the short messageID is rather complex.  (Even within just spamd it 
requires looking at multiple lines)


For example, this line contains the score:

Dec 28 02:37:35 quark spamd[9203]: spamd: identified spam (20.1/5.0) for 
danm:58 in 0.4 seconds, 3920 bytes.


But this one, which contains almost everything else of use, does NOT 
contain the decimal score, and there's nothing there at ALL to 
cross-correlate them (and this is running in debugmode).  I can guess 
based on the size, score, scantime and uid, but those are hardly unique, 
especially 
during a deluge.


Dec 28 02:37:35 quark spamd[9203]: spamd: result: Y 20 - 
ANY_BOUNCE_MESSAGE,BOUNCE_MESSAGE 
scantime=0.4,size=3920,user=danm,uid=58,required_score=5.0,rhost=prime.gushi.org,raddr=72.9.101.130,rport=53762,mid=<200912280733.nbs7xfkj049...@prime.gushi.org>,bayes=0.001253,autolearn=disabled,shortcircuit=spam


Somewhere within the spamd guts there's a print/printf line that prints 
that last line, and is supplied a list of variables.  I mean only to add a 
couple more.  I'm quite surprised it's not a tunable.


I'm also surprised that, unline sendmail, spamd doesn't put a single token 
in EVERY logline, even if that's a unique messageid known only to SA (as 
is the case with sendmail logs).


-Dan



On Sat, 26 Dec 2009, Dan Mahoney, System Admin wrote:


Hey there,

Background: Sendmail with spamd running on a different box, spamc called 
from global procmail file.


I'm doing some nightly log-combing to look for interesting patterns, 
including against other network traffic (like erroneous DNS lookups, I 
think I might be on to something).


However, one of the annoying things about spamassassin's logging is that it 
fails to log the remote connecting ip, even though it places it in special 
places in the logs:


take for example:

Dec 26 08:41:51 quark spamd[87490]: spamd: connection from prime.gushi.org 
[72.9.101.130] at port 62430
Dec 26 08:41:51 quark spamd[87490]: spamd: processing message 
 for danm:58
Dec 26 08:41:53 quark spamd[87490]: FuzzyOcr: Scan canceled, message has 
less than -5 points (-6.601).
Dec 26 08:41:53 quark spamd[87490]: spamd: clean message (-6.6/5.0) for 
danm:58 in 1.9 seconds, 3788 bytes.
Dec 26 08:41:53 quark spamd[87490]: spamd: result: . -6 - 
AWL,BAYES_00,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS 
scantime=1.9,size=3788,user=danm,uid=58,required_score=5.0,rhost=prime.gushi.org,raddr=72.9.101.130,rport=62430,mid=,bayes=0.00,autolearn=ham,shortcircuit=no


From those logs, there's nothing at all that tells me what the relaying ip 

is, even though it's "special" to spamd, used to determine the ASN, etc.

The sendmail logs (grepped for that messageid) are more useful:

Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: 
from=, size=2735, class=0, 
nrcpts=1, 
msgid=, 
proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17]


But again, those give me half the picture (and are on two different 
machines), and I'd need the long msgid line to correlate them.


Is logging output configurable that I could add the value of the "relay=" 
line into the output?  Or perhaps the value of "X-Envelope-To?"


Also, does spamc have any concept of the "short" (ESMTP) messageid, as 
defined by sendmail's queues? (nBQDcLck027423).  In terms of parsing logs, 
this is a much more useful correlation point, since it's that identifier 
that every other milter uses, and every other thing that writes to maillog 
uses. (But I understand if it's not possible since the API is different).


For example, grepping for that self-same messageid, other than spamc, gives 
me the whole story.  Sender, recipient, every milter it's been through.


Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: 
from=, size=2735, class=0, 
nrcpts=1, 
msgid=, 
proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17]
Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: Milter 
insert (1): header: X-DomainKeys:  Sendmail DomainKeys Filter v1.0.2 
prime.gushi.org nBQDcLck027423
Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck02

Adding remote-ip/ESMTPID/X-Envelope to logging output?

[Dan Mahoney, System Admin]

Hey there,

Background: Sendmail with spamd running on a different box, spamc called 
from global procmail file.


I'm doing some nightly log-combing to look for interesting patterns, 
including against other network traffic (like erroneous DNS lookups, I 
think I might be on to something).


However, one of the annoying things about spamassassin's logging is that 
it fails to log the remote connecting ip, even though it places it in 
special places in the logs:


take for example:

Dec 26 08:41:51 quark spamd[87490]: spamd: connection from prime.gushi.org 
[72.9.101.130] at port 62430
Dec 26 08:41:51 quark spamd[87490]: spamd: processing message 
 for danm:58
Dec 26 08:41:53 quark spamd[87490]: FuzzyOcr: Scan canceled, message has 
less than -5 points (-6.601).
Dec 26 08:41:53 quark spamd[87490]: spamd: clean message (-6.6/5.0) for 
danm:58 in 1.9 seconds, 3788 bytes.
Dec 26 08:41:53 quark spamd[87490]: spamd: result: . -6 - 
AWL,BAYES_00,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS 
scantime=1.9,size=3788,user=danm,uid=58,required_score=5.0,rhost=prime.gushi.org,raddr=72.9.101.130,rport=62430,mid=,bayes=0.00,autolearn=ham,shortcircuit=no


From those logs, there's nothing at all that tells me what the relaying ip 

is, even though it's "special" to spamd, used to determine the ASN, etc.

The sendmail logs (grepped for that messageid) are more useful:

Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: 
from=, size=2735, class=0, 
nrcpts=1, 
msgid=, 
proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17]


But again, those give me half the picture (and are on two different 
machines), and I'd need the long msgid line to correlate them.


Is logging output configurable that I could add the value of the "relay=" 
line into the output?  Or perhaps the value of "X-Envelope-To?"


Also, does spamc have any concept of the "short" (ESMTP) messageid, as 
defined by sendmail's queues? (nBQDcLck027423).  In terms of parsing logs, 
this is a much more useful correlation point, since it's that identifier 
that every other milter uses, and every other thing that writes to maillog 
uses. (But I understand if it's not possible since the API is different).


For example, grepping for that self-same messageid, other than spamc, 
gives me the whole story.  Sender, recipient, every milter it's been 
through.


Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: 
from=, size=2735, class=0, 
nrcpts=1, 
msgid=, 
proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17]
Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: Milter 
insert (1): header: X-DomainKeys:  Sendmail DomainKeys Filter v1.0.2 
prime.gushi.org nBQDcLck027423
Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: Milter 
insert (1): header: Authentication-Results:  prime.gushi.org; dkim=none 
(no signature)\n\theader.i=unknown; x-dkim-adsp=none
Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: Milter 
insert (1): header: X-DKIM:  Sendmail DKIM Filter v2.8.3 prime.gushi.org 
nBQDcLck027423
Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: Milter 
insert (1): header: Authentication-Results: prime.gushi.org; 
sender-id=pass header.sender=asterisk-users-boun...@lists.digium.com; 
spf=pass smtp.mfrom=asterisk-users-boun...@lists.digium.com
Dec 26 08:38:23  prime sm-mta[27423]: nBQDcLck027423: Milter 
insert (1): header: X-SenderID: Sendmail Sender-ID Filter v1.0.0 
prime.gushi.org nBQDcLck027423
Dec 26 08:38:24  prime sm-mta[27423]: nBQDcLck027423: Milter 
add: header: X-Greylist: Default is to whitelist mail, not delayed by 
milter-greylist-4.0.1 (prime.gushi.org [72.9.101.130]); Sat, 26 Dec 2009 
08:41:49 -0500 (EST)
Dec 26 08:38:28  prime sm-mta[27436]: nBQDcLck027423: 
to=, delay=00:00:05, xdelay=00:00:03, mailer=local, 
pri=33624, dsn=2.0.0, stat=Sent


Thoughts?

-Dan Mahoney

--

"When I'm lost, and confused, and trying to make a U-turn, nothing annoys
me more than someone telling me to watch out for the tombstone!"

"How often does that happen, Fab?"

-David Feld & Tom Fabry, sometime in High School.

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

bayes: cannot open bayes databases /home/user/.spamassassin/bayes_* R/W: tie failed: No such file or directory

[Admin]

Hi there,

Any clues how I can fix the following error?
sa-learn is failing

$ sa-learn --no-sync --spam --mbox ~/mail/Spam
bayes: cannot open bayes databases /home/user/.spamassassin/bayes_* R/O: 
tie failed:
bayes: cannot open bayes databases /home/user/.spamassassin/bayes_* R/O: 
tie failed: Bad file descriptor
bayes: cannot open bayes databases /home/user/.spamassassin/bayes_* R/W: 
tie failed: No such file or directory

Learned tokens from 0 message(s) (1 message(s) examined)
ERROR: the Bayes learn function returned an error, please re-run with -D 
for more information



Cheers,

Noah

Re: processing of mail considered spam

[Admin]

Jari Fredriksson wrote:

Hi there,

Some spam is getting past the spamassassin.  So I;d like to devise a
scheme where I manually place the emails not caught by spamassass in my
'spam-mail' folder.  Is there any way to get spamassassin to process the
contents of the folder so I those accepted messages are considered spam
in the future?



It depends. If the "spam-mail folder" is a folder in a POP3 client, it is
not easy to automate it. if the "spam-mail folder" is a folder in a
IMAP-server, and the server uses Maildir format under Linux or other Unix,
it is easy. The mails are plain text files in the server file system, and
a cron job can easily run sa-learn --spam with that folder.

If the server is some MS Exchange I have no idea. Maybe the folder could
be downloaded periodically with fetchmail or such, and fed to sa-learn.

Anyway, SpamAssassin itself does not help besides sa-learn, so this needs
scripting on your part.




okay how do I script sa-learn to learn the contents of a particular file.

Cheers,

Noah

processing of mail considered spam

[Admin]

Hi there,

Some spam is getting past the spamassassin.  So I;d like to devise a 
scheme where I manually place the emails not caught by spamassass in my 
'spam-mail' folder.  Is there any way to get spamassassin to process the 
contents of the folder so I those accepted messages are considered spam 
in the future?


Cheers,
Noah

spamassassin not working

[Admin]

Hi there,

I do not see spamassassin processing information in the SMTP header of 
incoming messages.  So I am fairly sure that the processing is not 
working.  I am hoping to get the postfix->procmail->spamc processing 
path working system-wide.  I need some help though since it is not working.


So here are the configuration files and postfix running configuration:


 postconf -n -
# postconf -n
alias_database = hash:/etc/postfix/aliases 
hash:/var/lib/mailman/data/aliases

alias_maps = hash:/etc/postfix/aliases hash:/var/lib/mailman/data/aliases
append_dot_mydomain = no
biff = no
broken_sasl_auth_clients = yes
config_directory = /etc/postfix
delay_warning_time = 4h
home_mailbox = Maildir/
inet_interfaces = all
inet_protocols = all
mailbox_command = /usr/bin/procmail
mailbox_size_limit = 0
mydestination = $myhostname, localhost.$mydomain $mydomain
mydomain = domain.com
myhostname = domain.com
mynetworks = 100.100.100.100/32 127.0.0.0/8 [:::127.0.0.0]/104 [::1]/128
myorigin = domain.com
readme_directory = no
recipient_delimiter = +
relayhost =
smtp_tls_note_starttls_offer = yes
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
smtpd_recipient_restrictions = 
permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination

smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain =
smtpd_sasl_security_options = noanonymous
smtpd_tls_CAfile = /etc/ssl/certs/cacert.pem
smtpd_tls_auth_only = no
smtpd_tls_cert_file = /etc/ssl/certs/smtpd.crt
smtpd_tls_key_file = /etc/ssl/private/smtpd.key
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_session_cache_timeout = 3600s
smtpd_use_tls = yes
tls_random_source = dev:/dev/urandom
unknown_local_recipient_reject_code = 550
virtual_alias_domains = virtual.org
virtual_alias_maps = hash:/etc/postfix/virtual 
hash:/var/lib/mailman/data/virtual-mailman


 postfix/main.cf --
$ cat /etc/postfix/main.cf
# See /usr/share/postfix/main.cf.dist for a commented, more complete version
# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
#append_dot_mydomain = yes
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
delay_warning_time = 4h

readme_directory = no

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myorigin = domain.com
myhostname = domain.com
mydomain = domain.com
mydestination = $myhostname, localhost.$mydomain $mydomain
alias_maps = hash:/etc/postfix/aliases hash:/var/lib/mailman/data/aliases
alias_database = hash:/etc/postfix/aliases 
hash:/var/lib/mailman/data/aliases

virtual_alias_domains = blah.org
virtual_alias_maps = hash:/etc/postfix/virtual 
hash:/var/lib/mailman/data/virtual-mailman

relayhost =
mynetworks = 127.0.0.0/8 [:::127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
unknown_local_recipient_reject_code = 550
inet_interfaces = all
inet_protocols = all
home_mailbox = Maildir/
mailbox_command = /usr/bin/procmail

# TLS parameters
smtpd_tls_cert_file = /etc/ssl/certs/smtpd.crt
smtpd_tls_key_file = /etc/ssl/private/smtpd.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_security_level = may
smtpd_tls_security_level = may
smtpd_tls_auth_only = no
smtp_tls_note_starttls_offer = yes
smtpd_tls_CAfile = /etc/ssl/certs/cacert.pem
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom
smtpd_sasl_local_domain =
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = 
permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination





- /etc/procmailrc -

$ cat /etc/procmailrc
# SpamAssassin sample procmailrc
# ==

# The following line is only used if you use a system-wide /etc/procmailrc.
# See procmailrc(5) for infos on what it exactly does, the short version:
#  * It ensures that the correct user is passed to spamd if spamc is used
#  * The folders the mail is filed to later on is owned by the user, not
#root.
DROPPRIVS=yes

# Pipe the mail through spamassassin (replace 'spamassassin' with 'spamc'
# if you use the spamc/spamd combination)
#
# The condition line ensures that only messages smaller than 250 kB
# (250 * 1024 = 256000 bytes) are processed by SpamAssassin. Most spam
# isn't bigger than a few k and working with big messages can bring
# SpamAss

Re: [sa-list] Re: A rant about FUZZY_OCR

[Dan Mahoney, System Admin]

On Mon, 27 Apr 2009, Jo Rhett wrote:


On Apr 27, 2009, at 1:16 PM, Dan Mahoney, System Admin wrote:
The problem exists now, there is PNG spam, and there will continue to be, 
because it gets through.  Right now the only way I find this blocked is if 
spamcop blocks it.



Just as a point of reference, I'd like to note that we haven't bothered with 
FuzzyOCR here and absolute none of the spam which reaches my inbox is a PNG 
or JPG or GIF spam.   SA does block it, and it does so without FuzzyOCR.


That said, we have jacked the scores for e-mail with images and no text and 
that might be why.   We never, ever receive valid e-mail with no text in it.


The spam I've been getting contains text, lots of it.  Markov-chain like 
crap that is 100 percent nonrelevant to the image.


-Dan


--

"She's NOT my girlfriend!"

-Dan Mahoney, Quite a bit recently.

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: A rant about FUZZY_OCR

[Dan Mahoney, System Admin]

On Mon, 27 Apr 2009, Henrik K wrote:

Nothing of this makes sense. If you don't have a test server, too bad. If
you don't trust the "score-changing values" too bad. It all worked for me.


It's a great idea, but I'd like to see it mature some first, especially
with respect to its documentation, test emails, word list, and live testing.


If was quickly developed to an ongoing problem. The problem disappeared
years ago. It was mature enough for 99% of users at that time. Though it did
add lots of complexity and stricter MTA rules etc handled the job just fine
also.


The problem exists now, there is PNG spam, and there will continue to be, 
because it gets through.  Right now the only way I find this blocked is if 
spamcop blocks it.


Ideally, what I'd probably like to see with regard to fuzzyOCR are:

1) Just patch it enough to work with 3.2 and 3.3 -- I don't have the 
internals know-how to do this, and I don't know if Decoder still reads 
this list.


2) A debug mode, whereby the plugin would note its own score, possibly by 
applying an equal negative value.


3) Wordlists loadable from userprefs, if not bayes.

4) A recommended configuration, along with "shortcircuit" documentation.

-Dan

--

"Ca. Tas. Tro. Phy."

-John Smedley, March 28th 1998, 3AM

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Code Rot?

[Dan Mahoney, System Admin]

Hey all,

While there's a decent amount of spamassassin list traffic to imply 
otherwise, is the SA project falling dormant?


the sare-rules claim they won't be updated due to lives, wives, and 
hockey.


the fuzzyOCR project claims the only thing that works with 3.2 is the SVN 
version, and on the same page claims you shouln't really expect the SVN 
version to work.


The wiki pages show the last release as almost a year ago, with no notice 
of any betas, pending releases, or whatnot.


Many commercial products have happily used SA in their core offering, is 
that where the future of development is?


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Image spam and failing rule

[Dan Mahoney, System Admin]

On Sat, 25 Apr 2009, John Hardin wrote:


On Sat, 25 Apr 2009, Gary Forrest wrote:

We are receiving the same image spam many times, random text within the 
body.


FuzzyOCR. It seems Spammers are trying image spam again, after giving up on 
it for a year or so.


Is there a version of FuzzyOCR that's actually supported with the current 
SA release?  Or under active development at all?


-Dan

--

"Man, this is such a trip"

-Dan Mahoney, October 25, 1997

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Spamd and ipv6

[Dan Mahoney, System Admin]

On Fri, 5 Dec 2008, Dan Mahoney, System Admin wrote:

Also, sorry about the subject headers.  I think I've fixed my procmail 
recipe.


-Dan

--

"I love you forever eternally."

-Connaian Expression

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: [sa-list] Re: [sa-list] Re: Spamd and ipv6

[Dan Mahoney, System Admin]

On Wed, 3 Dec 2008, SM wrote:


At 18:23 02-12-2008, Byung-Hee HWANG wrote:
Are you using FreeBSD or NetBSD? If so, i understand you. Unfortunately, SA 
developers do not care about IPv6 yet. So here SA program at first do 
action with "127.0.0.1" than "::1", i guess ;;


This was tested on a BSD system.  SpamAssassin developers are sharing their 
code for free.If we need a specific feature or find a bug, we can always 
send a patch.  If you read the URL I posted previously, you will see that the 
developers have been working on IPv6 support.


fwiw, I wasn't trying to sound abrasive, simply requesting that since the 
expected behavior is that if the behavior is that the client should try 
v6, then v4 -- that the server should have options to bind that way 
(assume I am running a spamd server that serves both v4 and v6 users). 
Right now I cannot multi-stack bind (is that being worked on?) or bind to 
multiple addresses (is that also being worked on?).


Alternatively, there should be an flag in the client to control whether 
it connects on v4 or v6, and the default should be consistent with how the 
server functions by default.  If v6 support in the server isn't done yet, 
then v4 should be the default.


Of course, opening a bug on this won't help since it's slated for fixing 
and ostensibly already have bugs open.


I suggested there might also be docbugs, but since this support is coming 
in the next release, amending the docs in the current version wouldn't 
help.


I've found bug reports to be a *terrible* method of communication unless 
someone on a list who knows the product better than I says "yeah, that's a 
bug, open one".


-Dan

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: [sa-list] Re: Spamd and ipv6

[Dan Mahoney, System Admin]

On Mon, 1 Dec 2008, SM wrote:


At 23:01 30-11-2008, Dan Mahoney, System Admin wrote:

So then, you're saying the behavior for ipv4 and ipv6 is somehow different?


If you start spamd without specifying the IP addresses to listen on, spamd 
will listen on the 127.0.0.1 IP address only.


And on an ip6 enabled system, where will "spamc localhost" try to connect 
to first?  127.0.0.1 or ::1?


You should have the IO::Socket::INET6 and Socket6 Perl modules installed to 
have IPv6 support in spamd.


I have both modules present:

quark# perl -e 'use IO::Socket::INET6'
quark# perl -e 'use Socket6'


You can start spamd as follows:

spamd -i 2001:DB8:1:1::1


Yes, but there's no way to listen on *both* addresses -- however, it's 
completely possible to listen on all ip4 addresses -- I'm just looking for 
a switch that will say "all ip4 AND all ip6".


Also, would be useful if I could specify to listen on "::" or "[::]" 
(neither worked when I tried it.)  Again, consistent behavior between v4 
and v6 is what I'm looking for.


spamd only allows connections from 127:0.0.1.  You can allow connections from 
other IP addresses with the -A parameter.  You may have to patch 
Mail::SpamAssassin::NetSet.  See 
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=4964


Additionally, even when I get this working, I am unable to specify ipv6 
addresses to -A, either with or without square brackets.


That part of the code is IPv4 specific.


Listening on v6 is pointless if I can't restrict.  Is the correct answer 
"open another bug?"  Or from these commit messages, should I simply assume 
the next 3.3 will have these (I see jm's note that the patches shouldn't 
cleanly apply to 3.2.x.)?


As stated, I've fixed this (for now) by changing my "spamc" args to have 
the v4 address.


-Dan

--

"Your future hasn't been written yet; no one's has.  So make it a good
one!"

-"Doc" Emmet L. Browne, Back to the Future III

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Spamd and ipv6

[Dan Mahoney, System Admin]

On Sun, 30 Nov 2008, SM wrote:


At 21:45 30-11-2008, Dan Mahoney, System Admin wrote:
Since getting my hosts natively speaking ipv6, I've been seeing a lot of 
initial timeouts connecting to spamc, because I believe it's apparently 
trying ipv6 first.


spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): 
Connection refused


[snip]

However, I cannot get the -A systax for spamd to accept connections from a 
given address, nor does it appear to be listening on said address:


quark# netstat -na | grep LIST | grep 783
tcp4   0  0  *.783  *.*LISTEN


Use the -i parameter to specify the IPv6 address.  The -A parameter to 
specify the host which can connect to spamd and not the IP address on which 
spamd should listen on.


So then, you're saying the behavior for ipv4 and ipv6 is somehow 
different?


I am starting spamd with -i but no ip specified, according to the docs:

"If you specify no IP address after the switch, spamd will listen on all 
interfaces.  (This is equal to the address 0.0.0.0)."


"All Interfaces" != "0.0.0.0"

At the very least, this is a docbug and should be amended to say "all 
ipv4 interfaces".


No mention is made of whether or not multiple -i arguments can be 
specified, but from my research, only the first -i is used, and you cannot 
comma-separate.


This is a second docbug, or a functionality that should be added to listen 
on v4 and v6 simultaneously.


Additionally, even when I get this working, I am unable to specify ipv6 
addresses to -A, either with or without square brackets.


Behaviorally, spamc *tries v6 by default* but spamd requires 
hoop-jumping.  This is a consistency problem and should also be looked 
into.


V6 is coming, fast.  Things like this are worth chasing down.  Let me know 
if you need me to run any other debugs or anything.


If you need access to my systems, please just say the word.  I like having 
something to offer in the solution of a problem, other than just 
complaints :)


-Dan

--

"Man, this is such a trip"

-Dan Mahoney, October 25, 1997

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Spamd and ipv6

[Dan Mahoney, System Admin]

Since getting my hosts natively speaking ipv6, I've been seeing a lot of 
initial timeouts connecting to spamc, because I believe it's apparently 
trying ipv6 first.


spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused
spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused
spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused
spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused
spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused
spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused
spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused
spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 
3): Connection refused


However, I cannot get the -A systax for spamd to accept connections from a 
given address, nor does it appear to be listening on said address:


quark# netstat -na | grep LIST | grep 783
tcp4   0  0  *.783  *.*LISTEN

I'm running a recent enough version that v6 *should* be supported.

Versions:

SpamAssassin Server version 3.2.5
  running on Perl 5.8.8
  with SSL support (IO::Socket::SSL 1.13)
  with zlib support (Compress::Zlib 2.008)

Any ideas?

--

"I can feel it, comin' back again...Like a rolling thunder chasin' the
wind..."

-Dan Mahoney, JS, JB & SL, May 10th, 1997, Approx 1AM

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Blogger URLs

[Dan Mahoney, System Admin]

On Sun, 20 Apr 2008, Theo Van Dinter wrote:


On Sun, Apr 20, 2008 at 12:39:29PM -0400, Dan Mahoney, System Admin wrote:

Can someone do a spam-versus-ham comparison for included links to
blogger.com (I don't have the corpus handy, nor do I know how to set up a
"proper" test.)


It's not really going to help you, you'd need to know the #s for your mail
flow.


Okay, so presumably then -- in my "normal" mail flow, there were all of 
six -- and those were with a client who was specifically giving me the URL 
to point her site at it.


Is there a tool, with the standard SA distribution, that can let me do a 
comparison analysis?


I found some tools here 
http://wiki.apache.org/spamassassin/StatsAndAnalyzers that gives me a 
whole bunch of after-the-fact info (i.e. on a live pool), but not 
something to let me say "okay, here's a defined rule, find all the X's and 
Y's (but I'm sure the SA team has something for such).


Didn't find anything in the rule submission guidelines either.


If it proves high enough, would a rule be possible?


Sure, go ahead, it's your setup. :)


Also, would it be possible to make spamassassin -r smart about reporting
such links straight to the feedback form here:

http://help.blogger.com/?page=troubleshooter.cs&problem=&ItemType=spam&contact_type=Spam&Submit=Continue


You could write a plugin to do it, but generally this is what spamcop is for
imo.


I've found spamcop to be a problem for two reasons:

1) It's an ANNOYING process.  And in theory I could automate it, but that 
circumvents the whole idea.


2) A plugin specifically targeted for blogger could check for the standard 
error messages -- not report if so, etc etc.


I've also had issues with spamcop not following the links right (for 
example, a popular ploy is to load the IMAGES in spam from other sources 
which SC doesn't follow)


The possibility of catering the reporting protocols to different sites 
(i.e. the major free sites have their own reporting systems that might be 
better used).  It's beyond the scope of this thread, but are there any 
docs on how to write a reporting protocol?


-Dan


--
Randomly Selected Tagline:
"Everyone looks like they're wearing the game board from Sorry."
 - Dennis Miller



--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Blogger URLs

[Dan Mahoney, System Admin]

Hello all,

A lot of the spam I'm seeing sneak past spamassassin has a blogger url in 
it (this seems to be a new favorite for spammers).


I've got about 200 such spams that have managed to sneak past (no idea how 
many of the 2 spams in my confirmed-kills folder also match).


So, that said:

Can someone do a spam-versus-ham comparison for included links to 
blogger.com (I don't have the corpus handy, nor do I know how to set up a 
"proper" test.)


If it proves high enough, would a rule be possible?

Also, would it be possible to make spamassassin -r smart about reporting 
such links straight to the feedback form here:


http://help.blogger.com/?page=troubleshooter.cs&problem=&ItemType=spam&contact_type=Spam&Submit=Continue

-Dan

--

"Be happy.  Try not to hurt each other.  Hope you fall in love."

--Mallory, Family Ties Finale (on the meaning of life)

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: Slow processing with 3.2.4

[Spam Admin]

Details on configuration. Both machines, ma1 and ma2 are identical.

We are running:

sendmail
spamhaus.org blacklist which rejects the majority of mail so 
SpamAssassin does not have to process that chunk of mail.

spamd (spamassassin) - 2 instances of spamd on each machine.
mimedefang
clamav
a few other milters

Dell 1750's 2.5G ram.
Dual Processor 2.4 GHz

Our graphs of memory usage do not show that we are using all of the RAM.

volume of mail is about 100K messages per day, per server, nearly 
equally balanced.

About 50-60% are discarded by one or more of the various filters.
About 40-50% is blocked by spamhaus.org -- thus not processed by 
SpamAssassin

About 10% is quarantined by Spam Assassin

The ma1 machine processors report a pretty steady CPU usage of about 
80-100% running SA 3.1.9.
The ma2 machine processors report a pretty steady CPU usage of about 
80-100% running SA 3.1.9, but when we upgrade ma2 to 3.2.4...
The ma2 machine processors report a pretty steady CPU usage of about 
300-400% running SA 3.2.4. With compiled rules (sa-compile). Also the 
similar performance with non-compiled rules.


Dan Zachary





  

 - is spamd taking up the CPU time or is it your MTA queuing taking up
   CPU time?  3.2.3+ will wait longer for DNS responses, decreasing
   throughput per child but not increasing CPU time per message

If you describe *how* you're using SA someone might have some more
suggestions.

Daryl



  

Re: Slow processing with 3.2.4

[Spam Admin]

Yes, the hardware is identical. The MX records are both '10', and the 
volume of mail is slightly LESS on the 3.2.4 machine over the 3.1.9 it's 
taking more time to process less mail on the newer machine.


We have 2.5 Gig memory on each machine. Our graphs show that we are 
using about 1.5 Gig of this. Swap usage is the same on both machines at 
500M each.


Thanks for the suggestions.  They are good factors to consider.

Dan Zachary


Kris Deugau wrote:

Spam Admin wrote:
I have two mail servers running Spamassassin.  One is running 3.1.9 
and the other 3.2.4, both with the same set of local rules, plus the 
standard rules that come with each version.


The 'load' on the processors for 3.2.4 is about *4 times more *than 
the 'load' on 3.1.9. 


I'm assuming the hardware is something resembling identical?  
Otherwise it's really hard to compare.


-> CPU speed?
-> Memory?
-> SA child limit parameters?

3.2 sucks down more memory than 3.1;  if the hardware is identical but 
the machine running 3.1 shows a small swap usage, the 3.2 machine is 
likely hitting swap a lot harder, causing your high load.


Do others have the same problem? Is this a typical change between the 
pre 3.2.x versions and the current version?


I don't know about 4x from 3.1.x to 3.2.x, but I certainly avoided 
upgrading from (patched-for-URI-RBLs) 2.64 for a LONG time because of 
the memory load of 3.x.


Of course, more rules means more CPU load, but the memory load was a 
far larger problem for me;  any more memory use on the old machine 
would have pushed the system into swap, which would have *really* 
killed performance...


-kgd

Slow processing with 3.2.4

[Spam Admin]

I have two mail servers running Spamassassin.  One is running 3.1.9 and 
the other 3.2.4, both with the same set of local rules, plus the 
standard rules that come with each version.


The 'load' on the processors for 3.2.4 is about *4 times more *than the 
'load' on 3.1.9. 

Do others have the same problem? Is this a typical change between the 
pre 3.2.x versions and the current version?


Dan Zachary

[no subject]

[Dan Mahoney, System Admin]

--

"Man, this is such a trip"

-Dan Mahoney, October 25, 1997

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Fri, 26 Oct 2007, Matthias Leisi wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Alex Woick schrieb:

[Spamcop]
I understand the two step reporting process too, and I too find it
annoying and timeconsuming to ack my (manually reviewed) 50 spams per
day to them, so I ceased to do it. There exist scripts for ack'ing
automatically, but this is not the intention of this process, so this is
no alternative for me.


I don't speak for Spamcop, but I do speak for dnswl.org. From our
experience I can tell that a manual review process is very important to
ensure data quality.

At least in the context of dnswl.org, there is little value in reporting
for the sake of reporting alone -- there needs to be some quality
control involved, or otherwise we run a high risk of including unwanted
IP addresses.

Having said that, we of course welcome all reports on false positives,
especially on IP addresses with a "low", "med" or "hi" score, and we
welcome all notifications of mailservers we do not yet know about.


It's rather simple, really.

If I'm auto-reporting spams with a score of (let's say, 15...enough that 
regardless of the DNSWL score's "negative" it would still be enough to 
auto-learn as "spam" to DNSWL (and DNSWL is passing complaints onto the 
original mailserver, which seems a logical thing) this serves as a 
reminder to the original mail server (let us say, in this case, two 
things).  This is the kind of thing that I would suggest be an enhancement 
to SA (but off by default for privacy reasons), on the spamd side, at the 
same time as bayes auto-learning happens.


1) That they are sending spam that risks their whitelist rating.

and

2) That the email they are sending is probably too spammish ANYWAY, if 
it's of a high enough threshhold ABOVE the DNSWL score to still be 
reported.


If you are a spammer, this allows you not only to listwash, but also to 
scrub and detail your email so it hits less SA rules -- of course, if you 
are any kind of pro spammer, presumably you are running your mails through 
at least a standard SA install anyway to test them.


If on the other hand you are a legitimate user of this service, *and* you 
are a producer of regular volumes of email, locally originated, that has 
some spammish tendencies (badly formed HTML parts, or being sent by a 
non-malicious script, then it allows you to correct other means of those 
false positive.


Naturally, if DNSWL isn't reporting back to the mailserver user, none of 
the above applies.


Manually reporting, on the other hand, is something that I would tie into 
the "spamassassin -r" functions, and much LIKE spamcop or the others, I'd 
suggest one or two extra pieces of data:


Some kind of a reporting ID, which determined the severity of the report 
(i.e. anonymous reports were given less credence).  And if the reports 
were going to be given back to the original mailserver again, some option 
to have the identifying data stripped.


Also, the ability to view the number of reports for a given server helps 
as well.


-Dan

 >

- -- Matthias
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFHIggQxbHw2nyi/okRAludAKC14sT7Ff3Ax4L9zpC/fWHx/xyUAwCfSUZ1
WB4q6mV08fa4Yhyx+aUtbEs=
=3yG4
-END PGP SIGNATURE-



--

Amerikanskaya firma Transceptor Technology pristupila k poizvodstu komputerov 
"Personal'ni Sputnik"

Translates as: 'American company Transceptor Technology commenced the production of the 
computer "personal sputnik"'

--Snap, "The Power"

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Wed, 17 Oct 2007, ram wrote:


Sorry I meant "like spamcop" .. I think I must proof-read my own mail
now before Ctrl-Enter :-)


The problem with SpamCop is: the two step reporting process makes things a 
bear to do.  I understand the logic behind it, but once or twice I've 
taken a couple hundred spam emails and spamassassin -r'd it...annoying as 
hell.


I'd like it if they open-sourced their analysis engine so people could use 
it to report spam privately, but I know it's not happening.


-Dan

--

"there is no loyalty in the business, so we stay away from things that piss people 
off"

-The Boss, November 12, 2002

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Rule for TLS verify=OK?

[Dan Mahoney, System Admin]

Hey all,

In looking through my sendmail logs, I've found that some connecting mail 
servers actually are correctly configured with a signed, valid cert from 
one of the major CA's.


Is there a rule that can match this, on sendmail, based on the connecting 
ip on your network edge?


This could be used to complement domain-assurance tools like SPF, DKIM or 
the like, since it not only matches the fact that in order to get one of 
these certs, the domain owner has had to match at least SOME kind of 
legitimacy test (even with the most automated signers).


This is a length I cannot imagine a spammer going to.

Better still, can someone with a better corpus than I confirm some hit/not 
hit ratios here?


-Dan

--

unless is a pr0no book he wont even come close to the bandwidth quota

-Racer-X, concerning DanMahoney.com's web hits.


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Wed, 17 Oct 2007, Alex Woick wrote:


Matthias Leisi schrieb am 17.10.2007 09:46:


Correct. But by setting (in your local.cf or equivalent)

| trusted_networks 204.9.177.18

you are telling SpamAssassin that this relay is not operated by a
spammer and that it should apply all black-/whitelist rules etc. to the
IP address one more hop away. Then, in the context of SpamAssassin, you
regain full control of connection-oriented rules.

That's not fully equivalent to having the actual "spamming connection"
to deal with, but as close as it gets -- if you need it "closer", you
should not use forwarding services.


Good point. I think I start to understand what trusted_network is for and how 
it works. Currently, I have a provider whose MX receives mail for me and 
forwards it to my local mail server. Spam detection improved much when I 
added its IP address to trusted_networks some time ago.


Now, I occasionly get spam to my users.sourceforge.net account, just like Dan 
Mahoney is getting spam to his Livejournal account. Sourceforge is also 
listed with LOW at dnswl and acts as a forwarder to my own mail server.


Since I never get spam from users.sourceforge.net accounts directly but only 
spam sent to my users.sourceforge.net account from random addresses, I 
suppose the Sourceforge mail server is trusted in that way that spam doesn't 
originate from it, and that's the purpose of trusted_network. Just like my 
Provider forwarding mail to me sent from random originators, but never 
produces spam itself.


Sure, but that means each person who is a member of one of these services 
has to:


* Look up their forwarded email address
* Look up the SPF record for that domain
  -or-
* Take a best guess as to the fact that the receiving MX will also be the 
sending.


THEN

* Translate that into trusted networks statements, which are GLOBALLY 
trusted (either per server or per used, but NOT per envelope-recipient) -- 
which is fine for Livejournal or Sourceforge, I guess, I'd imagine their 
MXes are pretty dedicated, but I'm sure there's smaller cases.


But it might help to have some series of dynamic rule...whereby an address 
is DNSWL'd with a special code that lists it as a known relay for certain 
domains, and the trusted_networks logic extends automatically (if the 
relaying domain matches).


Apologies if I've repeated anything already said.

-Dan

--

"there is no loyalty in the business, so we stay away from things that piss people 
off"

-The Boss, November 12, 2002

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Wed, 17 Oct 2007, Matthias Leisi wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Dan Mahoney, System Admin schrieb:


Livejournal's purely a mail forwarding service (i.e. there's no way to
POP/IMAP that account)


As far as I know, there are mails originating from LJ itself (eg
notifications etc)?


No, Livejournal also gives you a [EMAIL PROTECTED] email 
address.  Yes, they do also originate mail (for which we have things like 
SPF (which they do), DomainKeys, DKIM (which they don't, and in fact they 
may have an error for) -- as well as some of the more esoteric things like 
HashCash, GnuPG-signing, etc etc.)



and if they can't effect proper controls on how
mail is sent through them, then they shouldn't be trusted at all.



On my end, I have degrees of control (false MXes, Blacklists,
whitelists, greylists, sender callbacks, etc).  I have no such control
over the LJ MX'es.


Correct. But by setting (in your local.cf or equivalent)

| trusted_networks 204.9.177.18

you are telling SpamAssassin that this relay is not operated by a
spammer and that it should apply all black-/whitelist rules etc. to the
IP address one more hop away. Then, in the context of SpamAssassin, you
regain full control of connection-oriented rules.


interesting point, I suppose.  Kinda breaks the logic of "trusted 
networks".  On the same note, would it not be more useful to, instead of 
using the static trusted_networks configuration, to use the DNSWL to 
determine if that logic should be in play?  Or some kind of database of 
known forwarding services that work in such a manner?



That's not fully equivalent to having the actual "spamming connection"
to deal with, but as close as it gets -- if you need it "closer", you
should not use forwarding services.

Forwarding services are edge case in spamfiltering. Usually, such a
service is itself perfectly trustworthy and not the actual source of
spam, and care must be taken not to unduly penalize these services for
forwarded spam.


The problem therein lies in the fact that LJ notifications (comment 
notifications, friendslist notifications, account verification emails, 
etc) are passed through the exact same MXes as the 
[EMAIL PROTECTED] forwarding service.



I've proposed a reporting plugin on the sa-users list, that allows (both
for yourself, as well as other whitelists) for the list-owner to be
notified with details of high-spam activity (at which point, I guess,
you guys could pass that on to your whitelisted groups, and/or adjust
categories accordingly.


As I've answered before: That's already on the todo list. However, the
main problem is not the plugin per se (technically, that is rather
simple), but identifying trustworthy submitters.


I suppose that depends on what we submit.  If it's something verifiable 
(like, messageID:originating ip:spam level, it's easy).  Just as with 
spamcop, one can choose to omit the message-id so that the spammers cannot 
track who is the spamtrap and listwash, but such reports could be given a 
lower precedence.


--

"You're a nomad billygoat!"

-Juston, July 18th, 2002

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Wed, 17 Oct 2007, Matthias Leisi wrote:


I forwarded over 200 of them earlier today (as an attachment -- total
email size was about one meg).


OK, I now could have a look at them (well, a sample of them, not each of
the > 200 individually).

All samples in that set have been forwarded through your livejournal.com
account, and consequently sent to your server through a dnswl.org-listed
server of livejournal.com (204.9.177.18, see
http://www.dnswl.org/search.pl?s=1409).


Livejournal's purely a mail forwarding service (i.e. there's no way to 
POP/IMAP that account) and if they can't effect proper controls on how 
mail is sent through them, then they shouldn't be trusted at all.


On my end, I have degrees of control (false MXes, Blacklists, whitelists, 
greylists, sender callbacks, etc).  I have no such control over the LJ 
MX'es.


I've proposed a reporting plugin on the sa-users list, that allows (both 
for yourself, as well as other whitelists) for the list-owner to be 
notified with details of high-spam activity (at which point, I guess, you 
guys could pass that on to your whitelisted groups, and/or adjust 
categories accordingly.



Please configure your trusted_networks/internal_networks -- like that,


Like what?  I think I missed what you want me to do.


you'll even get the benefit that all RBL lookups, whitelist_from_rcvd
etc. profit from the correct information.


-Dan

--

"The first annual 5th of July party...have you been invited?"
"It's a Jack Party."
"Okay, so Long Island's been invited."

--Cali and Gushi, 6/23/02


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: [sa-list] Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Wed, 17 Oct 2007, Henrik Krohns wrote:


On Wed, Oct 17, 2007 at 02:48:49AM -0400, Dan Mahoney, System Admin wrote:

On Wed, 17 Oct 2007, Henrik Krohns wrote:


On Tue, Oct 16, 2007 at 06:16:49PM -0400, Dan Mahoney, System Admin wrote:

dnswl.org is either full of it, or not well maintained.

I've gotten at least 20 spams which I see are listed in dnswl.org as "low
trust" (which still merits -1.0).


Umm, did you actually read their pages?

Low Occasional spam occurrences, actively corrected but less promptly.


My point was more along the lines of the fact that there's no method (other
than manual notification) of doing "Active Correction".


Sure, I just felt like being rude also. ;) You say "at least 20 spam", but
since it depends on what your total traffic is, it doesn't mean much.


Actually, that was a typo, of sorts...a more accurate metric would be:

Over 200 hits on that rule, with spams mostly over scores of ten, since 
October 8th, with total spam volume (< 5) about 1000.


Or...roughly 1/5 to 1/4 of all the spam in the past couple weeks.

-Dan

--

"Is Gushi a person or an entity?"
"Yes"

-Bad Karma, August 25th 2001, Ezzi Computers, Quoting himself earler, referring 
to Gushi

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Wed, 17 Oct 2007, Henrik Krohns wrote:


On Tue, Oct 16, 2007 at 06:16:49PM -0400, Dan Mahoney, System Admin wrote:

dnswl.org is either full of it, or not well maintained.

I've gotten at least 20 spams which I see are listed in dnswl.org as "low
trust" (which still merits -1.0).


Umm, did you actually read their pages?

Low Occasional spam occurrences, actively corrected but less promptly.


My point was more along the lines of the fact that there's no method 
(other than manual notification) of doing "Active Correction".  DNSWL is a 
cool idea, but could we also come up with some sort of "reporting" plugin 
(disabled by default, optional) that could notify them when, say, a spam 
of score 15 or above also hits their rules.



If you dont like it, change the scores.


Why not change the system?

-Dan

--

"Why are you wearing TWO grounding straps?"

-John Evans, Ezzi Computers August 23, 2001


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

On Wed, 17 Oct 2007, Matthias Leisi wrote:

I forwarded over 200 of them earlier today (as an attachment -- total 
email size was about one meg).


It would have been from this address.

-Dan



-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Dan Mahoney, System Admin schrieb:

dnswl.org is either full of it, or not well maintained.

I've gotten at least 20 spams which I see are listed in dnswl.org as
"low trust" (which still merits -1.0).


All different IP addresses or some specific network?


Could we maybe please add a feature to spamassassin -r (or some other
hook to the generic whitelisting code) which reports this to the
appropriate whitelist owner?


Can you forward such "false positives" to admins -at- dnswl.org, please?

Thanks,
- -- Matthias, for dnswl.org

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFHFa31xbHw2nyi/okRAueXAJ9v7bs40kAz4UEry7dCKxYqWVnWFwCgjte/
N/CrJ3V4V3X1H+jkGhf/nb8=
=kIQd
-END PGP SIGNATURE-



--

"Oh, and we just recently got an invoice..."
"Congratulations!"

-JC and DM, regarding Unpredictable Billing, 8/18/2001

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

RCVD_IN_DNSWL_LOW

[Dan Mahoney, System Admin]

dnswl.org is either full of it, or not well maintained.

I've gotten at least 20 spams which I see are listed in dnswl.org as "low 
trust" (which still merits -1.0).


Could we maybe please add a feature to spamassassin -r (or some other hook 
to the generic whitelisting code) which reports this to the appropriate 
whitelist owner?


-Dan Mahoney

--

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Advice on MTA blacklist

[Dan Mahoney, System Admin]

On Wed, 10 Oct 2007, David B Funk wrote:


On Tue, 9 Oct 2007, Jo Rhett wrote:


On Oct 9, 2007, at 4:22 PM, Chris Edwards wrote:

Your server then enforces encryption and SMTP-AUTH, and the SSL will
(hopefully) defeat any man-in-the-middle attacks by trans-proxies.


That's exactly the problem I am reporting.  A lot of mail clients
don't enforce SSL connections, so man in the middle is silently
accepted.  Only T-bird can be configured to not work any other way,
TTBOMK.


Jo you didn't read Chris's statement closely. A conscientious mail server
administrator will configure the SERVER to -ONLY- accept encrypted
connections for SMTP-AUTH transactions; the server should enforce
the encryption requirements.
Thus it does not matter what the client wants to do, the server should
not let the client continue the SMTP-AUTH transaction until it has
completed the STARTTLS operation (or in the case of SMTPS, it's
already encrypted).

Back to Skip's question, possibly the easiest way to solve his
problem would be to run two SMTP servers, one on port 25 with full
spam/AV scanning for regular mail traffic, one on ports 587 & 645 with
SMTP-AUTH/TLS for his users' clients to submit messages, on that one
have AV scanning and possibly limited spam scanning.


Assuming sendmail (and we don't make such assumptions), you can specify 
different options per-port, such that you don't need to run "two" mail 
servers.


For example, I have no less than seven virtual daemons configured:

Submission agents on 587 and 2525, which require auth, and have encryption 
optional.  Also listens on 127.1.


A submission agent on 465 (not 645), configured the same way, but with 
encryption explicit.


Standard daemon on port 25 (and yes, it still supports the optional 
encryption).


As a bonus, my own server any port will present a FQDN, signed 
certificate (not self-signed).  I've actually found other servers out 
there in the wild that do the same, with a valid cert -- I've got my 
server configured with the CA root certs so it knows which are "true" 
(this doesn't affect ability to relay or anything, but it's cool to see 
others are doing it).


Of course, all this is wildly off the topic, but hey...

-Dan

--

"And, a special guest, from the future, miss Ria Pischell.  Miss Pischell,
as you all know, is the inventor of the Statiophonic Oxygenetic
Amplifiagraphaphonadelaverberator, and it's pretty hard to imagine life
without one of those.

-Rufus, Bill & Ted's Bogus Journey


Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] RE: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)

[Dan Mahoney, System Admin]

On Wed, 10 Oct 2007, Bret Miller wrote:


sa-update does NOT feed a local blocklist generated by *my*

particular

corpus of spam emails.  Think of it as the RBL equivalent of
sitewide-bayes.  Or think of it as a way of SA saying "when

I get twelve

spams of score 10+ from ip 208.23.118.172...I will feed the
auto-expiring RBL, which *SENDMAIL* works off of, thus keeping my
*SPAMASSASSIN* load lower.


How do you call SpamAssassin?

If whatever calls SpamAssassin in your setup knows what IP the
connecting relay has, it can hopefully also do what you describe
above. SpamAssassin doesn't really need to support this (through
plugins or anything else) for it to be possible (and feasible).


And I did something very similar as well. The problem I found is that you
need a very large white list to avoid blocking big ISPs for a sudden flood
of spam. I ended up rejecting legitimate email far too often from the
temporary block. I still like the idea and would do it in a second if I
could change the 5xx reject to a 4xx try later type of block. But I can't'
without switching to a different MTA.


milter-greylist lets me do this (reject 4XX based on a DNSBL).  I've found 
it to be highly customizable, if not a bit of a memory pig.


On the other hand, if there is a "big ISP" who is sending me spam...should 
they not be blocked, anyway?


-Dan

--

"Long live little fat girls!"

-Recent Taco Bell Ad Slogan, Literally Translated.  (Viva Gorditas)

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

A compound bounce/(spf/dk/dkim) rule I'd like to see.

[Dan Mahoney, System Admin]

In pseudocode...

IF (message is a recognizable bounce || message is from <>)...

AND (we can guess the domain being sent to (can't trust the "to" header, 
but maybe the X-Envelope-To or some MTA token?)


AND the domain being sent TO supports SPF and/or DKIM...(i.e. implying a 
misdirected bounce)


Score a compound rule hit.

My logic here is that I would eventually like to compile an rfc-ignorant 
list of the senders of such bounces, and aid them in not SENDING such 
bounce messages, or at the very least, set up a ruleset in the future to 
block bounces from them, based on a low signal/noise ratio.


I am not trying at all to claim that this should be something SCORABLE, 
immediately: I don't think SA's detection of legitimate bounce messages 
versus illegitmate bounce messages is good enough (please feel free to 
tell me differently).


-Dan Mahoney

--

"GO HOME AND COOK!!!"

Donielle Cocossa, Taco Bell, 2:30 AM

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)

[Dan Mahoney, System Admin]

On Tue, 9 Oct 2007, Steven Kurylo wrote:

Parsing the SA logs would be easy, but the connecting IP isn't listed 
there. 
As I mentioned, I'm parsing exim's logs.  It contains the spam score and the 
IP address.


Oh, that's true enough.  I was musing on parsing my own logfiles as 
opposed to plugins.  Not enough info since I'm rejecting at the procmail 
level, not the MTA (sendmail) level.


-Dan

--

"Ca. Tas. Tro. Phy."

-John Smedley, March 28th 1998, 3AM

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)

[Dan Mahoney, System Admin]

On Tue, 9 Oct 2007, Steven Kurylo wrote:

 Or think of it as a way of SA saying "when I get twelve spams of score 10+ 
from ip 208.23.118.172...I will feed the auto-expiring RBL, which 
*SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load lower.  Thus a 
spam deluge via a dictionary attack that may take hours is mitigated in the 
course of X number of mails. 
I already do something similar, but I haven't bothered to take it quite that 
far yet.


I use fail2ban to parse my exim logs.  If an IP address hits more than 5 
invalid accounts in 5 minutes, the IP is banned (fail2ban uses iptables) for 
24 hours.  As well if an IP address, which is listed on spamhause, hits me 
more than twice in 5 minutes it is banned for 24 hours.  Granted neither of 
these cases usually end up getting messages as far as spamassassin.


I've managed to drastically reduce the amount of simultaneous connections 
using this method; which was overloading the server.  The next step would be 
to add the "when I get twelve spams of score 10+ from [...]" parsing.  Though 
I hadn't thought of trying my hand at a SA plugin, I may do that.


Parsing the SA logs would be easy, but the connecting IP isn't listed 
there.


-Dan

--

"Man, this is such a trip"

-Dan Mahoney, October 25, 1997

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Re: [sa-list] Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,

[Dan Mahoney, System Admin]

On Mon, 8 Oct 2007, Rob McEwen wrote:

Therefore, I recommend that you re-think your choices here! Don't let your 
quest for "guaranteed long-term perfection" keep you from making 
**substantial** progress today!


Rob,

Then help rally the SA team to include those RBLs that you mentioned in 
the stock config.


Also, rally them to update the documentation on the wiki on how to 
configure SA for third-party DNSBL's, because it 
blows (and refers to years-old versions of SA).  Yes, I know the point of 
a wiki is that ANYONE can update it, but I'm not about to update it with 
information I don't understand for certain.


((Q: This documentation doesn't seem to cover how to configure 
dns-blocklists. It says "Support for these is built-in" but I can't 
believe that all free BL's is called each time a mail is beeing checked. 
There must be a way to configure which to use.


A: You're right. You might look at the [WWW] Mail::SpamAssassin::Conf 
documentation page which I admit doesn't really say how to configure which 
DNSBL to use, or the rules file [WWW] 20_dnsbl_tests.cf, for internal 
details, but no clear examples of how to configure the inclusion of 
various DNSBLs either. For the latest list of DNSBLs you want to be using 
SpamAssassin version 2.63 or 3.0.0-pre2, for the same reason that you 
wouldn't use an out-of-date virus scanner, but that also doesn't really 
have anything to do with the question.))


Finally, rally them to pay attention to the topic I'm proposing here, 
which is: allow users to run their own RBL + feeder so that they can 
auto-rbl and floodgate themselves (and yes, it allows me to combine your 
corpus, plus my corpus, plus HIS corpus) in a scoring config, which is 
FUN...or it lets you say, quite simply "SA said you sent too much spam, 
now sendmail won't listen for X hours per spam run".




While I've had a long history of getting decent responses from the 
developers on this list some of the time -- nobody has managed to answer 
the questions I've asked in the previous thread:


* can we do something with the ironport headers

* can we do something with the SPF softfail which my MTA registered but SA 
didn't (and why didn't it?)


* can we do something with the X-Originating-IP: 127:1 (is it a legit 
header, or is it there to evade filters?)


* can we fix something about the DKIM_POLICY_SIGNSOME,

* and after I changed the topic: Can we get a plugin that lets us feed our 
own blocklists, currently I get dictionary floods that are enough to 
overload SA (even right now).


and many is the time I've just sent an email out to this list on a given 
topic, seen a lack of useful answer, and shrugged it off.




--

"Check it out, it's just like Christmas.  Except it sucks."

-Jason Seguerra, 3/2/05

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---

Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)

[Dan Mahoney, System Admin]

On Mon, 8 Oct 2007, Matus UHLAR - fantomas wrote:


On Sat, 6 Oct 2007, Rob McEwen wrote:

FWIW... that IP, 220.226.197.15, is currently listed on four spam
blacklists ("RBLs"):

1) uceprotect
2) no-more-funn
3) psbl
4) ivmSIP.com (mine)


On 07.10.07 05:55, Dan Mahoney, System Admin wrote:

My problem is: blocklists come and go, and some blocklists, when they
"go", do things like "hang up because they're being flooded, thus slowing
my mail processes" or "flag all mail as spam" or "hand out stale data that
hasn't changed at all in months/years".


That's what sa-update is for.


Personally, I'd like it if SA came with a blocklist-feeder tool, where
upon, say, two auto-learns, a blocklist (or SQL database) could be fed.


Why do you think people would use them, when they don't already use
sa-update which does the same?


sa-update does NOT feed a local blocklist generated by *my* particular 
corpus of spam emails.  Think of it as the RBL equivalent of 
sitewide-bayes.  Or think of it as a way of SA saying "when I get twelve 
spams of score 10+ from ip 208.23.118.172...I will feed the auto-expiring 
RBL, which *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load 
lower.  Thus a spam deluge via a dictionary attack that may take hours is 
mitigated in the course of X number of mails.


Which is what I was (off-topicly) asking for,

-Dan

--

"I'll commit ritual suicide before I whore myself out to Disney."

--Emi Bryant
  April 26, 2004
  On the animation industry

Dan Mahoney
Techie,  Sysadmin,  WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144   AIM: LarpGM
Site:  http://www.gushi.org
---