Re: SA on a backup MX
The reason for read-only is that I am using PostgreSQL with hot standby streaming replication. The main MX server will use the master PostgreSQL server and the backup MX will use its own local PostgreSQL slave server. The slave server of a PostgreSQL hot standby pair is only available in read-only for queries, this is how PostgreSQL works for streaming replication as such the SQL queries on my slave server (the backup MX server) must be only SELECTs and such but no INSERTs. Thanks for pointing out the use_learner parameter, which I will definitely want to 1. I guess I will have to test and see if this is doable. In the worst case the INSERT queries would fail but maybe SA can handle that properly and still work or else I simply disable Bayes on the backup MX. On Wednesday, July 9, 2014 12:40 AM, RW wrote: On Tue, 8 Jul 2014 12:48:58 -0700 ML mail wrote: > Hello, > > I would like to run SpamAssassin on my backup MX as well in order to > avoid spam and backscatter. > ... > Now I have all my user configuration including SpamAssassin Bayes and > AWL stored in PostgreSQL and my backup MX will have a read-only > replica of that PostgreSQL database. So my plan would be to have SA > on the backup MX to use the Bayes+AWL data from the PostgreSQL in > read-only mode meaning that it would not write any new Bayes+AWL info > but just read it. So my question here is: is it possible to tell SA > through its config to use Bayes+AWL data in read-only mode? I can see why you'd want to use a copy, but why does it matter that it's not written to? On Tue, 08 Jul 2014 22:53:47 +0200 Axb wrote: > use_bayes 1 > use_bayes_rules 1 > use_learner 0 use_learner ( 0 | 1 ) (default: 1) Whether to use any machine-learning classifiers with SpamAssassin, such as the default 'BAYES_*' rules. Setting this to 0 will disable use of any and all human-trained classifiers. > bayes_auto_learn 0 > > that will use BAyes in read only mode. Aside from use_learner, it wouldn't turn-off atime updates. > Afaik, this will not control AWL - maybe disable that on the backup MX >
Re: help with a sintax rule appreciated
It seems that my rule using "Received" instead of "From" did the trick, the rule is working now. Thanks! Regards, Sergio On Tue, Jul 8, 2014 at 10:43 PM, Sergio wrote: > Hi all, > long time not bother you with my doubts, sorry if this has been posted > before and your help is appreciated. > > I have been hammered with a lot of spam that comes like this in the from: > > Example list: > bounces+974322-5ea9-user=domain@sendgrid.info > harprefinancelender-user=domain@formmobily.com > fldelitylife-user=domain@bajarvideos.net > whoswho-user=domain@bayangpinoy.com > garanciacambogia-user=domain@mymedcases.com > oceansbounty-user=domain@myivr.com > amazoncoupons-user=domain@lastawhdak.com > > These are the headers from amazoncoupons-user=domain@lastawhdak.com: > > Message Headers:Received: from tech.lastawhdak.com ([23.254.130.183]:5780) > by server.domain.com with esmtp (Exim 4.82) > (envelope-from ) > id 1X4VcB-004Aw1-EW > for u...@domain.com; Tue, 08 Jul 2014 08:39:23 -0500 > DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dkim; d= > LASTAWHDAK.COM; > h=Mime-Version:Content-Type:Message-Id:Date:From:To:Subject; i= > amazoncoup...@lastawhdak.com; > bh=VixSKqSnPl10ughWH0h+w7BHHVg=; > > b=fSr1ulVa9jHHrl9uO6cwHVfcn/7XO1trKlZqYwyWjhB0QF19t7mkqx8GeF9j6eA6N7gAqTL+EyXA > > 5ZIEPBli4fsSqced4ZwhNnc3SCFzGk+V6dqZCbVYsfUcO9hxFybv/YsHq00aiU7tbxbagvX96c/W > B7/2YgktkeAXy/D6aos= > Received: by tech.LASTAWHDAK.COM id hnfq3o0001gp for ; > Tue, 8 Jul 2014 13:18:07 + (envelope-from domain@lastawhdak.com>) > Mime-Version: 1.0 > Content-Type: multipart/alternative; > boundary="becf-9486-0840-97dd-1672-cc2d-bab3-5594" > Message-Id: < > 49553babd2cc2761dd7904806849fceb.10158442971ce...@lastawhdak.com> > Date: Tue, 8 Jul 2014 13:18:07 + > *From: *Amazon Coupons > To: u...@domain.comt > Subject: > =?utf-8?B?Q29uZ3JhdHVsYXRpb25zIG9uIHlvdXIgQW1hem9uIFN1cnZleSBSZXdhcmQ=?= > *From:*amazoncoupons-user=domain@lastawhdak.com > > I have created the following rule, because I thought that I could block > any "From" that includes a domain name with the extensions .com or .net or > .org or .biz before @ > > headerBLACKLIST_REGEXFrom:address =~ /\=.*\.(com|net|org|biz)\@/i > score BLACKLIST_REGEX5 > > But it is not working, the rule is not catching any of the "From" from > above example list. > > I have also tried but with no luck: > > headerBLACKLIST_REGEXFrom =~ /\=.*\.(com|net|org|biz)\@/i > score BLACKLIST_REGEX5 > > So, my question is, Do I have to go and better check for the "Received" ? > Something like: > > headerBLACKLIST_REGEXReceived =~ /\\=.*.(com|net|org|biz)\@/i > score BLACKLIST_REGEX5 > > Or if you have a better way on doing this, your advice is appreciated. > > Best Regards, > > Sergio >
help with a sintax rule appreciated
Hi all, long time not bother you with my doubts, sorry if this has been posted before and your help is appreciated. I have been hammered with a lot of spam that comes like this in the from: Example list: bounces+974322-5ea9-user=domain@sendgrid.info harprefinancelender-user=domain@formmobily.com fldelitylife-user=domain@bajarvideos.net whoswho-user=domain@bayangpinoy.com garanciacambogia-user=domain@mymedcases.com oceansbounty-user=domain@myivr.com amazoncoupons-user=domain@lastawhdak.com These are the headers from amazoncoupons-user=domain@lastawhdak.com: Message Headers:Received: from tech.lastawhdak.com ([23.254.130.183]:5780) by server.domain.com with esmtp (Exim 4.82) (envelope-from ) id 1X4VcB-004Aw1-EW for u...@domain.com; Tue, 08 Jul 2014 08:39:23 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dkim; d=LASTAWHDAK.COM ; h=Mime-Version:Content-Type:Message-Id:Date:From:To:Subject; i= amazoncoup...@lastawhdak.com; bh=VixSKqSnPl10ughWH0h+w7BHHVg=; b=fSr1ulVa9jHHrl9uO6cwHVfcn/7XO1trKlZqYwyWjhB0QF19t7mkqx8GeF9j6eA6N7gAqTL+EyXA 5ZIEPBli4fsSqced4ZwhNnc3SCFzGk+V6dqZCbVYsfUcO9hxFybv/YsHq00aiU7tbxbagvX96c/W B7/2YgktkeAXy/D6aos= Received: by tech.LASTAWHDAK.COM id hnfq3o0001gp for ; Tue, 8 Jul 2014 13:18:07 + (envelope-from ) Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="becf-9486-0840-97dd-1672-cc2d-bab3-5594" Message-Id: < 49553babd2cc2761dd7904806849fceb.10158442971ce...@lastawhdak.com> Date: Tue, 8 Jul 2014 13:18:07 + *From: *Amazon Coupons To: u...@domain.comt Subject: =?utf-8?B?Q29uZ3JhdHVsYXRpb25zIG9uIHlvdXIgQW1hem9uIFN1cnZleSBSZXdhcmQ=?= *From:*amazoncoupons-user=domain@lastawhdak.com I have created the following rule, because I thought that I could block any "From" that includes a domain name with the extensions .com or .net or .org or .biz before @ headerBLACKLIST_REGEXFrom:address =~ /\=.*\.(com|net|org|biz)\@/i score BLACKLIST_REGEX5 But it is not working, the rule is not catching any of the "From" from above example list. I have also tried but with no luck: headerBLACKLIST_REGEXFrom =~ /\=.*\.(com|net|org|biz)\@/i score BLACKLIST_REGEX5 So, my question is, Do I have to go and better check for the "Received" ? Something like: headerBLACKLIST_REGEXReceived =~ /\\=.*.(com|net|org|biz)\@/i score BLACKLIST_REGEX5 Or if you have a better way on doing this, your advice is appreciated. Best Regards, Sergio
Re: Ideas sought for blocking new variant of cryptolocker
On Tue, 2014-07-08 at 22:41 -0400, David F. Skoll wrote:
> On Tue, 08 Jul 2014 21:03:35 -0400, Kevin A. McGrail wrote:
>
> > So this sounds like you are searching the entire email for this
> > string which just sounds inefficient especially if they use some big
> > attachments.
>
> It's not too bad because the regex is simple.
The regex is dead simple, indeed. Even as full rule, it should perform
better than most complex body rules, since it is a very short pattern
with no backtracking.
For the benefit of easier discussion, the regex /\n\nTV[opqr]/ can be
translated to "an empty line, that hopefully matches the gap at the
beginning of a MIME attachment, with some certain base64 encoded bytes".
That last part should better be referred to as a base64 encoded DOS MZ
executable -- or simply .exe file. A base64 encoded /^TV[opqr]/ is
identical to /^MZ/ decoded.
Which of course means, that applies to more than just a new variant of
some malware. Though generally targeting any MS executable dropped as a
mail attachment is not a bad idea...
> The reason I did it with a SpamAssassin rule is that we have ways to
> push out SpamAssassin rules easily to our customers, but not so much
> code changes. :)
>
> The rule hits on surprisingly few messages (only two out of a couple of
> million so far), but it's not terribly accurate: One false-positive caused
> by a stupid base-64 encoder that leaves extra newlines between lines,
That should be solvable by adding stricter border to be beginning of the
regex. Since I don't know what your samples use in particular, here is a
quick first idea.
/(: base64|")\n\nTV[opqr]/
The idea is to base the regex at a char not part of the base64 set,
focusing on the Content-Transfer-Encoding and Content-Disposition MIME
headers. Whatever is the last one used.
> and one sort-of-false-positive that was a DLL renamed to .DAT to sneak
> past filename extension blocks, but wasn't otherwise malicious.
If you deliberately try to sneak past sensible security measures, you
should not be surprised to be blocked. The attempt by an honest user to
disguise any $file (he did it on purpose, so he knows there's issues
with that) is in no way better than a dis-honest user disguising a file.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Ideas sought for blocking new variant of cryptolocker
On Tue, 08 Jul 2014 21:03:35 -0400
"Kevin A. McGrail" wrote:
> So this sounds like you are searching the entire email for this
> string which just sounds inefficient especially if they use some big
> attachments.
It's not too bad because the regex is simple.
> Since I'm guessing you are using MD, wouldn't something like this be
> better? Untested, but based on some code for looking for rar files
> masquerading as zip files:
Yes, looking at file signatures (à la "file(1)") would be more robust.
> if (uc($header) eq "MZ") {
You don't want the uc(); that could lead to false-positives, but yes,
the idea is correct.
The reason I did it with a SpamAssassin rule is that we have ways to
push out SpamAssassin rules easily to our customers, but not so much
code changes. :)
The rule hits on surprisingly few messages (only two out of a couple of
million so far), but it's not terribly accurate: One false-positive caused
by a stupid base-64 encoder that leaves extra newlines between lines,
and one sort-of-false-positive that was a DLL renamed to .DAT to sneak
past filename extension blocks, but wasn't otherwise malicious.
Regards,
David.
Re: Ideas sought for blocking new variant of cryptolocker
On 7/7/2014 5:34 PM, David F. Skoll wrote:
Replying to myself...
full MSDOGEXE /\n\nTV[opqr]/
Seems to work. :)
So this sounds like you are searching the entire email for this string
which just sounds inefficient especially if they use some big attachments.
Since I'm guessing you are using MD, wouldn't something like this be
better? Untested, but based on some code for looking for rar files
masquerading as zip files:
sub filter_bad_filename {
# Check for hidden executables
unless (re_match($entity, '\.exe$') ) {
my $bh = $entity->bodyhandle();
if (defined($bh)) {
my $path = $bh->path();
if (defined($path)) {
if(&check_for_exe_signature($path,
$entity->head->recommended_filename())) {
action_add_header("X-Suspected-Hidden-EXE-Attachment","True");
}
}
}
}
...
}
sub check_for_exe_signature {
my ($path, $recommended_filename) = @_;
my ($filehandle, $header);
#OPEN THE FILE, GRAB THE HEADER AND TEST
$filehandle = new IO::File("< $path");
if (defined $filehandle) {
read($filehandle,$header,2);
close ($filehandle);
if (uc($header) eq "MZ") {
return 1;
}
return 0;
}
NOTE: I would actually use an action_add_header_immediately routine that
modifies the message that's passed to SA
But then add rules like this that use the header and other items to make
a poison pill score possible
body__KAM_CRYPTO1 /open the attached document/i
body__KAM_CRYPTO2 /add an extension/i
header __KAM_CRYPTO3 X-Suspected-Hidden-EXE-Attachment =~ /True/
meta KAM_CRYPTO (__KAM_CRYPTO1 + __KAM_CRYPTO2 + __KAM_CRYPTO3 >= 3)
describe KAM_CRYPTO Likely CryptoLocker Spam with Hidden EXE Malware
score KAM_CRYPTO 8.0
regards,
KAM
Re: SA on a backup MX
On Tue, 8 Jul 2014 12:48:58 -0700 ML mail wrote: > Hello, > > I would like to run SpamAssassin on my backup MX as well in order to > avoid spam and backscatter. > ... > Now I have all my user configuration including SpamAssassin Bayes and > AWL stored in PostgreSQL and my backup MX will have a read-only > replica of that PostgreSQL database. So my plan would be to have SA > on the backup MX to use the Bayes+AWL data from the PostgreSQL in > read-only mode meaning that it would not write any new Bayes+AWL info > but just read it. So my question here is: is it possible to tell SA > through its config to use Bayes+AWL data in read-only mode? I can see why you'd want to use a copy, but why does it matter that it's not written to? On Tue, 08 Jul 2014 22:53:47 +0200 Axb wrote: > use_bayes 1 > use_bayes_rules 1 > use_learner 0 use_learner ( 0 | 1 ) (default: 1) Whether to use any machine-learning classifiers with SpamAssassin, such as the default 'BAYES_*' rules. Setting this to 0 will disable use of any and all human-trained classifiers. > bayes_auto_learn 0 > > that will use BAyes in read only mode. Aside from use_learner, it wouldn't turn-off atime updates. > Afaik, this will not control AWL - maybe disable that on the backup MX >
Re: SA on a backup MX
On 07/08/2014 09:48 PM, ML mail wrote: Hello, I would like to run SpamAssassin on my backup MX as well in order to avoid spam and backscatter. My backup MX will have a similar setup and configuration as my main MX, that would be Postfix+amavisd-new+ClamAV+SpamAssassin. Now I have all my user configuration including SpamAssassin Bayes and AWL stored in PostgreSQL and my backup MX will have a read-only replica of that PostgreSQL database. So my plan would be to have SA on the backup MX to use the Bayes+AWL data from the PostgreSQL in read-only mode meaning that it would not write any new Bayes+AWL info but just read it. So my question here is: is it possible to tell SA through its config to use Bayes+AWL data in read-only mode? If that's not possible then I was thinking I could still use SA on my backup MX but I would then disable Bayes+AWL. What do you think? on your backup MX local.cf set: use_bayes 1 use_bayes_rules 1 use_learner 0 bayes_auto_learn 0 that will use BAyes in read only mode. Afaik, this will not control AWL - maybe disable that on the backup MX
Re: SA on a backup MX
My backup MX will be on located on another continent this is the main reason (latency) why I would like a local read-only replica of my database on the backup MX server. On Tuesday, July 8, 2014 9:57 PM, Antony Stone wrote: On Tuesday 08 July 2014 at 21:48:58, ML mail wrote: > Hello, > > I would like to run SpamAssassin on my backup MX as well in order to avoid > spam and backscatter. My backup MX will have a similar setup and > configuration as my main MX, that would be > Postfix+amavisd-new+ClamAV+SpamAssassin. > > Now I have all my user configuration including SpamAssassin Bayes and AWL > stored in PostgreSQL and my backup MX will have a read-only replica of > that PostgreSQL database. So my plan would be to have SA on the backup MX > to use the Bayes+AWL data from the PostgreSQL in read-only mode meaning > that it would not write any new Bayes+AWL info but just read it. So my > question here is: is it possible to tell SA through its config to use > Bayes+AWL data in read-only mode? > > If that's not possible then I was thinking I could still use SA on my > backup MX but I would then disable Bayes+AWL. What do you think? Why not provide r/w access to PostgreSQL on the main server from the MX machine? They both presumably have static IPs, so securing that connection shouldn't be a problem? Antony. -- The idea that Bill Gates appeared like a knight in shining armour to lead all customers out of a mire of technological chaos neatly ignores the fact that it was he who, by peddling second-rate technology, led them into it in the first place. - Douglas Adams in The Guardian, 25th August 1995 Please reply to the list; please *don't* CC me.
SA on a backup MX
Hello, I would like to run SpamAssassin on my backup MX as well in order to avoid spam and backscatter. My backup MX will have a similar setup and configuration as my main MX, that would be Postfix+amavisd-new+ClamAV+SpamAssassin. Now I have all my user configuration including SpamAssassin Bayes and AWL stored in PostgreSQL and my backup MX will have a read-only replica of that PostgreSQL database. So my plan would be to have SA on the backup MX to use the Bayes+AWL data from the PostgreSQL in read-only mode meaning that it would not write any new Bayes+AWL info but just read it. So my question here is: is it possible to tell SA through its config to use Bayes+AWL data in read-only mode? If that's not possible then I was thinking I could still use SA on my backup MX but I would then disable Bayes+AWL. What do you think? Regards ML
Re: I was wrong - Bayes filter not quite right
On Tue, 8 Jul 2014, motty cruz wrote: Hi Bruce, I was having similar issues, can you do su - vscan and restart amavisd service? user "vscan" != user "amavis". On Tue, Jul 8, 2014 at 8:54 AM, Bruce Sackett wrote: So I was able to get the Bayes filter working under spamassassin -D as the ‘amavis’ user (which should be correct), but when I run a live message through the mail server, it is apparently NOT using Bayes. Any ideas on where/how I could troubleshoot that specifically? Which user is amavisd running under? If it's indeed running under user "vscan" then training and testing Bayes as user "amavis" probably won't help any. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Gun Control laws cannot reduce violent crime, because gun control laws focus obsessively on a tool a criminal might use to commit a crime rather than the criminal himself and his act of violence. --- 12 days until the 45th anniversary of Apollo 11 landing on the Moon
Re: I was wrong - Bayes filter not quite right
Hi Bruce, I was having similar issues, can you do su - vscan and restart amavisd service? make sure you vscan user can read /var/amavisd. also, you must feed about 200+ ham/spam emails to activate bayes. Thanks, On Tue, Jul 8, 2014 at 8:54 AM, Bruce Sackett wrote: > So I was able to get the Bayes filter working under spamassassin -D as the > ‘amavis’ user (which should be correct), but when I run a live message > through the mail server, it is apparently NOT using Bayes. Any ideas on > where/how I could troubleshoot that specifically? > > --Bruce Sackett – e: br...@oecnw.com - w: www.oecnw.com - p: 541.342.3325 - > tw: @OECTECH - fb: www.facebook.com/oecnw > >
I was wrong - Bayes filter not quite right
So I was able to get the Bayes filter working under spamassassin -D as the ‘amavis’ user (which should be correct), but when I run a live message through the mail server, it is apparently NOT using Bayes. Any ideas on where/how I could troubleshoot that specifically? --Bruce Sackett – e: br...@oecnw.com - w: www.oecnw.com - p: 541.342.3325 - tw: @OECTECH - fb: www.facebook.com/oecnw
