Individual pre learning - Bayes in SQL
Hello I have Bayes in SQL for each users (emails) on test server. SA is trigger by /usr/local/bin/spamc -U /var/run/spamd/spamd.socket -u $local_part@$domain I looked at the results in database and have doubt. select * from bayes_vars; id | username| spam_count | ham_count | token_count 1 | a@x.x | 1 | 8 | 3937 13 | t@x.x | 0 | 1 | 356 15 | i@x.x | 0 | 1 | 360 Column skiped: last_expire | last_atime_delta | last_expire_reduce | oldest_token_age | newest_token_age | account id 1 is oldest created few days ago. Trained myself. 13 and 15 is new account received only one email: Why both account have token_count ~ 360 ? Not 1? whether these tokens are inherited? sa-learn -ut@x.x --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 0 0 non-token data: nspam 0.000 0 1 0 non-token data: nham 0.000 0356 0 non-token data: ntokens 0.000 0 1406154984 0 non-token data: oldest atime 0.000 0 1406154984 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count for id: 15 sa-learn -ui@x.x --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 0 0 non-token data: nspam 0.000 0 1 0 non-token data: nham 0.000 0360 0 non-token data: ntokens 0.000 0 1406159567 0 non-token data: oldest atime 0.000 0 1406159567 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count Probably I should make --sync. Second question: whether SA draws attention to mail's header TO, CC etc.? I want make pre learning. Collect dozens of super spam mails from different accounts and by script learn all accounts in loop sa-learn --spam --username=$account /spam/dir/* Mail addressed to another person will not be a problem in learning process? Best Regards.
Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
Howdy - I have two VMs at Digital Ocean, one on the east coast, one on the west. I'm running Sendmail-8.14.8-2.fc20.x86_64. I have several DNSBLs listed: FEATURE(`dnsbl',`in.dnsbl.org ')dnl FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl FEATURE(`dnsbl',`cbl.abuseat.org')dnl FEATURE(`dnsbl',`dul.dnsbl.sorbs.net')dnl Unfortunately, my home network is attached to a cable provider which shows up in dul.dnsbl.sorbs.net. Can I whitelist my IP address so that I can send mail through my mail servers? Right now, it gets rejected. Yeah, I know, I can always use my ISP's smtp server, I guess. But that kind of sucks. I would rather use mine. Purely a pride thing, I know. Thomas
Re: Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
On 07/24/2014 09:58 AM, Thomas Cameron wrote: Howdy - I have two VMs at Digital Ocean, one on the east coast, one on the west. I'm running Sendmail-8.14.8-2.fc20.x86_64. I have several DNSBLs listed: FEATURE(`dnsbl',`in.dnsbl.org ')dnl FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl FEATURE(`dnsbl',`cbl.abuseat.org')dnl FEATURE(`dnsbl',`dul.dnsbl.sorbs.net')dnl Unfortunately, my home network is attached to a cable provider which shows up in dul.dnsbl.sorbs.net. Can I whitelist my IP address so that I can send mail through my mail servers? Right now, it gets rejected. Yeah, I know, I can always use my ISP's smtp server, I guess. But that kind of sucks. I would rather use mine. Purely a pride thing, I know. Thomas Disregard. I was way over thinking it. A quick line in /etc/mail/access fixed it. Sorry for the noise. TC
Re: Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
s/somewhat// # ;)
On Thu, 2014-07-24 at 09:58 -0500, Thomas Cameron wrote:
I'm running Sendmail-8.14.8-2.fc20.x86_64. I have several DNSBLs listed:
FEATURE(`dnsbl',`in.dnsbl.org ')dnl
FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl
FEATURE(`dnsbl',`cbl.abuseat.org')dnl
FEATURE(`dnsbl',`dul.dnsbl.sorbs.net')dnl
Unfortunately, my home network is attached to a cable provider which
shows up in dul.dnsbl.sorbs.net.
Can I whitelist my IP address so that I can send mail through my mail
servers? Right now, it gets rejected.
Not sendmail specific, and I wouldn't know the exact sendmail conf for
this anyway, but:
You should use authentication, and configure your SMTP to accept
authenticated connections before rejecting based on DNSBLs.
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Individual pre learning - Bayes in SQL
On Thu, 24 Jul 2014 09:32:35 +0200 Adi wrote: Hello 13 and 15 is new account received only one email: Why both account have token_count ~ 360 ? Not 1? whether these tokens are inherited? A token is a word or some piece of derived data. I just means that that email contained 360 of them. Second question: whether SA draws attention to mail's header TO, CC etc.? Yes. I want make pre learning. Collect dozens of super spam mails from different accounts and by script learn all accounts in loop sa-learn --spam --username=$account /spam/dir/* Mail addressed to another person will not be a problem in learning process? Probably not. It wont make any difference in most cases, but if one of those addresses is in To/Cc , and the recipient hasn't yet trained it as ham, there's a small chance it might.
Re: Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
On Thu, 24 Jul 2014, Thomas Cameron wrote:
Howdy -
I have two VMs at Digital Ocean, one on the east coast, one on the west.
I'm running Sendmail-8.14.8-2.fc20.x86_64. I have several DNSBLs listed:
FEATURE(`dnsbl',`in.dnsbl.org ')dnl
FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl
FEATURE(`dnsbl',`cbl.abuseat.org')dnl
FEATURE(`dnsbl',`dul.dnsbl.sorbs.net')dnl
Unfortunately, my home network is attached to a cable provider which
shows up in dul.dnsbl.sorbs.net.
Can I whitelist my IP address so that I can send mail through my mail
servers? Right now, it gets rejected.
Yeah, I know, I can always use my ISP's smtp server, I guess. But that
kind of sucks. I would rather use mine. Purely a pride thing, I know.
Thomas
Thomas.
Do you have 'MSA' port enabled for your sendmail? (IE port 567) and
SMTP-AUTH? Then just skip the dnsbl checks for auth'ed mail submissions.
You could whitelist your client IP address in your 'access' file but
what happens when that address changes? (I assume your ISP gives you
a DHCP address).
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
On 07/24/2014 10:37 AM, Dave Funk wrote: Thomas. Do you have 'MSA' port enabled for your sendmail? (IE port 567) and SMTP-AUTH? Then just skip the dnsbl checks for auth'ed mail submissions. You could whitelist your client IP address in your 'access' file but what happens when that address changes? (I assume your ISP gives you a DHCP address). Hi, Dave - I actually have SMTP AUTH enabled, and it was working fine (albeit on port 25 with STARTTLS) until I added the DNSBL. Even connecting from my MUA (Thunderbird on Linux) to port 587 on my server, I get this (identifying info changed) in the log file if I enable the DNSBL: Jul 24 11:57:36 YYY dovecot: imap-login: Login: user=thomas.cameron, method=PLAIN, rip=1.2.3.4, lip=4.5.6.7 mpid=469, TLS, session=GG70g/L+xwBGw8l/ Jul 24 11:57:59 YYY sendmail[472]: ruleset=check_relay, arg1=cpe-.austin.res.rr.com, arg2=127.0.0.10, relay=cpe-.austin.res.rr.com [1.2.3.4], reject=550 5.7.1 Rejected: 68.203.17.142 listed at dul.dnsbl.sorbs.net TC
Re: Adding header depending on _SCORE_ and settings UserPref
Have you tried the amavis/sa-exim way? On 20.07.14 15:22, Adi wrote: Not yet. if I'm not mistaken amavis is working in delivery time not SMTP time? AFAIK, it can run in pre-queue mode too. Surely when running as milter. and IIRC it runs at SMTP time by default. Some my configuration is based on: http://www.janoszen.com/2013/07/24/filtering-spam-with-exim-and-spamassassin-properly/ I looked onto that and did not like it. Mostly because it does not run at the SMTP time... yes , but I SMTP time I can't analize spam according user prefs - for multi recipient :/ yes, that's feature of SMTP. That's why I advised you multiple times using sane defaults in such case. you can do two checks, one at SMTP time (with sane defaults), and second (user-personalized) at delivery time. single recipient e-mails can be scanned with user preferences at SMTP time and you can skip the latter step for them. Of course, spam rejected at SMTP time won't be scanned with per-user settings... Yes this is concept but how can I detect is it single or multi recipient? AND if it is multi recipient I can not reject mail in SMTP time. oh yes, you can - with sane defaults, e.g. when final score is over 10 (or whatever you configure) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The box said 'Requires Windows 95 or better', so I bought a Macintosh.
Re: Adding header depending on _SCORE_ and settings UserPref
Hello Thanks for answer. Not yet. if I'm not mistaken amavis is working in delivery time not SMTP time? AFAIK, it can run in pre-queue mode too. Surely when running as milter. and IIRC it runs at SMTP time by default. For now I'll leave my solution with exec spamc in Exim's transport (by pipe - second process of Exim). It's working good. I will back to the topic if have more time. another option to solve this in Exim may be PRDR (Per-Recipient-Data-Response). Become stable in new 4.83 (in Freebsd ports). http://www.eric-a-hall.com/specs/draft-hall-prdr-00.txt I must read this and test it. Yes this is concept but how can I detect is it single or multi recipient? AND if it is multi recipient I can not reject mail in SMTP time. oh yes, you can - with sane defaults, e.g. when final score is over 10 (or whatever you configure) OK. But what if some user want get SPAM, could become unhappy if I blocked him this opportunity :) Best Regards.
Re: Filters Don't Seem to Be Learning [SOLVED]
On Wed, 2014-07-23 at 14:34 -0700, Asai wrote: The mail server is running as a different user than amavis, so I ran this under the amavis user: 0.000 0 3 0 non-token data: bayes db version 0.000 0624 0 non-token data: nspam 0.000 0 11919 0 non-token data: nham While that seems a little biased, that's sufficient training with default thresholds of 200 each. 0.000 0 120783 0 non-token data: ntokens 0.000 0 1405862394 0 non-token data: oldest atime 0.000 0 1406151128 0 non-token data: newest atime Good, just a few hours ago. sa-learn is running under amavis as well. Yet, there are no BAYES_xx rules hit, so the Bayesian classifier somehow has been disabled. Check your SA conf with a fine-toothed comb. Grepping the .cf files for 'bayes' should get you most relevant options. In addition, check for the use_learner option. Besides the obvious use_bayes, the use_bayes_rules option seems a likely candidate for your issue. If that still doesn't explain it, it's time for some debugging. Both using the plain 'spamassassin' executable directly, as well as debugging Amavis. Thanks, Karsten. I got Bayes working. It had to do with the MySQL bayes_seen table being in latin1 and not in UTF8. Once I converted the table and data to UTF8, it worked.
Is this how this is supposed to work?
Not sure if I'm asking the right group but being new to all of this, it seems
like a good place to start. A little about my setup. I wanted to build a
front-end filter for my Exchange server so I put together
Postfix-Spamassassin-Amavis and tied in DCC, pyzor, and razor. I'm tailing the
mail.log and it seems to catch a lot of stuff but it lets a TON of stuff
through. I'll post the greater part of the email header on one seriously
obvious spam message and see if anyone can tell me what I'm missing here. I
appreciate any help and please be kind. I'm VERY new to this stuff. It was a
miracle I got this working at all. Trying to tie all of my domain names into
postfix config files was ridiculous!
Received: from data.gabowitztv.com (198.246.47.80) by mail.phhw.com (10.0.0.2)
with Microsoft SMTP Server id 14.3.181.6; Thu, 24 Jul 2014 10:49:29 -0500
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dkim; d=gabowitztv.com;
h=Mime-Version:Content-Type:Message-Id:Date:From:To:Subject;
i=pimsleurappro...@gabowitztv.com;
bh=bHddQVhew6uaKkn5Wru5J+kpECM=;
b=OMQ2jbeSaHzoNbvsPUfEFd0zfSgv9p9MCFxNrLkHEYwNmjuU0XUdKwLzGgvphTit6h7Ss5dYxIC7
3vIDjOVACIfKu5UL0X4Rr4AyNoQbsVWJe6477cM5rxydFeRoX7DTuhD/A0rdHhTzDXpA5rALsvGZ
NtMoMGCY9c+M7lEXVVs=
Received: by data.gabowitztv.com id hq4l9u0001gs for myem...@mydomain.com;
Thu,
24 Jul 2014 15:47:31 + (envelope-from
pimsleur-approach-myn...@gabowitztv.com)
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary=ade8-dbff-3531-7b86-2c0b-419d-0c87-208b
Message-ID: b80278c0d914b0c268b71353ffbd8eda.c3fe6a95aa809...@gabowitztv.com
Date: Thu, 24 Jul 2014 15:47:31 +
From: Pimsleur Approach pimsleurappro...@gabowitztv.com
To: myem...@mydomain.com
Subject:
=?utf-8?B?RldEOllvdSBjb3VsZCBsZWFybiBhIGxhbmd1YWdlIGluIGFzIGxpdHRsZSBhcyAxMCBEYXlz?=...
X-KSE-AntiSpam-Interceptor-Info: scan successful
X-KSE-AntiSpam-Version: 5.5.3, Database issued on: 07/24/2014 15:29:08
X-KSE-AntiSpam-Status: KAS_STATUS_NOT_DETECTED
X-KSE-AntiSpam-Method: none
X-KSE-AntiSpam-Rate: 19
X-KSE-AntiSpam-Info: Lua profiles 64420 [Jul 24 2014]
X-KSE-AntiSpam-Info: Version: 5.5.3
X-KSE-AntiSpam-Info: Envelope from:
pimsleur-approach-myn...@gabowitztv.com
X-KSE-AntiSpam-Info: {SMTP from is not routable}
X-KSE-AntiSpam-Info: SPF: pass
X-KSE-AntiSpam-Info: Rate: 19
X-KSE-AntiSpam-Info: Status: not_detected
X-KSE-AntiSpam-Info: Method: none
X-KSE-AntiSpam-Info: Moebius-Timestamps: 3032662, 3032697, 3032696
X-KSE-Antiphishing-Info: Clean
X-KSE-Antiphishing-Method: None
X-KSE-Antiphishing-Bases: 07/24/2014 15:29:00
X-KSE-Antivirus-Interceptor-Info: scan successful
X-KSE-Antivirus-Info: Clean
Resent-Message-ID: 20140724154947.33c981942...@smtp.mydomain.com
Resent-Date: Thu, 24 Jul 2014 10:49:47 -0500
Return-Path: myem...@mydomain.com
X-MS-Exchange-Organization-AuthSource: server.mail.mydomain.com
X-MS-Exchange-Organization-AuthAs: Anonymous
Greg Ledford
PHHW Technology Services LLC
1000 Corporate Centre Dr, Ste 200
Franklin, TN 37067
Office (615) 778-1777
Cell (615) 403-6989
Fax (615) 771-0081
Email gledf...@phhwtechnology.commailto:gledf...@phhwtechnology.com
Re: Is this how this is supposed to work?
On Thu, 2014-07-24 at 18:34 +, Greg Ledford wrote:
Not sure if I’m asking the right group but being new to all of this,
it seems like a good place to start. A little about my setup. I wanted
to build a front-end filter for my Exchange server so I put together
Postfix-Spamassassin-Amavis and tied in DCC, pyzor, and razor. I’m
tailing the mail.log and it seems to catch a lot of stuff but it lets
a TON of stuff through. I’ll post the greater part of the email header
on one seriously obvious spam message and see if anyone can tell me
what I’m missing here.
There is absolutely no SA header, including Amavis' flavor.
So there's nothing we can tell you, other than to review your entire
chain and verify the messages do get processed by Amavis and thus SA. If
they do, you will need to configure Amavis to add the usual headers --
which should be default, I believe.
(There are quite a few AntiSpam named headers which are totally
unrelated to Amavis and SA. Given their place in the headers, those
might be added by the sending SMTP.)
Next time pasting headers, please use a pastebin or tell your MUA to
behave and not inject empty lines...
I appreciate any help and please be kind. I’m VERY new to this stuff.
The first step would be to verify Postfix, Amavis and SpamAssassin do
get involved.
The headers you pasted seems to show a single Received header on your
side, Microsoft SMTP. There's no Postfix Received header.
The environment you try to set-up is SA, being called by Amavis, which
in turn is called by Postfix. For help on getting Postfix into your
chain, the postfix mailing-list would be a better place to start. ;)
Received: from data.gabowitztv.com (198.246.47.80)
by mail.phhw.com (10.0.0.2)
with Microsoft SMTP Server id 14.3.181.6; Thu, 24 Jul 2014 10:49:29 -0500
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Is this how this is supposed to work?
On Thu, 2014-07-24 at 21:43 +, Greg Ledford wrote:
So it looks like SA and Amavis are being totally bypassed?
Yes, there should be a few X-Spam-* headers added by SA or Amavis above
the pasted ones. And of course there should be a Received header by
postfix.
Since you didn't mentioned it in your reply, let me stress again that's
where you need to look into first -- if I understood your intended
set-up of Postfix with Amavis/SA *before* the MS server. Direct delivery
to the Microsoft SMTP (as it was till now) instead of Postfix might be
as easy as a bad or stale MX record...
FWIW, instead of only looking out for the relevant headers, both Postfix
and Amavis/SA have log files. Might be worth having a look at or tail -f
'ing.
Also, all this including X-Spam-* headers applies to ham, too. So you
can send yourself test mail, to observe it eventually flowing through
your Postfix environment.
Great. Looks like I'll be spending another night in front of this box.
Thanks for the input.
NP. Oh, and please always keep threads on-list, until you really mean to
reply in private only.
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: More text/plain questions
On Jul 23, 2014, at 11:45 AM, Amir 'CG' Caspi ceph...@3phase.com wrote:
On 2014-07-02 15:04, Amir Caspi wrote:
For what it's worth, I just received a spam that basically is the same
as what Philip complained about. I've posted a spample here:
http://pastebin.com/Y2YGwL49
[...]
I'm wondering if we shouldn't write a rule looking for lots of
#x0[0-9]{3}; patterns... say, 500 of them in one email. Or, would we
expect legitimate emails to have these?
So, to follow up on this... over the past couple of weeks I've been getting a
lot more FNs than normal, and almost every single one of these is an encoded
character spam like the example above. Bayes training does appear to work,
in that many of these FNs are already at BAYES_999... but there aren't enough
other rules hit to cause the FNs to cross the 5.0 threshold. (Other, similar
spams do cross the threshold, usually due to RAZOR and/or PYZOR hits.)
Since these are basically unicode character encodings, is there a move to
translate all charsets to UTF-8 (or some other fixed standard) before
applying body and/or URI rules? That would, presumably, help with trying to
catch these.
I'm definitely considering writing a rule to catch #x0[0-9]{3}; patterns.
I'm definitely worried it could cause FPs, but are there common circumstances
where legitimate emails would include dozens to hundreds of these? (The
latest FNs only include a few dozen, not the hundreds seen in the spample
above.)
Otherwise, I'm not sure what template rule I could write to catch these
things, and they're increasing in frequency (with more and more being missed
as FNs).
Thanks.
-- Amir
In text/plain with CTE of ‘7bit’ or ‘8bit’ it’s meaningless to use Unicode HTML
entity encodings. It’s obviously not HTML.
If you want Unicode in text/plain, it should be in base64 or quoted-printable
CTE.
-Philip
Re: More text/plain questions
On Jul 23, 2014, at 12:54 PM, Amir 'CG' Caspi ceph...@3phase.com wrote: Hope the patches above get pushed into production Indeed, though I'm still running SA v3.3.x ... I'm on a CentOS 5.10 platform and, because it's of the virtual-hosting control panel I use, I need my software distributed in RPMs. Until someone builds a proper 3.4 rpm for CentOS/RHEL 5, I'm stuck. (I could be the one to build it, but I'm certainly no expert at RPMs.) --- Amir You might have a shorter wait if you move to CentOS 6.5 instead. And I can help you with the RPM’s. I’m a fedora/epel packager. -Philip
Re: More text/plain questions
On Jul 23, 2014, at 1:21 PM, Amir 'CG' Caspi ceph...@3phase.com wrote: On 2014-07-23 13:14, Axb wrote: doesn't your VPS offer you shell access? if yes, uninstall the SA rpm stuff and install SA 3.4 from source/trunk. I think I didn't explain properly. I'm running the dedicated server on which there is VPS software. I need RPMs so that they get distributed to all the client sites. Installing from source/trunk at the root level won't distribute the tools to the individual sites. This is why I need 3.4 packaged as an rpm. I'm hoping someone will take up that task. 3.3.x was packaged as an rpm (on EPEL and other repos), so hopefully 3.4 will be, too. Thanks. --- Amir Sigh. Okay, I just did a blind build from fedpkg of spamassassin/master. http://fedorapeople.org/~philipp/spamassassin-3.4.0-7.el5.x86_64.rpm No warranties that this actually works. If you need i686 binaries I can make those too.
RE: Is this how this is supposed to work?
Sorry about that. I'm new to this list, too. It helps if I actually add
content_filter to postfix, I guess. This is all I'm seeing in the headers at
this point so it seems like I've got ONE part of it working. Does this look
like it's a start? Also my MX records are fine. I just removed them from the
headers I posted to keep people from seeing all my info but I guess that was
sort of pointless since they could have just done a nslookup and got that data
anyway. :/ Thanks again for your help.
X-Virus-Scanned: Debian amavisd-new at smtp.mymailserver.com
Received: from smtp.mymailserver.com ([127.0.0.1]) by localhost
(smtp.mymailserver.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id
Greg Ledford
PHHW Technology Services LLC
1000 Corporate Centre Dr, Ste 200
Franklin, TN 37067
Office (615) 778-1777
Cell (615) 403-6989
Fax (615) 771-0081
Email gledf...@phhwtechnology.com
-Original Message-
From: Karsten Bräckelmann [mailto:guent...@rudersport.de]
Sent: Thursday, July 24, 2014 4:54 PM
To: users@spamassassin.apache.org
Subject: Re: Is this how this is supposed to work?
On Thu, 2014-07-24 at 21:43 +, Greg Ledford wrote:
So it looks like SA and Amavis are being totally bypassed?
Yes, there should be a few X-Spam-* headers added by SA or Amavis above the
pasted ones. And of course there should be a Received header by postfix.
Since you didn't mentioned it in your reply, let me stress again that's where
you need to look into first -- if I understood your intended set-up of Postfix
with Amavis/SA *before* the MS server. Direct delivery to the Microsoft SMTP
(as it was till now) instead of Postfix might be as easy as a bad or stale MX
record...
FWIW, instead of only looking out for the relevant headers, both Postfix and
Amavis/SA have log files. Might be worth having a look at or tail -f 'ing.
Also, all this including X-Spam-* headers applies to ham, too. So you can send
yourself test mail, to observe it eventually flowing through your Postfix
environment.
Great. Looks like I'll be spending another night in front of this box.
Thanks for the input.
NP. Oh, and please always keep threads on-list, until you really mean to reply
in private only.
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: More text/plain questions
On 2014-07-24 16:11, Philip Prindeville wrote: You might have a shorter wait if you move to CentOS 6.5 instead. I would, but the VPS software I'm using does not run on CentOS 6.x, only 5.x. It's rather old software and I should convert to something else, but it's not worth the time I don't have, so I'm stuck with it. And I can help you with the RPM's. I'm a fedora/epel packager. Awesome. Perhaps you want to make an SA 3.4 package for EPEL 5? ;-) Of course, that helps more than just me... --- Amir
Re: More text/plain questions
On Jul 24, 2014, at 4:48 PM, Amir 'CG' Caspi ceph...@3phase.com wrote: On 2014-07-24 16:11, Philip Prindeville wrote: You might have a shorter wait if you move to CentOS 6.5 instead. I would, but the VPS software I'm using does not run on CentOS 6.x, only 5.x. It's rather old software and I should convert to something else, but it's not worth the time I don't have, so I'm stuck with it. And I can help you with the RPM’s. I’m a fedora/epel packager. Awesome. Perhaps you want to make an SA 3.4 package for EPEL 5? ;-) Of course, that helps more than just me... --- Amir Already done. I have no means to test it, however.
Re: Is this how this is supposed to work?
On Thu, 2014-07-24 at 22:33 +, Greg Ledford wrote:
Sorry about that. I'm new to this list, too.
Don't worry. I simply pointed it out so with further discussion,
everyone is on the same page. After all, there is more helpful folks on
this list -- and quite a few of them way better at Postfix and Amavis
stuff than I am.
Now if you could correct that top-posting... SCNR. ;)
It helps if I actually add content_filter to postfix, I guess. This is
all I'm seeing in the headers at this point so it seems like I've got
ONE part of it working. Does this look like it's a start?
That Received header below? Yeah, looks good, Amavis seems to be in the
loop.
I wonder how that could be all you're seeing at this point, though.
Amavis added a header it received a message locally, but who passed it
on? Isn't Postfix supposed to do that? So where is the Postfix Received
header?
It seems you're snipping too much stuff from the raw headers you may
believe to be irrelevant. However, unless you *know* it's irrelevant and
snipping it does *not* affect interpretation of the full mail flow, do
include it in the paste.
FWIW, in this case all headers beginning with the very first Received
from external by your server is likely to be relevant in some way.
Also my MX records are fine. I just removed them from the headers
So you ask about help with a set-up including Postfix being your MX, and
you snip all traces of Postfix acting like your MX. See where this is
bad?
[...] I posted to keep people from seeing all my info but I guess
that was sort of pointless since they could have just done a
nslookup and got that data anyway. :/ Thanks again for your help.
Correct. These public (sic) IPs are no secret. The mail you posted to
this list includes them...
Feel free to mask IP addresses and domain names if need be, in
particular email addresses. However, please keep it to a minimum and
definitely with a consistent pattern. Don't break headers, and don't
simply remove whole headers.
BTW, your outgoing SMTP server claims to be hostnamed smtp, though its
IP actually resolves to mail, breaking rDNS forward confirmation.
Probably outfall from adding that first line of defense Postfix
server...
X-Virus-Scanned: Debian amavisd-new at smtp.mymailserver.com
Received: from smtp.mymailserver.com ([127.0.0.1])
by localhost (smtp.mymailserver.com [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Alternate method to check for rule updates?
Sprint, which I use for net access is hijacking DNS. I cannot trust that the response received by sa-update is valid. Is there another method to check for updates? BTW, 1609892 is being given as the current version. It's been at this version for at least a few days. jd
Re: Alternate method to check for rule updates?
On Thu, 24 Jul 2014, jdebert wrote: BTW, 1609892 is being given as the current version. It's been at this version for at least a few days. Masscheck corpora are starved at the moment. It's being analyzed. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...the good of having the government prohibited from doing harm far outweighs the harm of having it obstructed from doing good. -- Mike@mike-istan --- 784 days since the first successful private support mission to ISS (SpaceX)
Re: Alternate method to check for rule updates?
On Thu, 2014-07-24 at 17:32 -0700, jdebert wrote:
Sprint, which I use for net access is hijacking DNS.
What exactly do you mean hijacking? Routing NXDOMAIN to some sort of
advertising web-server? Or serious packet-sniffing tampering with *any*
DNS query crossing their hardware?
I cannot trust that the response received by sa-update is valid. Is
there another method to check for updates?
If you really cannot trust *.updates.spamassassin.org DNS responses, you
cannot trust *any* DNS response. Including all the DNSxLs SA uses by
default. And rDNS rules. And your own SMTP's Received header.
In that case, I don't see how you can run SA at all, or even a trusted
SMTP MX. (Without VPN'ing out to a trusted DNS...)
And just in case your problem merely is with using your ISPs DNS server,
don't. Run your own local, caching DNS resolver (non-forwarding).
Unless we're really talking intercepting raw DNS traffic, that should
do.
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
URIDNSBL check return code
Hi,
Is there a way to get the return code in the generated reports?
eg:
uridnssub ALT_URI bl.foo A 127.0.0.2-127.0.0.11
body ALT_URI eval:check_uridnsbl('ALT_URI')
describe ALT_URI URL's domain A record listed in bl.foo ($RETRUN_CODE)
score ALT_URI 3.0
tflagsALT_URI net a
so if the check matched on 127.0.0.6, desc would be
describe ALT_URI URL's domain A record listed in bl.foo (127.0.0.6)
Thanks for any pointers
Re: Alternate method to check for rule updates?
On Fri, 2014-07-25 at 03:30 +0200, me wrote:
On Thu, 2014-07-24 at 17:32 -0700, jdebert wrote:
Sprint, which I use for net access is hijacking DNS.
I cannot trust that the response received by sa-update is valid. Is
there another method to check for updates?
Let me clarify a little.
If you really cannot trust *.updates.spamassassin.org DNS responses, you
False results here would in almost any case simply mean failing
sa-update. The odds for false TXT records, that (a) still are a valid
revision number and (b) do not result in either lint check failure or
simply downgrade to a previous working rule set are close to zero.
In other words, no rules update with an alert via cron. Or at worst,
revert to a previous known-to-work state.
cannot trust *any* DNS response. Including all the DNSxLs SA uses by
default. And rDNS rules. And your own SMTP's Received header.
False responses in those cases easily can result in both, FPs and FNs.
Lot's of them.
Thus, if you cannot even trust your ISP('s DNS) to get sa-updates right,
worrying about sa-update is the least of your problems.
(Unless, again, your issue actually is not running a local resolver.)
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Alternate method to check for rule updates?
On Fri, 25 Jul 2014 03:30:19 +0200 Karsten Bräckelmann guent...@rudersport.de wrote: On Thu, 2014-07-24 at 17:32 -0700, jdebert wrote: Sprint, which I use for net access is hijacking DNS. What exactly do you mean hijacking? Routing NXDOMAIN to some sort of advertising web-server? Or serious packet-sniffing tampering with *any* DNS query crossing their hardware? Yes. Also disabling dnssec, not responding to certain queries and modifying responses and queries. They like to call it transparent DNS proxying. But it's not proxying and obviously not transparent. I cannot trust that the response received by sa-update is valid. Is there another method to check for updates? If you really cannot trust *.updates.spamassassin.org DNS responses, you cannot trust *any* DNS response. Including all the DNSxLs SA uses by default. And rDNS rules. And your own SMTP's Received header. Wow. I never thought of that. :\ And just in case your problem merely is with using your ISPs DNS server, don't. Run your own local, caching DNS resolver (non-forwarding). Unless we're really talking intercepting raw DNS traffic, that should do. we are. jd
Re: Alternate method to check for rule updates?
On Thu, 24 Jul 2014, jdebert wrote: On Fri, 25 Jul 2014 03:30:19 +0200 Karsten Bräckelmann guent...@rudersport.de wrote: On Thu, 2014-07-24 at 17:32 -0700, jdebert wrote: Sprint, which I use for net access is hijacking DNS. What exactly do you mean hijacking? Routing NXDOMAIN to some sort of advertising web-server? Or serious packet-sniffing tampering with *any* DNS query crossing their hardware? Yes. Also disabling dnssec, not responding to certain queries and modifying responses and queries. They like to call it transparent DNS proxying. But it's not proxying and obviously not transparent. YGBFKM. Seriously? That kinda shoots the idea they are a Tier-1 ISP in the head... Maybe you have to pay extra to have them not fsck up your data. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The more you believe you can create heaven on earth the more likely you are to set up guillotines in the public square to hasten the process. -- James Lileks --- 784 days since the first successful private support mission to ISS (SpaceX)
Re: Alternate method to check for rule updates?
On 2014-07-24 18:56, jdebert wrote: On Fri, 25 Jul 2014 03:30:19 +0200 Karsten Bräckelmann guent...@rudersport.de wrote: On Thu, 2014-07-24 at 17:32 -0700, jdebert wrote: Sprint, which I use for net access is hijacking DNS. What exactly do you mean hijacking? Routing NXDOMAIN to some sort of advertising web-server? Or serious packet-sniffing tampering with *any* DNS query crossing their hardware? Yes. Also disabling dnssec, not responding to certain queries and modifying responses and queries. They like to call it transparent DNS proxying. But it's not proxying and obviously not transparent. If they're actually tampering with DNS requests made to other DNS servers, I'd give some serious thought to dropping them completely. If that's not an option, perhaps a $5 VPS at a network location that's reasonably near yourself, and then forwarding your own resolver to that resolver over port other than 53. -- Dave Warren http://www.hireahit.com/ http://ca.linkedin.com/in/davejwarren
Re: Alternate method to check for rule updates?
On Thu, 2014-07-24 at 18:56 -0700, jdebert wrote:
On Fri, 25 Jul 2014 03:30:19 +0200 Karsten Bräckelmann wrote:
On Thu, 2014-07-24 at 17:32 -0700, jdebert wrote:
Sprint, which I use for net access is hijacking DNS.
What exactly do you mean hijacking? Routing NXDOMAIN to some sort of
advertising web-server? Or serious packet-sniffing tampering with
*any* DNS query crossing their hardware?
Yes. Also disabling dnssec, not responding to certain queries and
modifying responses and queries.
Run.
Is that an option?
They like to call it transparent DNS proxying. But it's not
proxying and obviously not transparent.
I cannot trust that the response received by sa-update is valid. Is
there another method to check for updates?
If you really cannot trust *.updates.spamassassin.org DNS responses,
you cannot trust *any* DNS response. Including all the DNSxLs SA uses
by default. And rDNS rules. And your own SMTP's Received header.
Wow. I never thought of that. :\
And just in case your problem merely is with using your ISPs DNS
server, don't. Run your own local, caching DNS resolver
(non-forwarding).
Unless we're really talking intercepting raw DNS traffic, that should
do.
we are.
Got to admit, I wasn't expecting this. What you describe sounds major.
Just to be clear -- and absolutely no excuse to tamper with raw traffic
like this -- are we talking end-user / dial-up?
Sprint really even messes with DNS TXT records? What for?
Well, unless there is no way around that almost malicious tampering, I
guess the solution is to change ISP, regardless whether that's local
line or server housing.
I understand home ISP switching might be difficult. In which case I
guess Sprint would see exactly one type of traffic by me -- VPN traffic
using their line into a trustworthy network.
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
RE: Is this how this is supposed to work?
Good point. I'll post all of the headers and see if anyone has any tips.
Received: from smtp.phhwtechnology.com (10.0.1.7) by mail.phhwtechnology.com
(10.0.1.5) with Microsoft SMTP Server id 14.3.195.1; Thu, 24 Jul 2014
18:11:18 -0500
Received: from localhost (localhost [127.0.0.1])by
smtp.phhwtechnology.com
(Postfix) with ESMTP id E9A1D1943211 for gledf...@phhwtechnology.com; Thu,
24 Jul 2014 18:00:06 -0500 (CDT)
X-Virus-Scanned: Debian amavisd-new at smtp.phhwtechnology.com
Received: from smtp.phhwtechnology.com ([127.0.0.1])by localhost
(smtp.phhwtechnology.com [127.0.0.1]) (amavisd-new, port 10024)with
ESMTP id
l_YzbdhJopva for gledf...@phhwtechnology.com;Thu, 24 Jul 2014
18:00:02
-0500 (CDT)
Received: from sell.pywor.eu (sell.pywor.eu [23.249.160.158]) by
smtp.phhwtechnology.com (Postfix) with ESMTP id 29AFE194320C for
gledf...@phhwtechnology.com; Thu, 24 Jul 2014 17:59:50 -0500 (CDT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dkim; d=pywor.eu;
h=Content-Type:MIME-Version:From:To:Subject:Message-ID:References:In-Reply-To:Date;
i=smallbusinessl...@pywor.eu;
bh=WHIK8mEJdFg2fXt9Zz2ucXCGxPk=;
b=rxT2Eq1czjvL4ygXMD/Lxw4AZSYSW7ES8onIuJA3kpMwvVF3DUt7Oz3LP+r3jx6il9Y8ZjiFfIg2
B/JAjQQKIodjDTHxBvjIaoxaWP+b2vgl6AJxYf2PC9mWu1xmX+QEnNbY0ZbmVLmy24R+ODm7d0wU
P/eGjTK0IR/o4Uv6tts=
Content-Type: multipart/alternative;
boundary8108863872609529479==
MIME-Version: 1.0
From: Small Business Loan smallbusinessl...@pywor.eu
To: gledf...@phhwtechnology.com
Subject: Now trending: exciting small business loan opportunities
Message-ID: 615491d8b09c8278d9b65c2d2ffac...@pywor.eu
Thread-Topic: Now trending: exciting small business loan opportunities
References: 615491d8b09c8278d9b65c2d2ffac...@pywor.eu
In-Reply-To: 615491d8b09c8278d9b65c2d2ffac...@pywor.eu
Date: Thu, 24 Jul 2014 22:59:47 +
Return-Path: smallbusinessloan-gledford=phhwtechnology@pywor.eu
X-MS-Exchange-Organization-AuthSource: WEBSERVER01.mail.phhwtechnology.com
X-MS-Exchange-Organization-AuthAs: Anonymous
Greg Ledford
PHHW Technology Services LLC
1000 Corporate Centre Dr, Suite 200
Franklin, TN 37067
Office (615) 778-1777
Cell (615) 403-6989
Fax (615) 771-0081
-Original Message-
From: Karsten Bräckelmann [mailto:guent...@rudersport.de]
Sent: Thursday, July 24, 2014 6:38 PM
To: users@spamassassin.apache.org
Subject: Re: Is this how this is supposed to work?
On Thu, 2014-07-24 at 22:33 +, Greg Ledford wrote:
Sorry about that. I'm new to this list, too.
Don't worry. I simply pointed it out so with further discussion, everyone is on
the same page. After all, there is more helpful folks on this list -- and quite
a few of them way better at Postfix and Amavis stuff than I am.
Now if you could correct that top-posting... SCNR. ;)
It helps if I actually add content_filter to postfix, I guess. This
is all I'm seeing in the headers at this point so it seems like I've
got ONE part of it working. Does this look like it's a start?
That Received header below? Yeah, looks good, Amavis seems to be in the loop.
I wonder how that could be all you're seeing at this point, though.
Amavis added a header it received a message locally, but who passed it on?
Isn't Postfix supposed to do that? So where is the Postfix Received header?
It seems you're snipping too much stuff from the raw headers you may believe to
be irrelevant. However, unless you *know* it's irrelevant and snipping it does
*not* affect interpretation of the full mail flow, do include it in the paste.
FWIW, in this case all headers beginning with the very first Received from
external by your server is likely to be relevant in some way.
Also my MX records are fine. I just removed them from the headers
So you ask about help with a set-up including Postfix being your MX, and you
snip all traces of Postfix acting like your MX. See where this is bad?
[...] I posted to keep people from seeing all my info but I guess
that was sort of pointless since they could have just done a
nslookup and got that data anyway. :/ Thanks again for your help.
Correct. These public (sic) IPs are no secret. The mail you posted to this list
includes them...
Feel free to mask IP addresses and domain names if need be, in particular email
addresses. However, please keep it to a minimum and definitely with a
consistent pattern. Don't break headers, and don't simply remove whole headers.
BTW, your outgoing SMTP server claims to be hostnamed smtp, though its IP
actually resolves to mail, breaking rDNS forward confirmation.
Probably outfall from adding that first line of defense Postfix server...
X-Virus-Scanned: Debian amavisd-new at smtp.mymailserver.com
Received: from smtp.mymailserver.com ([127.0.0.1])
by localhost (smtp.mymailserver.com [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char
Re: Is this how this is supposed to work?
On Fri, 2014-07-25 at 03:16 +, Greg Ledford wrote:
Good point. I'll post all of the headers and see if anyone has any tips.
Read inline-comments in the headers bottom up.
And finally, Microsoft SMTP seems to have got the message from Postfix.
Received: from smtp.phhwtechnology.com (10.0.1.7)
by mail.phhwtechnology.com (10.0.1.5)
with Microsoft SMTP Server id 14.3.195.1; Thu, 24 Jul 2014 18:11:18 -0500
Postfix got the processed message back from Amavis.
Received: from localhost (localhost [127.0.0.1])
by smtp.phhwtechnology.com (Postfix) with ESMTP id E9A1D1943211
for gledf...@phhwtechnology.com; Thu, 24 Jul 2014 18:00:06 -0500 (CDT)
Amavis seems to have scanned for viruses.
X-Virus-Scanned: Debian amavisd-new at smtp.phhwtechnology.com
Amavis got the message locally from Postfix.
Received: from smtp.phhwtechnology.com ([127.0.0.1])
by localhost (smtp.phhwtechnology.com [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id l_YzbdhJopva for gledf...@phhwtechnology.com; Thu, 24
Jul 2014 18:00:02 -0500 (CDT)
Your Postfix MX SMTP accepted the message.
Received: from sell.pywor.eu (sell.pywor.eu [23.249.160.158])
by smtp.phhwtechnology.com (Postfix) with ESMTP id 29AFE194320C
for gledf...@phhwtechnology.com; Thu, 24 Jul 2014 17:59:50 -0500 (CDT)
Headers below this point are generated externally, irrelevant to your
problem.
Obviously, Postfix works as MX SMTP, feeds the message to Amavis and
get's it back, and finally hands it off to your old Exchange.
Good. Next step is properly configuring Amavis. You said you want to use
SpamAssassin, which Amavis does not seem to invoke. You didn't mention
Virus scanning, which Amavis does seem to perform.
What's missing is Amavis calling SA. If it does, there should be some
X-Spam-* headers as mentioned before, somewhere close above Postfix
receiving the message. And logs...
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
