Re: Start from scratch. Really needed?
Is there a smart way of deleting the bayes db? On 2.6x pretty much just delete the bayes db files and let it start over. I suppose restarting spamd would be good idea, it usually is when changing things. I'd hand-feed the first 200 hams and spams rather than autolearning them, were I doing it. Loren
RE: Load Balancing with Postfix [and SpamAssassin]
Alan Fullmer wrote: Yes they are rejecting mail for unknown users. However, currently I have it discard flagged spam, rather than reject it. Granted there are some that SA does not catch, therefore go into the whole limbo situation. I currently have no way for this machine to check the validity of a user. :( It resides on the 3rd box and by then it's already 'processed it'. I'm almost now wondering if there is another issue I may have overlooked. If you're running that on one machine, makes me wonder. I will investigate on that part. That's exactly what I was referring to. If you can get your machine to reject the invalid users, you can cut down your processing time significantly. Over 3/4 of the connections to my server are rejected for invalid users. If I had to accept all of those connections, there's no way I could do it with only one machine. You need to investigate methods for your server to validate the users. You could do this via LDAP or some sort of synchronization routine. -- Bowie
Re: Autolearn=failed, on SA version 3.0.4
[EMAIL PROTECTED] wrote: While referring the previous discussions regarding permissions on Bayes DB files, I would like to know what should be the permissions because the log files indicate autolearn=failed/no Well no merely indicates that the message did not score high or low enough to qualify for learning and thus learning was not attempted. Failed means that learning was attempted but could not complete. If it comes up occasionally, that's normal because SA will sometimes fail due to lock contention. Only one process can be writing learning data to the database at a time, and if two spamd's try learning at the same time, one will fail rather than backlog the mail queue waiting. However, if it's always coming back no or failed, that likely points to a permission problem. 9 times out of 10 this happens when spamd is running as root, and spamc is called site-wide as root. Spamd will never scan mail as root, so it falls back to nobody as a safety measure of last resort. nobody should not be able to write it's homedir, so when this happens learning will fail. The solution is to use -u to cause spamd to scan as another user with a homedir (most create a spamd user), not root. 1 time out of 10 this happens when someone specifies a bayes_path without bayes_file_mode 0777. So: 1) do you have a -u parameter to spamd or spamc? 2) if not, how do you call spamc? 3) do you have a bayes_path and/or bayes_file_mode statement in your config files (ie: local.cf)?
Exim 4.60 SpamAssassin 3.1.0 Problems
Hello all,I'm writing this list in regards to an issue that has developed after I upgraded to Exim 4.60 SpamAssassin 3.1.0. Originally I posted this on the Exim'suser mailing listwhere I got numerous replies, but nothing concrete answer wise to what the cause is or the solution is for this specific problem. Most often I was told "Contact the SA developers, it's their problem". Currently my platform is running SpamAssassin 3.1.0 on CentOS 3.4 with Exim 4.60 as my MTA. *** continued from the orginial email *** After upgrading to 4.60 I also upgraded to SpamAssassin 3.1.0 so as to continue offering my webhosting customers the best spam protection I could. However within hours of making this upgrade customers started calling me nonstop that their email "wasn't working". It finally took me a while to discover that what they were saying is that the email they were sending out was bouncing back with the following text listed: This message was created automatically by mail delivery software. A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed: [EMAIL PROTECTED] local delivery failed The following text was generated during the delivery attempt: -- [EMAIL PROTECTED] -- An error was detected while processing a file of BSMTP input. The error message was: 421 Lost incoming connection The SMTP transaction started in line 0. The error was detected in line 3. 0 previous messages were successfullyprocessed. The rest of the batch was abandoned. 421 Lost incoming connection Transaction started in line 0 Error detected in line 3 The more research I did, I couldn't figure out what in the world was causing this. I downgraded back to Exim 4.53 and SpamAssassin 3.0.4 (this setup worked fine before) and the problems continued. Finally after killing the SpamAssassin process, Exim 4.53 ran fine for a few days. Then I upgraded to 4.54 and finally 4.60 after each upgrade of Exim was confirmed to work. However, again this evening once I recompiled SpamAssassin 3.1.0 and started the process with the command /usr/bin/spamd -d -c -m 5 the same issues immediately started.Customers reported that it was very sporatic and random. They could email someone and get the above 421 error, then reemail them and it'd work. Several people who tried forwarding me copies of their error messages were unable to get copies of the emails to me. I immediatedly started checking my Exim logs and noticed BSMTP related errors there such as the following: 2006-01-10 23:05:23 SMTP connection from mail lost while reading message data (header) 2006-01-10 23:05:23 1EwX9q-00060G-ML [EMAIL PROTECTED]: spamcheck transport output: An error was detected while processing a file of BSMTP input. 2006-01-10 23:05:23 1EwX9q-00060G-ML ** [EMAIL PROTECTED] F=[EMAIL PROTECTED] R=spamcheck_director T=spamcheck: Child process of spamcheck transport returned 2 from command: /usr/sbin/exim (preceded by transport filter timeout while writing to pipe) 2006-01-10 23:05:23 1EwXEh-00062B-1P = R=1EwX9q-00060G-ML U=mail P=local S=34000 T="Mail delivery failed: returning message to sender" from for [EMAIL PROTECTED] 2006-01-10 23:05:23 1EwX9q-00060G-ML Completed I really don't know what to say outside of this appears to be an issue that Exim and SpamAssassin are having issues working together. All of my webhosting customers are expecting quality spam filtering and protection. I don't know what to do if SpamAssassin and Exim aren't working now, nor are they working if I downgrade either. My temporary solution has been to run Exim 4.60 but leave SA 3.1.0 turned off until I can get a solution implimented.Does ANYONE have any ideas on what direction to take??!ThanksBrad
SPF test clarification
Can someone point me in the right direction on exactly what the difference between the following SPF tests are, please? I assume that SPF_PASS means the sending domain has an SPF record and the sending server IP matches. However, the description for SPF_FAIL, SPF_SOFTFAIL, and SPF_NEUTRAL are all the same. In which case does the sender domain not have an SPF record? Which one is there a record, but the sending server IP doesn't match? What is the fourth case that I'm missing? Jason
Anti-phishing rules?
I've noticed that many phishing emails contain URLs with one of these two formats: http://trusteddomain.com.fakedomain.xx/... http://fakedomain.xx/.../trusteddomain.com/ where .xx is any TLD and ... is any series of characters. More specifically, the trusted domain usually ends in .com (paypal.com, ebay.com, some_bank_name.com, etc), but the phisher's domain (fakedomain.xx) can have any TLD (.net, .com, .org, or any of the country-specific TLDs). Of course, the protocol can be https as well (though this is rarer). Has anyone considered creating rules for emails containing URLs like those above? I realize that some legitimate sites use redirection in email: http://your_bank.com/please/visit/our/partner/third_party_product.com/ so this can't be scored too high, but it still might be useful. We do use clamav, but it doesn't block all phishing emails, and I thought this might help. I know there are SARE_SPOOF_COM2COM and SARE_SPOOF_COM2OTH rules in 70_sare_spoof.cf to catch things like a.com.b.com and a.com.b.c, but I wasn't sure if these quite caught what I'm suggesting. Has anyone tried creating rules like this and filtered out too much ham? Are there other better ways of scoring phishing emails? I've aware of the SARE_FORGED_PAYPAL and similar rules, but these assume the phisher will spoof a legitimate domain's email address, instead of just the URL. My apologies if this has been asked before. -- Sincerely, Sarang Gupta ([EMAIL PROTECTED])
sa-learn and user preferences
On the servers I admin, the user preferences are stored in SQL, yet sa-learn insists on there being a .spamassassin directory in the users' home directory, creating it and a default user_prefs file if they do not exist. Why? What does it need the prefs for? Can it use the SQL preferences? Is there no way to turn off this behavior other than passing a --prefs-file=/dev/null flag to sa-learn? Pertinent details: I'm using SA 3.1.0. sa-learn is invoked from a cron job running as root, but the -u flag passes in the username for each user whose spam folder is learned.
Re: SPF test clarification
Jason Bertoch wrote: Can someone point me in the right direction on exactly what the difference between the following SPF tests are, please? I assume that SPF_PASS means the sending domain has an SPF record and the sending server IP matches. However, the description for SPF_FAIL, SPF_SOFTFAIL, and SPF_NEUTRAL are all the same. In which case does the sender domain not have an SPF record? None of the above.. If the sender domain has no SPF record, there will be no SPF rules matching at all. Which one is there a record, but the sending server IP doesn't match? That depends what the sender's SPF record is set for in the all clause. If it's ?all you get SPF_NEUTRAL If it's ~all you get SPF_SOFTFAIL if it's !all you get SPF_FAIL. What is the fourth case that I'm missing? There is none.
Re: Exim 4.60 SpamAssassin 3.1.0 Problems
Bradley Walker [EMAIL PROTECTED] wrote: However within hours of making this upgrade customers started calling me nonstop that their email wasn't working. [snip] 2006-01-10 23:05:23 1EwX9q-00060G-ML ** mailto:[EMAIL PROTECTED] [EMAIL PROTECTED] F= mailto:[EMAIL PROTECTED] [EMAIL PROTECTED] R=spamcheck_director T=spamcheck: Child process of spamcheck transport returned 2 from command: /usr/sbin/exim (preceded by transport filter timeout while writing to pipe) My temporary solution has been to run Exim 4.60 but leave SA 3.1.0 turned off until I can get a solution implimented. my temporary solution would be to put: timeout_defer ignore_status into the SA exim router. you should see what happens during that timeout. most probably it is SA spinning the CPU for few minutes. are you using bayes? i've had such problem when the token file got *really* big. debug might help, too. and these two lines are useful even as something more than a workaround, they insure that whenever SA timeouts/dies, a message is deferred instead of bounced. -- Stanisław Halik, http://tehran.lain.pl pgpOp8i4XyvAt.pgp Description: PGP signature
Re: SPF test clarification
Jason Bertoch escreveu: Can someone point me in the right direction on exactly what the difference between the following SPF tests are, please? I assume that SPF_PASS means the sending domain has an SPF record and the sending server IP matches. However, the description for SPF_FAIL, SPF_SOFTFAIL, and SPF_NEUTRAL are all the same. In which case does the sender domain not have an SPF record? Which one is there a record, but the sending server IP doesn't match? What is the fourth case that I'm missing? SPF defines 3 kind of fail: neutral fail, soft fail and fail. This allows each domain to tell others what to do when SPF fails. If you're really concerned about your domain being forged, a fail would be the correct configuration (-all). In other cases, softfail (~all) or neutral fail (?all) can be used. http://www.openspf.org/mechanisms.html Mechanisms can be prefixed with one of four characters: - fail ~ softfail + pass ? neutral -- Atenciosamente / Sincerily, Leonardo Rodrigues Solutti Tecnologia http://www.solutti.com.br Minha armadilha de SPAM, NO mandem email [EMAIL PROTECTED] My SPAMTRAP, do not email it
Create address to pipe spams to spamassassin -r?
We're considering creating a spam reporting email address that would automatically pipe received messages to spamassassin -r. Is this a good idea? Details/thoughts: % I know that spamassassin -r just reports the hash of the message to Razor, Pyzor, etc, and therefore only increases our protection against receiving the exact same spam again (since we do use Razor and Pyzor with our spamassassin), but we tend to receive multiple copies of the same spam, so this might be useful. % We are running SpamAssassin version 3.1.0 and http://wiki.apache.org/spamassassin/ReportingSpam notes that the -r option wil automatically strip existing SpamAssassin markup. Will it also strip headers? Or do we need to strip headers before piping to spamassassin -r? % Are there better ways of allowing users at an organization to report spam in an effective way? -- Sincerely, Sarang Gupta ([EMAIL PROTECTED])
Re: Exim 4.60 SpamAssassin 3.1.0 Problems
Stanislaw Halik [EMAIL PROTECTED] wrote: my temporary solution would be to put: timeout_defer ignore_status into the SA exim router. d'oh, sorry. i've meant the transport, not the router. -- Stanisław Halik, http://tehran.lain.pl pgpAzWZdIzmNC.pgp Description: PGP signature
Re: Create address to pipe spams to spamassassin -r?
Sarang Gupta wrote: We're considering creating a spam reporting email address that would automatically pipe received messages to spamassassin -r. Is this a good idea? Details/thoughts: Depends on how you intend to get mail there. If you're talking about a spamtrap, go for it, just keep an eye on what it receives. If you're going to have users forward mail to it, you'll have to do some extra work. A forwarded message will not have the original headers, and will likely have the body re-encoded by your client and text added to it. This kind of message will be useless to spamassassin -r. You *MUST* feed a true, unadulterated copy of the message to spamassassin -r. The only molestation can be spamassassin markups, spamassassin headers, and extra received headers.
RE: SPF test clarification
Which case is there a record, but the sending server IP doesn't match? That depends what the sender's SPF record is set for in the all clause. If it's ?all you get SPF_NEUTRAL If it's ~all you get SPF_SOFTFAIL if it's -all you get SPF_FAIL. That makes sense but now the scores for these rules have me a little confused. If a domain administrator indicates that we should fail any message not sourced from his IP's, why is the score for SPF_FAIL the smallest of the three? Shouldn't it be set at or near the required_score, instead?
sa-learn done as root.
Hello all.. Novice SA Admin here (well, none of my users complain - wait.. I have none, just me). I recently read something that says sa-learn is learned for the user who runs sa-learn. I've always run sa-learn as root. Is there a easy way to copy the contents of what's been learned from root to my user? I did read through the sa-learn doc, but maybe the way I'm wording it isn't how it's worded in the docs. Thanks. Evan
Re: SPF test clarification
Jason Bertoch escreveu: That makes sense but now the scores for these rules have me a little confused. If a domain administrator indicates that we should fail any message not sourced from his IP's, why is the score for SPF_FAIL the smallest of the three? Shouldn't it be set at or near the required_score, instead? I have seen SEVERAL domains with misconfigured SPF values. So, getting SPF_FAILs near required_score would, for sure, block some messages coming from misconfigured SPF domains which also matches some other rules. False positives, I dont like that. Anyway, i have raised my SPF_FAILs scores to: score SPF_NEUTRAL 4 score SPF_SOFTFAIL 4.5 score SPF_FAIL 5 I'm running with required_score 8. Failing SPF will raise a lot the message score, but will not reject it only because of SPF failing. If it's really a SPAM, it will certainly hit several other rules (SARE rules helps a lot here) and, adding SPF fail scores to that, will reach required score. -- Atenciosamente / Sincerily, Leonardo Rodrigues Solutti Tecnologia http://www.solutti.com.br Minha armadilha de SPAM, NÃO mandem email [EMAIL PROTECTED] My SPAMTRAP, do not email it
Re: sa-learn done as root.
Hello all.. Novice SA Admin here (well, none of my users complain - wait.. I have none, just me). I recently read something that says sa-learn is learned for the user who runs sa-learn. I've always run sa-learn as root. Is there a easy way to copy the contents of what's been learned from root to my user? I did read through the sa-learn doc, but maybe the way I'm wording it isn't how it's worded in the docs. If it's saved in an SQL database, I imagine you could do a simple UPDATE query to change the username, like... UPDATE bayes_vars SET username='username' WHERE username='root'; (Apologies if that's MySQL-specific. It's what I use.) If you're not using SQL, you could follow the instructions to do a backup and restore of the database, which would go something like this... sa-learn --backup /tmp/file.txt sa-learn -u username --restore=/tmp/file.txt Then, in the future, just add -u username to your sa-learn command line to learn as the desired username rather than root.
Re: sa-learn done as root.
Mike Jackson wrote: Hello all.. Novice SA Admin here (well, none of my users complain - wait.. I have none, just me). I recently read something that says sa-learn is learned for the user who runs sa-learn. I've always run sa-learn as root. Is there a easy way to copy the contents of what's been learned from root to my user? I did read through the sa-learn doc, but maybe the way I'm wording it isn't how it's worded in the docs. If it's saved in an SQL database, I imagine you could do a simple UPDATE query to change the username, like... UPDATE bayes_vars SET username='username' WHERE username='root'; (Apologies if that's MySQL-specific. It's what I use.) If you're not using SQL, you could follow the instructions to do a backup and restore of the database, which would go something like this... sa-learn --backup /tmp/file.txt sa-learn -u username --restore=/tmp/file.txt Then, in the future, just add -u username to your sa-learn command line to learn as the desired username rather than root. I dont know what version of SA the OP is running but note that 2.64 has no -u parameter so you can not pass the username on the command line. -Jim
RE: spam scores low (Sendmail + smtp-vilter + SA )
-Original Message-
From: Mike Sassaman [mailto:[EMAIL PROTECTED]
Sent: Tuesday, January 17, 2006 5:48 PM
To: users@spamassassin.apache.org
Subject: RE: spam scores low (Sendmail + smtp-vilter + SA )
% spamassassin --lint shows no output, so I'm thinking
that means no
problems in my local.cf.
Good, 'spamassassin --lint' should show no outout, it ony barks when
there's something wrong. Now 'spamassassin --lint -D' gives -tons-
of output, but any error messages often get buried in with all the
debugging output.
% spamassassin /tmp/test-message.txt on a lowscoring spam
(-1.6 according
to smtp-vilter's headers) get scored a whopping 14.3 by
spamassassin! Tests
hit include HELO_DYNAMIC_IPADDR, BAYES_99, RCVD_IN_SORBS_DUL,
RCVD_IN_BL_SPAMCOP_NET, RCVD_IN_XBL, RCVD_IN_NJABL_DUL
OK, so that vets your basic spamassassin system. Now the next
thing to try
is to take that same test message and feed it to spamd via
spamc to see
what the daemon thinks about it. Do: '% spamc -R
/tmp/test-message.txt'
that should give a report output that shows the same tests
hit. If it
doesn't then that says that there's something about how
you're running
spamd that is causing problems.
I noticed that in your tests report you show most of the
score came from
network type tests. If you start your spamd with the -L
command line
option that will disable all network tests (and seriously
reduce your spam
recognising ability). Or if there's something about the way
that your
spamd starts up so that network tests are disabled, it
will have the
same net-not result.
So I think Dave is right - the problem is with the milter,
or at least the
milter / spamassassin communication.
It may be a milter issue but first we need to rule out
whether it's a
spamd issue (thus the spamc tests). IE the flow is
sendmail - milter
- spamd, spamd results - milter - sendmail.
Verified that spamassassin testmessage.txt and spamc -R
testmessage.txt
hit the same tests for my sample spam, specifically:
Content analysis details: (14.3 points, 4.0 required)
pts rule name description
--
--
0.0 SUB_HELLO Subject starts with Hello
4.4 HELO_DYNAMIC_IPADDRRelay HELO'd using suspicious
hostname (IP addr
1)
3.5 BAYES_99 BODY: Bayesian spam probability
is 99 to 100%
[score: 0.9937]
2.0 RCVD_IN_SORBS_DUL RBL: SORBS: sent directly from dynamic IP
address
[24.125.102.162 listed in dnsbl.sorbs.net]
1.2 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in
bl.spamcop.net
[Blocked - see
http://www.spamcop.net/bl.shtml?24.125.102.162]
3.1 RCVD_IN_XBLRBL: Received via a relay in Spamhaus XBL
[24.125.102.162 listed in
sbl-xbl.spamhaus.org]
0.1 RCVD_IN_NJABL_DUL RBL: NJABL: dialup sender did
non-local SMTP
[24.125.102.162 listed in
combined.njabl.org]
Again, rating this mail actually received when it passed thru
my system was
-1.6.
These are the entries in /etc/rc.local that start smtp-vilter
and spamd:
# start smtp-vilter
if [ X${smtp_vilter} != XNO -a \
-x /usr/local/sbin/smtp-vilter ]; then
echo -n ' smtp-vilter'
/usr/local/sbin/smtp-vilter
fi
# Start Spamassassin daemon
/usr/local/bin/spamd -u _vilter -d -D -s mail -x echo -e spamd
started...
...and here is where it is called in my sendmail .mc file:
INPUT_MAIL_FILTER(`smtp-vilter',
`S=unix:/var/smtp-vilter/smtp-vilter.sock,
F=T, T=S:10m;R:10m;E:10m')dnl
Starting spamd in debug mode, I see this message:
debug: Score set 0 chosen.
Doesn't that mean network tests are not being run? But as
you can see, I am
NOT starting spamd with a -L. Why would score set 0 be
chosen? Can I force
it to run network tests or choose the score set manually?
Ok, so according to the logs it seems that just about every spam message is
hitting the ALL_TRUSTED rule. Maybe this is my problem. I understand that
indicates a broken trust path, as told here:
http://wiki.apache.org/spamassassin/TrustPath
But why is my trust broken? My local.cf contains the lines:
clear_internal_networks
clear_trusted_networks
internal_networks x.x.x.x
trusted_networks x.x.x.x
Where x.x.x.x is the address of my mail server running SA. All other mail
(basically all mail period) should be external, untrusted. So how can spam
be hitting the ALL_TRUSTED rule?
Re: sa-learn done as root.
On Thu, January 19, 2006 11:14 am, Mike Jackson wrote: If it's saved in an SQL database, I imagine you could do a simple UPDATE query to change the username, like... UPDATE bayes_vars SET username='username' WHERE username='root'; (Apologies if that's MySQL-specific. It's what I use.) I don't use SQL for SA - is there an advantage (ie speed) in doing so, or a disadvantage? If you're not using SQL, you could follow the instructions to do a backup and restore of the database, which would go something like this... sa-learn --backup /tmp/file.txt sa-learn -u username --restore=/tmp/file.txt Then, in the future, just add -u username to your sa-learn command line to learn as the desired username rather than root. It doesn't appear that I'm using SQL, as I don't see any database entries. Running the --backup command, and then grepping the file.txt, I see: v 3 db_version # this must be the first line!!! v 412 num_spam v 2145num_nonspam t 1 0 1122461946 43da1d3f27 t 1 0 629865 803e78e189 t 1 0 743100 e670eeddbf t 2 0 1122630499 d607b2b6db t 0 1 1104779201 860d2c6001 SNIP so I assume that's correct. Thanks again. Evan
Re: SPF test clarification
Jason Bertoch wrote: Which case is there a record, but the sending server IP doesn't match? That depends what the sender's SPF record is set for in the all clause. If it's ?all you get SPF_NEUTRAL If it's ~all you get SPF_SOFTFAIL if it's -all you get SPF_FAIL. That makes sense but now the scores for these rules have me a little confused. If a domain administrator indicates that we should fail any message not sourced from his IP's, why is the score for SPF_FAIL the smallest of the three? I don't know about your SA, but on 3.1.0's set 3 it's the middle of the three. You're trying to apply simple logic to a non-simple system. Never expect the simple when it comes to SA rule scores, the system is many orders of magnitude more complex than you think, because it's based on REAL patterns of REAL email sent by human people. Let's look at some real-world data: OVERALL% SPAM% HAM% S/ORANK SCORE NAME 3.437 4.8942 0.03960.992 0.801.38 SPF_SOFTFAIL 2.550 3.5717 0.16760.955 0.531.14 SPF_FAIL 2.297 3.2090 0.16950.950 0.521.07 SPF_NEUTRAL Note that SPF_FAIL matched had a higher HAM% than SOFTFAIL did.. Just because it in theory should be a better test does not mean it will be. You've got humans involved here, and human behavior is a lot strange. My guess is that a careless admin who did not think the implications through would be prone to immediately go to SPF_FAIL. This careless admin is also more likely to have omissions from his SPF record. SOFTFAIL is more likely to be used by conservative admins who think out their needs more carefully. These sites are much less likely to have omissions in their records. But that's just a theory. I'm no psychologist, I just read the numbers.
Re: Exim 4.60 SpamAssassin 3.1.0 Problems
Bradley: Fought the same battle here just last week literally. With the help of Larry Rosenman from the SA/Exim lists we got it working VERY well here. It's basically a machine load issue for me, and I'm guessing for you as well. First thing...with SA are you running either of these rules: blacklist-uri.cafe blacklist.cf They are both HUGE CPU hogs, remove tem from your rule sets. Second: Are your cleaning up after exim/SA? If not this script will do it for you, I'd run it nightly around midnight here check the path names and correct to match your setup: # more /usr/sbin/exim-cleanup exim_dbdir=/var/spool/exim exim_tidydb=/usr/local/exim/sbin/exim_tidydb echo echo Tidying Exim hints databases: for db in $exim_dbdir/db/*.lockfile; do echo $exim_tidydb $exim_dbdir `basename $db .lockfile` done ll /usr/local/exim/exiscan/virusmails ll /usr/local/exim/spool/scan rm /usr/local/exim/exiscan/virusmails/* rm -r /usr/local/exim/spool/scan/* These 2 items basically took my system load from a 10-12 and put it at .89 and my mail queue from HOURS of queue time to avg less than a minute: Plus these command lines (with appropriate editing will give some nice stats: Subject: Cron /usr/sbin/sa-stats -l /var/log/exim -f mail From: [EMAIL PROTECTED] (Cron Daemon) Date: Wed, 18 Jan 2006 23:50:47 -0600 Email:22065 Autolearn: 2 AvgScore: 13.51 AvgScanTime: 32.73 sec Spam: 10091 Autolearn: 1 AvgScore: 29.70 AvgScanTime: 32.59 sec Ham: 11974 Autolearn: 1 AvgScore: -0.14 AvgScanTime: 32.85 sec Time Spent Running SA: 200.62 hours Time Spent Processing Spam: 91.34 hours Time Spent Processing Ham: 109.28 hours Subject: Cron /usr/local/exim/sbin/eximstats -ne -nr /var/log/exim/mainlog From: [EMAIL PROTECTED] (Cron Daemon) Date: Wed, 18 Jan 2006 23:50:58 -0600 Exim statistics from 2006-01-15 05:05:09 to 2006-01-18 23:50:48 Grand total summary --- At least one address TOTAL VolumeMessages Hosts Delayed Failed Received 107MB7712 1032 417 5.4% 113 1.5% Delivered266MB 36558 378 Deliveries by transport --- VolumeMessages address_file2518KB 544 address_pipe 11MB1196 procmail 84MB6393 remote_smtp 169MB 28425 Try these suggestions and let me know how it goes. I'm happy to try to help out more. George ===[George R. Kasica]===+1 262 677 0766 President +1 206 374 6482 FAX Netwrx Consulting Inc. Jackson, WI USA http://www.netwrx1.com [EMAIL PROTECTED] ICQ #12862186
Re: spam scores low (Sendmail + smtp-vilter + SA )
Mike Sassaman wrote: Ok, so according to the logs it seems that just about every spam message is hitting the ALL_TRUSTED rule. Maybe this is my problem. I understand that indicates a broken trust path, as told here: http://wiki.apache.org/spamassassin/TrustPath But why is my trust broken? My local.cf contains the lines: clear_internal_networks clear_trusted_networks internal_networks x.x.x.x trusted_networks x.x.x.x I know the docs claim you can do just an IP as a trusted_networks declaration, but I've had problems with SA misbehaving when you use that format. Try adding a /32 netmask on the end and see if that clears it up. It's a long shot, but worth a quick try. Where x.x.x.x is the address of my mail server running SA. All other mail (basically all mail period) should be external, untrusted. So how can spam be hitting the ALL_TRUSTED rule? Based on past posts I read you are using SA 3.0.4. Versions older than 3.0.5 can also have this problem if there's an unparaseable Received: header. Since the header is unparsable, it doesn't count as either trusted or untrusted, which is a problem. This is fixed in SA 3.0.5 by backporting the 3.1.0 trust path code that adds an unparsable counter to the equation.
Re: spam scores low (Sendmail + smtp-vilter + SA )
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matt Kettler writes: Mike Sassaman wrote: Ok, so according to the logs it seems that just about every spam message is hitting the ALL_TRUSTED rule. Maybe this is my problem. I understand that indicates a broken trust path, as told here: http://wiki.apache.org/spamassassin/TrustPath But why is my trust broken? My local.cf contains the lines: clear_internal_networks clear_trusted_networks internal_networks x.x.x.x trusted_networks x.x.x.x I know the docs claim you can do just an IP as a trusted_networks declaration, but I've had problems with SA misbehaving when you use that format. Try adding a /32 netmask on the end and see if that clears it up. It's a long shot, but worth a quick try. if that is the case, please open a bug, too... - --j. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Exmh CVS iD8DBQFDz/V4MJF5cimLx9ARAp5eAKCpdCBTfv2CTFy3SID3R8+nQKShAwCgiS/V TB/pARWuFlsWJf5RSYcQUpQ= =uHT6 -END PGP SIGNATURE-
RE: spam scores low (Sendmail + smtp-vilter + SA )
-Original Message- From: Matt Kettler [mailto:[EMAIL PROTECTED] Sent: Thursday, January 19, 2006 3:01 PM To: Mike Sassaman Cc: users@spamassassin.apache.org Subject: Re: spam scores low (Sendmail + smtp-vilter + SA ) Mike Sassaman wrote: Ok, so according to the logs it seems that just about every spam message is hitting the ALL_TRUSTED rule. Maybe this is my problem. I understand that indicates a broken trust path, as told here: http://wiki.apache.org/spamassassin/TrustPath But why is my trust broken? My local.cf contains the lines: clear_internal_networks clear_trusted_networks internal_networks x.x.x.x trusted_networks x.x.x.x I know the docs claim you can do just an IP as a trusted_networks declaration, but I've had problems with SA misbehaving when you use that format. Try adding a /32 netmask on the end and see if that clears it up. It's a long shot, but worth a quick try. Where x.x.x.x is the address of my mail server running SA. All other mail (basically all mail period) should be external, untrusted. So how can spam be hitting the ALL_TRUSTED rule? Based on past posts I read you are using SA 3.0.4. Versions older than 3.0.5 can also have this problem if there's an unparaseable Received: header. Since the header is unparsable, it doesn't count as either trusted or untrusted, which is a problem. This is fixed in SA 3.0.5 by backporting the 3.1.0 trust path code that adds an unparsable counter to the equation. Thanks - I tried the /32 but it doesn't appear to have worked. Because of shear volume of messages hitting ALL_TRUSTED, it seems that it must be more than unparsable Received: headers, unless there is an awful lot of mail with unparsable headers.
Re: spam scores low (Sendmail + smtp-vilter + SA )
Mike Sassaman wrote: Thanks - I tried the /32 but it doesn't appear to have worked. Because of shear volume of messages hitting ALL_TRUSTED, it seems that it must be more than unparsable Received: headers, unless there is an awful lot of mail with unparsable headers. Well, if SA can't parse the format generated by your mailserver, that would affect all messages which don't have any additional Received: headers beyond the local delivery (which would be nearly all your spam/virus email).
RE: spam scores low (Sendmail + smtp-vilter + SA )
-Original Message- From: Matt Kettler [mailto:[EMAIL PROTECTED] Sent: Thursday, January 19, 2006 3:37 PM To: Mike Sassaman Cc: users@spamassassin.apache.org Subject: Re: spam scores low (Sendmail + smtp-vilter + SA ) Mike Sassaman wrote: Thanks - I tried the /32 but it doesn't appear to have worked. Because of shear volume of messages hitting ALL_TRUSTED, it seems that it must be more than unparsable Received: headers, unless there is an awful lot of mail with unparsable headers. Well, if SA can't parse the format generated by your mailserver, that would affect all messages which don't have any additional Received: headers beyond the local delivery (which would be nearly all your spam/virus email). Fair enough. I am using a relatively basic Sendmail installation (on OpenBSD3.8). How could I check to see if I was generating unparseable headers (and hopefully fix)?
Re: SPF test clarification
Jason Bertoch a écrit : pedestal It's my opinion that if an administrator misconfigured his SPF record, or a number of other things on their side, it is their fault that mail cannot be delivered. In the case of SPF_FAIL, they have explicitly told us they don't want mail to come from a server not listed in their record and I believe we should follow their directive. In fact, isn't that the point of SPF; to help us reject forged messages coming from unauthorized servers? Why bother even dealing with SPF if we're still going to let people get away with poor administration? That's partly how we got here in the first place... /pedestal your server, your rules. I personally don't use SA to police the network, but to detect spam. In addition, I prefer to let spam slip than block legitimate mail. I also won't block forwarded mail just because it fails SPF. but of course, YMMV. since PSF fail doesn't mean spam (nor spf $success mean legitimate mail), it's here (in SA) only as an additionnal parameter which value contributes to a global result. now if you want to jump in the spf crusade, you should use it in the MTA and probably not care in a content filter.
Re: SPF test clarification
Jason Bertoch wrote: pedestal It's my opinion that if an administrator misconfigured his SPF record, or a number of other things on their side, it is their fault that mail cannot be delivered. In the case of SPF_FAIL, they have explicitly told us they don't want mail to come from a server not listed in their record and I believe we should follow their directive. In fact, isn't that the point of SPF; to help us reject forged messages coming from unauthorized servers? Why bother even dealing with SPF if we're still going to let people get away with poor administration? That's partly how we got here in the first place... /pedestal I don't disagree with this, though I can give you an example of what can happen. I used to host the DNS for my domain with speakeasy.net who I had in general been quite happy with. Then they decided they needed to implement SPF and they were going to do it RIGHT NOW. They didn't create an addition to their otherwise wonder DNS admin webapp (I guess they were in a panic and didn't have the time), so instead they generated SPF records based on the MX records for your domain. In my case I also used my employers mail server to send email (and therefore needed to add it to my SPF record), but it was certainly NOT OK to add them as a MX record for my domain, so I suddenly found my emails being bounced from some mailing lists (this might have been one of them). I tried to explain to Speakeasy.net that they were generating incorrect SPF records, but they thought they were close enough and tried to talk me into some unacceptable workarounds. I don't know if they ever changed this, but I was no longer using them for DNS hosting in a matter of days. Since Speakeasy really is pretty good in a lot of ways, I can only imagine what goofy things some other ISPs have come up with. Steve
lock files are not being deleted
Is Spamassassin supposed to automatically delete lock files when completed? I am just wondering why so many files are created, some timestamps are from the previous day. My log files show the following: Jan 19 14:17:06 mail spamd[22166]: debug: lock: 22166 trying to get lock on /home/filter/.spamassassin/bayes with 3 retries Jan 19 14:17:06 mail spamd[22167]: debug: lock: 22167 trying to get lock on /home/filter/.spamassassin/bayes with 3 retries Jan 19 14:17:06 mail spamd[22164]: debug: lock: 22164 trying to get lock on /home/filter/.spamassassin/bayes with 7 retries Jan 19 14:17:06 mail spamd[22168]: debug: lock: 22168 trying to get lock on /home/filter/.spamassassin/bayes with 6 retries Jan 19 14:17:08 mail spamd[22164]: debug: lock: 22164 trying to get lock on /home/filter/.spamassassin/bayes with 9 retries Jan 19 14:17:08 mail spamd[22164]: Cannot open bayes databases /home/filter/.spamassassin/bayes_* R/W: lock failed: File exists It then has several files such as: bayes.lock.mail.domain.com.21886 Is this a sign that a process is not completing? Just curious if anyone has an idea. Thanks in advance Alan Fullmer www.xnote.com www.zoobuh.com
Re: lock files are not being deleted
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alan Fullmer writes: Is Spamassassin supposed to automatically delete lock files when completed? I am just wondering why so many files are created, some timestamps are from the previous day. My log files show the following: Jan 19 14:17:06 mail spamd[22166]: debug: lock: 22166 trying to get lock on /home/filter/.spamassassin/bayes with 3 retries Jan 19 14:17:06 mail spamd[22167]: debug: lock: 22167 trying to get lock on /home/filter/.spamassassin/bayes with 3 retries Jan 19 14:17:06 mail spamd[22164]: debug: lock: 22164 trying to get lock on /home/filter/.spamassassin/bayes with 7 retries Jan 19 14:17:06 mail spamd[22168]: debug: lock: 22168 trying to get lock on /home/filter/.spamassassin/bayes with 6 retries Jan 19 14:17:08 mail spamd[22164]: debug: lock: 22164 trying to get lock on /home/filter/.spamassassin/bayes with 9 retries Jan 19 14:17:08 mail spamd[22164]: Cannot open bayes databases /home/filter/.spamassassin/bayes_* R/W: lock failed: File exists It then has several files such as: bayes.lock.mail.domain.com.21886 Is this a sign that a process is not completing? Just curious if anyone has an idea. check the FAQ, this sounds a lot like a situation discussed there. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Exmh CVS iD8DBQFD0ATpMJF5cimLx9ARAoZgAJ95Fe5l4pum9V5ixrqzCcDWFRjUmQCfRVgK PgaRdMZY/pBqSfFCi24fSjY= =LHoz -END PGP SIGNATURE-
Re: spam scores low (Sendmail + smtp-vilter + SA )
Mike Sassaman wrote: How could I check to see if I was generating unparseable headers (and hopefully fix)? You could run a message through spamassassin -D and look at the debug output. There's a section in there where it's parsing the Received: headers. Just make sure it's not missing any.
RE: lock files are not being deleted
Is Spamassassin supposed to automatically delete lock files when completed? I am just wondering why so many files are created, some timestamps are from the previous day. My log files show the following: Jan 19 14:17:06 mail spamd[22166]: debug: lock: 22166 trying to get lock on /home/filter/.spamassassin/bayes with 3 retries Jan 19 14:17:06 mail spamd[22167]: debug: lock: 22167 trying to get lock on /home/filter/.spamassassin/bayes with 3 retries Jan 19 14:17:06 mail spamd[22164]: debug: lock: 22164 trying to get lock on /home/filter/.spamassassin/bayes with 7 retries Jan 19 14:17:06 mail spamd[22168]: debug: lock: 22168 trying to get lock on /home/filter/.spamassassin/bayes with 6 retries Jan 19 14:17:08 mail spamd[22164]: debug: lock: 22164 trying to get lock on /home/filter/.spamassassin/bayes with 9 retries Jan 19 14:17:08 mail spamd[22164]: Cannot open bayes databases /home/filter/.spamassassin/bayes_* R/W: lock failed: File exists It then has several files such as: bayes.lock.mail.domain.com.21886 Is this a sign that a process is not completing? Just curious if anyone has an idea. Thanks in advance Alan Fullmer Are you using the default NFS-safe locking system? If so, consider flock: http://spamassassin.apache.org/full/3.1.x/dist/doc/Mail_SpamAssassin_Conf.html#miscellaneous_options Gary V _ Express yourself instantly with MSN Messenger! Download today - it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
Re: Exim 4.60 SpamAssassin 3.1.0 Problems
Does ANYONE have any ideas on what direction to take??! I can't specifically help with your problem, I don't use Exim and have never seein anything like this reported. However, if 3.0.5 will work for you that would certainly be a pretty gool alternative to 3.1.0 until whatever this problem is gets figured out. Loren
Re: spam scores low (Sendmail + smtp-vilter + SA )
Thanks - I tried the /32 but it doesn't appear to have worked. Because of shear volume of messages hitting ALL_TRUSTED, it seems that it must be more than unparsable Received: headers, unless there is an awful lot of mail with unparsable headers. You could post a set of headers or two. Lots of people here (not necessarily me!) can spot the ones that are unparsable. Loren
RE: spam scores low (Sendmail + smtp-vilter + SA )
On Thu, 19 Jan 2006, Mike Sassaman wrote:
Well, if SA can't parse the format generated by your
mailserver, that would
affect all messages which don't have any additional Received:
headers beyond the
local delivery (which would be nearly all your spam/virus email).
Fair enough. I am using a relatively basic Sendmail installation (on
OpenBSD3.8). How could I check to see if I was generating unparseable
headers (and hopefully fix)?
That may be the answer. In the sendmail milter API the milter gets an
original copy of the incoming message, before sendmail alters it in any
way, including -before- adding the local 'Received:' header.
Thus a spamassasin-milter must internally synthesize a 'Received:' header
that correctly mimics the sendmail generated one, as it passes the
message on to spamd. If the smtp-vilter code isn't doing that
(either not at all or not correctly) it could cause your problem.
I know that the 'miltrassassin' milter had a bug that would cause
it to generate broken 'Received:' headers under certain input
corner-cases.
Hmm, I've never looked at smtp-vilter before. Looking at the code
now, I'm underwhelmed by their 'Received:' header synthesis code
(IE it's pretty lame).
And I think that I may see what the cause of your problem is. For
some strange reason they're using the '{client_addr}' macro rather
than the '_' macro for the address of the sending host.
Now '_' is in the sendmail milter default macro list, '{client_addr}'
is NOT. Did you explicitly add the '{client_addr}' macro to your
sendmail config file Milter.macros.connect parameter?
If you're not wedded to smtp-vilter you might want to consider using
a different milter or spend time trying to enlighten the authors
of that code and seeing if you can get it fixed.
Dave
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Turning Auto-Delete back off
SpamAssassin is a wonderful program, thanks. My ISP has implemented it. My ISP gives me this option: To simply have the server DELETE and NOT deliver emails that are tagged as spam by SpamAssassin, click here now. My problem is that I enabled this automatic delete option and would like to disable it now. No disable auto-delete option exists. When I disable SpamAssassin and re-enable it, the automatic-delete option is enabled. What do I need to change to get it back to adding ***SPAM*** to the subject of the e-mails and then delivering them? Thanks so much. Alan
Re: Turning Auto-Delete back off
On Thu, Jan 19, 2006 at 01:39:58PM -0500, Alan Henney wrote: SpamAssassin is a wonderful program, thanks. :) My ISP gives me this option: To simply have the server DELETE and NOT deliver emails that are tagged as spam by SpamAssassin, click here now. My problem is that I enabled this automatic delete option and would like to disable it now. No disable auto-delete option exists. What do I need to change to get it back to adding ***SPAM*** to the subject of the e-mails and then delivering them? You'll have to ask your ISP about this. SpamAssassin has no auto-delete functionality, so it has to be something they've setup outside of SA. -- Randomly Generated Tagline: Any member introducing a dog into the Society's premises shall be liable to a fine of one pound. Any animal leading a blind person shall be deemed to be a cat. -- Rule 46, Oxford Union Society, London pgpexvXG5E5aO.pgp Description: PGP signature
Outbound spam filtering
Anyone have any pointers on setting up an outbound MTA spam filter with qmail? I have spamassassin working on inbound, but want to prevent/block users from sending spam. Thanks!
