Re: Single word mails .
Matt Kettler wrote: ram wrote: Are the spammers testing some new spamtool I am getting mails with just a single word like gushes using etc what is this about now ? Read the archives for more details, however the general consensus is it's due to: 1) a mass run of short-emails to a broader-range of randomly generated addresses in an attempt to disover new ones. (aka Rumpelstiltskin attack) - OR - 2) some spammer screwed up their template when they last pushed one out to their botnet, and as a result the bots are generating emails with no useful payload. Both are quite plausible. Is there a good test for these? The first days run seemed to be related to the german stock market, but todays run so far I can not seem to find a common thread as the subject is getting the same one word as the body
Re: Google Summer of Code 2007 ...
Justin Mason wrote: Graham Murray writes: Theo Van Dinter [EMAIL PROTECTED] writes: Doesn't SA have at least 3 of those already? Razor, DCC, and Pyzor. Not quite. Those show how many times *others* have seen it, not how many times *I* have seen it. Also, these have hysteresis so if you are unfortunately to be at the start of the spam run and receive multiple mails all with the same body then Razor, DCC and Pyzor might not help. Though if this were implemented then there would have to a whitelist for mailing lists to which multiple users have subscribed. I know that a few big organisations use a private DCC server for this purpose, with good results; doing it with a DCC server works well if you have multiple scanner machines. --j. Humm... I didn't realize DC C could be used this way... I will investigate
Re: Google Summer of Code 2007 ...
Justin Mason wrote: Theo Van Dinter writes: I'm assuming that there will be a Google Summer of Code 2007 going on, and that the ASF will be involved again. So it's a good time to start thinking about things we'd like to put up as possible projects. We still have a number of items from last year that we could use again. Anything else that we'd like people to code up? Also, any suggestions from outside the dev team? Anyone got good ideas for new SpamAssassin features that would be good to pay someone to work on for 3 months? --j. How about a How many times have I seen this message body plugin... So each time SA see's the same or similar enough message body, it increases the score.
Re: Botnet 0.7 soon
John Rudd wrote: New things: 1) BOTNET_SOHO -- If the sender's (chosen from Envelope-From, Return-Path, or From, in that order) mail domain (the part after the @ sign) resolves back to the relay's IP address, or has an MX host which resolves back to the IP address, AND the sender's mail domain does NOT match the PTR record for the relay, then we'll assume this is a small office/home office mail server. We'll exempt them from BOTNET being triggered. (note: someone suggested that this check also try to resolve the HELO string, I make a note in my code as to why this is an extremely bad idea, and have a commented out block of code there for anyone who wants to go down that path ... but, really, don't) 2) Botnet API -- want to include the Botnet.pm module in other Perl code? Maybe call check_botnet from mimedefang-filter so you can block before a message gets to SpamAssassin? I've made an API for it. The routines that SA calls use this API, so it's the _exact_same_ code. There's now an included perl program Botnet.pl which takes an IP address CLI argument, and an optional main-domain CLI argument. It will tell you which rules do and don't get triggered. It also serves as an example of using the API. (you will still need to have SpamAssassin installed in order to use Botnet.pm in this fashion, even if you're using the API in a program that doesn't call SA) 3) BOTNET_CLIENT and BOTNET are now actual rules instead of meta rules. The individual rules are still there, just with zero'd scores. You can now easily pick between 1 big rule (BOTNET doing eval:botnet()), meta rules (detailed in the file Botnet.variations.txt), or piece-meal calling of the individual checks (also detailed in Botnet.variations.txt). 4) config option: botnet_pass_trusted (all|public|private|ignore) This defaults to public. If you have any public IP addresses in your relays-trusted list, then Botnet wont trigger. Private means any private IP addresses, where that includes 127.*, 10.*, etc.. All means either of those two. Ignore means do what Botnet used to do: not even look at the trusted relays, just look past them. The idea is: if you got this from a trusted relay, we can assume it wasn't a Botnet. 5) botnet_pass_auth now looks at the trusted relays. It probably should have been doing that all along. It no longer looks at the untrusted relays. 6) Rules that get triggered now use $permsgstatus-test_log to record information. The individual rules just list [rulename,ip=$ip,hostname=$host,maildomain=$domain] or an appropriate subset of that based on which rule it is. BOTNET_CLIENT and BOTNET also include a list of sub-rule names that were triggered. So, you might see this: [botnet,ip=1.2.3.4,host=dsl-1-2-3-4.isp.net,domain=spammer.com,baddns,ipinhostname,clientwords,client] or [botnet_nordns,ip=2.3.4.5] or [botnet_soho,ip=3.4.5.6,hostname=3.4.5.6.isp.net,maildomain=non-spammer-soho.org] (once I'm more comfortable with the output, I'll probably take out the leading rule name, but for now, I'm keeping it there) 7) shawcable.net and ocn.ne.jp seem to also be botnet sources, but their hostnames don't fit any of my other patterns. Luckily, they DO fit some pattern, and it's simple enough to not need a code based rule, just a regular conventional expression based rule. I've created BOTNET_SHAWCABLE and BOTNET_OCNNEJP rules to cover these two. 8) The file Botnet.variations.txt exists now with different suggested alternative ways to do Botnet rules. 9) Botnet.credits.txt exists, but is far from complete. I think that's everything... Just need another day or two of testing before I release it. out of curiosity, which release branches of SA is supported with this plugin? the 3.1.x 3.0.x or just the 3.1.x?
Re: High CPU running SA in a VMware VM
Sammy Anderson wrote: We recently migrated our SpamAssassin installation from a physical 3.6 GHz system running RHEL 4 and SA 3.0.4 to a VMware VM (ESX 2.5.4) with RHEL 4 as the guest OS and SA 3.1.7. Each user has their own Bayes files (Berkeley DB) and these were copied from the old to the new server. Now whenever an expiry process runs on a user's database, the CPU spikes, sometimes for a minute or longer. We did not notice spikes on the old server, but it is really hammering the VM. Has anyone else experienced this problem? For now I have disabled Bayes altogether because of the unacceptable load. --SA Do you Yahoo!? Get on board. You're invited http://us.rd.yahoo.com/evt=40791/*http://advision.webevents.yahoo.com/mailbeta to try the new Yahoo! Mail. I'm no VMware expert, but it's been my experience that any kind of database should not be run in a VMware VM.
Re: Bombarded by German political spam
David B Funk wrote: Tonight our site is being bombarded by German political spam or Joe-jobbed bounce fall-out. So far it appears to all be coming from trojaned PCs. Other than the specific URLs in the messages havn't found any easily identified parts to create rules for. anybody else seeing this? Did this suddendly stop today for anyone else and now your just dealing with the NDR's?
Re: Bombarded by German political spam
Chr. von Stuckrad wrote: On Sun, May 15, 2005 at 10:59:12AM -0500, Steven Stern wrote: I received about 500 on the webmaster account. Now we know what sober was all about. I see *no* connection to any Virus or Trojan! I got about 200 of them into a few accounts and seemingly I'm receiving more every few minutes. BUT I do *not* think it is more than 'Propaganda'! It mostly is just one URL of a genuine Article of a german Newspaper (only the 'collection' of Articles and tendency of subject making it 'political'). No attachments seem to be sent and our Mail-filter would have 'eaten' anyway all the current Sober-Viruses/Variants. (I'm pretty sure about that, I'm its admin) Stucki (postmaster at math/inf/mi.fu-berlin.de) Look at your AV logs of those sending sober.p and look at the connections sending the german political spam. you will start to see a connection. In fact I'm going through my logs right now finding the hosts which sent sober.p and starting to block those because they so far seem to be the main ones sending the political spam
Re: SpamAssassin score factors?
Lisheng Sun wrote: Could anyone here tell me how many different factors that will involve with SA? Say, IP is belong to blacklist, URL is belong to blacklist, etc. What else? Not include user-defined one. Thanks. You should probably visit http://spamassassin.apache.org/tests_3_0_x.html
Re: Barracuda's Spam firewall
Gray, Richard wrote: Anyone care to comment on how successful/effective this particular product is? (http://www.barracudanetworks.com) There is something of a major dispute going regarding whether this represents better value for mney than other solutions (including our own, self built service) If any of you fine people has any experience with this (tested it, use it, know someone else who uses it) I'd really appreciate any feedback you could give me on its pros/cons. Thanks. Richard --- This email from dns has been validated by dnsMSS Managed Email Security and is free from all known viruses. For further information contact [EMAIL PROTECTED] Barracuda devices are a spammer's dream. They make it very easy to backscatter spam. They bounce EVERYTHING causing a ton of backscatter. They seem to accept all mail, process it then bounce it back to the sender which is usually fake. Here's one such log entry from one of my postfix servers... This is just one entry of over 10,000 in the last 6 hours alone (of course I removed the recipient address): Feb 28 06:52:23 inbound1 postfix/cleanup[24781]: 3065B31A769: reject: header Subject: **Message you sent blocked by our bulk email filter** from barracuda.stcc.cc.tx.us[67.67.36.7]; from= to=[EMAIL PROTECTED] proto=ESMTP helo=barracuda.stcc.cc.tx.us: 550 Uknown User
Spamc wrapper script to bypass spamscanning for some users?
I thought I saw here a while back a script which would check to see if user wanted spam scanning or not. I've tried going through the list but alas I just can't seem to find it. Any help would be appreciated. Tim
Re: Request for spam from Kennedy-Western/kw.edu
William Stearns wrote: Good evening, all, I have a favor to ask. Kennedy Western has written in asking to be removed from the sa-blacklist - the audacity! :-) Could I trouble any of you that keep your back spam to grab any Kennedy Wester spams and send them along to [EMAIL PROTECTED] (obviously, this address is for spam only; if you have questions or want to reach me, please use [EMAIL PROTECTED])? Strings to look for are: Kennedy-Western kw.edu kennedy-western-university.net KennedyWestern@ Kennedy Western I sincerely appreciate the help. Cheers, - Bill --- It is easy to be blinded to the essential uselessness of computers by the sense of accomplishment you get from getting them to work at all. -- Douglas Adams -- William Stearns ([EMAIL PROTECTED]). Mason, Buildkernel, freedups, p0f, rsync-backup, ssh-keyinstall, dns-check, more at: http://www.stearns.org -- if it's any help I have nothing for the last 4 weeks
Re: OT Boincing Spam
What I've do now is: 1) Spam over a certain score goes to /dev/null 2) Spam under a certain score, and over a certain score go to spamtrap incase someone's looking for something. 3) Low scoring spam gets delivered the user with **SPAM** in the subject which the users have a client side rules to move those to a spam folder. That seems sane. What levels do you set? First off, I use Postfix Policyd to greylist delivery. With all the sare rules for SA 3.x, Razor, Dcc, and pyzor, I set our thresholds to: Over 15 -- /dev/null Over 9 -- /spamtrap Over 5.5 -- Rewrite Subject Most false positives fall between 5.5 6.5 (maybe 2 a day and it is usually due to a raher high score RBL). I have yet to have to into spamtrap to find good mail and I've used this for 4 months now. Just once was I given a false negative.
Re: OT Boincing Spam
1. Generate a bounce message to the envelope sender of the message, and 2. During the SMTP session, refuse to accept mail from the client, by returning a 500-series SMTP error code. Option 1 is almost always a terrible idea, unless perhaps the sender has published an SPF record and the result of an SPF check at the server is pass (but definitely not anything else, including neutral or none). Anyway, option 1 is strictly inferior to option 2, as it will always require more resources at the mail server. Option 2 is actually in my opinion a good idea, because in the case of false positives, it lets the sender know that the mail might not be read. In the case of actual spam, assuming you refuse the mail at the outermost mail relay at your organization, very often the mail is coming from a spambot that will never generate the bounce message. In other cases, the client may be an open mail relay, but such machines are very often blacklisted anyway, so I don't view causing them to send bounces as a terrible thing. Others may disagree on that point, but at any rate you are not risking getting your own server blacklisted--only the open mail relay is in danger of being blacklisted (which it should be anyway). I've been quite interested in this issue of bouncing/refusing spam messages, and so built an SMTP server that makes it easy for individual users to refuse spam at the SMTP level. (See www.mailavenger.org for details.) Recently, I have set up my account to reject with a 554 SMTP error code anything that spamassassin flags as spam, using the default threshold of 5.0, which is more aggressive than other people have been suggesting here. However, I also keep a copy of the messages I bounce, both so as to monitor how this is working out, and to build a corpus with which to train the Bayesian filter. At least anecdotally, this seems to be working well for me. When I spot check spams, I don't think I'm causing a lot of innocent people to get bounce messages. Well I've reached the point with those that bounce spam using Option 1 I block with the following bounce: 554 Tell your admin to quit bouncing spam as that type of thing does nothing but DoS innocent domains. Bouncing spam is IMHO just as big a problem as the spam itself. It seems a certain appliance named after a fish likes to bounce spam by default which has caused our server to receive over 30,000 false bounces to legitimate email addresses in less an hour yesterday. What I've do now is: 1) Spam over a certain score goes to /dev/null 2) Spam under a certain score, and over a certain score go to spamtrap incase someone's looking for something. 3) Low scoring spam gets delivered the user with **SPAM** in the subject which the users have a client side rules to move those to a spam folder. Viruses 1) Identifiable viruses go to /dev/null 2) Executeable's get quarantined in a filetrap
Re: How can I catch these messages?
Rob Blomquist wrote: I run Kmail with SA 3.0.1, and I filter by piping incoming mail to spamc. I am currently using SARE_OEM SARE_GENLSUBJ SARE_GENLSUBJ_ENG SARE_HTML1 SARE_HTML2 SARE_HEADER1 SARE_HEADER2 SARE_HTML_ENG SARE_BML SARE_FRAUD SARE_SPOOF SARE_UNSUB SARE_RANDOM SARE_TOP_200 and BOGUSVIRUS as my rulesets. All I want to do is push the scores into the spam range. And frankly I think I could lower the bar, too. Are their rulesets that might help, or custom rules that I could write, and as a single user I don't need perfection, I just want something like a 95% catch ratio instead of the 60% I am currently getting. Foobar replaces a couple of the words in the headers that I am sensitive about releasing to the net. Here are the headers for brevity: Return-Path: [EMAIL PROTECTED] Received: from 43.bevivek.com ([192.168.1.3]) by mta010.foobar.net (InterMail vM.5.01.06.06 201-253-122-130-106-20030910) with ESMTP id [EMAIL PROTECTED] for [EMAIL PROTECTED]; Fri, 19 Nov 2004 01:59:35 -0600 Received: from 43.bevivek.com (66.63.188.43) by sc009pub.foobar.net (MailPass SMTP server v1.1.1 - 121803235448JY) with SMTP id 1-995-125-995-132708-13-1100851174 for mta010.foobar.net; Fri, 19 Nov 2004 01:59:36 -0600 From: Hair Care Specialist[EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Medical Hair Restoration - A Permanent Solution Date: 19 Nov 2004 02:52:49 -0500 Message-Id: [EMAIL PROTECTED]/peno MIME-Version: 1.0 Content-Type: multipart/alternative; boundary=09845039450394qame.kjY-mkxGxhki/penoirmar X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on Timmy X-Spam-Level: *** X-Spam-Status: No, score=3.1 required=5.0 tests=ALL_NATURAL,BAYES_99, HTML_IMAGE_RATIO_04,HTML_MESSAGE autolearn=no version=3.0.1 X-UID: Status: RO X-Status: RC X-KMail-EncryptionState: N X-KMail-SignatureState: N X-KMail-MDN-Sent: --09845039450394qame.kjY-mkxGxhki/penoirmar Content-Type: text/plain; charset = ISO-8859-1 Content-Transfer-Encoding: 8bit Next: Return-Path: [EMAIL PROTECTED] Received: from lamx25.havagreayday.com ([192.168.1.2]) by mta005.foobar.net (InterMail vM.5.01.06.06 201-253-122-130-106-20030910) with ESMTP id [EMAIL PROTECTED] for [EMAIL PROTECTED]; Fri, 19 Nov 2004 00:27:28 -0600 Received: from lamx25.havagreayday.com (66.63.182.25) by sc011pub.foobar.net (MailPass SMTP server v1.1.1 - 121803235448JY) with SMTP id 3-32004-215-32004-58673-27-1100845648 for mta005.foobar.net; Fri, 19 Nov 2004 00:27:29 -0600 From: Natural Beauty[EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Welcome Gifts from Yves Rocher Date: 19 Nov 2004 01:24:22 -0500 Message-Id: [EMAIL PROTECTED]/peno MIME-Version: 1.0 Content-Type: multipart/alternative; boundary=09845039450394qame.kjY-mkxGxhki/penoirmar X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on Timmy X-Spam-Level: ** X-Spam-Status: No, score=2.3 required=5.0 tests=BAYES_99,HTML_50_60, HTML_MESSAGE,HTML_TEXT_AFTER_BODY,HTML_TEXT_AFTER_HTML,HTML_WEB_BUGS, SARE_HTML_P_JUSTIFY autolearn=no version=3.0.1 X-UID: Status: RO X-Status: RC X-KMail-EncryptionState: N X-KMail-SignatureState: N X-KMail-MDN-Sent: --09845039450394qame.kjY-mkxGxhki/penoirmar Content-Type: text/plain; charset = ISO-8859-1 Content-Transfer-Encoding: 8bit next: Return-Path: [EMAIL PROTECTED] Received: from xxx.lt ([192.168.1.4]) by mta019.foobar.net (InterMail vM.5.01.06.06 201-253-122-130-106-20030910) with ESMTP id [EMAIL PROTECTED]; Thu, 18 Nov 2004 17:27:42 -0600 Received: from xxx.lt (211.230.54.86) by sc010pub.foobar.net (MailPass SMTP server v1.1.1 - 121803235448JY) with SMTP id 2-9271-77-9271-60461-1-1100820446 for mta019.foobar.net; Thu, 18 Nov 2004 17:27:43 -0600 Received: from 197.126.123.141 by smtp.leira.no; Thu, 18 Nov 2004 23:29:34 + Message-ID: [EMAIL PROTECTED] From: Brooke Corbett [EMAIL PROTECTED] To: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Order Rolex or other Swiss watches online Date: Thu, 18 Nov 2004 19:29:03 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on Timmy X-Spam-Level: X-Spam-Status: No, score=4.5 required=5.0 tests=BAYES_99,MSGID_DOLLARS autolearn=no version=3.0.1 X-UID: Status: RO X-Status: RC X-KMail-EncryptionState: N X-KMail-SignatureState: N X-KMail-MDN-Sent: next: Return-Path: [EMAIL PROTECTED] Received: from lamx26.havagreatday.com ([192.168.1.3]) by mta013.foobar.net (InterMail vM.5.01.06.06 201-253-122-130-106-20030910) with ESMTP id [EMAIL PROTECTED] for [EMAIL PROTECTED]; Thu, 18 Nov 2004 10:58:11 -0600 Received: from lamx26.havagreatday.com (66.63.182.26) by sc009pub.foobar.net (MailPass SMTP server v1.1.1 - 121803235448JY) with SMTP id 1-995-202-995-129387-4-1100797090 for mta013.foobar.net; Thu, 18 Nov 2004 10:58:11
Re: spamd still burning CPU in 3.0.1
email builder wrote:
I hurried out and installed 3.0.1, thinking one of those memory/language
improvements mentioned in the release notes were going to be my savior...
Sadly, 3.0.1's spamd has the same CPU-intensive behavior here. I am s at
a loss; tried everything I've read... spent days reading... please, anyone
have anything more?
If spamd isn't I/O bound, my memory isn't swapping, I have no other processes
that are out of control, I can't for the life of me figure out why this is
happening.
Again, my specs:
A sample from top:
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
1401 maildrop 16 0 39744 34m 6840 R 28.3 3.4 3:04.18 spamd
spamd children average around 30% CPU, but even 50% not too unusual.
load average is around 15 to 18 during the middle of the day
And this is how I start spamd:
LANG=en_US; export LANG; TMPDIR=/tmp/spamassassin; export TMPDIR
spamd -d -q -x --max-children=5 -H /etc/razor -u maildrop -r
/var/run/spamd/spamd.pid
(also tried with -L to no avail)
/tmp/spamassassin is mounted with tmpfs
prefs/bayes/awl all in SQL, but bayes/awl not being used right now
we also run named on the same machine
if it's important, this is 3.0.1, downloaded and compiled manually (not a
CPAN install)
I have installed no custom rulesets, nothing extra beside whatever comes 100%
stock. This is a Fedora Core 2 machine (2.8P-IV hyperthreaded, 1GB RAM)
spamc is called from maildrop as such:
if ( $SIZE 262144 )
{
exception {
xfilter /usr/bin/spamc -u $LOGNAME
}
}
(also tried running inside of amavis to no avail)
Any advice or even just pointers on any more reading I can do would be highly
appreciated!
What in the world is going on? Isn't it true that spamd (beside DCC)
does
its thing w/out disk I/O? If so, what else could be chewing up so much
CPU?
I don't know - The same thing happens to me a couple of times a day, and I
only get about 350 messages per day. Today it was at 12:25p:
12:25:07 4496511804 99.13 2532 9420 65088
432884 86.93
12:25:07091 5.47 2.35 0.89 LA
When this happens, the HDD is constantly active. I'm using v2.64 with
network checks. The load average for the 21 hrs of this day is about 0.1
__
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail
Have you tried doing a force-expire on your bayes db?
I found this helped me. Disabling autoexpire, and twice a day running
sa-learn --force-expire
SMP and Make
what are the best options to use when building SpamAssassin on an SMP system? and during which step do I use those switches?
Re: SMP and Make
Theo Van Dinter wrote: On Mon, Oct 04, 2004 at 05:24:02PM -0400, Tim B wrote: what are the best options to use when building SpamAssassin on an SMP system? and during which step do I use those switches? You don't really get any benefit out of the -j switch in our make. There's only 1 thing which gets compiled, and a handful of file/copy commands. But make -j 2 (or whatever number) will do it. Thanks I won't sweat it then, was just looking to make the most out of a nice shiny new 4way server
