verifying DNSBLs
How can I verify whether my system is really using DNSBLs? From what I have read all I need to do is install Net::DNS. I have read as well that if the filtering gateway is running a local nameserver that it should not be pointing to itself otherwise the DNSBLs mechanism will fail. And is there any difference in performance between implementing them via SA or the MTA? Thank you. __ Post your free ad now! http://personals.yahoo.ca
Re[2]: Blank Message Rule
Hello Loren, Stuart, Monday, December 6, 2004, 9:27:52 PM, you wrote: LW Most of the empty spams also lack a To: address, although LW they may have a From. I've found that checking for missing body, LW missing subject, and missing To: is pretty accurate. LW One could probably argue that a missing To: all by itself was LW reason to toss the mail, but I haven't tried a mass-test to see LW what that would do. Found in SARE's 70_sare_header3.cf header__SARE_TO_NONE To =~ /^UNSET$/ [if-unset: UNSET] header__SARE_CC_NONE Cc =~ /^UNSET$/ [if-unset: UNSET] meta SARE_TOCC_NONE __SARE_TO_NONE __SARE_CC_NONE describe SARE_TOCC_NONE No To header found in email score SARE_TOCC_NONE 0.491 #hist SARE_TOCC_NONE Originally submitted by Bob Menschel #counts SARE_TOCC_NONE 728s/45h of 71334 corpus (43633s/27701h RM) 10/03/04 #counts SARE_TOCC_NONE 168s/58h of 17050 corpus (14617s/2433h MY) 08/08/04 #counts SARE_TOCC_NONE 49s/3h of 38398 corpus (14914s/23484h JH) 08/14/04 TM2 SA3.0-pre2 Hits a lot of ham. Enough spam to make it worth while for systems not too tight on system resources. Bob Menschel
Re: can spamd be told what domains are local for spamc -u?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jason Haar writes: I'm the author of the Qmail content filter Qmail-Scanner, and currently it calls spamc as spamc -u [EMAIL PROTECTED] so as to help out the sites doing per-user SA configs. I've assumed that anyone wanting to do this would be using SQL backends (so requiring them to refer to local accounts as [EMAIL PROTECTED] is fine) - but apparently I presumed too much! Some are just interested in standard old /home/$USER/.spamassassin/ style lookups. Now calling spamc -u [EMAIL PROTECTED] doesn't work for them as there is no local username called [EMAIL PROTECTED]. So I could add yet another feature to Qmail-Scanner where it will strip back to the username - or SpamAssassin could. I don't mind either way - it's just that I wonder if this is also an issue for other SA-integrated MTAs (milter, postfix), so thought I'd post it out for comment? Maybe others can suggest another way of doing it? [Let's not dwell on the fact that spamd may have to run as root for this mode to work...] Hi Jason! There's a general problem, though -- which is that SpamAssassin doesn't have the information to deal with just one of those cases. Take a look. So there's two items of data: - RCPT TO address ([EMAIL PROTECTED]) - username (user) Now, we could add code to SpamAssassin to assume that [EMAIL PROTECTED] translates to the username user, but in reality *it may not* -- the MTA configuration could have an alias that translates it to username jm instead of user. So in that case, it makes more sense to wait for the MTA to translate it, and let the MTA pass on the 'real', fully-alias-resolved username instead. - --j. -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Exmh CVS iD8DBQFBtlTfMJF5cimLx9ARAlGhAKCykTUGmqJ/Fxey0++0AwCiOyjmoQCguoeK 7sGO4hPDb4QTNTVf+B5Nmw8= =GmI5 -END PGP SIGNATURE-
Re: verifying DNSBLs
Peter Matulis wrote: How can I verify whether my system is really using DNSBLs? If it's using them, many of your spams will show up with BL tests hitting in the header. You can also take a nice juicy spam and feed it to spamassassin with the -D flag and watch the trace messages. Best regards, Mojo -- Morris Jones Monrovia, CA http://www.whiteoaks.com Old Town Astronomers: http://www.otastro.org
sa-stats.pl - Syslog Error
I'm trying to run sa-stats.pl on my spamd logs and get this on every line it parses WARNING: line not in syslog format. Spamd is run with these options spamd -d -i interface -u spamd -s /var/log/spamd.log. sa-stats.pl finds the logfile automatically and begins parsing it but generates the errors above. In the end it shows the stats but shows that no spam has ever been processed, all percentages are 0. Am I logging spamd improperly? - James
Re: ARGH!!! Why the *#%^$* is this tagged ALL_TRUSTED???
On Tue, 2004-12-07 at 17:22 -0600, David B Funk wrote:
On Tue, 7 Dec 2004, Thomas Cameron wrote:
Hrm - that makes a lot of sense. I am using spamass-milter (the latest
from CVS as of about a week ago).
I actually have the following at the bottom of my sendmail.mc:
INPUT_MAIL_FILTER
(`clmilter',`S=local:/var/run/clamav/clmilter.sock,F=,T=S:4m;R:4m')dnl
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=,
T=C:15m;S:4m;R:4m;E:10m')dnl
define(`confMILTER_MACROS_CONNECT',`b, j, _, {daemon_name}, {if_name},
{if_addr}')dnl
INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-
greylist.sock')dnl
define(`confMILTER_MACROS_CONNECT', `j, {if_addr}')dnl
define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}')dnl
define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}')dnl
I just realized I have two confMILTER_MACROS_CONNECT definitions. I
don't think
that that would cause this but I need to address this tomorrow after
I've slept some. :-)
Thomas
Sorry, but that second confMILTER_MACROS_CONNECT -IS- what is causing
you all your grief.
In the m4 macro processing, last man wins, so that second
confMILTER_MACROS_CONNECT def is preventing sendmail from passing the
_, macro to your milter which causes it to not feed SA a valid
'Received:' header.
Thanks a million for educating me on that - I have fixed it, rebuilt
senmail.cf and restarted the milters and sendmail. I'm very interested
to see how that changes things.
Warmest regards,
Thomas
spamassassin and spamd/spamc
hi all, i'm a little confused with spamassassin/spamd/spamc and i hope somebody can make this clear to me. i have the problem that functions of spamd/spamc aren't supported by spamassassin. for example: i use the bayes with mysql that works fine with spamd/spamc and spamassassin. if i want to force spamassassin to lookup user_scores in a mysql database it only works with spamd/spamc, not with spamassassin. is this a known issue or just a bug? my userpref table looks like follows CREATE TABLE `userpref` ( `username` varchar(100) NOT NULL default '', `preference` varchar(30) NOT NULL default '', `value` varchar(100) NOT NULL default '', `prefid` int(11) NOT NULL auto_increment, PRIMARY KEY (`prefid`), KEY `username` (`username`) ) TYPE=MyISAM AUTO_INCREMENT=4 ; setup for spamassassin in local.cf /etc/spamassassin/local.cf user_scores_dsn DBI:mysql:spamassassin:localhost:3306 user_scores_sql_username spamassassin user_scores_sql_password password user_scores_sql_table userpref any hints appreciated :) regards, werner
Re: New rules
Matthew Newton wrote:
Hello,
I've recently installed SA 3.0.1, and found some junk was
getting through with scores too low for my liking, especially before the
URLs made it into SURBL. I've put together a few rules to match some
of these that you might find interesting.
They are:
Finally, a string of words (more than 15 here) that all begin with a
capital letter, and no punctuation (I'm only testing this one at the
moment, hence the low score):
body UOLCC_CAPWORD_TEST /([A-Z][a-z]{3,}\s{1,2}){15,}/s
describe UOLCC_CAPWORD_TEST String of words that all begin with caps letter
score UOLCC_CAPWORD_TEST 0.1
Hope these are of use to someone. If anyone can show me that they are
likely to pick up false positives, I'd be most grateful.
This will likely trigger on several airline ticket confirmation messages
which, for some unknown highly scientific reason, are always sent all caps.
Alex
Re: [SPAM-TAG] Further URIDNSBL problems..
t/dnsbl.Bareword found in conditional at t/dnsbl.t line 15. Not found: P_2 = dns:134.88.73.210.dnsbltest.spamassassin.org [127.0.0.4] # Failed test 1 in t/SATest.pm at line 530 Not found: P_7 = dns:134.88.73.210.sb.dnsbltest.spamassassin.org?type=TXT # Failed test 2 in t/SATest.pm at line 530 fail #2 Not found: P_4 = dns:14.35.17.212.dnsbltest.spamassassin.org [127.0.0.1, 127.0.0.1] # Failed test 3 in t/SATest.pm at line 530 fail #3 Not found: P_3 = dns:18.13.119.61.dnsbltest.spamassassin.org [127.0.0.12] # Failed test 4 in t/SATest.pm at line 530 fail #4 Not found: P_5 = dns:226.149.120.193.dnsbltest.spamassassin.org [127.0.0.1] # Failed test 5 in t/SATest.pm at line 530 fail #5 Not found: P_1 = dns:98.3.137.144.dnsbltest.spamassassin.org [127.0.0.2] # Failed test 6 in t/SATest.pm at line 530 fail #6 Not found: P_6 = dns:example.com.dnsbltest.spamassassin.org [127.0.0.2] # Failed test 7 in t/SATest.pm at line 530 fail #7 Not found: P_15 = DNSBL_RHS # Failed test 8 in t/SATest.pm at line 530 fail #8 Not found: P_17 = DNSBL_SB_FLOAT # Failed test 9 in t/SATest.pm at line 530 fail #9 Not found: P_18 = DNSBL_SB_STR # Failed test 10 in t/SATest.pm at line 530 fail #10 Not found: P_16 = DNSBL_SB_TIME # Failed test 11 in t/SATest.pm at line 530 fail #11 Not found: P_10 = DNSBL_TEST_DYNAMIC # Failed test 12 in t/SATest.pm at line 530 fail #12 Not found: P_12 = DNSBL_TEST_RELAY # Failed test 13 in t/SATest.pm at line 530 fail #13 Not found: P_11 = DNSBL_TEST_SPAM # Failed test 14 in t/SATest.pm at line 530 fail #14 Not found: P_8 = DNSBL_TEST_TOP # Failed test 15 in t/SATest.pm at line 530 fail #15 Not found: P_9 = DNSBL_TEST_WHITELIST # Failed test 16 in t/SATest.pm at line 530 fail #16 Not found: P_14 = DNSBL_TXT_RE # Failed test 17 in t/SATest.pm at line 530 fail #17 Not found: P_13 = DNSBL_TXT_TOP # Failed test 18 in t/SATest.pm at line 530 fail #18 t/dnsbl.FAILED tests 1-18 Failed 18/22 tests, 18.18% okay I did some looking, and came up with a previous thread about this at: http://archive.netbsd.se/?ml=spamassassin-usersa=2004-08t=282748 The resolution here was to update Net::DNS. Obviously, I've done that, as well as making sure Digest::SHA1 was in, and still I get these errors. On the perl side, is there anything I need to do to make sure they're working? CPAN says the latest versions are installed (I made doubly sure by manualling installing Net::DNS by hand), but it's just not working. Any pointers for where to look for more specific error messages would be appreciated, as well. I don't know why theses are failing, they just are. To recap, DNSBL worked when I ran 2.6. After I up'd to 3.0.1, they stopped working. SA -D reported timeouts at 15 seconds. I upped it to 30 seconds, and now it says 'complete' at 17 seconds, but still does not mark up messages that it should. Thanks! -- Matthew 'Shandower' Romanek IDS Analyst
URIDNSBL on freebsd?
How to configure URIDNSBL on Freebsd? It does not seem to work by default. -Andrew
Re: spamassassin and spamd/spamc
On Wed, Dec 08, 2004 at 12:11:53PM +0100, Werner Detter wrote: if i want to force spamassassin to lookup user_scores in a mysql database it only works with spamd/spamc, not with spamassassin. is this a known issue or just a bug? my userpref table looks like follows That is how it works, period. You can read all about them here: http://www.apache.org/~parker/presentations/ Michael pgpTRm0ATN0dX.pgp Description: PGP signature
Re: New rules
On Wed, Dec 08, 2004 at 02:22:07PM +0100, Alex Broens wrote:
Matthew Newton wrote:
I've recently installed SA 3.0.1, and found some junk was
getting through with scores too low for my liking, especially before the
URLs made it into SURBL. I've put together a few rules to match some
of these that you might find interesting.
They are:
Finally, a string of words (more than 15 here) that all begin with a
capital letter, and no punctuation (I'm only testing this one at the
moment, hence the low score):
body UOLCC_CAPWORD_TEST /([A-Z][a-z]{3,}\s{1,2}){15,}/s
describe UOLCC_CAPWORD_TEST String of words that all begin with caps
letter
score UOLCC_CAPWORD_TEST 0.1
Hope these are of use to someone. If anyone can show me that they are
likely to pick up false positives, I'd be most grateful.
This will likely trigger on several airline ticket confirmation messages
which, for some unknown highly scientific reason, are always sent all caps.
Do they send out e-mails with Each Word Starting With A Capital Letter
with no punctuation between 15 words and all words longer than 3
letters?
I would expect perhaps everything in capitals, but not the above?
Thanks
Matthew
--
Matthew Newton [EMAIL PROTECTED]
UNIX Systems Administrator, Network Support Section,
Computer Centre, University of Leicester,
Leicester LE1 7RH, United Kingdom
RE: verifying DNSBLs
-Original Message- From: Peter Matulis [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 07, 2004 6:55 PM Subject: verifying DNSBLs And is there any difference in performance between implementing them via SA or the MTA? Thank you. As far as I know, most uses of DNSBLs at the MTA level are for rejecting message prior to taking any other action on them (i.e., accepting the message and then scanning with SA, etc.). So, if you use BLs at the MTA level, you're going to cut down on the number of messages accepted by the server for further action. If you implement via SA, you've already accepted the message for delivery and are just using the BL information to adjust message scoring. It's really a matter of taste, are you more willing to reject messages at the MTA layer (prior to accepting) knowing that there are bound to be FPs on the DNSBLs? Or, are you more willing to accept all messages and let the end user do what they wish with the messages? Hope this helps. -Joe K.
Feature Request: Whitelist_DNSRBL
OK, we know that the popular domains like yahoo.com and such are hard coded into SA to be skipped on DNSRBL lookups. But it would be great to have a function to add more locally. Thinking one step bigger, it would be even better to feed this a file. This way maybe SURBL can create a file for the top hit legit domains. Then using SARE and RDJ, people could update that. This would reduce a lot of traffic and time. This might also help with the mysterious bug we have seen where some local domains are being flagged as SURBL hit, when they aren't in SURBL. Perhaps whitelisting local domains so they are skipped would do away with this. Thoughts, suggestions, or coffee? Chris Santerre System Admin and SARE Ninja http://www.rulesemporium.com http://www.surbl.org 'It is not the strongest of the species that survives, not the most intelligent, but the one most responsive to change.' Charles Darwin
Re: Feature Request: Whitelist_DNSRBL
At 10:17 AM 12/8/2004 -0500, Chris Santerre wrote: OK, we know that the popular domains like yahoo.com and such are hard coded into SA to be skipped on DNSRBL lookups. But it would be great to have a function to add more locally. Um. They are?? AFAIK there are absolutely no whitelists to the DNSRBLs in SA itself. Don't confuse the EXISTING_DOMAINS list in DNS.pm with a whitelist. That's actually a list of domains that are used to test if your DNS is working if you don't have dns_available set to yes. SA does a quick MX query for one of the domains in the list, and if it gets an answer, it knows it's working... However, I do agree it would be nice to be able to have a DNSBL whitelist capability, if for no other reason than fixing any listings that might cause short-term FPs. Thinking one step bigger, it would be even better to feed this a file. This way maybe SURBL can create a file for the top hit legit domains. Then using SARE and RDJ, people could update that. This would reduce a lot of traffic and time. Wait, now you're bringing SURBL into this.. are you talking normal DNSRBLS, or URIDNSBLS? Or both? Was the whitelist you were referring to really the SURBL server-side whitelist? This might also help with the mysterious bug we have seen where some local domains are being flagged as SURBL hit, when they aren't in SURBL. Perhaps whitelisting local domains so they are skipped would do away with this. Agreed.. It would provide users a short-term fix, although really the problem does need to be rooted out at some point.. Thoughts, suggestions, or coffee? All of the above?
RE: sa-stats.pl - Syslog Error
By default, the sa-stats.pl uses the log file /var/log/maillog You need to tell sa-stats to use a different log Example: ./sa-stats.pl -l /var/log/spamd.log -s midnight -e now Steve -Original Message- From: James [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 07, 2004 10:29 PM To: users@spamassassin.apache.org Subject: sa-stats.pl - Syslog Error I'm trying to run sa-stats.pl on my spamd logs and get this on every line it parses WARNING: line not in syslog format. Spamd is run with these options spamd -d -i interface -u spamd -s /var/log/spamd.log. sa-stats.pl finds the logfile automatically and begins parsing it but generates the errors above. In the end it shows the stats but shows that no spam has ever been processed, all percentages are 0. Am I logging spamd improperly? - James
Re: New rules
Matthew Newton wrote:
On Wed, Dec 08, 2004 at 02:22:07PM +0100, Alex Broens wrote:
Matthew Newton wrote:
I've recently installed SA 3.0.1, and found some junk was
getting through with scores too low for my liking, especially before the
URLs made it into SURBL. I've put together a few rules to match some
of these that you might find interesting.
They are:
Finally, a string of words (more than 15 here) that all begin with a
capital letter, and no punctuation (I'm only testing this one at the
moment, hence the low score):
body UOLCC_CAPWORD_TEST /([A-Z][a-z]{3,}\s{1,2}){15,}/s
describe UOLCC_CAPWORD_TEST String of words that all begin with caps
letter
score UOLCC_CAPWORD_TEST 0.1
Hope these are of use to someone. If anyone can show me that they are
likely to pick up false positives, I'd be most grateful.
This will likely trigger on several airline ticket confirmation messages
which, for some unknown highly scientific reason, are always sent all caps.
Do they send out e-mails with Each Word Starting With A Capital Letter
with no punctuation between 15 words and all words longer than 3
letters?
I would expect perhaps everything in capitals, but not the above?
Yep... all in capitals, not starting only.
goofed it.
Alex
Re: Feature Request: Whitelist_DNSRBL
Chris Santerre wrote: Was the whitelist you were referring to really the SURBL server-side whitelist? Yes! But local SURBL whitelists are needed to reduce traffic and time. I'd much rather see SURBL respond with 127.0.0.0 with a really large TTL for white listed domains. Any sensible setup will run a local DNS cache which will take care of the load and time issue. Daryl
SA auto-learn question
We have been using SA for over a year now. It has been great to us. I just upgraded both of our filters to SA 3.0.1 last night. I just have one question. When using the auto learn command, how do you get spam messages to the spam filter, in order to run auto learn? For example, our spam filter forwards clean email to our mail server. Users read the mail, some spam will get through. What is the best way for them to get the spam messages to me, so I can run auto learn on them? Thanks Shane
Re: Feature Request: Whitelist_DNSRBL
On Wed, Dec 08, 2004 at 10:26:15AM -0500, Matt Kettler wrote: At 10:17 AM 12/8/2004 -0500, Chris Santerre wrote: OK, we know that the popular domains like yahoo.com and such are hard coded into SA to be skipped on DNSRBL lookups. But it would be great to have a function to add more locally. Um. They are?? AFAIK there are absolutely no whitelists to the DNSRBLs in SA itself. I'm not sure if DNSRBLs are the same as URIDNSBLs, or if this was the intent of the original poster, but SA 3.0.1 added the configuration option 'uridnsbl_skip_domain' which does not check the urls in emails that are from the listed domains. The following domains have been added to this list by default in 25_uribl.cf: 4at1.com 5iantlavalamp.com adobe.com advertising.com afa.net akamai.net akamaitech.net amazon.com aol.com apache.org apple.com arcamax.com atdmt.com att.net bbc.co.uk bfi0.com bravenet.com bridgetrack.com cc-dt.com chase.com cheaptickets.com chtah.com citibank.com citizensbank.com classmates.com click-url.com cnet.com cnn.com com.com comcast.net constantcontact.com debian.org directtrack.com doubleclick.net dsbl.org dsi-enews.net e-trend.co.jp earthlink.net ebay.com ebaystatic.com ed10.net ed4.net edgesuite.net ediets.com exacttarget.com extm.us flowgo.com geocities.com gmail.com go.com google.com grisoft.com gte.net hitbox.com hotbar.com hotmail.com hyperpc.co.jp ibm.com ientrymail.com incredimail.com investorplace.com jexiste.fr joingevalia.com m0.net mac.com macromedia.com mail.com marketwatch.com mcafee.com mediaplex.com messagelabs.com microsoft.com monster.com moveon.org msn.com mycomicspage.com myweathercheck.com netatlantic.com netflix.com norman.com nytimes.com p0.com pandasoftware.com partner2profit.com paypal.com pcmag.com plaxo.com postdirect.com prserv.net quickinspirations.com redhat.com rm04.net roving.com rr.com rs6.net sbcglobal.net sears.com sf.net shockwave.com si.com sitesolutions.it smileycentral.com sourceforge.net spamcop.net speedera.net sportsline.com sun.com suntrust.com terra.com.br tiscali.co.uk topica.com ual.com uclick.com unitedoffers.com ups.com verizon.net w3.org washingtonpost.com weatherbug.com xmr3.com yahoo.co.uk yahoo.com yahoogroups.com yimg.com yourfreedvds.com Mike -- /-\ | Michael Barnes [EMAIL PROTECTED] | | UNIX Systems Administrator | | College of William and Mary | | Phone: (757) 879-3930 | \-/
Re: Feature Request: Whitelist_DNSRBL
- Original Message - From: Daryl C. W. O'Shea [EMAIL PROTECTED] Was the whitelist you were referring to really the SURBL server-side whitelist? Yes! But local SURBL whitelists are needed to reduce traffic and time. I'd much rather see SURBL respond with 127.0.0.0 with a really large TTL for white listed domains. Any sensible setup will run a local DNS cache which will take care of the load and time issue. I agree, and have suggested a whitelist SURBL several times on the SURBL discussion list, but it has always fallen on deaf ears - nary a response. It would be nice if someone would at least respond as to why this is not a reasonable suggestion. Bill
RE: Feature Request: Whitelist_DNSRBL
-Original Message- From: Bill Landry [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 08, 2004 11:04 AM To: users@spamassassin.apache.org; [EMAIL PROTECTED] Subject: Re: Feature Request: Whitelist_DNSRBL - Original Message - From: Daryl C. W. O'Shea [EMAIL PROTECTED] Was the whitelist you were referring to really the SURBL server-side whitelist? Yes! But local SURBL whitelists are needed to reduce traffic and time. I'd much rather see SURBL respond with 127.0.0.0 with a really large TTL for white listed domains. Any sensible setup will run a local DNS cache which will take care of the load and time issue. I agree, and have suggested a whitelist SURBL several times on the SURBL discussion list, but it has always fallen on deaf ears - nary a response. It would be nice if someone would at least respond as to why this is not a reasonable suggestion. Well we have talked about it and didn't come up with a solid answer. The idea would cause more lookups and time for those who don't cache dns. We do have a whitelist that our private research tools do poll. The idea is that if it isn't in SURBL then it is white. This also puts more work to the already overworked contributors. ;) --Chris
Re: SA auto-learn question
On Wed, Dec 08, 2004 at 10:57:44AM -0500, shane mullins wrote: We have been using SA for over a year now. It has been great to us. I just upgraded both of our filters to SA 3.0.1 last night. I just have one question. When using the auto learn command, how do you get spam messages to the spam filter, in order to run auto learn? For example, our spam filter forwards clean email to our mail server. Users read the mail, some spam will get through. What is the best way for them to get the spam messages to me, so I can run auto learn on them? First, that last question is kinda confusing. Autolearning is just that. Spam and ham that meet sufficiently high or low scores with some distribution of the scoring between the headers and the body are autolearned. What you are asking about is manual learning in the event of an error by SA. Unfortunately, once the mail gets to a user (depending on their computer skills), its pretty much gone. What I mean, is that to feed the mails back to the bayes learning process once it has gotten to the user is that the user somehow has to get _the whole message, headers and all_ somewhere to be fed to sa-learn. I work with pretty bright people, many are in graduate school for computer oriented, but I would never ask for an original mail back from one of them because it would be too difficult with the multitude of (usually broken) mail readers out there. Part of the email rfc (I guess 822, not sure if its actually in another rfc) contains a feature called resending a message or similar. My mailer, mutt, has this feature, and it describes it as: With resend-message, mutt takes the current message as a template for a new message. This function is best described as recall from arbitrary folders. It can conveniently be used to forward MIME messages while preserving the original mail structure. Note that the amount of headers included here depends on the value of the $weed variable. This function is also available from the attachment menu. You can use this to easily resend a message which was included with a bounce message as a message/rfc822 body part. Unfortunetly, I have only heard of one other old obsolete mailer that has this feature. I'm sure there are others, but its not too common. If you have a userbase with uniform mail programs, this could be easier to implement. Otherwise, I would not even attempt it. Mike -- /-\ | Michael Barnes [EMAIL PROTECTED] | | UNIX Systems Administrator | | College of William and Mary | | Phone: (757) 879-3930 | \-/
Re: New rules
Getting off topic here, but the all caps is probably a holdover from the old SABRE airline reservation system which used a 6-bit codeset to reduce the transmission time on their (at the time) slow data links. Actually it was because the SABRE machines also used a 5-bit code set (and still largely do). (Assuming of course SABRE was the Sperry rather than IBM reservation system; I forget which was which.) Loren
RE: spamd takes a long time to scan
On Mon, Dec 06, 2004 at 10:27:29AM -0600, Jon Dossey wrote: Wow! 0.1 seconds, now that's fast! Then I saw this: tests=none I guess it would be fast if it doesn't have to really *do* anything! tests=none just mean that it didn't hit any rules, not that it didn't run any rules. You can try sending a GTUBE through. So have we really narrowed it down at all? We know that spamd is taking a long time during some test it performs, but we don't really know if its related to RBL checks. Well, we've proven it's network related, but haven't narrowed it down to which network check. -D may help, I would probably try slowly reenabling things. ie: remove -L but disable razor, dcc, pyzor, URIBL, etc. Also, check to see if you have any timeouts set to 15s. The default RBL timeout is 15s, so it could be that, but most of the queries would have to fail to actually get to 15s (as queries return, the timeout gets lower). First off, I'd like to thank you again for your and others help, Theo. I would have been completely lost without your help and suggestions, and I really appreciate your patience. I finally got a chance to take a look at this with debugging enabled (been very busy here), and noticed the following output: Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for rfci_envfrom after 15 seconds Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for NO_DNS_FOR_FROM after 15 seconds Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for ahbl after 15 seconds Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for NO_DNS_FOR_FROM after 15 seconds Also worth noting: Dec 8 10:07:58 dhgsrv17 spamd[1880]: debug: URIDNSBL: domains to query: Dec 8 10:07:58 dhgsrv17 spamd[1880]: debug: is Net::DNS::Resolver available? yes Dec 8 10:07:58 dhgsrv17 spamd[1880]: debug: Net::DNS version: 0.45 Now, I believe the problem *may* be due to the fact that I'm relaying off this host by connecting from my workstation on the SMTP port, with no reverse DNS entry. Is it possible the resolver is timing trying to reverse my private IP address to a valid host name? Thanks, .jon __ The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers.
Re: Feature Request: Whitelist_DNSRBL
On Wed, 8 Dec 2004 08:03:35 -0800, Bill Landry [EMAIL PROTECTED] wrote: I agree, and have suggested a whitelist SURBL several times on the SURBL discussion list, but it has always fallen on deaf ears - nary a response. It would be nice if someone would at least respond as to why this is not a reasonable suggestion. The floor in offering a DNS based whitelist is that it encourages people to place a negative score on it. The problem with this is that spammers can poison messages with whitelisted domains, thereby bypassing the power of the SURBL The concept of Whitelist in the SURBL world is more of an Exclusion List as in we exclude these domains from being listed rather than we consider the presence of these domains in an email to be a good sign of ham. An excluded domain is therefore ignored in all data and not allocated a score positively or negatively, so trying to poison a message with whitelisted domains is therefore pointless. I think we either need to look at a DNS version of uridnsbl_skip_domain with long TTL's or we should look at releasing a .cf file. I personally think the more proper implementation may be the DNS based version in order to avoid BigEvil type situations. Cheers! -- Regards, David Hooton
SpamAssassin memory usage
Hi! I have three Sun Fire servers running Solaris 9 and SpamAssassin 3.0.1. SpamAssassin memory usage seems grow a lot. The machines have 2Gb RAM each, and I have an hourly cron job that restarts SpamAssassin if more than 1.5Gb memory is used (if the machine starts swapping, performance goes through the floor). So, the questions are a) Does SpamAssassin normally use a this much memory? b) If so, how much can I expect it to use? c) If not, does anyone know of any bugs in perl (5.8.0) or Solaris that could cause this and finally d) is restarting SpamAssassin an acceptable thing to do to stop it swapping? The machines each process around 8 mails/day and we have something like 25000 users. Thanks for any help/advice you can give. Matthew -- Matthew Newton [EMAIL PROTECTED] UNIX Systems Administrator, Network Support Section, Computer Centre, University of Leicester, Leicester LE1 7RH, United Kingdom
Re: [SPAM-TAG] Further URIDNSBL problems..
FYI (and for future list-searchers), the problem with URIDNSBL appearing to work but not actually scoring was because the host's resolv.conf included 127.0.0.1, which apparently something doesn't like. Peter Matulis just sent an unrelated email to the list mentioning this, and after checking it out and pointing hosts at each other instead of themselves, everything works fine. Ta-Da! Instantly my false-negative rate dropped. -- Matthew 'Shandower' Romanek IDS Analyst
RE: [SPAM-TAG] Further URIDNSBL problems..
FYI (and for future list-searchers), the problem with URIDNSBL appearing to work but not actually scoring was because the host's resolv.conf included 127.0.0.1, which apparently something doesn't like. I find it pretty hard to believe it couldn't resolve off itself. Have you checked your firewall rules, and your named.conf to see if you've allowed-query 127.0.0.1 in your options statement? Have you tried resolving anything locally, while ssh'ed into the box? What about using another IP address bound to a NIC on the machine, that named is configured to answer on? Thanks, .jon __ The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers.
Re: Feature Request: Whitelist_DNSRBL
- Original Message - From: Chris Santerre [EMAIL PROTECTED] -Original Message- From: Bill Landry [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 08, 2004 11:04 AM To: users@spamassassin.apache.org; [EMAIL PROTECTED] Subject: Re: Feature Request: Whitelist_DNSRBL - Original Message - From: Daryl C. W. O'Shea [EMAIL PROTECTED] Was the whitelist you were referring to really the SURBL server-side whitelist? Yes! But local SURBL whitelists are needed to reduce traffic and time. I'd much rather see SURBL respond with 127.0.0.0 with a really large TTL for white listed domains. Any sensible setup will run a local DNS cache which will take care of the load and time issue. I agree, and have suggested a whitelist SURBL several times on the SURBL discussion list, but it has always fallen on deaf ears - nary a response. It would be nice if someone would at least respond as to why this is not a reasonable suggestion. Well we have talked about it and didn't come up with a solid answer. The idea would cause more lookups and time for those who don't cache dns. We do have a whitelist that our private research tools do poll. The idea is that if it isn't in SURBL then it is white. This also puts more work to the already overworked contributors. ;) Actually, I was thinking of the whitelist that Jeff has already compiled at http://spamcheck.freeapp.net/whitelist-domains.sort (currently over 66,500 whitelisted domains). If you set a long TTL on the query responses, it would certainly cut down on follow-up queries for anyone that is running a caching dns. It would also be a lot less resource intensive then trying to run a local whitelist.cf of over 66,500 whitelisted domains. Anyway, just a thought... Bill
RE: spamd takes a long time to scan
On Mon, Dec 06, 2004 at 10:27:29AM -0600, Jon Dossey wrote: Wow! 0.1 seconds, now that's fast! Then I saw this: tests=none I guess it would be fast if it doesn't have to really *do* anything! tests=none just mean that it didn't hit any rules, not that it didn't run any rules. You can try sending a GTUBE through. So have we really narrowed it down at all? We know that spamd is taking a long time during some test it performs, but we don't really know if its related to RBL checks. Well, we've proven it's network related, but haven't narrowed it down to which network check. -D may help, I would probably try slowly reenabling things. ie: remove -L but disable razor, dcc, pyzor, URIBL, etc. Also, check to see if you have any timeouts set to 15s. The default RBL timeout is 15s, so it could be that, but most of the queries would have to fail to actually get to 15s (as queries return, the timeout gets lower). First off, I'd like to thank you again for your and others help, Theo. I would have been completely lost without your help and suggestions, and I really appreciate your patience. I finally got a chance to take a look at this with debugging enabled (been very busy here), and noticed the following output: Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for rfci_envfrom after 15 seconds Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for NO_DNS_FOR_FROM after 15 seconds Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for ahbl after 15 seconds Dec 8 10:08:13 dhgsrv17 spamd[1880]: debug: DNS: timeout for NO_DNS_FOR_FROM after 15 seconds Also worth noting: Dec 8 10:07:58 dhgsrv17 spamd[1880]: debug: URIDNSBL: domains to query: Dec 8 10:07:58 dhgsrv17 spamd[1880]: debug: is Net::DNS::Resolver available? yes Dec 8 10:07:58 dhgsrv17 spamd[1880]: debug: Net::DNS version: 0.45 Now, I believe the problem *may* be due to the fact that I'm relaying off this host by connecting from my workstation on the SMTP port, with no reverse DNS entry. Is it possible the resolver is timing trying to reverse my private IP address to a valid host name? I think I just answered my own question. I relayed from a host that it easily resolved, and still had the same timeout issues running the RBL tests: Dec 8 10:28:35 dhgsrv17 spamd[1883]: debug: RBL: success for 0 of 4 queries Dec 8 10:28:35 dhgsrv17 spamd[1883]: debug: DNS: timeout for rfci_envfrom after 15 seconds Dec 8 10:28:35 dhgsrv17 spamd[1883]: debug: DNS: timeout for NO_DNS_FOR_FROM after 15 seconds Dec 8 10:28:35 dhgsrv17 spamd[1883]: debug: DNS: timeout for ahbl after 15 seconds Dec 8 10:28:35 dhgsrv17 spamd[1883]: debug: DNS: timeout for NO_DNS_FOR_FROM after 15 seconds Any idea where I should go from here? Thanks, .jon __ The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers.
Re: Feature Request: Whitelist_DNSRBL
- Original Message - From: David Hooton [EMAIL PROTECTED] On Wed, 8 Dec 2004 08:03:35 -0800, Bill Landry [EMAIL PROTECTED] wrote: I agree, and have suggested a whitelist SURBL several times on the SURBL discussion list, but it has always fallen on deaf ears - nary a response. It would be nice if someone would at least respond as to why this is not a reasonable suggestion. The floor in offering a DNS based whitelist is that it encourages people to place a negative score on it. The problem with this is that spammers can poison messages with whitelisted domains, thereby bypassing the power of the SURBL I agree, it should not be used as a HAM indicator, way too easy to abuse. I was suggesting that the whitelist be used as a way to exclude the domain from being constantly queried against the SURBL name servers. The concept of Whitelist in the SURBL world is more of an Exclusion List as in we exclude these domains from being listed rather than we consider the presence of these domains in an email to be a good sign of ham. Exactly. An excluded domain is therefore ignored in all data and not allocated a score positively or negatively, so trying to poison a message with whitelisted domains is therefore pointless. Yep, agree wholeheartedly. I think we either need to look at a DNS version of uridnsbl_skip_domain with long TTL's or we should look at releasing a .cf file. I personally think the more proper implementation may be the DNS based version in order to avoid BigEvil type situations. Indeed, my thoughts exactly. Bill
Re: [SPAM-TAG] Further URIDNSBL problems..
I find it pretty hard to believe it couldn't resolve off itself. Have you checked your firewall rules, and your named.conf to see if you've allowed-query 127.0.0.1 in your options statement? Have you tried resolving anything locally, while ssh'ed into the box? What about using another IP address bound to a NIC on the machine, that named is configured to answer on? There was never a problem resolving anything with DNS. This was an issue getting URIDNSBL in SA 3.0.1 to score correctly. See previous thread. :) -- Matthew 'Shandower' Romanek IDS Analyst
RE: Feature Request: Whitelist_DNSRBL
How about a way to use wildcards with uridnsbl_skip_domain? I'd like to be able to tell the SURBL code not to look up *.gov *.mil *.edu and even *.??.us since these are unlikely to be hosting spammer web pages. Larry
RE: Feature Request: Whitelist_DNSRBL
-Original Message- From: Rosenbaum, Larry M. [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 08, 2004 11:47 AM To: users@spamassassin.apache.org Subject: RE: Feature Request: Whitelist_DNSRBL How about a way to use wildcards with uridnsbl_skip_domain? I'd like to be able to tell the SURBL code not to look up *.gov *.mil *.edu and even *.??.us LOL we've listed a few edu so far :) LOL @ BigEvil situation , its now famous! Actually I was only saying to list the top look ups from the whitelist, not the 66,500. That is more of a research and exclusion tool. So no more then 200-300 domains. Check it every month for changes and update. I'll probably make up a .cf file and start testing it. --Chris
Re: Feature Request: Whitelist_DNSRBL
Bill Landry wrote: From: Chris Santerre [EMAIL PROTECTED] Well we have talked about it and didn't come up with a solid answer. The idea would cause more lookups and time for those who don't cache dns. It doesn't cause more lookups for anyone. A local white list file would reduces lookups at the expense of process size (and time if the white list is very large). Besides, if someone doesn't want to take the 1-5 minutes it takes to setup a local DNS cache they're probably not too interested in saving time anyway. We do have a whitelist that our private research tools do poll. The idea is that if it isn't in SURBL then it is white. This also puts more work to the already overworked contributors. ;) How so? The lookup code is already compatible as is, it's just a matter of adding the records to each of the SURBL zones... from the already existing data files. That'd take some effort, but I can't imagine it would require anything more than trivial changes... although I've been wrong before. Actually, I was thinking of the whitelist that Jeff has already compiled at http://spamcheck.freeapp.net/whitelist-domains.sort (currently over 66,500 whitelisted domains). If you set a long TTL on the query responses, it would certainly cut down on follow-up queries for anyone that is running a caching dns. It would also be a lot less resource intensive then trying to run a local whitelist.cf of over 66,500 whitelisted domains. Ditto. Even if someone isn't running a caching name server, it's highly unlikely that their ISP isn't. Daryl
how to run SA3.0.1 on a existing mailbox
I want to run spamassassin on my existing /var/mail/mymailbox and only move all the spam mail into /var/mail/spam . Is there a way to do that? thanks Andrew
RE: Feature Request: Whitelist_DNSRBL
We do have a whitelist that our private research tools do poll. The idea is that if it isn't in SURBL then it is white. This also puts more work to the already overworked contributors. ;) How so? The lookup code is already compatible as is, it's just a matter of adding the records to each of the SURBL zones... from the already existing data files. That'd take some effort, but I can't imagine it would require anything more than trivial changes... although I've been wrong before. Assuming that this whitelist would be used to LOWER the score of an email, and not just exclude them from SURBL. Then we would go thru even moreresearch before we whitelist a domain. There is a LOT of work that goes into adding a domain to our whitelist, and that is JUST for exclusion! It takes at least twice as long to see if someone is white vs black. Thats where the more work would come from. You should see some of the long threads on a single domain up for being whitelisted. Its a good thing Jeff and I have a sense of humor with eachother ;) My whole idea was skipping the lookup entirley. Why would you want to do a lookup for google even if it is cached? --Chris
Re: sa-stats.pl - Syslog Error
Like I said before it finds and reads the spam log file fine. It occurs when parsing the actual log file, it does not have trouble locating it. Any other ideas? - James Steve Dimoff wrote: By default, the sa-stats.pl uses the log file /var/log/maillog You need to tell sa-stats to use a different log Example: ./sa-stats.pl -l /var/log/spamd.log -s midnight -e now Steve -Original Message- From: James [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 07, 2004 10:29 PM To: users@spamassassin.apache.org Subject: sa-stats.pl - Syslog Error I'm trying to run sa-stats.pl on my spamd logs and get this on every line it parses WARNING: line not in syslog format. Spamd is run with these options spamd -d -i interface -u spamd -s /var/log/spamd.log. sa-stats.pl finds the logfile automatically and begins parsing it but generates the errors above. In the end it shows the stats but shows that no spam has ever been processed, all percentages are 0. Am I logging spamd improperly? - James
Re: how to run SA3.0.1 on a existing mailbox
Andrew Xiang wrote: I want to run spamassassin on my existing /var/mail/mymailbox and only move all the spam mail into /var/mail/spam . Is there a way to do that? thanks Andrew you could run spamassassin -e on the message and then make a quick script to check the return status and move the messages accordingly. -e, --exit-code Exit with a non-zero exit code if the tested message was spam SA doesnt move anything on its own so you'd have to script it. -Jim
Re: SA auto-learn question
Michael Barnes wrote: What you are asking about is manual learning in the event of an error by SA. Unfortunately, once the mail gets to a user (depending on their computer skills), its pretty much gone. *Especially* if they're using Outlook. Ugh. :( What I mean, is that to feed the mails back to the bayes learning process once it has gotten to the user is that the user somehow has to get _the whole message, headers and all_ somewhere to be fed to sa-learn. I work with pretty bright people, many are in graduate school for computer oriented, but I would never ask for an original mail back from one of them because it would be too difficult with the multitude of (usually broken) mail readers out there. Part of the email rfc (I guess 822, not sure if its actually in another rfc) contains a feature called resending a message or similar. My mailer, mutt, has this feature, and it describes it as: [snip] Pegasus Mail supports a Bounce feature that pretty much resends the message, including all existing headers and body content (MIME goop and all). The major disadvantage of this is that it produces a message with additional Received: headers, and depending on the email path other bits of the message headers may get changed. :/ Unfortunetly, I have only heard of one other old obsolete mailer that has this feature. I'm sure there are others, but its not too common. *nix-based mailers are far more likely to have this than Windows-based ones. That said, I've asked (and asked, and asked, and PLEADED with) users to forward mail as an attachment - which (from just about anything except Eudora and Outlook) gets me *exactly* the message the user receives. It's a little more work to untangle the attached message, but I've had *no* trouble feeding these into sa-learn. Right-click, forward as attachment works pretty well. -kgd -- Get your mouse off of there! You don't know where that email has been!
Re: how to run SA3.0.1 on a existing mailbox
Good day, Andrew, On Wed, 8 Dec 2004, Andrew Xiang wrote: But I have a whole mbox with 10,000 messages. How can I pipe all the messages and move them? - Original Message - From: Jim Maul [EMAIL PROTECTED] To: Andrew Xiang [EMAIL PROTECTED] Andrew Xiang wrote: I want to run spamassassin on my existing /var/mail/mymailbox and only move all the spam mail into /var/mail/spam . Is there a way to do that? you could run spamassassin -e on the message and then make a quick script to check the return status and move the messages accordingly. -e, --exit-code Exit with a non-zero exit code if the tested message was spam SA doesnt move anything on its own so you'd have to script it. ...with something like the following: ~/bin/reprocess-mailbox snip #!/bin/bash #Copyright 2004 William Stearns [EMAIL PROTECTED] #Released under the GPL if [ -z $1 ] || [ ! -r $1 ]; then echo Usage: $0 folder-to-reprocess exit 1 fi TmpFile=`mktemp -q $1.XX` if [ $? -ne 0 ]; then echo $0: Can't create temp file, exiting... exit 1 fi if [ ! -w $TmpFile ]; then echo $TmpFile unwriteable, exiting exit 1 fi echo working with $1 and $TmpFile nice formail -f -ds /usr/bin/spamc $1 $TmpFile echo Processing done. Please check $TmpFile snip This should be run on a mailbox that is _not_ actively recieving mail. At the end, you'll need to rename the tmpfile back to the original file to keep the changes, and then start your mail flowing again. Cheers, - Bill --- 5) what are people like spaf/chris rouland/lance then? a) THEY ARE THE ENEMY. WHITEHATS = ENEMY. -- http://www.blackhatbloc.org/phrack/texts/faq1.txt -- William Stearns ([EMAIL PROTECTED]). Mason, Buildkernel, freedups, p0f, rsync-backup, ssh-keyinstall, dns-check, more at: http://www.stearns.org --
Re: how to run SA3.0.1 on a existing mailbox
William Stearns wrote: Good day, Andrew, On Wed, 8 Dec 2004, Andrew Xiang wrote: But I have a whole mbox with 10,000 messages. How can I pipe all the messages and move them? - Original Message - From: Jim Maul [EMAIL PROTECTED] To: Andrew Xiang [EMAIL PROTECTED] Andrew Xiang wrote: I want to run spamassassin on my existing /var/mail/mymailbox and only move all the spam mail into /var/mail/spam . Is there a way to do that? you could run spamassassin -e on the message and then make a quick script to check the return status and move the messages accordingly. -e, --exit-code Exit with a non-zero exit code if the tested message was spam SA doesnt move anything on its own so you'd have to script it. ...with something like the following: ~/bin/reprocess-mailbox Thanks William, i knew someone would come up with this before i could :) -Jim
Re: how to run SA3.0.1 on a existing mailbox
It seem to copy all the emails into the temp file. It does not remove spam from the mbox. The purpose is to remove all the spams inside mailbox. -Andrew - Original Message - From: William Stearns [EMAIL PROTECTED] To: Andrew Xiang [EMAIL PROTECTED] Cc: Jim Maul [EMAIL PROTECTED]; ML-spamassassin-talk users@spamassassin.apache.org; William Stearns [EMAIL PROTECTED] Sent: Wednesday, December 08, 2004 12:40 PM Subject: Re: how to run SA3.0.1 on a existing mailbox Good day, Andrew, On Wed, 8 Dec 2004, Andrew Xiang wrote: But I have a whole mbox with 10,000 messages. How can I pipe all the messages and move them? - Original Message - From: Jim Maul [EMAIL PROTECTED] To: Andrew Xiang [EMAIL PROTECTED] Andrew Xiang wrote: I want to run spamassassin on my existing /var/mail/mymailbox and only move all the spam mail into /var/mail/spam . Is there a way to do that? you could run spamassassin -e on the message and then make a quick script to check the return status and move the messages accordingly. -e, --exit-code Exit with a non-zero exit code if the tested message was spam SA doesnt move anything on its own so you'd have to script it. ...with something like the following: ~/bin/reprocess-mailbox snip #!/bin/bash #Copyright 2004 William Stearns [EMAIL PROTECTED] #Released under the GPL if [ -z $1 ] || [ ! -r $1 ]; then echo Usage: $0 folder-to-reprocess exit 1 fi TmpFile=`mktemp -q $1.XX` if [ $? -ne 0 ]; then echo $0: Can't create temp file, exiting... exit 1 fi if [ ! -w $TmpFile ]; then echo $TmpFile unwriteable, exiting exit 1 fi echo working with $1 and $TmpFile nice formail -f -ds /usr/bin/spamc $1 $TmpFile echo Processing done. Please check $TmpFile snip This should be run on a mailbox that is _not_ actively recieving mail. At the end, you'll need to rename the tmpfile back to the original file to keep the changes, and then start your mail flowing again. Cheers, - Bill -- - 5) what are people like spaf/chris rouland/lance then? a) THEY ARE THE ENEMY. WHITEHATS = ENEMY. -- http://www.blackhatbloc.org/phrack/texts/faq1.txt -- William Stearns ([EMAIL PROTECTED]). Mason, Buildkernel, freedups, p0f, rsync-backup, ssh-keyinstall, dns-check, more at: http://www.stearns.org --
[Fwd: Re: FW: Any idea what happened to exit0.us]
Here's the latest info on exit0.us wiki. Sorry for the inconvenience. ---BeginMessage--- Chris Santerre wrote: LOL, what happened to exit0.us --Chris -Original Message- From: Brylski, Markus [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 08, 2004 11:25 AM To: [EMAIL PROTECTED] Subject: Any idea what happened to exit0.us Hello. I just passed by to check out the new rulesets and a rules_du_jour link of rulesemporiom (http://www.exit0.us/). Exit0.us is happy to tell that jack dunbar celebrated his 50th birthday... Do you by chance have any idea what happened to www.exit0.us? Thank you very much for your 2ct. Yours. Markus -- Markus Brylski Systemadministration Unix -- VSA GmbH Tomannweg 6 81673 München Tel. 089 / 43 18 42 67 0176 / 21 04 06 03 mailto:[EMAIL PROTECTED] www.vsa.de -- A supercomputer is a machine, that runs an endless loop in just 2 seconds. -- Unknown Hopefully you'll get this this time. I lost my job and subsequently my hosting. I've been trying to get the site back online, and I've got an older version running at the moment. All the data is as of 6/2004 I believe. I haven't been able to email any of the community since I'm running on a dynamic IP address and that is a No-No, typically that is a high spam sign as you well know. I finally got things to a point where I'm using a outbound relay from DynDns.org. I'm quasi-online now and if you know of anyone looking for an Email administrator in the Mid-Atlantic region, please let me know. AltGrendel ---End Message---
RE: [Fwd: Re: FW: Any idea what happened to exit0.us]
-Original Message- From: AltGrendel [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 08, 2004 1:50 PM To: users@spamassassin.apache.org Subject: [Fwd: Re: FW: Any idea what happened to exit0.us] Here's the latest info on exit0.us wiki. Sorry for the inconvenience. Gah! Sorry to read about all of that! I wish you the best of luck in finding work. -Joe K.
Questions about clearing LDAP preferences between runs
[Originally sent this to dev, realized it's more of a user question,
posting here.]
Hey all,
I've been doing some hacks to Mimedefang to allow per-user configuration
to be read from LDAP (using the SA ldap stuff), and have it working,
except for the fact that preferences do not get cleared between runs of
SpamAssassin. I took a look at the way Mimedefang is doing spam checking
and hacked up a test script (attached) to verify this behavior. I've also
included the .cf file I use to grab prefs out of LDAP.
For this script, I have two users defined in ldap - 'testuser1' and
'testuser2'. testuser1 has no specific sa configuration set; testuser2 has a
all_spam_to entry for '[EMAIL PROTECTED]'. The script runs 3 SA tests on the
message 'mailmessage'. The first test is run as 'testuser1', the session is
ended, it's run as testuser2, session is ended again, and then run as testuser1
again. Here's the results I get:
$ perl sa-ldap-test.pl
Initializing SpamAssassin... compiling... done.
Testing for testuser1... -2.755 hits with ALL_TRUSTED,AWL.
Testing for testuser2... -102.705 hits with
ALL_TRUSTED,AWL,USER_IN_ALL_SPAM_TO.
Testing for testuser1... -102.666 hits with
ALL_TRUSTED,AWL,USER_IN_ALL_SPAM_TO.
I thought that doing a $status-finish (see script) would clear out all of the
whitelist/blacklist it learns from LDAP (so when going from testuser2 back to
testuser1 it would clear out the all_spam_to entry learned from testuser2), but
it doesn't seem to do it. I tried taking a look at the spamd code to see what
I'm doing differently, but can't figure it out - if someone could look at the
script I'm running, and let me know what's necessary to clear out the conf
files when switching users, I'd much appreciate it!
| nate carlson | [EMAIL PROTECTED] | http://www.natecarlson.com |
| depriving some poor village of its idiot since 1981|
#!/usr/bin/perl
$|=1;
use Mail::SpamAssassin;
my($SASpamTester);
open(MAIL, mailmessage);
@msg = MAIL;
close(MAIL);
sainit();
testmsg(testuser1);
testmsg(testuser2);
testmsg(testuser1);
sub sainit {
print Initializing SpamAssassin... ;
$SASpamTester = Mail::SpamAssassin-new(
{
local_tests_only = 1,
dont_copy_prefs = 1,
LOCAL_RULES_DIR = /etc/spamassassin,
userprefs_filename = /etc/sa-mimedefang.cf
}
);
print compiling... ;
$SASpamTester-compile_now(1);
print done.\n;
}
sub testmsg ($) {
my ($username) = @_;
print Testing for $username... ;
$SASpamTester-load_scoreonly_ldap($username);
$SASpamTester-signal_user_changed (
{
username = $username,
user_dir = undef
}
);
my($mail) = $SASpamTester-parse([EMAIL PROTECTED]);
my($status) = $SASpamTester-check($mail);
$mail-finish();
my($hits) = $status-get_hits;
my($tests) = $status-get_names_of_tests_hit();
print $hits hits with $tests.\n;
$status-finish();
}
Return-Path: [EMAIL PROTECTED]
Received: from localhost (localhost [127.0.0.1])
by localhost with ESMTP id iB8I09Ti019209
for [EMAIL PROTECTED]; Wed, 08 Dec 2004 12:01:18 -0600 (CST)
Date: Wed, 8 Dec 2004 12:00:09 -0600 (CST)
From: Test User [EMAIL PROTECTED]
To: Test User [EMAIL PROTECTED]
Subject: test
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
test
user_scores_dsn
ldap://localhost/dc=example,dc=com?spamassassin?sub?uid=__USERNAME__
user_scores_ldap_username cn=ldapuser,dc=example,dc=com
user_scores_ldap_password password
Re: SpamAssassin memory usage
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matthew Newton writes: I have three Sun Fire servers running Solaris 9 and SpamAssassin 3.0.1. SpamAssassin memory usage seems grow a lot. The machines have 2Gb RAM each, and I have an hourly cron job that restarts SpamAssassin if more than 1.5Gb memory is used (if the machine starts swapping, performance goes through the floor). So, the questions are a) Does SpamAssassin normally use a this much memory? b) If so, how much can I expect it to use? c) If not, does anyone know of any bugs in perl (5.8.0) or Solaris that could cause this and finally d) is restarting SpamAssassin an acceptable thing to do to stop it swapping? The machines each process around 8 mails/day and we have something like 25000 users. Hi Matthew -- how many children are running? does the memory usage rise, or is it constant from startup? are there patterns in RAM usage? are you using external rulesets, razor, pyzor, dcc, etc.? - --j. -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Exmh CVS iD8DBQFBt2caMJF5cimLx9ARAjTiAJ0R2KASc3Gwmz1cMsv604BCoRwmCACeIisF d3Zeb5V/Snmv5npyyZdw3fo= =lFhe -END PGP SIGNATURE-
Re: Subject rewriting not happening
Kevin W. Gagel wrote:
I'm having a bit of trouble getting my subjects rewriten
with sa 3.01. Any suggestions would be appreaciated. I'm
using spamd/spamc.
Here is an example of the headers that I do get added:
Subject: =?Windows-1251?B?4PHy8O7r7uPo/yDiIOHo5+3l8eU=?=
MIME-Version: 1.0
--snip--
My local.cf has this:
rewrite_header subject ***SPAM {_Score(0)_}***
I think that's rewrite_header Subject, not rewrite_header subject
Could be wrong though.
Regards,
Rick
Re: Questions about clearing LDAP preferences between runs
This sounds like it has something to do with the switching users code. I don't know how it is done, but I know it is in spamd somewhere, and uses Storable to store the user configs. There is special stuff (I believe) somewhere that is supposed to get invoked when switching users in a database case that is a little different than the non-database case. I may be wrong on that though. I'd think your best bet for an answer would be Theo or Michael, and I'm surprised one of them didn't respond. Loren
Re: Subject rewriting not happening
Maybe change the order of report_safe and rewrite_header? Loren
REPORTS
How i can disable the spamassassin report. I would like a Subject rewrite only, and not the report with original email as attachment. Thanks
Re: Questions about clearing LDAP preferences between runs
On Wed, 8 Dec 2004, Loren Wilton wrote: This sounds like it has something to do with the switching users code. I don't know how it is done, but I know it is in spamd somewhere, and uses Storable to store the user configs. There is special stuff (I believe) somewhere that is supposed to get invoked when switching users in a database case that is a little different than the non-database case. I may be wrong on that though. I've copied the part that I thought was relevant from spamd (load_scoreonly_ldap, and signal_user_changed), but I may've well missed something - I'll take a look at the Storable stuff. I'd think your best bet for an answer would be Theo or Michael, and I'm surprised one of them didn't respond. Well, I did just post this a couple hours ago. :) I then saw the type of traffic that's usually on the -dev list (bugzilla reports, etc), and figured this was probably more of an end-user question (it's probably my issue, not a bug in the code), so I reposted here. | nate carlson | [EMAIL PROTECTED] | http://www.natecarlson.com | | depriving some poor village of its idiot since 1981|
Re: SpamAssassin memory usage
Hello On Wed, Dec 08, 2004 at 12:42:02PM -0800, Justin Mason wrote: I have three Sun Fire servers running Solaris 9 and SpamAssassin 3.0.1. SpamAssassin memory usage seems grow a lot. The machines have 2Gb RAM each, and I have an hourly cron job that restarts SpamAssassin if more than 1.5Gb memory is used (if the machine starts swapping, performance goes through the floor). So, the questions are a) Does SpamAssassin normally use a this much memory? b) If so, how much can I expect it to use? c) If not, does anyone know of any bugs in perl (5.8.0) or Solaris that could cause this and finally d) is restarting SpamAssassin an acceptable thing to do to stop it swapping? The machines each process around 8 mails/day and we have something like 25000 users. how many children are running? does the memory usage rise, or is it constant from startup? are there patterns in RAM usage? are you using external rulesets, razor, pyzor, dcc, etc.? It is set up to use 16 children (-m 16). The memory usage does rise gradually over time. This afternoon it was taking one hour for the system memory usage (output from swap -s) to go from 120M (just after SA had been started) to over 1.4G. This evening it has taken slightly longer. I'm in the process of extracting stats from the mail machines, so hopefully I'll be able to get a rough idea of the amount of mail flowing during these periods, and see if it ties up in some way. The only external stuff I'm using is SURBL. Auto whitelists is turned on, too. Bayesian is off, as are razor/pyzor/dcc. I want to turn on some of these extra services sometime (looking at the possibility of running a DCC server), but none in use yet. I could turn the cron job off on one machine out of three and see how much memory it uses, if that's useful. The machines are configured to give them around 5Gb memory including swap, but I couldn't do this on all machines because of the perfomance hit of using swap. Thanks! -- Matthew Newton [EMAIL PROTECTED] UNIX Systems Administrator, Network Support Section, Computer Centre, University of Leicester, Leicester LE1 7RH, United Kingdom
Re: SpamAssassin memory usage
On Wed, Dec 08, 2004 at 10:28:28PM +, Matthew Newton wrote: The only external stuff I'm using is SURBL. Auto whitelists is turned on, too. Bayesian is off, as are razor/pyzor/dcc. I want to turn on some of these extra services sometime (looking at the possibility of running a DCC server), but none in use yet. Auto whitelists can consume large amounts of memory from what I hear. I personally don't believe in auto whitelists, and have never used them. Maybe I'm being stupid, but I don't see a need for them. SA traps a vast majority of spam without any false positives. This is with or without bayes. But I do run many more tests than you do. YMMV Mike -- /-\ | Michael Barnes [EMAIL PROTECTED] | | UNIX Systems Administrator | | College of William and Mary | | Phone: (757) 879-3930 | \-/
Re: Subject rewriting not happening
Right you are Rick, however its still not coming out
correctly I'm not getting the score in it.
This is what I got:
Subject: ***SPAM ***
With local.cf set to:
rewrite_header Subject ***SPAM _Score_***
- Original Message Follows -
Date: Wed, 08 Dec 2004 16:33:45 -0500
---snip---
My local.cf has this:
rewrite_header subject ***SPAM {_Score(0)_}***
I think that's rewrite_header Subject, not rewrite_header
subject
Could be wrong though.
Regards,
Rick
=
Kevin W. Gagel
Network Administrator
Information Technology Services
(250) 561-5848 local 448
--
The College of New Caledonia, Visit us at http://www.cnc.bc.ca
Virus scanning is done on all incoming and outgoing email.
--
Re: Subject rewriting not happening
On Wed, Dec 08, 2004 at 02:45:06PM -0800, Kevin W. Gagel wrote: With local.cf set to: rewrite_header Subject ***SPAM _Score_*** The tags are case sensisitve. _SCORE_ ... -- Randomly Generated Tagline: You guys are extremely inert today. - Prof. Brown pgppz9dSK8vAl.pgp Description: PGP signature
Re: Subject rewriting not happening
The template tags webpage has:
---Paste---
_SCORE(PAD)_ message score, if PAD is included and is
either spaces or
zeroes, then pad scores with that many
spaces or zeroes
(default, none) ie: _SCORE(0)_ makes 2.4
become 02.4,
_SCORE(00)_ is 002.4. 12.3 would be 12.3
and 012.3
respectively.
---End Paste---
Assuming this is for version 3.01 which I'm using then this
should work. I'm even trying _SCORE_ with no padding in it
and can't get the score to show.
Note: My original problem was using subject instead of
Subject. Its now tagging but not adding the score at all.
- Original Message Follows -
From: Michael Barnes [EMAIL PROTECTED]
To: Kevin W. Gagel [EMAIL PROTECTED]
Subject: Re: Subject rewriting not happening
Date: Wed, 8 Dec 2004 17:27:25 -0500
On Wed, Dec 08, 2004 at 01:28:11PM -0800, Kevin W. Gagel
wrote: My local.cf has this:
rewrite_header subject ***SPAM {_Score(0)_}***
My local.cf has:
rewrite_header subject *SPAM*
(score=_SCORE_/_REQD_)
SA might not like the syntax '_Score(0)_' part. I'm too
lazy to check, but that just does not look right to me.
Give mine a shot to see if it does anything.
I'm assuming you have done spamassassin --lint and started
spamd with the -D flag already. If not, 99% of the time
you will get an answer more quickly doing that than asking
this list.
Mike
--
/-\
| Michael Barnes [EMAIL PROTECTED] |
| UNIX Systems Administrator |
| College of William and Mary |
| Phone: (757) 879-3930 |
\-/
=
Kevin W. Gagel
Network Administrator
Information Technology Services
(250) 561-5848 local 448
--
The College of New Caledonia, Visit us at http://www.cnc.bc.ca
Virus scanning is done on all incoming and outgoing email.
--
Re: REPORTS
At 05:20 PM 12/8/2004, abusquets wrote: How i can disable the spamassassin report. I would like a Subject rewrite only, and not the report with original email as attachment. Thanks read up on the report_safe option in man Mail::SpamAssassin::Conf.
