sql-based global use_auto_whitelist error?
Hi,
I have a global setting in my SQL-based userprefs for use_auto_whitelist.
I am noticing that spamd -D shows these messages about it, which confuse me:
debug: retrieving prefs for [EMAIL PROTECTED] from SQL server
debug: config: not parsing, administrator setting: use_auto_whitelist 1
debug: config: SpamAssassin failed to parse line, skipping:
use_auto_whitelist 1
debug: user has changed
What's going on here? Same thing when I set it to zero. Obviously, it
seems to be querying the DB correctly, but barfing on what it finds. Why? I
also have use_bayes set up the same way, which is working just fine.
my version: SA 3.0.2
mysql select * from spamassassin_user_settings;
+---+++
| username | preference | value
|
+---+++
| !GLOBAL | use_bayes | 1
|
| !GLOBAL | use_auto_whitelist | 1
|
+---+++
in local.cf:
user_scores_sql_custom_querySELECT preference, value FROM
spamassassin_user_settings WHERE username = _USERNAME_ OR username =
'!GLOBAL' OR username = CONCAT('@', _DOMAIN_) ORDER BY username ASC
__
Celebrate Yahoo!'s 10th Birthday!
Yahoo! Netrospective: 100 Moments of the Web
http://birthday.yahoo.com/netrospective/
Re: sql-based global use_auto_whitelist error?
At 07:23 PM 3/8/2005, email builder wrote: I have a global setting in my SQL-based userprefs for use_auto_whitelist. I am noticing that spamd -D shows these messages about it, which confuse me: debug: retrieving prefs for [EMAIL PROTECTED] from SQL server debug: config: not parsing, administrator setting: use_auto_whitelist 1 debug: config: SpamAssassin failed to parse line, skipping: use_auto_whitelist 1 debug: user has changed What's going on here? use_auto_whitelist cannot ever be set as a user preference. It's an admin-only setting and must be set at the local.cf level.
multiple hosts for spamc -d ?
All, Some postings a while back led me to believe that I could specify multiple hosts for the -d option of spamc. I understood that it would operate basically on a fallback basis (not load balancing). However, I can't seem to get spamc to use more than one of the -d listings. I've tried: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 /usr/bin/spamc -d 123.45.67.8 127.0.0.1 And switched the order around and fiddled with hostnames vs IP addresses, but no dice. I understand the man page to say that it will use fallback logic if the hostname resolves (via DNS query, right?) to more than one host... so why can't I give it those hosts directly? TIA! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
Re: multiple hosts for spamc -d ?
email builder wrote: All, Some postings a while back led me to believe that I could specify multiple hosts for the -d option of spamc. I understood that it would operate basically on a fallback basis (not load balancing). However, I can't seem to get spamc to use more than one of the -d listings. I've tried: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 /usr/bin/spamc -d 123.45.67.8 127.0.0.1 And switched the order around and fiddled with hostnames vs IP addresses, but no dice. I understand the man page to say that it will use fallback logic if the hostname resolves (via DNS query, right?) to more than one host... so why can't I give it those hosts directly? TIA! From http://spamassassin.apache.org/full/3.0.x/dist/doc/spamc.html -d host In TCP/IP mode, connect to spamd server on given host (default: localhost). If host resolves to multiple addresses, then spamc will fail-over to the other addresses, if the first one cannot be connected to You need to have a host that has multiple A records. spamd.domain.com A 123.123.123.123 spamd.domain.com A 123.123.123.124 spamd.domain.com A 123.123.123.125 /usr/bin/spamc -d spamd.domain.com If your DNS server sends the results back in a different order each time then it will not be a fallback but a round robin. You might be able to simply use /etc/host entries. I've never tried it as I use qmail which will not use the host file, so I always rely on DNS. Don't know if spamc will use the host file or not. DAve -- Dave Goodrich Systems Administrator http://www.tls.net Get rid of Unwanted Emails...get TLS Spam Blocker!
Whitelist collection project
OK, based on what little discussion there's been so far, here's a draft proposal for people to think about. Summary: A group of volunteers will maintain a collected/distributed whitelist, using SpamAssassin's whitelist_from_rcvd capabilities, similar to (but in the opposite direction as) William Stearns' collected/distributed blacklist at http://www.stearns.org/sa-blacklist/sa-blacklist.current.cf Goal: There are public newsletters, services, etc., which a) do not spam, and b) can easily be mistaken as spam by SpamAssassin for a variety of reasons (overly aggressive custom rules, wrongly taught Bayes system, paid advertising listing SURBL URIs, etc). Though anti-spam devotees see these as opportunities for cleaning up our system, for the purposes of email reliability we want these emails to go through unhindered. Assumption: This activity will focus only on public newsletters, services, etc., which normally do not contain any private information. Therefore there will not be any privacy or confidentiality concerns for the great majority of emails from these sources. Distribution: The rules file which results from this activity will be maintained within the SARE system, as file 70_sare_whitelist.cf -- it can be downloaded manually or via RDJ. Rules: Since these rules are gathered by the community at large, rather than use the whitelist_from_rcvd rule which normally scores -100, we will use the def_whitelist_from_rcvd rule, which scores only -15. Any site that wishes can copy any of these rules to a def_whitelist_from rule to gain the full -100 points. [Question to the devs: would you agree this is a valid use for the def_whitelist_from_rcvd rule?] Adoption to SA Core: We expect that some/most of these rules will be adopted into the SpamAssassin core distribution when new releases are issued. At such times, rules adopted into SA will be migrated to a stable file named 70_sare_whitelist_pre_vv.cf (eg: 70_sare_whitelist_pre_030100 for SA 3.1.0). This file will be announced on the SA-Users list, and can be retrieved once into any/all systems. That file will not change. These rules will be deleted from the primary 70_sare_whitelist.cf file concurrent with the official release of the new SA release. Systems that migrate to a new SA release would then be expected to delete all pre files, though there wouldn't be any harm other than a minimal performance hit if they do not. Contributions: Anyone within the SA community can submit a single whitelist rule for inclusion into the active rules file, and/or submit a qualifying email (with full headers), from which the whitelist team will create a rule. Submissions with a full qualifying email are preferred. Submissions should be sent to [EMAIL PROTECTED] [Submissions can be sent to that address immediately -- something will be done with them, whether following this proposal or not.] Process: At least at first, we expect many/most of these submissions to be obvious ham sources (such as NYTimes.com would be). Obvious ham sources will be agreed upon unanimously by the whitelist team, and added to the whitelist. Any non-obvious ham source, any that has a majority agreement within the whitelist team but not unanimous agreement, will be examined in detail by one of the team. The results of that examination will be reviewed by the whitelist team, and any that now receives unanimous agreement will be added to the rules file. Any submission which does not obtain unanimous agreement will not be added to the rules file. Any submission which already matches a def_whitelist_from_rcvd rule within the SA distribution will be identified and ignored (after response back to the submitter). (We are not going to try to develop pre-3.0.0 files.) Team members: Anyone within the SpamAssassin community who has submitted a CLA to Apache, and who is accepted unanimously by the whitelist team, can join the team. [The CLA requirement is to ensure that rules can be adopted into the SA without any licensing concerns.] - - - - - - - Can anyone think of any guidelines to be added or changed? Any volunteers willing to help out? Bob Menschel
Re: Whitelist collection project
Robert Menschel wrote: snipped a little for brevity OK, based on what little discussion there's been so far, here's a draft proposal for people to think about. Summary: A group of volunteers will maintain a collected/distributed whitelist, using SpamAssassin's whitelist_from_rcvd capabilities, similar to (but in the opposite direction as) William Stearns' collected/distributed blacklist at http://www.stearns.org/sa-blacklist/sa-blacklist.current.cf Don't forget about the new whitelist_from_spf capability that should be in the next major release. I think we'd like to get away from whitelist_from_rcvd if possible for domains that appear to have sensible SPF records (records that actually list their hosts and don't use ?all, etc). There's no point in keeping up with the addition/removal of a domain's hosts if we don't have to (which will be more common than the current whitelist_from_rcvd domains if there are lots of them added). It's also somewhat unfair to restrict a domain to their current mail hosts due to concerns about 'their SpamAssassin score' if we don't have to. For info on whitelist_from_spf (and def_whitelist_from_spf) implementation see bug 3487. Assumption: This activity will focus only on public newsletters, services, etc., which normally do not contain any private information. Therefore there will not be any privacy or confidentiality concerns for the great majority of emails from these sources. What about emails from banks etc? I'd think they'd be a good candidate for something you want to whitelist based on their received headers or SPF. Distribution: The rules file which results from this activity will be maintained within the SARE system, as file 70_sare_whitelist.cf -- it can be downloaded manually or via RDJ. Rules: Since these rules are gathered by the community at large, rather than use the whitelist_from_rcvd rule which normally scores -100, we will use the def_whitelist_from_rcvd rule, which scores only -15. Any site that wishes can copy any of these rules to a def_whitelist_from rule to gain the full -100 points. While it doesn't really matter how it's done, I'd suggest that a user just sets the score for def_whitelist_from_rcvd to -100 or whatever they want if -15 isn't enough (which I don't think I've seen a case where it isn't enough). def_whitelist_from_spf currently doesn't assign the full -15, unless the 'From:' header matches the envelope sender (see bug 3487), and would have to be scored appropriately if desired. [Question to the devs: would you agree this is a valid use for the def_whitelist_from_rcvd rule?] I don't see a problem with it, although we may want to create an alias for it, such as sare_whitelist_from_(rcvd|spf), to prevent confusion between entries included in the distribution with those available from SARE. Any submission which already matches a def_whitelist_from_rcvd rule within the SA distribution will be identified and ignored (after response back to the submitter). (We are not going to try to develop pre-3.0.0 files.) There aren't too many of them so it shouldn't be a problem. I'd suggest listing those domains (along with new domains as added) on a web page at your site, along with the info on how to submit new domains. Can anyone think of any guidelines to be added or changed? You might want to setup a web form for submissions. You could use it (well, the script behind it a database) to automatically filter out duplicate submissions -- but still tally submissions for a domain. Then again I don't know how many submissions you are expecting, so that may be overkill. Daryl
Re: Rule_Du_Jour.sh
FI Lint output: debug: SpamAssassin version 3.0.2 FI ... FI config: SpamAssassin failed to parse line, skipping: rewrite_subject 1 FI config: SpamAssassin failed to parse line, skipping: subject_tag *SPAM* FI [...etc...] You have --lint errors here that are probably in your local.cf FI Can someone explain to me how to solve all the config issues FI detected ? (config: SpamAssassin failed to parse line) Have you checked the config docs? A good suggestion. Those two lines are SA 2.6 commands that have been removed in 3.0. This makes me suspect that other things might also not have been correctly moved forward to 3.0. Loren
Re: Whitelist collection project
On Tuesday, March 8, 2005, 8:13:05 PM, Robert Menschel wrote: OK, based on what little discussion there's been so far, here's a draft proposal for people to think about. Summary: A group of volunteers will maintain a collected/distributed whitelist, using SpamAssassin's whitelist_from_rcvd capabilities, similar to (but in the opposite direction as) William Stearns' collected/distributed blacklist at http://www.stearns.org/sa-blacklist/sa-blacklist.current.cf Goal: There are public newsletters, services, etc., which a) do not spam, and b) can easily be mistaken as spam by SpamAssassin for a variety of reasons (overly aggressive custom rules, wrongly taught Bayes system, paid advertising listing SURBL URIs, etc). Though anti-spam devotees see these as opportunities for cleaning up our system, for the purposes of email reliability we want these emails to go through unhindered. Assumption: This activity will focus only on public newsletters, services, etc., which normally do not contain any private information. Therefore there will not be any privacy or confidentiality concerns for the great majority of emails from these sources. Distribution: The rules file which results from this activity will be maintained within the SARE system, as file 70_sare_whitelist.cf -- it can be downloaded manually or via RDJ. This is something that could potentially be useful to SURBLs as a whitelist source (used for exclusion from SURBLs), so I'm in favor of it. Daryl's ideas of a web form feeding a database and a separately named rule to use it within SA seem like good suggestions. Because this would be a centrally maintained, hand-edited list I don't see it occupying the exact same space as SPF. Ultimately something like an RBL may be the best way to get the data out unless the list is relatively static and not too large. As you outline it above it seems like it would be a global publishing of local whitelists where there was strong consensus about what should be whitelisted. That could be a subset of the local whitelist_froms of all SpamAssassin installations. It could also grow into something larger, and that's not necessarily a bad thing. Collecting up SA local whitelist_froms is a reasonable place to start. Jeff C. -- Jeff Chan mailto:[EMAIL PROTECTED] http://www.surbl.org/
Re: None of the Stock rules hitting
John
only a few in few hours (china daily ones) or so, but not yukon
ones...here's my hit list for one I did catch.. (note I've bumoe my
BAYES_99 score to the pre SA3.0 score!)
5.40BAYES_99Bayesian spam probability is 99 to 100%
2.29BIZ_TLD Contains an URL in the BIZ top-level domain
3.02FB_FORWARD
1.00FB_SPACED_MORTG_PT2
0.45FM_WHITEONWHITE
1.40FU_TLD_BIZ
0.14HTML_FONT_BIG HTML tag for a big font size
0.00HTML_MESSAGEHTML included in message
0.60J_CHICKENPOX_42 {4}Letter - punctuation - {2}Letter
0.00MIME_BOUND_NEXTPART Spam tool pattern in MIME boundary
2.67MIME_BOUND_RKFINDY Spam tool pattern in MIME boundary (rfkindy)
0.18MIME_HTML_ONLY Message only has text/html MIME parts
2.44MIME_HTML_ONLY_MULTIMultipart message only has text/html MIME parts
0.14MORTGAGE_BEST Information on mortgages
0.07MPART_ALT_DIFF HTML and text parts are different
0.11RCVD_IN_SBL Received via a relay in Spamhaus SBL
1.14SARE_MSGID_EMPTYMessage ID is empty, or just spaces
2.00SARE_SELL_STOCK2
0.85SARE_SUB_INVESTMENTSSpammer subject - credit or money
4.00URIBL_JP_SURBL Has URI in JP at http://www.surbl.org/lists.html
3.21URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist
1.00URIBL_SBL Contains an URL listed in the SBL blocklist
1.46URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist
0.01URI_REDIRECTOR Message has HTTP redirector URI
--
Martin Hepworth
Snr Systems Administrator
Solid State Logic
Tel: +44 (0)1865 842300
John Andersen wrote:
I've been seeing a lot of stock-spams slipping through lately
and none of them seem to hit any of the standard stock rule
sets.
These guys are often not even tripping bayes rules, and
come in under the radar even though I run surbl and razor
rules at elevated scores.
The body clearly mentions stocks, and virtually the same
email will arrive via different routes all trying to sell me
gold mining stocks in the Yukon. Anybody else seeing these?
**
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.
This footnote confirms that this email message has been swept
for the presence of computer viruses and is believed to be clean.
**
Re: One single mailbox for all spam
Kris Deugau wrote: Stephane Parenton wrote: A mail server, serving several domains : domain_a.com, domain_b.com, domain_c.com etc... for the moment, domain_a and domain_b are filtered, but if everything runs ok, all the domains should be filtered When a mail arrives, it's controled, scored, and if it's spam, it appears as an attachement. What I want is when a mail arrives, it's scored, and if it's spam, it's not put in the user's mailbox but on a single box that will receive only spam, whatever domain it's adressed to. Since you're using procmail, you may be able to create a global procmailrc file (/etc/procmailrc) something like this: # File spam in the global spambox: :0: * ^X-Spam-Flag: YES /var/spool/mail/spambox Hi Kris, your mail made me read more carefully the man procmail file, and discovered that /etc/procmailrc was quite important ;-)... it also made me realize that procmail was not completely used in my config My postfix setup is quite ok, and i decided to setup SA and procmail... so spamassassin works ok since it tags the headers, but i guess that procmail doesn't do the whole job i'm asking, for the mails are still in my own mailbox, and not in a separate one I'll go back to my manual readings ;-) thanks a lot for your answer Regards Stephane
US_ASCII 8BIT rule in airmax.cf
Hi In the mailheader below there is (unless I'm blind) nowhere a trace of US-ASCII 8Bit encoding. Nevertheless the airmax.cf ruleset has a hit for 2 points. Could somebody please enlighten me why the airmax.cf rule below still applies? Thank you Philipp 2.0 US_8BIT US-ASCII isn't an eight bit charset. Received: by mx..xx (XX, from userid 50001) id 3F26F1564C; Wed, 9 Mar 2005 10:39:08 +0100 (CET) Received: from localhost by mx..xxx with SpamAssassin (version 3.0.1); Wed, 09 Mar 2005 10:39:08 +0100 From: senderdomain.tld [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: *SPAM(4.5)* senderdomain.tld - Newsletter Date: Wed, 09 Mar 2005 08:53:32 +0100 Message-Id: [EMAIL PROTECTED] X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on SpamAssassin X-Spam-Level: X-Spam-Status: Yes, score=4.5 required=3.0 tests=BAYES_00,HTML_40_50, HTML_MESSAGE,HTML_TITLE_EMPTY,URI_REDIRECTOR,US_8BIT,VOWEL_FROM_6, VOWEL_TOCC_6,VOWEL_URI_6 autolearn=no version=3.0.1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=--=_422EC43C.EDCE4840 Return-Path: [EMAIL PROTECTED] X-OriginalArrivalTime: 09 Mar 2005 09:38:13.0710 (UTC) FILETIME=[B6DFC6E0:01C5248B] =_422EC43C.EDCE4840 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: quoted-printable =_422EC43C.EDCE4840 Content-Type: message/rfc822; x-spam-type=original Content-Description: original message before SpamAssassin Content-Disposition: attachment Content-Transfer-Encoding: 7bit Received: from fw01a-qfe1.senderdomain.tld(fw01a-qfe1.senderdomain.tld [ip.address]) by mx.x.xxx (xxx) with SMTP id C91821564C for [EMAIL PROTECTED]; Wed, 9 Mar 2005 10:38:59 +0100 (CET) Received: from mailhost by fw01a-qfe1.senderdomain.tld via smtpd (for mx.xxx.xxx [ip.address]) with SMTP; 9 Mar 2005 09:38:59 UT Received: from senderdomain.tld [ip.address] by senderdomain.tld with ESMTP (SMTPD32-7.07) id AB32ECB013A; Wed, 09 Mar 2005 08:52:18 +0100 Message-ID: [EMAIL PROTECTED] Date: Wed, 09 Mar 2005 08:53:32 +0100 From: senderdomain.tld [EMAIL PROTECTED] User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.4) Gecko/20030619 Netscape/7.1 (ax) X-Accept-Language: de-de, de MIME-Version: 1.0 To: [EMAIL PROTECTED] Subject: senderdomain.tld - Newsletter Content-Type: multipart/alternative; boundary=060907090407000502070702 Precedence: bulk Sender: [EMAIL PROTECTED] --060907090407000502070702 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable --060907090407000502070702 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit --060907090407000502070702-- =_422EC43C.EDCE4840--
Re: a problem with linux 2.6.11 and sa
On Tue, 8 Mar 2005, George Georgalis announced authoritatively: Here's what I'm doing that is broken. I use tcpserver (functionally similar to inetd) to receive an incoming smtp connection. While the smtp session is still open, the message is piped to a temp file which is then scanned for spam, if it passes the temp file is piped to my Both of these sound like redirection, not piping. (I don't see what you mean by `a pipe rom /proc/kmsg', though: pipes connect processes, not files. File redirections are quite different and should work unchanged in 2.6.11.) An interesting technique that allows a program (such as a log writer) to run as an unprivileged user, while receiving privileged data. (taken almost verbatim from Gerrit Pape's socklog) #!/bin/sh exec /proc/kmsg exec 21 exec softlimit -m 200 setuidgid nobody socklog ucspi This script, run by root takes its stdin from /proc/kmsg then combines its stdout and stderr, and exec-switches to the socklog program run as an ucspi application listening to the domain stream socket, as nobody:nogroup, with memory consumption limited to 2Mb. (and sends log to stdout) This is definitely redirection, not piping. As far as I know the implementation of redirection in the kernel remains unchanged: certainly the need to buffer piped data doesn't exist in this case, and since the redesign was of the buffering, this is probably not your problem :) It worked flawlessly until several kernel revs back when the kernel started protecting kmsg and wouldn't allow the user program to receive it, Indeed. result: nothing sent to the logging program and no error. The fix was to run socklog as root instead of nobody. You should be able to open it as root and read from it as another user: i.e., your technique above shouldn't break. (I'd hope.) -- ...Hires Root Beer... What we need these days is a stable, fast, anti-aliased root beer with dynamic shading. Not that you can let just anybody have root. --- John M. Ford
RE: Whitelist collection project
-Original Message- From: Robert Menschel [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 08, 2005 11:13 PM To: users@spamassassin.apache.org Subject: Whitelist collection project OK, based on what little discussion there's been so far, here's a draft proposal for people to think about. Summary: A group of volunteers will maintain a collected/distributed whitelist, using SpamAssassin's whitelist_from_rcvd capabilities, similar to (but in the opposite direction as) William Stearns' collected/distributed blacklist at http://www.stearns.org/sa-blacklist/sa-blacklist.current.cf Goal: There are public newsletters, services, etc., which a) do not spam, and b) can easily be mistaken as spam by SpamAssassin for a variety of reasons (overly aggressive custom rules, wrongly taught Bayes system, paid advertising listing SURBL URIs, etc). Though anti-spam devotees see these as opportunities for cleaning up our system, for the purposes of email reliability we want these emails to go through unhindered. Assumption: This activity will focus only on public newsletters, services, etc., which normally do not contain any private information. Therefore there will not be any privacy or confidentiality concerns for the great majority of emails from these sources. *snip* - - - - - - - Can anyone think of any guidelines to be added or changed? Any volunteers willing to help out? Bob Menschel This might just be the first time I disagree with you Bob ;) I don't see how this ruleset will not get abused. If I was a spammer I would make sure all my spam hit these rules to let me in. As a research tool it is great! Already the SURBL whitelist is one of the best around thanks to Jeff's work. He went mad for a few weeks and collected just about every bit of whitelist info from the net. IMHO, public negative scoring rules are not a good idea, despite there best intentions. Maybe if the airlines made there emails look less like spam? :) -1 --Chris (Just my opinion. I've been outruled many times beforeits pretty fun!)
Re: a problem with linux 2.6.11 and sa
On Wed, Mar 09, 2005 at 01:06:11PM +, Nix wrote: An interesting technique that allows a program (such as a log writer) to run as an unprivileged user, while receiving privileged data. (taken almost verbatim from Gerrit Pape's socklog) #!/bin/sh exec /proc/kmsg exec 21 exec softlimit -m 200 setuidgid nobody socklog ucspi This script, run by root takes its stdin from /proc/kmsg then combines its stdout and stderr, and exec-switches to the socklog program run as an ucspi application listening to the domain stream socket, as nobody:nogroup, with memory consumption limited to 2Mb. (and sends log to stdout) This is definitely redirection, not piping. As far as I know the implementation of redirection in the kernel remains unchanged: certainly the need to buffer piped data doesn't exist in this case, and since the redesign was of the buffering, this is probably not your problem :) It worked flawlessly until several kernel revs back when the kernel started protecting kmsg and wouldn't allow the user program to receive it, Indeed. result: nothing sent to the logging program and no error. The fix was to run socklog as root instead of nobody. You should be able to open it as root and read from it as another user: i.e., your technique above shouldn't break. (I'd hope.) Here is a nice proof that kmsg did become a problem around 2.6.0 http://article.gmane.org/gmane.comp.misc.pape.general/595 http://thread.gmane.org/gmane.comp.misc.pape.general/590 It (Gerrit Pape's technique) very defiantly stopped working a few revs back (2.6.7?). I'm seeing a similar failed read from /dev/rtc and mplayer with 2.6.10, now too. http://lkml.org/lkml/2005/3/8/226 while read file; do mplayer $file ; done mediafiles.txt Failed to open /dev/rtc: Permission denied for file in `cat mediafiles.txt`; do mplayer $file ; done works. // George -- George Georgalis, systems architect, administrator Linux BSD IXOYE http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
URIBL_SBL Weirdness
Can anyone explain to me what the URIBL_SBL rule does (I.e. which list Is used) I have an email that this rule catches because of a email address inside it. The SpamAssassin report lists it as : 0.6 URIBL_SBL Contains an URL listed in the SBL blocklist [URIs: gov.ru] But no matter what I try, I can´t find the blacklist that Is used. Since we are an International Organisation, this is a very inconvenient situaton. - Ríkharður
Re: Whitelist collection project
On Wednesday, March 9, 2005, 7:30:20 AM, Chris Santerre wrote: I don't see how this ruleset will not get abused. If I was a spammer I would make sure all my spam hit these rules to let me in. As a research tool it is great! Already the SURBL whitelist is one of the best around thanks to Jeff's work. He went mad for a few weeks and collected just about every bit of whitelist info from the net. Yep, you're right. We don't want spammers packing their messages full of legitimate domains and getting them through SA because of it. Jeff C. -- Jeff Chan mailto:[EMAIL PROTECTED] http://www.surbl.org/
Re: Whitelist collection project
Chris Santerre wrote: This might just be the first time I disagree with you Bob ;) I don't see how this ruleset will not get abused. If I was a spammer I would make sure all my spam hit these rules to let me in. As a research tool it is great! Already the SURBL whitelist is one of the best around thanks to Jeff's work. He went mad for a few weeks and collected just about every bit of whitelist info from the net. IMHO, public negative scoring rules are not a good idea, despite there best intentions. Maybe if the airlines made there emails look less like spam? :) -1 --Chris (Just my opinion. I've been outruled many times beforeits pretty fun!) How do you propose that whitelist_from_rcvd or whitelist_from_spf be abused, other than due to a mis configured or compromised server? Daryl
Re: Whitelist collection project
On Wednesday, March 9, 2005, 8:03:49 AM, Daryl O'Shea wrote: Chris Santerre wrote: This might just be the first time I disagree with you Bob ;) I don't see how this ruleset will not get abused. If I was a spammer I would make sure all my spam hit these rules to let me in. As a research tool it is great! Already the SURBL whitelist is one of the best around thanks to Jeff's work. He went mad for a few weeks and collected just about every bit of whitelist info from the net. IMHO, public negative scoring rules are not a good idea, despite there best intentions. Maybe if the airlines made there emails look less like spam? :) -1 --Chris (Just my opinion. I've been outruled many times beforeits pretty fun!) How do you propose that whitelist_from_rcvd or whitelist_from_spf be abused, other than due to a mis configured or compromised server? Oh hang on, it depends on whether it's looking at appropriate headers like whitelist_from_rcvd or whitelist_from_spf do, or whether it's looking at message body URIs do like uridnsbl commands do. As the data was proposed to be collected from local whitelist_from_rcvd lists, apparently it would be for headers. That should be safe, and we could probably still use them to limited effect in SURBLs to keep those domains off SURBLs. It's still a good idea. Jeff C. -- Jeff Chan mailto:[EMAIL PROTECTED] http://www.surbl.org/
Re: Whitelist collection project
On Wed, Mar 09, 2005 at 08:08:36AM -0800, Jeff Chan wrote: That should be safe, and we could probably still use them to limited effect in SURBLs to keep those domains off SURBLs. It could also be used to generate default uridnsbl_skip_domain entries as well. ;) Including the domains in spams doesn't get the spammer anywhere, and it means less useless queries against URIBLs. :) -- Randomly Generated Tagline: Daffy Duck of Borg:You're too despicable to assimilate. pgpJeBqxjDhtD.pgp Description: PGP signature
Re: URIBL_SBL Weirdness
On Wednesday, March 9, 2005, 7:50:13 AM, Rikhardur EGILSSON wrote: Can anyone explain to me what the URIBL_SBL rule does (I.e. which list Is used) RTFM? uridnsbl checks a URI domain's nameserver against sbl.spamhaus.org. I have an email that this rule catches because of a email address inside it. The SpamAssassin report lists it as : 0.6 URIBL_SBL Contains an URL listed in the SBL blocklist [URIs: gov.ru] But no matter what I try, I can´t find the blacklist that Is used. What this means is that the nameserver for gov.ru is listed in SBL. http://www.spamhaus.org/sbl/sbl.lasso?query=SBL13545 Ref: SBL13545 213.59.0.0/23 is listed on the Spamhaus Block List (SBL) 26-Feb-2005 02:47 GMT | SR01 Ruslan Ibragimov / send-safe.com 213.59.0.0/23 is listed on the Register Of Known Spam Operations (ROKSO) database as being assigned to, under the control of, or providing service to a known professional spam operation run by Ruslan Ibragimov / send-safe.com. Rostelecom Corporate Mail Relays (escalation) It looks like Spamhaus has listed all of Rostelecom since it hosts send-safe.com. Personally I don't like escalations like that, but I don't run Spamhaus. Fortunately URIBL_SBL usually gets a fairly low score due to false positives like this. I'd say keep it low. Jeff C. -- Jeff Chan mailto:[EMAIL PROTECTED] http://www.surbl.org/
RE: URIBL_SBL Weirdness
The SpamAssassin report lists it as : 0.6 URIBL_SBL Contains an URL listed in the SBL blocklist [URIs: gov.ru] But no matter what I try, I can´t find the blacklist that Is used. What this means is that the nameserver for gov.ru is listed in SBL. http://www.spamhaus.org/sbl/sbl.lasso?query=SBL13545 Ref: SBL13545 213.59.0.0/23 is listed on the Spamhaus Block List (SBL) 26-Feb-2005 02:47 GMT | SR01 Ruslan Ibragimov / send-safe.com 213.59.0.0/23 is listed on the Register Of Known Spam Operations (ROKSO) database as being assigned to, under the control of, or providing service to a known professional spam operation run by Ruslan Ibragimov / send-safe.com. Rostelecom Corporate Mail Relays (escalation) It looks like Spamhaus has listed all of Rostelecom since it hosts send-safe.com. Personally I don't like escalations like that, but I don't run Spamhaus. Hell yeah to Spamhaus! I like escalations like that! Its the only way hosts are going to learn to keep there own backyard clean. They have no preasure if they don't get listed. Of course, an escalation like this won't happen on SURBL. There are ways of making contributors comply, and resistence is futile --Chris (We are SURBL...the collective...OK who left the cube door open again? )
RE: Windows with ESA - statistics?
From: Tim P [mailto:[EMAIL PROTECTED]
I am using spamassasin on windows with the ESA message sync (
http://www.christopherlewis.com/ExchangeSpamAssassin.htm ) and
would like to have some statistics so that I can better tune the
spam assassin (particularly with the minimum spam setting and the
purge on discovery setting).
As perl is already installed what I really need is a perl script
that can go through the messages that the ESA dumps out (.out
files that can be read by notepad), grab the following lines and
dump them into a file format (like a csv or a delimited text
file) so that I can use something to give me statistics.
X-Spam-Status: Yes, score=28.9
Date: Tue, 4 Jan 2005 06:36:10 -0700
I would really like to be able to give statistics by date (daily,
weekly, monthly), averages, numbers of spams/hams for a time
period. Most of that can be done using excel if I have to but
its getting that info into a readable fileformat that is beyond
me. Anyone out there know how to do something like that?
Parsing the file and producing delimited text that can be imported
into Excel is simple with Perl.
Assuming that each email scanned will produce both of the lines you
listed and assuming that those entries are on their own lines and
not part of a longer line, this should work for you (untested):
print Date\tStatus\tScore\n;
while ()
{
if (/^X-Spam-Status: (Yes|No), score=([0-9.]+)/)
{
$status = $1;
$score = $2;
}
if (/^Date: (.*) [-+]\d{5}$/)
{
$date = $1;
}
if ($status and $score and $date)
{
print $date\t$status\t$score\n;
undef $date;
undef $status;
undef $score;
}
}
Output would be tab separated like this (formatted for readability):
DateStatusScore
Tue, 4 Jan 2005 06:36:10Yes 28.9
Bowie
RE: Whitelist collection project
-Original Message- From: Daryl C. W. O'Shea [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 09, 2005 11:04 AM To: Chris Santerre Cc: 'Robert Menschel'; users@spamassassin.apache.org Subject: Re: Whitelist collection project Chris Santerre wrote: This might just be the first time I disagree with you Bob ;) I don't see how this ruleset will not get abused. If I was a spammer I would make sure all my spam hit these rules to let me in. As a research tool it is great! Already the SURBL whitelist is one of the best around thanks to Jeff's work. He went mad for a few weeks and collected just about every bit of whitelist info from the net. IMHO, public negative scoring rules are not a good idea, despite there best intentions. Maybe if the airlines made there emails look less like spam? :) -1 --Chris (Just my opinion. I've been outruled many times beforeits pretty fun!) How do you propose that whitelist_from_rcvd or whitelist_from_spf be abused, other than due to a mis configured or compromised server? Daryl First I propose I don't answer anymore email until I get off this cold medication :) Second, I believe SPF records can be spoofed/use in a disposibal manner. But thats my medicated brain half telling me I might remember reading something about that. As for rcvd, I'm not sure how secure that is to faked headers. But you can bet they are going to try to abuse it anyway they can. If you can get code to deal with those issues, then hell yeah! I also like Theos idea of using it for the 'skipped' domains. --Chris
Re: SPF problems with this list
Steven Stern wrote: v=spf1 a mx ptr include:earthlink.net ~all I smarthost my mail through smtpauth.earthlink.net. The mail path then bounces around inside earthlink for a while. The spf why page says: hermes.apache.org saw a message coming from the IP address 209.86.89.61 which is smtpauth01.mail.atl.earthlink.net; the sender claimed to be [EMAIL PROTECTED] Earthlink's SPF record at present appears to be: v=spf1 ip4:207.217.120.0/23 ip4:207.69.200.0/24 ip4:209.86.93.0/24 ip4:207.69.195.0/24 ?all This doesn't include the server they sent your message out from. How can I configure the SPF record to work? Your record's fine. Earthlink messed up by not listing all their outgoing servers in their SPF record, or by setting up new servers without updating their SPF record. Notify Earthlink of the problem, and once they've updated their own SPF record it should be fine. Is it possible to use SPF in this environment? Of course, there is one more issue: hermes.apache.org should not be rejecting mail based on a *softfail*, which is what that message should have triggered. ~all means mail will probably never come from any other servers than these, but we're not absolutely certain, not mail will only ever come from these servers. -- Kelson Vibber SpeedGate Communications www.speed.net
Re: Whitelist collection project
Chris Santerre wrote: Second, I believe SPF records can be spoofed Only to the extent that any DNS record can be spoofed. use in a disposibal manner. In the sense that you can create any SPF entry you want... for your own domain. I could set one up with +all indicating that mail sent via any server on the Internet using my domain should be considered valid. I could buy up 100 disposable domains and put SPF on all of them. That's why an SPF pass was never intended to bypass filters by itself... but it can be used to decide whether an address is reliable enough to check it against a whitelist. Within the SpamAssassin scoring paradigm: whitelisted domain without SPF = no rule triggered whitelisted domain with SPF pass = apply nice rule subtract points whitelisted domain with SPF fail = apply forgery rule add points (I'm assuming that's how the feature is to be implemented, because that seems to me the most reasonable way to do it.) -- Kelson Vibber SpeedGate Communications www.speed.net
Re: Whitelist collection project
Kelson wrote: Chris Santerre wrote: Second, I believe SPF records can be spoofed Only to the extent that any DNS record can be spoofed. use in a disposibal manner. In the sense that you can create any SPF entry you want... for your own domain. I could set one up with +all indicating that mail sent via any server on the Internet using my domain should be considered valid. I could buy up 100 disposable domains and put SPF on all of them. That's why an SPF pass was never intended to bypass filters by itself... but it can be used to decide whether an address is reliable enough to check it against a whitelist. Within the SpamAssassin scoring paradigm: whitelisted domain without SPF = no rule triggered whitelisted domain with SPF pass = apply nice rule subtract points Yup. Currently it's -7.5 for the whitelist match SPF pass and then another -7.5 if the envelope sender and header 'From:' match, for a total of -15 like the current whitelist_from_rcvd. whitelisted domain with SPF fail = apply forgery rule add points I had considered also adding a rule for whitelisted domains that failed an SPF check, but thought (for the time being anyway) it'd be best not to penalize them anymore than the current SPF fail rule since many domains don't have up-to-date SPF records (often because they forget to add a webserver that sends out mail directly). (I'm assuming that's how the feature is to be implemented, because that seems to me the most reasonable way to do it.) Daryl
running spamd remotely
What is involved in setting up spamassassin to run on a remote machine? After installing it on hostA (mailserver) and hostB, I get onto hostA and tweak my procmailrc to have :0fw: $HOME/spamassassin.lock | $SPAMHOME/bin/spamc -d hostB.FQDN ... (if spam, then filter) And on hostB, I start it up with $SPAMHOME/bin/spamd -d --syslog-socket=inet -r /var/tmp/spamdnew.pid \ --allowed-ips=ip of hostA,ip of hostB But I don't even see anything in syslog on hostB. TIA =-=-=-=-=-=-=-=-=-=- generated by /dev/dave -=-=-=-=-=-=-=-=-=-=-=-= David SternUniversity of Maryland Institute for Advanced Computer Studies
Re: running spamd remotely
[EMAIL PROTECTED] wrote: What is involved in setting up spamassassin to run on a remote machine? After installing it on hostA (mailserver) and hostB, I get onto hostA and tweak my procmailrc to have :0fw: $HOME/spamassassin.lock | $SPAMHOME/bin/spamc -d hostB.FQDN ... (if spam, then filter) And on hostB, I start it up with $SPAMHOME/bin/spamd -d --syslog-socket=inet -r /var/tmp/spamdnew.pid \ --allowed-ips=ip of hostA,ip of hostB But I don't even see anything in syslog on hostB. TIA I'm assuming when you say you don't see anything, you mean you don't see anything about 'hostA' trying to connect. Do you have a firewall running on the spamd host? You'll need to open up tcp port 783 if you do. Daryl
SA addr tests need to be updated
SA 3.0.2 currently performs a handful of tests against HELO greetings that contain an IP address. These tests don't currently fire when an address literal is used in the HELO greeting, but they should. See section 3.6 of RFC 2821: | - The domain name given in the EHLO command MUST BE either a primary |host name (a domain name that resolves to an A RR) or, if the host |has no name, an address literal as described in section 4.1.1.1. and section 4.1.3: 4.1.3 Address Literals | Sometimes a host is not known to the domain name system and | communication (and, in particular, communication to report and repair | the error) is blocked. To bypass this barrier a special literal form | of the address is allowed as an alternative to a domain name. For | IPv4 addresses, this form uses four small decimal integers separated | by dots and enclosed by brackets such as [123.255.37.2], which | indicates an (IPv4) Internet Address in sequence-of-octets form. For | IPv6 and other forms of addressing that might eventually be | standardized, the form consists of a standardized tag that | identifies the address syntax, a colon, and the address itself, in a | format specified as part of the IPv6 standards [17]. Technically, addresses that are NOT enclosed in brackets are illegal, but those are the only ones that SA sniffs out currently. Extending the current rules to include literals can probably be done by simply changing the sniff code to look for open and close brackets, but I haven't looked so I'm just guessing. As far as that goes, the tests might already do this, and just not firing. I think the four affected rules are RCVD_HELO_IP_MISMATCH, RCVD_NUMERIC_HELO, RCVD_ILLEGAL_IP, RCVD_BY_IP -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: running spamd remotely
[EMAIL PROTECTED] wrote: What is involved in setting up spamassassin to run on a remote machine? After installing it on hostA (mailserver) and hostB, I get onto hostA and tweak my procmailrc to have :0fw: $HOME/spamassassin.lock | $SPAMHOME/bin/spamc -d hostB.FQDN ... (if spam, then filter) What does a command line test tell you? cat your.testmsg | /your/path/bin/spamc -d yourhost.com And on hostB, I start it up with $SPAMHOME/bin/spamd -d --syslog-socket=inet -r /var/tmp/spamdnew.pid \ --allowed-ips=ip of hostA,ip of hostB It would tell you there is no spamd listening, you have spamc making a tcp connection to spamd on another host that is listening to a socket. You need -i ip.address in your spamd startup. http://spamassassin.apache.org/full/3.0.x/dist/doc/spamd.html DAve But I don't even see anything in syslog on hostB. TIA =-=-=-=-=-=-=-=-=-=- generated by /dev/dave -=-=-=-=-=-=-=-=-=-=-=-= David SternUniversity of Maryland Institute for Advanced Computer Studies -- Dave Goodrich Systems Administrator http://www.tls.net Get rid of Unwanted Emails...get TLS Spam Blocker!
Re: SPF problems with this list
I believe you have to rely on Earthlink's smtpauth for its SPF.
That's what I do.
{^_^}
- Original Message -
From: Steven Stern [EMAIL PROTECTED]
My SPF record was working, but is now failing for this list. So, it
seems that SPF checking has ratcheted up a notch. I've deleted my SPF
txt record in order to send this.
Anyhow...
The record was
v=spf1 a mx ptr include:earthlink.net ~all
I smarthost my mail through smtpauth.earthlink.net. The mail path then
bounces around inside earthlink for a while. The spf why page says:
hermes.apache.org saw a message coming from the IP address 209.86.89.61
which
is smtpauth01.mail.atl.earthlink.net; the sender claimed to be
[EMAIL PROTECTED]
How can I configure the SPF record to work? I can't put in an
a:smtpauth01.mail.atl.earthlink.net because next time it may go
through 02,
or 03, etc.
Is it possible to use SPF in this environment?
--
Steve
Re: URIBL_SBL Weirdness
At 10:50 AM 3/9/2005, [EMAIL PROTECTED] wrote: Can anyone explain to me what the URIBL_SBL rule does (I.e. which list Is used) I have an email that this rule catches because of a email address inside it. The SpamAssassin report lists it as : 0.6 URIBL_SBL Contains an URL listed in the SBL blocklist [URIs: gov.ru] The URIBL_SBL rule checks the IPs of the nameservers listed in the NS record against the spamhaus SBL list. [EMAIL PROTECTED] mail]# dig ns gov.ru ; DiG 9.2.1 ns gov.ru ;; global options: printcmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 61199 ;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 2 ;; QUESTION SECTION: ;gov.ru.IN NS ;; ANSWER SECTION: gov.ru. 345600 IN NS ns.rtcomm.ru. gov.ru. 345600 IN NS ns1.gov.ru. gov.ru. 345600 IN NS ns.gov.ru. gov.ru. 345600 IN NS ns.relarn.ru. host ns.rtcomm.ru ns.rtcomm.ru has address 213.59.0.3 213.59.0.3 is listed in the SBL, in the following records: * http://www.spamhaus.org/sbl/sbl.lasso?query=SBL13545SBL13545 http://www.spamhaus.org/sbl/sbl.lasso?query=SBL13545
Re: running spamd remotely
At 01:31 PM 3/9/2005, [EMAIL PROTECTED] wrote: $SPAMHOME/bin/spamd -d --syslog-socket=inet -r /var/tmp/spamdnew.pid \ --allowed-ips=ip of hostA,ip of hostB But I don't even see anything in syslog on hostB. erm, that's not going to cause it to syslog to hostB... that's going to cause spamd to allow hostB to run spamc and connect. If you want to redirect your syslog output, do that in syslog.conf.
Re: running spamd remotely
On Wed, 9 Mar 2005, Dave Goodrich wrote: [EMAIL PROTECTED] wrote: What is involved in setting up spamassassin to run on a remote machine? After installing it on hostA (mailserver) and hostB, I get onto hostA and tweak my procmailrc to have :0fw: $HOME/spamassassin.lock | $SPAMHOME/bin/spamc -d hostB.FQDN ... (if spam, then filter) What does a command line test tell you? cat your.testmsg | /your/path/bin/spamc -d yourhost.com Great idea. First, as mentioned below, I changed startup to add -i ie $SPAMHOME/bin/spamd -d --syslog-socket=inet -r /var/tmp/spamdnew.pid \ --allowed-ips=HostA-ip,HostB-ip,127.0.0.1 -i HostB then on HostA, % cat /etc/motd | $SPAMHOME/bin/spamc -d hostB X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on HostB.fqdn X-Spam-Level: X-Spam-Status: No, score=-0.2 required=4.0 tests=ALL_TRUSTED,MISSING_DATE, MISSING_SUBJECT,NONLOCAL autolearn=unavailable version=3.0.2 (HostA had SA v.265 installed, HostB has SA v3.02) and the above indeed spews spamd stuff to syslog on hostB And on hostB, I start it up with $SPAMHOME/bin/spamd -d --syslog-socket=inet -r /var/tmp/spamdnew.pid \ --allowed-ips=ip of hostA,ip of hostB It would tell you there is no spamd listening, you have spamc making a tcp connection to spamd on another host that is listening to a socket. You need -i ip.address in your spamd startup. Thanks a lot for your help! =-=-=-=-=-=-=-=-=-=- generated by /dev/dave -=-=-=-=-=-=-=-=-=-=-=-= David SternUniversity of Maryland Institute for Advanced Computer Studies
Re: SA addr tests need to be updated
Eric, I believe that you have misinterpreted (and only partially quoted) RFC2821. A more correct interpretation (or at least different) and a fuller set of quotations is below. SA 3.0.2 currently performs a handful of tests against HELO greetings that contain an IP address. These tests don't currently fire when an address literal is used in the HELO greeting, but they should. See section 3.6 of RFC 2821: | - The domain name given in the EHLO command MUST BE either a primary |host name (a domain name that resolves to an A RR) or, if the host |has no name, an address literal as described in section 4.1.1.1. 3.6 Domains Only resolvable, fully-qualified, domain names (FQDNs) are permitted when domain names are used in SMTP. In other words, names that can be resolved to MX RRs or A RRs (as discussed in section 5) are permitted, as are CNAME RRs whose targets can be resolved, in turn, to MX or A RRs. Local nicknames or unqualified names MUST NOT be used. There are two exceptions to the rule requiring FQDNs: ... Nothing in either the section you have quoted, or the one I have allows a hostname which is not a FQDN to be used. and then section 4.1.1.1 downgrades the MUST which you quote to a SHOULD. 4.1.1.1 Extended HELLO (EHLO) or HELLO (HELO) These commands are used to identify the SMTP client to the SMTP server. The argument field contains the fully-qualified domain name of the SMTP client if one is available. In situations in which the SMTP client system does not have a meaningful domain name (e.g., when its address is dynamically allocated and no reverse mapping record is available), the client SHOULD send an address literal (see section 4.1.3), optionally followed by information that will help to identify the client system. y The SMTP server identifies itself to the SMTP client in the connection greeting reply and in the response to this command. ... (The 'y' which is apparently misplaced exists in the original document, it is not a typo by me.) Please note there is absolutely no requirement for an 'A' record; My domain does not have one! The requirement is for either an 'A' record or an 'MX' record. Admittedly, the requirement for an 'A' record is indeed recommended as a SHOULD, but is not not a MUST. Section 4.1.1.1, while not as clear as it should be, is a catchall that basically allows anything at all that is a possible FQDN and that is not identifiable as a CNAME (it is legal in general for a CNAME to otherwise also be a FQDN, I have subdomains for which this is true - I just don't send mail from them). Also there is no wording requiring the recommended reverse mapping record be an 'A' record, any record will suffice (even a 'RP', 'TXT', 'WKS' or 'HINFO' etc.). Also, note that RFC2821 is *still* on the standards track and has not yet (and probably like most RFCs will never be) ratified (though RFC821 was and thus is still valid despite the intention for RFC2821 to replace it). Further, the section you do quote makes clear that hostnames which are not also FQDNs are illegal (yet that is actually the standard practice in use). In other words using any old hostname to a HELO/EHLO is *not* legal by RFC2821, but rather it is intended that one use a domain name (which, of course may be a subdomain) which SHOULD have an 'A' record *or* MUST have an 'MX' record *or* SHOULD use an address literal (with the caveat that 4.1.1.1 makes, most probably because of oversight, nearly anything legal which could be interpreted as a FQDN - hence hostnames, which are recognizable, but not identifiable domains themselves, strictly should get a 4xx response, but I've never seen a system which does that). The only examples I still know of which abide strictly by RFC2821 are sgi.com and ibm.com. sun.com used to, but doesn't anymore (I have over a decade of logs to demonstrate this). And the most common case of using a hostname which is not also a FQDN is clearly not allowed (again, despite that being the standard practice) - i.e. FQHN != FQDN. and section 4.1.3: 4.1.3 Address Literals | Sometimes a host is not known to the domain name system and | communication (and, in particular, communication to report and repair | the error) is blocked. To bypass this barrier a special literal form | of the address is allowed as an alternative to a domain name. For | IPv4 addresses, this form uses four small decimal integers separated | by dots and enclosed by brackets such as [123.255.37.2], which | indicates an (IPv4) Internet Address in sequence-of-octets form. For | IPv6 and other forms of addressing that might eventually be | standardized, the form consists of a standardized tag that | identifies the address syntax, a colon, and the address itself, in a | format specified as part of the IPv6 standards [17]. Technically, addresses that are NOT enclosed in brackets are illegal, but those
Re: SA addr tests need to be updated
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Eric A. Hall writes: SA 3.0.2 currently performs a handful of tests against HELO greetings that contain an IP address. These tests don't currently fire when an address literal is used in the HELO greeting, but they should. actually, that's deliberate -- compare the frequencies of an RFC-2821 address literal, vs. a raw address, and you'll notice that the latter is much more prevalent in spam. - --j. See section 3.6 of RFC 2821: | - The domain name given in the EHLO command MUST BE either a primary |host name (a domain name that resolves to an A RR) or, if the host |has no name, an address literal as described in section 4.1.1.1. and section 4.1.3: 4.1.3 Address Literals | Sometimes a host is not known to the domain name system and | communication (and, in particular, communication to report and repair | the error) is blocked. To bypass this barrier a special literal form | of the address is allowed as an alternative to a domain name. For | IPv4 addresses, this form uses four small decimal integers separated | by dots and enclosed by brackets such as [123.255.37.2], which | indicates an (IPv4) Internet Address in sequence-of-octets form. For | IPv6 and other forms of addressing that might eventually be | standardized, the form consists of a standardized tag that | identifies the address syntax, a colon, and the address itself, in a | format specified as part of the IPv6 standards [17]. Technically, addresses that are NOT enclosed in brackets are illegal, but those are the only ones that SA sniffs out currently. Extending the current rules to include literals can probably be done by simply changing the sniff code to look for open and close brackets, but I haven't looked so I'm just guessing. As far as that goes, the tests might already do this, and just not firing. I think the four affected rules are RCVD_HELO_IP_MISMATCH, RCVD_NUMERIC_HELO, RCVD_ILLEGAL_IP, RCVD_BY_IP -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Exmh CVS iD8DBQFCL2QQMJF5cimLx9ARAssJAJ9n0LXbN+O1mvJ0tZTljBx9GBi5fACfQpkv pGW3PFSvh56f/auv27fmMY8= =m4XF -END PGP SIGNATURE-
Re: SA addr tests need to be updated
On 3/9/2005 4:01 PM, Justin Mason wrote: SA 3.0.2 currently performs a handful of tests against HELO greetings that contain an IP address. These tests don't currently fire when an address literal is used in the HELO greeting, but they should. actually, that's deliberate -- compare the frequencies of an RFC-2821 address literal, vs. a raw address, and you'll notice that the latter is much more prevalent in spam. That's true, but the rules that compare for addresses should still check the address in literals. I think the four affected rules are RCVD_HELO_IP_MISMATCH, RCVD_NUMERIC_HELO, RCVD_ILLEGAL_IP, RCVD_BY_IP if the addr doesn't check out... -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: SPF problems with this list
At 07:57 AM 3/9/2005, Steven Stern wrote: The record was v=spf1 a mx ptr include:earthlink.net ~all I smarthost my mail through smtpauth.earthlink.net. The mail path then bounces around inside earthlink for a while. The spf why page says: hermes.apache.org saw a message coming from the IP address 209.86.89.61 which is smtpauth01.mail.atl.earthlink.net; the sender claimed to be [EMAIL PROTECTED] When you do a SPF include, it basicaly winds up including the SPF record for earthlink.net. ;; QUESTION SECTION: ;earthlink.net. IN TXT ;; ANSWER SECTION: earthlink.net. 1800IN TXT v=spf1 ip4:207.217.120.0/23 ip4:207.69.200.0/24 ip4:209.86.93.0/24 ip4:207.69.195.0/24 ?all However, if you look, earthlink's SPF record doesn't include 209.86.98.61, therefore you fail the test.
Re: SA addr tests need to be updated
On 3/9/2005 3:29 PM, List Mail User wrote: See section 3.6 of RFC 2821: | - The domain name given in the EHLO command MUST BE either a primary |host name (a domain name that resolves to an A RR) or, if the host |has no name, an address literal as described in section 4.1.1.1. 3.6 Domains used. There are two exceptions to the rule requiring FQDNs: ... Nothing in either the section you have quoted, or the one I have allows a hostname which is not a FQDN to be used. see the first exception, which is the text I cited above. Technically, addresses that are NOT enclosed in brackets are illegal, but those are the only ones that SA sniffs out currently. Of course, my machines just refuse these during the SMTP conversation, Many do. BTW, postfix has similar problems wrt literals. For example, if postfix gets a regular address (non-literal) in the HELO, it will split the address into octets and do lookups for PERMIT/REJECT ACLs on incrementally smaller sets, which is all very nice. But if it finds a literal, it doesn't parse for the address inside, and treats the literal like a domain name. Another bug here is that the strict-syntax checks in postfix don't match against non-literal addresses, which it should (RFC1123 spells out what is a valid hostname, and all-numerics is clearly not legal). Please be careful and check the definitions and references in each document indeed -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: Possible per-user levels of grey whitelist
Do you mean have it learn on my Sent folder list? Would it help that much though since the sender is always me and will be lacking all the Received markers that incoming mail normally has? Justin Mason wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Effectively, there's already a way to do that... use Bayes to learn those mails as ham! It works very effectively for that purpose. - --j. Linda W writes:
Re: Possible per-user levels of grey whitelist
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 should still be OK for whitelisting purposes -- as the words in the mail will be correct. I wouldn't use it to learn a mail as spam, of course, but as ham it won't do any harm. - --j. Linda W writes: Do you mean have it learn on my Sent folder list? Would it help that much though since the sender is always me and will be lacking all the Received markers that incoming mail normally has? Justin Mason wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Effectively, there's already a way to do that... use Bayes to learn those mails as ham! It works very effectively for that purpose. - --j. Linda W writes: -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Exmh CVS iD8DBQFCL3MIMJF5cimLx9ARAk3JAKCJD7mJkD3e6EyG5S3Du66rJGPWMACfdmCb 8cakxjiYcrXRdhKEXcdVwxc= =7YoN -END PGP SIGNATURE-
Re: multiple hosts for spamc -d ?
Some postings a while back led me to believe that I could specify multiple hosts for the -d option of spamc. I understood that it would operate basically on a fallback basis (not load balancing). However, I can't seem to get spamc to use more than one of the -d listings. I've tried: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 /usr/bin/spamc -d 123.45.67.8 127.0.0.1 And switched the order around and fiddled with hostnames vs IP addresses, but no dice. I understand the man page to say that it will use fallback logic if the hostname resolves (via DNS query, right?) to more than one host... so why can't I give it those hosts directly? TIA! From http://spamassassin.apache.org/full/3.0.x/dist/doc/spamc.html -d host In TCP/IP mode, connect to spamd server on given host (default: localhost). If host resolves to multiple addresses, then spamc will fail-over to the other addresses, if the first one cannot be connected to You need to have a host that has multiple A records. spamd.domain.com A 123.123.123.123 spamd.domain.com A 123.123.123.124 spamd.domain.com A 123.123.123.125 /usr/bin/spamc -d spamd.domain.com If your DNS server sends the results back in a different order each time then it will not be a fallback but a round robin. You might be able to simply use /etc/host entries. I've never tried it as I use qmail which will not use the host file, so I always rely on DNS. Don't know if spamc will use the host file or not. Huh, I am not familiar with how to use /etc/hosts as a DNS source. Can you clarify? Mainly my question was if/how I could avoid making it a DNS query. I'd like to simply hand spamc the two addresses that I want it to have manually, and I do *NOT* want round-robin, I want failover Your help is much appreciated! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
Re: SA addr tests need to be updated
On 3/9/2005 3:29 PM, List Mail User wrote: See section 3.6 of RFC 2821: | - The domain name given in the EHLO command MUST BE either a primary |host name (a domain name that resolves to an A RR) or, if the host |has no name, an address literal as described in section 4.1.1.1. 3.6 Domains used. There are two exceptions to the rule requiring FQDNs: ... Nothing in either the section you have quoted, or the one I have allows a hostname which is not a FQDN to be used. see the first exception, which is the text I cited above. Technically, addresses that are NOT enclosed in brackets are illegal, but those are the only ones that SA sniffs out currently. Of course, my machines just refuse these during the SMTP conversation, Many do. BTW, postfix has similar problems wrt literals. For example, if postfix gets a regular address (non-literal) in the HELO, it will split the address into octets and do lookups for PERMIT/REJECT ACLs on incrementally smaller sets, which is all very nice. But if it finds a literal, it doesn't parse for the address inside, and treats the literal like a domain name. Another bug here is that the strict-syntax checks in postfix don't match against non-literal addresses, which it should (RFC1123 spells out what is a valid hostname, and all-numerics is clearly not legal). Please be careful and check the definitions and references in each document indeed -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/ Postfix option reject_invalid_hostname will reject bare IPs (when used in the smtpd_helo_restrictions section of main.cf). Paul Shupak [EMAIL PROTECTED] P.S. I see hundreds of connection attempts rejected every day becuase of this.
Re: SA addr tests need to be updated
Justin Mason wrote: Eric A. Hall writes: SA 3.0.2 currently performs a handful of tests against HELO greetings that contain an IP address. These tests don't currently fire when an address literal is used in the HELO greeting, but they should. actually, that's deliberate -- compare the frequencies of an RFC-2821 address literal, vs. a raw address, and you'll notice that the latter is much more prevalent in spam. Do you mean it's deliberate to catch this (as a helo ip mismatch): Received: from unknown (HELO 212.27.42.19) (218.190.234.6) but not this Received: from unknown (HELO [212.27.42.19]) (218.190.234.6) I can see that the latter may hit ham (nat+no hostname). Is this the justification? (hit very few spam but hit some ham?)
Re: SA addr tests need to be updated
On 3/9/2005 5:17 PM, List Mail User wrote: Postfix option reject_invalid_hostname will reject bare IPs (when used in the smtpd_helo_restrictions section of main.cf). Good to hear this was fixed. I filed a bug report on it in May '04 but didn't get much of a response. I'll have to upgrade. -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: multiple hosts for spamc -d ?
Some postings a while back led me to believe that I could specify multiple hosts for the -d option of spamc. I understood that it would operate basically on a fallback basis (not load balancing). However, I can't seem to get spamc to use more than one of the -d listings. I've tried: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 /usr/bin/spamc -d 123.45.67.8 127.0.0.1 And switched the order around and fiddled with hostnames vs IP addresses, but no dice. I understand the man page to say that it will use fallback logic if the hostname resolves (via DNS query, right?) to more than one host... so why can't I give it those hosts directly? From http://spamassassin.apache.org/full/3.0.x/dist/doc/spamc.html -d host In TCP/IP mode, connect to spamd server on given host (default: localhost). If host resolves to multiple addresses, then spamc will fail-over to the other addresses, if the first one cannot be connected to You need to have a host that has multiple A records. spamd.domain.com A 123.123.123.123 spamd.domain.com A 123.123.123.124 spamd.domain.com A 123.123.123.125 /usr/bin/spamc -d spamd.domain.com If your DNS server sends the results back in a different order each time then it will not be a fallback but a round robin. You might be able to simply use /etc/host entries. I've never tried it as I use qmail which will not use the host file, so I always rely on DNS. Don't know if spamc will use the host file or not. Huh, I am not familiar with how to use /etc/hosts as a DNS source. Can you clarify? I didn't mean use it as a dns source, but many programs can look first to your /etc/hosts file prior to doing a DNS lookup. I do not know if spamc will do that, I do not believe it does. Mainly my question was if/how I could avoid making it a DNS query. I'd like to simply hand spamc the two addresses that I want it to have manually, and I do *NOT* want round-robin, I want failover I do not think you can do this. You can use a IP address with spamc to save a lookup. I use the following run script under demontools, #!/sbin/sh PATH=/usr/bin:/usr/local/bin exec /usr/local/bin/softlimit -a 12800 \ /usr/local/bin/spamd -i 10.0.240.253 -p 1783 -A 10.0.240.0/24 \ -m 10 --max-conn-per-child=200 -u vpopmail -x -q -s stderr 21 I do not believe you can have mulitple addresses behind the -i switch, at least the docs do not lead me to believe it is possible. Maybe someone else knows better. spamd is not a problem for me. I run spamd on two machines, one being my main SA server, and one being a fallback just in case something goes awry (something recently did). I'd really like to be able to tell spamc that it can go to a 2nd IP address in case the first one fails, possibly by doing as I wrote above: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 but it seems I can't do this unless I go the DNS route (which I don't know how to do, since my main SA server must be routed to using an internal network IP). Thanks! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
Re: SA addr tests need to be updated
On 3/9/2005 6:08 PM, Justin Mason wrote: mouss writes: Do you mean it's deliberate to catch this (as a helo ip mismatch): Received: from unknown (HELO 212.27.42.19) (218.190.234.6) but not this Received: from unknown (HELO [212.27.42.19]) (218.190.234.6) yes. (I'm not sure if we've retested that recently though.) I've been playing with this a lot recently. Hence the note. I've grepped my logs and every single instance of a helo literal is spam. But even if you want to accomodate NATs and such, the current exception allows spammers to bypass the tests just by excluding the brackets. If you want to preserve the distinction, add an extra score for non-literal addresses, since that's a rat-sign that's above-and-beyond the spam-sign from lack of a domain name. -- Eric A. Hallhttp://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
