Some emails pass spamassassin unprocessed
Hello, I am running the Spamd Daemon version 3.2.5 on my Linux web and mail server and in general it works well. From time to time (somewhere in between 1-10% of all emails) spam passes the filter - but not because spamassassin decides that it is ham but because the email never gets processed by spamassassin (the header shows no X-Spam at all). This is how I pass the emails to spamassassin: I am running qmail and each user has a identical .qmail file: # more .qmail |preline /usr/bin/procmail -m .procmailrc The .procmailrc looks like this: # more .procmailrc LOGFILE=procmail.log HOME=[... this folder] MAILDIR=$HOME/Maildir :0fw |spamc :0 * ^X-Spam-Flag: YES $HOME/junk :0 w | /usr/bin/deliverquota $MAILDIR As I mentioned before everything works fine in 90+% of all incoming mails. Any hints on what might happen to the emails that simply get passed along unchecked? Could it be that if the server is busy processing other email he skips a few? Any hints are welcome, thanks in advance. -- View this message in context: http://www.nabble.com/Some-emails-pass-spamassassin-unprocessed-tp22119041p22119041.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Some emails pass spamassassin unprocessed
On Friday 20 February 2009, Monky wrote: Hello, I am running the Spamd Daemon version 3.2.5 on my Linux web and mail server and in general it works well. From time to time (somewhere in between 1-10% of all emails) spam passes the filter - but not because spamassassin decides that it is ham but because the email never gets processed by spamassassin (the header shows no X-Spam at all). This is how I pass the emails to spamassassin: I am running qmail and each user has a identical .qmail file: # more .qmail |preline /usr/bin/procmail -m .procmailrc The .procmailrc looks like this: # more .procmailrc LOGFILE=procmail.log HOME=[... this folder] MAILDIR=$HOME/Maildir :0fw : |spamc and usually in your .procmailrc there will be a stanza that checks the mails size, and if over a certain size, skips the spamc piping. This is not I trust the whole .procmailrc file. If it is, its broken. There is also IIRC, a similar check for file size in spamc's config. | :0 * ^X-Spam-Flag: YES $HOME/junk :0 w : | /usr/bin/deliverquota $MAILDIR As I mentioned before everything works fine in 90+% of all incoming mails. Any hints on what might happen to the emails that simply get passed along unchecked? Could it be that if the server is busy processing other email he skips a few? Any hints are welcome, thanks in advance. -- Cheers, Gene There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. -Ed Howdershelt (Author) Even the best of friends cannot attend each other's funeral. -- Kehlog Albran, The Profit
Re: Some emails pass spamassassin unprocessed
On Fri, 2009-02-20 at 03:31 -0800, an anonymous Nabble user wrote:
Hello,
I am running the Spamd Daemon version 3.2.5 on my Linux web and mail server
and in general it works well. From time to time (somewhere in between 1-10%
of all emails) spam passes the filter - but not because spamassassin decides
that it is ham but because the email never gets processed by spamassassin
(the header shows no X-Spam at all).
Mentioning some numbers is good, though too qualitative. How many mails
is that per day, in absolute numbers?
The .procmailrc looks like this:
:0fw
|spamc
Without locking here, you can easily resource drain your server during
peaks. Think a smart spammer sending a couple mails to you (and/or
other users) at the same time. How many spamd children are running max?
While all of them are busy, excess mails won't be processed.
Keep in mind that the lock file probably is per user, while the max
spamd processes is server-wide.
Of course, never forget that there's a default max-size per message.
Unless told otherwise, spamc won't scan mail that's larger than 500
kByte. Probably not your issue, though, unless (most) of the unprocessed
mail you're talking about actually *is* large.
Also, this is the spamc default of safe fallback. That means, if there
is any issue in communicating with the daemon, the message will be
passed along unprocessed. But let's re-schedule that for later. (See
follow-up post.)
:0
* ^X-Spam-Flag: YES
$HOME/junk
:0 w
| /usr/bin/deliverquota $MAILDIR
Unrelated: Are you using mbox or maildir storage? The $MAILDIR hints you
actually are using maildir format, though the junk folder is an mbox
file!
With mbox, you seriously should add locking to any delivering recipe.
As I mentioned before everything works fine in 90+% of all incoming mails.
Any hints on what might happen to the emails that simply get passed along
unchecked? Could it be that if the server is busy processing other email he
skips a few?
Yes. Check your logs. If you are running out of spamd children, you'll
see something like this in the logs:
prefork: child states
One state indicating char per children. B is busy, I means idle. Idle
processes are ready to take a message. If you're seeing too many busy
children, you're server can't handle the load.
In that case, you /can/ increase the number of children, if you got
plenty of RAM. Limiting the parallel resource usage by locking (in
procmail) can help, too. And it definitely would be worth investigating
more, like how long the children take for processing a message. Too long
scan times can amplify this.
guenther
--
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Some emails pass spamassassin unprocessed
Monky wrote: Hello, I am running the Spamd Daemon version 3.2.5 on my Linux web and mail server and in general it works well. From time to time (somewhere in between 1-10% of all emails) spam passes the filter - but not because spamassassin decides that it is ham but because the email never gets processed by spamassassin (the header shows no X-Spam at all). Are the skipped messages over 500k in size? By default, spamc will not pass messages over 500k to spamd. To change this, pass the -s parameter to spamc and follow it with the size, in bytes, that should be the largest message passed to spamd. See also, the spamc docs: http://spamassassin.apache.org/full/3.2.x/doc/spamc.html
Re: Some emails pass spamassassin unprocessed
On 20.02.09 03:31, Monky wrote: I am running the Spamd Daemon version 3.2.5 on my Linux web and mail server and in general it works well. From time to time (somewhere in between 1-10% of all emails) spam passes the filter - but not because spamassassin decides that it is ham but because the email never gets processed by spamassassin (the header shows no X-Spam at all). if spamc can't connect spamd for any reason, it will use safe-fallback - pass mail unchecked. If you want to avoid this behaviour and cause a temporary failure, use the -x switch for spamc. Note that it also disables conectins multiple hosts if spamd is unreachable. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I feel like I'm diagonally parked in a parallel universe.
HELO checks give too high score together
Hello, I've received e-mail that received score 4.9 just because of the same problem - invalid HELO. * 2.8 RCVD_HELO_IP_MISMATCH Received: HELO and IP do not match, but should * 2.1 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO Received: from 88.102.6.114 (67.kcity.telenet.cz [194.228.203.67]) by 8.hotelulipy.cz (Postfix) with SMTP id censored for censored; date I think that combination above hits way too much. I know what is the meaning scoring and multiple rules but I even think they are too high. Mostly because those two score the same problem... I know I can configure local meta rule(s) to avoid this, however someone might think the same and agree that those scores should be reassigned. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. You have the right to remain silent. Anything you say will be misquoted, then used against you.
Re: Some emails pass spamassassin unprocessed
From: Monky promil...@yahoo.de Date: Fri, 20 Feb 2009 03:31:14 -0800 (PST) Hello, I am running the Spamd Daemon version 3.2.5 on my Linux web and mail server and in general it works well. From time to time (somewhere in between 1-10% of all emails) spam passes the filter - but not because spamassassin decides that it is ham but because the email never gets processed by spamassassin (the header shows no X-Spam at all). look in the mail log files to see what was happening when messages are passed through unprocessed. SpamAssassin could be waiting on lock files. For example, Bayes files are locked while an automatic Bayes expiry runs. -jeff
Spamassassin and KAV Antivirus
Hi, I'm using spamassassin called from amavisd-new and it's eating my server. Amavisd-new first calls KAV Antivirus (desktop version). I've been told that I could achieve better perfomance by using spamd only, and that it can call KAV by using a plugin. Is that true? If it is, where can I get such a plugin? Thanks Claudia Burman El Bolson - Patagonia Argentina
Re: HELO checks give too high score together
Matus UHLAR - fantomas wrote: Hello, I've received e-mail that received score 4.9 just because of the same problem - invalid HELO. * 2.8 RCVD_HELO_IP_MISMATCH Received: HELO and IP do not match, but should * 2.1 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO Received: from 88.102.6.114 (67.kcity.telenet.cz [194.228.203.67]) by 8.hotelulipy.cz (Postfix) with SMTP id censored for censored; date I think that combination above hits way too much. Why is a bogous HELO being generated in the first place? i.e.: why is an address literal used, but not the correct address literal? I've not seen a legitimate mail client do this, so I'm actually rather curious as to what happened. In the set0 mass-checks, this rule had a S/O of 0.996, which is *VERY* good. OVERALLSPAM% HAM% S/ORANK SCORE NAME 1.197 1.8719 0.00780.996 0.862.40 RCVD_HELO_IP_MISMATCH And that's a pretty large scale test of over 953k spam, and 540k nonspam emails. It matched a total of 43 of those nonspam messages.
vbounce and OTHER messaes.. FP's
vbounce does an increasable job of blocking those pesky backscatters from networks that do not validate their valid users on the 'bastion ' email server or proxy (and those who still insist on bouncing forged viruses, or spammers who create phoney email 'bounces') one of its strengths is also one of its weaknesses. in order to allow VALID bounces, it compares a set of whitelisted relays to the part of the original email (headers) included in the bounce. There comes its weakness. 'bounces' (or things vbounce things are bounces) like vacation, out of office and read receipt messages match signatures in vbounce, but never include the original emails. one more thing, well, many more do this also. Take automated ticketing systems (like RT). it adds a Auto-Submitted: auto-generated header line. so does bugzilla. so, ironically (as in both strange AND funny), if you post a bug to spamassassin's bugzilla, and get an automated email from it, vbounce marks it as BOUNCE_MESSAGE ANY_BOUNCE_MESSAGE. happens with emails from clamav when you submit a signature, and silly people at live (msn things) when you ask for a password reminder. I suggest taking things out of vbounce that do not and cannot ever have the whitelist relays in it. from a cpu / load standpoint, why bother checking whitelisted relays against signatures that cannot possibly have whitelist relays in it? From a maintenance and scoring standpoint, shouldn't the 'other bounces'/ machine generate emails be in a different file? Just my thoughts, what say you users who have to use this in production environments? (ps, I already posted a suggested patch to vbounce that takes (many) of the out of office messages out of the BOUNCE_MESSAGE and creates an OOO_BOUNCE_MESSAGE subsection, and will be experimenting with read receipt ones also) Id like to work on patches if there is an agreement as to what makes good practice. Like to keep it as upward compatible (as in not break anything) as possible. -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 *| *SECNAP Network Security Corporation * Certified SNORT Integrator * King of Spam Filters, SC Magazine 2008 * Information Security Award 2008, Info Security Products Guide * CRN Magazine Top 40 Emerging Security Vendors * Finalist 2009 Network Products Guide Hot Companies _ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ _
Re: HELO checks give too high score together
Matus UHLAR - fantomas wrote: I've received e-mail that received score 4.9 just because of the same problem - invalid HELO. * 2.8 RCVD_HELO_IP_MISMATCH Received: HELO and IP do not match, but should * 2.1 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO Received: from 88.102.6.114 (67.kcity.telenet.cz [194.228.203.67]) by 8.hotelulipy.cz (Postfix) with SMTP id censored for censored; date I think that combination above hits way too much. On 20.02.09 08:56, Matt Kettler wrote: Why is a bogous HELO being generated in the first place? i.e.: why is an address literal used, but not the correct address literal? I guess this happenns for hosts behing NAT, that do not know the real IP address under which they are accessing the internet. I've not seen a legitimate mail client do this, so I'm actually rather curious as to what happened. In the set0 mass-checks, this rule had a S/O of 0.996, which is *VERY* good. I've just seen another one... However the main problem is that most HELO rules fire independently together -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Remember half the people you know are below average.
Re: config no subject rewrite, learning spam headers
On Wed, 18 Feb 2009 23:00:03 + (UTC) Ray rsk-gm...@misinformation.org wrote: However, SA currently appears to be making spam judgement and to be bayes autolearning. (A reasonable default setup from the hosting provider.) * How do I determine what the current SA config is? The locations where spamassassin looks for configuration are listed in the main manpage. Specifically, can I see whether bayes is enabled, and whether it's auto-learning (if that's distinct from merely enabled)? If it appears to autolearning, then bayes and autolearning are enabled. Note that autolearning uses its own, more conservative, rules, it's not based on the normal single threshold - you should use sa-learn to manually train too, if you can. By default Bayes scoring wont turn-on until you've learned 200 spam, and 200 ham (non-spam) messages. If you are going to make a judgement about moving the threshold then you should ignore the early mails that lack BAYES_* hits. * Can I stop SA from judging spamminess (that is, making the binary declaration of whether something is spam, X-Spam-Status, X-Spam-Flag) and retain the scoring markup? I suppose this may not be important, as sa-learn is said to ignore prior SA markup, it's just that having the declaration sitting in the headers from there on makes these mails look spammy whether they truly are, and other more naive tools might be misled. Some third-party Baysian filters let you you ignore unwanted headers. Even if you use one that doesn't, a single spam/ham token isn't likely to have all that much effect compared to all the other SA tokens. There are two main ways to use SA with a separate Bayesian filter. One is to score it into SA (which you can't do) and the other is to let the Bayesian filter pick-up extra tokens from the SA headers. In the latter case you would probably want to leave in the result at the default threshold anyway. I think you could get rid of it by creating a custom header, but it's probably not worth the effort. * If I can't stop SA from judging spamminess, can I at least override the site-wide config to mark up subjects? I can't figure this out. Currently I have 'rewrite_header subject ', but that fails. The docs say the string should be set to 'a null value', but the config file's syntax for specifying nulls is not described. I believe it just means: rewrite_header subject
Re: HELO checks give too high score together
On Fri, 20 Feb 2009 15:11:42 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: On 20.02.09 08:56, Matt Kettler wrote: Why is a bogous HELO being generated in the first place? i.e.: why is an address literal used, but not the correct address literal? I guess this happenns for hosts behing NAT, that do not know the real IP address under which they are accessing the internet. Note that none of the addresses are private. The test ignores private addresses and mismatched addresses from the same /24.
netlawyers: why is this patentable?
wonder why this is patentable? sounds like preque filtering available in every mta since the early 90's... looks for 'helo/mailfrom/recpt to' then drops or accepts connection. http://www.freepatentsonline.com/7490128.html United States Patent 7490128 Abstract: The spam blocker monitors the SMTP/TCP/IP conversation between a sending message transfer agent MTA—0 and a receiving message transfer agent MTA—1; catches MTA—0's IP address IP—0, MTA—0's declared domain D—0, sender_address A—0; and recipient A—1; and uses this source and content based information to test for unsolicited messages. It interrupts the conversation when MTA—0 sends a command specifying the recipient (an “RCPT” command) and uses the various test results to decide if the message is suspected of being unsolicited. If the message is suspected of being unsolicited then it logs the rejected message, sends an error reply to MTA—0 which forces MTA—0 to terminate the connection with MTA—1 before the body of the message is transmitted; else it logs the allowed message, releases the intercepted RCPT command which allows the conversation between MTA—0 and MTA—1 to proceed. -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 *| *SECNAP Network Security Corporation * Certified SNORT Integrator * King of Spam Filters, SC Magazine 2008 * Information Security Award 2008, Info Security Products Guide * CRN Magazine Top 40 Emerging Security Vendors * Finalist 2009 Network Products Guide Hot Companies _ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ _
RE: netlawyers: why is this patentable?
-Original Message- From: Michael Scheidell [mailto:scheid...@secnap.net] Sent: Friday, February 20, 2009 9:24 PM wonder why this is patentable? sounds like preque filtering available in every mta since the early 90's... looks for 'helo/mailfrom/recpt to' then drops or accepts connection. Why are software ideas patentable, anyway? It is only to steal $$$ and to stop newcomers, which is the exact opposite of the original meaning of patenting... Have a read to http://www.nosoftwarepatents.com/ . It is an EU-based organization, but motivations are the same regardless of nationality. Here in EU we have a lot of (zealous?) public officers attempting to introduce software patens in any possible way. Giampaolo http://www.freepatentsonline.com/7490128.html United States Patent 7490128 Abstract: The spam blocker monitors the SMTP/TCP/IP conversation between a sending message transfer agent MTA-0 and a receiving message transfer agent MTA-1; catches MTA-0's IP address IP-0, MTA-0's declared domain D-0, sender_address A-0; and recipient A-1; and uses this source and content based information to test for unsolicited messages. It interrupts the conversation when MTA-0 sends a command specifying the recipient (an RCPT command) and uses the various test results to decide if the message is suspected of being unsolicited. If the message is suspected of being unsolicited then it logs the rejected message, sends an error reply to MTA-0 which forces MTA-0 to terminate the connection with MTA- 1 before the body of the message is transmitted; else it logs the allowed message, releases the intercepted RCPT command which allows the conversation between MTA-0 and MTA-1 to proceed. -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 *| *SECNAP Network Security Corporation * Certified SNORT Integrator * King of Spam Filters, SC Magazine 2008 * Information Security Award 2008, Info Security Products Guide * CRN Magazine Top 40 Emerging Security Vendors * Finalist 2009 Network Products Guide Hot Companies ___ __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ ___ __
Re: netlawyers: why is this patentable?
Giampaolo Tomassoni wrote: -Original Message- From: Michael Scheidell [mailto:scheid...@secnap.net] Sent: Friday, February 20, 2009 9:24 PM wonder why this is patentable? Perhaps just because someone has the Chutzpah to try to patent it and the patent office hasn't a clue. Technology of all sorts has moved too quickly for the patent office and/or the patent laws to keep up. Another example is a U.S. company that uses recombinant DNA to put an unusual color in a bean. Then they patent it and sue a Mexican company and block imports of a bean that the Mexicans have been growing for generations. That's just nucking futs. sounds like preque filtering available in every mta since the early 90's... looks for 'helo/mailfrom/recpt to' then drops or accepts connection. Why are software ideas patentable, anyway? It is only to steal $$$ and to stop newcomers, which is the exact opposite of the original meaning of patenting... or stop others from doing what they've been doing all along and gain a competitive edge in the process or make them pay. All of which works iff the patent office hasn't a clue. Have a read to http://www.nosoftwarepatents.com/ . It is an EU-based organization, but motivations are the same regardless of nationality. Here in EU we have a lot of (zealous?) public officers attempting to introduce software patens in any possible way. Giampaolo http://www.freepatentsonline.com/7490128.html United States Patent 7490128 Abstract: The spam blocker monitors the SMTP/TCP/IP conversation between a sending message transfer agent MTA-0 and a receiving message transfer agent MTA-1; catches MTA-0's IP address IP-0, MTA-0's declared domain D-0, sender_address A-0; and recipient A-1; and uses this source and content based information to test for unsolicited messages. It interrupts the conversation when MTA-0 sends a command specifying the recipient (an RCPT command) and uses the various test results to decide if the message is suspected of being unsolicited. If the message is suspected of being unsolicited then it logs the rejected message, sends an error reply to MTA-0 which forces MTA-0 to terminate the connection with MTA- 1 before the body of the message is transmitted; else it logs the allowed message, releases the intercepted RCPT command which allows the conversation between MTA-0 and MTA-1 to proceed. -- Michael Scheidell, CTO -- --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst hoogen...@bio.umass.edu --- Erdös 4
Re: netlawyers: why is this patentable?
On Fri, 2009-02-20 at 16:54 -0500, Chris Hoogendyk wrote: Perhaps just because someone has the Chutzpah to try to patent it and the patent office hasn't a clue. Technology of all sorts has moved too quickly for the patent office and/or the patent laws to keep up. Another example is a U.S. company that uses recombinant DNA to put an unusual color in a bean. Then they patent it and sue a Mexican company and block imports of a bean that the Mexicans have been growing for generations. That's just nucking futs. Sounds like Monsanto at work. -- Lindsay Haisley | Everything works|Accredited FMP Computer Services | if you let it | by the 512-259-1190 |(The Roadie) | Austin Better http://www.fmp.com| | Business Bureau
Re: netlawyers: why is this patentable?
Michael Scheidell wrote: wonder why this is patentable? Loads of things are patentable in the meaning that someone manages to get a patent. That doesn't mean the patent can witstand a challenge. You never know for sure wether a patent (or a trademark) is fully valid until it is is disputed (in court) and survives. sounds like preque filtering available in every mta since the early 90's... It sound like more like the bastard child of a packet sniffing trafic analyzing firewall and a spam-scanning smtp proxy. looks for 'helo/mailfrom/recpt to' then drops or accepts connection. From the abstract it's possible that it does so at a firewall level rather than as an MTA, though it might also describe a more common SMTP proxy. /J -- Jonas Eckerman, FSDB Fruktträdet http://whatever.frukt.org/ http://www.fsdb.org/ http://www.frukt.org/
Re: netlawyers: why is this patentable?
On Fri, 2009-02-20 at 17:01 -0600, Lindsay Haisley wrote: On Fri, 2009-02-20 at 16:54 -0500, Chris Hoogendyk wrote: Perhaps just because someone has the Chutzpah to try to patent it and the patent office hasn't a clue. Technology of all sorts has moved too quickly for the patent office and/or the patent laws to keep up. Another example is a U.S. company that uses recombinant DNA to put an unusual color in a bean. Then they patent it and sue a Mexican company and block imports of a bean that the Mexicans have been growing for generations. That's just nucking futs. Sounds like Monsanto at work. Exactly. Two words: prior art. If the US patent office had the balls to enforce that clause then it wouldn't be necessary to argue about software patents. Knuth would invalidate most of them and Hopper would have put paid to most of the rest. Another point: IIRC[1] the patent is required to describe an invention in enough detail to allow a 3rd party to make the patented item. If *that* was enforced then a whole bunch of speculative patents would die when the claimant would be unable to show a working model built from the patent description. [1] I know this applies to British patents but am uncertain whether it still applies to US patents. If it doesn't, it ought to. Martin
Re: HELO checks give too high score together
Since you're bouncing any off-list emails because you reject my entire ISP, I'm going to drop out of aiding on this matter. Fix your own domain's over-zealous behaviors first. Matus UHLAR - fantomas wrote: Matus UHLAR - fantomas wrote: I've received e-mail that received score 4.9 just because of the same problem - invalid HELO. * 2.8 RCVD_HELO_IP_MISMATCH Received: HELO and IP do not match, but should * 2.1 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO Received: from 88.102.6.114 (67.kcity.telenet.cz [194.228.203.67]) by 8.hotelulipy.cz (Postfix) with SMTP id censored for censored; date I think that combination above hits way too much. On 20.02.09 08:56, Matt Kettler wrote: Why is a bogous HELO being generated in the first place? i.e.: why is an address literal used, but not the correct address literal? I guess this happenns for hosts behing NAT, that do not know the real IP address under which they are accessing the internet. I've not seen a legitimate mail client do this, so I'm actually rather curious as to what happened. In the set0 mass-checks, this rule had a S/O of 0.996, which is *VERY* good. I've just seen another one... However the main problem is that most HELO rules fire independently together
Re: HELO checks give too high score together
Matus UHLAR - fantomas a écrit : Matus UHLAR - fantomas wrote: I've received e-mail that received score 4.9 just because of the same problem - invalid HELO. * 2.8 RCVD_HELO_IP_MISMATCH Received: HELO and IP do not match, but should * 2.1 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO Received: from 88.102.6.114 (67.kcity.telenet.cz [194.228.203.67]) by 8.hotelulipy.cz (Postfix) with SMTP id censored for censored; date I think that combination above hits way too much. On 20.02.09 08:56, Matt Kettler wrote: Why is a bogous HELO being generated in the first place? i.e.: why is an address literal used, but not the correct address literal? I guess this happenns for hosts behing NAT, that do not know the real IP address under which they are accessing the internet. $ host 88.102.6.114 114.6.102.88.in-addr.arpa domain name pointer 114.6.broadband7.iol.cz. Are - iol.cz - telenet.cz - hotelulipy.cz the same organisation? if not, this is direct to MX junk. BTW. which (legitimate and not outdated) mail clients helo with a bare IP? I've not seen a legitimate mail client do this, so I'm actually rather curious as to what happened. In the set0 mass-checks, this rule had a S/O of 0.996, which is *VERY* good. I've just seen another one... However the main problem is that most HELO rules fire independently together try a meta that uses an AND and run a mass check. I'm sure I would get a score of 5 :)
Re: Spamassassin and KAV Antivirus
Claudia Burman a écrit : Hi, I'm using spamassassin called from amavisd-new and it's eating my server. make sure you're not using huge rulesets. Amavisd-new first calls KAV Antivirus (desktop version). I've been told that I could achieve better perfomance by using spamd only, and that it can call KAV by using a plugin. Is that true? I doubt it. you need to measure before getting to any consequences. try finding out which part of the solution is eating your server. If it is, where can I get such a plugin?
Re: netlawyers: why is this patentable?
Michael Scheidell a écrit : wonder why this is patentable? sounds like preque filtering available in every mta since the early 90's... looks for 'helo/mailfrom/recpt to' then drops or accepts connection. - if their spam blocker is linked in the MTA or is a firewall, then this has been done since long - if it is a sniffer that sends fake RSTs, then it's silly. but I'm sure it's something else, I'm just not sure whether it's something to drink or something to smoke ;-p http://www.freepatentsonline.com/7490128.html United States Patent 7490128 Abstract: The spam blocker monitors the SMTP/TCP/IP conversation between a sending message transfer agent MTA—0 and a receiving message transfer agent MTA—1; catches MTA—0's IP address IP—0, MTA—0's declared domain D—0, sender_address A—0; and recipient A—1; and uses this source and content based information to test for unsolicited messages. It interrupts the conversation when MTA—0 sends a command specifying the recipient (an “RCPT” command) and uses the various test results to decide if the message is suspected of being unsolicited. If the message is suspected of being unsolicited then it logs the rejected message, sends an error reply to MTA—0 which forces MTA—0 to terminate the connection with MTA—1 before the body of the message is transmitted; else it logs the allowed message, releases the intercepted RCPT command which allows the conversation between MTA—0 and MTA—1 to proceed.
Re: HELO checks give too high score together
Matt Kettler a écrit : Since you're bouncing any off-list emails because you reject my entire ISP, I'm going to drop out of aiding on this matter. probably a rule that considers vms173007pub.verizon.net as a dynamic name... Fix your own domain's over-zealous behaviors first.
Re: netlawyers: why is this patentable?
Martin Gregorie wrote: On Fri, 2009-02-20 at 17:01 -0600, Lindsay Haisley wrote: On Fri, 2009-02-20 at 16:54 -0500, Chris Hoogendyk wrote: Perhaps just because someone has the Chutzpah to try to patent it and the patent office hasn't a clue. Technology of all sorts has moved too quickly for the patent office and/or the patent laws to keep up. Another example is a U.S. company that uses recombinant DNA to put an unusual color in a bean. Then they patent it and sue a Mexican company and block imports of a bean that the Mexicans have been growing for generations. That's just nucking futs. Sounds like Monsanto at work. Exactly. Two words: prior art. If the US patent office had the balls to enforce that clause then it wouldn't be necessary to argue about software patents. Knuth would invalidate most of them and Hopper would have put paid to most of the rest. IMHO software patents are a nonsense. Where is the *inventive* step that any knowledgeable programmer wouldn't determine to be obvious? Another point: IIRC[1] the patent is required to describe an invention in enough detail to allow a 3rd party to make the patented item. If *that* was enforced then a whole bunch of speculative patents would die when the claimant would be unable to show a working model built from the patent description. [1] I know this applies to British patents but am uncertain whether it still applies to US patents. If it doesn't, it ought to. In my limited experience (I hold a few British patents - not software related), it's actually more than that. One can file a Patent Application for almost anything, and quite speculatively. However, for the patent to be granted (normally a year after the Application) one must also have exemplified one's claims or must withdraw them. If one fails to exemplify claims made in the Application they then become public domain and hence prior art and are not subsequently patentable. My patents are in the field of cancer research. So for example, one could file an Application for a new anticancer drug. At the time of filing the Application it could just be an idea, but in order to get the patent awarded one must have made the drug and be able to present some evidence that it is effective against cancer as per the claims made in the Application. Simple preliminary results would normally suffice, but the more evidence one is able to present the stronger the case one has. Unless it's an extremely competitive field one would normally wait until you have some evidence before filing an Application given the costs associated with the process (patent lawyers don't come cheap), unless you're in the business of filing many speculative Applications. As usual, the real winners are the lawyers.
RE: Some emails pass spamassassin unprocessed
if spamc can't connect spamd for any reason, it will use safe-fallback - pass mail unchecked. If you want to avoid this behaviour and cause a temporary failure, use the -x switch for spamc. Note that it also disables conectins multiple hosts if spamd is unreachable. -- Matus UHLAR - matus what do you mean by temp failure? SA rejection? does SA rejection then go backwards on this and equal SMTP rejection depending on the setup? - rh
Re: HELO checks give too high score together
On Sat, 21 Feb 2009 02:19:30 +0100 mouss mo...@ml.netoyen.net wrote: $ host 88.102.6.114 114.6.102.88.in-addr.arpa domain name pointer 114.6.broadband7.iol.cz. Are - iol.cz - telenet.cz - hotelulipy.cz the same organisation? if not, this is direct to MX junk. BTW. which (legitimate and not outdated) mail clients helo with a bare IP? The OP didn't actually say it was a mail-client to server received header, or that it's spam. Since he's saying that the score is too high, and is speculating about it being due to NAT, I would assume it's a legitimate mail.
Re: HELO checks give too high score together
mouss wrote: Matt Kettler a écrit : Since you're bouncing any off-list emails because you reject my entire ISP, I'm going to drop out of aiding on this matter. probably a rule that considers vms173007pub.verizon.net as a dynamic name... No, rejecting anything listed postmaster.rfc-ignorant.org. Fix your own domain's over-zealous behaviors first.
false positive on X-Mailer: Microsoft Outlook
Hi I have a message in hand that is triggering false positives based on the ratware rules in 3.2.4. The specific headers are: Message-ID: blu0-smtp74e123fde12343a12de12bd1...@phx.gbl X-Mailer: Microsoft Outlook, Build 10.0.6838 Specifically, it seems that the X-Mailer header matches __OUTLOOK_DOLLARS_MUA, and the Message-ID matches __HOTMAIL_BAYDAV_MSGID The problem is that __OUTLOOK_DOLLARS_MUA is included in __FORGED_OUTLOOK_DOLLARS without an exception for __HOTMAIL_BAYDAV_MSGID so __FORGED_OUTLOOK_DOLLARS causes a hit on FORGED_MUA_OUTLOOK. It seems to me, given this valid, non spam, non-ratware originating message, that __FORGED_OUTLOOK_DOLLARS needs to include an ! __HOTMAIL_BAYDAV_MSGID in the exception list. Thots? b. P.S. I do see that trunk is handling this combination of headers in a fairly different manner. But that doesn't change the fact that this MUA is causing false positives on 3.2, even with the latest (sa- update_3.2_20081231172858 according to SVN) 3.2 udpate.
Re: Some emails pass spamassassin unprocessed
if spamc can't connect spamd for any reason, it will use safe-fallback - pass mail unchecked. If you want to avoid this behaviour and cause a temporary failure, use the -x switch for spamc. Note that it also disables conectins multiple hosts if spamd is unreachable. On 20.02.09 17:59, RobertH wrote: what do you mean by temp failure? that spamc returns error code, which should be handled with program calling it. SA rejection? does SA rejection then go backwards on this and equal SMTP rejection depending on the setup? I think that spamassassin milter is able to handle that and return temporary failure to the SMTP client. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Nothing is fool-proof to a talented fool.
Re: HELO checks give too high score together
On 20.02.09 19:26, Matt Kettler wrote: Since you're bouncing any off-list emails because you reject my entire ISP, I'm going to drop out of aiding on this matter. I'm not rejecting your ISP. I'm rejecting mail from addresses I could not complain back to. Fix your own domain's over-zealous behaviors first. Fix your domain's RFC conformity first -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. He who laughs last thinks slowest.
Re: config no subject rewrite, learning spam headers
RW rwmaillists at googlemail.com writes: On Wed, 18 Feb 2009 23:00:03 + (UTC) Ray rsk-gmane at misinformation.org wrote: * How do I determine what the current SA config is? The locations where spamassassin looks for configuration are listed in the main manpage. I managed to find the config directory on this system, thanks for the pointer. I guess I have to parse all of these files to know how SA is actually config'd? Alas, I was hoping for something like Postfix's `postconf` to show the active/final configuration in its entirety. Where can one submit a feature request, and does this sound like a sensible one? If it appears to autolearning, then bayes and autolearning are enabled. The magically incrementing `sa-learn --dump magic|grep am` values suggest so. It's odd that there isn't any indication in the X-Spam-Status header that this is happening, as one would expect after reading the wiki article AutolearnNotWorking. Note that autolearning uses its own, more conservative, rules, it's not based on the normal single threshold - you should use sa-learn to manually train too, if you can. I noticed the additional thresholds for autolearning. I was hoping to do manual training only, but maybe that level of control is just not achievable in my circumstance. (The problem being that my headers may be bad for sa-learn.) By default Bayes scoring wont turn-on until you've learned 200 spam, and 200 ham (non-spam) messages. If you are going to make a judgement about moving the threshold then you should ignore the early mails that lack BAYES_* hits. I imagine after Bayes scoring goes into effect I'll have a nicer distribution of scores (pushed towards the poles). * Can I stop SA from judging spamminess (that is, making the binary declaration of whether something is spam, X-Spam-Status, X-Spam-Flag) and retain the scoring markup? I suppose this may not be important, as sa-learn is said to ignore prior SA markup, it's just that having the declaration sitting in the headers from there on makes these mails look spammy whether they truly are, and other more naive tools might be misled. Some third-party Baysian filters let you you ignore unwanted headers. I think this response might mean that I can't stop SA adding X-Spam-Status and/or X-Spam-Flag, as the response proceeds without answering the question directly. I would like to have just the scoring without the judgements, but I suppose again this is not an issue with regards to future application of sa-learn. The only other markup I feel it's actually necessary to hinder is the subject markup. Even if you use one that doesn't, a single spam/ham token isn't likely to have all that much effect compared to all the other SA tokens. There That's reassuring. are two main ways to use SA with a separate Bayesian filter. One is to score it into SA (which you can't do) and the other is to let the Bayesian filter pick-up extra tokens from the SA headers. In the latter case you would probably want to leave in the result at the default threshold anyway. And I or another person shouldjust remember while looking at these emails that the judgement is not necessarily correct. I guess I'm including myself (and other humans) among the naive tools to worry about. I think you could get rid of it by creating a custom header, but it's probably not worth the effort. It here referring to the final spamminess judgement? Oh, sorry, I misunderstood earlier, then. * If I can't stop SA from judging spamminess, can I at least override the site-wide config to mark up subjects? I can't figure this out. Currently I have 'rewrite_header subject ', but that fails. The docs say the string should be set to 'a null value', but the config file's syntax for specifying nulls is not described. I believe it just means: rewrite_header subject Ah, that's one of the permutations I tried. Any idea why it may not have worked? I've been able to modify required_score, as is evidenced by mail headers that come through, so I must be working in a picked-up config file. (Again a `sa-conf` to view live/final config would be much better for me than tweaking my user config file's required_score and then waiting for a spam to arrive so I can know if a config specification went into effect.) My only guess now is that somehow site-wide config overrides user config for this item or that user config for this item is disallowed. Right now SA's config'd to prepend ***SPAM*** . But I don't see this string or the string rewrite_header anywhere in the discovered config directory or my user config. So this is part of what I meant earlier about how my headers may be bad. The subjects. I can't imagine SA would know how to ignore this prepended string -- could it? The wiki article LearningMarkedUpMessages suggests that Subject header tagged etc. are automatically removed for learning, but the POD says ``Note that you should only use the
