RE: Too false negative
> --[ UxBoD ]-- wrote: > > policyd works a treat :) V2 is also in development aswell. > > > > it's not the same. I don't know why they call it V2. > As far as I know, Cami is no more involved. so I would stick > with the "current" (which is a single C threaded program). So you still prefer policyd not policydV2.. Some questions: - Does any web interface for policyd exist? - I have different SMTP gateways, on each of which I have to install policyd. Is it possible to share a single DB between the different policyd servers? For other possible question I will refer to policyd ML. :-) Thanks, rocsca
RE: Too false negative
> policyd works a treat :) V2 is also in development aswell. I will take in account your judge.. :-) rocsca
Re: Too false negative
--[ UxBoD ]-- wrote: policyd works a treat :) V2 is also in development aswell. it's not the same. I don't know why they call it V2. As far as I know, Cami is no more involved. so I would stick with the "current" (which is a single C threaded program).
Re: Too false negative
policyd works a treat :) V2 is also in development aswell. Regards, -- --[ UxBoD ]-- // PGP Key: "curl -s http://www.splatnix.net/uxbod.asc | gpg --import" // Fingerprint: F57A 0CBD DD19 79E9 1FCC A612 CB36 D89D 2C5A 3A84 // Keyserver: www.keyserver.net Key-ID: 0x2C5A3A84 // Phone: +44 845 869 2749 SIP Phone: [EMAIL PROTECTED] - "Rocco Scappatura" <[EMAIL PROTECTED]> wrote: > > What do I need to set up GL? Only the command below or there is > > something other parameter that I could set up (eg: the time spent > > before a message is accepted and so on)? > > > > > > of course, you need to install a policy server! Cami's > policyd is a good choice (it also has other features such > throttling, blacklisting, ... > etc). for postfix config see below. I already sow it quickly.. I hope it usage is not too 'invasive' with my current system.. Any way I will try to use it and I let you know.. Thanks, rocsca -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
RE: Too false negative
> > What do I need to set up GL? Only the command below or there is > > something other parameter that I could set up (eg: the time spent > > before a message is accepted and so on)? > > > > > > of course, you need to install a policy server! Cami's > policyd is a good choice (it also has other features such > throttling, blacklisting, ... > etc). for postfix config see below. I already sow it quickly.. I hope it usage is not too 'invasive' with my current system.. Any way I will try to use it and I let you know.. Thanks, rocsca
Re: Too false negative
Rocco Scappatura wrote: And spammer are becoming more faster as the time goes on.. Is it convenient to use gray listing newer bots retry, so GL is only effective is the time interval is large enough, but that's not a neutral thing so should be restricted to suspicious mail. That's what I use GL for anyway. What do I need to set up GL? Only the command below or there is something other parameter that I could set up (eg: the time spent before a message is accepted and so on)? of course, you need to install a policy server! Cami's policyd is a good choice (it also has other features such throttling, blacklisting, ... etc). for postfix config see below. the spam you showed has: Received: from [125.128.59.158] (unknown [125.128.59.158]) which means the client is "unknown" and it helo'ed with a literal IP (it's from Korea too but let's ignore this). My postfix has a check_helo_acces with a pcre: /^[/ reject_unknown_client, policy_greylist This rejects mail if the client is unknown and helo's with a literal IP. It's very interesting.. In what restriction do I have to put the rulese above? see below. I've not seen literal IPs in ham on an MX. Note that this test must not be applied on an MSA: MUAs like Thunderbird do helo with a literal IP. Infact.. Indeed I'm not using MSA.. So this complicates the things.. :-( Not really, because when using port 25, submitted mail is whitelisted via permit_mynetworks, permit_sasl_authenticated. Here is a restrictions example. smtpd_recipient_restrictions = # allow submission via port 25 permit_mynetworks permit_sasl_authenticated # no relay from here reject_unauth_destination # non fqdn addresses are not valid reject_non_fqdn_sender reject_non_fqdn_recipient # recipient BL and WL, traps, spamlovers ... check_recipient_access ${pcre_prefix}/recipient_acl check_recipient_access ${hash_prefix}/recipient_acl # sender BL check_sender_access ${pcre_prefix}/sender_acl # address validation reject_unlisted_recipient reject_unlisted_sender # "site" client WL and BL check_client_access ${cidr_prefix}/client_acl check_client_access ${hash_prefix}/client_acl # DNSWL check_client_access ${cidr_prefix}/dnswl/postfix-dnswl-permit reject_invalid_helo_hostname # this may catch misconfigured MTAs: reject_non_fqdn_helo_hostname # obvious helo forgery (our domain, our IP, ...) check_helo_access ${hash_prefix}/helo_acl # helo discrepancies check_helo_access ${pcre_prefix}/helo_acl # if we can't reach them, reject them reject_unknown_sender_domain # block bogus MX, tld wildcard MX, ... check_sender_mx_access ${cidr_prefix}/sender_mx_acl # DNSBL checks reject_rbl_client ... smtpd_restriction_class = policy_greylist ... policy_greylist = check_policy_service inet:127.0.0.1:10031 the variables like cidr_prefix are defined like this: cidr_prefix = cidr:/etc/postfix/maps/cidr ... If you want to avoid further checks when greylisting, you need to configure the policy service to return DEFER instead of DEFER_IF_PERMIT. The test is run before DNSBL checks, so it saves some cycles and reduces the load on DNSBL sites. these days, the test catches about 15% of mail rejected at MTA time. Note that reject_unknown_client returns a temp error, but unlike GL, you'll need to whitelist the client if you want to accept his mail). if this is a real issue, just remove the reject_unknown_client part and leave the greylisting check. but So you are saying that I have to WL the client that present himself to my server with an IP rather than a hostname? I don't understand. you only need to whitelist a client if you want to accept his mail _and_ he triggers one of your checks. if you have a doubt, use this for some time: /^\[/ warn_if_reject reject_unknown_client, policy_greylist and check your logs during some period to see if you get legitimate clients using a literal IP and triggering reject_unknown_client. And how I could withelist that client? see above (the .../client_acl lines). but if you find yourself whitelisting many clients, then it's time to remove the check instead. manual whitelisting is only good if it is rare. Aren't we getting OT?
RE: Too false negative
> > And spammer are becoming more faster as the time goes on.. Is it > > convenient to use gray listing > > newer bots retry, so GL is only effective is the time > interval is large enough, but that's not a neutral thing so > should be restricted to suspicious mail. That's what I use GL > for anyway. What do I need to set up GL? Only the command below or there is something other parameter that I could set up (eg: the time spent before a message is accepted and so on)? > the spam you showed has: > > Received: from [125.128.59.158] (unknown [125.128.59.158]) > > > which means the client is "unknown" and it helo'ed with a > literal IP (it's from Korea too but let's ignore this). My > postfix has a check_helo_acces with a pcre: > > /^[/ reject_unknown_client, policy_greylist > > This rejects mail if the client is unknown and helo's with a > literal IP. It's very interesting.. In what restriction do I have to put the rulese above? > I've not seen literal IPs in ham on an MX. Note that this > test must not be applied on an MSA: MUAs like Thunderbird do > helo with a literal IP. Infact.. Indeed I'm not using MSA.. So this complicates the things.. :-( > The test is run before DNSBL checks, so it saves some cycles > and reduces the load on DNSBL sites. these days, the test > catches about 15% of mail rejected at MTA time. > > Note that reject_unknown_client returns a temp error, but > unlike GL, you'll need to whitelist the client if you want to > accept his mail). if this is a real issue, just remove the > reject_unknown_client part and leave the greylisting check. but So you are saying that I have to WL the client that present himself to my server with an IP rather than a hostname? And how I could withelist that client? > of course, this is mostly a temporary cure. if ratware learns > to helo with a hostname, it won't be caught. but let's fight > the spam of today for now ;-p I agree with.. Compliment for your exahustive argumentation.. rocsca
Re: Too false negative
Rocco Scappatura wrote: % telnet yourserver 25 ... EHLO somehostname ... MAIL FROM: ... RCPT TO: DATA copy-patse the message with full headers except the Delivered-To that contains your recipient address end with a line containing a dot ('.') like this: . QUIT Infact I get: Feb 26 23:07:50 av4 amavis[17589]: (17589-03) Blocked SPAM, [] [] <[EMAIL PROTECTED]> -> <>, quarantine: r/spam-rGPEbZ4mzhH4.gz, Message-ID: <[EMAIL PROTECTED]>, mail_id: rGPEbZ4mzhH4, Hits: 7.193, size: 4063, 1874 ms And spammer are becoming more faster as the time goes on.. Is it convenient to use gray listing newer bots retry, so GL is only effective is the time interval is large enough, but that's not a neutral thing so should be restricted to suspicious mail. That's what I use GL for anyway. or there is something other effective tecnique that I could use to reduce false negative? Thanks, rocsca the spam you showed has: Received: from [125.128.59.158] (unknown [125.128.59.158]) which means the client is "unknown" and it helo'ed with a literal IP (it's from Korea too but let's ignore this). My postfix has a check_helo_acces with a pcre: /^[/ reject_unknown_client, policy_greylist This rejects mail if the client is unknown and helo's with a literal IP. I've not seen literal IPs in ham on an MX. Note that this test must not be applied on an MSA: MUAs like Thunderbird do helo with a literal IP. The test is run before DNSBL checks, so it saves some cycles and reduces the load on DNSBL sites. these days, the test catches about 15% of mail rejected at MTA time. Note that reject_unknown_client returns a temp error, but unlike GL, you'll need to whitelist the client if you want to accept his mail). if this is a real issue, just remove the reject_unknown_client part and leave the greylisting check. but of course, this is mostly a temporary cure. if ratware learns to helo with a hostname, it won't be caught. but let's fight the spam of today for now ;-p
Re: Too false negative
On Tue, 2008-02-26 at 23:14 +0100, Rocco Scappatura wrote: > And spammer are becoming more faster as the time goes on.. Is it > convenient to use gray listing or there is something other effective > tecnique that I could use to reduce false negative? Grey-listing helps, but seldom because the URI/IP/body hash is added to a suribl/rbl/razor . Just confounding the BOT's that only try once is enough to get rid of piles of SPAM. -- Daniel J McDonald, CCIE #2495, CISSP #78281, CNX Austin Energy http://www.austinenergy.com signature.asc Description: This is a digitally signed message part
Re: Too false negative
> % telnet yourserver 25 > ... > EHLO somehostname > ... > MAIL FROM: > ... > RCPT TO: > DATA > copy-patse the message with full headers except the Delivered-To that > contains your recipient address > end with a line containing a dot ('.') like this: > . > QUIT Infact I get: Feb 26 23:07:50 av4 amavis[17589]: (17589-03) Blocked SPAM, [] [] <[EMAIL PROTECTED]> -> <>, quarantine: r/spam-rGPEbZ4mzhH4.gz, Message-ID: <[EMAIL PROTECTED]>, mail_id: rGPEbZ4mzhH4, Hits: 7.193, size: 4063, 1874 ms And spammer are becoming more faster as the time goes on.. Is it convenient to use gray listing or there is something other effective tecnique that I could use to reduce false negative? Thanks, rocsca
Re: Too false negative
Rocco Scappatura wrote: Rocco Scappatura wrote: [snip] Sorry It was not the case to send the entire email.. Here the X-Spam-Status after running the message against 'spamassassin -D': X-Spam-Status: Yes, score=11.2 required=5.0 tests=AWL,BAYES_50,HTML_MESSAGE, RATWARE_MS_HASH,RATWARE_OUTLOOK_NONAME,RDNS_NONE,URIBL_BLACK,URIBL_JP_SU RBL, URIBL_OB_SURBL,URIBL_SC_SURBL autolearn=unavailable version=3.2.4 But it is really strange from amavisd-new log I see that the message is passed as clean: the URL may have been added in $uri lists in the meantime. That said, make sure Bayes is using the right "user". rerun spamassassin as the amavisd user. if your Bayes db is in mysql, use bayes_sql_override_username to force a single user. X-Spam-Status: Yes, score=6.3 required=5.0 tests=AWL,BAYES_50,HTML_MESSAGE, RATWARE_MS_HASH,RATWARE_OUTLOOK_NONAME,RDNS_NONE,URIBL_BLACK,URIBL_JP_SURBL, URIBL_OB_SURBL,URIBL_SC_SURBL autolearn=unavailable version=3.2.4 What URL? What is $uri_list? URIBL, SURBL, ... etc. the message contains one or more URIs that are listed. but they may have been listed after you received the message which would explain why the message was not caught at reception time. To make sure, copy one of the messages, remove the Delivered-To header the top (if yoy leave it, you'll get a loop error from postfix) and resubmit the message for example using telnet: % telnet yourserver 25 ... EHLO somehostname ... MAIL FROM: ... RCPT TO: DATA copy-patse the message with full headers except the Delivered-To that contains your recipient address end with a line containing a dot ('.') like this: . QUIT you can retrieve the from the return-Path header, and the from the Delivered-To header that you removed before resubmitting the message, or use any address you want. make sure the message passes through amavisd-new (in case you submit from a "whitelisted" client). If the client is not in your trusted_network, the test may pollute your AWL. you could disable AWL while testing. when your receive the message, see if it was caught by the URI* tests. I had already set bayes_sql_override_username: [EMAIL PROTECTED]:/tmp> cat /etc/mail/spamassassin/local.cf | grep bayes_sql_override_username what's this? you should only have the following one: bayes_sql_override_username amavis Is it possible that there is a lack of spamhaus? I suppose that I query the DNSBL much more then 100.000 times per day.. :-( that doesn't explain the miss because the message is caught by other checks. to test for spamhaus access, try % host 2.0.0.127.zen.spamhaus.org you should see something like this: 2.0.0.127.zen.spamhaus.org has address 127.0.0.2 2.0.0.127.zen.spamhaus.org has address 127.0.0.10 2.0.0.127.zen.spamhaus.org has address 127.0.0.4 if you are doing to many queries, you may need to pay.
Re: Too false negative
> Rocco Scappatura wrote: >>> [snip] >> >> Sorry It was not the case to send the entire email.. Here the >> X-Spam-Status after running the message against 'spamassassin -D': >> >> X-Spam-Status: Yes, score=11.2 required=5.0 >> tests=AWL,BAYES_50,HTML_MESSAGE, >> >> RATWARE_MS_HASH,RATWARE_OUTLOOK_NONAME,RDNS_NONE,URIBL_BLACK,URIBL_JP_SU >> RBL, >> URIBL_OB_SURBL,URIBL_SC_SURBL autolearn=unavailable >> version=3.2.4 >> >> But it is really strange from amavisd-new log I see that the message is >> passed as clean: >> >> > > the URL may have been added in $uri lists in the meantime. That said, > make sure Bayes is using the right "user". rerun spamassassin as the > amavisd user. if your Bayes db is in mysql, use > bayes_sql_override_username to force a single user. X-Spam-Status: Yes, score=6.3 required=5.0 tests=AWL,BAYES_50,HTML_MESSAGE, RATWARE_MS_HASH,RATWARE_OUTLOOK_NONAME,RDNS_NONE,URIBL_BLACK,URIBL_JP_SURBL, URIBL_OB_SURBL,URIBL_SC_SURBL autolearn=unavailable version=3.2.4 What URL? What is $uri_list? I had already set bayes_sql_override_username: [EMAIL PROTECTED]:/tmp> cat /etc/mail/spamassassin/local.cf | grep bayes_sql_override_username bayes_sql_override_username amavis Is it possible that there is a lack of spamhaus? I suppose that I query the DNSBL much more then 100.000 times per day.. :-( Thanks, rocsca
Re: Too false negative
Rocco Scappatura wrote: [snip] Sorry It was not the case to send the entire email.. Here the X-Spam-Status after running the message against 'spamassassin -D': X-Spam-Status: Yes, score=11.2 required=5.0 tests=AWL,BAYES_50,HTML_MESSAGE, RATWARE_MS_HASH,RATWARE_OUTLOOK_NONAME,RDNS_NONE,URIBL_BLACK,URIBL_JP_SU RBL, URIBL_OB_SURBL,URIBL_SC_SURBL autolearn=unavailable version=3.2.4 But it is really strange from amavisd-new log I see that the message is passed as clean: the URL may have been added in $uri lists in the meantime. That said, make sure Bayes is using the right "user". rerun spamassassin as the amavisd user. if your Bayes db is in mysql, use bayes_sql_override_username to force a single user. Feb 26 08:09:48 av4 amavis[18267]: (18267-12) Passed CLEAN, [125.128.59.158] [125.128.59.158] <[EMAIL PROTECTED]> -> >,>,>, Message-ID: <[EMAIL PROTECTED]>, mail_id: kgXmlG1zg5ao, Hits: 3.558, size: 3731, queued_as: 9D8E775037D, 2132 ms rocsca
RE: Too false negative
> > Since some days the number of SMTP connections rejected by > my server > > is increased (maybe doubled). It doesn't worry me. But > there is a side > > effect because even the number of false negative is increased. > > > > For example, at the moment a spam message with this header is > > considered clean by Amavisd-new-2.5.3+SpamaAssiassin-3.2.4: > > > > > > > How I have to do to make my system more reliable? > > > The provided information isn't sufficient. Can you post the > X-Spam-Status for one of the affected emails? Sorry It was not the case to send the entire email.. Here the X-Spam-Status after running the message against 'spamassassin -D': X-Spam-Status: Yes, score=11.2 required=5.0 tests=AWL,BAYES_50,HTML_MESSAGE, RATWARE_MS_HASH,RATWARE_OUTLOOK_NONAME,RDNS_NONE,URIBL_BLACK,URIBL_JP_SU RBL, URIBL_OB_SURBL,URIBL_SC_SURBL autolearn=unavailable version=3.2.4 But it is really strange from amavisd-new log I see that the message is passed as clean: Feb 26 08:09:48 av4 amavis[18267]: (18267-12) Passed CLEAN, [125.128.59.158] [125.128.59.158] <[EMAIL PROTECTED]> -> >,>,>, Message-ID: <[EMAIL PROTECTED]>, mail_id: kgXmlG1zg5ao, Hits: 3.558, size: 3731, queued_as: 9D8E775037D, 2132 ms rocsca
Re: Too false negative
Rocco Scappatura wrote: Hello, Since some days the number of SMTP connections rejected by my server is increased (maybe doubled). It doesn't worry me. But there is a side effect because even the number of false negative is increased. For example, at the moment a spam message with this header is considered clean by Amavisd-new-2.5.3+SpamaAssiassin-3.2.4: How I have to do to make my system more reliable? The provided information isn't sufficient. Can you post the X-Spam-Status for one of the affected emails? To try to figure out why SA is scoring one way or another, we generally need either a whole message, not just headers, or at the very least the output that SpamAssassin generated for it. Or is the point here that no SA analysis was run at all?
Too false negative
Hello, Since some days the number of SMTP connections rejected by my server is increased (maybe doubled). It doesn't worry me. But there is a side effect because even the number of false negative is increased. For example, at the moment a spam message with this header is considered clean by Amavisd-new-2.5.3+SpamaAssiassin-3.2.4: Received: from ([]) by ntfi10.hq.ignesti.it with Microsoft SMTPSVC(6.0.3790.3959); Tue, 26 Feb 2008 08:09:48 +0100 Received: from localhost (localhost [127.0.0.1]) by (Postfix) with ESMTP id 9D8E775037D; Tue, 26 Feb 2008 08:09:48 +0100 (CET) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="_=_NextPart_004_01C87846.932E4D28" Received: from ([127.0.0.1]) by localhost (av4.stt.vir [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kgXmlG1zg5ao; Tue, 26 Feb 2008 08:09:46 +0100 (CET) X-MimeOLE: Produced By Microsoft Exchange V6.5 Received: from [125.128.59.158] (unknown [125.128.59.158]) by (Postfix) with ESMTP id 9CF34750371; Tue, 26 Feb 2008 08:09:45 +0100 (CET) Received: from [125.128.59.158] by dator.plaahn.com; Tue, 26 Feb 2008 16:38:13 +0900 Content-class: urn:content-classes:message Subject: Comprate la forza per il pene, e salvate 85 %. Date: Tue, 26 Feb 2008 08:38:13 +0100 Message-ID: <[EMAIL PROTECTED]> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Comprate la forza per il pene, e salvate 85 %. Thread-Index: Aca6QAN67HSGN9YGB40WPNS14XFFVQ== From: "Wesley Hutchinson" <[EMAIL PROTECTED]> To: "Mosconi Raoul" I use a PRE-LISTING : reject_rbl_client zen.spamhaus.org reject_rbl_client list.dsbl.org And I update SA ruleset regularly with rules_du_jour and sa-update. How I have to do to make my system more reliable? Thanks in advance, rocsca