Re: Smtp auth and trusted_networks

2014-07-11 Thread Nick I
I implemented your patch, but unfortunatelly it did not work for me.
Authenticated sender IP address was recognised as trusted.

I still need to have 'smtpd_sasl_authenticated_header = yes' in my postfix
so i commented out these 3 lines.
And it does work for my installation.

---
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Message/Metadata/Received.pm.orig
2010-03-16 14:49:21.0 +
+++
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Message/Metadata/Received.pm
2014-07-11 17:20:21.497687731 +
@@ -389,9 +389,9 @@
   # with ASMTP (Authenticated SMTP) is used by Earthlink, Exim 4.34, and
others
   # with HTTP should only be authenticated webmail sessions
   # with HTTPU is used by Communigate Pro with Pronto! webmail interface
-  if (/ by /  / with (ESMTPA|ESMTPSA|LMTPA|LMTPSA|ASMTP|HTTPU?)(?:
|$)/i) {
-$auth = $1;
-  }
+#if (/ by /  / with (ESMTPA|ESMTPSA|LMTPA|LMTPSA|ASMTP|HTTPU?)(?:
|$)/i) {
+#  $auth = $1;
+#}
   # Courier v0.47 and possibly others
   elsif (/^from .*?(?:\]\)|\)\]) \(AUTH: (LOGIN|PLAIN|DIGEST-MD5|CRAM-MD5)
\S+(?:, .*?)?\) by /) {
 $auth = $1;


Thanks All for the help.

2014-07-11 17:26 GMT+03:00 Kevin A. McGrail kmcgr...@pccc.com:

 On 7/10/2014 5:55 PM, Giampaolo Tomassoni wrote:

 Il 2014-07-10 17:36 Nick I ha scritto:

  Hi

 In the following example our mx received message with ESMTPSA from
 1.1.1.1 and that ip detected as trusted.
 Our trusted_networks list do not have this ip configured.

 I need to run rbl check against 1.1.1.1.
 Is there any settings to not add authenticated host to trusted hosts ?

 We use SpamAssassin version 3.3.1.


 You case is exactly what the patch in bug#6430 (
 https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6430) attempts to
 cover.

 Unfortunately, that patch never went into any SA version, so you have to
 apply it by yourself if you really need to let your MX act as an MSA in
 case of authenticated submissions.

 If you use amavis, there is another option: move mail submission to
 another instance of your smtp daemon and configure it to submit received
 (and authenticated) message to an amavis channel you prepared for outgoing
 mail.

 Regards,

 Giampaolo

 If you use that patch and it works, please weigh in on the bugzilla or at
 least on the list.  The patch was considered pretty esoteric and didn't
 justify yet another option in the code.  But if people need it and use it,
 we will of course reconsider.



Smtp auth and trusted_networks

2014-07-10 Thread Nick I
Hi

In the following example our mx received message with ESMTPSA from 1.1.1.1
and that ip detected as trusted.
Our trusted_networks list do not have this ip configured.

I need to run rbl check against 1.1.1.1.
Is there any settings to not add authenticated host to trusted hosts ?

We use SpamAssassin version 3.3.1.

Jul 10 14:27:34.275 [9780] dbg: received-header: parsed as [ ip=1.1.1.1
rdns=sender1.domain.com helo=mail.domain.com by=mx.domain.com ident=
envfrom= intl=0 id= auth=ESMTPSA msa=0 ]
Jul 10 14:27:34.275 [9780] dbg: received-header: relay 1.1.1.1 trusted? yes
internal? yes msa? no
Jul 10 14:27:34.277 [9780] dbg: received-header: parsed as [ ip=2.2.2.2
rdns= helo= by=mail.domain.com ident= envfrom= intl=0 id= auth= msa=0 ]
Jul 10 14:27:34.277 [9780] dbg: received-header: relay 2.2.2.2 trusted? no
internal? no msa? no
Jul 10 14:27:34.277 [9780] dbg: metadata: X-Spam-Relays-Trusted: [
ip=1.1.1.1 rdns=sender1.domain.com helo=mail.domain.com by=mx.domain.com
ident= envfrom= intl=1 id= auth=ESMTPSA msa=0 ]
Jul 10 14:27:34.277 [9780] dbg: metadata: X-Spam-Relays-Untrusted: [
ip=2.2.2.2 rdns= helo= by=mail.domain.com ident= envfrom= intl=0 id= auth=
msa=0 ]
Jul 10 14:27:34.277 [9780] dbg: metadata: X-Spam-Relays-Internal: [
ip=1.1.1.1 rdns=sender1.domain.com helo=mail.domain.com by=mx.domain.com
ident= envfrom= intl=1 id= auth=ESMTPSA msa=0 ]
Jul 10 14:27:34.277 [9780] dbg: metadata: X-Spam-Relays-External: [
ip=2.2.2.2 rdns= helo= by=mail.domain.com ident= envfrom= intl=0 id= auth=
msa=0 ]

Thanks.


Sub-test only for first ip

2014-06-05 Thread Nick I
Hi

Can someone let me know which sub-test can i use to check only 1st real ip
addess (68.142.230.77) from this header against rbl dns server?

Received: from server (server
 [1.1.1.1])(using TLSv1 with cipher AES256-SHA (256/256 bits))(No
 client certificate requested)by server2 (MTA) with ESMTPS
for user; Wed,  4 Jun 2014 21:49:22 + (UTC)

Received: from nm20.bullet.mail.bf1.yahoo.com ([98.139.212.179])by
 server ([2.2.2.2])with ESMTPS id 125F946210C00DS4013;
Wed, 04 Jun 2014 17:49:22 -0400

Received: from [98.139.212.151] by nm20.bullet.mail.bf1.yahoo.com with
NNFMP;
 04 Jun 2014 21:49:22 -

Received: from [68.142.230.77] by tm8.bullet.mail.bf1.yahoo.com with NNFMP;
04
 Jun 2014 21:49:22 -

Received: from [127.0.0.1] by smtp234.mail.bf1.yahoo.com with NNFMP; 04 Jun
 2014 21:49:22 -

Thanks.


Re: high cpu load

2014-04-24 Thread Nick I
Finally i found message caused high load.

It looks like one message sent all the time from ticket system.
Message size is ~4M. We scan messages with this size in amavis.

Content of message is complex and has multiple items
Content-Type: image/gif
Content-Transfer-Encoding: base64
Content-Type: application/pdf

Results from debug, with %  1:
 dbg: rules: timing: Total time: 131.6748 s
 dbg: rules: [...] rulename ovl(s) max(s) #run %tot
 dbg: rules: [...] __FILL_THIS_FORM_LONG2 26.3811 26.3811 1 20.04%
 dbg: rules: [...] __FILL_THIS_FORM_SHORT2 26.3050 26.3050 1 19.98%
 dbg: rules: [...] __FILL_THIS_FORM_FRAUD_PHISH1 10.0878 10.0878 1 7.66%
 dbg: rules: [...] __FILL_THIS_FORM_LOAN1 7.2766 7.2766 1 5.53%
 dbg: rules: [...] __FILL_THIS_FORM_SHORT1 2.3360 2.3360 1 1.77%
 dbg: rules: [...] __FILL_THIS_FORM_LONG1 2.3051 2.3051 1 1.75%


 1.8 FUZZY_XPILLBODY: Attempt to obfuscate words in spam
 0.0 FUZZY_CPILLBODY: Attempt to obfuscate words in spam
 0.5 FUZZY_VPILLBODY: Attempt to obfuscate words in spam
 0.8 HTML_IMAGE_RATIO_02BODY: HTML has a low ratio of text to image area
 0.0 HTML_MESSAGE   BODY: HTML included in message
 0.0 LOTS_OF_MONEY  Huge... sums of money

Thanks all for the help!


2014-04-24 1:39 GMT+03:00 John Hardin jhar...@impsec.org:

 On Wed, 23 Apr 2014, Nick I wrote:

  Another interesting thing. Today when daily cron executed at 5 am load
 calmed to ~0. As it was before. Sa-update executed at that time.
 Amavisd has been reloaded at 7 am and load raised back again.
 Also i see that some messages checked 150329 ms, 158742 ms. But most
 messages checked ~400ms.

 I used @debug_recipient_maps and sa_debug but did not see any userful
 info.
 Can anyone suggest how to look inside tests_pri_0 ?


 The first thing you need to do is capture one of the messages that took a
 very long time to scan, so that it can be tested in a controlled
 environment. There are tools that will allow you to capture timing data for
 every rule, and if the message is a spam you could provide it to us for
 analysis.

 --
  John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
  jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
 ---
  Today: Max Planck's 156th birthday



Re: high cpu load

2014-04-23 Thread Nick I
Another interesting thing. Today when daily cron executed at 5 am load
calmed to ~0. As it was before. Sa-update executed at that time.
Amavisd has been reloaded at 7 am and load raised back again.
Also i see that some messages checked 150329 ms, 158742 ms. But most
messages checked ~400ms.

I used @debug_recipient_maps and sa_debug but did not see any userful info.
Can anyone suggest how to look inside tests_pri_0 ?


2014-04-23 14:49 GMT+03:00 RW rwmailli...@googlemail.com:

 On Wed, 23 Apr 2014 02:43:25 +0200
 Karsten Bräckelmann wrote:

  On Tue, 2014-04-22 at 22:31 +0300, Nick I wrote:
  
   Here is timing from amavis. tests_pri_0 is around 90% all the time :
   amavis[26002]: (26002-05) TIMING-SA total 759 ms - parse: 1.92
   (0.3%), extract_message_metadata: 31 (4.1%), get_uri_detail_list: 6
 
 I'm not entirely sure about that amavis log-line, but it most likely
  maps to the SA priorities rules are run at.



 The text after TIMING-SA looks to be SA's own _TIMING_ tag.






high cpu load

2014-04-22 Thread Nick I
Hi,

I use SpamAssassin version 3.3.1 running on Perl version 5.10.1,
amavisd-new-2.8.0-8.el6 as before-queue filter.
Today for unknown reason i noticed high load on my server. Mail flow is as
usual. About 8k in hour checked by amavisd.

Here is timing from amavis. tests_pri_0 is around 90% all the time :
amavis[26002]: (26002-05) TIMING-SA total 759 ms - parse: 1.92 (0.3%),
extract_message_metadata: 31 (4.1%), get_uri_detail_list: 6 (0.8%),
tests_pri_-1000: 17 (2.2%), tests_pri_-950: 0.93 (0.1%), tests_pri_-900:
1.00 (0.1%), tests_pri_-400: 0.79 (0.1%), *tests_pri_0: 668 (88.0%)*,
check_dkim_signature: 9 (1.2%), check_spf: 364 (47.9%), poll_dns_idle: 338
(44.5%), tests_pri_500: 7 (1.0%), get_report: 0.99 (0.1%)

In my another test i see async completed from 0.016 s till 0.174 s.

How can i disable/speed up tests_pri_0 ?
What is inside tests_pri_0?
Thanks.


Re: check_rbl sometimes does not work

2014-02-21 Thread Nick I
I made a test with message size 941105 bytes and rbl does work.
In amavisd
$sa_mail_body_size_limit = 1024*4096;

Also i did not found size limit control when spamassassin called by amavisd.

2014-02-20 18:18 GMT+02:00 Martin Gregorie mar...@gregorie.org:

 On Thu, 2014-02-20 at 14:32 +0200, Nick I wrote:
  The problems we have are:
  1) very rare spamassassin do not make any test at custom check_rbl
  for IP which exist it rbldnsd
 
 Do the unchecked message exceed the maximum message size?
 Default is 512Kb. If the unchecked messages are bigger, raise the limit.


 Martin





check_rbl sometimes does not work

2014-02-20 Thread Nick I
Hello,

We use internal rbldnsd 0.996b, postfix-2.11, amavisd-new-2.8.0-4.el6,
SpamAssassin version 3.3.1running on Perl version 5.10.1, CentOS 6.5.
On the same server used bind9 as cache DNS server

Example of spamassassin rbl check.
header __RCVD_IN_DNSWL eval:check_rbl('rep-domain',
'rep.domain.com.')
describe __RCVD_IN_DNSWL   Sender listed at rep.domain.com
tflags  __RCVD_IN_DNSWLnet nice

header REP_N_1 eval:check_rbl_sub('rep-domain', '127.1.1.0')
describe REP_N_1   Sender listed at rep.domain.com
tflags REP_N_1 nice net
score REP_N_1 -1

The problems we have are:
1) very rare spamassassin do not make any test at custom check_rbl
for IP which exist it rbldnsd
2) check_rbl never executed if user sasl authenticated.

Please suggest a solution.

Thanks.