Re: Multiple test failures

[Sidney Markowitz]

Hi Scott,

Your question is timely. When you posted that, I wasn't aware of 
problems with t/spamd_client.t, but now I have enough examples that it 
is the next failure case I'm tracking down. Can you email me directly 
(don't have to do the detailed back and forth to the entire mailing 
list) with enough configuration information so I could set up an ec2 
instance and install and test SpamAssassin with the same environment you 
have, including any quirks about version of OS, version of perl , 
network and firewall setup, etc? If I can't make it happen for myself, I 
may have some debugging things to add to t/spamd_client.t to produce 
some diagnostics.


Thanks,

 Sidney


Scott Ellentuch wrote on 25/04/24 7:28 am:

Hi,

Any updates on this ?

Tnx, Tuc

On Tue, Apr 9, 2024 at 6:24 PM Scott Ellentuch > wrote:


Hi,

Yes, as ec2-user running the make and then make test ends up
failing. There are no issues with the port as a previous tcpdump has
shown, it transfers data back and forth. It gets through some of the
tests and then it sends a RST. Amazon only goes as far as
spamassassin-3.4.3 in Amazon Linux 2 and they removed it in Amazon
Linux 2023.

Test Summary Report
---
t/spamd_client.t                (Wstat: 26624 Tests: 4 Failed: 0)
   Non-zero exit status: 104
   Parse errors: Bad plan.  You planned 52 tests but ran 4.
Files=217, Tests=3765, 940 wallclock secs ( 1.24 usr  0.22 sys +
280.71 cusr 26.08 csys = 308.25 CPU)
Result: FAIL
Failed 1/217 test programs. 0/3765 subtests failed.
make: *** [test_dynamic] Error 255

Tuc



On Tue, Apr 9, 2024 at 4:03 PM Sidney Markowitz mailto:sid...@sidney.com>> wrote:

Scott Ellentuch wrote on 10/04/24 5:15 am:
 > Apologies, but I don't understand.
 >
 > I am running "make test" as the AWS user "ec2-user" when
getting these
 > errors. Are you saying that its an acceptable error right
now, and I can
 > just do the "sudo make install"?
 >

If you ran "make test" as user "ec2-user", not "sudo make test"
then I
misread this thread and looked at the wrong thing. The bug I
found is an
unexpected problem when running the test as root even when home
directory permissions are relaxed.

Unless what happened was that you ran "sudo make test" and then
tried
"make test" without deleting the files in the t/log directory
that were
created owned by root by the "sudo make test", which would then
cause
failures in the "make test".

To be clear: On a clean system that has everything needed by
SpamAssassin installed, running as user "ec2-user", you should
be able
to run

    perl Makefile.PL < /dev/null
    make
    make test

see no errors in the tests, and then run

    sudo make install

If you are getting errors in spamd tests when running make test as
ec2-user then that might be indicating that something about the
configuration on aws regarding the network and access to ports is
getting in the way.

I don't know if there are any gotchas like that about setting up
on aws,
but if there are, there are probably people on this mailing list
who are
more familiar with any complexities in making a virtual machine
on aws
properly configured to run SpamAssassin.

Re: Multiple test failures

[Scott Ellentuch]

Hi,

Any updates on this ?

Tnx, Tuc

On Tue, Apr 9, 2024 at 6:24 PM Scott Ellentuch  wrote:

> Hi,
>
> Yes, as ec2-user running the make and then make test ends up failing.
> There are no issues with the port as a previous tcpdump has shown, it
> transfers data back and forth. It gets through some of the tests and then
> it sends a RST. Amazon only goes as far as spamassassin-3.4.3 in Amazon
> Linux 2 and they removed it in Amazon Linux 2023.
>
> Test Summary Report
> ---
> t/spamd_client.t(Wstat: 26624 Tests: 4 Failed: 0)
>   Non-zero exit status: 104
>   Parse errors: Bad plan.  You planned 52 tests but ran 4.
> Files=217, Tests=3765, 940 wallclock secs ( 1.24 usr  0.22 sys + 280.71
> cusr 26.08 csys = 308.25 CPU)
> Result: FAIL
> Failed 1/217 test programs. 0/3765 subtests failed.
> make: *** [test_dynamic] Error 255
>
> Tuc
>
>
>
> On Tue, Apr 9, 2024 at 4:03 PM Sidney Markowitz  wrote:
>
>> Scott Ellentuch wrote on 10/04/24 5:15 am:
>> > Apologies, but I don't understand.
>> >
>> > I am running "make test" as the AWS user "ec2-user" when getting these
>> > errors. Are you saying that its an acceptable error right now, and I
>> can
>> > just do the "sudo make install"?
>> >
>>
>> If you ran "make test" as user "ec2-user", not "sudo make test" then I
>> misread this thread and looked at the wrong thing. The bug I found is an
>> unexpected problem when running the test as root even when home
>> directory permissions are relaxed.
>>
>> Unless what happened was that you ran "sudo make test" and then tried
>> "make test" without deleting the files in the t/log directory that were
>> created owned by root by the "sudo make test", which would then cause
>> failures in the "make test".
>>
>> To be clear: On a clean system that has everything needed by
>> SpamAssassin installed, running as user "ec2-user", you should be able
>> to run
>>
>>perl Makefile.PL < /dev/null
>>make
>>make test
>>
>> see no errors in the tests, and then run
>>
>>sudo make install
>>
>> If you are getting errors in spamd tests when running make test as
>> ec2-user then that might be indicating that something about the
>> configuration on aws regarding the network and access to ports is
>> getting in the way.
>>
>> I don't know if there are any gotchas like that about setting up on aws,
>> but if there are, there are probably people on this mailing list who are
>> more familiar with any complexities in making a virtual machine on aws
>> properly configured to run SpamAssassin.
>>
>>

Re: Tips for improving bounce message deliverability?

[Benny Pedersen]

Bill Cole skrev den 2024-04-24 19:37:

On 2024-04-24 at 12:27:01 UTC-0400 (Wed, 24 Apr 2024 18:27:01 +0200)
Benny Pedersen 
is rumored to have said:


For example, it matches on
*  3.1 URI_IMG_CWINDOWSNET Non-MSFT image hosted by Microsoft Azure
infra, possible phishing


this is not in spamassassin core rules


Yes, it is:

updates_spamassassin_org # grep -n '[^A-Z]* URI_IMG_CWINDOWSNET' *
72_active.cf:5635:##{ URI_IMG_CWINDOWSNET
	72_active.cf:5637:meta   URI_IMG_CWINDOWSNET 
__URI_IMG_CWINDOWSNET && !__RCD_RDNS_SMTP && !__REPTO_QUOTE && 
!__URI_DOTEDU
	72_active.cf:5638:#score  URI_IMG_CWINDOWSNET 3.500	# 
limit
	72_active.cf:5639:describe   URI_IMG_CWINDOWSNET Non-MSFT 
image hosted by Microsoft Azure infra, possible phishing

72_active.cf:5640:tflags URI_IMG_CWINDOWSNET publish


it is publish, so waste of config in public :)


72_active.cf:5641:##} URI_IMG_CWINDOWSNET
	72_scores.cf:408:score URI_IMG_CWINDOWSNET   3.136 
3.060 3.136 3.060


It is being drawn in from John Hardin's sandbox, where he committed the 
rule on 2024-01-21 in r1915356


i know this, so if its stable rule in core rules i can turn of to get it 
from trunk



 *  2.6 HOSTED_IMG_DIRECT_MX Image hosted at large ecomm, CDN or
hosting
 *  site, message direct-to-mx


also not in default rule sets


Also NOT TRUE. That one is in the same sandbox source and was last 
tweaked in r1915433 on 2024-01-28


i do not always check changelogs on core rulesets, sorry for that

It also matches on ANY_BOUNCE_MESSAGE and BOUNCE_MESSAGE. Should 
metas

be created to avoid adding the above scores?

What more can be done to improve deliverability of these messages?
Perhaps this is something postfix can identify and bypass scanning?


it matches bounces since its a bounce, alt that is seen as a results 
of forwarding emails


More helpfully, it is possible to exempt bounces from filtering by 
SpamAssassin, a trick that is accomplished by whatever mechanism you 
use to 'glue' SA and your MTA (postfix, I assume...) not by SA itself. 
In the case of postfix, there are about a half-dozen mechanisms one can 
use so I can't say for sure. However, in general, if you are using a 
milter interface you must do the discrimination in the milter, while 
other glue mechanisms can provide selective filtering in postfix (at 
the cost of doing it within postfix.)


yes, vbounce should handle this to only make noice on localy bounces, 
and since spamassassin does not reject we all have to see external 
bounces aswell


A message which matches BOUNCE_MESSAGE (and hence also 
ANY_BOUNCE_MESSAGE) is fairly unlikely to be spam, but we have pegged 
the scores for all the *BOUNCE_MESSAGE rules at 0.1 just to make sure 
that they are always published and visible as control points that can 
be used by sites that have a particular need to accept (or shun) some 
or all bounces.


i have disabled this plugin, if that matter

Re: authres missing spf-helo ?

[Benny Pedersen]

Matus UHLAR - fantomas skrev den 2024-04-24 18:58:

On 24.04.24 18:50, Benny Pedersen wrote:

unsure so i ask :)


try to explain your question a bit more


perldoc Mail::SpamAssassin::Plugin::AuthRes

EVAL FUNCTIONS
header RULENAME eval:check_authres_result(method, result)
Can be used to check results.

  ifplugin Mail::SpamAssassin::Plugin::AuthRes
  ifplugin !(Mail::SpamAssassin::Plugin::SPF)
header  SPF_PASS  eval:check_authres_result('spf', 
'pass')
header  SPF_FAIL  eval:check_authres_result('spf', 
'fail')
header  SPF_SOFTFAIL  eval:check_authres_result('spf', 
'softfail')
header  SPF_TEMPFAIL  eval:check_authres_result('spf', 
'tempfail')

  endif
  ifplugin !(Mail::SpamAssassin::Plugin::DKIM)
header  DKIM_VERIFIED  eval:check_authres_result('dkim', 
'pass')
header  DKIM_INVALID   eval:check_authres_result('dkim', 
'fail')

  endif
  endif


imho this example is not correct, authres have the results from another 
header


grep eval:

=item header RULENAME eval:check_authres_result(method, result)
header  SPF_PASS  eval:check_authres_result('spf', 'pass')
header  SPF_FAIL  eval:check_authres_result('spf', 'fail')
header  SPF_SOFTFAIL  eval:check_authres_result('spf', 'softfail')
header  SPF_TEMPFAIL  eval:check_authres_result('spf', 'tempfail')
header  DKIM_VERIFIED  eval:check_authres_result('dkim', 'pass')
header  DKIM_INVALID   eval:check_authres_result('dkim', 'fail')


i asked why is SPF_HELO evals missing ?

can authres test for dkim none ?

if its just missing examples, all well

Re: Tips for improving bounce message deliverability?

[Bill Cole]

On 2024-04-24 at 12:27:01 UTC-0400 (Wed, 24 Apr 2024 18:27:01 +0200)
Benny Pedersen 
is rumored to have said:

>> For example, it matches on
>> *  3.1 URI_IMG_CWINDOWSNET Non-MSFT image hosted by Microsoft Azure
>> infra, possible phishing
>
> this is not in spamassassin core rules

Yes, it is:

updates_spamassassin_org # grep -n '[^A-Z]* URI_IMG_CWINDOWSNET' *
72_active.cf:5635:##{ URI_IMG_CWINDOWSNET
72_active.cf:5637:meta   URI_IMG_CWINDOWSNET 
__URI_IMG_CWINDOWSNET && !__RCD_RDNS_SMTP && !__REPTO_QUOTE && !__URI_DOTEDU
72_active.cf:5638:#score  URI_IMG_CWINDOWSNET 3.500 # limit
72_active.cf:5639:describe   URI_IMG_CWINDOWSNET Non-MSFT image 
hosted by Microsoft Azure infra, possible phishing
72_active.cf:5640:tflags URI_IMG_CWINDOWSNET publish
72_active.cf:5641:##} URI_IMG_CWINDOWSNET
72_scores.cf:408:score URI_IMG_CWINDOWSNET   3.136 
3.060 3.136 3.060

It is being drawn in from John Hardin's sandbox, where he committed the rule on 
2024-01-21 in r1915356

>>  *  2.6 HOSTED_IMG_DIRECT_MX Image hosted at large ecomm, CDN or
>> hosting
>>  *  site, message direct-to-mx
>
> also not in default rule sets

Also NOT TRUE. That one is in the same sandbox source and was last tweaked in 
r1915433 on 2024-01-28

>> It also matches on ANY_BOUNCE_MESSAGE and BOUNCE_MESSAGE. Should metas
>> be created to avoid adding the above scores?
>>
>> What more can be done to improve deliverability of these messages?
>> Perhaps this is something postfix can identify and bypass scanning?
>
> it matches bounces since its a bounce, alt that is seen as a results of 
> forwarding emails

More helpfully, it is possible to exempt bounces from filtering by 
SpamAssassin, a trick that is accomplished by whatever mechanism you use to 
'glue' SA and your MTA (postfix, I assume...) not by SA itself. In the case of 
postfix, there are about a half-dozen mechanisms one can use so I can't say for 
sure. However, in general, if you are using a milter interface you must do the 
discrimination in the milter, while other glue mechanisms can provide selective 
filtering in postfix (at the cost of doing it within postfix.)

A message which matches BOUNCE_MESSAGE (and hence also ANY_BOUNCE_MESSAGE) is 
fairly unlikely to be spam, but we have pegged the scores for all the 
*BOUNCE_MESSAGE rules at 0.1 just to make sure that they are always published 
and visible as control points that can be used by sites that have a particular 
need to accept (or shun) some or all bounces.

-- 
Bill Cole

Re: Tips for improving bounce message deliverability?

[Matus UHLAR - fantomas]

Alex skrev den 2024-04-24 15:45:

I'm using SA 4.0.1 and amavisd with postfix. I've identified a few
bounce messages in the quarantine because they weren't identified
properly. Here's one:
https://pastebin.com/RMNkcyhF


1.3 RDNS_NONE  Delivered to internal network by a host 
with no rDNS



This is apparently related to this:


Received: from gambit.example.com ([130.250.178.199])
by localhost (iceman.example.com [127.0.0.1]) (amavis, port 10024)
with ESMTP id D5Mo318nYFrZ; Wed, 24 Apr 2024 08:17:07 -0400 (EDT)



Alex:
Is gambit.example.com ([130.250.178.199]) your server?

If so, it should be in trusted_networks and internal_networks

Also, why don't you resolve DNS?
That IP has valid fcrdns name gambit.guardiandigital.com.


For example, it matches on
*  3.1 URI_IMG_CWINDOWSNET Non-MSFT image hosted by Microsoft Azure
infra, possible phishing


On 24.04.24 18:27, Benny Pedersen wrote:

this is not in spamassassin core rules


I _can_ see this in 4.0 rules


*  2.6 HOSTED_IMG_DIRECT_MX Image hosted at large ecomm, CDN or
hosting
*  site, message direct-to-mx


also not in default rule sets


also this one.
Perhaps Benny uses older SA?



It also matches on ANY_BOUNCE_MESSAGE and BOUNCE_MESSAGE. Should metas
be created to avoid adding the above scores?

What more can be done to improve deliverability of these messages?
Perhaps this is something postfix can identify and bypass scanning?


BOUNCE_MESSAGE requires setting up welcomelist_bounce_relays, which defines
servers who send your e-mail - thus you know bounces from those hosts are
legitimate.  the original message opriginated from mailgun, perhaps you need
to add its servers.

it matches bounces since its a bounce, alt that is seen as a results 
of forwarding emails


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Linux IS user friendly, it's just selective who its friends are...

Re: authres missing spf-helo ?

[Matus UHLAR - fantomas]

On 24.04.24 18:50, Benny Pedersen wrote:

unsure so i ask :)


try to explain your question a bit more

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
   One OS to rule them all, One OS to find them,
One OS to bring them all and into darkness bind them

authres missing spf-helo ?

[Benny Pedersen]

unsure so i ask :)

Re: Tips for improving bounce message deliverability?

[Benny Pedersen]

Alex skrev den 2024-04-24 15:45:

Hi,
I'm using SA 4.0.1 and amavisd with postfix. I've identified a few
bounce messages in the quarantine because they weren't identified
properly. Here's one:
https://pastebin.com/RMNkcyhF


Content preview: Delivery has failed to these recipients or groups: 
CURTIS
  RICCIARDI (cuberi...@msn.com) The 
recipient's mailbox
   is full and can't accept messages now. Please try resending your 
message

  later, or contact the recipient directly [...]

Content Domains: banno.com jshorefcu.org mailgun.net office365.com 
outlook.com windows.net


Content analysis details:   (11.9 points, 5.0 required)

 pts rule name  description
 -- 
--

 2.3 SPF_HELO_NONE  SPF: HELO does not publish an SPF Record
 0.0 ARC_VALID  Message has a valid ARC signature
 0.0 ARC_SIGNED Message has a ARC signature
 0.0 KAM_DMARC_STATUS   Test Rule for DKIM or SPF Failure with 
Strict

Alignment
 0.5 AUTHRES_ARC_NONE   Authentication-Results: has "arc=none" 
result
 0.5 AUTHRES_DKIM_NONE  Authentication-Results: has "dkim=none" 
result
 0.5 AUTHRES_DMARC_NONE Authentication-Results: has "dmarc=none" 
result

 2.0 URL_GREYLIST   Other untrustworthy TLDs
 [URI: bannoinstitutionassets.blob.core.windows.net 
(windows.net)]
 1.5 AUTHRES_SPF_NONE   Authentication-Results: has "spf=none" 
result

 0.0 HTML_MESSAGE   BODY: HTML included in message
 3.1 URI_IMG_CWINDOWSNETNon-MSFT image hosted by Microsoft Azure 
infra,

possible phishing
 1.3 RDNS_NONE  Delivered to internal network by a host with 
no rDNS

 0.2 KAM_LOTSOFHASH Emails with lots of hash-like gibberish
 0.0 T_KAM_HTML_FONT_INVALID Test for Invalidly Named or Formatted 
Colors

in HTML


For example, it matches on
*  3.1 URI_IMG_CWINDOWSNET Non-MSFT image hosted by Microsoft Azure
infra, possible phishing


this is not in spamassassin core rules


 *  2.6 HOSTED_IMG_DIRECT_MX Image hosted at large ecomm, CDN or
hosting
 *  site, message direct-to-mx


also not in default rule sets


It also matches on ANY_BOUNCE_MESSAGE and BOUNCE_MESSAGE. Should metas
be created to avoid adding the above scores?

What more can be done to improve deliverability of these messages?
Perhaps this is something postfix can identify and bypass scanning?


it matches bounces since its a bounce, alt that is seen as a results of 
forwarding emails

Tips for improving bounce message deliverability?

[Alex]

Hi,
I'm using SA 4.0.1 and amavisd with postfix. I've identified a few bounce
messages in the quarantine because they weren't identified properly. Here's
one:
https://pastebin.com/RMNkcyhF

For example, it matches on
*  3.1 URI_IMG_CWINDOWSNET Non-MSFT image hosted by Microsoft Azure
infra, possible phishing
 *  2.6 HOSTED_IMG_DIRECT_MX Image hosted at large ecomm, CDN or hosting
 *  site, message direct-to-mx

It also matches on ANY_BOUNCE_MESSAGE and BOUNCE_MESSAGE. Should metas be
created to avoid adding the above scores?

What more can be done to improve deliverability of these messages? Perhaps
this is something postfix can identify and bypass scanning?