Re: Multiple test failures

[Sidney Markowitz]

Hi Scott,

Your question is timely. When you posted that, I wasn't aware of 
problems with t/spamd_client.t, but now I have enough examples that it 
is the next failure case I'm tracking down. Can you email me directly 
(don't have to do the detailed back and forth to the entire mailing 
list) with enough configuration information so I could set up an ec2 
instance and install and test SpamAssassin with the same environment you 
have, including any quirks about version of OS, version of perl , 
network and firewall setup, etc? If I can't make it happen for myself, I 
may have some debugging things to add to t/spamd_client.t to produce 
some diagnostics.


Thanks,

 Sidney


Scott Ellentuch wrote on 25/04/24 7:28 am:

Hi,

Any updates on this ?

Tnx, Tuc

On Tue, Apr 9, 2024 at 6:24 PM Scott Ellentuch > wrote:


Hi,

Yes, as ec2-user running the make and then make test ends up
failing. There are no issues with the port as a previous tcpdump has
shown, it transfers data back and forth. It gets through some of the
tests and then it sends a RST. Amazon only goes as far as
spamassassin-3.4.3 in Amazon Linux 2 and they removed it in Amazon
Linux 2023.

Test Summary Report
---
t/spamd_client.t                (Wstat: 26624 Tests: 4 Failed: 0)
   Non-zero exit status: 104
   Parse errors: Bad plan.  You planned 52 tests but ran 4.
Files=217, Tests=3765, 940 wallclock secs ( 1.24 usr  0.22 sys +
280.71 cusr 26.08 csys = 308.25 CPU)
Result: FAIL
Failed 1/217 test programs. 0/3765 subtests failed.
make: *** [test_dynamic] Error 255

Tuc



On Tue, Apr 9, 2024 at 4:03 PM Sidney Markowitz mailto:sid...@sidney.com>> wrote:

Scott Ellentuch wrote on 10/04/24 5:15 am:
 > Apologies, but I don't understand.
 >
 > I am running "make test" as the AWS user "ec2-user" when
getting these
 > errors. Are you saying that its an acceptable error right
now, and I can
 > just do the "sudo make install"?
 >

If you ran "make test" as user "ec2-user", not "sudo make test"
then I
misread this thread and looked at the wrong thing. The bug I
found is an
unexpected problem when running the test as root even when home
directory permissions are relaxed.

Unless what happened was that you ran "sudo make test" and then
tried
"make test" without deleting the files in the t/log directory
that were
created owned by root by the "sudo make test", which would then
cause
failures in the "make test".

To be clear: On a clean system that has everything needed by
SpamAssassin installed, running as user "ec2-user", you should
be able
to run

    perl Makefile.PL < /dev/null
    make
    make test

see no errors in the tests, and then run

    sudo make install

If you are getting errors in spamd tests when running make test as
ec2-user then that might be indicating that something about the
configuration on aws regarding the network and access to ports is
getting in the way.

I don't know if there are any gotchas like that about setting up
on aws,
but if there are, there are probably people on this mailing list
who are
more familiar with any complexities in making a virtual machine
on aws
properly configured to run SpamAssassin.

Re: Multiple test failures

[Scott Ellentuch]

Hi,

Any updates on this ?

Tnx, Tuc

On Tue, Apr 9, 2024 at 6:24 PM Scott Ellentuch  wrote:

> Hi,
>
> Yes, as ec2-user running the make and then make test ends up failing.
> There are no issues with the port as a previous tcpdump has shown, it
> transfers data back and forth. It gets through some of the tests and then
> it sends a RST. Amazon only goes as far as spamassassin-3.4.3 in Amazon
> Linux 2 and they removed it in Amazon Linux 2023.
>
> Test Summary Report
> ---
> t/spamd_client.t(Wstat: 26624 Tests: 4 Failed: 0)
>   Non-zero exit status: 104
>   Parse errors: Bad plan.  You planned 52 tests but ran 4.
> Files=217, Tests=3765, 940 wallclock secs ( 1.24 usr  0.22 sys + 280.71
> cusr 26.08 csys = 308.25 CPU)
> Result: FAIL
> Failed 1/217 test programs. 0/3765 subtests failed.
> make: *** [test_dynamic] Error 255
>
> Tuc
>
>
>
> On Tue, Apr 9, 2024 at 4:03 PM Sidney Markowitz  wrote:
>
>> Scott Ellentuch wrote on 10/04/24 5:15 am:
>> > Apologies, but I don't understand.
>> >
>> > I am running "make test" as the AWS user "ec2-user" when getting these
>> > errors. Are you saying that its an acceptable error right now, and I
>> can
>> > just do the "sudo make install"?
>> >
>>
>> If you ran "make test" as user "ec2-user", not "sudo make test" then I
>> misread this thread and looked at the wrong thing. The bug I found is an
>> unexpected problem when running the test as root even when home
>> directory permissions are relaxed.
>>
>> Unless what happened was that you ran "sudo make test" and then tried
>> "make test" without deleting the files in the t/log directory that were
>> created owned by root by the "sudo make test", which would then cause
>> failures in the "make test".
>>
>> To be clear: On a clean system that has everything needed by
>> SpamAssassin installed, running as user "ec2-user", you should be able
>> to run
>>
>>perl Makefile.PL < /dev/null
>>make
>>make test
>>
>> see no errors in the tests, and then run
>>
>>sudo make install
>>
>> If you are getting errors in spamd tests when running make test as
>> ec2-user then that might be indicating that something about the
>> configuration on aws regarding the network and access to ports is
>> getting in the way.
>>
>> I don't know if there are any gotchas like that about setting up on aws,
>> but if there are, there are probably people on this mailing list who are
>> more familiar with any complexities in making a virtual machine on aws
>> properly configured to run SpamAssassin.
>>
>>

Re: Multiple test failures

[Scott Ellentuch]

Hi,

Yes, as ec2-user running the make and then make test ends up failing. There
are no issues with the port as a previous tcpdump has shown, it transfers
data back and forth. It gets through some of the tests and then it sends a
RST. Amazon only goes as far as spamassassin-3.4.3 in Amazon Linux 2 and
they removed it in Amazon Linux 2023.

Test Summary Report
---
t/spamd_client.t(Wstat: 26624 Tests: 4 Failed: 0)
  Non-zero exit status: 104
  Parse errors: Bad plan.  You planned 52 tests but ran 4.
Files=217, Tests=3765, 940 wallclock secs ( 1.24 usr  0.22 sys + 280.71
cusr 26.08 csys = 308.25 CPU)
Result: FAIL
Failed 1/217 test programs. 0/3765 subtests failed.
make: *** [test_dynamic] Error 255

Tuc



On Tue, Apr 9, 2024 at 4:03 PM Sidney Markowitz  wrote:

> Scott Ellentuch wrote on 10/04/24 5:15 am:
> > Apologies, but I don't understand.
> >
> > I am running "make test" as the AWS user "ec2-user" when getting these
> > errors. Are you saying that its an acceptable error right now, and I can
> > just do the "sudo make install"?
> >
>
> If you ran "make test" as user "ec2-user", not "sudo make test" then I
> misread this thread and looked at the wrong thing. The bug I found is an
> unexpected problem when running the test as root even when home
> directory permissions are relaxed.
>
> Unless what happened was that you ran "sudo make test" and then tried
> "make test" without deleting the files in the t/log directory that were
> created owned by root by the "sudo make test", which would then cause
> failures in the "make test".
>
> To be clear: On a clean system that has everything needed by
> SpamAssassin installed, running as user "ec2-user", you should be able
> to run
>
>perl Makefile.PL < /dev/null
>make
>make test
>
> see no errors in the tests, and then run
>
>sudo make install
>
> If you are getting errors in spamd tests when running make test as
> ec2-user then that might be indicating that something about the
> configuration on aws regarding the network and access to ports is
> getting in the way.
>
> I don't know if there are any gotchas like that about setting up on aws,
> but if there are, there are probably people on this mailing list who are
> more familiar with any complexities in making a virtual machine on aws
> properly configured to run SpamAssassin.
>
>

Re: Multiple test failures

[Sidney Markowitz]

Scott Ellentuch wrote on 10/04/24 5:15 am:

Apologies, but I don't understand.

I am running "make test" as the AWS user "ec2-user" when getting these 
errors. Are you saying that its an acceptable error right now, and I can 
just do the "sudo make install"?




If you ran "make test" as user "ec2-user", not "sudo make test" then I 
misread this thread and looked at the wrong thing. The bug I found is an 
unexpected problem when running the test as root even when home 
directory permissions are relaxed.


Unless what happened was that you ran "sudo make test" and then tried 
"make test" without deleting the files in the t/log directory that were 
created owned by root by the "sudo make test", which would then cause 
failures in the "make test".


To be clear: On a clean system that has everything needed by 
SpamAssassin installed, running as user "ec2-user", you should be able 
to run


  perl Makefile.PL < /dev/null
  make
  make test

see no errors in the tests, and then run

  sudo make install

If you are getting errors in spamd tests when running make test as 
ec2-user then that might be indicating that something about the 
configuration on aws regarding the network and access to ports is 
getting in the way.


I don't know if there are any gotchas like that about setting up on aws, 
but if there are, there are probably people on this mailing list who are 
more familiar with any complexities in making a virtual machine on aws 
properly configured to run SpamAssassin.

Re: Multiple test failures

[Scott Ellentuch]

Apologies, but I don't understand.

I am running "make test" as the AWS user "ec2-user" when getting these
errors. Are you saying that its an acceptable error right now, and I can
just do the "sudo make install"?

Thanks, Tuc

On Fri, Apr 5, 2024 at 9:58 PM Sidney Markowitz  wrote:

> Scott Ellentuch wrote on 4/04/24 9:43 am:
> > File attached. However, I don't see any smoking gun.
>
> I've verified the problem. I ran sudo make test in a directory tree in
> /tmp with world r-x access, and got the error in t/spamd_client.t as
> well as in t/spamc_optL.t. I don't know why you didn't see the second of
> those, but that doesn't matter for fixing it.
>
> It looks like the fix to bug 5529 was only a partial solution, made even
> worse by the way that it put nothing in place to ensure that newly
> created tests would not have the same problem as the ones it "fixed".
>
> I'll open a new bug for this and finish the job bug 5529 started.
>
> However, it remains a good idea to not run the tests as root.
> Even if you need root to install SpamAssassin you can do it either by using
>
>perl Makefile.PL < /dev/null
>make
>make test
>sudo make install
>
> or if you have been using sudo cpan to do the installation, instead use
> cpanm with the -S option. That option allows you to run cpanm as
> non-root and it does the sudo make install for you.
>
> If you do that it will work before I fix the bug.
>
>   Sidney
>
>

Re: Multiple test failures

[Sidney Markowitz]

Scott Ellentuch wrote on 4/04/24 9:43 am:

File attached. However, I don't see any smoking gun.


I've verified the problem. I ran sudo make test in a directory tree in 
/tmp with world r-x access, and got the error in t/spamd_client.t as 
well as in t/spamc_optL.t. I don't know why you didn't see the second of 
those, but that doesn't matter for fixing it.


It looks like the fix to bug 5529 was only a partial solution, made even 
worse by the way that it put nothing in place to ensure that newly 
created tests would not have the same problem as the ones it "fixed".


I'll open a new bug for this and finish the job bug 5529 started.

However, it remains a good idea to not run the tests as root.
Even if you need root to install SpamAssassin you can do it either by using

  perl Makefile.PL < /dev/null
  make
  make test
  sudo make install

or if you have been using sudo cpan to do the installation, instead use 
cpanm with the -S option. That option allows you to run cpanm as 
non-root and it does the sudo make install for you.


If you do that it will work before I fix the bug.

 Sidney

Re: Multiple test failures

[Sidney Markowitz]

Loren Wilton wrote on 4/04/24 9:26 pm:
Would it be worth adding some sort of test for this kind of thing 


I started to look at where in SATest.pm it ought to go, and I found
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=5529

I think from the comments I left there 17 years ago that is is already 
supposed to be handled. I'll look at the emails in this thread more 
carefully to see if perhaps we re-introduced the problem in some change 
we made since then.


 Sidney

Re: Multiple test failures

[Loren Wilton]

I haven't had a chance yet to read this thread carefully, but spamd when 
run as root in tests will, at least in some cases, set itself to run as 
user "nobody". If you do that in a subdirectory of your non-nobody user's 
HOME, the usual permission configuration will not provide read access to 
nobody and the test will fail.


Would it be worth adding some sort of test for this kind of thing that could 
make a reasonably explicit "don't run install as root" or "incorrect 
directory permissions" or some such, to make it more obvious what is going 
wrong? I seem to remember at least one or two very similar postings here in 
the last year or so, so it is something that does happen and confuses 
people.

Re: Multiple test failures

[Sidney Markowitz]

I haven't had a chance yet to read this thread carefully, but spamd when 
run as root in tests will, at least in some cases, set itself to run as 
user "nobody". If you do that in a subdirectory of your non-nobody 
user's HOME, the usual permission configuration will not provide read 
access to nobody and the test will fail.


Basically, the only tests that should be run as root are the t/root*.t 
tests, and even those have comments in their source telling you about 
running under a directory with world read permissions, and those tests 
are not run by default, being used as part of our release testing on 
machines that do not run SpamAssassin in production.


 Sidney


Bill Cole wrote on 4/04/24 3:46 am:

On 2024-04-02 at 18:18:09 UTC-0400 (Tue, 2 Apr 2024 18:18:09 -0400)
Scott Ellentuch 
is rumored to have said:


Hi,

Trying to install SA 4.0.1 from scratch. Tried via CPAN, that didn't
go
well, so trying from tarball. (Enabled SSL when doing Makefile.PL)


NEVER run 'make' as root except when you're ready to commit with 'make
install' unless you're doing it on a sacrificial system.

Re: Multiple test failures

[Scott Ellentuch]

File attached. However, I don't see any smoking gun.

Nothing in t/log
[ec2-user@ip-172-31-131-251 Mail-SpamAssassin-4.0.1]$ ls -lR t/log
t/log:
total 0

Nothing big in iptables
[ec2-user@ip-172-31-131-251 Mail-SpamAssassin-4.0.1]$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source   destination

Chain FORWARD (policy ACCEPT)
target prot opt source   destination
ACCEPT all  --  ip-10-8-0-0.ec2.internal/24  anywhere
ACCEPT all  --  anywhere anywhere state
RELATED,ESTABLISHED

Chain OUTPUT (policy ACCEPT)
target prot opt source   destination

Tnx, Tuc

On Wed, Apr 3, 2024 at 3:31 PM Bill Cole <
sausers-20150...@billmail.scconsult.com> wrote:

> On 2024-04-03 at 14:01:44 UTC-0400 (Wed, 3 Apr 2024 14:01:44 -0400)
> Scott Ellentuch 
> is rumored to have said:
>
> > Hi,
> >
> > Ok, deleted the directory and started again.
> >
> > Test Summary Report
> > ---
> > t/spamd_client.t(Wstat: 26624 Tests: 4 Failed: 0)
> >   Non-zero exit status: 104
> >   Parse errors: Bad plan.  You planned 52 tests but ran 4.
> > Files=217, Tests=3765, 890 wallclock secs ( 1.21 usr  0.19 sys +
> > 271.62
> > cusr 25.51 csys = 298.53 CPU)
> > Result: FAIL
> > Failed 1/217 test programs. 0/3765 subtests failed.
> > make: *** [test_dynamic] Error 255
> >
> > Script file attached.
>
> This error appears to be a problem launching a spamd instance from the
> test harness, verifying its PID, and getting responses from it. You can
> get more details logged by clearing the test logs and re-running just
> the one test file which displays the problem:
>
>rm -r t/log
>make test TEST_FILES="t/spamd_client.t"  TEST_VERBOSE=1
>
> That should provide the precise command used to launch spamd and
> hopefully a clue about why it failed. There may also be useful
> information logged under t/log/ after a failed test.
>
> One possibility is a local packet filter (iptables, nftables, etc.)
> blocking the port spamd uses for testing. That is rare because it
> selects an unused high  port on the loopback interface for the test run,
> but if you have a very tight network security policy in place, that can
> fail. SELinux and AppArmor can also interfere.
>
>
> >
> > Thanks Tuc
> >
> > On Wed, Apr 3, 2024 at 10:46 AM Bill Cole <
> > sausers-20150...@billmail.scconsult.com> wrote:
> >
> >> On 2024-04-02 at 18:18:09 UTC-0400 (Tue, 2 Apr 2024 18:18:09 -0400)
> >> Scott Ellentuch 
> >> is rumored to have said:
> >>
> >>> Hi,
> >>>
> >>> Trying to install SA 4.0.1 from scratch. Tried via CPAN, that didn't
> >>> go
> >>> well, so trying from tarball. (Enabled SSL when doing Makefile.PL)
> >>
> >> NEVER run 'make' as root except when you're ready to commit with
> >> 'make
> >> install' unless you're doing it on a sacrificial system.
> >>
> >> Think about how unsafe it could be...
> >>
> >> These test failures look like you did that. I am flattered that you
> >> trust the SpamAssassin team that much, but don't, please. We are only
> >> human. In the past there have been bugs in the test suite that have
> >> polluted the running config of the system if run as root. It is
> >> possible
> >> in principle for there to still be such bugs.
> >>
> >>> I'm on Amazon Linux 2 , 4.0.1 SA, and not sure what other info I can
> >>> give.
> >>> I installed every perl module it wanted.
> >>
> >> FWIW, non-root 'make test' has been clean for PMC members on a wide
> >> range of systems, so a real test failure would be both a shock and a
> >> serious problem. If this is happening with a normal user running
> >> 'make
> >> test' we definitely need to
> >> address it.
> >>
> >>>
> >>> The final summary is -
> >>> Test Summary Report
> >>> ---
> >>> t/spamc_optL.t  (Wstat: 2560 Tests: 18 Failed: 10)
> >>>   Failed tests:  2, 5-8, 10, 12, 15-16, 18
> >>>   Non-zero exit status: 10
> >>> t/spamd_client.t(Wstat: 3584 Tests: 52 Failed: 14)
> >>>   Failed tests:  35, 37-42, 44, 46-51
> >>>   Non-zero exit status: 14
> >>> Files=217, Tests=3807, 904 wallclock secs ( 1.21 usr  0.22 sys +
> >>> 273.72
> >>> cusr 26.33 csys = 301.48 CPU)
> >>> Result: FAIL
> >>> Failed 2/217 test programs. 24/3807 subtests failed.
> >>> make: *** [test_dynamic] Error 255
> >>>
> >>> During the run it seems to output :
> >>>
> >>> t/spamd_client.t .. 32/52
> >>> #   Failed test at t/spamd_client.t line 152.
> >>> ERROR: Bayes dump returned an error, please re-run with -D for more
> >>> information
> >>> t/spamd_client.t .. 37/52
> >>> #   Failed test at t/spamd_client.t line 157.
> >>> Not found: spam in database = 1 0  non-token data: nspam at
> >>> t/spamd_client.t line 158.
> >>>
> >>> #   Failed test at t/SATest.pm line 926.
> >>>
> >>> #   Failed test at t/spamd_client.t line 161.
> >>> ERROR: Bayes dump returned an error, please re-run with -D for more
> >>> information
> >>>
> >>> #   Failed test 

Re: Multiple test failures

[Bill Cole]

On 2024-04-03 at 14:01:44 UTC-0400 (Wed, 3 Apr 2024 14:01:44 -0400)
Scott Ellentuch 
is rumored to have said:


Hi,

Ok, deleted the directory and started again.

Test Summary Report
---
t/spamd_client.t(Wstat: 26624 Tests: 4 Failed: 0)
  Non-zero exit status: 104
  Parse errors: Bad plan.  You planned 52 tests but ran 4.
Files=217, Tests=3765, 890 wallclock secs ( 1.21 usr  0.19 sys + 
271.62

cusr 25.51 csys = 298.53 CPU)
Result: FAIL
Failed 1/217 test programs. 0/3765 subtests failed.
make: *** [test_dynamic] Error 255

Script file attached.


This error appears to be a problem launching a spamd instance from the 
test harness, verifying its PID, and getting responses from it. You can 
get more details logged by clearing the test logs and re-running just 
the one test file which displays the problem:


  rm -r t/log
  make test TEST_FILES="t/spamd_client.t"  TEST_VERBOSE=1

That should provide the precise command used to launch spamd and 
hopefully a clue about why it failed. There may also be useful 
information logged under t/log/ after a failed test.


One possibility is a local packet filter (iptables, nftables, etc.) 
blocking the port spamd uses for testing. That is rare because it 
selects an unused high  port on the loopback interface for the test run, 
but if you have a very tight network security policy in place, that can 
fail. SELinux and AppArmor can also interfere.





Thanks Tuc

On Wed, Apr 3, 2024 at 10:46 AM Bill Cole <
sausers-20150...@billmail.scconsult.com> wrote:


On 2024-04-02 at 18:18:09 UTC-0400 (Tue, 2 Apr 2024 18:18:09 -0400)
Scott Ellentuch 
is rumored to have said:


Hi,

Trying to install SA 4.0.1 from scratch. Tried via CPAN, that didn't
go
well, so trying from tarball. (Enabled SSL when doing Makefile.PL)


NEVER run 'make' as root except when you're ready to commit with 
'make

install' unless you're doing it on a sacrificial system.

Think about how unsafe it could be...

These test failures look like you did that. I am flattered that you
trust the SpamAssassin team that much, but don't, please. We are only
human. In the past there have been bugs in the test suite that have
polluted the running config of the system if run as root. It is 
possible

in principle for there to still be such bugs.


I'm on Amazon Linux 2 , 4.0.1 SA, and not sure what other info I can
give.
I installed every perl module it wanted.


FWIW, non-root 'make test' has been clean for PMC members on a wide
range of systems, so a real test failure would be both a shock and a
serious problem. If this is happening with a normal user running 
'make

test' we definitely need to
address it.



The final summary is -
Test Summary Report
---
t/spamc_optL.t  (Wstat: 2560 Tests: 18 Failed: 10)
  Failed tests:  2, 5-8, 10, 12, 15-16, 18
  Non-zero exit status: 10
t/spamd_client.t(Wstat: 3584 Tests: 52 Failed: 14)
  Failed tests:  35, 37-42, 44, 46-51
  Non-zero exit status: 14
Files=217, Tests=3807, 904 wallclock secs ( 1.21 usr  0.22 sys +
273.72
cusr 26.33 csys = 301.48 CPU)
Result: FAIL
Failed 2/217 test programs. 24/3807 subtests failed.
make: *** [test_dynamic] Error 255

During the run it seems to output :

t/spamd_client.t .. 32/52
#   Failed test at t/spamd_client.t line 152.
ERROR: Bayes dump returned an error, please re-run with -D for more
information
t/spamd_client.t .. 37/52
#   Failed test at t/spamd_client.t line 157.
Not found: spam in database = 1 0  non-token data: nspam at
t/spamd_client.t line 158.

#   Failed test at t/SATest.pm line 926.

#   Failed test at t/spamd_client.t line 161.
ERROR: Bayes dump returned an error, please re-run with -D for more
information

#   Failed test at t/spamd_client.t line 165.
Not found: ham in database = 0 0  non-token data: nham at
t/spamd_client.t
line 166.

#   Failed test at t/SATest.pm line 926.
Not found: spam in database = 0 0  non-token data: nspam at
t/spamd_client.t line 166.

#   Failed test at t/SATest.pm line 926.
t/spamd_client.t .. 44/52
#   Failed test at t/spamd_client.t line 172.
ERROR: Bayes dump returned an error, please re-run with -D for more
information

#   Failed test at t/spamd_client.t line 177.
Not found: ham in database = 1 0  non-token data: nham at
t/spamd_client.t
line 178.

#   Failed test at t/SATest.pm line 926.

#   Failed test at t/spamd_client.t line 181.
ERROR: Bayes dump returned an error, please re-run with -D for more
information
t/spamd_client.t .. 49/52
#   Failed test at t/spamd_client.t line 185.
Not found: ham in database = 0 0  non-token data: nham at
t/spamd_client.t
line 186.

#   Failed test at t/SATest.pm line 926.
Not found: spam in database = 0 0  non-token data: nspam at
t/spamd_client.t line 186.

#   Failed test at t/SATest.pm line 926.
t/spamd_client.t .. 52/52 # Looks like you failed 14
tests
of 52.
t/spamd_client.t .. 

Re: Multiple test failures

[Scott Ellentuch]

Hi,

Ok, deleted the directory and started again.

Test Summary Report
---
t/spamd_client.t(Wstat: 26624 Tests: 4 Failed: 0)
  Non-zero exit status: 104
  Parse errors: Bad plan.  You planned 52 tests but ran 4.
Files=217, Tests=3765, 890 wallclock secs ( 1.21 usr  0.19 sys + 271.62
cusr 25.51 csys = 298.53 CPU)
Result: FAIL
Failed 1/217 test programs. 0/3765 subtests failed.
make: *** [test_dynamic] Error 255

Script file attached.

Thanks Tuc

On Wed, Apr 3, 2024 at 10:46 AM Bill Cole <
sausers-20150...@billmail.scconsult.com> wrote:

> On 2024-04-02 at 18:18:09 UTC-0400 (Tue, 2 Apr 2024 18:18:09 -0400)
> Scott Ellentuch 
> is rumored to have said:
>
> > Hi,
> >
> > Trying to install SA 4.0.1 from scratch. Tried via CPAN, that didn't
> > go
> > well, so trying from tarball. (Enabled SSL when doing Makefile.PL)
>
> NEVER run 'make' as root except when you're ready to commit with 'make
> install' unless you're doing it on a sacrificial system.
>
> Think about how unsafe it could be...
>
> These test failures look like you did that. I am flattered that you
> trust the SpamAssassin team that much, but don't, please. We are only
> human. In the past there have been bugs in the test suite that have
> polluted the running config of the system if run as root. It is possible
> in principle for there to still be such bugs.
>
> > I'm on Amazon Linux 2 , 4.0.1 SA, and not sure what other info I can
> > give.
> > I installed every perl module it wanted.
>
> FWIW, non-root 'make test' has been clean for PMC members on a wide
> range of systems, so a real test failure would be both a shock and a
> serious problem. If this is happening with a normal user running 'make
> test' we definitely need to
> address it.
>
> >
> > The final summary is -
> > Test Summary Report
> > ---
> > t/spamc_optL.t  (Wstat: 2560 Tests: 18 Failed: 10)
> >   Failed tests:  2, 5-8, 10, 12, 15-16, 18
> >   Non-zero exit status: 10
> > t/spamd_client.t(Wstat: 3584 Tests: 52 Failed: 14)
> >   Failed tests:  35, 37-42, 44, 46-51
> >   Non-zero exit status: 14
> > Files=217, Tests=3807, 904 wallclock secs ( 1.21 usr  0.22 sys +
> > 273.72
> > cusr 26.33 csys = 301.48 CPU)
> > Result: FAIL
> > Failed 2/217 test programs. 24/3807 subtests failed.
> > make: *** [test_dynamic] Error 255
> >
> > During the run it seems to output :
> >
> > t/spamd_client.t .. 32/52
> > #   Failed test at t/spamd_client.t line 152.
> > ERROR: Bayes dump returned an error, please re-run with -D for more
> > information
> > t/spamd_client.t .. 37/52
> > #   Failed test at t/spamd_client.t line 157.
> > Not found: spam in database = 1 0  non-token data: nspam at
> > t/spamd_client.t line 158.
> >
> > #   Failed test at t/SATest.pm line 926.
> >
> > #   Failed test at t/spamd_client.t line 161.
> > ERROR: Bayes dump returned an error, please re-run with -D for more
> > information
> >
> > #   Failed test at t/spamd_client.t line 165.
> > Not found: ham in database = 0 0  non-token data: nham at
> > t/spamd_client.t
> > line 166.
> >
> > #   Failed test at t/SATest.pm line 926.
> > Not found: spam in database = 0 0  non-token data: nspam at
> > t/spamd_client.t line 166.
> >
> > #   Failed test at t/SATest.pm line 926.
> > t/spamd_client.t .. 44/52
> > #   Failed test at t/spamd_client.t line 172.
> > ERROR: Bayes dump returned an error, please re-run with -D for more
> > information
> >
> > #   Failed test at t/spamd_client.t line 177.
> > Not found: ham in database = 1 0  non-token data: nham at
> > t/spamd_client.t
> > line 178.
> >
> > #   Failed test at t/SATest.pm line 926.
> >
> > #   Failed test at t/spamd_client.t line 181.
> > ERROR: Bayes dump returned an error, please re-run with -D for more
> > information
> > t/spamd_client.t .. 49/52
> > #   Failed test at t/spamd_client.t line 185.
> > Not found: ham in database = 0 0  non-token data: nham at
> > t/spamd_client.t
> > line 186.
> >
> > #   Failed test at t/SATest.pm line 926.
> > Not found: spam in database = 0 0  non-token data: nspam at
> > t/spamd_client.t line 186.
> >
> > #   Failed test at t/SATest.pm line 926.
> > t/spamd_client.t .. 52/52 # Looks like you failed 14
> > tests
> > of 52.
> > t/spamd_client.t .. Dubious, test returned 14 (wstat
> > 3584,
> > 0xe00)
> > Failed 14/52 subtests
> >
> > Any indications as to the issue?
> >
> > Thanks, Tuc
>
>
> --
> Bill Cole
> b...@scconsult.com or billc...@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Not Currently Available For Hire
>
Script started on Wed 03 Apr 2024 05:40:56 PM UTC
]0;ec2-user@ip-172-31-131-251:/usr/local/src/Mail-SpamAssassin-4.0.1[?1034h[ec2-user@ip-172-31-131-251
 Mail-SpamAssassin-4.0.1]$ make test
"/usr/bin/perl" build/mkrules --exit_on_no_src --src rulesrc --out rules 
--manifest MANIFEST --manifestskip MANIFEST.SKIP
no source 

Re: Multiple test failures

[Bill Cole]

On 2024-04-02 at 18:18:09 UTC-0400 (Tue, 2 Apr 2024 18:18:09 -0400)
Scott Ellentuch 
is rumored to have said:


Hi,

Trying to install SA 4.0.1 from scratch. Tried via CPAN, that didn't 
go

well, so trying from tarball. (Enabled SSL when doing Makefile.PL)


NEVER run 'make' as root except when you're ready to commit with 'make 
install' unless you're doing it on a sacrificial system.


Think about how unsafe it could be...

These test failures look like you did that. I am flattered that you 
trust the SpamAssassin team that much, but don't, please. We are only 
human. In the past there have been bugs in the test suite that have 
polluted the running config of the system if run as root. It is possible 
in principle for there to still be such bugs.


I'm on Amazon Linux 2 , 4.0.1 SA, and not sure what other info I can 
give.

I installed every perl module it wanted.


FWIW, non-root 'make test' has been clean for PMC members on a wide 
range of systems, so a real test failure would be both a shock and a 
serious problem. If this is happening with a normal user running 'make 
test' we definitely need to

address it.



The final summary is -
Test Summary Report
---
t/spamc_optL.t  (Wstat: 2560 Tests: 18 Failed: 10)
  Failed tests:  2, 5-8, 10, 12, 15-16, 18
  Non-zero exit status: 10
t/spamd_client.t(Wstat: 3584 Tests: 52 Failed: 14)
  Failed tests:  35, 37-42, 44, 46-51
  Non-zero exit status: 14
Files=217, Tests=3807, 904 wallclock secs ( 1.21 usr  0.22 sys + 
273.72

cusr 26.33 csys = 301.48 CPU)
Result: FAIL
Failed 2/217 test programs. 24/3807 subtests failed.
make: *** [test_dynamic] Error 255

During the run it seems to output :

t/spamd_client.t .. 32/52
#   Failed test at t/spamd_client.t line 152.
ERROR: Bayes dump returned an error, please re-run with -D for more
information
t/spamd_client.t .. 37/52
#   Failed test at t/spamd_client.t line 157.
Not found: spam in database = 1 0  non-token data: nspam at
t/spamd_client.t line 158.

#   Failed test at t/SATest.pm line 926.

#   Failed test at t/spamd_client.t line 161.
ERROR: Bayes dump returned an error, please re-run with -D for more
information

#   Failed test at t/spamd_client.t line 165.
Not found: ham in database = 0 0  non-token data: nham at 
t/spamd_client.t

line 166.

#   Failed test at t/SATest.pm line 926.
Not found: spam in database = 0 0  non-token data: nspam at
t/spamd_client.t line 166.

#   Failed test at t/SATest.pm line 926.
t/spamd_client.t .. 44/52
#   Failed test at t/spamd_client.t line 172.
ERROR: Bayes dump returned an error, please re-run with -D for more
information

#   Failed test at t/spamd_client.t line 177.
Not found: ham in database = 1 0  non-token data: nham at 
t/spamd_client.t

line 178.

#   Failed test at t/SATest.pm line 926.

#   Failed test at t/spamd_client.t line 181.
ERROR: Bayes dump returned an error, please re-run with -D for more
information
t/spamd_client.t .. 49/52
#   Failed test at t/spamd_client.t line 185.
Not found: ham in database = 0 0  non-token data: nham at 
t/spamd_client.t

line 186.

#   Failed test at t/SATest.pm line 926.
Not found: spam in database = 0 0  non-token data: nspam at
t/spamd_client.t line 186.

#   Failed test at t/SATest.pm line 926.
t/spamd_client.t .. 52/52 # Looks like you failed 14 
tests

of 52.
t/spamd_client.t .. Dubious, test returned 14 (wstat 
3584,

0xe00)
Failed 14/52 subtests

Any indications as to the issue?

Thanks, Tuc



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Multiple test failures

[Scott Ellentuch]

Hi,

Trying to install SA 4.0.1 from scratch. Tried via CPAN, that didn't go
well, so trying from tarball. (Enabled SSL when doing Makefile.PL)

I'm on Amazon Linux 2 , 4.0.1 SA, and not sure what other info I can give.
I installed every perl module it wanted.

The final summary is -
Test Summary Report
---
t/spamc_optL.t  (Wstat: 2560 Tests: 18 Failed: 10)
  Failed tests:  2, 5-8, 10, 12, 15-16, 18
  Non-zero exit status: 10
t/spamd_client.t(Wstat: 3584 Tests: 52 Failed: 14)
  Failed tests:  35, 37-42, 44, 46-51
  Non-zero exit status: 14
Files=217, Tests=3807, 904 wallclock secs ( 1.21 usr  0.22 sys + 273.72
cusr 26.33 csys = 301.48 CPU)
Result: FAIL
Failed 2/217 test programs. 24/3807 subtests failed.
make: *** [test_dynamic] Error 255

During the run it seems to output :

t/spamd_client.t .. 32/52
#   Failed test at t/spamd_client.t line 152.
ERROR: Bayes dump returned an error, please re-run with -D for more
information
t/spamd_client.t .. 37/52
#   Failed test at t/spamd_client.t line 157.
Not found: spam in database = 1 0  non-token data: nspam at
t/spamd_client.t line 158.

#   Failed test at t/SATest.pm line 926.

#   Failed test at t/spamd_client.t line 161.
ERROR: Bayes dump returned an error, please re-run with -D for more
information

#   Failed test at t/spamd_client.t line 165.
Not found: ham in database = 0 0  non-token data: nham at t/spamd_client.t
line 166.

#   Failed test at t/SATest.pm line 926.
Not found: spam in database = 0 0  non-token data: nspam at
t/spamd_client.t line 166.

#   Failed test at t/SATest.pm line 926.
t/spamd_client.t .. 44/52
#   Failed test at t/spamd_client.t line 172.
ERROR: Bayes dump returned an error, please re-run with -D for more
information

#   Failed test at t/spamd_client.t line 177.
Not found: ham in database = 1 0  non-token data: nham at t/spamd_client.t
line 178.

#   Failed test at t/SATest.pm line 926.

#   Failed test at t/spamd_client.t line 181.
ERROR: Bayes dump returned an error, please re-run with -D for more
information
t/spamd_client.t .. 49/52
#   Failed test at t/spamd_client.t line 185.
Not found: ham in database = 0 0  non-token data: nham at t/spamd_client.t
line 186.

#   Failed test at t/SATest.pm line 926.
Not found: spam in database = 0 0  non-token data: nspam at
t/spamd_client.t line 186.

#   Failed test at t/SATest.pm line 926.
t/spamd_client.t .. 52/52 # Looks like you failed 14 tests
of 52.
t/spamd_client.t .. Dubious, test returned 14 (wstat 3584,
0xe00)
Failed 14/52 subtests

Any indications as to the issue?

Thanks, Tuc