There are failures in the build log but they are mostly expected failures.
They are designed to test deliberately failing cases.
The design of the regression test system is that you create an input file,
capture the output, and append it after the command with a prefix.
So
1+1
in an input file generates
(1) 2
Type: PositiveInteger
Now you construct an input file that has some special comments
around this test case (see any file in src/input/*.input.pamphlet):
--S 1 of 32
1+1
--R (1) 2
--R Type: PositiveInteger
--E 1
Since anything with "--" is an axiom comment almost all of the lines
are ignored when the file is read. If you read the file and spool the
output you will see:
--S 1 of 32 <-- the testcase number
1+1 <-- the axiom input
<-- the actual output
(1) 2 <--
Type: PositiveInteger <--
--R <-- the expected output
--R (1) 2 <--
--R Type: PositiveInteger <--
--E 1 <-- end of this testcase
We see the input (1+1) and the Axiom output. We also have comments that show
the expected Axiom output. There is a function called "regress" which
will take this spool
output file (say, foo.output") and compare the actual Axiom output from
the expected Axiom
output.
)lisp (regress "foo.output")
It lists the successful and the failing test cases (look in
int/input/*.regress)
At the very end of the build Axiom scans the regress files looking for
any that fail.
Some can fail due to the fact that the algorithm is random. Others fail
for various
reasons. I check these failure cases after every Axiom build.
Any failures either get fixed in the next update or get added to the
known bugs list.
So, to answer your question, there are cases where the Axiom input fails
intentionally
and the failing output is captured. This way it is possible to test
known bad inputs.
The regression test system only cares that the answers are the same, it
does not care
that the Axiom input is a known, intentional failure.
Thus, the regression test system can test both successful and failing
input but will
only complain if some output did not match the previous results.
Tim
Camm Maguire wrote:
Congratulations, Tim!
Separate question -- I notice there are many regression failures in
the build log. Do you have any mechanism to identify "expected"
failures? Or is there any other automated way to ensure the
correctness of the build? For example, with maxima, the autobuilder
is instructed to abort if there are any "unexpected" test suite
failures.
Take care,
Tim Daly <d...@axiom-developer.org> writes:
The Axiom Version January 2010 has been released to
the source code servers:
from github: git clone git://github.com/daly/axiom.git
from savannah: git clone ax...@git.sv.nongnu.org:/srv/git/axiom.git
from sourceforge: |git clone
git://axiom.git.sourceforge.net/gitroot/axiom/axiom|
(commit: 7aa4ca083d79f63bc46b0c21af41e9d79191f390)
Binaries will become available as time permits at:
http://axiom-developer.org/axiom-website/download.html
Tim
_______________________________________________
Axiom-developer mailing list
Axiom-developer@nongnu.org
http://lists.nongnu.org/mailman/listinfo/axiom-developer
_______________________________________________
Axiom-developer mailing list
Axiom-developer@nongnu.org
http://lists.nongnu.org/mailman/listinfo/axiom-developer