On Wed, 27 May 2015, Arthur Schwarz wrote:
TAP is supposed to be a custom test driver. Is the interface and manner of
calling different from other custom drivers, and the API and other comments
describing custom drivers in the manual?
I think that the method of calling is similar to other type of tests
except that the list of tests 'TESTS' is a list of scripts which
produce TAP output strings.
It depends on subordinate
scripts printing
I'm very confused here. It looks to me like tap-driver.sh is a standalone
script and doesn't need any help scripts. The input data is processed in awk
and all the needed functions are defined in this context.
If the test program was a C program so it was a binary and printed
'ok' and 'not ok' outputs, and is able to find any input files without
assistance, then no extra shim script should be required.
In my case I was replacing perhaps 1000 individual classic Automake
test scripts and wanted to replace them with far fewer TAP scripts.
I am confused. Using Automake the Developer can generate a reference to a
class of test cases defined in the TESTS variable. Each one of the scripts
is required to output the results of one of more tests that they run in
quasi-TAP format. The TAP script, tap-driver.sh, takes the output and
generates a trs file. Included with the tap-driver.sh there is a means to
generate XFAIL and XPASS, however this seems to be global to all subtests
executed in a TESTS defined test. Each ok or not ok returned will be
translated (as required) to XPASS or XFAIL depending on
--expect-failure=yes.
A TAP test program/script may contain many tests and it may output
multiple test lines. The script indicates how many tests it plans to
run by first printing a line like
1..532
to indicate that it plans to run 532 tests. Each test gets is own
PASS, FAIL, XFAIL indication.
As an example:
TEST_EXTENSIONS =.abc
ABC_LOG_DRIVER = tap-driver.sh
TESTS = test1.abc test2.abc
Means that tap-driver.sh is called twice, once for test1.abc, once for
test2.abc. Each of these tests can return one or more of ok, not ok, skip,
bail, or a YAML-like text string defined by --diagnostic-string and
--comments.
Yes, but tap-driver.sh interprets the output while the test
program/scripts runs and it can handle an arbitary number of
tests/results in the same test script.
Anyhow, thanks for the heads up, I will be looking at your files. And, as
always, I am confused.
I am not surprised. :-)
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
