Module Name: src
Committed By: jmmv
Date: Sat Jun 26 11:15:28 UTC 2010
Modified Files:
src/share/man/man7: Makefile
Added Files:
src/share/man/man7: tests.7
Log Message:
Add the tests(7) manual page, which describes why and how to run the
test suite and how to configure it.
To generate a diff of this commit:
cvs rdiff -u -r1.21 -r1.22 src/share/man/man7/Makefile
cvs rdiff -u -r0 -r1.1 src/share/man/man7/tests.7
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/share/man/man7/Makefile
diff -u src/share/man/man7/Makefile:1.21 src/share/man/man7/Makefile:1.22
--- src/share/man/man7/Makefile:1.21 Fri Mar 2 11:28:16 2007
+++ src/share/man/man7/Makefile Sat Jun 26 11:15:27 2010
@@ -1,10 +1,11 @@
-# $NetBSD: Makefile,v 1.21 2007/03/02 11:28:16 wiz Exp $
+# $NetBSD: Makefile,v 1.22 2010/06/26 11:15:27 jmmv Exp $
# @(#)Makefile 8.1 (Berkeley) 6/5/93
# missing: eqnchar.7 man.7 ms.7 term.7
MAN= ascii.7 environ.7 hier.7 hostname.7 intro.7 mailaddr.7 \
nls.7 operator.7 pkgsrc.7 release.7 \
- script.7 setuid.7 signal.7 sticky.7 symlink.7 sysctl.7
+ script.7 setuid.7 signal.7 sticky.7 symlink.7 sysctl.7 \
+ tests.7
.include <bsd.man.mk>
Added files:
Index: src/share/man/man7/tests.7
diff -u /dev/null src/share/man/man7/tests.7:1.1
--- /dev/null Sat Jun 26 11:15:28 2010
+++ src/share/man/man7/tests.7 Sat Jun 26 11:15:27 2010
@@ -0,0 +1,172 @@
+.\" $NetBSD: tests.7,v 1.1 2010/06/26 11:15:27 jmmv Exp $
+.\"
+.\" Copyright (c) 2010 The NetBSD Foundation, Inc.
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions
+.\" are met:
+.\" 1. Redistributions of source code must retain the above copyright
+.\" notice, this list of conditions and the following disclaimer.
+.\" 2. Redistributions in binary form must reproduce the above copyright
+.\" notice, this list of conditions and the following disclaimer in the
+.\" documentation and/or other materials provided with the distribution.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
+.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
+.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
+.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.\"
+.Dd June 26, 2010
+.Dt TESTS 7
+.Os
+.Sh NAME
+.Nm tests
+.Nd An introduction to the NetBSD test suite
+.Sh DESCRIPTION
+The
+.Nx
+test suite provides a collection of automated tests for two major purposes.
+On the one hand, the test suite aids
+.Em developers
+in catching bugs and regressions in the code when they performing modifications
+to the source tree.
+On the other hand, the test suite allows
+.Em end users
+(and, in particular, system administrators) to verify that fresh installations
+of the
+.Nx
+operating system behave correctly in their hardware platform and also to ensure
+that the system does not suffer from regressions during regular system
+operation and maintenance.
+.Pp
+The
+.Nx
+tests are implemented using the
+.Em Automated Testing Framework (ATF) ,
+a third-party package shipped with
+.Nx ;
+see
+.Xr atf 7
+for details.
+The
+.Nx
+test suite is distributed as a separate installation set, named
+.Pa tests.tgz ,
+and the test programs are all installed under the
+.Pa /usr/tests
+hieararchy.
+.Pp
+This manual page describes how to execute the test suite and how to configure
+some of its optional features.
+.Ss When to run the tests?
+Before diving into the details of how to run the test suite, here are some
+scenarios in which you should be running them:
+.Bl -bullet -offset indent
+.It
+After a fresh installation of
+.Nx
+to ensure that the system works correctly on your hardware platform.
+.It
+After an upgrade of
+.Nx
+to a different version to ensure that the new code works well on your
+hardware platform and that the upgrade did not introduce regressions in your
+configuration.
+.It
+After performing changes to the source tree to catch any bugs and/or regressions
+introduced by the modifications.
+.It
+Periodically, maybe from a
+.Xr cron 8
+job, to ensure that any changes to the system (such as the installation of
+third-party packages or manual modifications to configuration files) do not
+introduce unexpected failures.
+.El
+.Ss Running the tests
+Use the following commands to run the whole test suite:
+.Bd -literal -offset indent
+$ cd /usr/tests
+$ atf-run | atf-report
+.Ed
+.Pp
+The above will go through all test programs in
+.Pa /usr/tests
+recursive, execute them, and, at the very end, show a report of the results of
+the test suite.
+These results include the count of tests that succeeded (passed), the names of
+the tests that failed, and the count of the tests that were not executed
+(skipped) because the system configuration did not meet their requirements.
+.Pp
+If you are interested in saving the whole output of the test suite execution so
+that you can later investigate failures, use the following idiom instead:
+.Bd -literal -offset indent
+$ cd /usr/tests
+$ atf-run | tee ~/tests.log | atf-report
+.Ed
+.Pp
+The above command will save the raw output of the test suite in
+.Pa ~/tests.log ,
+which you can later inspect manually to look for failures.
+Note that the file contains a copy of the
+.Sq stdout
+and
+.Sq stderr
+of each test case, which becomes valuable during debugging.
+.Pp
+It is also possible to restrict which tests to execute so that only a small
+subsystem is tested; see
+.Xr atf-run 1
+for details.
+Additionally, it is also possible to run the test programs themselves by hand;
+see
+.Xr atf-test-program 1
+for more details, but be aware that you should only be doing this if you are
+debugging failing tests.
+.Ss Configuring the tests
+Some test cases in the
+.Nx
+test suite require the administrator to manually set up some configuration
+properties before they can run.
+Unless these properties are defined, the tests that require them will be marked
+as skipped and thus they will not be really executed.
+.Pp
+Each test suite is configured through a separate file that lives under
+.Pa /etc/atf/
+and that carries the name of the test suite.
+Henceforth, to configure the properties that affect the execution of the
+.Nx
+test suite, you need to edit
+.Pa /etc/atf/NetBSD.conf .
+These files conform to the configuration file format described in
+.Xr atf-formats 5 .
+.Pp
+The following properties are recognized:
+.Bl -tag -offset indent -width unprivilegedXuserXX
+.It Va unprivileged_user
+Specifies the name of a local, unprivileged user, that will be used by those
+tests that need to perform checks as non-root.
+.El
+.Sh SEE ALSO
+.Xr atf 7
+.Sh HISTORY
+The
+.Xr tests 7
+manual page first appeared in
+.Nx 6.0 .
+.Pp
+The ATF testing framework was first distributed with
+.Nx 5.0
+and the collection of test programs in
+.Pa /usr/tests
+has been growing since then.
+.Sh AUTHORS
+.An Julio Merino Aq j...@netbsd.org
