So...
sqlite-3.3.5
Would you expect ./testfixture ./test/quick.test to pass with "0 errors out
of 24922 tests"
But ./testfixture ./test/all.test to bail abruptly with:
btree2-2.5... Ok
btree2-2.6...
Error: invalid command name "btree_cursor_info"
btree2-2.7... Ok
btree2-2.8.1.0... Ok
/opt/alarm/src/sqlite-3.3.5/.libs/lt-testfixture: invalid command name
"btree_cu
rsor_info"
while executing
"btree_cursor_info [set ::c$i]"
("foreach" body line 15)
invoked from within
"foreach {n I K D} {
0.5 0.5 0.1 0.1
1.0 0.2 0.1 0.1
1.0 0.8 0.1 0.1
2.0 0.0 0.1 0.1
2.0 1.0 0.1 0.1
2.0 0.0 0.0 0.0
2.0 1"
("foreach" body line 60)
invoked from within
"foreach {N L} {
10 2
50 2
200 3
2000 5
} {
puts " N=$N L=$L "
set hash [md5file test2.bt]
do_test btree2-$testno.1 [subst -nocom..."
invoked from within
"if {[info commands btree_open]!=""} {
# Create a new database file containing no entries. The database should
# contain 5 tables:
#
# 2 The de..."
(file "./test/btree2.test" line 20)
invoked from within
"source $testfile"
("foreach" body line 5)
invoked from within
"foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
set tail [file tail $testfile]
if {[lsearch -exact $EXCLUDE $tail]>=0} continue
..."
("for" body line 7)
invoked from within
"for {set Counter 0} {$Counter<$COUNT && $nErr==0} {incr Counter} {
if {$Counter%2} {
set ::SETUP_SQL {PRAGMA default_synchronous=off;}
} else ..."
(file "./test/all.test" line 76)
On the following platform??
Linux 2.4.31 #4 SMP Wed Apr 19 16:30:29 CDT 2006 i686 unknown unknown
GNU/Linux
Christian Werner
Unicel Traffic Engineering
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: Friday, June 23, 2006 1:08 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] Need Regression Testing Insight
"Frank Gordon" <[EMAIL PROTECTED]> wrote:
> I'm a newbie. I am also a test engineer interested in the amount and
types
> of testing that are performed on SQLite. Especially on releases of new
> versions.
> Can anyone shed any light on this topic?
>
> I have spent a considerable amount of days looking for SQLite regression
> tests that are available, how often are they run, and where to find the
> results. So far I feel I have exhausted my search and came up way to
short
> on information. I have only found regression tcl scripts at the following
> url: http://www.sqlite.org/cvstrac/dir?d=sqlite/test However, no
> documentation summarizing this testing and no posted results from
> SQLite.org. My
> understanding is that over 200,000 test cases are being run by SQLite.org,
> according to check-in [3144] there is 98% test coverage, and that the
> community of developers utilizing SQLite in there projects are also
running
> these regression tests on there individual platforms.
>
Testing of SQLite is done with a program called "testfixture".
Testfixture contains an instrumented version of SQLite plus
a TCL interpreter for running the tests. You can build
testfixture by typing "make testfixture" on unix. It is
more complicated on windows. Once you have your testfixture,
you run the tests by typing:
./testfixture ../sqlite/test/quick.test
or
./testfixture ../sqlite/test/all.test
The "all.test" script runs in excess of 95% of source lines,
measured using gcov. The object code coverage is lower but
is still relatively high. The code that is not run is
usually unreachable.
Special compile-time options are needed to get high
code coverage.
SQLite does a lot of very speciallized tests. For example,
there is a customized backend to sqlite (similar to os_unix.c
and os_win.c) that simulates a computer that can suffer a
random power failure in the middle of a transaction. This
is used to verify that the recovery and rollback logic is
all working correctly. This customized backend is only
available when you compile with the right flags.
Over 60% of the non-comment source code in SQLite is
devoted purely to testing.
I generally run quick.test on Linux before each check-in
and all.test on both Linux and windows and quick.test
on MacOSX before each release.
--
D. Richard Hipp <[EMAIL PROTECTED]>
