Re: [sqlite] Why is there no test suite for ''make check' ?
On 25 Jun 2010, at 7:06pm, Dr. David Kirkby wrote: > On 06/24/10 05:13 PM, Richard Hipp wrote: > >> I see. You are building from the amalgamation. And you should be too. But >> you are right - we do not ship test suites with the amalgamation. To use >> the free test suite for SQLite, you have to build from canonical source >> code. We have a separate test harness for the amalgamation (the one that >> provides 100% branch test coverage) but that one is not free, I'm afraid. > > Do you (or anyone else for that matter), not believe it is reasonable to > include > a test suite with any distribution? You'll find a number of FREE distributions of SQLite source code files here: http://www.sqlite.org/download.html The full distribution is the one called sqlite-3_6_23-tea.tar.gz (1.14 MiB) And, as you can see, it includes a test suite. The purpose of the test suite is /not/ to test SQLite itself because the authors have done that. It's to test your hardware, OS, compiler and linker to make sure that your own compilation of SQLite is sane on your particular platform. The only people who run the test suite in anger are those who are compiling for non-standard hardware (embedded platforms) who are rare, and those who think they've spotted a bug in SQLite who are even rarer. Many people who run SQLite are not really concerned about the inner workings of SQLite. They are dealing with a big and complicated set of source files themselves and they don't want one tiny piece of their program -- a database engine -- to look big and complicated in their development environment. For those people, SQLite makes the amalgamation package available: one file of C code and one header file, which is as simple as C gets. That's the one you were told to use for your programming: sqlite-amalgamation-3_6_23_1.zip (1.07 MiB) If you have no reason to be suspicious of SQLite because you use the same hardware and compiler as everyone else, the amalgamation files are the only ones you need, and that's the way almost everyone does it. So you have the test suite right there, for free, if you need it. The reason you were initially told about the amalgamation version is that almost nobody does. > I'm not talking about something that uses weeks of CPU time, gives 100% > branch > coverage and requires commerical licences to use. Instead, I am suggesting a > more limited set of tests that try to find problems within a reasonable time > frame - perhaps 15-30 on a modern computer, without requiring any commercial > software. That's the way SQLite splits it: the really big and complicated test suite is not a freebie (and is not mentioned on the above page). However, there's a smaller simpler and faster set of tests, still quite thorough, which you can get free as described above. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
On Fri, Jun 25, 2010 at 2:06 PM, Dr. David Kirkby wrote: > On 06/24/10 05:13 PM, Richard Hipp wrote: > > > I see. You are building from the amalgamation. And you should be too. > But > > you are right - we do not ship test suites with the amalgamation. To use > > the free test suite for SQLite, you have to build from canonical source > > code. We have a separate test harness for the amalgamation (the one that > > provides 100% branch test coverage) but that one is not free, I'm afraid. > > Do you (or anyone else for that matter), not believe it is reasonable to > include > a test suite with any distribution? > > I'm not talking about something that uses weeks of CPU time, gives 100% > branch > coverage and requires commerical licences to use. Instead, I am suggesting > a > more limited set of tests that try to find problems within a reasonable > time > frame - perhaps 15-30 on a modern computer, without requiring any > commercial > software. > Creating and supporting yet another test suite takes time. We don't have any spare time right now. > > Dave > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- - D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
On 06/24/10 05:13 PM, Richard Hipp wrote: > I see. You are building from the amalgamation. And you should be too. But > you are right - we do not ship test suites with the amalgamation. To use > the free test suite for SQLite, you have to build from canonical source > code. We have a separate test harness for the amalgamation (the one that > provides 100% branch test coverage) but that one is not free, I'm afraid. Do you (or anyone else for that matter), not believe it is reasonable to include a test suite with any distribution? I'm not talking about something that uses weeks of CPU time, gives 100% branch coverage and requires commerical licences to use. Instead, I am suggesting a more limited set of tests that try to find problems within a reasonable time frame - perhaps 15-30 on a modern computer, without requiring any commercial software. Dave ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
Does the TEA tarball have the tests? -- __ 3.14159265358979323846264338327950 Let the spirit of pi 2884197169399375105820974944592307 spread all over the world! 8164062862089986280348253421170679 http://pi314.at PI VOBISCUM! == ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/24/2010 12:38 PM, Darren Duncan wrote: > The test suite being AFAIK > essentially an application that links against the already-compiled SQLite, Not quite the case. The test suite includes large amounts of code to induce various faults, track memory, reset various things back to pre-initialized values etc. This requires recompiling the SQLite library to get those various hooks and tweaks. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkwjv+MACgkQmOOfHg372QQm6QCfWjN2mmIjAibKGtKTLVH3sine 5HUAn0iVK3aZmPwrgg8XQQ+YVATuWSSF =rVZK -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
Replying to an earlier post, I remember "make test" working for SQLite when I tried it in the past. I'm actually used to running that, this practice ingrained as standard behaviour with Perl and Perl modules, and I usually expect any other projects to have a "make test" too. Besides Perl, I know that Git, Python, PHP, and SQLite, "make test" works for. Richard Hipp wrote: > I see. You are building from the amalgamation. And you should be too. But > you are right - we do not ship test suites with the amalgamation. To use > the free test suite for SQLite, you have to build from canonical source > code. We have a separate test harness for the amalgamation (the one that > provides 100% branch test coverage) but that one is not free, I'm afraid. Why is that? I would have thought that with respect to users of SQLite the amalgamation is drop-in-substitutable for the separate files, with the test suite mainly testing the collection as a whole. The test suite being AFAIK essentially an application that links against the already-compiled SQLite, at which point differences may have gone away. What subtleties does the test suite depend on that the same one doesn't work for both versions. -- Darren Duncan ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
On 06/24/10 05:13 PM, Richard Hipp wrote: > On Thu, Jun 24, 2010 at 12:03 PM, Dr. David Kirkby > wrote: > >> On 06/24/10 04:20 PM, Black, Michael (IS) wrote: >>> Try one directory up. It's not in the src directory Makefile. >> >> >> There is only one directory - there is no 'src' directory at all. If you >> look at >> the path what I posted below, you do see 'src' but that is part of the >> directory >> structure in Sage. In other words >> >> >> ~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src >> >> IS the top level directory in sqlite. (That's assuming some idiot has not >> screwed something up in Sage). I guess I'll need to double check this, but >> it >> seems unlikely. The 'configure' script creates a Makefile, but that >> Makefile has >> no test/check/fulltest targets. >> >> Dave >> >> > I see. You are building from the amalgamation. And you should be too. But > you are right - we do not ship test suites with the amalgamation. Is there a a good reason for that? To say I'm building from the right source code, but then to not have the test packages seems a bit odd. One might reasonably expect some way to test the software no matter what source code one uses. > To use > the free test suite for SQLite, you have to build from canonical source > code. But by implication of what you said earlier, I should be building from the amalgamation source code. > We have a separate test harness for the amalgamation (the one that > provides 100% branch test coverage) but that one is not free, I'm afraid. I'm not looking for 100% branch coverage. I would like something that I can use to be reasonably confident the code is working correctly. Dave ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
On Thu, Jun 24, 2010 at 12:03 PM, Dr. David Kirkby wrote: > On 06/24/10 04:20 PM, Black, Michael (IS) wrote: > > Try one directory up. It's not in the src directory Makefile. > > > There is only one directory - there is no 'src' directory at all. If you > look at > the path what I posted below, you do see 'src' but that is part of the > directory > structure in Sage. In other words > > > ~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src > > IS the top level directory in sqlite. (That's assuming some idiot has not > screwed something up in Sage). I guess I'll need to double check this, but > it > seems unlikely. The 'configure' script creates a Makefile, but that > Makefile has > no test/check/fulltest targets. > > Dave > > I see. You are building from the amalgamation. And you should be too. But you are right - we do not ship test suites with the amalgamation. To use the free test suite for SQLite, you have to build from canonical source code. We have a separate test harness for the amalgamation (the one that provides 100% branch test coverage) but that one is not free, I'm afraid. > > > > > > > > Michael D. Black > > Senior Scientist > > Northrop Grumman Mission Systems > > > > > > ________ > > > > From: sqlite-users-boun...@sqlite.org on behalf of Dr. David Kirkby > > Sent: Thu 6/24/2010 10:10 AM > > To: General Discussion of SQLite Database > > Subject: Re: [sqlite] Why is there no test suite for ''make check' ? > > > > > > > > On 06/24/10 02:55 PM, Richard Hipp wrote: > >> On Thu, Jun 24, 2010 at 9:33 AM, Dr. David Kirkby > >> wrote: > >> > >>> I know the SQlite developers take testing quite seriously > >>> > >>> http://www.sqlite.org/testing.html > >>> > >>> with 679 times as much test code as actual code in the database. > >>> > >>> In fact, I've often pointed to the above page when trying to get the > >>> developers > >>> of the Sage maths software to pay a bit more attention to testing. > >>> > >>> > >>> > http://groups.google.co.uk/group/sage-devel/browse_thread/thread/eba06b53be7cd176/e938667e199bfce4?lnk=gst&q=sqlite#e938667e199bfce4 > >>> > >>> However, I was somewhat surprised when I run 'make check' to see there > is > >>> no > >>> test suite shipped as part of sqlite. Whilst I realise you do a lot of > >>> testing > >>> and it's impractical for an end user to do all this, I would have > thought a > >>> minimal test suite that tests 100 or so things would be useful. Often > bugs > >>> in > >>> compilers or operating systems display bugs that are not seen by the > >>> developers. > >>> That would be especially useful if someone is using a rarer platform > like > >>> AIX, > >>> HP-UX, or building in a less common way (like forcing 64-bit builds on > >>> Solaris, > >>> as someone was asking about). > >>> > >>> > >> Try instead: > >> > >> make test > > > > > > Thank you. > > > > Is this quite new? I don't see it in version 3.6.22. > > > > drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ > make test > > make: *** No rule to make target `test'. Stop. > > > > > >> Or, to keep your server busy all afternoon: > >> > >> make fulltest > > > > drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ > make fulltest > > make: *** No rule to make target `fulltest'. Stop. > > > > drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ > uname -a > > SunOS hawk 5.11 snv_134 i86pc i386 i86pc > > > > > > Dave > > ___ > > sqlite-users mailing list > > sqlite-users@sqlite.org > > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > > > > > > > > > > ___ > > sqlite-users mailing list > > sqlite-users@sqlite.org > > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- - D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
On 06/24/10 04:20 PM, Black, Michael (IS) wrote: > Try one directory up. It's not in the src directory Makefile. There is only one directory - there is no 'src' directory at all. If you look at the path what I posted below, you do see 'src' but that is part of the directory structure in Sage. In other words ~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src IS the top level directory in sqlite. (That's assuming some idiot has not screwed something up in Sage). I guess I'll need to double check this, but it seems unlikely. The 'configure' script creates a Makefile, but that Makefile has no test/check/fulltest targets. Dave > Michael D. Black > Senior Scientist > Northrop Grumman Mission Systems > > > > > From: sqlite-users-boun...@sqlite.org on behalf of Dr. David Kirkby > Sent: Thu 6/24/2010 10:10 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Why is there no test suite for ''make check' ? > > > > On 06/24/10 02:55 PM, Richard Hipp wrote: >> On Thu, Jun 24, 2010 at 9:33 AM, Dr. David Kirkby >> wrote: >> >>> I know the SQlite developers take testing quite seriously >>> >>> http://www.sqlite.org/testing.html >>> >>> with 679 times as much test code as actual code in the database. >>> >>> In fact, I've often pointed to the above page when trying to get the >>> developers >>> of the Sage maths software to pay a bit more attention to testing. >>> >>> >>> http://groups.google.co.uk/group/sage-devel/browse_thread/thread/eba06b53be7cd176/e938667e199bfce4?lnk=gst&q=sqlite#e938667e199bfce4 >>> >>> However, I was somewhat surprised when I run 'make check' to see there is >>> no >>> test suite shipped as part of sqlite. Whilst I realise you do a lot of >>> testing >>> and it's impractical for an end user to do all this, I would have thought a >>> minimal test suite that tests 100 or so things would be useful. Often bugs >>> in >>> compilers or operating systems display bugs that are not seen by the >>> developers. >>> That would be especially useful if someone is using a rarer platform like >>> AIX, >>> HP-UX, or building in a less common way (like forcing 64-bit builds on >>> Solaris, >>> as someone was asking about). >>> >>> >> Try instead: >> >> make test > > > Thank you. > > Is this quite new? I don't see it in version 3.6.22. > > drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ make test > make: *** No rule to make target `test'. Stop. > > >> Or, to keep your server busy all afternoon: >> >> make fulltest > > drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ make > fulltest > make: *** No rule to make target `fulltest'. Stop. > > drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ uname -a > SunOS hawk 5.11 snv_134 i86pc i386 i86pc > > > Dave > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > > > > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
Try one directory up. It's not in the src directory Makefile. Michael D. Black Senior Scientist Northrop Grumman Mission Systems From: sqlite-users-boun...@sqlite.org on behalf of Dr. David Kirkby Sent: Thu 6/24/2010 10:10 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Why is there no test suite for ''make check' ? On 06/24/10 02:55 PM, Richard Hipp wrote: > On Thu, Jun 24, 2010 at 9:33 AM, Dr. David Kirkby > wrote: > >> I know the SQlite developers take testing quite seriously >> >> http://www.sqlite.org/testing.html >> >> with 679 times as much test code as actual code in the database. >> >> In fact, I've often pointed to the above page when trying to get the >> developers >> of the Sage maths software to pay a bit more attention to testing. >> >> >> http://groups.google.co.uk/group/sage-devel/browse_thread/thread/eba06b53be7cd176/e938667e199bfce4?lnk=gst&q=sqlite#e938667e199bfce4 >> >> However, I was somewhat surprised when I run 'make check' to see there is >> no >> test suite shipped as part of sqlite. Whilst I realise you do a lot of >> testing >> and it's impractical for an end user to do all this, I would have thought a >> minimal test suite that tests 100 or so things would be useful. Often bugs >> in >> compilers or operating systems display bugs that are not seen by the >> developers. >> That would be especially useful if someone is using a rarer platform like >> AIX, >> HP-UX, or building in a less common way (like forcing 64-bit builds on >> Solaris, >> as someone was asking about). >> >> > Try instead: > > make test Thank you. Is this quite new? I don't see it in version 3.6.22. drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ make test make: *** No rule to make target `test'. Stop. > Or, to keep your server busy all afternoon: > > make fulltest drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ make fulltest make: *** No rule to make target `fulltest'. Stop. drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ uname -a SunOS hawk 5.11 snv_134 i86pc i386 i86pc Dave ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
On 06/24/10 02:55 PM, Richard Hipp wrote: > On Thu, Jun 24, 2010 at 9:33 AM, Dr. David Kirkby > wrote: > >> I know the SQlite developers take testing quite seriously >> >> http://www.sqlite.org/testing.html >> >> with 679 times as much test code as actual code in the database. >> >> In fact, I've often pointed to the above page when trying to get the >> developers >> of the Sage maths software to pay a bit more attention to testing. >> >> >> http://groups.google.co.uk/group/sage-devel/browse_thread/thread/eba06b53be7cd176/e938667e199bfce4?lnk=gst&q=sqlite#e938667e199bfce4 >> >> However, I was somewhat surprised when I run 'make check' to see there is >> no >> test suite shipped as part of sqlite. Whilst I realise you do a lot of >> testing >> and it's impractical for an end user to do all this, I would have thought a >> minimal test suite that tests 100 or so things would be useful. Often bugs >> in >> compilers or operating systems display bugs that are not seen by the >> developers. >> That would be especially useful if someone is using a rarer platform like >> AIX, >> HP-UX, or building in a less common way (like forcing 64-bit builds on >> Solaris, >> as someone was asking about). >> >> > Try instead: > > make test Thank you. Is this quite new? I don't see it in version 3.6.22. drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ make test make: *** No rule to make target `test'. Stop. > Or, to keep your server busy all afternoon: > > make fulltest drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ make fulltest make: *** No rule to make target `fulltest'. Stop. drkir...@hawk:~/parallel/sage-4.4.4/spkg/standard/sqlite-3.6.22/src$ uname -a SunOS hawk 5.11 snv_134 i86pc i386 i86pc Dave ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Why is there no test suite for ''make check' ?
On Thu, Jun 24, 2010 at 9:33 AM, Dr. David Kirkby wrote: > I know the SQlite developers take testing quite seriously > > http://www.sqlite.org/testing.html > > with 679 times as much test code as actual code in the database. > > In fact, I've often pointed to the above page when trying to get the > developers > of the Sage maths software to pay a bit more attention to testing. > > > http://groups.google.co.uk/group/sage-devel/browse_thread/thread/eba06b53be7cd176/e938667e199bfce4?lnk=gst&q=sqlite#e938667e199bfce4 > > However, I was somewhat surprised when I run 'make check' to see there is > no > test suite shipped as part of sqlite. Whilst I realise you do a lot of > testing > and it's impractical for an end user to do all this, I would have thought a > minimal test suite that tests 100 or so things would be useful. Often bugs > in > compilers or operating systems display bugs that are not seen by the > developers. > That would be especially useful if someone is using a rarer platform like > AIX, > HP-UX, or building in a less common way (like forcing 64-bit builds on > Solaris, > as someone was asking about). > > Try instead: make test Or, to keep your server busy all afternoon: make fulltest > > Dave > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- - D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users