-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14/10/2014 8:40 p.m., Tianyin Xu wrote: > Thanks, Amos! > > One more question. I see test-builds.sh tries to run the tests in > the "test-suites" at the top directory. What's the difference > between the test under "src/tests" and "test-suites"? > > I'm a little bit confused since you said the tests in "src/tests" > contains all the unit tests. > > Also, I can successfully ran make check. But when I checked out the > trunk and ran ./test-builds.sh at the top directory, many tests > fail. Is this normal? (sorry I actually do not understand what > "test-suites" does)
That depends on how complete your build environment is. The script begins with testing ./bootstrap.sh to ensure the toolchain is complete and working. Recent autoconf produce a lot of warnings about subdir-objects feature, but should still finish saying "success". If it does not then the error it exits with should explain what needs to be resolved, usually some portion of the autotools build environment is outdated or missing. For the later "layers" being tested (eg the ESI layer) there are some additional libraries needed to be installed which are optional for normal users. You *need* this script to run to completion with no errors on the current 3.HEAD code for it to be worth going any further into the testing. So, what OS are you building on? and what are the specific error(s) the script is failing with? > > Oh, I forget to mention the web page I referred to. Actually I'm > referring to the "tasks needing a volunteer" in the following wiki > page, http://wiki.squid-cache.org/Features/CodeTestBed Are these > tasks still alive? Since I'm looking at the tests, I think I can > give a hand. Aha. Wonderful thank you. In that case I better explain in a bit more detail than just how to use them. :-) src/tests/ content is the main code for what the wiki page Details section refer to as "ad-hoc CPP Unit-tests". Other unit-tests also exist elsewhere in the sources (eg pre-compiler unit tests in compat/). Each of those src/tests/test*.cc is a test of a single "unit" (aka. C++ class). "make check" turns it into a binary that is supposed to exhaustively test what happens when any permutation of input is presented to the class API/methods, including edge cases and soemtimes wrong inputs. Unit-tests like that can only test one way of compiling the code at a time. Which is fine for validating that some pre-built binary will work as intended. But fails miserably to validate source code for software like Squid with many possible features built (or not) by users. That is where the ./test-builds.sh script comes in. (See under "Component Test Controls".) It runs the unit tests with different permutations of ./configure settings. It also runs some large scale tests, such as squid.conf parsing for various known squid.conf contents which have or might cause trouble, or the bootstrap to check the build chain is operational. Ideally both unit-test and the ./configure tests would be exhaustive and cover every bit of code at least once. That is a very long way off still, and may actually never happen. But that is the goal. So, yes the items in both "Tasks needing a volunteer:" lists still need doing. The work is being coordinated in wiki page <http://wiki.squid-cache.org/Features/SourceLayout>. The main table first column indicates whether the new polished convenience libraries have unit-tests for the classes inside. That column content is outdated, in fact many of those classes do have unit-tests but nobody has audited to check if they are complete and correct for the current API. So part of the needed work is correcting that column entries. There is another item related to the testing which is not documented there. That is the compiler build options which are available to detect potential issues. Squid is only able to build cleanly with a small set at present, additional work is needed to try building with others and fix the issues they highlight. I hope that is clear enough to understand. I am assuming you at least know the basics of what testing is all about. PS. since 3.5 is now in beta I will this week be auditing the ./configure tests options are up to date. That may affect the test coverage and results somewhat in the coming days. Amos -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUPPpqAAoJELJo5wb/XPRjKhEH/1QjhBXv6n3MM+EhduJ5q5NP vKYzNTrg71vkoCcZ/7VoTTp5NXCdS9D4mg5cfaZtjHYU5HhZJqaJ3sxp3iuRE8Sh EWHEIUWPu3+95PAdnnB4FAzEcwrDZoTvbMc7P7Mo+/vKQp3ljfA2x+uYLUZjtucV h/N5W96ZfAWqYFdHUeZFh0xJ6+bNtsV1ndjlnSddPfMFfCCZIzEk/+QQacuZm9d7 aS7lO8Ig7IlZ+v6FJglWbVt7PGEnvQclS/J9hH6lQqV5bwQPnGZiS4zsLm2MFfPK vMIRngmKrAw6vDrdCpuEXHjFBxosaGdLHTvrwwMMTFTMbFo0fRN9lcFJiZ7qI+g= =zK+F -----END PGP SIGNATURE----- _______________________________________________ squid-dev mailing list [email protected] http://lists.squid-cache.org/listinfo/squid-dev
