+Roger, for Travis-CI Hi Teddy,
On 3 July 2016 at 13:17, Teddy Reed <teddy.r...@gmail.com> wrote: > Hi Simon, > > On Sun, Jul 3, 2016 at 8:40 AM, Simon Glass <s...@chromium.org> wrote: >> Add a few notes about how testing works in U-Boot. >> >> Signed-off-by: Simon Glass <s...@chromium.org> > > Reviewed-by: Teddy Reed <teddy.r...@gmail.com> Thanks for the comments. Will tidy up. > >> --- >> >> test/README | 82 >> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 82 insertions(+) >> create mode 100644 test/README >> >> diff --git a/test/README b/test/README >> new file mode 100644 >> index 0000000..dfd83d6 >> --- /dev/null >> +++ b/test/README >> @@ -0,0 +1,82 @@ >> +Testing in U-Boot >> +================= >> + >> +U-Boot has a large amount of code. This file describes how this code is >> +tested and what tests you should write when adding a new feature. >> + >> + >> +Sandbox >> +------- >> +U-Boot can be built as a user-space application (e.g. for Linux). This >> +allows test to be executed without needing target hardware. The 'sandbox' >> +target provides this feature and it is widely used in tests. >> + >> + >> +Pytest Suite >> +------------ >> + >> +Many tests are available using the pytest suite, in test/py. This can run >> +either on sandbox or on real hardware. It relies on the U-Boot console to >> +inject test commands and check the result. It is slower to run than C code, >> +but provides the ability to unify lots of test and summarise their results. > > lots of tests > >> + >> +You can run the tests on sandbox with: >> + >> + ./test/py/test.py --bd sandbox --build >> + >> +This will produce HTML output in build-sandbox/test-log.html >> + >> +See test/py/README.md for more information about the pytest suite. >> + >> + >> +tbot >> +---- >> + >> +Tbot provides a way to execute tests on target hardware. It is intended for >> +trying out both U-Boot and Linux (and potentially other software) on a >> +number of boards automatically. It can be used to create a continuous test >> +environment. See tools/tbot/README for more information. >> + >> + >> +Ad-hoc tests >> +------------ >> + >> +There are several ad-hoc tests which run outside the pytest environment: >> + >> + test/fs - File system test (shell script) >> + test/image - FIT and lagacy image tests (shell script and Python) > > s/lagacy/legacy/ > >> + test/stdint - A test that stdint.h can be used in U-Boot (shell script) >> + trace - Test for the tracing feature (shell script) >> + vboot - Test for verified boot (shell script) >> + >> +The above should be converted to run as part of the pytest suite. > > Is this a NOTE or a TODO? TODO - I'll reword it. > >> + >> + >> +When to write tests >> +------------------- >> + >> +If you add code to U-Boot without a test you are taking a risk. Even if you >> +perform thorough manual testing at the time of submission, it may break when >> +future changes are made to U-Boot. It may even break when applied to >> mainline, >> +if other changes interact with it. A good mindset is that untested code >> +probably doesn't work and should be deleted. >> + >> +You can assume that the Pytest suite will be run before patches are accepted >> +to mainline, so this provides protection against future breakage. >> + >> +On the other hand there is quite a bit of code that is not covered with >> tests, >> +or is covered sparingly. So here are some suggestions: >> + >> +- If you are adding a new uclass, add a sandbox driver and a test that uses >> it >> +- If you are modifying code covered by an existing test, add a new test case >> + to cover your changes >> +- If the code you are modifying has not tests, consider writing one. Even a >> + very basic test is useful, and may be picked up and enhanced by others. It >> + is much easier to add onto a test - writing a new large test can seem >> + daunting to most contributors. >> + >> + >> +Future work >> +----------- >> + >> +Converting existing shell scripts into pytest tests. >> -- >> 2.8.0.rc3.226.g39d4020 >> >> _______________________________________________ >> U-Boot mailing list >> U-Boot@lists.denx.de >> http://lists.denx.de/mailman/listinfo/u-boot > > Thank for these docs! I wonder if a clone/mirror of the repo on Github > can be set up to run the sandbox/Pytest tests in TravisCI with minimum > effort? :) There is a .travis.yml file - I have not tried this but I believe people are already doing it somewhere. Regards. Simon _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot