Hi Elmar,
I think your plan covers the question "HOW do we want to unit test the
software" well. However, we have not thought much about the "WHAT do we
want to test?" question. Essentially, we need to think about which
classes/functions to test first.
I think it is not realistic to aim for a high test coverage with software
as large as LyX. Unit tests make sense in certain cases and less in others.
So we should identify these use cases first, and start with a few selective
unit tests. We can then grow them as we see fit.
For user-driven actions (LFUNs), LyX already has a test framework. However,
these tests do not cover internals of LyX.
Which internals do not require a complex sequence of actions (or a complex
internal state) to be tested? (Those that do are maybe better covered with
other types of tests.) Where has the code changed often in the past, and is
expected to change often in the future? What kinds of (internal) functions
are often covered by bug reports and thus warrant better test automation?
I hope some of the veteran developers can help answer these questions.
Elmar Hinz wrote:
Hello list,
I'd like to come up with a small plan for getting started with unit testing:
==========================
1.) Directory structure: "tests/unit/"
==========================
* Unit tests stay their own directory separated from src/.
* Below "tests/unit" the directory structure mirrors the structure of "src/".
* The reason for the subfolder "unit" is, that unit tests are not the
only type of tests. Unit tests test the single units of the source.
==========================
2.) Mocking: googlemock
==========================
This weekend I researched for alternatives, but there was none
that made a stable impression to me apart from googlemock.
With C++ mocking can take two approaches:
2.a) Turning classes into Templates, to be able to replace members by mocks.
http://programmaticallyspeaking.blogspot.de/2010/04/beautiful-dependency-injection-in-c.html
This seems to invasive into the existing sources for me.
2.b) Inheriting the mocks from the real objects, so that they are of the
same type.
This is the approach of googlemock.
Googlemock can be used with any mocking framework. It best integrates with
googletest.
==========================
3.) Testing framework: any
==========================
Unit tests are independent units, so any framework or none can be used.
For the reason of mocks, googletest is recommended.
==========================
4.) Runnig all tests
==========================
To be able to run all tests at once they need to be detected.
* The naming pattern of the binary to support this: whateverTest
* A successful test binary returns 0, the others an Error.
* The global test detector and runner is a Python script.
==========================
5.) Dependency injection
==========================
Dependency injection (the mock objects) is more difficult with C++
for multiple reasons like the missing garbage collection or reflection.
The approach that looks most forward and less intrusive for the sources
to me, is to directly access the private members by making them public
during testing and only during testing.
That is not really kosher, but at least a simple way to get started directly.
It is done by use of a simple preprocessor directive:
#public_on_testing
Once a wall of tests is created to ensure the behaviour of the given API,
more skilled approaches can be introduced.
\Elmar
--
Elmar Hinz
Freiherr-vom-Stein-Str. 1
33014 Bad Driburg
TYPO3 community contact: t.3.e.l.m.a...@.g.m.a.i.l.dot.c.o.m
personal contact: e.l.m.a.r.dot.h.i.n...@.g.m.a.i.l.dot.c.o.m
--
Regards,
Cyrille Artho - http://artho.com/
Love is the delusion that one woman differs from another.
-- H.L. Mencken
