On Fri, 2020-04-17 at 17:51 +0000, Jon Degenhardt via Digitalmars-d- learn wrote: > On Friday, 17 April 2020 at 16:56:57 UTC, Russel Winder wrote: > > Hi, > > > > Thinking of trying to do the next project in D rather than > > Rust, but… > > > > Rust has built in unit testing on a module basis. D has this so > > no problem. > > > > Rust allows for integration tests in the tests directory of a > > project. These are automatically build and run along with all > > unit tests as part of "cargo test". > > > > Does D have any integrated support for integration tests in the > > way > > Rust does? > > Automated testing is important, perhaps you describe further > what's needed? I haven't worked with Rust test frameworks, but I > took a look at the description of the integration tests and unit > tests. It wasn't immediately obvious what can be done with the > Rust integration test framework that cannot be done with D's > unittest framework.
I should point out that Rust is far behind Python's PyTest in terms of testing sophistication, and I fear D lags behind Rust. I think the following is just saying what everyone knows, but it seems making it explicit for this thread. Let us distinguish unit testing, integration testing, and system testing using the definitions: unit testing is self standing testing the code with no extra dependencies; integration testing is testing some functionality of the system with all external resources mocked out (possibly by using a process to simulate the external resource, which is what I want for this project); system testing is testing some functionality with real external resources. D, like Rust, supports unit testing very nicely, well unit_threading seems essential, on a per module basis. Python unit testing with PyTest is very straightforward. Rust supports integration testing explicitly, D does not. Python PyTest does but only because with Python you need no extra infrastructure. Integration testing must not have access to the internals of a module or group of modules, but should appear as a using application. Rust/Cargo integrates this using the tests directory as a peer to the src directory: each file in the tests directory represents a crate that uses the code under test as a crate. Integration and automated system testing in Rust is really a question of whether external resources are mocked or used for real. Python PyTest integration and system testing is very much the same. As far as I know, D/Dub, D/Meson, D/SCons, D/CMake, and D/Make do not provide an out of the box way of building integration tests. Hidden agenda item: system testing really needs to be in a sandbox of some sort. > An important concept described was testing a module as an > external caller. That would seem very be doable using D's > unittest framework. For example, one could create a set of tests > against Phobos, put them in a separate location (e.g. a separate > file), and arrange to have the unittests run as part of a CI > process run along with a build. > > My look was very superficial, perhaps you could explain more. The important difference between using a unit test infrastructure and an integration/system test infrastructure is that: – unit tests are inside the system testing functions and other code features; – integration tests are outside the system testing functionality with mocked external resources. So the build system needs to build all the codes that realise the mocks of the external resources. So for a vestigial Rust project: . ├── Cargo.lock ├── Cargo.toml ├── README.md ├── src │ ├── arcam_protocol.rs │ ├── lib.rs │ └── main.rs └── tests ├── integration_tests.rs └── mock_avr850.rs src/arcam_protocol/rs has all the unit tests in it. D can do the equivalent. tests/mock_avr850.rs is an application that mocks a real amplifier; tests/integration_tests is an application that uses the project as an external crate and starts the mock amplifier for each of the tests. As far as I can tell there is no Dub, Meson, Scons, CMake, Make infrastructure for building D code that supports this sort of thing. Python using PyTest can do all the above really very trivially, D/Cargo has yet to catch up! -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part