Re: Integration tests
On Wednesday, 22 April 2020 at 10:32:48 UTC, Russel Winder wrote:
I ended up creating the following project structure:
.
├── dub.sdl
├── dub.selections.json
├── source
│ ├── arcam_protocol.d
│ └── main.d
├── tests
│ └── integration_tests.d
└── test_support
└── mock_avr850
└── main.d
with the following Dub control file:
name "arcamclient"
description "arcamclient is a Rust/gtk-rs/GTK+ desktop
application to control an Arcam amplifier over the Ethernet
connection."
authors "Russel Winder"
copyright "Copyright © 2020 Russel Winder."
license "GPL-3.0"
targetType "executable"
targetPath "bin"
configuration "application" {
}
configuration "unittest" {
targetName "arcamclient_test"
dependency "unit-threaded" version="*"
mainSourceFile "bin/ut.d"
excludedSourceFiles "source/main.d"
preBuildCommands "$DUB run --compiler=$$DC unit-threaded -c
gen_ut_main -- -f bin/ut.d -d $DUB"
preBuildCommands "$DUB build arcamclient:mock_avr850"
importPaths "tests"
sourcePaths "tests"
}
subPackage {
name "mock_avr850"
targetName "mock_avr850"
targetType "executable"
targetPath "bin"
sourcePaths "source" "test_support/mock_avr850"
importPaths "source" "test_support/mock_avr850"
excludedSourceFiles "source/main.d"
}
This seems a bit more sensible that what I have been able to
achieve with Rust, but is still second rate compared to how
easy things are using Python.
Have you try Silly? I found far more straightforward to use, that
unit-threaded.
On my pet game engine, I got it working with this :
"configurations": [
...
{
"dependencies": {
"beep": "~>0.0.2",
"silly": "~>1.0.2"
},
"name": "unittest",
"targetType": "library"
}
],
Sadly, Silly only is the test runner (and one far prettier that
unit-threaded). I need to add separated dub module to make
assertion more easy and fluent. Beep go on the correct path with
a API like this :
@("My test")
unittest {
myFunc(1, 2, 3).expect!equals(42);
auto a = [1, 2 ,3];
a.expect!contains(2);
a.lenght.expect!great(0);
true.expect!true();
}
And the output on case of fail, shows a human friendly message
with the expected value and the real value.
Another assertion module that not depends on unit-threaded and
have better API, is pyjamas . But actually is abandoned. I did a
try to get it working again, but depends on another dead dub
module... another test runner called "bed" (that I did a PR to
fix it to get it working against the actual DMD version).
Finally... What I really miss are two things :
- A test/integration framework like Spock (Yeah, I professionally
work on Java world :( )
- Test runners had a common output format that make IDE/tools
work far more easy. See
https://github.com/nomad-software/dunit/issues/19#issuecomment-435209223
Offtopic: Should be a way to mark a dub module dead on dub
register. This last weeks I don't stop of finding dead/not
working stuff and given a really bad image of the state of D.
Re: Integration tests
On Wed, 2020-04-22 at 11:19 +, aliak via Digitalmars-d-learn wrote: > On Wednesday, 22 April 2020 at 10:32:48 UTC, Russel Winder wrote: > > Now I discover Python, Rust, and Go have far nicer abstractions > > for writing Internet code than D does. Does D really not have a > > TcpListener abstraction? > > It really doesn't :( :-( Even GTK+ has it's own wrappers around the base socket API to make it sensible for programmers. GtkD offers these for GtkD-based applications, but D has no language support for asynchronous (via Futures/Promises/event loops) which leaves Rust (and Python) far ahead in this race to support asynchronous programming. > And D has so much potential as server tech with the potential > combination of fibers + TLS + shared + static introspection. Potential is necessary but not sufficient. Vibe.d seems to be one solution (in the Rust Async_std and Tokio sense) but Rust has language level support for Futures that make everything a lot easier in Rust than seemingly in D. > The package Vibe-d is quite nice though. I don't know if you've > tried it but it's very simple to get a listener up with it. And now it seems we have Hunt. I am now dithering whether to use Vibe.d or Hunt for my async TCP (but not HTTP(S)) server. […] -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Option and Result [was Integration tests]
On Tue, 2020-04-21 at 21:29 +, aliak via Digitalmars-d-learn wrote: > On Tuesday, 21 April 2020 at 16:30:15 UTC, Russel Winder wrote: […] > > There've been a few attempts at building a Result type: > > https://code.dlang.org/search?q=expect > And here: > https://github.com/aliak00/ddash/blob/master/utils/source/ddash/utils/expect.d > > > Is Option intended for adding to Phobos? > > Not that I am aware of. There was an attempt to PR an Option type > way back when which never made it: > https://github.com/dlang/phobos/pull/3915 > > There was a post here: > https://forum.dlang.org/thread/hqtdekjtdgbhhbjgy...@forum.dlang.org Perhaps there can be one implementation package of Nullable, Option, and Result that is in the Dub repository that everyone can use even though they really ought to be part of Phobos. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Integration tests
On Wednesday, 22 April 2020 at 10:32:48 UTC, Russel Winder wrote: Now I discover Python, Rust, and Go have far nicer abstractions for writing Internet code than D does. Does D really not have a TcpListener abstraction? It really doesn't :( And D has so much potential as server tech with the potential combination of fibers + TLS + shared + static introspection. The package Vibe-d is quite nice though. I don't know if you've tried it but it's very simple to get a listener up with it. To date all I can get is: std.socket.SocketOSException@std/socket.d(2792): Unable to bind socket: Bad file descriptor when trying to open a TCP server on 127.0.0.1:5, with Python, Rust, or Go it all worked first time. This is really sad given D has so many advantages over Rust. :-(
Re: Integration tests
I ended up creating the following project structure:
.
├── dub.sdl
├── dub.selections.json
├── source
│ ├── arcam_protocol.d
│ └── main.d
├── tests
│ └── integration_tests.d
└── test_support
└── mock_avr850
└── main.d
with the following Dub control file:
name "arcamclient"
description "arcamclient is a Rust/gtk-rs/GTK+ desktop application to control
an Arcam amplifier over the Ethernet connection."
authors "Russel Winder"
copyright "Copyright © 2020 Russel Winder."
license "GPL-3.0"
targetType "executable"
targetPath "bin"
configuration "application" {
}
configuration "unittest" {
targetName "arcamclient_test"
dependency "unit-threaded" version="*"
mainSourceFile "bin/ut.d"
excludedSourceFiles "source/main.d"
preBuildCommands "$DUB run --compiler=$$DC unit-threaded -c gen_ut_main --
-f bin/ut.d -d $DUB"
preBuildCommands "$DUB build arcamclient:mock_avr850"
importPaths "tests"
sourcePaths "tests"
}
subPackage {
name "mock_avr850"
targetName "mock_avr850"
targetType "executable"
targetPath "bin"
sourcePaths "source" "test_support/mock_avr850"
importPaths "source" "test_support/mock_avr850"
excludedSourceFiles "source/main.d"
}
This seems a bit more sensible that what I have been able to achieve
with Rust, but is still second rate compared to how easy things are
using Python.
Now I discover Python, Rust, and Go have far nicer abstractions for
writing Internet code than D does. Does D really not have a TcpListener
abstraction?
To date all I can get is:
std.socket.SocketOSException@std/socket.d(2792): Unable to bind socket: Bad
file descriptor
when trying to open a TCP server on 127.0.0.1:5, with Python, Rust,
or Go it all worked first time. This is really sad given D has so many
advantages over Rust. :-(
--
Russel.
===
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Roadm: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
Re: Option and Result [was Integration tests]
On Tuesday, 21 April 2020 at 16:30:15 UTC, Russel Winder wrote: On Mon, 2020-04-20 at 20:19 +, aliak via Digitalmars-d-learn wrote: […] [0]: https://github.com/aliak00/optional Rust has Option and Result, and most languages are rapidly introducing at least Option if not Result – and yes it is almost certain all this comes from Haskell. Yeah it's a great abstraction that's part of modern languages now. Some static languages are even building syntax for it right in (e.g. swift, kotlin, v, zig - to name a few). There've been a few attempts at building a Result type: https://code.dlang.org/search?q=expect And here: https://github.com/aliak00/ddash/blob/master/utils/source/ddash/utils/expect.d Is Option intended for adding to Phobos? Not that I am aware of. There was an attempt to PR an Option type way back when which never made it: https://github.com/dlang/phobos/pull/3915 There was a post here: https://forum.dlang.org/thread/hqtdekjtdgbhhbjgy...@forum.dlang.org
Re: Option and Result [was Integration tests]
On Tuesday, 21 April 2020 at 16:30:15 UTC, Russel Winder wrote: On Mon, 2020-04-20 at 20:19 +, aliak via Digitalmars-d-learn wrote: […] [0]: https://github.com/aliak00/optional Rust has Option and Result, and most languages are rapidly introducing at least Option if not Result – and yes it is almost certain all this comes from Haskell. Is Option intended for adding to Phobos? There is Nullable in Phobos. This is not really an Option, I know. But just in case. But back to integration tests, there are indeed only few possibilities to mock something. However, the basics are there. I found a section of code generation a while ago in std.typecons. https://dlang.org/library/std/typecons.html There are Black and White holes there, for example, which could help to implement a basic mocking framework.
Re: Option and Result [was Integration tests]
On Mon, 2020-04-20 at 20:19 +, aliak via Digitalmars-d-learn wrote: > > […] > [0]: https://github.com/aliak00/optional Rust has Option and Result, and most languages are rapidly introducing at least Option if not Result – and yes it is almost certain all this comes from Haskell. Is Option intended for adding to Phobos? -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Integration tests
On Friday, 17 April 2020 at 16:56:57 UTC, Russel Winder wrote: Hi, Thinking of trying to do the next project in D rather than Rust, but… Rust has built in unit testing on a module basis. D has this so no problem. Rust allows for integration tests in the tests directory of a project. These are automatically build and run along with all unit tests as part of "cargo test". Does D have any integrated support for integration tests in the way Rust does? D does not recognise any special testing directories afaik. But you'll get the same affect as in rust if you have your integration tests in a file that is not the modules you are testing and run it after building the main library - or during even. With Dub it's quite easy to set up, you can see the optional package as an example [0]. There's a tests directory that does not have access to the optional package's internals. There's also a unittest-compat configuration in dub.json that tests the package alongside vibed (another d package) For auto mocking, the only thing I've see that provides some functionality is the unit-threaded package. But I've never tried it. [0]: https://github.com/aliak00/optional
Re: Integration tests
On Fri, 2020-04-17 at 17:51 +, Jon Degenhardt via Digitalmars-d- learn wrote: > On Friday, 17 April 2020 at 16:56:57 UTC, Russel Winder wrote: > > Hi, > > > > Thinking of trying to do the next project in D rather than > > Rust, but… > > > > Rust has built in unit testing on a module basis. D has this so > > no problem. > > > > Rust allows for integration tests in the tests directory of a > > project. These are automatically build and run along with all > > unit tests as part of "cargo test". > > > > Does D have any integrated support for integration tests in the > > way > > Rust does? > > Automated testing is important, perhaps you describe further > what's needed? I haven't worked with Rust test frameworks, but I > took a look at the description of the integration tests and unit > tests. It wasn't immediately obvious what can be done with the > Rust integration test framework that cannot be done with D's > unittest framework. I should point out that Rust is far behind Python's PyTest in terms of testing sophistication, and I fear D lags behind Rust. I think the following is just saying what everyone knows, but it seems making it explicit for this thread. Let us distinguish unit testing, integration testing, and system testing using the definitions: unit testing is self standing testing the code with no extra dependencies; integration testing is testing some functionality of the system with all external resources mocked out (possibly by using a process to simulate the external resource, which is what I want for this project); system testing is testing some functionality with real external resources. D, like Rust, supports unit testing very nicely, well unit_threading seems essential, on a per module basis. Python unit testing with PyTest is very straightforward. Rust supports integration testing explicitly, D does not. Python PyTest does but only because with Python you need no extra infrastructure. Integration testing must not have access to the internals of a module or group of modules, but should appear as a using application. Rust/Cargo integrates this using the tests directory as a peer to the src directory: each file in the tests directory represents a crate that uses the code under test as a crate. Integration and automated system testing in Rust is really a question of whether external resources are mocked or used for real. Python PyTest integration and system testing is very much the same. As far as I know, D/Dub, D/Meson, D/SCons, D/CMake, and D/Make do not provide an out of the box way of building integration tests. Hidden agenda item: system testing really needs to be in a sandbox of some sort. > An important concept described was testing a module as an > external caller. That would seem very be doable using D's > unittest framework. For example, one could create a set of tests > against Phobos, put them in a separate location (e.g. a separate > file), and arrange to have the unittests run as part of a CI > process run along with a build. > > My look was very superficial, perhaps you could explain more. The important difference between using a unit test infrastructure and an integration/system test infrastructure is that: – unit tests are inside the system testing functions and other code features; – integration tests are outside the system testing functionality with mocked external resources. So the build system needs to build all the codes that realise the mocks of the external resources. So for a vestigial Rust project: . ├── Cargo.lock ├── Cargo.toml ├── README.md ├── src │ ├── arcam_protocol.rs │ ├── lib.rs │ └── main.rs └── tests ├── integration_tests.rs └── mock_avr850.rs src/arcam_protocol/rs has all the unit tests in it. D can do the equivalent. tests/mock_avr850.rs is an application that mocks a real amplifier; tests/integration_tests is an application that uses the project as an external crate and starts the mock amplifier for each of the tests. As far as I can tell there is no Dub, Meson, Scons, CMake, Make infrastructure for building D code that supports this sort of thing. Python using PyTest can do all the above really very trivially, D/Cargo has yet to catch up! -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Integration tests
On Friday, 17 April 2020 at 16:56:57 UTC, Russel Winder wrote: Hi, Thinking of trying to do the next project in D rather than Rust, but… Rust has built in unit testing on a module basis. D has this so no problem. Rust allows for integration tests in the tests directory of a project. These are automatically build and run along with all unit tests as part of "cargo test". Does D have any integrated support for integration tests in the way Rust does? Automated testing is important, perhaps you describe further what's needed? I haven't worked with Rust test frameworks, but I took a look at the description of the integration tests and unit tests. It wasn't immediately obvious what can be done with the Rust integration test framework that cannot be done with D's unittest framework. An important concept described was testing a module as an external caller. That would seem very be doable using D's unittest framework. For example, one could create a set of tests against Phobos, put them in a separate location (e.g. a separate file), and arrange to have the unittests run as part of a CI process run along with a build. My look was very superficial, perhaps you could explain more.
Integration tests
Hi, Thinking of trying to do the next project in D rather than Rust, but… Rust has built in unit testing on a module basis. D has this so no problem. Rust allows for integration tests in the tests directory of a project. These are automatically build and run along with all unit tests as part of "cargo test". Does D have any integrated support for integration tests in the way Rust does? -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
