Moritz Lenz wrote:
A few months ago Larry proposed to add some testing
facilites to the language itself, because we want to
culturally encourage testing, and because the test
suite defines the language, so we need to specify the
behaviour of our testing facilities anyway.
If we're going to revamp the testing primitives, then I'd like to
suggest importing some concepts from hardware verification langauges,
whose entire purpose is to define tests. Not too much, but just a few defns:
* Define a "property" as an expression whose truth is of interest
(properties may be named, or may be anonymous inline).
* An "assert <property>" statement (aka "ok <property>") indicates that
a violation of the property is to be considered an error
* An "assume <property>" statement indicates that a violation of the
property implies an incorrect test.
Assumptions are very important when you write automated test generators,
or need to validate that your tests are not doing something illegal. If
you violate an assumption when running normal code then it's not really
any different from hitting an assertion. You want as many assumptions as
possible to be part of the type system (which we already do with "where"
clauses).
(I'm using the phrase "assert" instead of "ok" because that's the
standard terminology: In perl, "ok" is standard, so no need to rename
it. But I do think that we need to qualify it with whether it's an
assumption or an assertion. You can also think of an assumption as a
precondition. But adding "PRE" blocks to every function tends to
encourage cargo-cult DBC programming.)
The other thing I'd like to point out is that the concept of a
"property" can be very general. We shouldn't assume that they just sit
in the middle of procedural code. Specifically, it should be possible to
define invariants on objects, which should be true at some specified
point in time (however, it's not always obvious what that point in time is).
I'm thinking we might have something like:
class Foo {
has $.a;
has $.b;
property conserve_sum { $.a + $.b == 42 },
"a+b must sum to 42, but a=$.a + b=$.b == { $.a+$.b }";
method foo() does assume<conserve_sum> { ... }
method bar() does ensure<conserve_sum> { ... }
method baz() { bar; ok conserve_sum; foo; }
}
A property is just a method that returns a Bool but, if you associate
the failure message with it, then it becomes simple to assert/assumme it
in multiple places without needing to keep repeating the message.
An interesting type of property is one that tracks a series of events
through time: a so called "temporal" property. A simple idea might be
that "conserve_sum" should actually mean "sum does't change", instead of
"is constant 42":
class Foo {
...
coro property conserve_sum {
my $sum = $.a + $.b;
leave True;
ok $.a + $.b == $sum,
"sum not conserved: expected $sum, actual {$.a+$.b}"
}
method foo() does maintain<conserve_sum> { --$.a; ++$.b }
}
I don't think that it is necessary to be too cute with huffmannization
of testing primitives. All my examples here are just thinking aloud.
