Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[Jacques Carette]

On 12-06-23 04:38 AM, José Pedro Magalhães wrote:



On Tue, Jun 19, 2012 at 4:04 PM, Jacques Carette > wrote:


User beware: this is gencheck-0.1, there are still a few rough
edges.  We plan to add a Template Haskell feature to this which
should make deriving enumerators automatic for version 0.2.


Can you provide me a quick pointer into what methods need to be 
generated automatically?


Sure.  Given a data definition such as
data Zipper a = Zip ![a] ![a] deriving (Eq,Show)
(where the strictness annotations are not relevant), one would want to 
automatically generate


instance Enumerated Zipper where
   enumeration = eMemoize $  eProd Zip (eList A) (eList A)

and for
data BinTree a = BTNode a | BTBr (BinTree a) (BinTree a)

generate
instance Enumerated (BinTree Label) where
  enumeration = eMemoize $ eSum (eNode (BTNode A))
(eProd BTBr eBinTree eBinTree)

Jacques
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[Jacques Carette]

On 12-06-23 04:26 AM, Roman Cheplyaka wrote:

Hi,

1. Minor correction for your tutorial: "reverse . reverse = id" is
called the involution property, not idempotency.


Indeed!  Fixed, thanks.


2. Writing haddock documentation would definitely increase the chances
for GenCheck wide adoption.


Absolutely - I am working on that now.  It was always the plan, but we 
were sitting on GenCheck for too long, so I decided to push out an early 
0.1 release and fix that up for 0.2


Jacques


___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[José Pedro Magalhães]

On Tue, Jun 19, 2012 at 4:04 PM, Jacques Carette wrote:

> User beware: this is gencheck-0.1, there are still a few rough edges.  We
> plan to add a Template Haskell feature to this which should make deriving
> enumerators automatic for version 0.2.
>

Can you provide me a quick pointer into what methods need to be generated
automatically?


Thanks,
Pedro
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[Roman Cheplyaka]

Hi,

1. Minor correction for your tutorial: "reverse . reverse = id" is
   called the involution property, not idempotency.

2. Writing haddock documentation would definitely increase the chances
   for GenCheck wide adoption.

-- 
Roman I. Cheplyaka :: http://ro-che.info/

___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[Jacques Carette]

On 20/06/2012 6:56 AM, Henning Thielemann wrote:
QuickCheck is Haskell-98 and thus is very portable. I see that 
GenCheck needs some more extensions - type families, multi-parameter 
type classes, what else?


FlexibleContexts and FlexibleInstances.

However, I am fairly sure that the multi-parameter type classes are not 
in fact needed.  Certainly a branch of the current implementation could 
easily be done that removes these without harming the functionality 
currently implemented, although it might harm some of the 
extensibility.  And I think most of the uses of 
FlexibleContexts/Instances are tied to these MPTCs.


The one use of type families is to implement a view or, if you prefer, 
the 'get' part of a lens.  With type families, this is extremely 
elegant.  If portability is really important, this too could probably be 
branched into an implementation that does this otherwise.


I am actively working on refactoring the LabelledPartition MPTC into 
simpler pieces, but you'll have to wait until gencheck-0.2 for that.


Jacques

___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[Henning Thielemann]

On Tue, 19 Jun 2012, Jacques Carette wrote:


Its main novel features are:

* introduces a number of /testing strategies/ and /strategy combinators/
* introduces a variety of test execution methods
* guarantees uniform sampling (at each rank) for the random strategy
* guarantees both uniqueness and coverage of all structures for the
  exhaustive strategy
* introduces an /extreme/ strategy for testing unbalanced structures
* also introduces a /uniform/ strategy which does uniform sampling
  along an enumeration
* allows different strategies to be mixed; for example one can
  exhaustively test all binary trees up to a certain size, filled with
  random integers.
* complete separation between properties, generators, testing
  strategies and test execution methods



This sounds very interesting to me since I had a lot of trouble with 
changed test case distributions when switching from QuickCheck-1 to 
QuickCheck-2. It was mainly that tested numbers became much bigger in 
QuickCheck-2 and if I used the numbers as size of lists, then tests could 
not be run in reasonable time anymore. Thus I think more control over 
test-case generation is necessary.


QuickCheck is Haskell-98 and thus is very portable. I see that GenCheck 
needs some more extensions - type families, multi-parameter type classes, 
what else?


___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[Jacques Carette]

[Only cc:ing cafe]

There are definite similarities, yes - I only became aware of 
testing-feat very recently.   You seem to have concentrated more on 
efficiency, while we have focused more on the high-level modular design 
and on strategies.  We should probably merge our efforts, if you are 
willing.


I mistakenly build the release to upload to hackage from an experimental 
branch (which is currently broken).  The sources on github work.  I'll 
update gencheck shortly to fix this, sorry.


Jacques

On 12-06-19 12:58 PM, Jonas Almström Duregård wrote:

HI Jacques,

This looks very similar to the recently released testing-feat library:
http://hackage.haskell.org/package/testing-feat-0.2

I get a build error on the latest platform:

Test\GenCheck\Base\LabelledPartition.lhs:126:3:
 The equation(s) for `new' have two arguments,
 but its type `[a] ->  Map k a' has only one
 In the instance declaration for `LabelledPartition (Map k) a'

Regards,
Jonas




___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] [Haskell] [ANN] GenCheck - a generalized property-based testing framework

[Jonas Almström Duregård]

HI Jacques,

This looks very similar to the recently released testing-feat library:
http://hackage.haskell.org/package/testing-feat-0.2

I get a build error on the latest platform:

Test\GenCheck\Base\LabelledPartition.lhs:126:3:
The equation(s) for `new' have two arguments,
but its type `[a] -> Map k a' has only one
In the instance declaration for `LabelledPartition (Map k) a'

Regards,
Jonas


On 19 June 2012 17:04, Jacques Carette  wrote:
> Test.GenCheck is a Haskell library for generalized proposition-based
> testing. It simultaneously generalizes QuickCheck and SmallCheck.
>
> Its main novel features are:
>
> introduces a number of testing strategies and strategy combinators
> introduces a variety of test execution methods
> guarantees uniform sampling (at each rank) for the random strategy
> guarantees both uniqueness and coverage of all structures for the exhaustive
> strategy
> introduces an extreme strategy for testing unbalanced structures
> also introduces a uniform strategy which does uniform sampling along an
> enumeration
> allows different strategies to be mixed; for example one can exhaustively
> test all binary trees up to a certain size, filled with random integers.
> complete separation between properties, generators, testing strategies and
> test execution methods
>
> The package is based on a lot of previous research in combinatorics
> (combinatorial enumeration of structures and the theory of Species), as well
> as a number of established concepts in testing (from a software engineering
> perspective). In other words, further to the features already implemented in
> this first release, the package contains an extensible, general framework
> for generators, test case generation and management. It can also be very
> easily generalized to cover many more combinatorial structures unavailable
> as Haskell types.
>
> The package also provides interfaces for different levels of usage. In other
> words, there is a 'simple' interface for dealing with straightforward
> testing, a 'medium' interface for those who want to explore different
> testing strategies, and an 'advanced' interface for access to the full power
> of GenCheck.
>
> See http://hackage.haskell.org/package/gencheck for further details.
>
> In the source repository (https://github.com/JacquesCarette/GenCheck), the
> file tutorial/reverse/TestReverseList.lhs shows the simplest kinds of tests
> (standard and deep for structures, or base for unstructured types) and
> reporting (checking, testing and full report) for the classical list reverse
> function. The files in tutorial/list_zipper show what can be done with the
> medium level interface (this tutorial is currently incomplete). The brave
> user can read the source code of the package for the advanced usage -- but
> we'll write a tutorial for this too, later.
>
> User beware: this is gencheck-0.1, there are still a few rough edges.  We
> plan to add a Template Haskell feature to this which should make deriving
> enumerators automatic for version 0.2.
>
> Jacques and Gordon
>
>
> ___
> Haskell mailing list
> hask...@haskell.org
> http://www.haskell.org/mailman/listinfo/haskell
>

___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe