Re: Spec & test.check generators for complex data structures

[Alex Miller]

The general approach to this is to first create a "model" of your domain, 
which might be some random seed info and then generate one or more entities 
that are consistent with the model.

Stu does a simple example of this in 
http://blog.cognitect.com/blog/2016/8/10/clojure-spec-screencast-customizing-generators
 
using gen/fmap and gen/bind which is probably the best way to build 
something like this.

As an example, if you need to test on a graph of nodes, your model might 
consist of first creating random nodes, then generating random edges 
between the nodes (rather than randomly generating nodes and edges, which 
will likely have no commonality). 



On Tuesday, November 1, 2016 at 2:14:28 PM UTC-5, Maarten Truyens wrote:
>
> Completely convinced of Spec, I am diving into property-based testing with 
> test.check. However, I encounter some difficulties for which I would like 
> to get the community's input. 
>
> The central part of the software I am developing, consists of a tree of 
> different kinds of nodes that can refer to — and interact with — each 
> other. There are many business rules that specify how those nodes can 
> interact — e.g., a node of type A can refer to other nodes of types B and C 
> (but not D); or a node can have max. 3 child nodes of type E but more than 
> 3 child nodes of type F. 
>
> It is trivial to have Spec generate completely random nodes to test 
> functions that can ignore the business rules. However, other functions do 
> depend on the business rules, and for those it is necessary to somewhat 
> "guide" the generators towards random-but-compliant output. And here I'm 
> stumbling: due to the business rules' complexity, this is not a matter of 
> simply inserting a few such-that generators, so that I have already spent 
> many days on writing a business rule compliant generator. Part of this is 
> due to the multitude and complexity of the business rules, part due to the 
> difficulty of writing generators (which operate at a higher level of 
> abstraction, and for which only a subset of the normal Clojure constructs 
> can be used).
>
> How do others cope with this issue? I have the feeling that I am pushing 
> the generators far beyond what they were designed to do, but at the same I 
> have to acknowledge that the time required to write traditional tests for 
> functions dealing with the business rules is also substantial — so maybe 
> the time required to write those generators is not abnormal.   
>
> Any suggestions are welcomed! I did not find any in-depth observations of 
> other developers about property-based testing on the Net.
>
> Maarten
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Spec & test.check generators for complex data structures

[Maarten Truyens]

Thanks - the start of that thread was indeed interesting, but it quickly 
changed course towards a discussion on whether Spec should check function 
outputs. Rich hinted at exactly the kind of in-depth discussion I wanted to 
find — but much too briefly... 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Spec & test.check generators for complex data structures

[Josh Tilles]

I don’t have any answers, but you might find the discussion in the first 
several messages of this thread 
 to be 
relevant.

On Tuesday, November 1, 2016 at 3:14:28 PM UTC-4, Maarten Truyens wrote:
>
> Completely convinced of Spec, I am diving into property-based testing with 
> test.check. However, I encounter some difficulties for which I would like 
> to get the community's input. 
>
> The central part of the software I am developing, consists of a tree of 
> different kinds of nodes that can refer to — and interact with — each 
> other. There are many business rules that specify how those nodes can 
> interact — e.g., a node of type A can refer to other nodes of types B and C 
> (but not D); or a node can have max. 3 child nodes of type E but more than 
> 3 child nodes of type F. 
>
> It is trivial to have Spec generate completely random nodes to test 
> functions that can ignore the business rules. However, other functions do 
> depend on the business rules, and for those it is necessary to somewhat 
> "guide" the generators towards random-but-compliant output. And here I'm 
> stumbling: due to the business rules' complexity, this is not a matter of 
> simply inserting a few such-that generators, so that I have already spent 
> many days on writing a business rule compliant generator. Part of this is 
> due to the multitude and complexity of the business rules, part due to the 
> difficulty of writing generators (which operate at a higher level of 
> abstraction, and for which only a subset of the normal Clojure constructs 
> can be used).
>
> How do others cope with this issue? I have the feeling that I am pushing 
> the generators far beyond what they were designed to do, but at the same I 
> have to acknowledge that the time required to write traditional tests for 
> functions dealing with the business rules is also substantial — so maybe 
> the time required to write those generators is not abnormal.   
>
> Any suggestions are welcomed! I did not find any in-depth observations of 
> other developers about property-based testing on the Net.
>
> Maarten
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Spec & test.check generators for complex data structures

[Maarten Truyens]

Completely convinced of Spec, I am diving into property-based testing with 
test.check. However, I encounter some difficulties for which I would like 
to get the community's input. 

The central part of the software I am developing, consists of a tree of 
different kinds of nodes that can refer to — and interact with — each 
other. There are many business rules that specify how those nodes can 
interact — e.g., a node of type A can refer to other nodes of types B and C 
(but not D); or a node can have max. 3 child nodes of type E but more than 
3 child nodes of type F. 

It is trivial to have Spec generate completely random nodes to test 
functions that can ignore the business rules. However, other functions do 
depend on the business rules, and for those it is necessary to somewhat 
"guide" the generators towards random-but-compliant output. And here I'm 
stumbling: due to the business rules' complexity, this is not a matter of 
simply inserting a few such-that generators, so that I have already spent 
many days on writing a business rule compliant generator. Part of this is 
due to the multitude and complexity of the business rules, part due to the 
difficulty of writing generators (which operate at a higher level of 
abstraction, and for which only a subset of the normal Clojure constructs 
can be used).

How do others cope with this issue? I have the feeling that I am pushing 
the generators far beyond what they were designed to do, but at the same I 
have to acknowledge that the time required to write traditional tests for 
functions dealing with the business rules is also substantial — so maybe 
the time required to write those generators is not abnormal.   

Any suggestions are welcomed! I did not find any in-depth observations of 
other developers about property-based testing on the Net.

Maarten

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.