Gentlemen, Between the two of you you have provided the perfect combination of explanations that will allow me to speak to and from your input and straight into property based testing Nirvana!
I am stuck with day job tasks tonight, so I will get back in the saddle tomorrow and I am so looking forward to completing this. A heads up: I talked to Kostis who is on the PropEr team and they are coming out with something this week that allows for stateful testing - I know it does not mean much to you, but that feature is so golden that you wouldn't believe it! When we are done with erlware_commons we can find a piece of code that is best tested with this approach and then we document how that is done. Cheers, Torben On Tue, Mar 22, 2011 at 17:43, Martin Logan <[email protected]> wrote: > That last sentence should be: > > As the code is tried in further exploratory testing and in production > new input parameter sets for which the given function does not meet > the stated contract are discovered and added to the test case once a > fix has been put into place. > > On Tue, Mar 22, 2011 at 11:41 AM, Martin Logan <[email protected]> > wrote: > > For me unit testing is about contracts. I think about the same things > > I think about when I write statements like {ok, Resp} = > > Mod:Func(Args). Unit testing and writing specs are very close for me. > > Hypothetically speaking lets say a function should return return {ok, > > string()} | {error, term()} for all given input parameters then my > > unit tests should be able to show that for a representative set of > > input parameters that those contracts are honored. The art comes in > > thinking about what that set is. > > > > The trap in writing all your own tests can often be that we think > > about the set in terms of what we coded for and not what may indeed be > > asked of our function. As the code is tried in further exploratory > > testing and in production new input parameter sets that don't meet the > > contract are discovered and added to the test case once a fix has been > > put into place. > > > > Cheers, > > Martin > > > > On Tue, Mar 22, 2011 at 8:04 AM, Torben Hoffmann > > <[email protected]> wrote: > >> Eric, > >> > >> And if you could send me some words about the thinking that you put into > the > >> creation of those unit tests it would make it so much evident where the > two > >> approaches are similar and where they differ. > >> > >> I can fabulate on the reasoning, but I'd rather have the thought process > >> described by some one who has actually done a lot of unit tests. > >> > >> TO ALL: **** Anyone can chip in and describe the way they think when > doing > >> unit testing, it does not have to be Eric.*** > >> > >> It is not going to be used to point fingers, but to make it easy to > >> understand what you have to do differently when changing to property > based > >> testing. > >> > >> I can promise you this: if you jump on the property based testing wagon > you > >> will experience fun with testing like never before. > >> A new and better you will emerge on the other side. > >> > >> Cheers, > >> Torben aka The ErlangPriest > >> > >> > >> On Mon, Mar 21, 2011 at 23:59, Eric Merritt <[email protected]> > wrote: > >>> > >>> Torben, > >>> > >>> I will put together an eunit test for add in ec_dictionary by > >>> tomorrow. And then we can move from there. > >>> > >>> Eric > >>> > >>> On Mon, Mar 21, 2011 at 5:57 PM, Torben Hoffmann > >>> <[email protected]> wrote: > >>> > Hi again, > >>> > > >>> > Manolis helped me debug a very stupid thing on my side, so now we > have a > >>> > running property based test suite for ec_dictionary that tests > >>> > ec_gb_trees > >>> > for now. > >>> > > >>> > Action plan - not necessarily in strict order, but close and some > items > >>> > can > >>> > be repeated as desired... > >>> > > >>> > receive a description of how to do a unit test of > ec_dictionary:add/2. > >>> > write a description of how a property based test of > ec_dictionary:add/2 > >>> > looks like and how it differs from unit testing. > >>> > receive suggestions for new properties from the group - I will help > turn > >>> > them into correct properties. > >>> > make it possible to test different ec_dictionary implementations. > >>> > figure out (with the help of Manolis) how to use the Auto-ADT > >>> > functionality. > >>> > enhance the description with how to use Auto-ADT. > >>> > support others in applying PropEr on selected modules in the Erlware > >>> > suite. > >>> > be happy. > >>> > > >>> > Cheers, > >>> > Torben > >>> > > >>> > > >>> > On Mon, Mar 21, 2011 at 20:57, Torben Hoffmann < > [email protected]> > >>> > wrote: > >>> >> > >>> >> > >>> >> On Mon, Mar 21, 2011 at 16:11, Eric Merritt <[email protected] > > > >>> >> wrote: > >>> >>> > >>> >>> I went a bit nuts over the weekend and pulled Robert Virding's > rbtree > >>> >>> implementation and changed it to be a native implementation of our > >>> >>> dictionary. I wanted something a bit more complex then the assoc > list > >>> >>> stuff. > >>> >> > >>> >> Walking on fire... but that is also where the fun is!! > >>> >> > >>> >>> > >>> >>> Most of us don't have much experience with quick check, proper or > >>> >>> property based testing. We are using this opportunity to go through > >>> >>> and figure it all out so we can start using it. Is there any chance > >>> >>> you could go through this first test and give some specific > >>> >>> documentation of what is going on in each case? Then I will try to > add > >>> >>> some tests to it myself and see how it goes. > >>> >> > >>> >> I have given a bit of thought to this too, and I think that if I get > >>> >> get a > >>> >> good explanation of how one of you guys would test the > >>> >> ec_dictionary:add/2 > >>> >> function then I could explain what the differences are when doing it > >>> >> with > >>> >> property based testing. I think that would be a good context for > >>> >> understanding how the two approaches differ and how property based > >>> >> testing > >>> >> provides more value. > >>> >> > >>> >> Does this make sense? > >>> >> > >>> >>> > >>> >>> More inline: > >>> >>> > >>> >>> On Sun, Mar 20, 2011 at 10:53:04PM +0100, Torben Hoffmann wrote: > >>> >>> > Hi, > >>> >>> > > >>> >>> > I have added more properties to test/ec_dictionary_proper - > >>> >>> > > >>> >>> > > >>> >>> > [1] > https://github.com/lehoff/erlware_commons/blob/master/test/ec_dictionary_proper.erl > >>> >>> > > >>> >>> > I would encourage you all to go a and look and them and send > me > >>> >>> > suggestions to new thing to be tested for a dictionary. > >>> >>> > > >>> >>> > I have run into a problem where it seems that PropEr is able > to > >>> >>> > create a > >>> >>> > dictionary based on gb_trees where the same key is present > more > >>> >>> > than > >>> >>> > once. > >>> >>> > That is bad(tm). > >>> >>> > Unfortunately I have not been able to work well enough with > >>> >>> > PropEr > >>> >>> > to get > >>> >>> > a sequence of operations out that shows how it actually > created > >>> >>> > such > >>> >>> > an > >>> >>> > instance of gb_trees, but when I do I will explain the finding > in > >>> >>> > details. > >>> >>> > >>> >>> Our 'add' maps to the gb_trees enter function. According to the > docs > >>> >>> it should insert if the key doesn't exist or update if the key > >>> >>> exists. Could this be a key equality problem. That is =:= vs ==, I > >>> >>> suspect not since you are using integers for the keys but it > doesn't > >>> >>> hurt to ask the obvious questions. > >>> >> > >>> >> Hmmm, that is actually something that I did not pay attention to. It > >>> >> won't > >>> >> do any harm with the integers I am using now, but for others it > could > >>> >> be an > >>> >> issue, so I will think about how this is tested the best. > >>> >> > >>> >> Cheers, > >>> >> Torben > >>> >> > >>> >>> > >>> >>> > The failing property is called prop_to_list_mathes_get() and > it > >>> >>> > tries to > >>> >>> > compare the output of ec_dictionary:to_list to the output of > >>> >>> > using > >>> >>> > ec_dictionary:get/2 on all the keys in the to_list output. > >>> >>> > They are not the same in some cases due to the gb_trees thing > >>> >>> > above. > >>> >>> > > >>> >>> > Most likely I am doing something extremely stupid, but it > could > >>> >>> > be > >>> >>> > fun if > >>> >>> > there was a subtle bug in gb_trees... or maybe the right word > is > >>> >>> > scary... > >>> >>> > > >>> >>> > Cheers, > >>> >>> > Torben > >>> >>> > -- > >>> >>> > [2]http://www.linkedin.com/in/torbenhoffmann > >>> >>> > > >>> >>> > References > >>> >>> > > >>> >>> > Visible links > >>> >>> > 1. > >>> >>> > > >>> >>> > > https://github.com/lehoff/erlware_commons/blob/master/test/ec_dictionary_proper.erl > >>> >>> > 2. http://www.linkedin.com/in/torbenhoffmann > >>> >>> > >>> >>> -- > >>> >>> Eric Merritt > >>> >>> Erlang & OTP in Action (Manning) http://manning.com/logan > >>> >>> http://twitter.com/ericbmerritt > >>> >>> http://erlware.org > >>> >>> > >>> >>> -- > >>> >>> You received this message because you are subscribed to the Google > >>> >>> Groups > >>> >>> "erlware-dev" group. > >>> >>> To post to this group, send email to [email protected]. > >>> >>> To unsubscribe from this group, send email to > >>> >>> [email protected]. > >>> >>> For more options, visit this group at > >>> >>> http://groups.google.com/group/erlware-dev?hl=en. > >>> >>> > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> http://www.linkedin.com/in/torbenhoffmann > >>> > > >>> > > >>> > > >>> > -- > >>> > http://www.linkedin.com/in/torbenhoffmann > >>> > > >> > >> > >> > >> -- > >> http://www.linkedin.com/in/torbenhoffmann > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "erlware-dev" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]. > >> For more options, visit this group at > >> http://groups.google.com/group/erlware-dev?hl=en. > >> > > > > > > > > -- > > Martin Logan > > Erlang & OTP in Action (Manning) http://manning.com/logan > > http://twitter.com/martinjlogan > > http://erlware.org > > > > > > -- > Martin Logan > Erlang & OTP in Action (Manning) http://manning.com/logan > http://twitter.com/martinjlogan > http://erlware.org > -- http://www.linkedin.com/in/torbenhoffmann -- You received this message because you are subscribed to the Google Groups "erlware-dev" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/erlware-dev?hl=en.
