That last sentence should be: As the code is tried in further exploratory testing and in production new input parameter sets for which the given function does not meet the stated contract are discovered and added to the test case once a fix has been put into place.
On Tue, Mar 22, 2011 at 11:41 AM, Martin Logan <[email protected]> wrote: > For me unit testing is about contracts. I think about the same things > I think about when I write statements like {ok, Resp} = > Mod:Func(Args). Unit testing and writing specs are very close for me. > Hypothetically speaking lets say a function should return return {ok, > string()} | {error, term()} for all given input parameters then my > unit tests should be able to show that for a representative set of > input parameters that those contracts are honored. The art comes in > thinking about what that set is. > > The trap in writing all your own tests can often be that we think > about the set in terms of what we coded for and not what may indeed be > asked of our function. As the code is tried in further exploratory > testing and in production new input parameter sets that don't meet the > contract are discovered and added to the test case once a fix has been > put into place. > > Cheers, > Martin > > On Tue, Mar 22, 2011 at 8:04 AM, Torben Hoffmann > <[email protected]> wrote: >> Eric, >> >> And if you could send me some words about the thinking that you put into the >> creation of those unit tests it would make it so much evident where the two >> approaches are similar and where they differ. >> >> I can fabulate on the reasoning, but I'd rather have the thought process >> described by some one who has actually done a lot of unit tests. >> >> TO ALL: **** Anyone can chip in and describe the way they think when doing >> unit testing, it does not have to be Eric.*** >> >> It is not going to be used to point fingers, but to make it easy to >> understand what you have to do differently when changing to property based >> testing. >> >> I can promise you this: if you jump on the property based testing wagon you >> will experience fun with testing like never before. >> A new and better you will emerge on the other side. >> >> Cheers, >> Torben aka The ErlangPriest >> >> >> On Mon, Mar 21, 2011 at 23:59, Eric Merritt <[email protected]> wrote: >>> >>> Torben, >>> >>> I will put together an eunit test for add in ec_dictionary by >>> tomorrow. And then we can move from there. >>> >>> Eric >>> >>> On Mon, Mar 21, 2011 at 5:57 PM, Torben Hoffmann >>> <[email protected]> wrote: >>> > Hi again, >>> > >>> > Manolis helped me debug a very stupid thing on my side, so now we have a >>> > running property based test suite for ec_dictionary that tests >>> > ec_gb_trees >>> > for now. >>> > >>> > Action plan - not necessarily in strict order, but close and some items >>> > can >>> > be repeated as desired... >>> > >>> > receive a description of how to do a unit test of ec_dictionary:add/2. >>> > write a description of how a property based test of ec_dictionary:add/2 >>> > looks like and how it differs from unit testing. >>> > receive suggestions for new properties from the group - I will help turn >>> > them into correct properties. >>> > make it possible to test different ec_dictionary implementations. >>> > figure out (with the help of Manolis) how to use the Auto-ADT >>> > functionality. >>> > enhance the description with how to use Auto-ADT. >>> > support others in applying PropEr on selected modules in the Erlware >>> > suite. >>> > be happy. >>> > >>> > Cheers, >>> > Torben >>> > >>> > >>> > On Mon, Mar 21, 2011 at 20:57, Torben Hoffmann <[email protected]> >>> > wrote: >>> >> >>> >> >>> >> On Mon, Mar 21, 2011 at 16:11, Eric Merritt <[email protected]> >>> >> wrote: >>> >>> >>> >>> I went a bit nuts over the weekend and pulled Robert Virding's rbtree >>> >>> implementation and changed it to be a native implementation of our >>> >>> dictionary. I wanted something a bit more complex then the assoc list >>> >>> stuff. >>> >> >>> >> Walking on fire... but that is also where the fun is!! >>> >> >>> >>> >>> >>> Most of us don't have much experience with quick check, proper or >>> >>> property based testing. We are using this opportunity to go through >>> >>> and figure it all out so we can start using it. Is there any chance >>> >>> you could go through this first test and give some specific >>> >>> documentation of what is going on in each case? Then I will try to add >>> >>> some tests to it myself and see how it goes. >>> >> >>> >> I have given a bit of thought to this too, and I think that if I get >>> >> get a >>> >> good explanation of how one of you guys would test the >>> >> ec_dictionary:add/2 >>> >> function then I could explain what the differences are when doing it >>> >> with >>> >> property based testing. I think that would be a good context for >>> >> understanding how the two approaches differ and how property based >>> >> testing >>> >> provides more value. >>> >> >>> >> Does this make sense? >>> >> >>> >>> >>> >>> More inline: >>> >>> >>> >>> On Sun, Mar 20, 2011 at 10:53:04PM +0100, Torben Hoffmann wrote: >>> >>> > Hi, >>> >>> > >>> >>> > I have added more properties to test/ec_dictionary_proper - >>> >>> > >>> >>> > >>> >>> > [1]https://github.com/lehoff/erlware_commons/blob/master/test/ec_dictionary_proper.erl >>> >>> > >>> >>> > I would encourage you all to go a and look and them and send me >>> >>> > suggestions to new thing to be tested for a dictionary. >>> >>> > >>> >>> > I have run into a problem where it seems that PropEr is able to >>> >>> > create a >>> >>> > dictionary based on gb_trees where the same key is present more >>> >>> > than >>> >>> > once. >>> >>> > That is bad(tm). >>> >>> > Unfortunately I have not been able to work well enough with >>> >>> > PropEr >>> >>> > to get >>> >>> > a sequence of operations out that shows how it actually created >>> >>> > such >>> >>> > an >>> >>> > instance of gb_trees, but when I do I will explain the finding in >>> >>> > details. >>> >>> >>> >>> Our 'add' maps to the gb_trees enter function. According to the docs >>> >>> it should insert if the key doesn't exist or update if the key >>> >>> exists. Could this be a key equality problem. That is =:= vs ==, I >>> >>> suspect not since you are using integers for the keys but it doesn't >>> >>> hurt to ask the obvious questions. >>> >> >>> >> Hmmm, that is actually something that I did not pay attention to. It >>> >> won't >>> >> do any harm with the integers I am using now, but for others it could >>> >> be an >>> >> issue, so I will think about how this is tested the best. >>> >> >>> >> Cheers, >>> >> Torben >>> >> >>> >>> >>> >>> > The failing property is called prop_to_list_mathes_get() and it >>> >>> > tries to >>> >>> > compare the output of ec_dictionary:to_list to the output of >>> >>> > using >>> >>> > ec_dictionary:get/2 on all the keys in the to_list output. >>> >>> > They are not the same in some cases due to the gb_trees thing >>> >>> > above. >>> >>> > >>> >>> > Most likely I am doing something extremely stupid, but it could >>> >>> > be >>> >>> > fun if >>> >>> > there was a subtle bug in gb_trees... or maybe the right word is >>> >>> > scary... >>> >>> > >>> >>> > Cheers, >>> >>> > Torben >>> >>> > -- >>> >>> > [2]http://www.linkedin.com/in/torbenhoffmann >>> >>> > >>> >>> > References >>> >>> > >>> >>> > Visible links >>> >>> > 1. >>> >>> > >>> >>> > https://github.com/lehoff/erlware_commons/blob/master/test/ec_dictionary_proper.erl >>> >>> > 2. http://www.linkedin.com/in/torbenhoffmann >>> >>> >>> >>> -- >>> >>> Eric Merritt >>> >>> Erlang & OTP in Action (Manning) http://manning.com/logan >>> >>> http://twitter.com/ericbmerritt >>> >>> http://erlware.org >>> >>> >>> >>> -- >>> >>> You received this message because you are subscribed to the Google >>> >>> Groups >>> >>> "erlware-dev" group. >>> >>> To post to this group, send email to [email protected]. >>> >>> To unsubscribe from this group, send email to >>> >>> [email protected]. >>> >>> For more options, visit this group at >>> >>> http://groups.google.com/group/erlware-dev?hl=en. >>> >>> >>> >> >>> >> >>> >> >>> >> -- >>> >> http://www.linkedin.com/in/torbenhoffmann >>> > >>> > >>> > >>> > -- >>> > http://www.linkedin.com/in/torbenhoffmann >>> > >> >> >> >> -- >> http://www.linkedin.com/in/torbenhoffmann >> >> -- >> You received this message because you are subscribed to the Google Groups >> "erlware-dev" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/erlware-dev?hl=en. >> > > > > -- > Martin Logan > Erlang & OTP in Action (Manning) http://manning.com/logan > http://twitter.com/martinjlogan > http://erlware.org > -- Martin Logan Erlang & OTP in Action (Manning) http://manning.com/logan http://twitter.com/martinjlogan http://erlware.org -- You received this message because you are subscribed to the Google Groups "erlware-dev" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/erlware-dev?hl=en.
