I would certainly welcome and use an `overinstrument` function if added to schpec (or another library).
One reason spec is great because it provides "a la carte" validation: a can add an appropriate amount of validation to the parts of my code that would most benefit from it. In my experience using spec so far, optionally running `fn` and `ret` specs during instrumentation would give me more options to pick the correct amount of validation. For instance, since side-effecting functions are harder to unit test (and difficult to test generatively), I would like to spec the functions and then run integration tests. Running `ret`/`fn` specs would help me detect errors closer to their source and also confirm the specs are accurate, which improves the value specs as documentation. In my current project, I have been avoiding writing `ret` specs for side-effecting functions because I worry they'll get out-of-sync and mislead other programmers. Also, having `ret`/`fn` specs checked during instrumentation would add value during development when I'm not yet ready to build generative tests. For example, during early development of a function, I might just spec a parameter with `map?` even though it actually requires certain keys. That spec far too vague for generative testing, but immediately adds value when I turn on instrumentation, since it catches bugs where I've passed, say, `nil` as the parameter. When the feature is more stable, I can invest the time in creating a generator and running generative tests. With the current implementation of `instrument`, I find that I've been tempted to over-spec early in development in order to get working generative tests that will confirm my `ret` and `fn` specs are correct. I understand Rich and Alex have thought all this through and have provided a detailed rationale for the current behavior. But in my own workflow, being able to turn on `ret`/`fn` specs during instrumentation would be a big help, and I'd very much like to see this feature added to schpec. I'm happy to look at this when my schedule allows (not this week or next due to travel). Perhaps the best approach is to add an `over-instrument` function that has options for turning on/off `args`, `fn`, and `ret` specs (enabling all by default) and see how people use it in practice? Thanks to everyone who have worked so hard on all aspects of spec! Ben On Tuesday, December 6, 2016 at 4:42:44 PM UTC-7, Brandon Bloom wrote: > > I was just very surprised to discover my :ret specs were not being checked > via instrument. I've read the rationale above, but I'm not yet convinced. > While of course I can (and will) use spec/assert from a post condition, I > lose the nice selective instrumentation features. > > I'd also like to make a counter-point to this reasoning: > > >> Running return-value instrument-style checking on whatever few >> hand-written tests you might have isn’t going to give you better coverage >> than a simple (even hardwired) generator that captures similar ranges. > > > In my case, I was already testing the root-level return value, but this > function is deeply recursive. A small non-conforming value deep in the > evaluation stack was yielding non-conformance at a path deep in the root > return value. I was confused by why the :ret check wasn't finding it > earlier. By relying on the top-level check rather than inline checks > per-call (as per instrument or :pre/:post) I lost access to the call stack. > Adding the post condition pin-pointed the problem instantly. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
