Hi Chris and Paul,
Let me please answer your messages in one go as they are related.
Paul wrote:
> For most single use (or infrequently used) functions, I'd argue that the
> trade-off *isn't* worth it.
>
> Here's a quick example from the pip codebase:
>
> # Retry every half second for up to 3 seconds
> @retry(stop_max_delay=3000, wait_fixed=500)
> def rmtree(dir, ignore_errors=False):
> shutil.rmtree(dir, ignore_errors=ignore_errors,
> onerror=rmtree_errorhandler)
Absolutely, I agree. If it's a single-use or infrequently used function
that hardly anybody uses, it's not worth it. Moreover, if some contracts
are harder to figure out than the implementation, then they are probably in
most cases not worth the effort, too.
Please mind that I said: any *library* would benefit from it, as in the
users of any library on pipy would benefit from better, formal and more
precise documentation. That doesn't imply that all the contracts need to be
specified or that you have to specify the contract for *every *function, or
that you omit the documentation altogether. Some contracts are simply too
hard to get explicitly. Some are not meaningful even if you could write
them down. Some you'd like to add, but run only at test time since too slow
in production. Some run in production, but are not included in the
documentation (*e.g., *to prevent the system to enter a bad state though it
is not meaningful for the user to actually *read* that contract).
Since contracts are formally written, they can be verified. Human text *can
not*. Specifying all the contracts is in most cases *not *meaningful. In my
day-to-day programming, I specify contracts on the fly and they help me
express formally to the next girl/guy who will use my code what to expect
or what not. That does not mean that s/he can just skip the documentation
or that contracts describe fully what the function does. They merely help
-- help him/her what arguments and results are expected. That *does not
mean *that I fully specified all the predicates on the arguments and the
result. It's merely a help à la
* "Ah, this argument needs to be bigger than that argument!"
* "Ah, the resulting dictionary is shorter than the input list!"
* "Ah, the result does not include the input list"
* "Ah, this function accepts only files (no directories) and relative
paths!"
* "Ah, I put the bounding box outside of the image -- something went wrong!"
* "Ah, this method allows me to put the bounding box outside of the image
and will fill all the outside pixels with black!"
*etc.*
For example, if I have an object detector operating on a region-of-interest
and returning bounding boxes of the objects, the postconditions will not
be: "all the bounding boxes are cars", that would impossible. But the
postcondition might check that all the bounding boxes are within the
region-of-interest or slightly outside, but not completely outside *etc.*
Let's be careful not to make a straw-man here, *i.e. *to push DbC *ad
absurdum *and then discard it that way.
Also, the implicit contracts code currently has are typically pretty loose.
> What you need to "figure out" is very general. Explicit contracts are
> typically demonstrated as being relatively strict, and figuring out and
> writing such contracts is more work than writing code with loose implicit
> contracts. Whether the trade-off of defining tight explicit contracts once
> vs inferring a loose implicit contract every time you call the function is
> worth it, depends on how often the function is called. For most single use
> (or infrequently used) functions, I'd argue that the trade-off *isn't*
> worth it.
I don't believe such an approach would ever be pragmatical, *i.e. *automatic
versioning based on the contracts. It might hint it (as a warning: you
changed a contract, but did not bump the version), but relying solely on
this mechanism to get the versioning would imply that you specified *all *the
contracts. Though Bertrand might have always envisioned it as the best
state of the world, even he himself repeatedly said that it's better to
specify rather 10% than 0% contracts and 20% rather than 10% contracts.
Chris wrote:
> def fibber(n):
> return n if n < 2 else fibber(n-1) + fibber(n-2)
It depends who you are writing this function for -- for example, instead of
the contract, why not just include the code implementation as the
documentation? The only *meaningful* contract I could imagine:
@pre n >= 0
Everything else would be just repetition of the function. If you
implemented some optimized and complex fibber_optimized() function, then
your contracts would probably look like:
@pre n >= 0
@post (n < 2 ) or result == fibber_optimized(n-1) + fibber_optimized(n-2)
@post not (n < 2) or result == n
> Here is my try at the contracts. Assuming that there is a list of figures
> and that they have a property "displayed" and that "State.blocked" global
> variable refers to whether the interface is blocked or not::
> > @post(lambda: all(figure.displayed for figure in figures)
> > @post(lambda: not ipython.in_pylab_mode() or not State.blocked)
> > @post(lambda: not interactive() or State.blocked)
> > matplotlib.pyplot.show()
> >
>
> There is no such thing as "State.blocked". It blocks. The function
> *does not return* until the figure has been displayed, and dismissed.
> There's no way to recognize that inside the function's state.
>
> Contracts are great when every function is 100% deterministic and can
> maintain invariants and/or transform from one set of invariants to
> another. Contracts are far less clear when the definitions are
> muddier.
>
Sorry, it has been ages that I used matplotlib. I thought it meant
"blocking" as in "the drawing thread blocks". Since blocking basically
means halting the execution-- then the contracts can't help. They are
checked *before *and *after *the function executes. They can not solve the
halting problem. For that you need formal methods (and I doubt that formal
methods would ever work for matplotlib). The contracts *do not check *what
happens *during *the execution of the function. They are not meant to do
that. Even the invariants of the class are checked *before *and *after the
call to public methods *(and the private methods are allowed to break
them!).
Please mind that this is actually not the argument pro/against the
contracts -- you are discussing in this particular case a tool (different
to DbC) which should test the behavior *during the execution *of a function.
Chirs wrote:
> > However, contracts can be useful when testing the GUI -- often it is
> difficult to script the user behavior. What many people do is record a
> session and re-play it. If there is a bug, fix it. Then re-record. While
> writing unit tests for GUI is hard since GUI changes rapidly during
> development and scripting formally the user behavior is tedious, DbC might
> be an alternative where you specify as much as you can, and then just
> re-run through the session. This implies, of course, a human tester.
> >
>
> That doesn't sound like the function's contract. That sounds like a
> test case - of which you would have multiple, using different
> "scripted session" inputs and different outputs (some of success, some
> of expected failure).
>
Well, yes, you can view it as a testing technique; it assumes that
scripting a session is often difficult for GUIs and sometimes harder (since
combinatorial) than the implementation itself. What I saw people do is
write the contracts, put the program in debug mode and let the human tester
test it. Think of it as a random test where only checks are your
pre/postconditions. The human annotator mimics a meaningful random walk and
uses the application as s/he sees fit. I'm not championing this approach,
just noting it here as a potential use case.
There's certainly benefits for the "contracts as additional tests"
> viewpoint. But whenever that's proposed as what people understand by
> DbC, the response is "no, it's not like that at all". So going back to
> the "why isn't DbC more popular" question - because no-one can get a
> clear handle on whether they are "like tests" or "like assertions" or
> "something else" :-)
I think that it's a tool that you can use for many things:
* verifiable documentation
* deeper testing (every test case tests also other parts of the system,
akin to asserts)
* automatic test generation
* hand-break akin to asserts
I find the first one, verifiable documentation, to be the most useful one
in working with my team at the moment.
Cheers,
Marko
On Wed, 26 Sep 2018 at 10:59, Paul Moore <p.f.mo...@gmail.com> wrote:
> On Wed, 26 Sep 2018 at 09:45, Chris Angelico <ros...@gmail.com> wrote:
> >
> > On Wed, Sep 26, 2018 at 6:36 PM Paul Moore <p.f.mo...@gmail.com> wrote:
> > >
> > > On Wed, 26 Sep 2018 at 06:41, Chris Angelico <ros...@gmail.com> wrote:
> > > >
> > > > On Wed, Sep 26, 2018 at 2:47 PM Marko Ristin-Kaufmann
> > > > <marko.ris...@gmail.com> wrote:
> > > > > * The contracts written in documentation as human text inevitably
> rot and they are much harder to maintain than automatically verified formal
> contracts.
> > > >
> > > > Agreed.
> > >
> > > Agreed, if contracts are automatically verified. But when runtime cost
> > > comes up, people suggest that contracts can be disabled in production
> > > code - which invalidates the "automatically verified" premise.
> >
> > Even if they're only verified as a dedicated testing pass ("okay,
> > let's run the unit tests, let's run the contract verifier, let's run
> > the type checker, cool, we're good"), they're still better off than
> > unchecked comments/documentation in terms of code rot.
>
> Absolutely. But if the contracts are checked at runtime, they are
> precisely as good as tests - they will flag violations *in any
> circumstances you check*. That's great, but nothing new. I understood
> that one of the benefits of contracts was that it would handle cases
> that you *forgot* to test - like assertions do, in essence - and would
> need to be active in production (again, like assertions, if we assume
> we've not explicitly chosen to run with assertions disabled) to get
> that benefit.
>
> There's certainly benefits for the "contracts as additional tests"
> viewpoint. But whenever that's proposed as what people understand by
> DbC, the response is "no, it's not like that at all". So going back to
> the "why isn't DbC more popular" question - because no-one can get a
> clear handle on whether they are "like tests" or "like assertions" or
> "something else" :-)
>
> Paul
> _______________________________________________
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
