Re: [Python-ideas] "old" values in postconditions
Hi James,
I'm a bit short on time today, and would need some more time and attention
to understand the proposal you wrote. I'll try to come back to you
tomorrow.
In any case, I need to refactor icontract's decorators to use conditions
like lambda P: and lambda P, result: first before adding snapshot
functionality.
What about having @snapshot_with and @snapshot? @Snapshot_with does what
you propose and @snapshot expects a lambda P, identifier: ?
After the refactoring, maybe the same could be done for defining contracts
as well? (Requires and requires_that?)
If the documentation is clear, I'd expect the user to be able to
distinguish the two. The first approach is shorter, and uses magic, but
fails in some rare situations. The other method is more verbose, but always
works.
Cheers,
Marko
Le sam. 29 sept. 2018 à 00:35, James Lu a écrit :
> I am fine with your proposed syntax. It’s certainly lucid. Perhaps it
> would be a good idea to get people accustomed to “non-magic” syntax.
>
> I still have a feeling that most developers would like to store the state
> in many different custom ways.
>
> Please explain. (Expressions like thunk(all)(a == b for a, b in
> P.arg.meth()) would be valid.)
>
> I'm thinking mostly about all the edge cases which we would not be able to
> cover (and how complex that would be to cover them).
>
>
> Except for a > b > c being one flat expression with 5 members, it seems
> fairly easy to recreate an AST, which can then be compiled down to a code
> object. The code object can be fun with a custom “locals()”
>
> Below is my concept code for such a P object.
>
> from ast import *
>
> # not done: enforce Singleton property on EmptySymbolType
>
> class EmptySymbolType(object): ...
>
> EmptySymbol = EmptySymbolType() # empty symbols are placeholders
>
> class MockP(object):
>
> # "^" is xor
>
> @icontract.pre(lambda symbol, astnode: (symbol is None) ^ (astnode is
> None))
>
> def __init__(self, symbol=None, value=EmptySymbol, astnode=None,
> initsymtable=(,)):
>
> self.symtable = dict(initsymtable)
>
> if symbol:
>
> self.expr = Expr(value=Name(id=symbol, ctx=Load()))
>
> self.symtable = {symbol: value}
>
> else:
>
> self.expr = astnode
>
> self.frozen = False
>
> def __add__(self, other):
>
> wrapped = MockP.wrap_value(other)
>
> return MockP(astnode=Expr(value=BinOp(self.expr, Add(),
> wrapped.expr), initsymtable={**self.symtable, **wrapped.symtable})
>
> def compile(self): ...
>
> def freeze(self):
>
> # frozen objects wouldn’t have an overrided getattr, allowing for
> icontract to manipulate the MockP object using its public interface
>
> self.frozen = True
>
> @classmethod
>
> def wrap_value(cls, obj):
>
># create a MockP object from a value. Generate a random identifier
> and set that as the key in symtable, the AST node is the name of that
> identifier, retrieving its value through simple expression evaluation.
>
>...
>
>
> thunk = MockP.wrap_value
>
> P = MockP('P')
>
> # elsewhere: ensure P is only accessed via valid “dot attribute access”
> inside @snapshot so contracts fail early, or don’t and allow Magic like
> __dict__ to occur on P.
>
> On Sep 27, 2018, at 9:49 PM, Marko Ristin-Kaufmann
> wrote:
>
> Hi James,
>
> I still have a feeling that most developers would like to store the state
> in many different custom ways. I see also thunk and snapshot with wrapper
> objects to be much more complicated to implement and maintain; I'm thinking
> mostly about all the edge cases which we would not be able to cover (and
> how complex that would be to cover them). Then the linters need also to
> work around such wrappers... It might also scare users off since it looks
> like too much magic. Another concern I also have is that it's probably very
> hard to integrate these wrappers with mypy later -- but I don't really have
> a clue about that, only my gut feeling?
>
> What about we accepted to repeat "lambda P, " prefix, and have something
> like this:
>
> @snapshot(
> lambda P, some_name: len(P.some_property),
> lambda P, another_name: hash(P.another_property)
> )
>
> It's not too verbose for me and you can still explain in three-four
> sentences what happens below the hub in the library's docs. A
> pycharm/pydev/vim/emacs plugins could hide the verbose parts.
>
> I performed a small experiment to test how this solution plays with pylint
> and it seems OK that arguments are not used in lambdas.
>
> Cheers,
> Marko
>
>
> On Thu, 27 Sep 2018 at 12:27, James Lu wrote:
>
>> Why couldn’t we record the operations done to a special object and
>> replay them?
>>
>> Actually, I think there is probably no way around a decorator that
>>> captures/snapshots the data before the function call with a lambda (or even
>>> a separate function). "Old" construct, if we are to parse it somehow from
>>> the condition function, would limit us only to
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Sat, Sep 29, 2018 at 12:30:45PM +1200, Greg Ewing wrote:
> Chris Angelico wrote:
> >But as a part of the
> >function's declared intent, it's fine to have a postcondition of "the
> >directory will be empty" even though something could drop a file into
> >it.
>
> If you only intend the contract to serve as documentation,
> I suppose that's okay, but it means you can't turn on runtime
> checking of contracts, otherwise your program could suffer
> spurious failures.
If your code can cope with a particular file system state ("directory
isn't empty") then you don't need to specify it as a precondition. If it
can't cope, then it isn't a spurious failure, its a real failure. You
either get an error when the condition is checked, up front, or you get
an error in the middle of processing the directory.
Of course for many things (especially file system operations) you need
to be prepared to handle I/O errors *even if the precondition passed*.
So what? That hasn't changed, and nobody said that contracts were a cure
for Time Of Check To Time Of Use bugs.
Contracts are a tool to help developers write better code, not a magic
wand. You still need to write your code in a way which isn't vulnerable
to TOCTTOU failures, contracts or not.
Anticipating an objection: why bother with the precondition check if the
code has to handle an error anyway?
Because it is better to get an upfront error than an error halfway
through processing. In general you get a better, more informative error
message, closer to the place where it matters, if you fail fast.
Yes yes yes, in theory you might have a precondition requirement which
would fail up front but resolve itself before it matters:
directory not empty
start running your program
precondition check fails
... later
directory is emptied by another process
your program actually needs to use the empty directory
but do you really think it is good practice to design your code on the
basis of that? Better to write your code conservatively: if the
directory isn't empty up front, don't hope it will empty itself, fail
fast, but be prepared to handle the error case as well.
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] "while:" for the loop
I put the list of related discussion here, just in case. Same suggestion: https://mail.python.org/pipermail/python-dev/2005-July/054914.html Idea for the "loop" keyword: https://mail.python.org/pipermail/python-ideas/2014-June/028202.html (followed by the same suggestion from @Random832: https://mail.python.org/pipermail/python-ideas/2014-June/028220.html) Somewhat related PEP (rejected) + discussion links inside: https://legacy.python.org/dev/peps/pep-0315/ (I've meditated a bit on this - but could not get what was actually the point of that idea.) Plus some other related discussions, maybe not directly related, but something along the lines might be interesting. An old and lively one: https://mail.python.org/pipermail/python-list/1999-April/002557.html https://mail.python.org/pipermail/python-list/1999-May/008535.html Another one: https://mail.python.org/pipermail/python-list/2000-December/029972.html ( Particularly this: https://mail.python.org/pipermail/python-list/2000-December/052694.html ) Yet another "1" vs "True" debate: https://mail.python.org/pipermail/python-list/2012-January/618649.html New era: https://mail.python.org/pipermail/python-list/2017-April/721182.html ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Sun, Sep 23, 2018 at 07:09:37AM +0200, Marko Ristin-Kaufmann wrote: > After the discussion we had on the list and after browsing the internet a > bit, I'm still puzzled why design-by-contract was not more widely adopted > and why so few languages support it. [...] > *. *After properly reading about design-by-contract and getting deeper into > the topic, there is no rational argument against it and the benefits are > obvious. And still, people just wave their hand and continue without > formalizing the contracts in the code and keep on writing them in the > descriptions. > > * Why is that so? [...] You are asking a question about human psychology but expecting logical, technical answers. I think that's doomed to failure. There is no nice way to say this, because it isn't nice. Programmers and language designers don't use DbC because it is new and different and therefore scary and wrong. Programmers, as a class, are lazy (they even call laziness a virtue), deeply conservative, superstitious, and don't like change. They do what they do because they're used to it, not because it is the technically correct thing to do, unless it is by accident. (Before people's hackles raise too much, I'm speaking in generalities, not about you personally. Of course you are one of the 1% awesomely logical, technically correct programmers who know what you are doing and do it for the right reasons after careful analysis. I'm talking about the other 99%, you know the ones. You probably work with them. You've certainly read their answers on Stackoverflow or The Daily WTF and a million blogs.) They won't use it until there is a critical mass of people using it, and then it will suddenly flip from "that weird shit Eiffel does" to "oh yeah, this is a standard technique that everyone uses, although we don't make a big deal about it". Every innovation in programming goes through this. Whether the innovation goes mainstream or not depends on getting a critical mass, and that is all about psychology and network effects and nothing to do with the merit of the idea. Remember the wars over structured programming? Probably not. In 2018, the idea that people would *seriously argue against writing subroutines* seems insane, but they did. As late as 1999, a former acquaintance of mine was working on a BASIC project for a manager who insisted they use GOTO and GOSUB in preference to subroutines. Testing: the idea that we should have *repeatable automated tests* took a long time to be accepted, and is still resisted by both developers and their managers. What's wrong with just sitting a person down in front of the program and checking for bugs by hand? We still sometimes have to fight for an automated test suite, never mind something like test driven development. ML-based languages have had type inference for decades, and yet people still think of type checking in terms of C and Java declarations. Or they still think in terms of static VERSUS dynamic typing, instead of static PLUS dynamic typing. I could go on, but I think I've made my point. I can give you some technical reasons why I don't use contracts in my own Python code, even though I want to: (1) Many of my programs are too small to bother. If I'm writing a quick script, I don't even write tests. Sometimes "be lazy" is the right answer, when the cost of bugs is small enough and the effort to avoid them is greater. And that's fine. Nobody says that contracts must be mandatory. (2) Python doesn't make it easy to write contracts. None of the solutions I've seen are nice. Ironically, the least worst I've seen is a quick and dirty metaclass solution given by Guido in an essay way back in Python 1.5 days: https://www.python.org/doc/essays/metaclasses/ His solution relies only on a naming convention, no decorators, no lambdas: class C(Eiffel): def method(self, arg): return whatever def method_pre(self, arg): assert arg > 0 def method_post(self, Result, arg): assert Result > arg Still not pretty, but at least we get some block structure instead of a wall of decorators. Syntax matters. Without good syntax that makes it easy to write contracts, it will never be anything but niche. (3) In a sense, *of course I write contracts*. Or at least precondition checks. I just don't call them that, and I embed them in the implementation of the method, and have no way to turn them off. Nearly all of us have done the same, we start with a method like this: class C: def method(self, alist, astring, afloat): # do some work... using nothing but naming conventions and an informal (and often vague) specification in the docstring, and while that is sometimes enough in small projects, the third time we get bitten we start adding defensive checks so we get sensible exceptions: def method(self, alist, astring, afloat): if not isinstance(alist, list): raise
Re: [Python-ideas] Why is design-by-contracts not widely
On 9/28/18 8:32 PM, Chris Angelico wrote: On Sat, Sep 29, 2018 at 10:22 AM Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> wrote: On 9/28/18 7:39 PM, Steven D'Aprano wrote: But none of that compares to C undefined behaviour. People who think that they are equivalent, don't understand C undefined behaviour. Well, yes: Some syntactically legal C results in nasal demons, and some of that code is harder to spot than others. AFAIK, syntactically legal Python can only do that if the underlying C code invokes undefined behaviour. What should happen here? [examples of what Steven would call non-sensible, non-non-weird objects doing non-non-weird things snipped] AFAIK, "AFAIK" is a weasel word: It allows me to proclaim my own ignorance without providing further examples, evidence, or counter examples. :-) Python has its own set of "well don't do that then" situations. In fact, I would say that *most* languages do. Yep. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On 9/28/18 8:12 PM, Chris Angelico wrote: On Sat, Sep 29, 2018 at 7:19 AM Greg Ewing wrote: Chris Angelico wrote: It is still fundamentally difficult to make assertions about the file system as pre/post contracts. When you consider race conditions, I'd say it's impossible. A postcondition check involving the file system could fail even if the function does its job perfectly. I guess that depends on the meaning of "contract". If it means "there is a guarantee that this is true after this function returns", then yes, the race condition means it's impossible. But as a part of the function's declared intent, it's fine to have a postcondition of "the directory will be empty" even though something could drop a file into it. But if it's the latter... contracts are just a different way of defining unit tests and we're right back where we started. At the risk of pedantry (on a Python list? I'm *shocked*): I call BS on any contract that requires on entry or guarantees on exit the state of the file system. At best, a function can guarantee that it will make (or made) a request to the OS, and that the OS returned "success" before the function continued. Then again, a function that guarantees to work in a particular way based on some condition of the file system would be okay. For example, a function might claim to create a temporary file with some content *if* some directory exists when the function tries to create the temporary file. But as I think both of you are claiming, the best that function can guarantee on exit is that it asked the OS to write to the file, and that the OS agreed to do so. The function cannot guarantee that the file or the content still exists when the function finally returns. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely
On Sat, Sep 29, 2018 at 10:22 AM Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> wrote: > > On 9/28/18 7:39 PM, Steven D'Aprano wrote: > > But none of that compares to C undefined behaviour. People who think > > that they are equivalent, don't understand C undefined behaviour. > > Well, yes: Some syntactically legal C results in nasal demons, and some > of that code is harder to spot than others. AFAIK, syntactically legal > Python can only do that if the underlying C code invokes undefined > behaviour. What should happen here? >>> import ctypes >>> ctypes.cast(id(1), ctypes.POINTER(ctypes.c_int))[6] = 0 >>> 1 Nothing here invokes C's undefined behaviour. Or what about here: >>> import sys; sys.setrecursionlimit(2147483647) >>> def f(): f() ... >>> f() Python has its own set of "well don't do that then" situations. In fact, I would say that *most* languages do. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
Chris Angelico wrote: But as a part of the function's declared intent, it's fine to have a postcondition of "the directory will be empty" even though something could drop a file into it. If you only intend the contract to serve as documentation, I suppose that's okay, but it means you can't turn on runtime checking of contracts, otherwise your program could suffer spurious failures. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely
On 9/28/18 7:39 PM, Steven D'Aprano wrote: On Fri, Sep 28, 2018 at 07:18:54PM +0200, 2qdxy4rzwzuui...@potatochowder.com wrote: On 9/28/18 12:45 PM, Steven D'Aprano wrote: On Tue, Sep 25, 2018 at 09:59:53PM +1000, Hugh Fisher wrote: C and Python (currently) are known as simple languages. o_O That's a usage of "simple" I haven't come across before. Especially in the case of C, which is a minefield of *intentionally* underspecified behaviour which makes it near to impossible for the developer to tell what a piece of syntactically legal C code will actually do in practice. s/C/Python/ s/underspecified/dynamic/ ;-) I see the wink, but I don't see the relevance. Are you agreeing with me or disagreeing? I agree that Hugh's use of "simple" is unfamiliar. I disagree that C is is a bigger offender than Python when it comes to a developer telling what a piece of syntactically legal code will actually do in practice. If that's not what you meant by "Especially in the case of C...," then I mis-interpreted or read too much into your wording. Python is "simple" in the sense that the execution model is *relatively* simple, but its not a minimalist language by any definition. And as you say, the execution model is dynamic: we can't be sure what legal code will do until you know the runtime state. That's my point: What you emphasized about C can be applied equally to Python. In C, it's because the standard is intentionally underspecified; in Python, it's y because the language is intentionally dynamic. When you said "underspecified," I didn't make the leap to "undefined behaviour" (although I think I know from past experience how you feel about C and its undefined behaviour). Instead, I jumped to things like the ambiguity in the size of an int, or the freedom the compiler has to pack/align struct values or implement integers as something other than two's complement. (Although we can often guess, based on assumptions about sensible, non-weird objects that don't do weird things.) Again, the same is true of C. In Python, weird objects might override getattr; in C weird objects might point to hardware registers, or depend on implementation specific detail(s). But none of that compares to C undefined behaviour. People who think that they are equivalent, don't understand C undefined behaviour. Well, yes: Some syntactically legal C results in nasal demons, and some of that code is harder to spot than others. AFAIK, syntactically legal Python can only do that if the underlying C code invokes undefined behaviour. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Sat, Sep 29, 2018 at 7:19 AM Greg Ewing wrote: > > Chris Angelico wrote: > > It is still fundamentally difficult to make assertions about the file > > system as pre/post contracts. > > When you consider race conditions, I'd say it's impossible. > A postcondition check involving the file system could fail > even if the function does its job perfectly. I guess that depends on the meaning of "contract". If it means "there is a guarantee that this is true after this function returns", then yes, the race condition means it's impossible. But as a part of the function's declared intent, it's fine to have a postcondition of "the directory will be empty" even though something could drop a file into it. But if it's the latter... contracts are just a different way of defining unit tests and we're right back where we started. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Simplicity of C (was why is design-by-contracts not widely)
> Date: Sat, 29 Sep 2018 02:45:06 +1000 > From: Steven D'Aprano > To: python-ideas@python.org > Subject: Re: [Python-ideas] Why is design-by-contracts not widely > Message-ID: <20180928164506.gn19...@ando.pearwood.info> > Content-Type: text/plain; charset=us-ascii > > On Tue, Sep 25, 2018 at 09:59:53PM +1000, Hugh Fisher wrote: > > > C and Python (currently) are known as simple languages. > > o_O > > That's a usage of "simple" I haven't come across before. Especially in > the case of C, which is a minefield of *intentionally* underspecified > behaviour which makes it near to impossible for the developer to tell > what a piece of syntactically legal C code will actually do in practice. Oh FFS. You couldn't make the effort to read the very next sentence, let alone the next paragraph, before responding? -- cheers, Hugh Fisher ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely
On Fri, Sep 28, 2018 at 07:18:54PM +0200, 2qdxy4rzwzuui...@potatochowder.com wrote: > > On 9/28/18 12:45 PM, Steven D'Aprano wrote: > >On Tue, Sep 25, 2018 at 09:59:53PM +1000, Hugh Fisher wrote: > > > >>C and Python (currently) are known as simple languages. > > > >o_O > > > >That's a usage of "simple" I haven't come across before. Especially in > >the case of C, which is a minefield of *intentionally* underspecified > >behaviour which makes it near to impossible for the developer to tell > >what a piece of syntactically legal C code will actually do in practice. > > s/C/Python/ > > s/underspecified/dynamic/ > > ;-) I see the wink, but I don't see the relevance. Are you agreeing with me or disagreeing? Python is "simple" in the sense that the execution model is *relatively* simple, but its not a minimalist language by any definition. And as you say, the execution model is dynamic: we can't be sure what legal code will do until you know the runtime state. (Although we can often guess, based on assumptions about sensible, non-weird objects that don't do weird things.) But none of that compares to C undefined behaviour. People who think that they are equivalent, don't understand C undefined behaviour. https://blog.regehr.org/archives/213 http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_14.html -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
Many editors highlight decorators in a different color that makes it easier to ignore and can also fold decorators. Contracts can also sometimes actively improve the flow of code. I personally find a formal contract easier to read than informal documentation. It also reduces the times where you need to spend time figuring out if documentation actually accurate and up to date ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Exception handling in contracts
Let’s get some ideas for how icontract can say “it should throw an exception if this happens.” ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] "old" values in postconditions
> The problem with readability might be easier to solve than I thought, and > your pointer to coconut gave me the idea. What if we make a utility that > takes the python source code, examines the decorators pre/post/inv (or > whatever we call them) and transforms them back and forth from/to valid > python code? > > Pretty much any IDE has after load / before save handlers. When you load a > python file, you'd transform it from less readable python code, write using a > concise form and when you save it, it's transformed back to valid python. Hmm yes. Look at my previous email- the proposed syntax there does not require changing or transforming Python code. It’s missing many magic methods, but it would work something like this: @snapshot(some_identifier=P.self.another_property + 10) would internally turn into something like this: lambda P: P.self.another_property + 10 Maybe it’s possible to modify PyCharm parsing by altering its Python grammar file? Sent from my iPhone > On Sep 28, 2018, at 10:27 AM, Marko Ristin-Kaufmann > wrote: > > Hi James, > > The problem with readability might be easier to solve than I thought, and > your pointer to coconut gave me the idea. What if we make a utility that > takes the python source code, examines the decorators pre/post/inv (or > whatever we call them) and transforms them back and forth from/to valid > python code? > > Pretty much any IDE has after load / before save handlers. When you load a > python file, you'd transform it from less readable python code, write using a > concise form and when you save it, it's transformed back to valid python. > > Then we need to pick the python form that is easiest to implement (and still > readable enough for, say, code reviews on github), but writing and reading > the contracts in the code would be much more pleasant. > > As long as the "readable" form has also valid python syntax, the tool can be > implemented with ast module. > > For example: > @snapshot(some_identifier=self.another_property + 10) > @post(self.some_property => old.some_identifier > 100) > > would transform into > @snapshot(lambda P, some_identifier: P.self.another_property + 10) > @post(lambda O, P: not self.some_property and O.some_identifier > 100) > > Cheers, > Marko > >> On Fri, 28 Sep 2018 at 03:49, Marko Ristin-Kaufmann >> wrote: >> Hi James, >> >> I still have a feeling that most developers would like to store the state in >> many different custom ways. I see also thunk and snapshot with wrapper >> objects to be much more complicated to implement and maintain; I'm thinking >> mostly about all the edge cases which we would not be able to cover (and how >> complex that would be to cover them). Then the linters need also to work >> around such wrappers... It might also scare users off since it looks like >> too much magic. Another concern I also have is that it's probably very hard >> to integrate these wrappers with mypy later -- but I don't really have a >> clue about that, only my gut feeling? >> >> What about we accepted to repeat "lambda P, " prefix, and have something >> like this: >> >> @snapshot( >> lambda P, some_name: len(P.some_property), >> lambda P, another_name: hash(P.another_property) >> ) >> >> It's not too verbose for me and you can still explain in three-four >> sentences what happens below the hub in the library's docs. A >> pycharm/pydev/vim/emacs plugins could hide the verbose parts. >> >> I performed a small experiment to test how this solution plays with pylint >> and it seems OK that arguments are not used in lambdas. >> >> Cheers, >> Marko >> >> >>> On Thu, 27 Sep 2018 at 12:27, James Lu wrote: >>> Why couldn’t we record the operations done to a special object and replay >>> them? >>> >> Actually, I think there is probably no way around a decorator that >> captures/snapshots the data before the function call with a lambda (or >> even a separate function). "Old" construct, if we are to parse it >> somehow from the condition function, would limit us only to shallow >> copies (and be complex to implement as soon as we are capturing >> out-of-argument values such as globals etc.). Moreove, what if we don't >> need shallow copies? I could imagine a dozen of cases where shallow copy >> is not what the programmer wants: for example, s/he might need to make >> deep copies, hash or otherwise transform the input data to hold only >> part of it instead of copying (e.g., so as to allow equality check >> without a double copy of the data, or capture only the value of certain >> property transformed in some way). >>> >>> >>> from icontract import snapshot, P, thunk >>> @snapshot(some_identifier=P.self.some_method(P.some_argument.some_attr)) >>> >>> P is an object of our own type, let’s call the type MockP. MockP returns >>> new MockP objects when any operation is done to it. MockP * MockP = MockP. >>> MockP.attr =
Re: [Python-ideas] "old" values in postconditions
I am fine with your proposed syntax. It’s certainly lucid. Perhaps it would be
a good idea to get people accustomed to “non-magic” syntax.
> I still have a feeling that most developers would like to store the state in
> many different custom ways.
Please explain. (Expressions like thunk(all)(a == b for a, b in P.arg.meth())
would be valid.)
> I'm thinking mostly about all the edge cases which we would not be able to
> cover (and how complex that would be to cover them).
Except for a > b > c being one flat expression with 5 members, it seems fairly
easy to recreate an AST, which can then be compiled down to a code object. The
code object can be fun with a custom “locals()”
Below is my concept code for such a P object.
from ast import *
# not done: enforce Singleton property on EmptySymbolType
class EmptySymbolType(object): ...
EmptySymbol = EmptySymbolType() # empty symbols are placeholders
class MockP(object):
# "^" is xor
@icontract.pre(lambda symbol, astnode: (symbol is None) ^ (astnode is None))
def __init__(self, symbol=None, value=EmptySymbol, astnode=None,
initsymtable=(,)):
self.symtable = dict(initsymtable)
if symbol:
self.expr = Expr(value=Name(id=symbol, ctx=Load()))
self.symtable = {symbol: value}
else:
self.expr = astnode
self.frozen = False
def __add__(self, other):
wrapped = MockP.wrap_value(other)
return MockP(astnode=Expr(value=BinOp(self.expr, Add(), wrapped.expr),
initsymtable={**self.symtable, **wrapped.symtable})
def compile(self): ...
def freeze(self):
# frozen objects wouldn’t have an overrided getattr, allowing for
icontract to manipulate the MockP object using its public interface
self.frozen = True
@classmethod
def wrap_value(cls, obj):
# create a MockP object from a value. Generate a random identifier and
set that as the key in symtable, the AST node is the name of that identifier,
retrieving its value through simple expression evaluation.
...
thunk = MockP.wrap_value
P = MockP('P')
# elsewhere: ensure P is only accessed via valid “dot attribute access” inside
@snapshot so contracts fail early, or don’t and allow Magic like __dict__ to
occur on P.
> On Sep 27, 2018, at 9:49 PM, Marko Ristin-Kaufmann
> wrote:
>
> Hi James,
>
> I still have a feeling that most developers would like to store the state in
> many different custom ways. I see also thunk and snapshot with wrapper
> objects to be much more complicated to implement and maintain; I'm thinking
> mostly about all the edge cases which we would not be able to cover (and how
> complex that would be to cover them). Then the linters need also to work
> around such wrappers... It might also scare users off since it looks like too
> much magic. Another concern I also have is that it's probably very hard to
> integrate these wrappers with mypy later -- but I don't really have a clue
> about that, only my gut feeling?
>
> What about we accepted to repeat "lambda P, " prefix, and have something like
> this:
>
> @snapshot(
> lambda P, some_name: len(P.some_property),
> lambda P, another_name: hash(P.another_property)
> )
>
> It's not too verbose for me and you can still explain in three-four sentences
> what happens below the hub in the library's docs. A pycharm/pydev/vim/emacs
> plugins could hide the verbose parts.
>
> I performed a small experiment to test how this solution plays with pylint
> and it seems OK that arguments are not used in lambdas.
>
> Cheers,
> Marko
>
>
>> On Thu, 27 Sep 2018 at 12:27, James Lu wrote:
>> Why couldn’t we record the operations done to a special object and replay
>> them?
>>
> Actually, I think there is probably no way around a decorator that
> captures/snapshots the data before the function call with a lambda (or
> even a separate function). "Old" construct, if we are to parse it somehow
> from the condition function, would limit us only to shallow copies (and
> be complex to implement as soon as we are capturing out-of-argument
> values such as globals etc.). Moreove, what if we don't need shallow
> copies? I could imagine a dozen of cases where shallow copy is not what
> the programmer wants: for example, s/he might need to make deep copies,
> hash or otherwise transform the input data to hold only part of it
> instead of copying (e.g., so as to allow equality check without a double
> copy of the data, or capture only the value of certain property
> transformed in some way).
>>
>>
>> from icontract import snapshot, P, thunk
>> @snapshot(some_identifier=P.self.some_method(P.some_argument.some_attr))
>>
>> P is an object of our own type, let’s call the type MockP. MockP returns new
>> MockP objects when any operation is done to it. MockP * MockP = MockP.
>> MockP.attr = MockP. MockP objects remember all the operations done to them,
>>
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
Chris Angelico wrote: It is still fundamentally difficult to make assertions about the file system as pre/post contracts. When you consider race conditions, I'd say it's impossible. A postcondition check involving the file system could fail even if the function does its job perfectly. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On 2018-09-28 05:23, David Mertz wrote: On Thu, Sep 27, 2018, 9:25 PM Marko Ristin-Kaufmann mailto:marko.ris...@gmail.com>> wrote: Try to question whether the contracts I wrote are so obvious to everybody even if they are obvious to you and keep in mind that the user does not look into the implementation. I had missed this comment, but this seems to be the biggest disconnect, or talking past each other. I'm a user of many libraries. I USUALLY look at the implementation when in doubt about a function. If contracts are meant only for users who don't look at code, the detrimental effect on code readability is mitigated. I agree with you that this seems to be a major disconnect in the discussion here. However, on the issue itself, I quite agree with Marko that it is *much* more important for the documentation to be readable than for the function to be readable. I too read the source of functions sometimes, and whenever I find myself having to do so, I grouse at the authors of whatever libraries I'm using for not making the documentation more clear. Ideally, no user should *ever* have to read the function source code, because the documentation should make it completely clear how to use the function without knowing *anything* about how it is implemented. Of course, this ideal will never be achieved, but I think it's something to aim towards, and the idea that adding a few lines of DbC decorators makes the source code too cluttered seems quite incorrect to me. I glanced through the source code and didn't find it hard to read at all. The contracts are all cleanly separated from the function bodies because they're in decorators up front. I'm frankly quite baffled that other people on this thread find that hard to read. The problem with reading the source code is that you can't tell what parts of the behavior are specified and which are implementation details. The appeal of something like DbC is that it encourages (some might say painfully forces) programmers to be very explicit about what behavior they want to guarantee. Whether writing these guarantees as Python expressions is better than writing them as prose is another matter. Personally I do see some value in the modifications that Marko made to pathlib. In a sense, writing "documentation as Python code" is like annotating the source code to mark which specific parts of the implementation are guarantees and which may be implementation details. I think there is significant value in knowing precisely what an API allows, in an explicit and verifiable form such as that provided by DbC, rather than using human language, which is much less precise, can leave room for misinterpretation, and, perhaps most importantly, is harder to verify automatically. Ultimately, I'm firmly of the opinion that, in publicly distributed code, the function *IS* its documentation, not its source code. When a function's actual behavior conflicts with its documented behavior, that is a bug. When meaningful aspects of a functions behavior are not specified in the documentation, that is also a bug. These may be bugs in the documentation or in the behavior, but either way the point is that reading the source code is not an acceptable substitute for making the documentation a complete and self-sufficient vehicle for total understanding of the function's behavior. It doesn't matter if the function behaves as the author intended; it only matters if it behaves as documented. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely
On 9/28/18 12:45 PM, Steven D'Aprano wrote: On Tue, Sep 25, 2018 at 09:59:53PM +1000, Hugh Fisher wrote: C and Python (currently) are known as simple languages. o_O That's a usage of "simple" I haven't come across before. Especially in the case of C, which is a minefield of *intentionally* underspecified behaviour which makes it near to impossible for the developer to tell what a piece of syntactically legal C code will actually do in practice. s/C/Python/ s/underspecified/dynamic/ ;-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely
On Tue, Sep 25, 2018 at 09:59:53PM +1000, Hugh Fisher wrote: > C and Python (currently) are known as simple languages. o_O That's a usage of "simple" I haven't come across before. Especially in the case of C, which is a minefield of *intentionally* underspecified behaviour which makes it near to impossible for the developer to tell what a piece of syntactically legal C code will actually do in practice. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
Hi, (Posting from work, so sorry for the short response.) @Paul Moore icontract.pre/post/inv have the enabled argument; if not enabled, the contract is ignored. Similarly with rmdir() -- "the directory must be empty" -- but how exactly >> am I supposed to check that? >> > > Isn't that the whole point? The prose statement "the directory must be > empty" is clear. But the exact code check isn't - and may be best handled > by a series of unit tests, rather than a precondition. > I meant "check" as a user, not as a developer. As in "What did the implementer think -- how am I supposed to check that the directory is empty?" A la: "Dear user, if you want to rmdir, here is what you need to check that it is indeed a dir, and here is what you need to check that it is empty. If both checks pass, run me." @David patching __doc__ automatically is on the short-term todo list. I suppose we'll just add sphinx directives (:requires:, :ensures: etc.) * Marko isn't that familiar with the codebase, so there may be better > ways to express certain things > This is true :) * Sometimes it's just plain hard to express a verbal constraint in code > In these cases you simply don't express it in code. Why would you? If it's complex code, possibility that you have an error is probably equal or higher than that your comment rots. @pre(lambda args, result: not any(Path(arg).is_absolute() for arg in args) > or > (result == [pth for arg in args for pth in [Path(arg)] if > pth.is_absolute()][-1]), > "When several absolute paths are given, the last is taken as an anchor > (mimicking os.path.join()’s behaviour)") > I'm really not familiar with the code base nor with how to write stuff nice and succinct in python. This particular contract was hard to define because there were no last() and no arg_is_absolute() functions. Otherwise, it would have read: @pre(lambda args, result: not any(arg_is_absolute(arg) for arg in args) or result == Path(last(arg for arg in args if arg_is_absolute(arg))) When rendered, this wouldn't look too bad to read. @Chris > It is still fundamentally difficult to make assertions about the file > system as pre/post contracts. Are you becoming aware of this? > Contracts, as has been stated multiple times, look great for > mathematically pure functions that have no state outside of their own > parameters and return values (and 'self', where applicable), but are > just poor versions of unit tests when there's anything external to > consider. > I never thought of these particular contracts as running in the production. I would set them to run only in testing and only on part of the tests where they are safe from race conditions (e.g., setting enabled=test_pathlib.SERIAL; toggling mechanism is something I haven't spent too much thought about either and would also need to be discussed/implemented.). I really thought about them as documentation, not for correctness (or at best deeper tests during the testing in a "safe" local environment, for example when you want to check if all the contracts also hold on situations in *my *testsuite, not only during the test suite of pathlib). In the end, I'm calling it the day. I really got tired in the last few days. Standardizing contracts for python is not worth the pain. We'll continue to develop icontract for our internal needs and keep it open source, so anybody who's interested can have a look. Thank you all for a very lively discussions! Cheers, Marko On Fri, 28 Sep 2018 at 14:49, Paul Moore wrote: > On Fri, 28 Sep 2018 at 13:23, David Mertz wrote: > > I agree that all the Sphinx documentation examples shown are very nice. > Prose descriptions would often be nicer still, but the Boolean expressions > are helpful for those unusual cases where I don't want to look at the code. > > I'm ambivalent about the Sphinx examples. I find the highly detailed > code needed to express a condition fairly unreadable (and I'm an > experienced Python programmer). For example > > @pre(lambda args, result: not any(Path(arg).is_absolute() for arg in args) > or > (result == [pth for arg in args for pth in [Path(arg)] if > pth.is_absolute()][-1]), > "When several absolute paths are given, the last is taken as an anchor > (mimicking os.path.join()’s behaviour)") > > The only way I'd read that is by looking at the summary text - I'd > never get the sense of what was going on from the code alone. There's > clearly a number of trade-offs going on here: > > * Conditions should be short, to avoid clutter > * Writing helper functions that are *only* used in conditions is more > code to test or get wrong > * Sometimes it's just plain hard to express a verbal constraint in code > * Marko isn't that familiar with the codebase, so there may be better > ways to express certain things > > But given that *all* the examples I've seen of contracts have this > issue (difficult to read expressions) I suspect the problem is > inherent. > > Another thing that I haven't yet seen clearly explained.
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Fri, 28 Sep 2018 at 13:23, David Mertz wrote: > I agree that all the Sphinx documentation examples shown are very nice. Prose > descriptions would often be nicer still, but the Boolean expressions are > helpful for those unusual cases where I don't want to look at the code. I'm ambivalent about the Sphinx examples. I find the highly detailed code needed to express a condition fairly unreadable (and I'm an experienced Python programmer). For example @pre(lambda args, result: not any(Path(arg).is_absolute() for arg in args) or (result == [pth for arg in args for pth in [Path(arg)] if pth.is_absolute()][-1]), "When several absolute paths are given, the last is taken as an anchor (mimicking os.path.join()’s behaviour)") The only way I'd read that is by looking at the summary text - I'd never get the sense of what was going on from the code alone. There's clearly a number of trade-offs going on here: * Conditions should be short, to avoid clutter * Writing helper functions that are *only* used in conditions is more code to test or get wrong * Sometimes it's just plain hard to express a verbal constraint in code * Marko isn't that familiar with the codebase, so there may be better ways to express certain things But given that *all* the examples I've seen of contracts have this issue (difficult to read expressions) I suspect the problem is inherent. Another thing that I haven't yet seen clearly explained. How do these contracts get *run*? Are they checked on every call to the function, even in production code? Is there a means to turn them off? What's the runtime overhead of a "turned off" contract (even if it's just an if-test per condition, that can still add up)? And what happens if a contract fails - is it an exception/traceback (which is often unacceptable in production code such as services)? The lack of any clear feel for the cost of adding contracts is part of what makes people reluctant to use them (particularly in the face of the unfortunately still common assertion that "Python is slow" :-() Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Thu, Sep 27, 2018, 9:25 PM Marko Ristin-Kaufmann wrote: > Try to question whether the contracts I wrote are so obvious to everybody > even if they are obvious to you and keep in mind that the user does not > look into the implementation. > I had missed this comment, but this seems to be the biggest disconnect, or talking past each other. I'm a user of many libraries. I USUALLY look at the implementation when in doubt about a function. If contracts are meant only for users who don't look at code, the detrimental effect on code readability is mitigated. The other place I look, if not the actual implementation, is at the docstring. I don't remember if icontracts patches the docstring when it decorates a function. If not, that would be very helpful. I agree that all the Sphinx documentation examples shown are very nice. Prose descriptions would often be nicer still, but the Boolean expressions are helpful for those unusual cases where I don't want to look at the code. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Fri, Sep 28, 2018, 4:38 AM Paul Moore wrote: > On Fri, 28 Sep 2018 at 02:24, Marko Ristin-Kaufmann < > marko.ris...@gmail.com> wrote: > >> I annotated pathlib with contracts: >> https://github.com/mristin/icontract-pathlib-poc. I zipped the HTML docs >> into >> https://github.com/mristin/icontract-pathlib-poc/blob/master/contracts-pathlib-poc.zip, >> you can just unpack and view the index.html. >> > > The thing that you didn't discuss in the above was the effect on the > source code. Looking at your modified sources, I found it *significantly* > harder to read your annotated version than the original. Basically every > function and class was cluttered with irrelevant[1] conditions, which > obscured the logic and the basic meaning of the code. > My reaction was just the same as Paul's. I read the modified source, and found that the invariant declarations made it *dramatically* harder to read. The ratio was almost exactly as I characterized in a recent note: 15 lines of pre/post-conditions on a 10 like function. Like Paul, I understand the documentation and testing value of these, but they were highly disruptive to the readability of the functions themselves. As a result of reading the example, I'd be somewhat less likely to use a DbC library, and much more strongly opposed to having one in the standards library (and aghast at the idea of dedicated syntax) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Fri, 28 Sep 2018 at 10:37, Chris Angelico wrote: > > On Fri, Sep 28, 2018 at 7:29 PM Jonathan Fine wrote: > > > > I like this discussion. I'd like to add another theme, namely what > > should happen when there is an error. (This is prompted by race > > hazards when performing file system operations.) > > > > Suppose fn_a() calls fn_b(), and fn_b() raises an exception. What then > > should fn_a() do? It may be that this exception has left part or all > > of the system in an inconsistent (invalid) state. > > That's why try/finally exists. You shouldn't have to worry about > contracts for that. > > (Similarly, context managers, which are a way of wrapping up > try/finally into a convenient package.) However, a contract would need to be able to express "Returns a fully initialised widget or raises a BrokenWidget exception". Unit tests do this sort of thing all the time. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Fri, Sep 28, 2018 at 7:29 PM Jonathan Fine wrote: > > I like this discussion. I'd like to add another theme, namely what > should happen when there is an error. (This is prompted by race > hazards when performing file system operations.) > > Suppose fn_a() calls fn_b(), and fn_b() raises an exception. What then > should fn_a() do? It may be that this exception has left part or all > of the system in an inconsistent (invalid) state. That's why try/finally exists. You shouldn't have to worry about contracts for that. (Similarly, context managers, which are a way of wrapping up try/finally into a convenient package.) ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
I like this discussion. I'd like to add another theme, namely what should happen when there is an error. (This is prompted by race hazards when performing file system operations.) Suppose fn_a() calls fn_b(), and fn_b() raises an exception. What then should fn_a() do? It may be that this exception has left part or all of the system in an inconsistent (invalid) state. At this level of abstraction, it's not possible to sensibly answer this question. Sometimes the whole system should be stopped. Other times, an invalidation of an object is enough. And sometimes, a rollback of the transaction is what's wanted. Here's a well-known example (overflow exception in Ariane 5), which to me shows that these problems can be very hard to get right. https://en.wikipedia.org/wiki/Cluster_(spacecraft) According to wikipedia (above) this failure resulted in "the first example of large-scale static code analysis by abstract interpretation". I expect that in some situations design-by-contract will help here, by encouraging a focus on providing a more complete specification of behaviour. It would be good to have some real-life Python examples. I'd afraid I don't have any (although I've looked either). -- Jonathan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Fri, Sep 28, 2018 at 11:25 AM Marko Ristin-Kaufmann wrote: > > Hi, > > I annotated pathlib with > contracts:https://github.com/mristin/icontract-pathlib-poc. I zipped the HTML > docs into > https://github.com/mristin/icontract-pathlib-poc/blob/master/contracts-pathlib-poc.zip, > you can just unpack and view the index.html. > > One thing I did observe was that there were contracts written in text all > over the documentation -- I tried to formulate most of them in code. Since > I'm not the implementer nor very familiar with the module, please consider > that many of these contracts can be definitely made more beautiful. There > were some limitations to icontract-sphinx extension and icontracts which I > noted at the top of the document. > You do a lot of contracts that involve is_absolute and other checks. But the postcondition on is_absolute merely says: not result or self.root != "" (Also: I'm not sure, but I think maybe that should be _root?? Leaving that aside.) So I'm not sure how much you can really ascertain about absolute paths. You guarantee that is_absolute will return something plausible, but unless you guarantee that it's returning the correct value, depending on it for your preconditions seems dubious. A buggy is_absolute could break basically everything, and your contracts wouldn't notice. It is still fundamentally difficult to make assertions about the file system as pre/post contracts. Are you becoming aware of this? Contracts, as has been stated multiple times, look great for mathematically pure functions that have no state outside of their own parameters and return values (and 'self', where applicable), but are just poor versions of unit tests when there's anything external to consider. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add .= as a method return value assignment operator
On Fri, Sep 28, 2018 at 6:56 PM Jonathan Fine wrote: > Finally, I note > > >>> a = 2 > >>> a **= 3 > >>> a > 8 > ? Yes? That's what 2 ** 3 is, so that's what I would expect. All other augmented assignment operators take an assignment target on the left and a (single) value on the right. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add .= as a method return value assignment operator
Summary: I recast an example in a more abstract form.
Steve D'Aprano wrote:
> Think about a more complex assignment:
>text .= encode(spam) + str(eggs)
I find this example instructive. I hope the following is also instructive:
$ python3
>>> obj += incr
NameError: name 'obj' is not defined
>>> obj = object()
>>> obj += incr
NameError: name 'incr' is not defined
>>> incr = 1
>>> obj += incr
TypeError: unsupported operand type(s) for +=: 'object' and 'int'
>>> incr = object()
>>> obj += incr
TypeError: unsupported operand type(s) for +=: 'object' and 'object'
>>> obj += [] + ()
TypeError: can only concatenate list (not "tuple") to list
To me this shows that
LHS += RHS
works as follows:
1. Evaluate the LHS (as an assignable object).
2. Evaluate the RHS (as a value).
and then some more steps, not covered in my example.
As syntax the compound symbols '+=' and '.=' are similar. But in
semantics, '+=' does and '.=' does not have an evaluation of the RHS
as an expression. This is, in abstract terms, the origin of Steve's
example. Someone else has noted that '+=' and its variants are focused
on numeric operations, such as addition.
This shows, to me, that the simplification provided by use cases such as
text = text.replace("foo","bar")
has to be compared to the complexity introduced by
text .= encode(spam) + str(eggs)
In other words, I've restated Steve's example, in a more abstract
form. I hope it helps to have another way to look at this example.
Finally, I note
>>> a = 2
>>> a **= 3
>>> a
8
--
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Fri, 28 Sep 2018 at 02:24, Marko Ristin-Kaufmann wrote: > Hi, > > I annotated pathlib with contracts: > https://github.com/mristin/icontract-pathlib-poc. I zipped the HTML docs > into > https://github.com/mristin/icontract-pathlib-poc/blob/master/contracts-pathlib-poc.zip, > you can just unpack and view the index.html. > Thanks, for doing this! It's probably not going to result in the reaction you hoped for (see below) but I appreciate you taking the time to do it. Some of the contracts might seem trivial -- but mind that you, as a writer, > want to tell the user what s/he is expected to fulfill before calling the > function. For example, if you say: > rmdir() > > Remove this directory. The directory must be empty. > Requires: > >- not list(self.iterdir()) (??? There must be a way to check this more >optimally) >- self.is_dir() > > > self.is_dir() contract might seem trivial -- but it's actually not. You > actually want to convey the message: dear user, if you are iterating > through a list of paths, use this function to decide if you should call > rmdir() or unlink(). Analogously with the first contract: dear user, please > check if the directory is empty before calling rmdir() and this is what you > need to call to actually check that. > The problem (IMO) with both of these is precisely that they are written as Python expressions. Your prose descriptions of what they mean are fine, and *those* are what I would hope to see in documentation. This is far more obvious in later examples, where the code needed to check certain conditions is extremely unclear unless you spend time trying to interpret it. > I also finally assembled the puzzle. Most of you folks are all older and > were exposed to DbC in the 80ies championed by DbC zealots who advertised > it as *the *tool for software development. You were repulsed by their > fanaticism -- the zealots also pushed for all the contracts to be defined, > and none less. Either you have 100% DbC or no sane software development at > all. > Well, yes, but your claims were *precisely* the same as those I saw back then. "All projects on PyPI would benefit", "the benefits are obvious", ... As I said, DbC is a reasonable methodology, but the reason it's not more widely used is (in the first instance) because it has a *marketing* problem. Solving that with more of the same style of marketing won't help. Once you get beyond the marketing, there are *still* questions (see above and below), but if you can't even get people past the first step, you've lost. > And that's why I said that the libraries on pypi meant to be used by > multiple people and which already have type annotations would obviously > benefit from contracts -- while you were imagining that all of these > libraries need to be DbC'ed 100%, I was imagining something much more > humble. Thus the misunderstanding. > No, I was imagining that some libraries were small, some were used by small, specialised groups, and some were immensely successful without DbC. So claiming that they would "obviously" benefit is a very strong claim. > After annotating pathlib, I find that it actually needs contracts more > thain if it had type annotations. For example: > stat() > > Return the result of the stat() system call on this path, like os.stat() > does. > Ensures: > >- result is not None ⇒ self.exists() >- result is not None ⇒ os.stat(str(self)).__dict__ == result.__dict__ >(??? This is probably not what it was meant with ‘like os.stat() does’?) > > > But what does it mean "like os.stat() does"? I wrote equality over > __dict__'s in the contract. That was my idea of what the implementer was > trying to tell me. But is that the operator that should be applied? Sure, > the contract merits a description. But without it, how am I supposed to > know what "like" means? > > Similarly with rmdir() -- "the directory must be empty" -- but how exactly > am I supposed to check that? > Isn't that the whole point? The prose statement "the directory must be empty" is clear. But the exact code check isn't - and may be best handled by a series of unit tests, rather than a precondition. Anyhow, please have a look at the contracts and let me know what you think. > Please consider it an illustration. Try to question whether the contracts I > wrote are so obvious to everybody even if they are obvious to you and keep > in mind that the user does not look into the implementation. And please try > to abstract away the aesthetics: neither icontract library that I wrote nor > the sphinx extension are of sufficient quality. We use them for our company > code base, but they still need a lot of polishing. So please try to focus > only on the content. We are still talking about contracts in general, not > about the concrete contract implementation > The thing that you didn't discuss in the above was the effect on the source code. Looking at your modified sources, I found it *significantly* harder to read your
Re: [Python-ideas] Add .= as a method return value assignment operator
On Fri, Sep 28, 2018 at 05:34:58PM +1200, Greg Ewing wrote: > Steven D'Aprano wrote: > >Think about a more complex assignment: > > > >text .= encode(spam) + str(eggs) > > I think the only sane thing would be to disallow that, and > require the RHS to have the form of a function call, which is > always interpreted as a method of the LHS. You obviously have a different idea of what is "sane" than I do :-) But okay, so we cripple the RHS so that it can only be a single method call. So useful things like these are out: target .= method(arg) or default target .= foo(arg) if condition else bar(arg) and even target .= method(args) + 1 making the syntax pure sugar for target = target.method(args) and absolutely nothing else. I think that's the sort of thing which gives syntactic sugar a bad name. The one positive I can see is that if the target is a compound expression, it could be evaluated once only: spam[0](x, y, z).eggs['cheese'].aardvark .= method(args) I suppose if I wrote a lot of code like that, aside from (probably?) violating the Law of Demeter, I might like this syntax because it avoids repeating a long compound target. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add .= as a method return value assignment operator
I had the number 4 in mind Though I think your idea is way better, as it's more flexible and less confusing On Fri, Sep 28, 2018, 10:33 Brice Parent wrote: > > Le 27/09/2018 à 12:48, Ken Hilton a écrit : > > Hi Jasper, > This seems like a great idea! It looks so much cleaner, too. > > Would there be a dunder method handling this? Or since it's explicitly > just a syntax for "obj = obj.method()" is that not necessary? > My only qualm is that this might get PHP users confused; that's really not > an issue, though, since Python is not PHP. > > Anyway, I fully support this idea. > > What would the following evaluate to? > a .= b + c(d) > > 1: a = a.b + a.c(a.d) # everything is prepended an "a." > it means we dn't have access to any external elements, making the > functionality only useful in a few cases > > 2: a = a.b + a.c(d) # every first level element (if that means something) > is prepended an "a." > We still lose some of the ability to access anything outside of `a`, but a > bit less than in #1. The effort to understand the line as grown a bit, > though. > > 3: a = a.(b + c(d)) # everything is evaluated, and an "a." is prepended > to that result > (the same way `a *= 2 + 3` is equivalent to `a *= 5`) > I believe in most cases, this wouldn't mean anything to evaluate `b + > c(d)` on their own, and expect a return that can be used as an attribute of > `a`. > > 4: a = a.b + c(d) # "a." is prepended to the first element after the `=` > It is probably quite easy to read and understand, but it removes the > transitivity of the operators we have on the right, and is a bit limiting. > > 5: SyntaxError: Can only use the [whatever the name] augmented operator > with a single expression > Why not, it's a bit limiting, but is clear enough to me. > > Maybe, a simpler thing to do for this problem would be to make something > like this: > a = .b(5) + c(.d) + 3 > being the equivalent of > a = a.b(5) + c(a.d) + 3 > > I don't see any ambiguity anymore, it shortens the code a lot, and I guess > it wouldn't be hard for the compiler to recompose the line as a first > parsing step, and create the same AST with both syntaxes. > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add .= as a method return value assignment operator
Le 27/09/2018 à 12:48, Ken Hilton a écrit : Hi Jasper, This seems like a great idea! It looks so much cleaner, too. Would there be a dunder method handling this? Or since it's explicitly just a syntax for "obj = obj.method()" is that not necessary? My only qualm is that this might get PHP users confused; that's really not an issue, though, since Python is not PHP. Anyway, I fully support this idea. What would the following evaluate to? a .= b + c(d) 1: a = a.b + a.c(a.d) # everything is prepended an "a." it means we dn't have access to any external elements, making the functionality only useful in a few cases 2: a = a.b + a.c(d) # every first level element (if that means something) is prepended an "a." We still lose some of the ability to access anything outside of `a`, but a bit less than in #1. The effort to understand the line as grown a bit, though. 3: a = a.(b + c(d)) # everything is evaluated, and an "a." is prepended to that result (the same way `a *= 2 + 3` is equivalent to `a *= 5`) I believe in most cases, this wouldn't mean anything to evaluate `b + c(d)` on their own, and expect a return that can be used as an attribute of `a`. 4: a = a.b + c(d) # "a." is prepended to the first element after the `=` It is probably quite easy to read and understand, but it removes the transitivity of the operators we have on the right, and is a bit limiting. 5: SyntaxError: Can only use the [whatever the name] augmented operator with a single expression Why not, it's a bit limiting, but is clear enough to me. Maybe, a simpler thing to do for this problem would be to make something like this: a = .b(5) + c(.d) + 3 being the equivalent of a = a.b(5) + c(a.d) + 3 I don't see any ambiguity anymore, it shortens the code a lot, and I guess it wouldn't be hard for the compiler to recompose the line as a first parsing step, and create the same AST with both syntaxes. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
