> The problem with readability might be easier to solve than I thought, and
> your pointer to coconut gave me the idea. What if we make a utility that
> takes the python source code, examines the decorators pre/post/inv (or
> whatever we call them) and transforms them back and forth from/to valid
> python code?
>
> Pretty much any IDE has after load / before save handlers. When you load a
> python file, you'd transform it from less readable python code, write using a
> concise form and when you save it, it's transformed back to valid python.
Hmm yes. Look at my previous email- the proposed syntax there does not require
changing or transforming Python code. It’s missing many magic methods, but it
would work something like this:
@snapshot(some_identifier=P.self.another_property + 10)
would internally turn into something like this: lambda P:
P.self.another_property + 10
Maybe it’s possible to modify PyCharm parsing by altering its Python grammar
file?
Sent from my iPhone
> On Sep 28, 2018, at 10:27 AM, Marko Ristin-Kaufmann <marko.ris...@gmail.com>
> wrote:
>
> Hi James,
>
> The problem with readability might be easier to solve than I thought, and
> your pointer to coconut gave me the idea. What if we make a utility that
> takes the python source code, examines the decorators pre/post/inv (or
> whatever we call them) and transforms them back and forth from/to valid
> python code?
>
> Pretty much any IDE has after load / before save handlers. When you load a
> python file, you'd transform it from less readable python code, write using a
> concise form and when you save it, it's transformed back to valid python.
>
> Then we need to pick the python form that is easiest to implement (and still
> readable enough for, say, code reviews on github), but writing and reading
> the contracts in the code would be much more pleasant.
>
> As long as the "readable" form has also valid python syntax, the tool can be
> implemented with ast module.
>
> For example:
> @snapshot(some_identifier=self.another_property + 10)
> @post(self.some_property => old.some_identifier > 100)
>
> would transform into
> @snapshot(lambda P, some_identifier: P.self.another_property + 10)
> @post(lambda O, P: not self.some_property and O.some_identifier > 100)
>
> Cheers,
> Marko
>
>> On Fri, 28 Sep 2018 at 03:49, Marko Ristin-Kaufmann <marko.ris...@gmail.com>
>> wrote:
>> Hi James,
>>
>> I still have a feeling that most developers would like to store the state in
>> many different custom ways. I see also thunk and snapshot with wrapper
>> objects to be much more complicated to implement and maintain; I'm thinking
>> mostly about all the edge cases which we would not be able to cover (and how
>> complex that would be to cover them). Then the linters need also to work
>> around such wrappers... It might also scare users off since it looks like
>> too much magic. Another concern I also have is that it's probably very hard
>> to integrate these wrappers with mypy later -- but I don't really have a
>> clue about that, only my gut feeling?
>>
>> What about we accepted to repeat "lambda P, " prefix, and have something
>> like this:
>>
>> @snapshot(
>> lambda P, some_name: len(P.some_property),
>> lambda P, another_name: hash(P.another_property)
>> )
>>
>> It's not too verbose for me and you can still explain in three-four
>> sentences what happens below the hub in the library's docs. A
>> pycharm/pydev/vim/emacs plugins could hide the verbose parts.
>>
>> I performed a small experiment to test how this solution plays with pylint
>> and it seems OK that arguments are not used in lambdas.
>>
>> Cheers,
>> Marko
>>
>>
>>> On Thu, 27 Sep 2018 at 12:27, James Lu <jam...@gmail.com> wrote:
>>> Why couldn’t we record the operations done to a special object and replay
>>> them?
>>>
>>>>>> Actually, I think there is probably no way around a decorator that
>>>>>> captures/snapshots the data before the function call with a lambda (or
>>>>>> even a separate function). "Old" construct, if we are to parse it
>>>>>> somehow from the condition function, would limit us only to shallow
>>>>>> copies (and be complex to implement as soon as we are capturing
>>>>>> out-of-argument values such as globals etc.). Moreove, what if we don't
>>>>>> need shallow copies? I could imagine a dozen of cases where shallow copy
>>>>>> is not what the programmer wants: for example, s/he might need to make
>>>>>> deep copies, hash or otherwise transform the input data to hold only
>>>>>> part of it instead of copying (e.g., so as to allow equality check
>>>>>> without a double copy of the data, or capture only the value of certain
>>>>>> property transformed in some way).
>>>
>>>
>>> from icontract import snapshot, P, thunk
>>> @snapshot(some_identifier=P.self.some_method(P.some_argument.some_attr))
>>>
>>> P is an object of our own type, let’s call the type MockP. MockP returns
>>> new MockP objects when any operation is done to it. MockP * MockP = MockP.
>>> MockP.attr = MockP. MockP objects remember all the operations done to them,
>>> and allow the owner of a MockP object to re-apply the same operations
>>>
>>> “thunk” converts a function or object or class to a MockP object, storing
>>> the function or object for when the operation is done.
>>>
>>> thunk(function)(<MockP expression>)
>>>
>>> Of course, you could also thunk objects like so: thunk(3) * P.number.
>>> (Though it might be better to keep the 3 after P.number in this case so
>>> P.number’s __mult__ would be invoked before 3’s __mult__ is invokes.
>>>
>>>
>>> In most cases, you’d save any operations that can be done on a copy of the
>>> data as generated by @snapshot in @postcondiion. thunk is for rare
>>> scenarios where 1) it’s hard to capture the state, for example an object
>>> that manages network state (or database connectivity etc) and whose stage
>>> can only be read by an external classmethod 2) you want to avoid using
>>> copy.deepcopy.
>>>
>>> I’m sure there’s some way to override isinstance through a meta class or
>>> dunder subclasshook.
>>>
>>> I suppose this mocking method could be a shorthand for when you don’t need
>>> the full power of a lambda. It’s arguably more succinct and readable,
>>> though YMMV.
>>>
>>> I look forward to reading your opinion on this and any ideas you might have.
>>>
>>>> On Sep 26, 2018, at 3:56 PM, James Lu <jam...@gmail.com> wrote:
>>>>
>>>> Hi Marko,
>>>>
>>>>> Actually, following on #A4, you could also write those as multiple
>>>>> decorators:
>>>>> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr)
>>>>> @snpashot(lambda _, other_identifier: other_func(_.self))
>>>>
>>>> Yes, though if we’re talking syntax using kwargs would probably be better.
>>>> Using “P” instead of “_”: (I agree that _ smells of ignored arguments)
>>>>
>>>> @snapshot(some_identifier=lambda P: ..., some_identifier2=lambda P: ...)
>>>>
>>>> Kwargs has the advantage that you can extend multiple lines without
>>>> repeating @snapshot, though many lines of @capture would probably be more
>>>> intuitive since each decorator captures one variable.
>>>>
>>>>> Why uppercase "P" and not lowercase (uppercase implies a constant for me)?
>>>>
>>>> To me, the capital letters are more prominent and explicit- easier to see
>>>> when reading code. It also implies its a constant for you- you shouldn’t
>>>> be modifying it, because then you’d be interfering with the function
>>>> itself.
>>>>
>>>> Side node: maybe it would be good to have an @icontract.nomutate (probably
>>>> use a different name, maybe @icontract.readonly) that makes sure a method
>>>> doesn’t mutate its own __dict__ (and maybe the __dict__ of the members of
>>>> its __dict__). It wouldn’t be necessary to put the decorator on every read
>>>> only function, just the ones your worried might mutate.
>>>>
>>>> Maybe a @icontract.nomutate(param=“paramname”) that ensures the __dict__
>>>> of all members of the param name have the same equality or identity before
>>>> and after. The semantics would need to be worked out.
>>>>
>>>>> On Sep 26, 2018, at 8:58 AM, Marko Ristin-Kaufmann
>>>>> <marko.ris...@gmail.com> wrote:
>>>>>
>>>>> Hi James,
>>>>>
>>>>> Actually, following on #A4, you could also write those as multiple
>>>>> decorators:
>>>>> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr)
>>>>> @snpashot(lambda _, other_identifier: other_func(_.self))
>>>>>
>>>>> Am I correct?
>>>>>
>>>>> "_" looks a bit hard to read for me (implying ignored arguments).
>>>>>
>>>>> Why uppercase "P" and not lowercase (uppercase implies a constant for
>>>>> me)? Then "O" for "old" and "P" for parameters in a condition:
>>>>> @post(lambda O, P: ...)
>>>>> ?
>>>>>
>>>>> It also has the nice property that it follows both the temporal and the
>>>>> alphabet order :)
>>>>>
>>>>>> On Wed, 26 Sep 2018 at 14:30, James Lu <jam...@gmail.com> wrote:
>>>>>> I still prefer snapshot, though capture is a good name too. We could use
>>>>>> generator syntax and inspect the argument names.
>>>>>>
>>>>>> Instead of “a”, perhaps use “_”. Or maybe use “A.”, for arguments. Some
>>>>>> people might prefer “P” for parameters, since parameters sometimes means
>>>>>> the value received while the argument means the value passed.
>>>>>>
>>>>>> (#A1)
>>>>>>
>>>>>> from icontract import snapshot, __
>>>>>> @snapshot(some_func(_.some_argument.some_attr) for some_identifier, _ in
>>>>>> __)
>>>>>>
>>>>>> Or (#A2)
>>>>>>
>>>>>> @snapshot(some_func(some_argument.some_attr) for some_identifier, _,
>>>>>> some_argument in __)
>>>>>>
>>>>>> —
>>>>>> Or (#A3)
>>>>>>
>>>>>> @snapshot(lambda some_argument,_,some_identifier:
>>>>>> some_func(some_argument.some_attr))
>>>>>>
>>>>>> Or (#A4)
>>>>>>
>>>>>> @snapshot(lambda _,some_identifier: some_func(_.some_argument.some_attr))
>>>>>> @snapshot(lambda _,some_identifier, other_identifier:
>>>>>> some_func(_.some_argument.some_attr), other_func(_.self))
>>>>>>
>>>>>> I like #A4 the most because it’s fairly DRY and avoids the extra
>>>>>> punctuation of
>>>>>>
>>>>>> @capture(lambda a: {"some_identifier":
>>>>>> some_func(a.some_argument.some_attr)})
>>>>>>
>>>>>> On Sep 26, 2018, at 12:23 AM, Marko Ristin-Kaufmann
>>>>>> <marko.ris...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Franklin wrote:
>>>>>>>> The name "before" is a confusing name. It's not just something that
>>>>>>>> happens before. It's really a pre-`let`, adding names to the scope of
>>>>>>>> things after it, but with values taken before the function call. Based
>>>>>>>> on that description, other possible names are `prelet`, `letbefore`,
>>>>>>>> `predef`, `defpre`, `beforescope`. Better a name that is clearly
>>>>>>>> confusing than one that is obvious but misleading.
>>>>>>>
>>>>>>> James wrote:
>>>>>>>> I suggest that instead of “@before” it’s “@snapshot” and instead of
>>>>>>>> “old” it’s “snapshot”.
>>>>>>>
>>>>>>>
>>>>>>> I like "snapshot", it's a bit clearer than prefixing/postfixing verbs
>>>>>>> with "pre" which might be misread (e.g., "prelet" has a meaning in
>>>>>>> Slavic languages and could be subconsciously misread, "predef" implies
>>>>>>> to me a pre-definition rather than prior-to-definition , "beforescope"
>>>>>>> is very clear for me, but it might be confusing for others as to what
>>>>>>> it actually refers to ). What about "@capture" (7 letters for captures
>>>>>>> versus 8 for snapshot)? I suppose "@let" would be playing with fire if
>>>>>>> Python with conflicting new keywords since I assume "let" to be one of
>>>>>>> the candidates.
>>>>>>>
>>>>>>> Actually, I think there is probably no way around a decorator that
>>>>>>> captures/snapshots the data before the function call with a lambda (or
>>>>>>> even a separate function). "Old" construct, if we are to parse it
>>>>>>> somehow from the condition function, would limit us only to shallow
>>>>>>> copies (and be complex to implement as soon as we are capturing
>>>>>>> out-of-argument values such as globals etc.). Moreove, what if we don't
>>>>>>> need shallow copies? I could imagine a dozen of cases where shallow
>>>>>>> copy is not what the programmer wants: for example, s/he might need to
>>>>>>> make deep copies, hash or otherwise transform the input data to hold
>>>>>>> only part of it instead of copying (e.g., so as to allow equality check
>>>>>>> without a double copy of the data, or capture only the value of certain
>>>>>>> property transformed in some way).
>>>>>>>
>>>>>>> I'd still go with the dictionary to allow for this extra freedom. We
>>>>>>> could have a convention: "a" denotes to the current arguments, and "b"
>>>>>>> denotes the captured values. It might make an interesting hint that we
>>>>>>> put "b" before "a" in the condition. You could also interpret "b" as
>>>>>>> "before" and "a" as "after", but also "a" as "arguments".
>>>>>>>
>>>>>>> @capture(lambda a: {"some_identifier":
>>>>>>> some_func(a.some_argument.some_attr)})
>>>>>>> @post(lambda b, a, result: b.some_identifier > result +
>>>>>>> a.another_argument.another_attr)
>>>>>>> def some_func(some_argument: SomeClass, another_argument: AnotherClass)
>>>>>>> -> SomeResult:
>>>>>>> ...
>>>>>>> "b" can be omitted if it is not used. Under the hub, all the arguments
>>>>>>> to the condition would be passed by keywords.
>>>>>>>
>>>>>>> In case of inheritance, captures would be inherited as well. Hence the
>>>>>>> library would check at run-time that the returned dictionary with
>>>>>>> captured values has no identifier that has been already captured, and
>>>>>>> the linter checks that statically, before running the code. Reading
>>>>>>> values captured in the parent at the code of the child class might be a
>>>>>>> bit hard -- but that is case with any inherited methods/properties. In
>>>>>>> documentation, I'd list all the captures of both ancestor and the
>>>>>>> current class.
>>>>>>>
>>>>>>> I'm looking forward to reading your opinion on this and alternative
>>>>>>> suggestions :)
>>>>>>> Marko
>>>>>>>
>>>>>>>> On Tue, 25 Sep 2018 at 18:12, Franklin? Lee
>>>>>>>> <leewangzhong+pyt...@gmail.com> wrote:
>>>>>>>> On Sun, Sep 23, 2018 at 2:05 AM Marko Ristin-Kaufmann
>>>>>>>> <marko.ris...@gmail.com> wrote:
>>>>>>>> >
>>>>>>>> > Hi,
>>>>>>>> >
>>>>>>>> > (I'd like to fork from a previous thread, "Pre-conditions and
>>>>>>>> > post-conditions", since it got long and we started discussing a
>>>>>>>> > couple of different things. Let's discuss in this thread the
>>>>>>>> > implementation of a library for design-by-contract and how to push
>>>>>>>> > it forward to hopefully add it to the standard library one day.)
>>>>>>>> >
>>>>>>>> > For those unfamiliar with contracts and current state of the
>>>>>>>> > discussion in the previous thread, here's a short summary. The
>>>>>>>> > discussion started by me inquiring about the possibility to add
>>>>>>>> > design-by-contract concepts into the core language. The idea was
>>>>>>>> > rejected by the participants mainly because they thought that the
>>>>>>>> > merit of the feature does not merit its costs. This is quite
>>>>>>>> > debatable and seems to reflect many a discussion about
>>>>>>>> > design-by-contract in general. Please see the other thread, "Why is
>>>>>>>> > design-by-contract not widely adopted?" if you are interested in
>>>>>>>> > that debate.
>>>>>>>> >
>>>>>>>> > We (a colleague of mine and I) decided to implement a library to
>>>>>>>> > bring design-by-contract to Python since we don't believe that the
>>>>>>>> > concept will make it into the core language anytime soon and we
>>>>>>>> > needed badly a tool to facilitate our work with a growing code base.
>>>>>>>> >
>>>>>>>> > The library is available at http://github.com/Parquery/icontract.
>>>>>>>> > The hope is to polish it so that the wider community could use it
>>>>>>>> > and once the quality is high enough, make a proposal to add it to
>>>>>>>> > the standard Python libraries. We do need a standard library for
>>>>>>>> > contracts, otherwise projects with conflicting contract libraries
>>>>>>>> > can not integrate (e.g., the contracts can not be inherited between
>>>>>>>> > two different contract libraries).
>>>>>>>> >
>>>>>>>> > So far, the most important bits have been implemented in icontract:
>>>>>>>> >
>>>>>>>> > Preconditions, postconditions, class invariants
>>>>>>>> > Inheritance of the contracts (including strengthening and weakening
>>>>>>>> > of the inherited contracts)
>>>>>>>> > Informative violation messages (including information about the
>>>>>>>> > values involved in the contract condition)
>>>>>>>> > Sphinx extension to include contracts in the automatically generated
>>>>>>>> > documentation (sphinx-icontract)
>>>>>>>> > Linter to statically check that the arguments of the conditions are
>>>>>>>> > correct (pyicontract-lint)
>>>>>>>> >
>>>>>>>> > We are successfully using it in our code base and have been quite
>>>>>>>> > happy about the implementation so far.
>>>>>>>> >
>>>>>>>> > There is one bit still missing: accessing "old" values in the
>>>>>>>> > postcondition (i.e., shallow copies of the values prior to the
>>>>>>>> > execution of the function). This feature is necessary in order to
>>>>>>>> > allow us to verify state transitions.
>>>>>>>> >
>>>>>>>> > For example, consider a new dictionary class that has "get" and
>>>>>>>> > "put" methods:
>>>>>>>> >
>>>>>>>> > from typing import Optional
>>>>>>>> >
>>>>>>>> > from icontract import post
>>>>>>>> >
>>>>>>>> > class NovelDict:
>>>>>>>> > def length(self)->int:
>>>>>>>> > ...
>>>>>>>> >
>>>>>>>> > def get(self, key: str) -> Optional[str]:
>>>>>>>> > ...
>>>>>>>> >
>>>>>>>> > @post(lambda self, key, value: self.get(key) == value)
>>>>>>>> > @post(lambda self, key: old(self.get(key)) is None and
>>>>>>>> > old(self.length()) + 1 == self.length(),
>>>>>>>> > "length increased with a new key")
>>>>>>>> > @post(lambda self, key: old(self.get(key)) is not None and
>>>>>>>> > old(self.length()) == self.length(),
>>>>>>>> > "length stable with an existing key")
>>>>>>>> > def put(self, key: str, value: str) -> None:
>>>>>>>> > ...
>>>>>>>> >
>>>>>>>> > How could we possible implement this "old" function?
>>>>>>>> >
>>>>>>>> > Here is my suggestion. I'd introduce a decorator "before" that would
>>>>>>>> > allow you to store whatever values in a dictionary object "old"
>>>>>>>> > (i.e. an object whose properties correspond to the key/value pairs).
>>>>>>>> > The "old" is then passed to the condition. Here is it in code:
>>>>>>>> >
>>>>>>>> > # omitted contracts for brevity
>>>>>>>> > class NovelDict:
>>>>>>>> > def length(self)->int:
>>>>>>>> > ...
>>>>>>>> >
>>>>>>>> > # omitted contracts for brevity
>>>>>>>> > def get(self, key: str) -> Optional[str]:
>>>>>>>> > ...
>>>>>>>> >
>>>>>>>> > @before(lambda self, key: {"length": self.length(), "get":
>>>>>>>> > self.get(key)})
>>>>>>>> > @post(lambda self, key, value: self.get(key) == value)
>>>>>>>> > @post(lambda self, key, old: old.get is None and old.length + 1
>>>>>>>> > == self.length(),
>>>>>>>> > "length increased with a new key")
>>>>>>>> > @post(lambda self, key, old: old.get is not None and old.length
>>>>>>>> > == self.length(),
>>>>>>>> > "length stable with an existing key")
>>>>>>>> > def put(self, key: str, value: str) -> None:
>>>>>>>> > ...
>>>>>>>> >
>>>>>>>> > The linter would statically check that all attributes accessed in
>>>>>>>> > "old" have to be defined in the decorator "before" so that attribute
>>>>>>>> > errors would be caught early. The current implementation of the
>>>>>>>> > linter is fast enough to be run at save time so such errors should
>>>>>>>> > usually not happen with a properly set IDE.
>>>>>>>> >
>>>>>>>> > "before" decorator would also have "enabled" property, so that you
>>>>>>>> > can turn it off (e.g., if you only want to run a postcondition in
>>>>>>>> > testing). The "before" decorators can be stacked so that you can
>>>>>>>> > also have a more fine-grained control when each one of them is
>>>>>>>> > running (some during test, some during test and in production). The
>>>>>>>> > linter would enforce that before's "enabled" is a disjunction of all
>>>>>>>> > the "enabled"'s of the corresponding postconditions where the old
>>>>>>>> > value appears.
>>>>>>>> >
>>>>>>>> > Is this a sane approach to "old" values? Any alternative approach
>>>>>>>> > you would prefer? What about better naming? Is "before" a confusing
>>>>>>>> > name?
>>>>>>>>
>>>>>>>> The dict can be splatted into the postconditions, so that no special
>>>>>>>> name is required. This would require either that the lambdas handle
>>>>>>>> **kws, or that their caller inspect them to see what names they take.
>>>>>>>> Perhaps add a function to functools which only passes kwargs that fit.
>>>>>>>> Then the precondition mechanism can pass `self`, `key`, and `value` as
>>>>>>>> kwargs instead of args.
>>>>>>>>
>>>>>>>> For functions that have *args and **kwargs, it may be necessary to
>>>>>>>> pass them to the conditions as args and kwargs instead.
>>>>>>>>
>>>>>>>> The name "before" is a confusing name. It's not just something that
>>>>>>>> happens before. It's really a pre-`let`, adding names to the scope of
>>>>>>>> things after it, but with values taken before the function call. Based
>>>>>>>> on that description, other possible names are `prelet`, `letbefore`,
>>>>>>>> `predef`, `defpre`, `beforescope`. Better a name that is clearly
>>>>>>>> confusing than one that is obvious but misleading.
>>>>>>>>
>>>>>>>> By the way, should the first postcondition be `self.get(key) is
>>>>>>>> value`, checking for identity rather than equality?
>>>>>>> _______________________________________________
>>>>>>> Python-ideas mailing list
>>>>>>> Python-ideas@python.org
>>>>>>> https://mail.python.org/mailman/listinfo/python-ideas
>>>>>>> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
