I am fine with your proposed syntax. It’s certainly lucid. Perhaps it would be a good idea to get people accustomed to “non-magic” syntax.
> I still have a feeling that most developers would like to store the state in > many different custom ways. Please explain. (Expressions like thunk(all)(a == b for a, b in P.arg.meth()) would be valid.) > I'm thinking mostly about all the edge cases which we would not be able to > cover (and how complex that would be to cover them). Except for a > b > c being one flat expression with 5 members, it seems fairly easy to recreate an AST, which can then be compiled down to a code object. The code object can be fun with a custom “locals()” Below is my concept code for such a P object. from ast import * # not done: enforce Singleton property on EmptySymbolType class EmptySymbolType(object): ... EmptySymbol = EmptySymbolType() # empty symbols are placeholders class MockP(object): # "^" is xor @icontract.pre(lambda symbol, astnode: (symbol is None) ^ (astnode is None)) def __init__(self, symbol=None, value=EmptySymbol, astnode=None, initsymtable=(,)): self.symtable = dict(initsymtable) if symbol: self.expr = Expr(value=Name(id=symbol, ctx=Load())) self.symtable = {symbol: value} else: self.expr = astnode self.frozen = False def __add__(self, other): wrapped = MockP.wrap_value(other) return MockP(astnode=Expr(value=BinOp(self.expr, Add(), wrapped.expr), initsymtable={**self.symtable, **wrapped.symtable}) def compile(self): ... def freeze(self): # frozen objects wouldn’t have an overrided getattr, allowing for icontract to manipulate the MockP object using its public interface self.frozen = True @classmethod def wrap_value(cls, obj): # create a MockP object from a value. Generate a random identifier and set that as the key in symtable, the AST node is the name of that identifier, retrieving its value through simple expression evaluation. ... thunk = MockP.wrap_value P = MockP('P') # elsewhere: ensure P is only accessed via valid “dot attribute access” inside @snapshot so contracts fail early, or don’t and allow Magic like __dict__ to occur on P. > On Sep 27, 2018, at 9:49 PM, Marko Ristin-Kaufmann <marko.ris...@gmail.com> > wrote: > > Hi James, > > I still have a feeling that most developers would like to store the state in > many different custom ways. I see also thunk and snapshot with wrapper > objects to be much more complicated to implement and maintain; I'm thinking > mostly about all the edge cases which we would not be able to cover (and how > complex that would be to cover them). Then the linters need also to work > around such wrappers... It might also scare users off since it looks like too > much magic. Another concern I also have is that it's probably very hard to > integrate these wrappers with mypy later -- but I don't really have a clue > about that, only my gut feeling? > > What about we accepted to repeat "lambda P, " prefix, and have something like > this: > > @snapshot( > lambda P, some_name: len(P.some_property), > lambda P, another_name: hash(P.another_property) > ) > > It's not too verbose for me and you can still explain in three-four sentences > what happens below the hub in the library's docs. A pycharm/pydev/vim/emacs > plugins could hide the verbose parts. > > I performed a small experiment to test how this solution plays with pylint > and it seems OK that arguments are not used in lambdas. > > Cheers, > Marko > > >> On Thu, 27 Sep 2018 at 12:27, James Lu <jam...@gmail.com> wrote: >> Why couldn’t we record the operations done to a special object and replay >> them? >> >>>>> Actually, I think there is probably no way around a decorator that >>>>> captures/snapshots the data before the function call with a lambda (or >>>>> even a separate function). "Old" construct, if we are to parse it somehow >>>>> from the condition function, would limit us only to shallow copies (and >>>>> be complex to implement as soon as we are capturing out-of-argument >>>>> values such as globals etc.). Moreove, what if we don't need shallow >>>>> copies? I could imagine a dozen of cases where shallow copy is not what >>>>> the programmer wants: for example, s/he might need to make deep copies, >>>>> hash or otherwise transform the input data to hold only part of it >>>>> instead of copying (e.g., so as to allow equality check without a double >>>>> copy of the data, or capture only the value of certain property >>>>> transformed in some way). >> >> >> from icontract import snapshot, P, thunk >> @snapshot(some_identifier=P.self.some_method(P.some_argument.some_attr)) >> >> P is an object of our own type, let’s call the type MockP. MockP returns new >> MockP objects when any operation is done to it. MockP * MockP = MockP. >> MockP.attr = MockP. MockP objects remember all the operations done to them, >> and allow the owner of a MockP object to re-apply the same operations >> >> “thunk” converts a function or object or class to a MockP object, storing >> the function or object for when the operation is done. >> >> thunk(function)(<MockP expression>) >> >> Of course, you could also thunk objects like so: thunk(3) * P.number. >> (Though it might be better to keep the 3 after P.number in this case so >> P.number’s __mult__ would be invoked before 3’s __mult__ is invokes. >> >> >> In most cases, you’d save any operations that can be done on a copy of the >> data as generated by @snapshot in @postcondiion. thunk is for rare scenarios >> where 1) it’s hard to capture the state, for example an object that manages >> network state (or database connectivity etc) and whose stage can only be >> read by an external classmethod 2) you want to avoid using copy.deepcopy. >> >> I’m sure there’s some way to override isinstance through a meta class or >> dunder subclasshook. >> >> I suppose this mocking method could be a shorthand for when you don’t need >> the full power of a lambda. It’s arguably more succinct and readable, though >> YMMV. >> >> I look forward to reading your opinion on this and any ideas you might have. >> >>> On Sep 26, 2018, at 3:56 PM, James Lu <jam...@gmail.com> wrote: >>> >>> Hi Marko, >>> >>>> Actually, following on #A4, you could also write those as multiple >>>> decorators: >>>> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr) >>>> @snpashot(lambda _, other_identifier: other_func(_.self)) >>> >>> Yes, though if we’re talking syntax using kwargs would probably be better. >>> Using “P” instead of “_”: (I agree that _ smells of ignored arguments) >>> >>> @snapshot(some_identifier=lambda P: ..., some_identifier2=lambda P: ...) >>> >>> Kwargs has the advantage that you can extend multiple lines without >>> repeating @snapshot, though many lines of @capture would probably be more >>> intuitive since each decorator captures one variable. >>> >>>> Why uppercase "P" and not lowercase (uppercase implies a constant for me)? >>> >>> To me, the capital letters are more prominent and explicit- easier to see >>> when reading code. It also implies its a constant for you- you shouldn’t be >>> modifying it, because then you’d be interfering with the function itself. >>> >>> Side node: maybe it would be good to have an @icontract.nomutate (probably >>> use a different name, maybe @icontract.readonly) that makes sure a method >>> doesn’t mutate its own __dict__ (and maybe the __dict__ of the members of >>> its __dict__). It wouldn’t be necessary to put the decorator on every read >>> only function, just the ones your worried might mutate. >>> >>> Maybe a @icontract.nomutate(param=“paramname”) that ensures the __dict__ of >>> all members of the param name have the same equality or identity before and >>> after. The semantics would need to be worked out. >>> >>>> On Sep 26, 2018, at 8:58 AM, Marko Ristin-Kaufmann >>>> <marko.ris...@gmail.com> wrote: >>>> >>>> Hi James, >>>> >>>> Actually, following on #A4, you could also write those as multiple >>>> decorators: >>>> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr) >>>> @snpashot(lambda _, other_identifier: other_func(_.self)) >>>> >>>> Am I correct? >>>> >>>> "_" looks a bit hard to read for me (implying ignored arguments). >>>> >>>> Why uppercase "P" and not lowercase (uppercase implies a constant for me)? >>>> Then "O" for "old" and "P" for parameters in a condition: >>>> @post(lambda O, P: ...) >>>> ? >>>> >>>> It also has the nice property that it follows both the temporal and the >>>> alphabet order :) >>>> >>>>> On Wed, 26 Sep 2018 at 14:30, James Lu <jam...@gmail.com> wrote: >>>>> I still prefer snapshot, though capture is a good name too. We could use >>>>> generator syntax and inspect the argument names. >>>>> >>>>> Instead of “a”, perhaps use “_”. Or maybe use “A.”, for arguments. Some >>>>> people might prefer “P” for parameters, since parameters sometimes means >>>>> the value received while the argument means the value passed. >>>>> >>>>> (#A1) >>>>> >>>>> from icontract import snapshot, __ >>>>> @snapshot(some_func(_.some_argument.some_attr) for some_identifier, _ in >>>>> __) >>>>> >>>>> Or (#A2) >>>>> >>>>> @snapshot(some_func(some_argument.some_attr) for some_identifier, _, >>>>> some_argument in __) >>>>> >>>>> — >>>>> Or (#A3) >>>>> >>>>> @snapshot(lambda some_argument,_,some_identifier: >>>>> some_func(some_argument.some_attr)) >>>>> >>>>> Or (#A4) >>>>> >>>>> @snapshot(lambda _,some_identifier: some_func(_.some_argument.some_attr)) >>>>> @snapshot(lambda _,some_identifier, other_identifier: >>>>> some_func(_.some_argument.some_attr), other_func(_.self)) >>>>> >>>>> I like #A4 the most because it’s fairly DRY and avoids the extra >>>>> punctuation of >>>>> >>>>> @capture(lambda a: {"some_identifier": >>>>> some_func(a.some_argument.some_attr)}) >>>>> >>>>> On Sep 26, 2018, at 12:23 AM, Marko Ristin-Kaufmann >>>>> <marko.ris...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Franklin wrote: >>>>>>> The name "before" is a confusing name. It's not just something that >>>>>>> happens before. It's really a pre-`let`, adding names to the scope of >>>>>>> things after it, but with values taken before the function call. Based >>>>>>> on that description, other possible names are `prelet`, `letbefore`, >>>>>>> `predef`, `defpre`, `beforescope`. Better a name that is clearly >>>>>>> confusing than one that is obvious but misleading. >>>>>> >>>>>> James wrote: >>>>>>> I suggest that instead of “@before” it’s “@snapshot” and instead of >>>>>>> “old” it’s “snapshot”. >>>>>> >>>>>> >>>>>> I like "snapshot", it's a bit clearer than prefixing/postfixing verbs >>>>>> with "pre" which might be misread (e.g., "prelet" has a meaning in >>>>>> Slavic languages and could be subconsciously misread, "predef" implies >>>>>> to me a pre-definition rather than prior-to-definition , "beforescope" >>>>>> is very clear for me, but it might be confusing for others as to what it >>>>>> actually refers to ). What about "@capture" (7 letters for captures >>>>>> versus 8 for snapshot)? I suppose "@let" would be playing with fire if >>>>>> Python with conflicting new keywords since I assume "let" to be one of >>>>>> the candidates. >>>>>> >>>>>> Actually, I think there is probably no way around a decorator that >>>>>> captures/snapshots the data before the function call with a lambda (or >>>>>> even a separate function). "Old" construct, if we are to parse it >>>>>> somehow from the condition function, would limit us only to shallow >>>>>> copies (and be complex to implement as soon as we are capturing >>>>>> out-of-argument values such as globals etc.). Moreove, what if we don't >>>>>> need shallow copies? I could imagine a dozen of cases where shallow copy >>>>>> is not what the programmer wants: for example, s/he might need to make >>>>>> deep copies, hash or otherwise transform the input data to hold only >>>>>> part of it instead of copying (e.g., so as to allow equality check >>>>>> without a double copy of the data, or capture only the value of certain >>>>>> property transformed in some way). >>>>>> >>>>>> I'd still go with the dictionary to allow for this extra freedom. We >>>>>> could have a convention: "a" denotes to the current arguments, and "b" >>>>>> denotes the captured values. It might make an interesting hint that we >>>>>> put "b" before "a" in the condition. You could also interpret "b" as >>>>>> "before" and "a" as "after", but also "a" as "arguments". >>>>>> >>>>>> @capture(lambda a: {"some_identifier": >>>>>> some_func(a.some_argument.some_attr)}) >>>>>> @post(lambda b, a, result: b.some_identifier > result + >>>>>> a.another_argument.another_attr) >>>>>> def some_func(some_argument: SomeClass, another_argument: AnotherClass) >>>>>> -> SomeResult: >>>>>> ... >>>>>> "b" can be omitted if it is not used. Under the hub, all the arguments >>>>>> to the condition would be passed by keywords. >>>>>> >>>>>> In case of inheritance, captures would be inherited as well. Hence the >>>>>> library would check at run-time that the returned dictionary with >>>>>> captured values has no identifier that has been already captured, and >>>>>> the linter checks that statically, before running the code. Reading >>>>>> values captured in the parent at the code of the child class might be a >>>>>> bit hard -- but that is case with any inherited methods/properties. In >>>>>> documentation, I'd list all the captures of both ancestor and the >>>>>> current class. >>>>>> >>>>>> I'm looking forward to reading your opinion on this and alternative >>>>>> suggestions :) >>>>>> Marko >>>>>> >>>>>>> On Tue, 25 Sep 2018 at 18:12, Franklin? Lee >>>>>>> <leewangzhong+pyt...@gmail.com> wrote: >>>>>>> On Sun, Sep 23, 2018 at 2:05 AM Marko Ristin-Kaufmann >>>>>>> <marko.ris...@gmail.com> wrote: >>>>>>> > >>>>>>> > Hi, >>>>>>> > >>>>>>> > (I'd like to fork from a previous thread, "Pre-conditions and >>>>>>> > post-conditions", since it got long and we started discussing a >>>>>>> > couple of different things. Let's discuss in this thread the >>>>>>> > implementation of a library for design-by-contract and how to push it >>>>>>> > forward to hopefully add it to the standard library one day.) >>>>>>> > >>>>>>> > For those unfamiliar with contracts and current state of the >>>>>>> > discussion in the previous thread, here's a short summary. The >>>>>>> > discussion started by me inquiring about the possibility to add >>>>>>> > design-by-contract concepts into the core language. The idea was >>>>>>> > rejected by the participants mainly because they thought that the >>>>>>> > merit of the feature does not merit its costs. This is quite >>>>>>> > debatable and seems to reflect many a discussion about >>>>>>> > design-by-contract in general. Please see the other thread, "Why is >>>>>>> > design-by-contract not widely adopted?" if you are interested in that >>>>>>> > debate. >>>>>>> > >>>>>>> > We (a colleague of mine and I) decided to implement a library to >>>>>>> > bring design-by-contract to Python since we don't believe that the >>>>>>> > concept will make it into the core language anytime soon and we >>>>>>> > needed badly a tool to facilitate our work with a growing code base. >>>>>>> > >>>>>>> > The library is available at http://github.com/Parquery/icontract. The >>>>>>> > hope is to polish it so that the wider community could use it and >>>>>>> > once the quality is high enough, make a proposal to add it to the >>>>>>> > standard Python libraries. We do need a standard library for >>>>>>> > contracts, otherwise projects with conflicting contract libraries can >>>>>>> > not integrate (e.g., the contracts can not be inherited between two >>>>>>> > different contract libraries). >>>>>>> > >>>>>>> > So far, the most important bits have been implemented in icontract: >>>>>>> > >>>>>>> > Preconditions, postconditions, class invariants >>>>>>> > Inheritance of the contracts (including strengthening and weakening >>>>>>> > of the inherited contracts) >>>>>>> > Informative violation messages (including information about the >>>>>>> > values involved in the contract condition) >>>>>>> > Sphinx extension to include contracts in the automatically generated >>>>>>> > documentation (sphinx-icontract) >>>>>>> > Linter to statically check that the arguments of the conditions are >>>>>>> > correct (pyicontract-lint) >>>>>>> > >>>>>>> > We are successfully using it in our code base and have been quite >>>>>>> > happy about the implementation so far. >>>>>>> > >>>>>>> > There is one bit still missing: accessing "old" values in the >>>>>>> > postcondition (i.e., shallow copies of the values prior to the >>>>>>> > execution of the function). This feature is necessary in order to >>>>>>> > allow us to verify state transitions. >>>>>>> > >>>>>>> > For example, consider a new dictionary class that has "get" and "put" >>>>>>> > methods: >>>>>>> > >>>>>>> > from typing import Optional >>>>>>> > >>>>>>> > from icontract import post >>>>>>> > >>>>>>> > class NovelDict: >>>>>>> > def length(self)->int: >>>>>>> > ... >>>>>>> > >>>>>>> > def get(self, key: str) -> Optional[str]: >>>>>>> > ... >>>>>>> > >>>>>>> > @post(lambda self, key, value: self.get(key) == value) >>>>>>> > @post(lambda self, key: old(self.get(key)) is None and >>>>>>> > old(self.length()) + 1 == self.length(), >>>>>>> > "length increased with a new key") >>>>>>> > @post(lambda self, key: old(self.get(key)) is not None and >>>>>>> > old(self.length()) == self.length(), >>>>>>> > "length stable with an existing key") >>>>>>> > def put(self, key: str, value: str) -> None: >>>>>>> > ... >>>>>>> > >>>>>>> > How could we possible implement this "old" function? >>>>>>> > >>>>>>> > Here is my suggestion. I'd introduce a decorator "before" that would >>>>>>> > allow you to store whatever values in a dictionary object "old" (i.e. >>>>>>> > an object whose properties correspond to the key/value pairs). The >>>>>>> > "old" is then passed to the condition. Here is it in code: >>>>>>> > >>>>>>> > # omitted contracts for brevity >>>>>>> > class NovelDict: >>>>>>> > def length(self)->int: >>>>>>> > ... >>>>>>> > >>>>>>> > # omitted contracts for brevity >>>>>>> > def get(self, key: str) -> Optional[str]: >>>>>>> > ... >>>>>>> > >>>>>>> > @before(lambda self, key: {"length": self.length(), "get": >>>>>>> > self.get(key)}) >>>>>>> > @post(lambda self, key, value: self.get(key) == value) >>>>>>> > @post(lambda self, key, old: old.get is None and old.length + 1 >>>>>>> > == self.length(), >>>>>>> > "length increased with a new key") >>>>>>> > @post(lambda self, key, old: old.get is not None and old.length >>>>>>> > == self.length(), >>>>>>> > "length stable with an existing key") >>>>>>> > def put(self, key: str, value: str) -> None: >>>>>>> > ... >>>>>>> > >>>>>>> > The linter would statically check that all attributes accessed in >>>>>>> > "old" have to be defined in the decorator "before" so that attribute >>>>>>> > errors would be caught early. The current implementation of the >>>>>>> > linter is fast enough to be run at save time so such errors should >>>>>>> > usually not happen with a properly set IDE. >>>>>>> > >>>>>>> > "before" decorator would also have "enabled" property, so that you >>>>>>> > can turn it off (e.g., if you only want to run a postcondition in >>>>>>> > testing). The "before" decorators can be stacked so that you can also >>>>>>> > have a more fine-grained control when each one of them is running >>>>>>> > (some during test, some during test and in production). The linter >>>>>>> > would enforce that before's "enabled" is a disjunction of all the >>>>>>> > "enabled"'s of the corresponding postconditions where the old value >>>>>>> > appears. >>>>>>> > >>>>>>> > Is this a sane approach to "old" values? Any alternative approach you >>>>>>> > would prefer? What about better naming? Is "before" a confusing name? >>>>>>> >>>>>>> The dict can be splatted into the postconditions, so that no special >>>>>>> name is required. This would require either that the lambdas handle >>>>>>> **kws, or that their caller inspect them to see what names they take. >>>>>>> Perhaps add a function to functools which only passes kwargs that fit. >>>>>>> Then the precondition mechanism can pass `self`, `key`, and `value` as >>>>>>> kwargs instead of args. >>>>>>> >>>>>>> For functions that have *args and **kwargs, it may be necessary to >>>>>>> pass them to the conditions as args and kwargs instead. >>>>>>> >>>>>>> The name "before" is a confusing name. It's not just something that >>>>>>> happens before. It's really a pre-`let`, adding names to the scope of >>>>>>> things after it, but with values taken before the function call. Based >>>>>>> on that description, other possible names are `prelet`, `letbefore`, >>>>>>> `predef`, `defpre`, `beforescope`. Better a name that is clearly >>>>>>> confusing than one that is obvious but misleading. >>>>>>> >>>>>>> By the way, should the first postcondition be `self.get(key) is >>>>>>> value`, checking for identity rather than equality? >>>>>> _______________________________________________ >>>>>> Python-ideas mailing list >>>>>> Python-ideas@python.org >>>>>> https://mail.python.org/mailman/listinfo/python-ideas >>>>>> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/