Re: [Python-Dev] misbehaving __contains__
On Jan 23, 2008 3:19 AM, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > [Steven Bethard] > > >We've already lost this if anyone really wants to break it:: > > > >>>> class C(object): > >... def __iter__(self): > >... return iter(xrange(3)) > >... def __contains__(self, item): > >... return False [snip] > I'm more concerned about introducing an API where well-meaning attempts to > code a __contains__ override will implicitly shoot the toes off of client > code that innocently assumes a somewhat sensible invariant relation between > looping over elements in a container and those elements being in the > container. The code for sets and frozensets makes that assumption, for > example. And, when the code does break, the errors will be darned hard to > find. well, i don't see how limiting __contains__ to returning booleans solves the problem you mentioned. what you're talking about is *semantics*, and that's always up to the programmer to get right. allowing __contains__ to return a FooBar object won't change that a bit -- but it will allow me to construct expression objects more easily. and, statements like if x in y: pass will work as expected, assuming the object returned by y.__contains__(x) has a proper __bool__/__nonzero__ method. we're just deferring the type-cast to the point it's actually needed. as per performance, i don't believe changing a slot method to return PyObject* instead of int would change anything... am i wrong? -tomer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] misbehaving __contains__
>> And, would we lose the nice relationship expressed by: >> >> for elem in container: >> assert elem in container [Steven Bethard] >We've already lost this if anyone really wants to break it:: > >>>> class C(object): >... def __iter__(self): >... return iter(xrange(3)) >... def __contains__(self, item): >... return False I think this is in the same category as writing a __hash__ that doens't match __eq__. It is explicitly shooting yourself in the foot. I'm more concerned about introducing an API where well-meaning attempts to code a __contains__ override will implicitly shoot the toes off of client code that innocently assumes a somewhat sensible invariant relation between looping over elements in a container and those elements being in the container. The code for sets and frozensets makes that assumption, for example. And, when the code does break, the errors will be darned hard to find. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] misbehaving __contains__
On Jan 22, 2008 5:40 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > [Daniel Stutzbach] > > There are many places in the C implementation where a slot > > returns an int rather than a PyObject. There other replies > > in this thread seem to support altering the signature of the > > slot to return a PyObject. Is this setting a precedent that > > _all_ slots should return a PyObject? > > I hope not. [snip] > And, would we lose the nice relationship expressed by: > > for elem in container: > assert elem in container We've already lost this if anyone really wants to break it:: >>> class C(object): ... def __iter__(self): ... return iter(xrange(3)) ... def __contains__(self, item): ... return False ... >>> c = C() >>> for item in c: ... print item in c ... False False False Of course, anyone who decides to break their container classes in that way is asking for trouble. ;-) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] misbehaving __contains__
[Daniel Stutzbach] > There are many places in the C implementation where a slot > returns an int rather than a PyObject. There other replies > in this thread seem to support altering the signature of the > slot to return a PyObject. Is this setting a precedent that > _all_ slots should return a PyObject? I hope not. One cost of these hypergeneralizations is that it makes Python slower. Also, it makes it harder to write code when you can no longer count on simple predicates to return either True or False. [tomer filiba] > i see no reason it should behave differently than > __eq__ and the rest of comparison operators The rest of the equality operators were a special case to allow rich comparisons which were needed to implement some vector arithmetic idioms. It was thought that those use case had enough benefit to pay for the negative impacts: * making the language more complex * slowing everything down a bit. * complicating the underlying implementation * making the language more difficult to learn (rich operators are harder than cmp) Does making __contains__ return a PyObject have use cases that are worth those costs? It is reminiscent of the ill-fated proposals to overrride 'and', 'or', 'is', and 'is not'. A couple more questions about the proposal: If __contains__ returns 17 in Tomer's example, what would "8 not in f" return? And, would we lose the nice relationship expressed by: for elem in container: assert elem in container It looks like changing the semantics would introduce some weirdness. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] misbehaving __contains__
On Jan 22, 2008 3:46 PM, Daniel Stutzbach <[EMAIL PROTECTED]> wrote: > On Jan 22, 2008 1:26 PM, tomer filiba <[EMAIL PROTECTED]> wrote: > > >>> class Foo(object): > > ... def __contains__(self, key): > > ... return 17 > > ... def __eq__(self, other): > > ... return 19 > > ... > > >>> > > >>> f=Foo() > > >>> f == 8 > > 19 > > >>> 8 in f > > True > > There are many places in the C implementation where a slot returns an > int rather than a PyObject. There other replies in this thread seem > to support altering the signature of the slot to return a PyObject. > Is this setting a precedent that _all_ slots should return a PyObject? > > Consider the following third behavior: > > >>> class foo(object): > ... def __len__(self): > ... return 'foo' > ... > >>> x = foo() > >>> len(x) > Traceback (most recent call last): > File "", line 1, in > TypeError: an integer is required Possibly. I guess it depends on the use case. It would be nice to research this more, e.g. figure out how much code needs to be changed to make each of these possible changes, and how likely there will be a use case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] misbehaving __contains__
On Jan 22, 2008 1:26 PM, tomer filiba <[EMAIL PROTECTED]> wrote: > >>> class Foo(object): > ... def __contains__(self, key): > ... return 17 > ... def __eq__(self, other): > ... return 19 > ... > >>> > >>> f=Foo() > >>> f == 8 > 19 > >>> 8 in f > True There are many places in the C implementation where a slot returns an int rather than a PyObject. There other replies in this thread seem to support altering the signature of the slot to return a PyObject. Is this setting a precedent that _all_ slots should return a PyObject? Consider the following third behavior: >>> class foo(object): ... def __len__(self): ... return 'foo' ... >>> x = foo() >>> len(x) Traceback (most recent call last): File "", line 1, in TypeError: an integer is required -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] misbehaving __contains__
The issue is that the sq_contains slot is defined as returning an int, while the tp_richcompare slot is defined as returning a PyObject *. Your first opportunity for changing this will be Py3k. Please submit a patch! On Jan 22, 2008 11:26 AM, tomer filiba <[EMAIL PROTECTED]> wrote: > i'm using python to create expression objects, for more intuitive > usage as > predicates, for instance: > x = (Arg(0) > 17) & (Arg(1).foo == "bar") > instead of > x = And(Gt(Arg(0), 17), Eq(GetAttr(Arg(1), "foo"), "bar")) > > so now i can use x.eval(18, "spam") and get the result of the > expression. > > i'm doing it by overriding all the operators to return expression > objects, > instead of evaluating the result immediately. it works fine, but i > encountered > a problem with making __contains__ behave so. > > it turns out __contains__ coerces the return value into a bool. this > might > seem logical at first, but is not consistent with the result of the > language. > > consider the following code: > > >>> class Foo(object): > ... def __contains__(self, key): > ... return 17 > ... def __eq__(self, other): > ... return 19 > ... > >>> > >>> f=Foo() > >>> f == 8 > 19 > >>> 8 in f > True > > as you can see, __eq__ does not coerces the result to bool, so why > should > __contains__ do that? > > i've looked at PySequence_Contains in objects/abstract.c, but i can't > quite > understand where the coercion is made. is because the function is > typed > as int? if so, i believe it should either be changed to PyObject, or > have > cmp_outcome (in ceval.c) not use this API directly, and rather return > the > result as it is returned from __contains__. > > i see no reason it should behave differently than __eq__ and the rest > of comparison operators. > > > -tomer > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] misbehaving __contains__
My guess is that results from the bottom of cmp_outcome (ceval.c), The default case handles the PyCmp_EQ case via PyObject_RichCompare, which might not return Py_True or Py_False. But 'in' is handled inside the switch and is subject to the final statements: v = res ? Py_True : Py_False; Py_INCREF(v); return v Though, IANAPydev. C -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of tomer filiba Sent: Tuesday, January 22, 2008 11:27 AM To: python-dev@python.org Subject: [Python-Dev] misbehaving __contains__ i'm using python to create expression objects, for more intuitive usage as predicates, for instance: x = (Arg(0) > 17) & (Arg(1).foo == "bar") instead of x = And(Gt(Arg(0), 17), Eq(GetAttr(Arg(1), "foo"), "bar")) so now i can use x.eval(18, "spam") and get the result of the expression. i'm doing it by overriding all the operators to return expression objects, instead of evaluating the result immediately. it works fine, but i encountered a problem with making __contains__ behave so. it turns out __contains__ coerces the return value into a bool. this might seem logical at first, but is not consistent with the result of the language. consider the following code: >>> class Foo(object): ... def __contains__(self, key): ... return 17 ... def __eq__(self, other): ... return 19 ... >>> >>> f=Foo() >>> f == 8 19 >>> 8 in f True as you can see, __eq__ does not coerces the result to bool, so why should __contains__ do that? i've looked at PySequence_Contains in objects/abstract.c, but i can't quite understand where the coercion is made. is because the function is typed as int? if so, i believe it should either be changed to PyObject, or have cmp_outcome (in ceval.c) not use this API directly, and rather return the result as it is returned from __contains__. i see no reason it should behave differently than __eq__ and the rest of comparison operators. -tomer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/cmason%40vcentertainme nt.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] misbehaving __contains__
i'm using python to create expression objects, for more intuitive usage as predicates, for instance: x = (Arg(0) > 17) & (Arg(1).foo == "bar") instead of x = And(Gt(Arg(0), 17), Eq(GetAttr(Arg(1), "foo"), "bar")) so now i can use x.eval(18, "spam") and get the result of the expression. i'm doing it by overriding all the operators to return expression objects, instead of evaluating the result immediately. it works fine, but i encountered a problem with making __contains__ behave so. it turns out __contains__ coerces the return value into a bool. this might seem logical at first, but is not consistent with the result of the language. consider the following code: >>> class Foo(object): ... def __contains__(self, key): ... return 17 ... def __eq__(self, other): ... return 19 ... >>> >>> f=Foo() >>> f == 8 19 >>> 8 in f True as you can see, __eq__ does not coerces the result to bool, so why should __contains__ do that? i've looked at PySequence_Contains in objects/abstract.c, but i can't quite understand where the coercion is made. is because the function is typed as int? if so, i believe it should either be changed to PyObject, or have cmp_outcome (in ceval.c) not use this API directly, and rather return the result as it is returned from __contains__. i see no reason it should behave differently than __eq__ and the rest of comparison operators. -tomer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
