Hi Armin and Alex,
> This is (if we are positive) an internal implementation detail and (if
> we are negative) a bug in CPython. There is no way to define in pure
> Python a list-like type that would have the exact same behavior.
I don't disagree with this -- indeed, I think a strict reading of the
language reference would clearly identify this as wrong behavior by
CPython.
Nevertheless, real-world programs do exist that rely on this behavior.
I don't have a broad survey of real programs (I don't think anyone
does), but this issue causes Quora's codebase to fail a ton of unit
tests and basically not work at all under PyPy without modification.
The reason is that we have this idiom in thousands of places:
def f():
z = []
z += Foo()
z += Bar()
for x in something():
z += Baz(x)
return z
When the classes Foo, Bar, Baz, etc were implemented, the author gave
them __radd__ methods to implement the above behavior -- after all,
addition is precisely what we're doing here, right? So they look like
class Foo(object):
def __radd__(self, other):
other.append("something")
return other
as in my original mail.
Now, knowing how this all works, it's not too hard to fix my company's
code so that it works here -- I just add an __iter__ method like so:
class Foo(object):
[...]
def __iter__(self):
yield "something"
And if the original author had been developing under PyPy, no doubt he
would have seen the TypeError, accepted the behavior as is, and
quickly figured out this approach. Programmers adapt to all kinds of
details in their development environments, whether those details are
justified well or poorly.
But for someone considering PyPy and bringing over an existing
codebase, it's awfully disconcerting to see a bunch of tests fail and
have to go debugging. Especially so when it's pure Python and doesn't
muck around with GC, or in sys, or use anything that a non-expert
would recognize as an implementation detail. I know I'm an early,
early adopter and so I don't mind, but for the broader class of users
that I hope PyPy acquires in the next few years, an issue like this
would be a pretty negative signal.
As I outlined at the end of my first email, I don't think this would
require a great deal of code. I appreciate that there's a cost to
making things more complex, but I think it's worth it to make the way
smooth for people moving from CPython.
Let me know what you think. I'm happy to write up a patch.
Greg
On Thu, Mar 10, 2011 at 12:53 PM, Armin Rigo <ar...@tunes.org> wrote:
> Hi Greg,
>
> On Wed, Mar 9, 2011 at 7:13 PM, Greg Price <g...@quora.com> wrote:
>> The following program works in CPython, but fails in PyPy:
>
> This is (if we are positive) an internal implementation detail and (if
> we are negative) a bug in CPython. There is no way to define in pure
> Python a list-like type that would have the exact same behavior. We
> already have one such special case for __add__/__radd__ on subclasses
> of strings and unicodes, which appears to be the use case that people
> rely on most often, but I don't see the point in duplicating this
> strange behavior for lists, tuples, and so on, unless there are really
> several programs out there that rely on it.
>
>
> A bientôt,
>
> Armin.
>
_______________________________________________
pypy-dev@codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev
