Re: StopIteration in the if clause of a generator expression
On Apr 5, 2005 2:04 AM, Raymond Hettinger [EMAIL PROTECTED] wrote: [Steven Bethard] So do I read this right in preferring [x for y in z] over list(x for y in z) Yes! Why? (Serious question. I'm sure that you have a good reason - I just can't figure out what it is.) The generator expression has the advantage of not leaking references into the enclosing namespace. What's advantage of the list comp? -- Cheers, Simon B, [EMAIL PROTECTED], http://www.brunningonline.net/simon/blog/ -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
Simon Brunning wrote: On Apr 5, 2005 2:04 AM, Raymond Hettinger [EMAIL PROTECTED] wrote: [Steven Bethard] So do I read this right in preferring [x for y in z] over list(x for y in z) Yes! Why? (Serious question. I'm sure that you have a good reason - I just can't figure out what it is.) The generator expression has the advantage of not leaking references into the enclosing namespace. What's advantage of the list comp? The list comprehension is about 15-20% faster according to timeit.py: C:\Python24\Lib..\python.exe timeit.py -s t = range(1000) [ x for x in t] 1 loops, best of 3: 116 usec per loop C:\Python24\Lib..\python.exe timeit.py -s t = range(1000) list(x for x in t) 1000 loops, best of 3: 144 usec per loop C:\Python24\Lib..\python.exe timeit.py -s t = range(10) [ x for x in t] 10 loops, best of 3: 13.9 msec per loop C:\Python24\Lib..\python.exe timeit.py -s t = range(10) list(x for x in t) 10 loops, best of 3: 16.3 msec per loop Alternatively you could just regard the list comprehension as having less clutter on the screen so it may be clearer. -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
[Steven Bethard] and I often find myself alternating between the two when I can't decide which one seems more Pythonic. Both are pythonic. Use a genexp when you need a generator and use a listcomp when you need a list. Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
Raymond Hettinger wrote: [Steven Bethard] and I often find myself alternating between the two when I can't decide which one seems more Pythonic. Both are pythonic. Use a genexp when you need a generator and use a listcomp when you need a list. So do I read this right in preferring [x for y in z] over list(x for y in z) ? STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
Use a genexp when you need a generator and use a listcomp when you need a list. [Steven Bethard] So do I read this right in preferring [x for y in z] over list(x for y in z) Yes! Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
Raymond Hettinger wrote: (quoting Bengt) I assumed that all standard sequence consumers (including list, of course) would intercept the StopIteration of a sequence given them in the form of a generator expression, so your lyst example would have an analogue for other sequence consumers as well, right? I.e., there's not a hidden list(genex) in those others I would hope ;-) Right. I see I followed the historical evolvement and saw generator expressions as a lazy listcomp rather than a cool new way to write a generator. That turned out to be the road to confusion. Thanks Carl, thanks Raymond for setting me straight. I would call it an unfortunate assymmetry -- one the never comes up unless you're up to no good ;-) Do you see any chance that list comprehensions will be redefined as an alternative spelling for list(generator expression)? Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
jfj wrote: To make it a bit clearer, a StopIteration raised in a generator expression silently terminates that generator: *any* exception raised from a generator, terminates the generator Yeah, but StopIteration is the only expected exception and therefore the only one that client code (nearly) always knows to deal with: def choke(): raise ValueError ... list(i for i in range(10) if i 3 or choke()) Traceback (most recent call last): File stdin, line 1, in ? File stdin, line 1, in generator expression File stdin, line 1, in choke ValueError [i for i in range(10) if i 3 or choke()] Traceback (most recent call last): File stdin, line 1, in ? File stdin, line 1, in choke ValueError Here you can *not* tell apart list(genexp) and listcomp. (Of course, as has since been pointed out, the StopIteration is actually caught in the list constructor, so nothing magic to the example in my initial post) Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
[Peter Otten] Do you see any chance that list comprehensions will be redefined as an alternative spelling for list(generator expression)? Not likely. It is possible that the latter spelling would make it possible for Py3.0. eliminate list comps entirely. However, they are very popular and practical, so my bet is that they will live on. The more likely change is that in Py3.0 list comps will no longer expose the loop variable outside the loop. Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
Raymond Hettinger wrote: [Peter Otten] Do you see any chance that list comprehensions will be redefined as an alternative spelling for list(generator expression)? Not likely. It is possible that the latter spelling would make it possible for Py3.0. eliminate list comps entirely. However, they are very popular and practical, so my bet is that they will live on. I suspect you're right, but I certainly wouldn't complain if list comps disappeared. TOOWTDI and all, and I often find myself alternating between the two when I can't decide which one seems more Pythonic. (These days I generally write a listcomp, but I wouldn't put any money on my code being entirely consistent about this...) STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
This is all just making everything far too complicated. What you really want to do is quite simple: import itertools def condition(x): return x 5 list(itertools.takewhile(condition, (i for i in range(10 The 'Stop Iteration In Generator Expression' problem was solved in the language that List Comprehensions came from, Haskell. Haskell's basic library, prelude, had a series of functions that have found their way into the itertools toolbox. I highly recommend having a read of the itertools docs if you want to continue hacking around with generators. Regards, Stephen Thorne -- http://mail.python.org/mailman/listinfo/python-list
StopIteration in the if clause of a generator expression
To confuse a newbies and old hands alike, Bengt Richter wrote:
Need something more straightforward, e.g., a wrapped one-liner:
def guess(n=3): print (You're right!, 'No more tries for
you!!!')[n-1 in
...(x for x in xrange(n) for t in [raw_input('Guess my name:
')=='Ben']
...if not t or iter([]).next())]
...
guess()
To make it a bit clearer, a StopIteration raised in a generator expression
silently terminates that generator:
def stop(): raise StopIteration
...
list(i for i in range(10) if i 5 or stop())
[0, 1, 2, 3, 4]
In a list comprehension, on the other hand, it is propagated:
[i for i in range(10) if i 5 or stop()]
Traceback (most recent call last):
File stdin, line 1, in ?
File stdin, line 1, in stop
StopIteration
Is that an intentional difference?
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
Peter Otten wrote: To confuse a newbies and old hands alike, Bengt Richter wrote: got me for one:) To make it a bit clearer, a StopIteration raised in a generator expression silently terminates that generator: *any* exception raised from a generator, terminates the generator jfj -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
Peter Otten wrote:
To confuse a newbies and old hands alike, Bengt Richter wrote:
Need something more straightforward, e.g., a wrapped one-liner:
def guess(n=3): print (You're right!, 'No more tries for
you!!!')[n-1 in
...(x for x in xrange(n) for t in [raw_input('Guess my name:
')=='Ben']
...if not t or iter([]).next())]
...
guess()
To make it a bit clearer, a StopIteration raised in a generator
expression
silently terminates that generator:
def stop(): raise StopIteration
...
list(i for i in range(10) if i 5 or stop())
[0, 1, 2, 3, 4]
In a list comprehension, on the other hand, it is propagated:
[i for i in range(10) if i 5 or stop()]
Traceback (most recent call last):
File stdin, line 1, in ?
File stdin, line 1, in stop
StopIteration
Is that an intentional difference?
Very interesting. I'm not sure if the designers even considered this
particular subtlety. Why it happens is pretty plain. In the generator
expression case, the generator expression does propogate the
StopIteration, but list() traps it. List comprehensions are internally
treated as a for-loop (kind of), which doesn't trap StopIteration.
Maybe it should.
The list comprehension [ x for x in y ] is currently treated as
equivalent to the following, with byte-code optimizations:
. _ = []
. for x in y:
. _.append(x)
Perhaps it ought to be equivalent to:
. _ = []
. try:
. for x in y:
. _.append(x)
. except StopIteration:
. pass
However, I would guess the Python gods wouldn't approve of this use of
StopIteration, and so would make no sacrifices to get it.
Nevertheless, it seems likely to be how a list comprehension would
behave in Python 3.0, so maybe we should do it.
--
CARL BANKS
--
http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
[Peter Otten] a StopIteration raised in a generator expression silently terminates that generator: def stop(): raise StopIteration ... list(i for i in range(10) if i 5 or stop()) [0, 1, 2, 3, 4] In a list comprehension, on the other hand, it is propagated: [i for i in range(10) if i 5 or stop()] Traceback (most recent call last): File stdin, line 1, in ? File stdin, line 1, in stop StopIteration Is that an intentional difference? I would call it an unfortunate assymmetry -- one the never comes up unless you're up to no good ;-) In a way, both behave identically. They both raise StopIteration. In the case of the generator expression, that StopIteration is intercepted by the enclosing list() call. That becomes obvious if you write a pure python equivalent for list: def lyst(s): it = iter(s) result = [] try: while 1: result.append(it.next()) except StopIteration:# guess who trapped StopIter return result Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
On Fri, 01 Apr 2005 16:34:32 GMT, Raymond Hettinger [EMAIL PROTECTED] wrote: [Peter Otten] a StopIteration raised in a generator expression silently terminates that generator: def stop(): raise StopIteration ... list(i for i in range(10) if i 5 or stop()) [0, 1, 2, 3, 4] In a list comprehension, on the other hand, it is propagated: [i for i in range(10) if i 5 or stop()] Traceback (most recent call last): File stdin, line 1, in ? File stdin, line 1, in stop StopIteration Is that an intentional difference? I would call it an unfortunate assymmetry -- one the never comes up unless you're up to no good ;-) ;-) In a way, both behave identically. They both raise StopIteration. In the case of the generator expression, that StopIteration is intercepted by the enclosing list() call. That becomes obvious if you write a pure python equivalent for list: def lyst(s): it = iter(s) result = [] try: while 1: result.append(it.next()) except StopIteration:# guess who trapped StopIter return result I assumed that all standard sequence consumers (including list, of course) would intercept the StopIteration of a sequence given them in the form of a generator expression, so your lyst example would have an analogue for other sequence consumers as well, right? I.e., there's not a hidden list(genex) in those others I would hope ;-) E.g., in in my toy exposed more clearly, using Peter's stop: def show(x): print x,; return x ... def stop(): raise StopIteration ... 2 in (x for x in xrange(5) if show(x)4 or stop()) 0 1 2 True 7 in (x for x in xrange(5) if show(x)4 or stop()) 0 1 2 3 4 False BTW I notice that this also nicely shortcuts when the 2 is found. Regards, Bengt Richter -- http://mail.python.org/mailman/listinfo/python-list
Re: StopIteration in the if clause of a generator expression
I assumed that all standard sequence consumers (including list, of course) would intercept the StopIteration of a sequence given them in the form of a generator expression, so your lyst example would have an analogue for other sequence consumers as well, right? I.e., there's not a hidden list(genex) in those others I would hope ;-) Right. E.g., in in my toy exposed more clearly, using Peter's stop: def show(x): print x,; return x ... def stop(): raise StopIteration ... 2 in (x for x in xrange(5) if show(x)4 or stop()) 0 1 2 True 7 in (x for x in xrange(5) if show(x)4 or stop()) 0 1 2 3 4 False BTW I notice that this also nicely shortcuts when the 2 is found. That's a fact. Raymond -- http://mail.python.org/mailman/listinfo/python-list
