Re: Dangerous behavior of list(generator)

2010-01-03 Thread Steven D'Aprano
On Wed, 30 Dec 2009 23:20:06 -0500, Benjamin Kaplan wrote: >>> I used to have that a lot in cases where not finding at least one >>> valid foo is an actual fatal error. >> >> What's wrong with the obvious solution? >> >> if not any(foo for foo in foos if foo.bar): >>    raise ValueError('need at l

Re: Dangerous behavior of list(generator)

2010-01-02 Thread Martin v. Loewis
> I'm asking about why the behavior of a StopIteration exception being > handled from the `expression` of a generator expression to mean "stop > the loop" is accepted by "the devs" as acceptable. I may be late to this discussion, but the answer is "most definitely yes". *Any* exception leads to t

Re: Dangerous behavior of list(generator)

2010-01-02 Thread Martin v. Loewis
>> Bottom line, I'm going to have to remove this pattern from my code: >> >> foo = (foo for foo in foos if foo.bar).next() I recommend to rewrite this like so: def first(gen): try: return gen.next() except StopIteration: raise ValueError, "No first value" foo = first(foo for foo in

Re: Dangerous behavior of list(generator)

2010-01-01 Thread Steven D'Aprano
On Fri, 01 Jan 2010 05:19:02 -0800, Wolfram Hinderer wrote: > On 1 Jan., 02:47, Steven D'Aprano cybersource.com.au> wrote: >> On Thu, 31 Dec 2009 11:34:39 -0800, Tom Machinski wrote: [...] >> > As for what's wrong with the "if not any" solution, Benjamin Kaplan's >> > post hits the nail on its he

Re: Dangerous behavior of list(generator)

2010-01-01 Thread Wolfram Hinderer
On 1 Jan., 02:47, Steven D'Aprano wrote: > On Thu, 31 Dec 2009 11:34:39 -0800, Tom Machinski wrote: > > On Wed, Dec 30, 2009 at 4:01 PM, Steven D'Aprano > > wrote: > >> On Wed, 30 Dec 2009 15:18:11 -0800, Tom Machinski wrote: > >>> Bottom line, I'm going to have to remove this pattern from my c

Re: Dangerous behavior of list(generator)

2009-12-31 Thread Steven D'Aprano
On Thu, 31 Dec 2009 11:34:39 -0800, Tom Machinski wrote: > On Wed, Dec 30, 2009 at 4:01 PM, Steven D'Aprano > wrote: >> On Wed, 30 Dec 2009 15:18:11 -0800, Tom Machinski wrote: >>> Bottom line, I'm going to have to remove this pattern from my code: >>> >>>   foo = (foo for foo in foos if foo.bar)

Re: Dangerous behavior of list(generator)

2009-12-31 Thread Tom Machinski
On Thu, Dec 31, 2009 at 12:18 PM, Stephen Hansen wrote: > Hmm? Just use a sentinel which /can't/ exist in the list: then its truly > safe. If the list can contain all the usual sort of sentinels (False, None, > 0, -1, whatever), then just make a unique one all your own. > sentinel = object() > if

Re: Dangerous behavior of list(generator)

2009-12-31 Thread Stephen Hansen
On Thu, Dec 31, 2009 at 11:42 AM, Tom Machinski wrote: > On Thu, Dec 31, 2009 at 1:54 AM, Peter Otten <__pete...@web.de> wrote: > > Somewhat related in 2.6 there's the next() built-in which accepts a > default > > value. You can provide a sentinel and test for that instead of using > > try...excep

Re: Dangerous behavior of list(generator)

2009-12-31 Thread Tom Machinski
On Thu, Dec 31, 2009 at 1:54 AM, Peter Otten <__pete...@web.de> wrote: > Somewhat related in 2.6 there's the next() built-in which accepts a default > value. You can provide a sentinel and test for that instead of using > try...except: Thanks. This can be useful in some of the simpler cases. As yo

Re: Dangerous behavior of list(generator)

2009-12-31 Thread Tom Machinski
On Wed, Dec 30, 2009 at 4:01 PM, Steven D'Aprano wrote: > On Wed, 30 Dec 2009 15:18:11 -0800, Tom Machinski wrote: >> Bottom line, I'm going to have to remove this pattern from my code: >> >>   foo = (foo for foo in foos if foo.bar).next() > > I don't see why. What's wrong with it? Unless you embe

Re: Dangerous behavior of list(generator)

2009-12-31 Thread Peter Otten
Tom Machinski wrote: > It would be nice if there was a builtin for "get the first element in > a genexp, or raise an exception (which isn't StopIteration)", sort of > like: > > from itertools import islice > > def first_or_raise(genexp): > L = list(islice(genexp, 1)) > if not L:

Re: Dangerous behavior of list(generator)

2009-12-30 Thread Benjamin Kaplan
On Wed, Dec 30, 2009 at 7:01 PM, Steven D'Aprano wrote: > > I don't see why. What's wrong with it? Unless you embed it in a call to > list, or similar, it will explicitly raise StopIteration as expected. > > >> I used to have that a lot in cases where not finding at least one valid >> foo is an ac

Re: Dangerous behavior of list(generator)

2009-12-30 Thread Steven D'Aprano
On Wed, 30 Dec 2009 15:18:11 -0800, Tom Machinski wrote: > Thanks for the comment and discussion guys. > > Bottom line, I'm going to have to remove this pattern from my code: > > foo = (foo for foo in foos if foo.bar).next() I don't see why. What's wrong with it? Unless you embed it in a call

Re: Dangerous behavior of list(generator)

2009-12-30 Thread Tom Machinski
Thanks for the comment and discussion guys. Bottom line, I'm going to have to remove this pattern from my code: foo = (foo for foo in foos if foo.bar).next() I used to have that a lot in cases where not finding at least one valid foo is an actual fatal error. But using StopIteration to signal

Re: Dangerous behavior of list(generator)

2009-12-19 Thread Albert van der Horst
In article , Gabriel Genellina wrote: > >Despite a promise in PEP 289, generator expressions semantics isn't >explained in detail in the language reference. I can't tell if the >difference is intentional, accidental, undocumented behavior, an >implementation accident, a bug, or what... Philosoph

Re: Dangerous behavior of list(generator)

2009-12-18 Thread Gregory Ewing
Albert van der Horst wrote: An important feature that is not documented is a severe defect. This isn't something that I would expect to find documented under the heading of generator expressions, because it doesn't have anything to do with them. It's an interaction between the iterator protoco

Re: Dangerous behavior of list(generator)

2009-12-16 Thread Gregory Ewing
exar...@twistedmatrix.com wrote: Which is unfortunate, because it's not that hard to get StopIteration without explicitly raising it yourself and this behavior makes it difficult to debug such situations. It might not be hard if you set out to do it, but in my experience it's pretty rare to en

Re: Dangerous behavior of list(generator)

2009-12-16 Thread Michele Simionato
On Dec 14, 11:05 pm, Carl Banks wrote: > But to answer your question, I think "simple is better than complex" > rules the day.  Right now StopIteration stops an iteration, simple as > that.  Any fix would add complexity. +1 -- http://mail.python.org/mailman/listinfo/python-list

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Steven D'Aprano
On Mon, 14 Dec 2009 15:26:25 -0800, Carl Banks wrote: > On Dec 14, 2:48 pm, Steven D'Aprano cybersource.com.au> wrote: >> On Mon, 14 Dec 2009 14:31:44 +, exarkun wrote: >> > On 06:46 am, tjre...@udel.edu wrote: >> >>On 12/13/2009 10:29 PM, exar...@twistedmatrix.com wrote: >> >>>Doesn't matter

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Carl Banks
On Dec 14, 2:48 pm, Steven D'Aprano wrote: > On Mon, 14 Dec 2009 14:31:44 +, exarkun wrote: > > On 06:46 am, tjre...@udel.edu wrote: > >>On 12/13/2009 10:29 PM, exar...@twistedmatrix.com wrote: > >>>Doesn't matter. Sometimes it makes sense to call it directly. > > >>It only makes sense to call

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Steven D'Aprano
On Mon, 14 Dec 2009 14:31:44 +, exarkun wrote: > On 06:46 am, tjre...@udel.edu wrote: >>On 12/13/2009 10:29 PM, exar...@twistedmatrix.com wrote: >>>Doesn't matter. Sometimes it makes sense to call it directly. >> >>It only makes sense to call next (or .__next__) when you are prepared to >>expl

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Carl Banks
On Dec 14, 7:21 am, exar...@twistedmatrix.com wrote: > Note, I know *why* the implementation leads to this behavior.  I'm > asking why "the devs" *accept* this. As noted, the problem isn't with generators but with iteration protocol. The devs "allowed" this because it was a necessary evil for cor

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Antoine Pitrou
Le Mon, 14 Dec 2009 15:21:09 +, exarkun a écrit : > > I'm asking about why the behavior of a StopIteration exception being > handled from the `expression` of a generator expression to mean "stop > the loop" is accepted by "the devs" as acceptable. It's not "accepted as acceptable", it's just

Re: Dangerous behavior of list(generator)

2009-12-14 Thread exarkun
On 06:00 pm, tjre...@udel.edu wrote: On 12/14/2009 10:21 AM, exar...@twistedmatrix.com wrote: I'm asking about why the behavior of a StopIteration exception being handled from the `expression` of a generator expression to mean "stop the loop" is accepted by "the devs" as acceptable. Any unhand

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Terry Reedy
On 12/14/2009 10:21 AM, exar...@twistedmatrix.com wrote: I'm asking about why the behavior of a StopIteration exception being handled from the `expression` of a generator expression to mean "stop the loop" is accepted by "the devs" as acceptable. Any unhandled exception within a loop stops the

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Mel
exar...@twistedmatrix.com wrote: [ ... ] it's as if a loop like this: > > for a in b: > c > > actually meant this: > > for a in b: > try: > c > except StopIteration: > break > > Note, I know *why* the implementation leads to this behavior.

Re: Dangerous behavior of list(generator)

2009-12-14 Thread exarkun
On 02:58 pm, m...@egenix.com wrote: exar...@twistedmatrix.com wrote: On 08:45 am, tjre...@udel.edu wrote: Tom Machinski wrote: In most cases, `list(generator)` works as expected. Thus, `list()` is generally equivalent to `[ expression>]`. Here's a minimal case where this equivalence breaks,

Re: Dangerous behavior of list(generator)

2009-12-14 Thread M.-A. Lemburg
exar...@twistedmatrix.com wrote: > On 08:45 am, tjre...@udel.edu wrote: >> Tom Machinski wrote: >>> In most cases, `list(generator)` works as expected. Thus, >>> `list()` is generally equivalent to `[>> expression>]`. >>> >>> Here's a minimal case where this equivalence breaks, causing a serious >>

Re: Dangerous behavior of list(generator)

2009-12-14 Thread exarkun
On 06:46 am, tjre...@udel.edu wrote: On 12/13/2009 10:29 PM, exar...@twistedmatrix.com wrote: Doesn't matter. Sometimes it makes sense to call it directly. It only makes sense to call next (or .__next__) when you are prepared to explicitly catch StopIteration within a try..except construct.

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Peter Otten
Terry Reedy wrote: > On 12/13/2009 11:33 PM, exar...@twistedmatrix.com wrote: >> This could provide behavior roughly equivalent to the behavior >> of a list comprehension. > > Impossible. The only serious option for consistency is to special case > list comps to also trap StopIteration raised in

Re: Dangerous behavior of list(generator)

2009-12-14 Thread Lie Ryan
On 12/14/09, exar...@twistedmatrix.com wrote: > On 02:50 am, lie.1...@gmail.com wrote: >>On 12/14/2009 9:45 AM, exar...@twistedmatrix.com wrote: >>>On 08:18 pm, st...@remove-this-cybersource.com.au wrote: On Sun, 13 Dec 2009 14:35:21 +, exarkun wrote: >>StopIteration is intended to be

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Terry Reedy
On 12/13/2009 11:33 PM, exar...@twistedmatrix.com wrote: But if you mistakenly don't catch it, and you're trying to debug your code to find this mistake, you probably won't be aided in this pursuit by the exception-swallowing behavior of generator expressions. As I remember, it was the call to

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Terry Reedy
On 12/13/2009 10:29 PM, exar...@twistedmatrix.com wrote: Doesn't matter. Sometimes it makes sense to call it directly. It only makes sense to call next (or .__next__) when you are prepared to explicitly catch StopIteration within a try..except construct. You did not catch it, so it stopped e

Re: Dangerous behavior of list(generator)

2009-12-13 Thread exarkun
On 04:11 am, ste...@remove.this.cybersource.com.au wrote: On Sun, 13 Dec 2009 22:45:58 +, exarkun wrote: On 08:18 pm, st...@remove-this-cybersource.com.au wrote: On Sun, 13 Dec 2009 14:35:21 +, exarkun wrote: StopIteration is intended to be used only within the .__next__ method of ite

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Steven D'Aprano
On Sun, 13 Dec 2009 22:45:58 +, exarkun wrote: > On 08:18 pm, st...@remove-this-cybersource.com.au wrote: >>On Sun, 13 Dec 2009 14:35:21 +, exarkun wrote: StopIteration is intended to be used only within the .__next__ method of iterators. The devs know that other 'off-label' us

Re: Dangerous behavior of list(generator)

2009-12-13 Thread exarkun
On 02:50 am, lie.1...@gmail.com wrote: On 12/14/2009 9:45 AM, exar...@twistedmatrix.com wrote: On 08:18 pm, st...@remove-this-cybersource.com.au wrote: On Sun, 13 Dec 2009 14:35:21 +, exarkun wrote: StopIteration is intended to be used only within the .__next__ method of iterators. The dev

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Lie Ryan
On 12/14/2009 9:45 AM, exar...@twistedmatrix.com wrote: On 08:18 pm, st...@remove-this-cybersource.com.au wrote: On Sun, 13 Dec 2009 14:35:21 +, exarkun wrote: StopIteration is intended to be used only within the .__next__ method of iterators. The devs know that other 'off-label' use result

Re: Dangerous behavior of list(generator)

2009-12-13 Thread exarkun
On 08:18 pm, st...@remove-this-cybersource.com.au wrote: On Sun, 13 Dec 2009 14:35:21 +, exarkun wrote: StopIteration is intended to be used only within the .__next__ method of iterators. The devs know that other 'off-label' use results in the inconsistency you noted, but their and my view

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Steven D'Aprano
On Sun, 13 Dec 2009 14:35:21 +, exarkun wrote: >>StopIteration is intended to be used only within the .__next__ method of >>iterators. The devs know that other 'off-label' use results in the >>inconsistency you noted, but their and my view is 'don't do that'. > > Which is unfortunate, because

Re: Dangerous behavior of list(generator)

2009-12-13 Thread exarkun
On 08:45 am, tjre...@udel.edu wrote: Tom Machinski wrote: In most cases, `list(generator)` works as expected. Thus, `list()` is generally equivalent to `[]`. Here's a minimal case where this equivalence breaks, causing a serious and hard-to-detect bug in a program: >>> def sit(): raise StopI

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Gabriel Genellina
En Sat, 12 Dec 2009 23:43:20 -0300, Ned Deily escribió: In article , Ned Deily wrote: In article , Benjamin Kaplan wrote: > On Sat, Dec 12, 2009 at 7:15 PM, Tom Machinski > wrote: > > >>> def sit(): raise StopIteration() > > ... > > >>> [f() for f in (lambda:1, sit, lambda:2)]

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Gabriel Genellina
En Sat, 12 Dec 2009 23:43:20 -0300, Ned Deily escribió: In article , Ned Deily wrote: In article , Benjamin Kaplan wrote: > On Sat, Dec 12, 2009 at 7:15 PM, Tom Machinski > wrote: > > >>> def sit(): raise StopIteration() > > ... > > >>> [f() for f in (lambda:1, sit, lambda:2)]

Re: Dangerous behavior of list(generator)

2009-12-13 Thread Terry Reedy
Tom Machinski wrote: In most cases, `list(generator)` works as expected. Thus, `list()` is generally equivalent to `[]`. Here's a minimal case where this equivalence breaks, causing a serious and hard-to-detect bug in a program: >>> def sit(): raise StopIteration() StopIteration is intended

Re: Dangerous behavior of list(generator)

2009-12-12 Thread Ned Deily
In article , Ned Deily wrote: > In article > , > Benjamin Kaplan wrote: > > On Sat, Dec 12, 2009 at 7:15 PM, Tom Machinski > > wrote: > > > In most cases, `list(generator)` works as expected. Thus, > > > `list()` is generally equivalent to `[ > > expression>]`. > > Actually, it's list(gener

Re: Dangerous behavior of list(generator)

2009-12-12 Thread Benjamin Kaplan
On Sat, Dec 12, 2009 at 9:01 PM, Ned Deily wrote: > In article > , >  Benjamin Kaplan wrote: >> On Sat, Dec 12, 2009 at 7:15 PM, Tom Machinski >> wrote: >> > In most cases, `list(generator)` works as expected. Thus, >> > `list()` is generally equivalent to `[> > expression>]`. >> Actually, it's

Re: Dangerous behavior of list(generator)

2009-12-12 Thread Ned Deily
In article , Benjamin Kaplan wrote: > On Sat, Dec 12, 2009 at 7:15 PM, Tom Machinski > wrote: > > In most cases, `list(generator)` works as expected. Thus, > > `list()` is generally equivalent to `[ > expression>]`. > Actually, it's list(generator) vs. a list comprehension. I agree that > it c

Re: Dangerous behavior of list(generator)

2009-12-12 Thread Benjamin Kaplan
On Sat, Dec 12, 2009 at 7:15 PM, Tom Machinski wrote: > In most cases, `list(generator)` works as expected. Thus, > `list()` is generally equivalent to `[ expression>]`. > Actually, it's list(generator) vs. a list comprehension. I agree that it can be confusing, but Python considers them to be tw

Dangerous behavior of list(generator)

2009-12-12 Thread Tom Machinski
In most cases, `list(generator)` works as expected. Thus, `list()` is generally equivalent to `[]`. Here's a minimal case where this equivalence breaks, causing a serious and hard-to-detect bug in a program: >>> def sit(): raise StopIteration() ... >>> [f() for f in (lambda:1, sit, lambda:2