Re: Why are broken iterators broken?
On Sep 21, 10:13 pm, Steven D'Aprano [EMAIL PROTECTED] cybersource.com.au wrote: According to the Python docs, once an iterator raises StopIteration, it should continue to raise StopIteration forever. Iterators that fail to behave in this fashion are deemed to be broken: http://docs.python.org/lib/typeiter.html I don't understand the reasoning behind this. As I understand it, an iterator is something like a stream. There's no constraint that once a stream is empty it must remain empty forever. I think empty != StopIteration. StopIteration (IMHO) shouldn't be raised when the stream is empty, instead a sentinel value specifying that there is no data yet, but if you wait there might be should be returned (possibly None or empty string). When you raise StopIteration, it is a signal that I don't have any more data and there is no use in waiting. Can somebody explain why broken iterators are broken? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
Craig Allen wrote: In the end I interpreted that statement as if unless __iter__() is called again It seems you were confusing iterators and iterables. The iterator is the object that is returned by calling __iter__() on an iterable, and yes, you are expected to get a new one each time you want to start iterating again. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
I'm interested what others think of this because at first I couldn't get it... I have an object which can iterate over its parts... and at first I thought, what? I'm supposed to create a new object every time the user needs to iterate the contents? In the end I interpreted that statement as if unless __iter__() is called again, in which case it makes sense that an iterator should stay finished until it's told to start to iterate again. Then even the stream analogy holds, you don't expect a stream to say EOF then start giving you bytes (from beyond the end, I guess)... instead, such a stream would more likely block if it got to the end of available data. I'm probably being stupid in this... perhaps I'm the only one that at first interpreted the phrase as being regardless of a fresh call to __iter__()... and that the OP was worried about some other implication. But if I was wrong to think it's ok for one and the same iterator to reset when __iter__() is called again, then I seriously don't understand. what does forever mean in that dictum? -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
Cameron Simpson wrote: you probably want the consumer thread to block when it catches up with the producer, rather than exit. It sounds like he wants non-blocking behaviour in his consumer. Roy gave an example, he didn't post a requirements specification. A common example is try to gather a lot of stuff into a single packet, but send a smaller packet promptly if there isn't much stuff. that use case is better solved with a plain list object. no need to make things harder than they are. /F -- http://mail.python.org/mailman/listinfo/python-list
Why are broken iterators broken?
According to the Python docs, once an iterator raises StopIteration, it should continue to raise StopIteration forever. Iterators that fail to behave in this fashion are deemed to be broken: http://docs.python.org/lib/typeiter.html I don't understand the reasoning behind this. As I understand it, an iterator is something like a stream. There's no constraint that once a stream is empty it must remain empty forever. Can somebody explain why broken iterators are broken? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
Steven D'Aprano wrote: According to the Python docs, once an iterator raises StopIteration, it should continue to raise StopIteration forever. Iterators that fail to behave in this fashion are deemed to be broken: http://docs.python.org/lib/typeiter.html I don't understand the reasoning behind this. As I understand it, an iterator is something like a stream. There's no constraint that once a stream is empty it must remain empty forever. it's a design guideline, not an absolute rule. but I disagree that an iterator is something like a stream. it's rather something like a pointer or an index, that is, an object that helps you iterate over all members in a collection. /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
In article [EMAIL PROTECTED], Fredrik Lundh [EMAIL PROTECTED] wrote: Steven D'Aprano wrote: According to the Python docs, once an iterator raises StopIteration, it should continue to raise StopIteration forever. Iterators that fail to behave in this fashion are deemed to be broken: http://docs.python.org/lib/typeiter.html I don't understand the reasoning behind this. As I understand it, an iterator is something like a stream. There's no constraint that once a stream is empty it must remain empty forever. it's a design guideline, not an absolute rule. but I disagree that an iterator is something like a stream. it's rather something like a pointer or an index, that is, an object that helps you iterate over all members in a collection. /F There are plausible examples of collections which grow while you're iterating over them. I'm thinking specifically of a queue in a multi-threaded application. One thread pushes work onto the back of the queue while another pops from the front. The queue could certainly go empty at times. But, maybe a Python iterator is just the wrong way to model such behavior. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
Roy Smith wrote: There are plausible examples of collections which grow while you're iterating over them. I'm thinking specifically of a queue in a multi-threaded application. One thread pushes work onto the back of the queue while another pops from the front. The queue could certainly go empty at times. But, maybe a Python iterator is just the wrong way to model such behavior. you probably want the consumer thread to block when it catches up with the producer, rather than exit. (that's the default behaviour of Python's Queue object, btw) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
Steven D'Aprano wrote: According to the Python docs, once an iterator raises StopIteration, it should continue to raise StopIteration forever. Iterators that fail to behave in this fashion are deemed to be broken: http://docs.python.org/lib/typeiter.html I don't understand the reasoning behind this. As I understand it, an iterator is something like a stream. There's no constraint that once a stream is empty it must remain empty forever. It is quite possible that a stream reader will return '' on one call and then something non-empty the next. An iterator that reads a stream and yields chunks of whatever size should either block until it gets sufficient data or yield nulls as long as the stream is open and not raise StopIteration until the steam is closed and it has yielded the last chunk of data. Can somebody explain why broken iterators are broken? There is an important different between a store that is closed until the next day and one that closed - out of business. Similarly, there is a difference between an item being out-of-stock until the next delivery and out-of-stock and discontinued permanently, or between a road closed for repairs versus removal for something else. Using the same sign or signal for temporary and permanent conditions is confusing and therefore 'broken'. Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
On 21Sep2008 18:36, Fredrik Lundh [EMAIL PROTECTED] wrote: Roy Smith wrote: There are plausible examples of collections which grow while you're iterating over them. I'm thinking specifically of a queue in a multi-threaded application. One thread pushes work onto the back of the queue while another pops from the front. The queue could certainly go empty at times. But, maybe a Python iterator is just the wrong way to model such behavior. you probably want the consumer thread to block when it catches up with the producer, rather than exit. (that's the default behaviour of Python's Queue object, btw) It sounds like he wants non-blocking behaviour in his consumer. A common example is try to gather a lot of stuff into a single packet, but send a smaller packet promptly if there isn't much stuff. You could make the next() method return a sentinal value like None when the queue is empty. That would mean your consumer must recognise the special value and also precludes delivering that value through the queue. I'm not convinced my suggestion here is any better than just doubling up every call to next() with an empty() check immediately beforehand. You could write a trivial wrapping generator to take the original blocking queue and return a sentinel value on empty, too. My suggestion is also an excellent way of getting programs that fail-busy (i.e. they spin out) if you make a logic error in your consumer. Ouch. Cheers, -- Cameron Simpson [EMAIL PROTECTED] DoD#743 http://www.cskk.ezoshosting.com/cs/ Kill, v.t. To create a vacancy without nominating a successor. Ambrose Bierce (1842-1914), U.S. author. The Devil's Dictionary (1881-1906). -- http://mail.python.org/mailman/listinfo/python-list
Re: Why are broken iterators broken?
On Sun, Sep 21, 2008 at 11:13 AM, Steven D'Aprano
[EMAIL PROTECTED] wrote:
According to the Python docs, once an iterator raises StopIteration, it
should continue to raise StopIteration forever. Iterators that fail to
behave in this fashion are deemed to be broken:
http://docs.python.org/lib/typeiter.html
I don't understand the reasoning behind this. As I understand it, an
iterator is something like a stream. There's no constraint that once a
stream is empty it must remain empty forever.
Can somebody explain why broken iterators are broken?
It's not a terribly onerous restriction. If you're iterating with a
for-loop, you can make the iterable return a new iterator object when
the old one is exhausted, and if the intent is for the next()-method
to be called directly, you don't have to conform to the iterator
protocol.
Strictly speaking, file objects are broken iterators:
Python 2.5.1 (r251:54863, Jan 17 2008, 19:35:16)
f = open('foo')
it = iter(f)
it.next()
'hi\n'
it.next()
'bye\n'
it.next()
Traceback (most recent call last):
File stdin, line 1, in module
StopIteration
f.seek(0)
it.next()
'hi\n'
-Miles
--
http://mail.python.org/mailman/listinfo/python-list
