Re: [Tutor] iter class
Keith Winston wrote: On Thu, Jan 23, 2014 at 12:21 AM, Devin Jeanpierre jeanpierr...@gmail.com wrote: in Python 3, it should be __next__, not next. Ah! That's it! Thanks!!! I'd suggest staying away from any old blog posts and articles, unless you'd care to learn Python 2.x instead of 3.x. ;) Yeah, but this is a REALLY GOOD resource. But your point is well-taken: on the simple stuff, i.e. print statements, I can see the difference quickly. I suppose I should practice running my questions on old code through 2to3 before I pester the Tutor list, since that's probably also a good way to learn the differences. Yes, that would have worked here. But why not install Python 2.7 on your machine, too? That would allow you run the examples as is. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On 01/23/2014 06:53 AM, Keith Winston wrote: I suppose I should practice running my questions on old code through 2to3 before I pester the Tutor list, since that's probably also a good way to learn the differences. Yes, but that way others learn as well :-) And many people prefere learning via human interaction then dealing with arid texts (which, also, are unable to adapt to you, are they? maybe the day we have bio-animate AI text...). Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 12:53 AM, Keith Winston keithw...@gmail.com wrote: in Python 3, it should be __next__, not next. Ah! That's it! Thanks!!! Generally you'll make `__iter__` a generator, so you don't have to worry about implementing `__next__`. Also, the built-in function `next` was added in 2.6, so you don't have to worry about the method name difference between 2.x and 3.x, either. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 5:56 AM, spir denis.s...@gmail.com wrote: Yes, but that way others learn as well :-) And many people prefere learning via human interaction then dealing with arid texts Well, you caught me. I do run out of steam just plowing through lessons such: it really helps to have actual humans to interact with (well, as far as I know you're all actual humans... I can think of Tutor as a kind of Turing test :) I know that some people may prefer fewer rather than more posts on the list, but I subscribe to your others learn as well approach: I read all the posts and answers here, including those that are above my understanding, and often get little tidbits of insight. And I really can't rave too much about how helpful people are here. That said, I have been trying to be a little less dominating of the list! -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 3:09 AM, Peter Otten __pete...@web.de wrote: But why not install Python 2.7 on your machine, too? That would allow you run the examples as is. Y'know, it's funny, but I have 2.7 installed. But since I was almost certain it was a 2to3 kind of problem, I wanted to figure it out. Ok, maybe... get you all to figure it out :) That said, I did spend a while poking around trying to figure it out. -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 7:05 AM, eryksun eryk...@gmail.com wrote: Generally you'll make `__iter__` a generator, so you don't have to worry about implementing `__next__`. Also, the built-in function `next` was added in 2.6, so you don't have to worry about the method name difference between 2.x and 3.x, either. Yes, the exercise was about implementing an iter incorrectly, to see the difference. But I don't really understand your second point: when I changed the method name, it worked...? -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 10:50 AM, Keith Winston keithw...@gmail.com wrote: On Thu, Jan 23, 2014 at 7:05 AM, eryksun eryk...@gmail.com wrote: Generally you'll make `__iter__` a generator, so you don't have to worry about implementing `__next__`. Also, the built-in function `next` was added in 2.6, so you don't have to worry about the method name difference between 2.x and 3.x, either. Yes, the exercise was about implementing an iter incorrectly, to see the difference. But I don't really understand your second point: when I changed the method name, it worked...? The page you sited discusses iterating the old-fashioned way in a loop that calls `some_iter.next()`. Nowadays it's `next(some_iter)`, which works in 2.6+ and 3.x. So if `__iter__` is a generator, you don't implement '__next__` and generally won't have to call it explicitly -- unless for some reason you need to pass the bound method as a callable. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 7:50 AM, Keith Winston keithw...@gmail.com wrote: Yes, the exercise was about implementing an iter incorrectly, to see the difference. But I don't really understand your second point: when I changed the method name, it worked...? Again, nothing was incorrect about the example. Every iterator has this problem. I question how good your resource is, if this is the impression it's leaving you. -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 1:36 PM, Devin Jeanpierre
jeanpierr...@gmail.com wrote:
Again, nothing was incorrect about the example. Every iterator has
this problem.
Hmmm. Well, here's what he actually said about that example, since I
don't think I've explained correctly:
*
With iterators, one thing to watch out for is the return of self from
the __iter__ function. You can all too easily write an iterator that
isn't as re-usable as you think it is. For example, suppose you had
the following class:
{my example here}
This works just like you'd expect as long as you create a new object each time:
for i in MyTrickyIter(['a', 'b']):
... for j in MyTrickyIter(['a', 'b']):
... print i, j
a a
a b
b a
b b
but it will break if you create the object just once:
{my example here, yielding only a single a b from the above loop}
**
The difference essentially comes down to the line
... self.thelist = thelist # in the broken example, vs.
... self.thelist = iter(thelist) # the better way to do it,
restarts the iterator each time a new loop uses it
I'm pretty sure I'm not really saying this right, it takes time
(apparently, for me) to understand how to speak clearly of these
things. The more important element, of course, is when my fuzzy
speaking speaks to fuzzy thinking, of which I am grateful for
correction.
--
Keith
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 7:05 AM, eryksun eryk...@gmail.com wrote: Generally you'll make `__iter__` a generator, so you don't have to worry about implementing `__next__`. Also, the built-in function `next` was added in 2.6, so you don't have to worry about the method name difference between 2.x and 3.x, either. I'm now realizing I don't understand this comment at all. First, there IS a __iter__ method in the example: does the presence of such make it a generator already, or is it a usage thing, or something else? I don't yet completely understand the difference/point... or maybe you mean inherit the generator class? Second: you state that I don't have to worry about the name difference, but when I changed the method name from next to __next__ it worked in 3.3. So what's your point here? I'm just realizing I'm missing your pearls of wisdom, not intending on being combative or something. -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 2:24 PM, Keith Winston keithw...@gmail.com wrote: On Thu, Jan 23, 2014 at 7:05 AM, eryksun eryk...@gmail.com wrote: Generally you'll make `__iter__` a generator, so you don't have to worry about implementing `__next__`. Also, the built-in function `next` was added in 2.6, so you don't have to worry about the method name difference between 2.x and 3.x, either. I'm now realizing I don't understand this comment at all. First, there IS a __iter__ method in the example: does the presence of such make it a generator already, or is it a usage thing, or something else? I don't yet completely understand the difference/point... or maybe you mean inherit the generator class? Sorry, sloppy language. You'll make `__iter__` a generator *function* that returns a generator, which is an iterator (i.e. has a `__next__` method). This works in 2.6+ and 3.x without needing any kludges or 2to3. For example: class MyIter: def __init__(self, thelist): self.thelist = thelist def __iter__(self): for item in self.thelist: yield item my_iter = MyIter(['a', 'b']) it = iter(my_iter) type(it) class 'generator' next(it) 'a' next(it) 'b' list(it) # exhausted [] Second: you state that I don't have to worry about the name difference, but when I changed the method name from next to __next__ it worked in 3.3. So what's your point here? The 2nd statement in my original comment was contingent on the first, which I apparently failed to clarify in the followup. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 02:18:33PM -0500, Keith Winston wrote:
On Thu, Jan 23, 2014 at 1:36 PM, Devin Jeanpierre
jeanpierr...@gmail.com wrote:
Again, nothing was incorrect about the example. Every iterator has
this problem.
Hmmm. Well, here's what he actually said about that example, since I
don't think I've explained correctly:
Who is he?
*
With iterators, one thing to watch out for is the return of self from
the __iter__ function. You can all too easily write an iterator that
isn't as re-usable as you think it is. For example, suppose you had
the following class:
To be pedantic, an iterator isn't an iterator unless __iter__ returns
self. Let's look at something which is *iterable* but not an iterator: a
lsit. You can iterate over a list, as you know, but iter() (which calls
__iter__ under the hood) does not return the list itself:
py L = [1, 2, 4, 8]
py iter(L) is L
False
py iter(L)
list_iterator object at 0xb7b77f6c
Notice that list.__iter__ returns a special list_iterator object. That
is an actual iterator:
py it = iter(L)
py iter(it) is it
True
One definining characteristic of an iterator is that you can call iter()
on it as many times as you like, and you'll always get the same object.
[...]
{my example here}
This works just like you'd expect as long as you create a new object each
time:
for i in MyTrickyIter(['a', 'b']):
... for j in MyTrickyIter(['a', 'b']):
... print i, j
a a
a b
b a
b b
but it will break if you create the object just once:
{my example here, yielding only a single a b from the above loop}
I think your example was something like this:
it = MyTrickyIter(['a', 'b'])
for i in it:
for j in it:
print i, j
which just prints a b and then is done.
This should not be considered a bug in iterators! It's a feature,
working as designed. If you don't want that behaviour, then you
shouldn't use the same iterator in both the outer and inner loops.
For example, if you want to grab items of a sequence in lots of three,
you can do something like this:
py L = iter(range(12))
py for a, b, c in zip(L, L, L):
... print(a, b, c)
...
0 1 2
3 4 5
6 7 8
9 10 11
Each time you go around the loop, zip() grabs one item from each
argument. Since the arguments all correspond to the same iterator, that
is equivalent to grabbing three items each time around the loop.
**
The difference essentially comes down to the line
... self.thelist = thelist # in the broken example, vs.
But it's not broken at all. Remember that mysterious list_iterator
object we saw earlier? That is probably implemented something quite like
this so-called broken (not really) example.
... self.thelist = iter(thelist) # the better way to do it,
restarts the iterator each time a new loop uses it
But then it isn't an iterator. It's something else. Perhaps something
useful, but not an iterator.
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 02:24:20PM -0500, Keith Winston wrote: On Thu, Jan 23, 2014 at 7:05 AM, eryksun eryk...@gmail.com wrote: Generally you'll make `__iter__` a generator, so you don't have to worry about implementing `__next__`. Also, the built-in function `next` was added in 2.6, so you don't have to worry about the method name difference between 2.x and 3.x, either. I'm now realizing I don't understand this comment at all. First, there IS a __iter__ method in the example: does the presence of such make it a generator already, or is it a usage thing, or something else? I don't yet completely understand the difference/point... or maybe you mean inherit the generator class? Generators are a kind of function, which are special. You can't inherit from them: py def generator(): ... yield 1 ... py class MyGenerator(type(generator)): ... pass ... Traceback (most recent call last): File stdin, line 1, in module TypeError: type 'function' is not an acceptable base type although I expect that should probably be considered an implementation detail of CPython rather than a language features. (Some other Pythons may allow you to inherit from function, if they wanted to.) Eryksun means that when you write an iterable class, you might be tempted to manage the next method yourself, but nine times out of ten you don't need to, you can delegate the annoying part of the coding to Python. That is, instead of creating an *actual* iterator: # Pseudocode class MyIterator: def __iter__(self): return self def __next__(self): # I have to manage this myself... if there still are items to process: return the next item else: raise StopIteration you can *return* an iterator. This makes your class a mock or pseudo-iterator, I suppose, although for the most part only pedants normally worry about the difference and we normally call it an iterator and hope nobody gets confused. But for now, I'm going to be pedantic and be absolutely firm on the difference: class MyPseudoIterator: def __iter__(self): return iter(some data) (It's not a true iterator, because it doesn't return self. Instead, it returns a true iterator. But 95% of the time, we don't care too much about the difference.) Sometimes you already have a bunch of data ready to use, like a list, and calling iter() is easy. But sometimes you want to calculate the data on the fly, and that's where Eryksun's suggestion to use a generator is specially useful: class MyPseudoIterator: def __iter__(self): # use yield instead of return yield something yield another thing result = calculate_stuff() yield result Notice that in both cases I don't declare a __next__ method! That's because I never call next() on MyPseudoIterator instances directly, I always call iter() first: # this won't work instance = MyPseudoIterator() result = next(instance) # fails # this does instance = MyPseudoIterator() it = iter(instance) result = next(it) # works For-loops are okay since Python automatically calls iter for you: instance = MyPseudoIterator() for item in instance: ... Second: you state that I don't have to worry about the name difference, but when I changed the method name from next to __next__ it worked in 3.3. So what's your point here? I think that Eryksun means that in Python 2, you can call: it.next() but in Python 3 you can call: it.__next__() but neither are recommended, instead you should call: next(it) and let the built-in next() function worry about the difference. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 7:43 PM, Steven D'Aprano st...@pearwood.info wrote: Generators are a kind of function, which are special. You can't inherit from them: I clarified my sloppy language in a reply. `__iter__` should be a generator function, not a generator. A generator function uses `yield` and `yield from` expressions, and returns a `generator` when called. In CPython, the code for a generator function is flagged as CO_GENERATOR: def genfun(): yield 1 flags = genfun.__code__.co_flags inspect.CO_GENERATOR 32 flags inspect.CO_GENERATOR 32 But that's CPython specific; use the inspect API instead (2.6+): inspect.isgeneratorfunction(genfun) True A generator expression also creates a generator. In CPython, this is implemented by calling an anonymous generator function that gets instantiated and called automagically. When a function is called, the interpreter creates a frame to evaluate the function's bytecode, given the function's globals and the current thread state. It combines the call's positional and keyword arguments with the `__defaults__` tuple and `__kwdefaults__` dict to set the parameters as local variables in the frame. For a closure, it also loads the free-variable `cell` objects from the `__closure__` tuple. Everything is ready to call PyEval_EvalFrameEx to evaluate the frame... Except if the code is flagged as CO_GENERATOR that doesn't happen. Instead the interpreter creates and returns a `generator` that references the frame as its `gi_frame` attribute. Calling the generator's `__next__` method in turn evaluates the frame up to a `yield` or `yield from` expression. `StopIteration` is raised when the frame executes a `return` statement or explicit `return None` -- or implicitly returns from the last line of the function. Like the function type, the generator type is finalized (i.e. doesn't allow subclassing). It's clear why that would be the case for the type of a singleton object such as None, Ellipsis, NotImplemented, and True/False (bool). In other cases, supporting subclassing would be more work (initial coding, bugs, maintenance -- who's volunteering?) for little gain. For example: slice and range. Other types are so closely linked to the language that I don't see why anyone would care to subclass them, such as function, method, generator, frame, code, and cell [1]. There's a fuzzy line there; property, classmethod, and staticmethod aren't finalized. [1] Listing every finalized type in CPython would be tedious. Here's a few more, if anyone cares: builtin_function_or_method, method_descriptor, classmethod_descriptor, member_descriptor, getset_descriptor, wrapper_descriptor, and method-wrapper. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] iter class
I'm working my way through some of the examples in http://ivory.idyll.org/articles/advanced-swc/#list-comprehensions And tried this one: class MyTrickyIter: ... def __init__(self, thelist): ... self.thelist = thelist ... self.index = -1 ... ... def __iter__(self): ... return self ... ... def next(self): ... self.index += 1 ... if self.index len(self.thelist): ... return self.thelist[self.index] ... raise StopIteration FYI, this is supposed to be an example of how NOT to do an iterator, because of the way it handles thelist (which is supposed to be an iteratable construct, like a list, if I'm not confused). Anyway, my efforts to recreate the following example: mi = MyTrickyIter(['a', 'b']) for i in mi: ... for j in mi: ... print i, j which should result in a b instead results in Traceback (most recent call last): File pyshell#57, line 1, in module for i in mi: TypeError: iter() returned non-iterator of type 'MyTrickyIter' I'm sort of wondering if there's a Py2 vs. Py3 issue here, but I don't see it. -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Wed, Jan 22, 2014 at 8:57 PM, Keith Winston keithw...@gmail.com wrote: I'm working my way through some of the examples in http://ivory.idyll.org/articles/advanced-swc/#list-comprehensions And tried this one: class MyTrickyIter: ... def __init__(self, thelist): ... self.thelist = thelist ... self.index = -1 ... ... def __iter__(self): ... return self ... ... def next(self): ... self.index += 1 ... if self.index len(self.thelist): ... return self.thelist[self.index] ... raise StopIteration FYI, this is supposed to be an example of how NOT to do an iterator, because of the way it handles thelist (which is supposed to be an iteratable construct, like a list, if I'm not confused). thelist is more like a sequence, meaning you can use myseq[n] for every value of n between and including 0 and len(myseq) -1. (Although, sequences usually have other things, too.) An iterable is anything you can loop over with a for loop. This is a perfectly reasonable iterator, but iterators are always tricky when used as an iterable -- you can't do a nested loop over a single iterator, because the iterator used for both loops will be the same iterator, and keep the same state. Anyway, my efforts to recreate the following example: mi = MyTrickyIter(['a', 'b']) for i in mi: ... for j in mi: ... print i, j which should result in a b instead results in Traceback (most recent call last): File pyshell#57, line 1, in module for i in mi: TypeError: iter() returned non-iterator of type 'MyTrickyIter' I'm sort of wondering if there's a Py2 vs. Py3 issue here, but I don't see it. in Python 3, it should be __next__, not next. I'd suggest staying away from any old blog posts and articles, unless you'd care to learn Python 2.x instead of 3.x. ;) -- Devin ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iter class
On Thu, Jan 23, 2014 at 12:21 AM, Devin Jeanpierre jeanpierr...@gmail.com wrote: in Python 3, it should be __next__, not next. Ah! That's it! Thanks!!! I'd suggest staying away from any old blog posts and articles, unless you'd care to learn Python 2.x instead of 3.x. ;) Yeah, but this is a REALLY GOOD resource. But your point is well-taken: on the simple stuff, i.e. print statements, I can see the difference quickly. I suppose I should practice running my questions on old code through 2to3 before I pester the Tutor list, since that's probably also a good way to learn the differences. Thanks for your help. -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
