Re: iterators and views of lists

2009-12-19 Thread Terry Reedy
On 12/19/2009 12:10 AM, Gregory Ewing wrote: Terry Reedy wrote: On the other hand, Python indexes are a form of random access iterator, the top of the hierarchy. The term random access iterator seems oxymoronic to me. Iteration is all about operating on things in sequence. If you're accessing

Re: iterators and views of lists

2009-12-19 Thread Anh Hai Trinh
On Dec 19, 5:47 am, Bearophile bearophileh...@lycos.com wrote: It seems you have missed my post, so here it is, more explicitly: http://www.boostcon.com/site-media/var/sphene/sphwiki/attachment/2009... Interestingly, my `listagent` can be used as a lazy iterator and thus using itertools we

Re: iterators and views of lists

2009-12-19 Thread Carl Banks
On Dec 18, 12:18 pm, Alf P. Steinbach al...@start.no wrote: [lots of tangential information snipped] ...but I don't understand why you're limiting yourself to the STL. Because I believe the vast majority of iterators used in C++ in practice are the ones provided by STL types, therefore I felt a

Re: iterators and views of lists

2009-12-19 Thread Rhodri James
On Fri, 18 Dec 2009 21:10:28 -, Brendan Miller catph...@catphive.net wrote: When I said they are weak I meant it in sense that the algorithms writeable against an InputerIterator interface (which is what python's iterator protocol provides) is a proper subset of the algorithms that can be

Re: iterators and views of lists

2009-12-19 Thread Anh Hai Trinh
On Dec 20, 12:04 am, Anh Hai Trinh anh.hai.tr...@gmail.com wrote: chain:   sorted(itertools.chain(listagent(x)[::2], listagent(y)[-1:1:-2]))   [0, 4, 8, 12, 13, 15, 16, 17, 19] zip:   sorted(itertools.izip(listagent(z)[1::3], listagent(x)[2::3]))   [(452, 16), (758, 4), (898, 10)] I

Re: iterators and views of lists

2009-12-18 Thread Anh Hai Trinh
On Dec 18, 3:07 am, Brendan Miller catph...@catphive.net wrote: Well, it doesn't really need to be any slower than a normal list. You only need to use index and do extra additions because it's in python. However, if listagent were written in C, you would just have a pointer into the contents

Re: iterators and views of lists

2009-12-18 Thread Alf P. Steinbach
* Carl Banks: On Dec 17, 10:00 pm, Brendan Miller catph...@catphive.net wrote: On Thu, Dec 17, 2009 at 6:44 PM, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: On Thu, 17 Dec 2009 12:07:59 -0800, Brendan Miller wrote: I was thinking it would be cool to make python more usable in

Re: iterators and views of lists

2009-12-18 Thread Lie Ryan
On 12/18/2009 7:07 AM, Brendan Miller wrote: As for copying pointers not taking much time... that depends on how long the list is. if you are working with small sets of data, you can do almost anything and it will be efficient. However, if you have megabytes or gigabytes of data (say you are

Re: iterators and views of lists

2009-12-18 Thread Carl Banks
On Dec 18, 11:08 am, Alf P. Steinbach al...@start.no wrote: * Carl Banks: On Dec 17, 10:00 pm, Brendan Miller catph...@catphive.net wrote: On Thu, Dec 17, 2009 at 6:44 PM, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: On Thu, 17 Dec 2009 12:07:59 -0800, Brendan Miller

Re: iterators and views of lists

2009-12-18 Thread Alf P. Steinbach
* Carl Banks: On Dec 18, 11:08 am, Alf P. Steinbach al...@start.no wrote: * Carl Banks: On Dec 17, 10:00 pm, Brendan Miller catph...@catphive.net wrote: On Thu, Dec 17, 2009 at 6:44 PM, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: On Thu, 17 Dec 2009 12:07:59 -0800, Brendan

Re: iterators and views of lists

2009-12-18 Thread Brendan Miller
On Fri, Dec 18, 2009 at 10:39 AM, Carl Banks pavlovevide...@gmail.com wrote: On Dec 17, 10:00 pm, Brendan Miller catph...@catphive.net wrote: On Thu, Dec 17, 2009 at 6:44 PM, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: On Thu, 17 Dec 2009 12:07:59 -0800, Brendan Miller wrote:

Re: iterators and views of lists

2009-12-18 Thread Terry Reedy
On 12/18/2009 1:00 AM, Brendan Miller wrote: For the benefit of those of us who aren't C++ programmers, what do its iterators do that Python's don't? It depends on what one means by 'iterator'. Python iterators do not fit in the STL hierarchy. On the other hand, Python indexes are a form of

Re: iterators and views of lists

2009-12-18 Thread Bearophile
Brendan Miller: I agree though, it doesn't matter to everyone and anyone. The reason I was interested was because i was trying to solve some specific problems in an elegant way. I was thinking it would be cool to make python more usable in programming competitions by giving it its own port of

Re: iterators and views of lists

2009-12-18 Thread Steven D'Aprano
On Fri, 18 Dec 2009 10:39:15 -0800, Carl Banks wrote: It is true that Python iterators can't be used to mutate the underlying structure--if there is actual underlying data structure-- An iterator is a protocol. So long as you have __iter__ and next (or __next__ in Python 3) methods, your

Re: iterators and views of lists

2009-12-18 Thread Gregory Ewing
Terry Reedy wrote: On the other hand, Python indexes are a form of random access iterator, the top of the hierarchy. The term random access iterator seems oxymoronic to me. Iteration is all about operating on things in sequence. If you're accessing elements arbitrarily, then you're not

Re: iterators and views of lists

2009-12-18 Thread Brendan Miller
On Fri, Dec 18, 2009 at 2:47 PM, Bearophile bearophileh...@lycos.com wrote: Brendan Miller: I agree though, it doesn't matter to everyone and anyone. The reason I was interested was because i was trying to solve some specific problems in an elegant way. I was thinking it would be cool to make

Re: iterators and views of lists

2009-12-18 Thread Lie Ryan
On 12/17/2009 4:44 AM, Francesco Bochicchio wrote: On Dec 16, 1:58 pm, Anh Hai Trinhanh.hai.tr...@gmail.com wrote: You might be interested in this libraryhttp://pypi.python.org/pypi/ stream. You can easily create arbitrary slice, for example i = mylist takei(primes()) will return an

Re: iterators and views of lists

2009-12-17 Thread Anh Hai Trinh
I have a couple of thoughts: 1. Since [:] by convention already creates a copy, it might violate people's expectations if that syntax were used. Indeed, listagent returns self on __getitem__[:]. What I meant was this: x = [0, 1, 2, 3, 4, 5, 6, 7] a = listagent(x)[::2] a[:] =

Re: iterators and views of lists

2009-12-17 Thread Brendan Miller
On Thu, Dec 17, 2009 at 8:41 AM, Anh Hai Trinh anh.hai.tr...@gmail.com wrote: I have a couple of thoughts: 1. Since [:] by convention already creates a copy, it might violate people's expectations if that syntax were used. Indeed, listagent returns self on __getitem__[:]. What I meant was

Re: iterators and views of lists

2009-12-17 Thread Steven D'Aprano
On Thu, 17 Dec 2009 12:07:59 -0800, Brendan Miller wrote: I was thinking it would be cool to make python more usable in programming competitions by giving it its own port of the STL's algorithm library, which needs something along the lines of C++'s more powerful iterators. For the benefit

Re: iterators and views of lists

2009-12-17 Thread Brendan Miller
On Thu, Dec 17, 2009 at 6:44 PM, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: On Thu, 17 Dec 2009 12:07:59 -0800, Brendan Miller wrote: I was thinking it would be cool to make python more usable in programming competitions by giving it its own port of the STL's algorithm

Re: iterators and views of lists

2009-12-16 Thread Bearophile
Brendan Miller: Currently people slice and dice with well... slices, but those are copying, so if you want to operate over part of a range you make a copy, perform the operation, then copy the results back in. I was thinking you'd want something like random access iterators in c++, or

Re: iterators and views of lists

2009-12-16 Thread Steven D'Aprano
On Tue, 15 Dec 2009 23:48:04 -0800, Brendan Miller wrote: On Tue, Dec 15, 2009 at 9:09 PM, Terry Reedy tjre...@udel.edu wrote: On 12/15/2009 10:39 PM, Brendan Miller wrote: I'm wondering if anyone has done work towards creating more powerful iterators for python, or creating some more

Re: iterators and views of lists

2009-12-16 Thread Paul Rudin
Steven D'Aprano st...@remove-this-cybersource.com.au writes: I'm sympathetic to your request for list views. I've often wanted some way to cleanly and neatly do this: for item in seq[1:]: process(item) without making an unnecessary copy of almost all of seq. I don't know how it's

Re: iterators and views of lists

2009-12-16 Thread Anh Hai Trinh
On Dec 16, 10:39 am, Brendan Miller catph...@catphive.net wrote: I was trying to reimplement some of the c++ library of generic algorithms in c++ in python, but I was finding that this is problematic to do this in a generic way because there isn't any equivalent of c++'s forward iterators,

Re: iterators and views of lists

2009-12-16 Thread Peter Otten
Paul Rudin wrote: Steven D'Aprano st...@remove-this-cybersource.com.au writes: I'm sympathetic to your request for list views. I've often wanted some way to cleanly and neatly do this: for item in seq[1:]: process(item) without making an unnecessary copy of almost all of seq.

Re: iterators and views of lists

2009-12-16 Thread Carl Banks
On Dec 15, 11:48 pm, Brendan Miller catph...@catphive.net wrote: I was thinking you'd want something like random access iterators in c++, or pointers in c, to write typical in place algorithmic code. To me, something like non-copying slices (maybe you'd call it a list view?) would seem

Re: iterators and views of lists

2009-12-16 Thread Daniel Stutzbach
On Wed, Dec 16, 2009 at 7:33 AM, Peter Otten __pete...@web.de wrote: islice() could be changed to special-case lists and tuples, but that feels a bit unclean. How about special-casing objects that implement collections.Sequence? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises,

Re: iterators and views of lists

2009-12-16 Thread Daniel Stutzbach
On Wed, Dec 16, 2009 at 5:39 AM, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: for item in seq[1:]: process(item) without making an unnecessary copy of almost all of seq. I use the following idiom: for i in range(1, len(seq)): process(seq[i]) Alternately, if I'm using

Re: iterators and views of lists

2009-12-16 Thread Francesco Bochicchio
On Dec 16, 1:58 pm, Anh Hai Trinh anh.hai.tr...@gmail.com wrote: You might be interested in this library http://pypi.python.org/pypi/ stream. You can easily create arbitrary slice, for example   i = mylist takei(primes()) will return an iterator over the items of mylist with a prime

Re: iterators and views of lists

2009-12-16 Thread Brendan Miller
On Wed, Dec 16, 2009 at 4:16 AM, Paul Rudin paul.nos...@rudin.co.uk wrote: Steven D'Aprano st...@remove-this-cybersource.com.au writes: I'm sympathetic to your request for list views. I've often wanted some way to cleanly and neatly do this: for item in seq[1:]:     process(item) without

Re: iterators and views of lists

2009-12-16 Thread Anh Hai Trinh
On Dec 16, 2:48 pm, Brendan Miller catph...@catphive.net wrote: No, that's what I'm getting at... Most of the existing mutating algorithms in python (sort, reverse) operate over entire collections, not partial collections delimited by indexes... which would be really awkward anyway. Ok it

Re: iterators and views of lists

2009-12-16 Thread Brendan Miller
On Wed, Dec 16, 2009 at 12:38 PM, Anh Hai Trinh anh.hai.tr...@gmail.com wrote: On Dec 16, 2:48 pm, Brendan Miller catph...@catphive.net wrote: No, that's what I'm getting at... Most of the existing mutating algorithms in python (sort, reverse) operate over entire collections, not partial

iterators and views of lists

2009-12-15 Thread Brendan Miller
I was trying to reimplement some of the c++ library of generic algorithms in c++ in python, but I was finding that this is problematic to do this in a generic way because there isn't any equivalent of c++'s forward iterators, random access iterators, etc. i.e. all python iterators are just input

Re: iterators and views of lists

2009-12-15 Thread Terry Reedy
On 12/15/2009 10:39 PM, Brendan Miller wrote: I was trying to reimplement some of the c++ library of generic algorithms in c++ in python, but I was finding that this is problematic to do this in a generic way because there isn't any equivalent of c++'s forward iterators, random access iterators,

Re: iterators and views of lists

2009-12-15 Thread Brendan Miller
On Tue, Dec 15, 2009 at 9:09 PM, Terry Reedy tjre...@udel.edu wrote: On 12/15/2009 10:39 PM, Brendan Miller wrote: I'm wondering if anyone has done work towards creating more powerful iterators for python, or creating some more pythonic equivalent. For sequences, integer indexes let you do