Re: list.pop(0) vs. collections.dequeue

2010-01-27 Thread Steve Howell
On Jan 26, 11:34 pm, Arnaud Delobelle wrote: > Steve Howell writes: > > On Jan 25, 1:32 pm, Arnaud Delobelle wrote: > >> Steve Howell writes: > > >> [...] > > >> > My algorithm does exactly N pops and roughly N list accesses, so I > >> > would be going from N*N + N to N + N log N if switched to

Re: list.pop(0) vs. collections.dequeue

2010-01-26 Thread Arnaud Delobelle
Steve Howell writes: > On Jan 25, 1:32 pm, Arnaud Delobelle wrote: >> Steve Howell writes: >> >> [...] >> >> > My algorithm does exactly N pops and roughly N list accesses, so I >> > would be going from N*N + N to N + N log N if switched to blist. >> >> Can you post your algorithm?  It would be

Re: list.pop(0) vs. collections.dequeue

2010-01-26 Thread Steve Holden
Steve Howell wrote: > On Jan 24, 11:28 am, a...@pythoncraft.com (Aahz) wrote: >> In article >> , >> Steve Howell wrote: >> >> >> >>> Even with realloc()'s brokenness, you could improve pop(0) in a way >>> that does not impact list access at all, and the patch would not change >>> the time comple

Re: list.pop(0) vs. collections.dequeue

2010-01-26 Thread Steve Howell
On Jan 25, 9:00 pm, Steve Howell wrote: > On Jan 24, 11:28 am, a...@pythoncraft.com (Aahz) wrote: > > > > > In article > > , > > Steve Howell   wrote: > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > > >that does not impact list access at all, and the patch would not c

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 25, 8:31 pm, Paul Rubin wrote: > Steve Howell writes: > > I haven't profiled deque vs. list, but I think you are correct about > > pop() possibly being a red herring > > For really large lists, I suppose memmove() would eventually start to > > become a bottleneck, but it's brutally fas

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 24, 11:28 am, a...@pythoncraft.com (Aahz) wrote: > In article , > Steve Howell   wrote: > > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > >that does not impact list access at all, and the patch would not change > >the time complexity of any operation; it would ju

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Paul Rubin
Steve Howell writes: > I haven't profiled deque vs. list, but I think you are correct about > pop() possibly being a red herring > For really large lists, I suppose memmove() would eventually start to > become a bottleneck, but it's brutally fast when it just moves a > couple kilobytes of data

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Jerry Hill
On Sat, Jan 23, 2010 at 4:38 AM, Alf P. Steinbach wrote: > Hm, it would be nice if > the Python docs offered complexity (time) guarantees in general... Last time it came up, I don't think there was any core developer interest in putting complexity guarantees in the Python Language Reference. Som

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Alf P. Steinbach
* Ethan Furman: Steve Howell wrote: On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: So, we're right back to my statement earlier in this thread that the docs are deficient in that they describe behavior with no hint about cost. Given that, it should be no surprise that users make incorrect

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Ethan Furman
Steve Howell wrote: On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: So, we're right back to my statement earlier in this thread that the docs are deficient in that they describe behavior with no hint about cost. Given that, it should be no surprise that users make incorrect assumptions abou

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread geremy condra
On Sat, Jan 23, 2010 at 4:38 AM, Alf P. Steinbach wrote: > Hm, it would be nice if the Python docs offered complexity (time) > guarantees in general... > > Cheers, > > - Alf This would be a very welcome improvement IMHO- especially in collections. Geremy Condra -- http://mail.python.org/mail

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
--- On Mon, 1/25/10, Chris Colbert wrote: > > looking at that code, i think you could solve > your whole problem with a single called to reversed() (which > is NOT the same as list.reverse())  > I do not think that's actually true. It does no good to pop elements off a copy of the list if th

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 25, 1:32 pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm?  It would be interesting to have a concrete >

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 25, 1:00 pm, Paul Rubin wrote: > Steve Howell writes: > > These are the reasons I am not using deque: > > Thanks for these.  Now we are getting somewhere. > > >   1) I want to use native lists, so that downstream methods can use > > them as lists. > > It sounds like that could be fixed by

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Chris Colbert
On Mon, Jan 25, 2010 at 5:09 PM, Steve Howell wrote: > On Jan 25, 1:32 pm, Arnaud Delobelle wrote: > > Steve Howell writes: > > > > [...] > > > > > My algorithm does exactly N pops and roughly N list accesses, so I > > > would be going from N*N + N to N + N log N if switched to blist. > > > > C

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 25, 1:32 pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm?  It would be interesting to have a concrete >

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 25, 1:32 pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm?  It would be interesting to have a concrete >

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Stephen Hansen
On Mon, Jan 25, 2010 at 9:31 AM, Steve Howell wrote: > Another way of looking at it is that you would need to have 250 or so > lists in memory at the same time before the extra pointer was even > costing you kilobytes of memory. My consumer laptop has 3027908k of > memory. > Umm, I think the is

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Arnaud Delobelle
Steve Howell writes: [...] > My algorithm does exactly N pops and roughly N list accesses, so I > would be going from N*N + N to N + N log N if switched to blist. Can you post your algorithm? It would be interesting to have a concrete use case to base this discussion on. -- Arnaud -- http://m

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Paul Rubin
Steve Howell writes: > These are the reasons I am not using deque: Thanks for these. Now we are getting somewhere. > 1) I want to use native lists, so that downstream methods can use > them as lists. It sounds like that could be fixed by making the deque API a proper superset of the list API

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Daniel Stutzbach
On Mon, Jan 25, 2010 at 12:24 PM, Steve Howell wrote: > Hi Daniel, I agree with what Raymond Hettinger says toward the top of > the PEP. Blist, while extremely useful, does seem to have to trade > off performance of common operations, notably "get item", in order to > get better performance for

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 24, 1:51 pm, Daniel Stutzbach wrote: > On Sun, Jan 24, 2010 at 1:53 PM, Steve Howell wrote: > > I don't think anybody provided an actual link, but please correct me > > if I overlooked it. > > I have to wonder if my messages are all ending up in your spam folder > for some reason. :-) > >

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 25, 9:31 am, Steve Howell wrote: > On Jan 24, 11:24 pm, Paul Rubin wrote: > > > Steve Howell writes: > > > There is nothing wrong with deque, at least as far as I know, if the > > > data strucure actually applies to your use case.  It does not apply to > > > my use case. > > > You haven't

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 24, 10:07 pm, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 20:12:11 -0800, Steve Howell wrote: > >> > The most ambitious proposal is to fix the memory manager itself to > >> > allow the release of memory from the start of the chunk. > > >> That's inappropriate given the memory fragmentation

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Steve Howell
On Jan 24, 11:24 pm, Paul Rubin wrote: > Steve Howell writes: > > There is nothing wrong with deque, at least as far as I know, if the > > data strucure actually applies to your use case.  It does not apply to > > my use case. > > You haven't explained why deque doesn't apply to your use case.  U

Re: list.pop(0) vs. collections.dequeue

2010-01-25 Thread Antoine Pitrou
Le Sun, 24 Jan 2010 11:28:53 -0800, Aahz a écrit : > > Again, your responsibility is to provide a patch and a spectrum of > benchmarking tests to prove it. Then you would still have to deal with > the objection that extensions use the list internals -- that might be an > okay sell given the effor

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Paul Rubin
Steve Howell writes: > There is nothing wrong with deque, at least as far as I know, if the > data strucure actually applies to your use case. It does not apply to > my use case. You haven't explained why deque doesn't apply to your use case. Until a convincing explanation emerges, the sentimen

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Steven D'Aprano
On Sun, 24 Jan 2010 20:12:11 -0800, Steve Howell wrote: >> > The most ambitious proposal is to fix the memory manager itself to >> > allow the release of memory from the start of the chunk. >> >> That's inappropriate given the memory fragmentation it would cause. >> >> > Bullshit. Memory managers

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Steve Howell
On Jan 24, 12:44 pm, Paul Rubin wrote: > Steve Howell writes: > > Proposal: Improve list's implementation so that deleting elements from > > the front of the list does not require an O(N) memmove operation. ... > > It is possible now, of course, to use a data structure in > > Python that has O(1)

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Daniel Stutzbach
On Sun, Jan 24, 2010 at 1:53 PM, Steve Howell wrote: > I don't think anybody provided an actual link, but please correct me > if I overlooked it. I have to wonder if my messages are all ending up in your spam folder for some reason. :-) PEP 3128 (which solves your problem, but not using the impl

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Paul Rubin
Steve Howell writes: > Proposal: Improve list's implementation so that deleting elements from > the front of the list does not require an O(N) memmove operation. ... > It is possible now, of course, to use a data structure in > Python that has O(1) for deleting off the top of the list, but none of

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Terry Reedy
On 1/24/2010 2:26 PM, Steve Howell wrote: I think it's a good idea to write a PEP on this issue, and I will attempt a first draft. I think I should submit the first draft to python-ideas, correct? That is not a *requirement* for drafts in general, but it is a good idea for a community or com

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Steve Howell
On Jan 24, 11:28 am, a...@pythoncraft.com (Aahz) wrote: > In article , > Steve Howell   wrote: > > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > >that does not impact list access at all, and the patch would not change > >the time complexity of any operation; it would ju

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Aahz
In article , Steve Howell wrote: > >Even with realloc()'s brokenness, you could improve pop(0) in a way >that does not impact list access at all, and the patch would not change >the time complexity of any operation; it would just add negligible >extract bookkeeping within list_resize() and a few

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Steve Howell
On Jan 23, 3:04 pm, Terry Reedy wrote: > On 1/23/2010 12:17 PM, Steve Howell wrote: > > > Terry Reedy said: > > > ''' > > If you try writing a full patch, as I believe someone did, or at least > > a > > prototype thereof, when the idea was discussed, you might have a > > better > > idea of what th

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Steve Howell
On Jan 24, 3:20 am, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 02:33:36 -0800, Steve Howell wrote: > > You are also a brilliant computer scientist, despite the fact that you > > are defending a list implemenation that can't pop the first element off > > the list in O(1) time. > > You say that li

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Steven D'Aprano
On Sun, 24 Jan 2010 02:33:36 -0800, Steve Howell wrote: > You are also a brilliant computer scientist, despite the fact that you > are defending a list implemenation that can't pop the first element off > the list in O(1) time. You say that like it's a bad thing. It's very simple: the trade-offs

Re: list.pop(0) vs. collections.dequeue

2010-01-24 Thread Steve Howell
On Jan 23, 8:00 pm, Raymond Hettinger wrote: > [Steve Howell] > > > Why wouldn't you get a competent C programmer simply make > > list_ass_slice smart enough to make list.pop(0) O(1)? > > When this suggestion was discussed on python-dev years ago, > it was rejected.  One reason is that it was some

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Raymond Hettinger
[Steve Howell] > Why wouldn't you get a competent C programmer simply make > list_ass_slice smart enough to make list.pop(0) O(1)? When this suggestion was discussed on python-dev years ago, it was rejected. One reason is that it was somewhat common for C code to access the list data structure di

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Roy Smith
In article , Terry Reedy wrote: > >> The page you should probably be looking at is > >> http://wiki.python.org/moin/TimeComplexity > > > > I was not aware of this page; thanks for pointing it out. > > Perhaps you could suggest on the tracker a place or places in the doc > where this relatively

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Terry Reedy
On 1/23/2010 2:02 PM, Roy Smith wrote: In article, Duncan Booth wrote: Roy Smith wrote: I'm looking at http://tinyurl.com/cdbwog. It shows all the operations of a list. What it does not show is their cost. For pop(), it has a note: "The pop() method is only supported by the list and a

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Terry Reedy
On 1/23/2010 12:17 PM, Steve Howell wrote: Terry Reedy said: ''' If you try writing a full patch, as I believe someone did, or at least a prototype thereof, when the idea was discussed, you might have a better idea of what the tradeoffs are and why it was rejected. ''' I have to run, but tomor

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Roy Smith
In article , Duncan Booth wrote: > Roy Smith wrote: > > > I'm looking at http://tinyurl.com/cdbwog. It shows all the operations > > of a list. What it does not show is their cost. For pop(), it has a > > note: > > > > "The pop() method is only supported by the list and array types. The >

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Aahz
In article <8e4d3fe2-c4bd-4a73-9c50-7a336dab2...@o28g2000yqh.googlegroups.com>, Steve Howell wrote: >On Jan 22, 11:10=A0pm, a...@pythoncraft.com (Aahz) wrote: >> >>>I know Python's number one concern will never be speed, but if Python >>>makes an O(1) operation into an unnecessarily O(N) operatio

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 7:54 am, Steven D'Aprano wrote: > On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > > In article , > >  "Alf P. Steinbach" wrote: > > >> But it would IMHO have been better if it wasn't called "list", which > >> brings in the wrong associations for someone used to other languages.

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 4:12 am, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 21:42:43 -0800, Steve Howell wrote: > > This innocent program here literally moves about a million bytes of > > memory around for no good reason: > > >     lst = [] > >     for i in range(2000): > >         lst.append(i) > >     whil

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 4:12 am, Steven D'Aprano wrote: > > > An alternative would be to do exactly what you want lists to do: track > the start of the list. Untested: > >     def recurse(prefix_lines): >         start = 0 >         end = len(prefix_lines) >         while start < end: >             prefix, lin

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 6:40 am, Roy Smith wrote: > In article > , >  Steve Howell wrote: > > > This innocent program here literally moves about a million bytes of > > memory around for no good reason: > > >     lst = [] > >     for i in range(2000): > >         lst.append(i) > >     while lst: > >         pr

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 5:46 am, Christian Heimes wrote: > Steve Howell wrote: > > Another benchmark is that deques are slower than lists for accessing > > elements. > > deques are optimized for accessing, inserting and removing data from > both ends. For anything else it's slower than the list type. The fact

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Alf P. Steinbach
* Steven D'Aprano: On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: In article , "Alf P. Steinbach" wrote: But it would IMHO have been better if it wasn't called "list", which brings in the wrong associations for someone used to other languages. +1. When I first started using Python (

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Duncan Booth
Roy Smith wrote: > I'm looking at http://tinyurl.com/cdbwog. It shows all the operations > of a list. What it does not show is their cost. For pop(), it has a > note: > > "The pop() method is only supported by the list and array types. The > optional argument i defaults to -1, so that by de

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steven D'Aprano
On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > In article , > "Alf P. Steinbach" wrote: > >> But it would IMHO have been better if it wasn't called "list", which >> brings in the wrong associations for someone used to other languages. > > +1. > > When I first started using Python (bac

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Roy Smith
In article , "Alf P. Steinbach" wrote: > But it would IMHO have been better if it wasn't called "list", which brings > in the wrong associations for someone used to other languages. +1. When I first started using Python (back in the 1.4 days), I assumed a list was a singly-linked list. Whic

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Roy Smith
In article , Steve Howell wrote: > This innocent program here literally moves about a million bytes of > memory around for no good reason: > > lst = [] > for i in range(2000): > lst.append(i) > while lst: > print lst.pop(0) > > Why? Because list.pop(0) is implemen

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Christian Heimes
Steve Howell wrote: > Another benchmark is that deques are slower than lists for accessing > elements. deques are optimized for accessing, inserting and removing data from both ends. For anything else it's slower than the list type. The fact was explained in this very thread yesterday. Christian

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Arnaud Delobelle
Dave Angel writes: > Arnaud Delobelle wrote: >> Steve Howell writes: >> >> >>> On Jan 22, 12:14 pm, Chris Rebert wrote: >>> >> I made the comment you quoted. In CPython, it is O(n) to delete/insert >> an element at the start of the list - I know it because I looked at the >> implementa

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steven D'Aprano
On Fri, 22 Jan 2010 21:42:43 -0800, Steve Howell wrote: > This innocent program here literally moves about a million bytes of > memory around for no good reason: > > lst = [] > for i in range(2000): > lst.append(i) > while lst: > print lst.pop(0) > > Why? Because lis

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 1:24 am, Terry Reedy wrote: > On 1/23/2010 3:23 AM, Steve Howell wrote: > > > On Jan 23, 12:13 am, Terry Reedy  wrote: > > >> Challenge yes, mock no. > > >> Part of writing good basic data structures is not adding needless > >> complication from featuritis and not penalizing 99.99% of a

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Alf P. Steinbach
* Steve Howell: On Jan 23, 12:32 am, "Alf P. Steinbach" wrote: * Steve Howell: On Jan 23, 12:13 am, Terry Reedy wrote: Challenge yes, mock no. Part of writing good basic data structures is not adding needless complication from featuritis and not penalizing 99.99% of access to satify a .01%

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 12:13 am, Terry Reedy wrote: > On 1/23/2010 12:58 AM, Steve Howell wrote: > > > I really want to use list *normally* with all its perfectly good > > semantics and reasonable implementation, except for its blind spot > > with respect to popping the first element off the list. > > It was

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Terry Reedy
On 1/23/2010 3:23 AM, Steve Howell wrote: On Jan 23, 12:13 am, Terry Reedy wrote: Challenge yes, mock no. Part of writing good basic data structures is not adding needless complication from featuritis and not penalizing 99.99% of access to satify a .01% need better satisfied another way. I

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 12:32 am, "Alf P. Steinbach" wrote: > * Steve Howell: > > > On Jan 23, 12:13 am, Terry Reedy wrote: > >> Challenge yes, mock no. > > >> Part of writing good basic data structures is not adding needless > >> complication from featuritis and not penalizing 99.99% of access to > >> satify

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Alf P. Steinbach
* Steve Howell: On Jan 23, 12:13 am, Terry Reedy wrote: Challenge yes, mock no. Part of writing good basic data structures is not adding needless complication from featuritis and not penalizing 99.99% of access to satify a .01% need better satisfied another way. I would like to challenge yo

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 23, 12:13 am, Terry Reedy wrote: > > Challenge yes, mock no. > > Part of writing good basic data structures is not adding needless > complication from featuritis and not penalizing 99.99% of access to > satify a .01% need better satisfied another way. > I would like to challenge your asser

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Terry Reedy
On 1/23/2010 12:58 AM, Steve Howell wrote: I really want to use list *normally* with all its perfectly good semantics and reasonable implementation, except for its blind spot with respect to popping the first element off the list. It was not designed for that. .pop() was added to lists about 1

Re: list.pop(0) vs. collections.dequeue

2010-01-23 Thread Steve Howell
On Jan 22, 11:10 pm, a...@pythoncraft.com (Aahz) wrote: > In article <83082e19-9130-45a8-91f2-8601c1fda...@22g2000yqr.googlegroups.com>, > Steve Howell   wrote: > > > > > > >I really want to use list *normally* with all its perfectly good > >semantics and reasonable implementation, except for its b

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 11:10 pm, a...@pythoncraft.com (Aahz) wrote: > > >I know Python's number one concern will never be speed, but if Python > >makes an O(1) operation into an unnecessarily O(N) operation for no > >good reasons other than "it's too complicated, " or it "adds another > >pointer to the structu

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Alf P. Steinbach
* Steve Howell: On Jan 22, 7:09 pm, Roy Smith wrote: In article <3ac173bd-4124-434d-b726-0b9baaeec...@36g2000yqu.googlegroups.com>, Steve Howell wrote: In my case I'm not really concerned about giving the memory back right away, it's more about keeping my code simple. Once I'm done with an

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Aahz
In article <83082e19-9130-45a8-91f2-8601c1fda...@22g2000yqr.googlegroups.com>, Steve Howell wrote: > >I really want to use list *normally* with all its perfectly good >semantics and reasonable implementation, except for its blind spot >with respect to popping the first element off the list. The

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 7:09 pm, Roy Smith wrote: > In article > <3ac173bd-4124-434d-b726-0b9baaeec...@36g2000yqu.googlegroups.com>, >  Steve Howell wrote: > > > In my case I'm not really concerned about giving the memory back right > > away, it's more about keeping my code simple.  Once I'm done with an > >

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 6:20 pm, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 14:38:18 -0800, Steve Howell wrote: > > I know the Python programmer could simply iterate through the list > > rather than popping off unused elements, but that just means that you > > not only tie up the memory for the pointers just

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Roy Smith
In article <3ac173bd-4124-434d-b726-0b9baaeec...@36g2000yqu.googlegroups.com>, Steve Howell wrote: > In my case I'm not really concerned about giving the memory back right > away, it's more about keeping my code simple. Once I'm done with an > element, I do just want to pop it off and keep all

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Roy Smith
In article , Christian Heimes wrote: > Steve Howell wrote: > > Is that really true in CPython? It seems like you could advance the > > pointer instead of shifting all the elements. It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steven D'Aprano
On Fri, 22 Jan 2010 14:38:18 -0800, Steve Howell wrote: > I know the Python programmer could simply iterate through the list > rather than popping off unused elements, but that just means that you > not only tie up the memory for the pointers just as long, but you also > prevent the objects themse

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Daniel Stutzbach
On Fri, Jan 22, 2010 at 5:27 PM, Steve Howell wrote: > I actually do expect Python to solve performance problems for me that > are more easily solved in core than in Python itself. So I guess > that's where we differ. > You might be interested in the extension type I wrote (the "blist") that lo

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 3:17 pm, Christian Heimes wrote: > Steve Howell wrote: > > That maybe would be an argument for just striking the paragraph from > > the tutorial.  If it's rare that people pop the head off the list in > > code that is performance critical or prominent, why bother to even > > mention it

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 2:54 pm, Dave Angel wrote: > Steve Howell wrote: > > On Jan 22, 12:40 pm, Christian Heimes wrote: > > >> Steve Howell wrote: > > >>> Is that really true in CPython?  It seems like you could advance the > >>> pointer instead of shifting all the elements.  It would create some > >>> nuan

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Christian Heimes
Steve Howell wrote: > That maybe would be an argument for just striking the paragraph from > the tutorial. If it's rare that people pop the head off the list in > code that is performance critical or prominent, why bother to even > mention it in the tutorial? How else are you going to teach new P

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Dave Angel
Arnaud Delobelle wrote: Steve Howell writes: On Jan 22, 12:14 pm, Chris Rebert wrote: I made the comment you quoted. In CPython, it is O(n) to delete/insert an element at the start of the list - I know it because I looked at the implementation a while ago. This is why collections.

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 1:32 pm, Terry Reedy wrote: > On 1/22/2010 2:14 PM, Steve Howell wrote: > > > The v2.6.4 version of the tutorial says this: > > Is that really true in CPython?  It seems like you could advance the > > pointer instead of shifting all the elements.  It would create some > > nuances with r

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Dave Angel
Steve Howell wrote: On Jan 22, 12:40 pm, Christian Heimes wrote: Steve Howell wrote: Is that really true in CPython? It seems like you could advance the pointer instead of shifting all the elements. It would create some nuances with respect to reclaiming the memory, but it seems like

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 1:29 pm, Christian Heimes wrote: > Steve Howell wrote: > > I disagree that Python code rarely pops elements off the top of a > > list.  There are perfectly valid use cases for wanting a list over a > > dequeue without having to pay O(N) for pop(0).  Maybe we are just > > quibbling over

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Christian Heimes
Arnaud Delobelle wrote: > I made the comment you quoted. In CPython, it is O(n) to delete/insert > an element at the start of the list - I know it because I looked at the > implementation a while ago. This is why collections.deque exists I > guess. I don't know how they are implemented but inser

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Terry Reedy
On 1/22/2010 2:14 PM, Steve Howell wrote: The v2.6.4 version of the tutorial says this: Is that really true in CPython? It seems like you could advance the pointer instead of shifting all the elements. It would create some nuances with respect to reclaiming the memory, but it seems like an e

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 1:08 pm, Arnaud Delobelle wrote: > Steve Howell writes: > > On Jan 22, 12:14 pm, Chris Rebert wrote: > >> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > >> > The v2.6.4 version of the tutorial says this: > > >> > ''' > >> > It is also possible to use a list as a queue, where

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Christian Heimes
Steve Howell wrote: > I disagree that Python code rarely pops elements off the top of a > list. There are perfectly valid use cases for wanting a list over a > dequeue without having to pay O(N) for pop(0). Maybe we are just > quibbling over the meaning of "rarely." I was speaking from my own po

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Arnaud Delobelle
Steve Howell writes: > On Jan 22, 12:14 pm, Chris Rebert wrote: >> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: >> > The v2.6.4 version of the tutorial says this: >> >> > ''' >> > It is also possible to use a list as a queue, where the first element >> > added is the first element retr

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 12:40 pm, Christian Heimes wrote: > Steve Howell wrote: > > Is that really true in CPython?  It seems like you could advance the > > pointer instead of shifting all the elements.  It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way t

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Christian Heimes
Steve Howell wrote: > Is that really true in CPython? It seems like you could advance the > pointer instead of shifting all the elements. It would create some > nuances with respect to reclaiming the memory, but it seems like an > easy way to make lists perform better under a pretty reasonable us

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
On Jan 22, 12:14 pm, Chris Rebert wrote: > On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > > The v2.6.4 version of the tutorial says this: > > > ''' > > It is also possible to use a list as a queue, where the first element > > added is the first element retrieved (“first-in, first-out”);

Re: list.pop(0) vs. collections.dequeue

2010-01-22 Thread Chris Rebert
On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > The v2.6.4 version of the tutorial says this: > > ''' > It is also possible to use a list as a queue, where the first element > added is the first element retrieved (“first-in, first-out”); however, > lists are not efficient for this purpose.

list.pop(0) vs. collections.dequeue

2010-01-22 Thread Steve Howell
The v2.6.4 version of the tutorial says this: ''' It is also possible to use a list as a queue, where the first element added is the first element retrieved (“first-in, first-out”); however, lists are not efficient for this purpose. While appends and pops from the end of list are fast, doing inser