Re: list.clear() missing?!?

2006-04-15 Thread Fredrik Lundh
"Ben C" wrote:

> I used to think it assigned with things like integers, because if you
> write:
>
> a = 5
> b = a
> b += 1
> print a
>
> a is still 5. So it looked like a and b stored values and b got a "copy"
> of a's value. But this is the wrong interpretation,
>
> b += 1
>
> is really b = b + 1, and rebinds b.

almost: "b += obj" is really "b = b.__iadd__(obj)" with "b = b + obj"
as a fallback if __iadd__ isn't supported by "b".

> If it weren't for the id() function I think the difference between
> "variable stores value", "variable stores immutable reference" and
> "variable stores copy-on-write reference" would be implementation
> detail and never visible to the programmer.

except when they are:

>>> a = [1, 2, 3]
>>> b = a
>>> b += [4]
>>> a
[1, 2, 3, 4]
>>> b
[1, 2, 3, 4]





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-14 Thread Ville Vainio
Duncan Booth wrote:

> Looking in the 'obvious' place in the Tutorial, section 5.1 'More on
> Lists' I found in the immediately following section 5.2 'The del
> statement':

I read the tutorial 6 years ago, and don't read it regularly. What's in
the tutorial is not really important, what can be easily looked up in
library reference or interctive prompt is.

There is a bit of "elitism" regarding the defense of del lst[:]. Many
python programmers are "casual" programmers (was it Tim Berners-Lee
that explained how great python is for casual users?) who know the
language but don't remember the way everything is done. Of course
someone that uses python 8 hours a day instantly recall lst[:], but a
casual one will more probably launch an ipython prompt and do:

[ipython]|1> l = []
[ipython]|2> l.
l.__add__  l.__getslice__ l.__ne__   l.append
l.__class__l.__gt__   l.__new__  l.count
l.__contains__ l.__hash__ l.__reduce__   l.extend
l.__delattr__  l.__iadd__ l.__reduce_ex__l.index
l.__delitem__  l.__imul__ l.__repr__ l.insert
l.__delslice__ l.__init__ l.__reversed__ l.pop
l.__doc__  l.__iter__ l.__rmul__ l.remove
l.__eq__   l.__le__   l.__setattr__  l.reverse
l.__ge__   l.__len__  l.__setitem__  l.sort
l.__getattribute__ l.__lt__   l.__setslice__
l.__getitem__  l.__mul__  l.__str__
[ipython]|2> # wtf?

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-14 Thread Steven D'Aprano
Raymond, I suspect we're not seeing eye to eye on this issue, but I do
appreciate you taking the time and effort. Thank you. My comments follow.

On Thu, 13 Apr 2006
09:34:46 -0700, Raymond Hettinger wrote:

> Both the pros and cons
> were quipped with abrupt perjoratives so the bullet points could be
> stated succinctly and with a bit of levity.  

Humour so often doesn't come across in text -- I didn't see the levity you
intended.


> "request is inane" --> "A generation of python programmers has found
> list clearing to be like other parts of the language that you get used
> to very quickly and do not prove to be a problem in practice.  The
> request is in the same category as others which challenge api choices
> made 16 years ago; in particular, the decision to have compact APIs
> where the named methods do not duplicate functionality provided by
> syntax using operators and keywords. The request is less of a bug
> report and more a rejection of Guido's sense of design and his
> subsequent experience using his own language."

Of course it isn't a bug report. It's a feature request.

As for Guido's sense of design, in *this particular instance* I think he
got it wrong. That's hardly a rejection of the man's overall design skills.

In any case, as the creator of Python, Guido has never seen the language
with the eyes of a Python beginner. All the more credit to him for
creating a language which is simultaneously friendly to beginners and
powerful for experts. But he isn't superhuman, his experience is not the
only experience. (True, as BDFL, his experience has the final vote, but
that's a whole different kettle of fish.)

A bit of searching on Google finds this issue coming up frequently. It
seems to me that there is a steady stream of Python programmers asking
"where's the list clear method?". Perhaps it isn't a monthly occurrence,
but it is still very common. That surely suggests that, as powerful
as slicing is, people aren't getting the connection between slicing and
emptying a list, and even when they do get it, many dislike it.

Since there is no evidence that these people are universally stupid, it
suggests strongly that the connection is too subtle. Some may protest that
the idiom is in the tutorial, that it is obvious once you know slicing and
del, but the fact is that many people aren't finding it obvious or
self-evident at all -- not even those who have read the tutorial.

Here is another possible solution: have the list.clear method be defined
this way:

def clear(self):
print "Haven't you read the tutorial? Use del self[:] or self[:] = []"

That at least will stop the procession of people asking how to clear a
list, and Fredrik won't have to spend his time telling them to read the
tutorial.

(Hey, if it's good enough for quit and exit... *wink*)


>> A list.clear method will make deleting items from a list more OO,
>> consistent with almost everything else you do to lists, and less
>> procedural. This is especially true if clear() takes an optional index (or
>> two), allowing sections of the list to be cleared, not just the entire
>> list.
> 
> Don't shoot yourself in the foot here.  If you want to advocate
> list.clear(), then you're hurting your chances by pushing for it to
> take an optional argument.  Essentially, this amounts to an
> unwillingness to use the del-keyword and to duplicate its functionality
> with a named method.

Deleting names from namespaces is conceptually distinct from deleting
components of compound objects like lists, and should be handled
differently: objects know their own contents, and can operate on
themselves, but they don't know what namespace they live in or what
name(s) they are known by. Deleting items from a list should be a case of
"object, operate on yourself", just like remove, append and so on, and
hence should be specified by an object method, not a keyword like del.

So you are right: I am uncomfortable having del do double duty to both
delete names and operate on the internals of compound objects. It feels
wrong to me. If insert were a keyword (as in "insert list[i:j], obj") it
would feel wrong to me too, and for the same reason.

I haven't been using Python for sixteen years, but at six years and
counting I'm not exactly a newbie. At first I accepted the del obj[:]
idiom implicitly -- the whole language was new to me, and it was no more
strange to me than any other Python syntax or idiom. But as I've got more
familiar with Python, the sense of two distinct concepts being
artificially forced into the same syntax has just gotten stronger.

As a pragmatist, I'd be happy to see a bare list.clear() method; that
would at least have the advantages of being easy to find with dir(list)
and accessible with help(). But as a purist, I really think it needs
optional arguments. Make of that what you will.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-14 Thread Ben C
On 2006-04-14, Sergei Organov <[EMAIL PROTECTED]> wrote:
> Dennis Lee Bieber <[EMAIL PROTECTED]> writes:
>>  It always means bind... But if the LHS is a mutable object, AND you
>> have specified a component of that object, it is the component that is
>> being rebound...
>>
>>  lst[:] = [] 

>> [...]

> Me gets corrected, thanks. Now I need to unroll my mind somewhat back to
> figure out when and why I started to believe it sometimes assigns ;)

I used to think it assigned with things like integers, because if you
write:

a = 5
b = a
b += 1
print a

a is still 5. So it looked like a and b stored values and b got a "copy"
of a's value. But this is the wrong interpretation,

b += 1

is really b = b + 1, and rebinds b.

You can see what's really going on if you use the id() function on a and
b during these operations.

The other reason for the confusion is that I think in Java a variable
either stores a value, in the case of numbers, or a reference in the
case of objects (or a copy-on-write reference, which behaves like a
value, in the case of strings). In Python it's better to think of it as
always a reference, and to think in terms of immutable vs. mutable
objects that are referred to.

If it weren't for the id() function I think the difference between
"variable stores value", "variable stores immutable reference" and
"variable stores copy-on-write reference" would be implementation detail
and never visible to the programmer. That's why it's easy to be
"confused"-- most of the time these interpretations are equivalent, so
it doesn't matter which you work with.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-14 Thread Sergei Organov
Dennis Lee Bieber <[EMAIL PROTECTED]> writes:

> On Fri, 14 Apr 2006 09:17:05 +0400, Sergei Organov <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
>
>> 
>> I, as a newcomer, don't have much trouble understanding the binding vs
>> the assignment by themselves. What does somewhat confuse is dual role of
>> the "=" operator, -- sometimes it means "bind" and other times it means
>> "assign", right? For me it seems that the language would be less
>
>   It always means bind... But if the LHS is a mutable object, AND you
> have specified a component of that object, it is the component that is
> being rebound...
>
>   lst[:] = [] 
>
> is rebinding the elements inside the list "lst", and not rebinding the
> name "lst". Essentially, once you add any "selector" to the name
> (object[...]= or object.xxx=) you are going "inside" the object, and
> manipulating (rebinding) what is inside. If the name is used "pure"
> (object=), you are rebinding the /name/ to a different object.

Me gets corrected, thanks. Now I need to unroll my mind somewhat back to
figure out when and why I started to believe it sometimes assigns ;)

-- Sergei.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Felipe Almeida Lessa
Em Sex, 2006-04-14 às 09:17 +0400, Sergei Organov escreveu:
> I, as a newcomer, don't have much trouble understanding the binding vs
> the assignment by themselves. What does somewhat confuse is dual role of
> the "=" operator, -- sometimes it means "bind" and other times it means
> "assign", right? For me it seems that the language would be less
> error-prone and easier to grok if these two operations had different
> names/syntax (thinking about lisp "let" vs "set"), though it seems to be
> too drastic change even for Python3000.

The "=" operator *always* binds.

-- 
Felipe.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-13 Thread Sergei Organov
Peter Hansen <[EMAIL PROTECTED]> writes:
[...]
> Then it's a good reason we had this thread, so you could learn something 
> *crucial* to understanding Python and writing non-buggy code: name 
> binding versus variables which occupy fixed memory locations like in 
> some other languages.  This has to be by far the most frequent area that 
> newcomer's trip up.  But that's another story...

I, as a newcomer, don't have much trouble understanding the binding vs
the assignment by themselves. What does somewhat confuse is dual role of
the "=" operator, -- sometimes it means "bind" and other times it means
"assign", right? For me it seems that the language would be less
error-prone and easier to grok if these two operations had different
names/syntax (thinking about lisp "let" vs "set"), though it seems to be
too drastic change even for Python3000.

-- Sergei.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Raymond Hettinger
[Dan Christensen]
> It's true that this runs at the same speed as the del variants on my
> machine.  That's not too surprising to me, but I still don't
> understand why the del variants are more than 5% faster than the first
> version.

Understanding it involves looking at implementation specific details
such as the overallocation scheme for growing lists and the performance
of your machine's underlying memory allocator.


> Once this is understood, is it something that could be optimized?
> It's pretty common to rebind a variable to a new value, and if
> this could be improved 5%, that would be cool.

Sorry, that's not how the language works.  Something like
"a=range(10)" means:
* build a NEW list for 10 elements
* THEN assign it to the variable "a"
* which THEN reduces the ref count to the previous binding for "a"
* and THEN IF the ref count is zero, free the list previously bound to
"a"

In other words, if "a" is already bound to a large list, then the above
assignment necessarily creates a second, non-overlapping list in
memory.

However, if you write "a=None;a=range(10)", then the original list
gets freed BEFORE the new list is created and the system has a chance
to re-use that large, contiguous block of memory.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Sion Arrowsmith
Fredrik Lundh <[EMAIL PROTECTED]> wrote:
>except that arguments along the line of "if the syntax is not obj.method(),
>it's not OO enough" are likely to be mostly ignored.
>
>(nobody's going to be impressed by yet another "len(obj) isn't OO" variant)

Does that suggest that what's needed is clear(obj) and __clear__
methods? 8-)

-- 
\S -- [EMAIL PROTECTED] -- http://www.chaos.org.uk/~sion/
  ___  |  "Frankly I have no feelings towards penguins one way or the other"
  \X/  |-- Arthur C. Clarke
   her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-13 Thread Fredrik Lundh
Raymond Hettinger wrote:

> Also, in the python-dev world, making something "more OO"  is neither a
> virtue nor a vice.

except that arguments along the line of "if the syntax is not obj.method(),
it's not OO enough" are likely to be mostly ignored.

(nobody's going to be impressed by yet another "len(obj) isn't OO" variant)





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Raymond Hettinger
> > * the request is inane, the underlying problem is trivial, and the
> > relevant idiom is fundamental (api expansions should be saved for rich
> > new functionality and not become cluttered with infrequently used
> > redundant entries)
>
> Is this sort of editorialising fair, or just a way of not-so-subtly
> encouraging Guido to reject the whole idea, now and forever?

Bah.  Guido is not stupid, nor easily misled.  Both the pros and cons
were quipped with abrupt perjoratives so the bullet points could be
stated succinctly and with a bit of levity.  The translation to
verbose, soft, politically correct statements is self-evident.

"request is inane" --> "A generation of python programmers has found
list clearing to be like other parts of the language that you get used
to very quickly and do not prove to be a problem in practice.  The
request is in the same category as others which challenge api choices
made 16 years ago; in particular, the decision to have compact APIs
where the named methods do not duplicate functionality provided by
syntax using operators and keywords. The request is less of a bug
report and more a rejection of Guido's sense of design and his
subsequent experience using his own language."

"underlying problem is trivial" --> "Books such as the Python Pocket
Reference or Python in a Nutshell are able to cover this idiom with
just a single sentence. Once known and understood, the application of
the current-way-to-do-it is immediate, compact, and effective."

"the existing alternatives are a bit perlish" --> "Both alternatives
involve a bit of inventiveness in combining two ideas (either the del
keyword and its interaction with slicing notation or the assignment of
an emtpy list to a slice).  Each approach has a visual appearance of
being a syntax trick.  The effect contrasts with much of the rest of
the language where it is usually possible to write code is a way that
can be read and understood by non-python programmers.  The existing
solution trades readability for the succinctness of a compact
syntactical idiom."



> A list.clear method will make deleting items from a list more OO,
> consistent with almost everything else you do to lists, and less
> procedural. This is especially true if clear() takes an optional index (or
> two), allowing sections of the list to be cleared, not just the entire
> list.

Don't shoot yourself in the foot here.  If you want to advocate
list.clear(), then you're hurting your chances by pushing for it to
take an optional argument.  Essentially, this amounts to an
unwillingness to use the del-keyword and to duplicate its functionality
with a named method.

Also, in the python-dev world, making something "more OO"  is neither a
virtue nor a vice.   It is better to argue for rich functionality,
succeptibility to errors, or dramatic improvements of existing
real-world code.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Fredrik Lundh
Mel Wilson wrote:

> > for item in seq:
> > L.append(item)
>
> Both extend and append have one-line slice equivalents,
> except that the equivalents have to keep referring to
> the length of the list.. (have to keep finding the
> len function.)

fwiw, the *tutorial* defines append and extend in terms of slicing...





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Steven Bethard
Raymond Hettinger wrote:
> [Steven Bethard]
>> I think these are all good reasons for adding a clear method, but being
>> that it has been so hotly contended in the past, I don't think it will
>> get added without a PEP.  Anyone out there willing to take out the best
>> examples from this thread and turn it into a PEP?
> 
> Something this small doesn't need a PEP.  I'll just send a note to
> Guido asking for a pronouncement.

Thanks.  It'd be nice to have something to refer to the next time this 
comes up.

> Here's a draft list of pros and cons (any changes or suggestions are
> welcome):
> 
> Pros:
> -
> 
> * s.clear() is more obvious in intent
> 
> * easier to figure-out, look-up, and remember than either s[:]=[] or
> del s[:]
> 
> * parallels the api for dicts, sets, and deques (increasing the
> expecation that lists will too)
>
> * the existing alternatives are a bit perlish
> 

I'd snip the one below.  It doesn't really contribute anything, and was 
sarcastic in the first place. ;)

> * the OP is shocked, SHOCKED that python got by for 16 years without
> list.clear()
> 
> 
> Cons:
> -
> 
> * makes the api fatter (there are already two ways to do it)
> 
> * expanding the api makes it more difficult to write classes that can
> be polymorphically substituted for lists
> 
> * learning slices is basic to the language (this lesson shouldn't be
> skipped)
> 
> * while there are valid use cases for re-using lists, the technique is
> already overused for unsuccessful attempts to micro-optimize (creating
> new lists is surprisingly fast)
> 

In fairness, if we're going to drop the "SHOCKED" comment, we should 
drop the first two clauses of the point below (but the "relevant idiom" 
part is clearly a good point).

> * the request is inane, the underlying problem is trivial, and the
> relevant idiom is fundamental (api expansions should be saved for rich
> new functionality and not become cluttered with infrequently used
> redundant entries)

Other than those minor edits, I think you've pretty much gotten 
everything.  I look forward to a pronouncement.

Thanks again!

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread gry
A perspective that I haven't seen raised here is inheritance.
I often say
mylist = []
if I'm done with the current contents and just want a fresh list.

But the cases where I have really needed list.clear [and laboriously
looked for it and ended up with
   del l[:]
were when the object was my own class that inherits from list, adding
some state and other functionality.  Of course I *could* have added my
own 'clear' function member, but I *wanted* it to behave like a
standard
python list in it's role of maintaining a sequence of processing steps.
So, I end up doing
   del current_process[:]
which, IMO, looks a bit queer, especially when current_process
object is a fairly elaborate data object.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Richie Hindle

[Mystilleef]
> Lists should have a clear method. But what's shocking is that
> it doesn't seem obvious to others. list.clear() is a whole lot more
> readable, intuitive, "flowable" and desirable than [the alternatives]

+1 to all of that.

-- 
Richie Hindle
[EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Mystilleef
I agree. Lists should have a clear method. But what's shocking is that
it doesn't seem obvious to others. list.clear() is a whole lot more
readable, intuitive, "flowable" and desirable than del list. Or maybe I
haven't had enough coffee this morning. I'd go as far as saying all
container objects should have a clear method. This is the reason why
those Rubyist whine about Python being inconsistent and not being "OO
enough."

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Fredrik Lundh
Peter Hansen wrote:

> You're spending a lot of time trying to convince me I'm wrong

no, I'm just posting observable facts in response to various "I'm too
lazy to look this up, but I'll assume that things are this way" posts.

> Thankfully (to Georg)

it's fixed in the wiki too...

talking about the wiki, are you and rurpy the same person, btw?  the
implied "I'm going to pretend that you've never done anything to im-
prove the situation" insinuations are getting a bit tiresome...





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Peter Hansen
Fredrik Lundh wrote:
> Peter Hansen wrote:
>>One of very first things newcomers learn (I believe, though I don't know
>>how soon the tutorial teaches it)
> 
> let's see.  lists are introduced on page 19, a more extensive discussion of 
> lists is
> found on page 33, the del statement appears on page 34, and the dir() function
> is introduced on page 46.

You're spending a lot of time trying to convince me I'm wrong, yet until 
  Georg did something about it just now, nowhere in those sections did 
it talk about [:] specifically, which is sort of the whole point.  It is 
*not* obvious that although [x:] and [:y] and [x:y] take subsets of 
things, that leaving them both out is either legal or useful.

Thankfully (to Georg), it now is.  (Well, I haven't read it yet, but 
I'll take his word for it.)

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Mel Wilson
Steven D'Aprano wrote:
> Convenience and obviousness are important for APIs -- that's why lists
> have pop, extend and remove methods. The only difference I can see between
> a hypothetical clear and these is that clear can be replaced with a
> one-liner, while the others need at least two, e.g. for extend:
> 
> for item in seq: 
> L.append(item)

Both extend and append have one-line slice equivalents,
except that the equivalents have to keep referring to
the length of the list.. (have to keep finding the
len function.)

Mel.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Mel Wilson
Alan Morgan wrote:
> In article <[EMAIL PROTECTED]>,
> Raymond Hettinger <[EMAIL PROTECTED]> wrote:
>>* s.clear() is more obvious in intent
> 
> Serious question: Should it work more like "s=[]" or more like
> "s[:]=[]".  I'm assuming the latter, but the fact that there is
> a difference is an argument for not hiding this operation behind
> some syntactic sugar. 


It has to work more like s[:]=[]

It's not easy for an object method to bind a whole different 
object to the first object's name.  Generally only 
statements can affect namespaces (hence the uses of del that
everyone remembers.)

Mel.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Fredrik Lundh
Peter Hansen wrote:

> One of very first things newcomers learn (I believe, though I don't know
> how soon the tutorial teaches it)

let's see.  lists are introduced on page 19, a more extensive discussion of 
lists is
found on page 33, the del statement appears on page 34, and the dir() function
is introduced on page 46.

(page numbers from the current pytut PDF copy)

 



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Peter Hansen
Duncan Booth wrote:
> Peter Hansen wrote:
> Looking in the 'obvious' place in the Tutorial, section 5.1 'More on
> Lists' I found in the immediately following section 5.2 'The del 
> statement': 
> 
>>There is a way to remove an item from a list given its index instead
>>of its value: the del statement. Unlike the pop()) method which
>>returns a value, the del keyword is a statement and can also be used
>>to remove slices from a list (which we did earlier by assignment of an
>>empty list to the slice). 

I saw that section too, but was scanning for any example of wiping out 
the whole list.  As you point out, it's not mentioned.  I don't think 
there's even an example of slicing with no arguments [:] for copying a 
list (i.e. on the right side of the =), and it's reasonable to assume (I 
originally did, as I recall) that this would be some kind of error...

> Both of these talk about ways to remove slices from a list. Perhaps the 
> wording could be clearer to make it obvious that they can also be used to 
> clear a list entirely (using the word 'clear' would certainly help people 
> Googling for the answer). So maybe 'this can even change the size of the 
> list or clear it completely' would be a good change for 3.1.4.

This is quite true.  After all, who imagines when offered a "slice of 
cake" that a slice might be the entire thing!  The concept of "slice" in 
English strongly implies a *subset*, not the whole, so if we're not 
going to get a .clear() method, I do believe that the various uses of 
[:] should be much more explicitly pointed out in the docs.  At least 
we'd have a ready place to point to in the tutorial, instead of this 
discussion cropping up every month.

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Peter Hansen
Alan Morgan wrote:
> Ah, but if you know your basic python then you wouldn't be looking for
> s.clear() in the first place; you'd just use s[:]=[] (or s=[], whichever
> is appropriate).  

One of very first things newcomers learn (I believe, though I don't know 
how soon the tutorial teaches it) is to use "dir()" on objects.  The 
clear() method would show up there and quickly attract attention. 
Neither of the other techniques are likely to be discovered as quickly. 
  (Well, okay, s=[] would be, I'm sure, but to many newcomers that might 
"feel wrong" as the way to empty out a list, but then we're back to 
wondering how often there's really a usecase for doing so.)

> Personally, it seems more *reasonable* to me, a novice python user,
> for s.clear() to behave like s=[] (or, perhaps, for s=[] and s[:]=[] to
> mean the same thing).  The fact that it can't might be an argument for
> not having it in the first place.

Then it's a good reason we had this thread, so you could learn something 
*crucial* to understanding Python and writing non-buggy code: name 
binding versus variables which occupy fixed memory locations like in 
some other languages.  This has to be by far the most frequent area that 
newcomer's trip up.  But that's another story...

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Steven D'Aprano
On Thu, 13 Apr 2006 09:11:31 +0200, Fredrik Lundh wrote:

> Peter Hansen wrote:
> 
>> It's not even clear that extend needs two lines:
>>
>>  >>> s = range(5)
>>  >>> more = list('abc')
>>  >>> s[:] = s + more
>>  >>> s
>> [0, 1, 2, 3, 4, 'a', 'b', 'c']
>>
>> Okay, it's not obvious, but I don't think s[:]=[] is really any more
>> obvious as a way to clear the list.
>>
>> Clearly .extend() needs to be removed from the language as it is an
>> unnecessary extension to the API using slicing
> 
> you just flunked the "what Python has to do to carry out a certain operation"
> part of the "how Python works, intermediate level" certification.

So implementation details are part of the language now? 

Out of curiosity, are those things that Python has to do the same for
CPython, Jython, IronPython and PyPy?

Personally, I think it is a crying shame that we're expected to be experts
on the specific internals of the Python interpreter before we're allowed
to point out that "only one obvious way to do it" just is not true, no
matter what the Zen says.

>>> L = []
>>> L.append(0)
>>> L[:] = L + [1]
>>> L[:] += [2]
>>> L[len(L):] = [3]
>>> L.__setslice__(len(L), -1, [4])
>>> L.__setslice__(sys.maxint, sys.maxint, [5])
>>> L += [6]
>>> L
[0, 1, 2, 3, 4, 5, 6]

That's at least seven ways to do an append, and it is a double-standard to
declare that slice manipulation is the One and Only True obvious way
to clear a list, but slice manipulation is not obvious enough for
appending to a list.

No doubt under the hood, these seven ways are implemented differently.
They certainly differ in their obviousness, and I'm willing to bet that
practically nobody thinks that the slicing methods are more obvious than
append. Perhaps we're not Dutch. I daresay one method is better, faster,
or more efficient than the others, but remember the warning against
premature optimisation.

Whenever I see "only one obvious way to do it" being used as a knee-jerk
excuse for rejecting any change, my eyes roll. Nobody wants Python to
turn into Perl plus the kitchen sink, but it isn't as if Python is some
absolutely minimalist language with two objects and three functions. There
is no shortage of "more than one way to do it" convenience methods,
functions and even entire modules. And that's why Python is such a fun,
easy to use language: because most things in Python are just convenient.
When you want to append to a list, or insert into a list, you don't have
to muck about with slices, you call the obvious list method.

And so it should be for clearing all or part of a list.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Steven D'Aprano
On Thu, 13 Apr 2006 07:49:11 +, Duncan Booth wrote:

> Raymond Hettinger wrote:
> 
>> Cons:
>> -
>> 
>> * learning slices is basic to the language (this lesson shouldn't be
>> skipped)
>> 
> also, a clear method would simply clear the entire list. You still need to 
> learn the assigning to/deleting slices technique any time you want to clear 
> out part of a list.

A clear method does not *necessarily* clear only the entire list. That's
an choice that can be made. I for one would vote +1 for clear taking an
optional index or slice, or two indices, to clear only a part of the list.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Georg Brandl
Duncan Booth wrote:
> Peter Hansen wrote:
> 
>>> * learning slices is basic to the language (this lesson shouldn't be
>>> skipped)
>> 
>> And yet it doesn't appear to be in the tutorial.  I could have missed 

> Both of these talk about ways to remove slices from a list. Perhaps the 
> wording could be clearer to make it obvious that they can also be used to 
> clear a list entirely (using the word 'clear' would certainly help people 
> Googling for the answer). So maybe 'this can even change the size of the 
> list or clear it completely' would be a good change for 3.1.4.

I added two examples of clearing a list to the section about slice assignment
and del.

Georg
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Duncan Booth
Raymond Hettinger wrote:

> Cons:
> -
> 
> * learning slices is basic to the language (this lesson shouldn't be
> skipped)
> 
also, a clear method would simply clear the entire list. You still need to 
learn the assigning to/deleting slices technique any time you want to clear 
out part of a list.

Every so often I still get an "oh, I didn't know Python could do *that* 
moment", just had one now:

>>> s = range(10)
>>> s[::2] = reversed(s[::2])
>>> s
[8, 1, 6, 3, 4, 5, 2, 7, 0, 9]

I've no idea when I might need it, but it just never occurred to me before 
that you can also assign/del non-contiguous slices.

The symmetry does breaks down a bit here as assigning to an extended slice 
only lets you assign a sequence of the same length as the slice, so you 
can't delete an extended slice by assignment, only by using del.

>>> s = range(10)
>>> del s[::2]
>>> s
[1, 3, 5, 7, 9]
>>> s = range(10)
>>> s[::2] = []

Traceback (most recent call last):
  File "", line 1, in -toplevel-
s[::2] = []
ValueError: attempt to assign sequence of size 0 to extended slice of size 
5
>>> 
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Duncan Booth
Peter Hansen wrote:

>> * learning slices is basic to the language (this lesson shouldn't be
>> skipped)
> 
> And yet it doesn't appear to be in the tutorial.  I could have missed 
> it, but I've looked in a number of the obvious places, without
> actually going through it (again) from start to finish.  Also,
> googling for "slice site:docs.python.org", you have to go to the
> *sixth* entry before you can find the first mention of "del x[:]" and
> what it does.  I think given the current docs it's possible to learn
> all kinds of things about slicing and still not make the non-intuitive
> leap that "del x[slice]" is actually how you spell "delete contents of
> list in-place". 

Looking in the 'obvious' place in the Tutorial, section 5.1 'More on
Lists' I found in the immediately following section 5.2 'The del 
statement': 

> There is a way to remove an item from a list given its index instead
> of its value: the del statement. Unlike the pop()) method which
> returns a value, the del keyword is a statement and can also be used
> to remove slices from a list (which we did earlier by assignment of an
> empty list to the slice). 

The 'earlier showing assignment of an empty list to a slice' is a reference 
to section 3.1.4 'Lists': 

> Assignment to slices is also possible, and this can even change the
> size of the list: 
> 
> 
 # Replace some items:
> ... a[0:2] = [1, 12]
 a
> [1, 12, 123, 1234]
 # Remove some:
> ... a[0:2] = []
 a
> [123, 1234]
> 

Both of these talk about ways to remove slices from a list. Perhaps the 
wording could be clearer to make it obvious that they can also be used to 
clear a list entirely (using the word 'clear' would certainly help people 
Googling for the answer). So maybe 'this can even change the size of the 
list or clear it completely' would be a good change for 3.1.4.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Fredrik Lundh
Peter Hansen wrote:

> It's not even clear that extend needs two lines:
>
>  >>> s = range(5)
>  >>> more = list('abc')
>  >>> s[:] = s + more
>  >>> s
> [0, 1, 2, 3, 4, 'a', 'b', 'c']
>
> Okay, it's not obvious, but I don't think s[:]=[] is really any more
> obvious as a way to clear the list.
>
> Clearly .extend() needs to be removed from the language as it is an
> unnecessary extension to the API using slicing

you just flunked the "what Python has to do to carry out a certain operation"
part of the "how Python works, intermediate level" certification.





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-13 Thread Fredrik Lundh
Peter Hansen wrote:

> > * learning slices is basic to the language (this lesson shouldn't be
> > skipped)
>
> And yet it doesn't appear to be in the tutorial.

oh, please.

slices are explained in the section on strings, and in the section on lists,
and used to define the behaviour of the list methods in the second section
on lists, ...

> I could have missed it, but I've looked in a number of the obvious places

http://docs.python.org/tut/node5.html#SECTION00514

section 3.1.2 contains an example that shows to remove stuff from a list,
in place.

if you want a clearer example, please consider donating some of your time
to the pytut wiki:

http://pytut.infogami.com/





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Dan Christensen
Raymond Hettinger <[EMAIL PROTECTED]> writes:

> Felipe Almeida Lessa writes:
> 
> > I love benchmarks, so as I was testing the options, I saw something very
> > strange:
> >
> > $ python2.4 -mtimeit 'x = range(10); '
> > 100 loops, best of 3: 6.7 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); del x[:]'
> > 100 loops, best of 3: 6.35 msec per loop
> >
> > Why the first benchmark is the slowest? I don't get it... could someone
> > test this, too?
>
> It is an effect of the memory allocator and fragmentation.  The first
> builds up a list with increasingly larger sizes.  

I don't see what you mean by this.  There are many lists all of
the same size.  Do you mean some list internal to the memory
allocator? 

> It periodically
> cannot grow in-place because something is in the way (some other
> object) so it needs to move its current entries to another, larger
> block and grow from there.  In contrast, the other entries are reusing
> a the previously cleared out large block.
>
> Just for grins, replace the first with"
>   'x=None; x=range(10)'
> The assignment to None frees the reference to the previous list and
> allows it to be cleared so that its space is immediately available to
> the new list being formed by range().

It's true that this runs at the same speed as the del variants on my
machine.  That's not too surprising to me, but I still don't
understand why the del variants are more than 5% faster than the first
version.

Once this is understood, is it something that could be optimized?
It's pretty common to rebind a variable to a new value, and if
this could be improved 5%, that would be cool.  But maybe it
wouldn't affect anything other than such micro benchmarks.

Dan
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Terry Reedy

"Peter Hansen" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> It's not even clear that extend needs two lines:
>
> >>> s = range(5)
> >>> more = list('abc')
> >>> s[:] = s + more
> >>> s
> [0, 1, 2, 3, 4, 'a', 'b', 'c']

This is not the same as list.extend because it makes a separate 
intermediate list instead of doing the extension completely in place. 
However, the following does mimic .extend.

>>> s=range(5)
>>> s[len(s):] = list('abc')
>>> s
[0, 1, 2, 3, 4, 'a', 'b', 'c']

So. at the cost of writing and calling len(s), you are correct that .extend 
is not necessary.

Terry Jan Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Alan Morgan
In article <[EMAIL PROTECTED]>,
Peter Hansen  <[EMAIL PROTECTED]> wrote:
>Alan Morgan wrote:
>> Right.  I was wondering what would happen in this case:
>> 
>> s=[1,2,3]
>> t=s
>> s.clear()
>> t   # [] or [1,2,3]?? 
>> 
>> If you know your basic python it is "obvious" what would happen
>> if you do s=[] or s[:]=[] instead of s.clear() and I guess it is
>> equally "obvious" which one s.clear() must mimic.  I'm still not
>> used to dealing with mutable lists.
>
>If you know your basic python :-), you know that s[:] = [] is doing the 
>only thing that s.clear() could possibly do,

Ah, but if you know your basic python then you wouldn't be looking for
s.clear() in the first place; you'd just use s[:]=[] (or s=[], whichever
is appropriate).  IOW, the people most in need of s.clear() are those
least likely to be able to work out what it is actually doing.
Personally, it seems more *reasonable* to me, a novice python user,
for s.clear() to behave like s=[] (or, perhaps, for s=[] and s[:]=[] to
mean the same thing).  The fact that it can't might be an argument for
not having it in the first place.

Alan
-- 
Defendit numerus
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Peter Hansen
Steven D'Aprano wrote:
> Convenience and obviousness are important for APIs -- that's why lists
> have pop, extend and remove methods. The only difference I can see between
> a hypothetical clear and these is that clear can be replaced with a
> one-liner, while the others need at least two, e.g. for extend:
> 
> for item in seq: 
> L.append(item)

It's not even clear that extend needs two lines:

 >>> s = range(5)
 >>> more = list('abc')
 >>> s[:] = s + more
 >>> s
[0, 1, 2, 3, 4, 'a', 'b', 'c']

Okay, it's not obvious, but I don't think s[:]=[] is really any more 
obvious as a way to clear the list.

Clearly .extend() needs to be removed from the language as it is an 
unnecessary extension to the API using slicing, which everyone should 
already know about...

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Peter Hansen
Alan Morgan wrote:
> Right.  I was wondering what would happen in this case:
> 
> s=[1,2,3]
> t=s
> s.clear()
> t   # [] or [1,2,3]?? 
> 
> If you know your basic python it is "obvious" what would happen
> if you do s=[] or s[:]=[] instead of s.clear() and I guess it is
> equally "obvious" which one s.clear() must mimic.  I'm still not
> used to dealing with mutable lists.

If you know your basic python :-), you know that s[:] = [] is doing the 
only thing that s.clear() could possibly do, which is changing the 
contents of the list which has the name "s" bound to it (and which might 
have other names bound to it, just like any object in Python).  It 
*cannot* be doing the same as "s=[]" which does not operate on the list 
but creates an entirely new one and rebinds the name "s" to it.

The only possible answer for your question above is "t is s" and "t == 
[]" because you haven't rebound the names.

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Alan Morgan
In article <[EMAIL PROTECTED]>,
Steven D'Aprano  <[EMAIL PROTECTED]> wrote:
>On Wed, 12 Apr 2006 15:36:47 -0700, Alan Morgan wrote:
>
>> Serious question: Should it work more like "s=[]" or more like
>> "s[:]=[]".  I'm assuming the latter, but the fact that there is
>> a difference is an argument for not hiding this operation behind
>> some syntactic sugar. 
>
>Er, I don't see how it can possibly work like s = []. That merely
>reassigns a new empty list to the name s, it doesn't touch the existing
>list (which may or may not be garbage collected soon/immediately
>afterwards).

Right.  I was wondering what would happen in this case:

s=[1,2,3]
t=s
s.clear()
t   # [] or [1,2,3]?? 

If you know your basic python it is "obvious" what would happen
if you do s=[] or s[:]=[] instead of s.clear() and I guess it is
equally "obvious" which one s.clear() must mimic.  I'm still not
used to dealing with mutable lists.

Alan
-- 
Defendit numerus
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Steven D'Aprano
On Wed, 12 Apr 2006 15:36:47 -0700, Alan Morgan wrote:

> Serious question: Should it work more like "s=[]" or more like
> "s[:]=[]".  I'm assuming the latter, but the fact that there is
> a difference is an argument for not hiding this operation behind
> some syntactic sugar. 

Er, I don't see how it can possibly work like s = []. That merely
reassigns a new empty list to the name s, it doesn't touch the existing
list (which may or may not be garbage collected soon/immediately
afterwards).

As far as I know, it is impossible -- or at least very hard -- for an
object to know which namesspaces it is in, so it can reassign one name but
not the others. Even if such a thing was possible, I think it is an
absolutely bad idea.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Steven D'Aprano
On Wed, 12 Apr 2006 12:40:52 -0700, Raymond Hettinger wrote:

> Something this small doesn't need a PEP.  I'll just send a note to
> Guido asking for a pronouncement.

Raymond, if you're genuinely trying to help get this sorted in the
fairest, simplest way possible, I hope I speak for everyone when
I say thank you, your efforts are appreciated.

But there is this:

> * the request is inane, the underlying problem is trivial, and the
> relevant idiom is fundamental (api expansions should be saved for rich
> new functionality and not become cluttered with infrequently used
> redundant entries)

Is this sort of editorialising fair, or just a way of not-so-subtly
encouraging Guido to reject the whole idea, now and forever? 

Convenience and obviousness are important for APIs -- that's why lists
have pop, extend and remove methods. The only difference I can see between
a hypothetical clear and these is that clear can be replaced with a
one-liner, while the others need at least two, e.g. for extend:

for item in seq: 
L.append(item)

Here is another Pro for your list:

A list.clear method will make deleting items from a list more OO,
consistent with almost everything else you do to lists, and less
procedural. This is especially true if clear() takes an optional index (or
two), allowing sections of the list to be cleared, not just the entire
list.


-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Alan Morgan
In article <[EMAIL PROTECTED]>,
Raymond Hettinger <[EMAIL PROTECTED]> wrote:
>[Steven Bethard]
>> I think these are all good reasons for adding a clear method, but being
>> that it has been so hotly contended in the past, I don't think it will
>> get added without a PEP.  Anyone out there willing to take out the best
>> examples from this thread and turn it into a PEP?
>
>Something this small doesn't need a PEP.  I'll just send a note to
>Guido asking for a pronouncement.
>
>Here's a draft list of pros and cons (any changes or suggestions are
>welcome):
>
>Pros:
>-
>
>* s.clear() is more obvious in intent

Serious question: Should it work more like "s=[]" or more like
"s[:]=[]".  I'm assuming the latter, but the fact that there is
a difference is an argument for not hiding this operation behind
some syntactic sugar. 

Alan
-- 
Defendit numerus
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Ville Vainio
Ville Vainio wrote:

> Assigning to slices is much less important, and is something I always
> never do (and hence forget).

ALMOST never, of course.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Ville Vainio
Raymond Hettinger wrote:

> * easier to figure-out, look-up, and remember than either s[:]=[] or
> del s[:]

Easier is an understatement - it's something you figure out
automatically. When I want to do something w/ an object, looking at its
methods (via code completion) is the very first thing.

> * the OP is shocked, SHOCKED that python got by for 16 years without
> list.clear()

I'm sure you realize I was being sarcastic...

> * learning slices is basic to the language (this lesson shouldn't be
> skipped)

Assigning to slices is much less important, and is something I always
never do (and hence forget).

> * the request is inane, the underlying problem is trivial, and the
> relevant idiom is fundamental (api expansions should be saved for rich
> new functionality and not become cluttered with infrequently used
> redundant entries)

I understand that these are the main arguments. However, as it stands
there is no one *obvious* way to clear a list in-place. I agree that
it's rare to even need it, but when you do a it's a little bit of a
surprise.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Peter Hansen
Raymond Hettinger wrote:
> Cons:
> -
> * learning slices is basic to the language (this lesson shouldn't be
> skipped)

And yet it doesn't appear to be in the tutorial.  I could have missed 
it, but I've looked in a number of the obvious places, without actually 
going through it (again) from start to finish.  Also, googling for 
"slice site:docs.python.org", you have to go to the *sixth* entry before 
you can find the first mention of "del x[:]" and what it does.  I think 
given the current docs it's possible to learn all kinds of things about 
slicing and still not make the non-intuitive leap that "del x[slice]" is 
actually how you spell "delete contents of list in-place".

> * while there are valid use cases for re-using lists, the technique is
> already overused for unsuccessful attempts to micro-optimize (creating
> new lists is surprisingly fast)

Not just valid use-cases, but ones for which "x = []" is entirely buggy, 
yet not obviously so, especially to newcomers.

> * the request is inane, the underlying problem is trivial, and the
> relevant idiom is fundamental (api expansions should be saved for rich
> new functionality and not become cluttered with infrequently used
> redundant entries)

The first phrase is insulting and untrue, the second merely untrue (as 
even your own list of pros and cons shows), and the last completely 
valid and highly relevant, yet not overriding.

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Raymond Hettinger
[Felipe Almeida Lessa]
> > I love benchmarks, so as I was testing the options, I saw something very
> > strange:
> >
> > $ python2.4 -mtimeit 'x = range(10); '
> > 100 loops, best of 3: 6.7 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); del x[:]'
> > 100 loops, best of 3: 6.35 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); x[:] = []'
> > 100 loops, best of 3: 6.36 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); del x'
> > 100 loops, best of 3: 6.46 msec per loop
> >
> > Why the first benchmark is the slowest? I don't get it... could someone
> > test this, too?

[Dan Christensen]
> I get similar behaviour.  No idea why.

It is an effect of the memory allocator and fragmentation.  The first
builds up a list with increasingly larger sizes.  It periodically
cannot grow in-place because something is in the way (some other
object) so it needs to move its current entries to another, larger
block and grow from there.  In contrast, the other entries are reusing
a the previously cleared out large block.

Just for grins, replace the first with"
  'x=None; x=range(10)'
The assignment to None frees the reference to the previous list and
allows it to be cleared so that its space is immediately available to
the new list being formed by range().

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Felipe Almeida Lessa
Em Qua, 2006-04-12 às 12:40 -0700, Raymond Hettinger escreveu:
> * the existing alternatives are a bit perlish

I love this argument =D! "perlish"... lol...

Cheers,

-- 
Felipe.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-12 Thread Raymond Hettinger
[Steven Bethard]
> I think these are all good reasons for adding a clear method, but being
> that it has been so hotly contended in the past, I don't think it will
> get added without a PEP.  Anyone out there willing to take out the best
> examples from this thread and turn it into a PEP?

Something this small doesn't need a PEP.  I'll just send a note to
Guido asking for a pronouncement.

Here's a draft list of pros and cons (any changes or suggestions are
welcome):

Pros:
-

* s.clear() is more obvious in intent

* easier to figure-out, look-up, and remember than either s[:]=[] or
del s[:]

* parallels the api for dicts, sets, and deques (increasing the
expecation that lists will too)

* the existing alternatives are a bit perlish

* the OP is shocked, SHOCKED that python got by for 16 years without
list.clear()


Cons:
-

* makes the api fatter (there are already two ways to do it)

* expanding the api makes it more difficult to write classes that can
be polymorphically substituted for lists

* learning slices is basic to the language (this lesson shouldn't be
skipped)

* while there are valid use cases for re-using lists, the technique is
already overused for unsuccessful attempts to micro-optimize (creating
new lists is surprisingly fast)

* the request is inane, the underlying problem is trivial, and the
relevant idiom is fundamental (api expansions should be saved for rich
new functionality and not become cluttered with infrequently used
redundant entries)

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Mel Wilson
Ville Vainio wrote:
> Fredrik Lundh wrote:
>>because Python already has a perfectly valid way to clear a list,
>>perhaps ?
>>
>>del l[:]
> 
> 
> Ok. That's pretty non-obvious but now that I've seen it I'll probably
> remember it. I did a stupid "while l: l.pop()" loop myself.

Actually, it's in the Library Reference (that we keep under 
our pillows) section 2.3.6.4 Mutable Sequence Types.

 Cheers,Mel.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Martin v. Löwis
Steven D'Aprano wrote:
>>> $ python2.4 -mtimeit 'x = range(10); '
>>> 100 loops, best of 3: 6.7 msec per loop
>>> $ python2.4 -mtimeit 'x = range(10); del x[:]'
>>> 100 loops, best of 3: 6.35 msec per loop
>>> $ python2.4 -mtimeit 'x = range(10); x[:] = []'
>>> 100 loops, best of 3: 6.36 msec per loop
>>> $ python2.4 -mtimeit 'x = range(10); del x'
>>> 100 loops, best of 3: 6.46 msec per loop
>>>
>>> Why the first benchmark is the slowest? I don't get it... could someone
>>> test this, too?
>> In the first benchmark, you need space for two lists: the old one and
>> the new one; 
> 
> Er, what new list? I see only one list, x = range(10), which is merely
> created then nothing done to it. Have I missed something?

See Duncan's explanation. This code is run many times, allocating many
lists. However, only two different lists exist at any point at time.

A Python list consists of two memory blocks: the list proper (a few
bytes), plus the "guts", i.e. a variable-sized block of pointers to
the objects. del x[:] frees the guts.

> I understood Felipe to be asking, why does it take longer to just create a
> list, than it takes to create a list AND then do something to it? 

Actually, the same code (deallocation of all integers and the list
blocks) appears in either case. However, in the one case it is triggered
explicitly, and before the new allocation; in the other case, the
allocation of the new objects occurs before the old ones are released.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Steven Bethard
John Salerno wrote:
> Steven Bethard wrote:
> 
>> I think these are all good reasons for adding a clear method, but 
>> being that it has been so hotly contended in the past, I don't think 
>> it will get added without a PEP.  Anyone out there willing to take out 
>> the best examples from this thread and turn it into a PEP?
> 
> What are the usual arguments against adding it?

I think most of the other folks have already given you the answers, but 
the main ones I remember:

(1) There are already at least two ways of spelling it. We don't need 
another.

(2) Clearing a list is a pretty uncommon operation. Usually just 
creating a new list is fine (and easier).

Disclaimer: I'm happy with adding list.clear(), so it's quite possible 
I'm misrepresenting the argument here.  I guess that's just all the more 
reason to have a PEP for it: to lay out all the pros and cons.

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Peter Hansen
John Salerno wrote:
> Steven Bethard wrote:
>>I think these are all good reasons for adding a clear method, but being 
>>that it has been so hotly contended in the past, I don't think it will 
>>get added without a PEP.  Anyone out there willing to take out the best 
>>examples from this thread and turn it into a PEP?
> 
> What are the usual arguments against adding it?

I think one is that it's very rare to need it.  Most of the time, just 
rebinding the name to a new empty list is easier, and possibly faster. 
The main problem with that approach is that in multithreaded code that 
can be a source of subtle bugs.  On the other hand, most things can be a 
source of subtle bugs in multithreaded code, so maybe that's not a good 
reason to add clear().

Saving us time answering the question repeatedly (and trying to justify 
the current state of affairs) might be justification enough...

-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Peter Hansen
Georg Brandl wrote:
> John Salerno wrote:
>>Steven Bethard wrote:
>>>I think these are all good reasons for adding a clear method, but being 
>>>that it has been so hotly contended in the past, I don't think it will 
>>>get added without a PEP.  Anyone out there willing to take out the best 
>>>examples from this thread and turn it into a PEP?
>>
>>What are the usual arguments against adding it?
> 
> 
> That there should be one obvious way to do it.
> 
> Yes, I know that it can be debated whether "del x[:]" is obvious, and
> fortunately I'm not the one to decide .

It's so "obvious", I couldn't even find it in the tutorial the other day 
when this thread came up.  It's a wonder I ever learned that one myself, 
and I don't know where I saw it.  .clear(), on the other hand, would 
clearly be easy to add/find in the tutorial... and we'd all save time 
answering the question over and over around here (valuable time we would 
then be able to using telling people who hadn't, to go and read the 
tutorial! :-) ).


-Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Georg Brandl
John Salerno wrote:
> Steven Bethard wrote:
> 
>> I think these are all good reasons for adding a clear method, but being 
>> that it has been so hotly contended in the past, I don't think it will 
>> get added without a PEP.  Anyone out there willing to take out the best 
>> examples from this thread and turn it into a PEP?
> 
> What are the usual arguments against adding it?

That there should be one obvious way to do it.

Yes, I know that it can be debated whether "del x[:]" is obvious, and
fortunately I'm not the one to decide .

Georg
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread John Salerno
Steven Bethard wrote:

> I think these are all good reasons for adding a clear method, but being 
> that it has been so hotly contended in the past, I don't think it will 
> get added without a PEP.  Anyone out there willing to take out the best 
> examples from this thread and turn it into a PEP?

What are the usual arguments against adding it?
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Steven Bethard
Steven D'Aprano wrote:
> On Tue, 11 Apr 2006 14:49:04 -0700, Ville Vainio wrote:
> 
>> John Salerno wrote:
>>
>>> Thanks guys, your explanations are really helpful. I think what had me
>>> confused at first was my understanding of what L[:] does on either side
>>> of the assignment operator. On the left, it just chooses those elements
>>> and edits them in place; on the right, it makes a copy of that list,
>>> right? (Which I guess is still more or less *doing* the same thing, just
>>> for different purposes)
>> Interestingly, if it was just a "clear" method nobody would be confused.
> 
> Even more importantly, you could say help(list.clear) and learn something
> useful, instead of trying help(del) and getting a syntax error.
[snip more reasons to add list.clear()]

I think these are all good reasons for adding a clear method, but being 
that it has been so hotly contended in the past, I don't think it will 
get added without a PEP.  Anyone out there willing to take out the best 
examples from this thread and turn it into a PEP?

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread John Salerno
Steven D'Aprano wrote:

> But name.clear() meaning "mutate the object referenced by name to the
> empty state" is a very natural candidate for a method, and I don't
> understand why lists shouldn't have it.

Funny this even comes up, because I was just trying to 'clear' a list 
the other day. But it sounds like it's been an issue for a while. :)
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Duncan Booth
Steven D'Aprano wrote:

> But that is precisely the same for the other timeit tests too.
> 
> for _i in _it:
> x = range(10)
Allocate list.
Allocate ob_item array to hold pointers to 1 objects
Allocate 99900 integer objects
setup list

> del x[:]
Calls list_clear which:
  decrements references to 99900 integer objects, freeing them
  frees the ob_item array

... next time round ...
> x = range(10)

Allocate new list list.
Allocate ob_item array probably picks up same memory block as last time
Allocate 99900 integer objects, probably reusing same memory as last time
setup list

> 
> etc.
> 
> The question remains -- why does it take longer to do X than it takes to
> do X and then Y?

>>> for _i in _it:
...   x = range(10); 

Allocate list.
Allocate ob_item array to hold pointers to 1 objects
Allocate 99900 integer objects
setup list

... next time round ...
Allocate another list
allocate a second ob_item array
allocate another 99900 integer objects
setup the list
then deletes the original list, decrements and releases original integers, 
frees original ob_item array.

Uses twice as much everything except the actual list object. The actual 
work done is the same but I guess there are likely to be more cache misses.
Also there is the question whether the memory allocation does or does not 
manage to reuse the recently freed blocks. With one large block I expect it 
might well end up reusing it, with two large blocks being freed alternately 
it might not manage to reuse either (but that is just a guess and maybe 
system dependant).
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Steven D'Aprano
On Wed, 12 Apr 2006 00:33:29 -0700, Serge Orlov wrote:

> 
> Felipe Almeida Lessa wrote:
>> Em Qua, 2006-04-12 às 11:36 +1000, Steven D'Aprano escreveu:
>> > On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote:
>> >
>> > > Felipe Almeida Lessa wrote:
>> > >> I love benchmarks, so as I was testing the options, I saw something very
>> > >> strange:
>> > >>
>> > >> $ python2.4 -mtimeit 'x = range(10); '
>> > >> 100 loops, best of 3: 6.7 msec per loop
>> > >> $ python2.4 -mtimeit 'x = range(10); del x[:]'
>> > >> 100 loops, best of 3: 6.35 msec per loop
>> > >> $ python2.4 -mtimeit 'x = range(10); x[:] = []'
>> > >> 100 loops, best of 3: 6.36 msec per loop
>> > >> $ python2.4 -mtimeit 'x = range(10); del x'
>> > >> 100 loops, best of 3: 6.46 msec per loop
>> > >>
>> > >> Why the first benchmark is the slowest? I don't get it... could someone
>> > >> test this, too?
>> > >
>> > > In the first benchmark, you need space for two lists: the old one and
>> > > the new one;
>> >
>> > Er, what new list? I see only one list, x = range(10), which is merely
>> > created then nothing done to it. Have I missed something?
>>
>> He's talking about the garbage collector.
> 
> To be exact the reason for two array is timeit.py. It doesn't place the
> code to time into a separate namespace but injects it into a for loop,
> so the actual code timed is:
> for _i in _it:
> x = range(10)
> and that makes two arrays with 100.000 items exist for a short time
> starting from second iteration.

But that is precisely the same for the other timeit tests too.

for _i in _it:
x = range(10)
del x[:]

etc.

The question remains -- why does it take longer to do X than it takes to
do X and then Y?


-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-12 Thread Serge Orlov

Felipe Almeida Lessa wrote:
> Em Qua, 2006-04-12 às 11:36 +1000, Steven D'Aprano escreveu:
> > On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote:
> >
> > > Felipe Almeida Lessa wrote:
> > >> I love benchmarks, so as I was testing the options, I saw something very
> > >> strange:
> > >>
> > >> $ python2.4 -mtimeit 'x = range(10); '
> > >> 100 loops, best of 3: 6.7 msec per loop
> > >> $ python2.4 -mtimeit 'x = range(10); del x[:]'
> > >> 100 loops, best of 3: 6.35 msec per loop
> > >> $ python2.4 -mtimeit 'x = range(10); x[:] = []'
> > >> 100 loops, best of 3: 6.36 msec per loop
> > >> $ python2.4 -mtimeit 'x = range(10); del x'
> > >> 100 loops, best of 3: 6.46 msec per loop
> > >>
> > >> Why the first benchmark is the slowest? I don't get it... could someone
> > >> test this, too?
> > >
> > > In the first benchmark, you need space for two lists: the old one and
> > > the new one;
> >
> > Er, what new list? I see only one list, x = range(10), which is merely
> > created then nothing done to it. Have I missed something?
>
> He's talking about the garbage collector.

To be exact the reason for two array is timeit.py. It doesn't place the
code to time into a separate namespace but injects it into a for loop,
so the actual code timed is:
for _i in _it:
x = range(10)
and that makes two arrays with 100.000 items exist for a short time
starting from second iteration.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-12 Thread Serge Orlov

Martin v. Löwis wrote:
> Felipe Almeida Lessa wrote:
> > I love benchmarks, so as I was testing the options, I saw something very
> > strange:
> >
> > $ python2.4 -mtimeit 'x = range(10); '
> > 100 loops, best of 3: 6.7 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); del x[:]'
> > 100 loops, best of 3: 6.35 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); x[:] = []'
> > 100 loops, best of 3: 6.36 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); del x'
> > 100 loops, best of 3: 6.46 msec per loop
> >
> > Why the first benchmark is the slowest? I don't get it... could someone
> > test this, too?
>
> In the first benchmark, you need space for two lists: the old one and
> the new one; the other benchmarks you need only a single block of
> memory (*). Concluding from here gets difficult - you would have to study
> the malloc implementation to find out whether it works better in one
> case over the other. Could also be an issue of processor cache: one
> may fit into the cache, but the other may not.

Addition to the previous message. Now I follow you :) There are indeed
two arrays and cache seems to be the second reason for slowdown, but
iterating backwards is also contributing to slowdown.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Serge Orlov
Martin v. Löwis wrote:
> Felipe Almeida Lessa wrote:
> > I love benchmarks, so as I was testing the options, I saw something very
> > strange:
> >
> > $ python2.4 -mtimeit 'x = range(10); '
> > 100 loops, best of 3: 6.7 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); del x[:]'
> > 100 loops, best of 3: 6.35 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); x[:] = []'
> > 100 loops, best of 3: 6.36 msec per loop
> > $ python2.4 -mtimeit 'x = range(10); del x'
> > 100 loops, best of 3: 6.46 msec per loop
> >
> > Why the first benchmark is the slowest? I don't get it... could someone
> > test this, too?
>
> In the first benchmark, you need space for two lists: the old one and
> the new one; the other benchmarks you need only a single block of
> memory (*).

I don't follow you here :)

> Concluding from here gets difficult - you would have to study
> the malloc implementation to find out whether it works better in one
> case over the other.

That's not the case here. The following program prints the same
addresses whether you comment or uncomment del
-
ids = range(10)
for i in xrange(10):
x = range(10)
#del x[:]
ids[i] = id(x)

print ids



> Could also be an issue of processor cache: one
> may fit into the cache, but the other may not.

That's not the reason, since the same time difference happens with
smaller arrays. I think the difference is how items are deallocated in
these two cases.

Calling del invokes list_ass_slice that deallocates from 0 to end
whereas ordinary removal of a list was changed to backward iteration
(the reason is in the comment:
/* Do it backwards, for Christian Tismer.
   There's a simple test case where somehow this reduces
   thrashing when a *very* large list is created and
   immediately deleted. */

Usually iterating from low addresses to higher addresses is better for
CPU. On my CPU (Pentium M, 1.7Ghz) it's 20% faster:

Here is my results:

C:\py>python -mtimeit "x = range(1); del x[:]"
1000 loops, best of 3: 213 usec per loop

C:\py>python -mtimeit "x = range(1); del x"
1000 loops, best of 3: 257 usec per loop

C:\py>python -mtimeit "x = range(1); "
1000 loops, best of 3: 258 usec per loop

C:\py>python -mtimeit "x = range(1000); del x[:]"
1 loops, best of 3: 21.4 usec per loop

C:\py>python -mtimeit "x = range(1000); del x"
1 loops, best of 3: 25.2 usec per loop

C:\py>python -mtimeit "x = range(1000); "
1 loops, best of 3: 25.6 usec per loop

I don't have a development environment on my computer so I can't test
my thoughts. I could be wrong about the reason.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Dan Christensen
Felipe Almeida Lessa <[EMAIL PROTECTED]> writes:

> I love benchmarks, so as I was testing the options, I saw something very
> strange:
>
> $ python2.4 -mtimeit 'x = range(10); '
> 100 loops, best of 3: 6.7 msec per loop
> $ python2.4 -mtimeit 'x = range(10); del x[:]'
> 100 loops, best of 3: 6.35 msec per loop
> $ python2.4 -mtimeit 'x = range(10); x[:] = []'
> 100 loops, best of 3: 6.36 msec per loop
> $ python2.4 -mtimeit 'x = range(10); del x'
> 100 loops, best of 3: 6.46 msec per loop
>
> Why the first benchmark is the slowest? I don't get it... could someone
> test this, too?

I get similar behaviour.  No idea why.

$ python2.4 -mtimeit 'x = range(10); '
100 loops, best of 3: 6.99 msec per loop
$ python2.4 -mtimeit 'x = range(10); del x[:]'
100 loops, best of 3: 6.49 msec per loop
$ python2.4 -mtimeit 'x = range(10); x[:] = []'
100 loops, best of 3: 6.47 msec per loop
$ python2.4 -mtimeit 'x = range(10); del x'
100 loops, best of 3: 6.6 msec per loop

Dan
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Felipe Almeida Lessa
Em Qua, 2006-04-12 às 11:36 +1000, Steven D'Aprano escreveu:
> On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote:
> 
> > Felipe Almeida Lessa wrote:
> >> I love benchmarks, so as I was testing the options, I saw something very
> >> strange:
> >> 
> >> $ python2.4 -mtimeit 'x = range(10); '
> >> 100 loops, best of 3: 6.7 msec per loop
> >> $ python2.4 -mtimeit 'x = range(10); del x[:]'
> >> 100 loops, best of 3: 6.35 msec per loop
> >> $ python2.4 -mtimeit 'x = range(10); x[:] = []'
> >> 100 loops, best of 3: 6.36 msec per loop
> >> $ python2.4 -mtimeit 'x = range(10); del x'
> >> 100 loops, best of 3: 6.46 msec per loop
> >> 
> >> Why the first benchmark is the slowest? I don't get it... could someone
> >> test this, too?
> > 
> > In the first benchmark, you need space for two lists: the old one and
> > the new one; 
> 
> Er, what new list? I see only one list, x = range(10), which is merely
> created then nothing done to it. Have I missed something?

He's talking about the garbage collector.

Talking about the GC, do you want to see something *really* odd?

$ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
range(10); '
100 loops, best of 3: 13 msec per loop

$ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
range(10); del x[:]'
100 loops, best of 3: 8.19 msec per loop

$ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
range(10); x[:] = []'
100 loops, best of 3: 8.16 msec per loop

$ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
range(10); del x'
100 loops, best of 3: 8.3 msec per loop


But in this case I got the answer (I think):
- When you explicitly delete the objects, the GC already know that it
can be collected, so it just throw the objects away.
- When we let the "x" variable continue to survive, the GC has to look
at all the 11 objects to see if they can be collected -- just to see
that it can't.

Also, IIRC "del x" is slower than "x = []" because removing a name from
the namespace is more expensive than just assigning something else to
it. Right?

> I understood Felipe to be asking, why does it take longer to just create a
> list, than it takes to create a list AND then do something to it? 

I see dead people... ;-)

-- 
Felipe.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-11 Thread Steven D'Aprano
On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote:

> Felipe Almeida Lessa wrote:
>> I love benchmarks, so as I was testing the options, I saw something very
>> strange:
>> 
>> $ python2.4 -mtimeit 'x = range(10); '
>> 100 loops, best of 3: 6.7 msec per loop
>> $ python2.4 -mtimeit 'x = range(10); del x[:]'
>> 100 loops, best of 3: 6.35 msec per loop
>> $ python2.4 -mtimeit 'x = range(10); x[:] = []'
>> 100 loops, best of 3: 6.36 msec per loop
>> $ python2.4 -mtimeit 'x = range(10); del x'
>> 100 loops, best of 3: 6.46 msec per loop
>> 
>> Why the first benchmark is the slowest? I don't get it... could someone
>> test this, too?
> 
> In the first benchmark, you need space for two lists: the old one and
> the new one; 

Er, what new list? I see only one list, x = range(10), which is merely
created then nothing done to it. Have I missed something?

I understood Felipe to be asking, why does it take longer to just create a
list, than it takes to create a list AND then do something to it? 



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-11 Thread Steven D'Aprano
On Tue, 11 Apr 2006 14:49:04 -0700, Ville Vainio wrote:

> John Salerno wrote:
> 
>> Thanks guys, your explanations are really helpful. I think what had me
>> confused at first was my understanding of what L[:] does on either side
>> of the assignment operator. On the left, it just chooses those elements
>> and edits them in place; on the right, it makes a copy of that list,
>> right? (Which I guess is still more or less *doing* the same thing, just
>> for different purposes)
> 
> Interestingly, if it was just a "clear" method nobody would be confused.

Even more importantly, you could say help(list.clear) and learn something
useful, instead of trying help(del) and getting a syntax error.
 
>>> help(del)
  File "", line 1
help(del)
   ^
SyntaxError: invalid syntax

I know Python isn't a purely OOP language, but in my opinion using
statements like del should be avoided when there is an easy and natural OO
way of doing it. Something like name.del() which means "delete the
reference to name in name's namespace" feels wrong, and may not even be
possible with Python's object model, so del name is a natural way to do
it. 

But name.clear() meaning "mutate the object referenced by name to the
empty state" is a very natural candidate for a method, and I don't
understand why lists shouldn't have it.

For lists, it would be natural to have a hypothetical clear() method
accept an index or slice as an argument, so that these are equivalent:

del L[:]  <=>  L.clear()
del L[n]  <=>  L.clear(n)
del L[a:b]  <=>  L.clear(slice(a, b))


# untested reference implementation:
class ClearableList(list):
def clear(self, obj=None):
if obj is None:
obj = slice(0, len(self))
if isinstance(obj, int):
self.__delitem__(obj)
elif isinstance(obj, slice):
self.__delslice__(obj.start, obj.stop)
else:
raise TypeError


-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Ville Vainio
John Salerno wrote:

> Thanks guys, your explanations are really helpful. I think what had me
> confused at first was my understanding of what L[:] does on either side
> of the assignment operator. On the left, it just chooses those elements
> and edits them in place; on the right, it makes a copy of that list,
> right? (Which I guess is still more or less *doing* the same thing, just
> for different purposes)

Interestingly, if it was just a "clear" method nobody would be confused.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread John Salerno
Fredrik Lundh wrote:
> John Salerno wrote:
> 
>> Steven Bethard wrote:
>>
>>
>>> lst[:] = []
>>> lst = []
>> What's the difference here?
> 
> L[:]= modifies the object in place, L=[] binds the variable to a
> new object.  compare and contrast:

Thanks guys, your explanations are really helpful. I think what had me 
confused at first was my understanding of what L[:] does on either side 
of the assignment operator. On the left, it just chooses those elements 
and edits them in place; on the right, it makes a copy of that list, 
right? (Which I guess is still more or less *doing* the same thing, just 
for different purposes)
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread John Salerno
Felipe Almeida Lessa wrote:

> You see? lst[:] removes all elements from the list that lst refers to,
> while lst = [] just creates a new list and discard the only one. The
> difference is, for example:

Thanks, your explanation was great!
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Duncan Smith
John Salerno wrote:
> Steven Bethard wrote:
> 
> 
>> lst[:] = []
>> lst = []
> 
> 
> What's the difference here?

>>> lst = [1,2,3]
>>> lst2 = lst
>>> lst[:] = []
>>> lst2
[]
>>> lst = [1,2,3]
>>> lst2 = lst
>>> lst = []
>>> lst2
[1, 2, 3]
>>>

Duncan
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Fredrik Lundh
John Salerno wrote:

> Steven Bethard wrote:
>
>
> > lst[:] = []
> > lst = []
>
> What's the difference here?

L[:]= modifies the object in place, L=[] binds the variable to a
new object.  compare and contrast:

>>> L = ["a", "b", "c"]
>>> M = L
>>> L
['a', 'b', 'c']
>>> M
['a', 'b', 'c']
>>> L is M
True
>>> L[:] = []
>>> L
[]
>>> M
[]
>>> L is M
True

>>> L = ["a", "b", "c"]
>>> M = L
>>> L
['a', 'b', 'c']
>>> M
['a', 'b', 'c']
>>> L = []
>>> L
[]
>>> M
['a', 'b', 'c']
>>> L is M
False





-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Felipe Almeida Lessa
Em Ter, 2006-04-11 às 17:56 +, John Salerno escreveu:
> Steven Bethard wrote:
> 
> 
> > lst[:] = []
> > lst = []
> 
> What's the difference here?

lst[:] = [] makes the specified slice become []. As we specified ":", it
transforms the entire list into [].

lst = [] assigns the value [] to the variable lst, deleting any previous
one.

This might help:

>>> lst = range(10)
>>> id(lst), lst
(-1210826356, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> lst[:] = []
>>> id(lst), lst
(-1210826356, [])

>>> lst = range(10)
>>> id(lst), lst
(-1210844052, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> lst = []
>>> id(lst), lst
(-1210826420, [])


You see? lst[:] removes all elements from the list that lst refers to,
while lst = [] just creates a new list and discard the only one. The
difference is, for example:

>>> lst = range(3)
>>> x = [lst, lst, lst]
>>> x
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
>>> lst[:] = []
>>> x
[[], [], []]

>>> lst = range(3)
>>> x = [lst, lst, lst]
>>> x
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
>>> lst = []
>>> x
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]

HTH,

-- 
Felipe.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-11 Thread John Salerno
Steven Bethard wrote:


> lst[:] = []
> lst = []

What's the difference here?
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Martin v. Löwis
Ville Vainio wrote:
> It's just that, when I have an object, and am wondering how I can clear
> it, I tend to look what methods it has first and go to google looking
> for "idioms" second.

I guess del on a list is not that common, so people tend to not know
that it works on lists (and slices!), too. It's too bad that lists have
a pop() method these days, so people can do x.pop() even if they don't
need the value, instead of doing del x[-1]. I don't think I ever needed
to del a slice except for clearing the entire list (and I don't need to
do that often, either - I just throw the list away).

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Martin v. Löwis
Felipe Almeida Lessa wrote:
> I love benchmarks, so as I was testing the options, I saw something very
> strange:
> 
> $ python2.4 -mtimeit 'x = range(10); '
> 100 loops, best of 3: 6.7 msec per loop
> $ python2.4 -mtimeit 'x = range(10); del x[:]'
> 100 loops, best of 3: 6.35 msec per loop
> $ python2.4 -mtimeit 'x = range(10); x[:] = []'
> 100 loops, best of 3: 6.36 msec per loop
> $ python2.4 -mtimeit 'x = range(10); del x'
> 100 loops, best of 3: 6.46 msec per loop
> 
> Why the first benchmark is the slowest? I don't get it... could someone
> test this, too?

In the first benchmark, you need space for two lists: the old one and
the new one; the other benchmarks you need only a single block of
memory (*). Concluding from here gets difficult - you would have to study
the malloc implementation to find out whether it works better in one
case over the other. Could also be an issue of processor cache: one
may fit into the cache, but the other may not.

Regards,
Martin

(*) plus, you also need the integer objects twice.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Ville Vainio
Steven Bethard wrote:

> If you feel really strongly about this though, you might consider
> writing up a PEP.  It's been contentious enough that there's not much
> chance of getting a change without one.

No strong feelings here, and I'm sure greater minds than me have
already hashed this over sufficiently.

It's just that, when I have an object, and am wondering how I can clear
it, I tend to look what methods it has first and go to google looking
for "idioms" second.

Perhaps "clear" method could be added that raises
PedagogicException("Use del lst[:], stupid!")? 

*ducks*

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Felipe Almeida Lessa
Em Ter, 2006-04-11 às 10:42 -0600, Steven Bethard escreveu:
> one of::
> 
>  del lst[:]
> 
>  lst[:] = []
> 
> or if you don't need to modify the list in place,
> 
>  lst = []
> 
> Personally, I tend to go Fredrik's route and use the first.

I love benchmarks, so as I was testing the options, I saw something very
strange:

$ python2.4 -mtimeit 'x = range(10); '
100 loops, best of 3: 6.7 msec per loop
$ python2.4 -mtimeit 'x = range(10); del x[:]'
100 loops, best of 3: 6.35 msec per loop
$ python2.4 -mtimeit 'x = range(10); x[:] = []'
100 loops, best of 3: 6.36 msec per loop
$ python2.4 -mtimeit 'x = range(10); del x'
100 loops, best of 3: 6.46 msec per loop

Why the first benchmark is the slowest? I don't get it... could someone
test this, too?

Cheers,

-- 
Felipe.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: list.clear() missing?!?

2006-04-11 Thread Steven Bethard
Ville Vainio wrote:
> I tried to clear a list today (which I do rather rarely, considering
> that just doing l = [] works most of the time) and was shocked, SHOCKED
> to notice that there is no clear() method. Dicts have it, sets have it,
> why do lists have to be second class citizens?

This gets brought up all the time (search the archives for your 
favorite), but your options are basically (renaming your list to lst for 
readability) one of::

 del lst[:]

 lst[:] = []

or if you don't need to modify the list in place,

 lst = []

Personally, I tend to go Fredrik's route and use the first.

If you feel really strongly about this though, you might consider 
writing up a PEP.  It's been contentious enough that there's not much 
chance of getting a change without one.

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Ville Vainio
Fredrik Lundh wrote:

> > I tried to clear a list today (which I do rather rarely, considering
> > that just doing l = [] works most of the time) and was shocked, SHOCKED
> > to notice that there is no clear() method. Dicts have it, sets have it,
> > why do lists have to be second class citizens?
>
> because Python already has a perfectly valid way to clear a list,
> perhaps ?
>
> del l[:]

Ok. That's pretty non-obvious but now that I've seen it I'll probably
remember it. I did a stupid "while l: l.pop()" loop myself.

> (lists are not mappings, so the duck typing argument don't really
> apply here.)

I was thinking of list as a "mutable collection", and clear() is
certainly a very natural operation for them.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list.clear() missing?!?

2006-04-11 Thread Fredrik Lundh
Ville Vainio wrote:

> I tried to clear a list today (which I do rather rarely, considering
> that just doing l = [] works most of the time) and was shocked, SHOCKED
> to notice that there is no clear() method. Dicts have it, sets have it,
> why do lists have to be second class citizens?

because Python already has a perfectly valid way to clear a list,
perhaps ?

del l[:]

(lists are not mappings, so the duck typing argument don't really
apply here.)





-- 
http://mail.python.org/mailman/listinfo/python-list


list.clear() missing?!?

2006-04-11 Thread Ville Vainio
I tried to clear a list today (which I do rather rarely, considering
that just doing l = [] works most of the time) and was shocked, SHOCKED
to notice that there is no clear() method. Dicts have it, sets have it,
why do lists have to be second class citizens?

-- 
http://mail.python.org/mailman/listinfo/python-list