RE: properly delete item during "for item in..."
> -Original Message- > From: [EMAIL PROTECTED] [mailto:python- > [EMAIL PROTECTED] On Behalf Of Ratko > Sent: Thursday, July 17, 2008 12:27 PM > To: python-list@python.org > Subject: properly delete item during "for item in..." > > Say you have something like this: > > for item in myList: >del item > > Would this actually delete the item from the list or just decrement > the reference counter because the item in myList is not associated > with name "item" anymore (but still is with myList[itemIndex])? In > other words, is "item" a temporary reference to myList[itemIndex] or > is it actually that reference that myList has stored? > > I am not sure if this question even makes any sense anymore. I've been > using python for years and never had any problems (and I don't now > either) but now that I had to revisit c++/STL, I had to deal about > these issues and was wondering how python does it. > Walk the list backwards when deleting. master = ['a', 'b', 'c', 'd', 'e', 'f', 'g'] print "Deletes nothing" a = master[:] print a for i in a: del i print a print print print "Deletes safely from end" a = master[:] print a for i in range(len(a)-1, -1, -1): print i if i % 2 == 0: print "removing ", master[i] del a[i] print "", a, if master[i] in a: print "Ooops, deleted wrong thing...", print print a print print print "Delete from front. Deletes wrong things and throws an exception..." a = master[:] print a #for i in range(len(a)-1, -1, -1): for i in range(len(a)): print i if i % 2 == 0: print "removing ", master[i] del a[i] print "", a, if master[i] in a: print "Ooops, deleted wrong thing...", print print a -- http://mail.python.org/mailman/listinfo/python-list
Re: properly delete item during "for item in..."
> > For dictionaries we can just iterate over values() or items() as > > opposed to itervalues() or iteritems() since that's technically a copy > > of values or items in the dict, right? > > No! In fact the whole point of iteritems and itervalues and iterkeys is > that they *DO NOT* make copies, so changing the dictionary out from > under them is a programming error. > > If you use dict.items(), dict.keys() or dict.values(), then you're OK, > because these methods *do* create new lists for both. That's what I meant, it just didn't come across correctly I guess. Thanks for clarifying these issues. I think I have a better understanding now. R -- http://mail.python.org/mailman/listinfo/python-list
Re: properly delete item during "for item in..."
Ratko wrote: On Jul 17, 9:57 am, mk <[EMAIL PROTECTED]> wrote: Gary Herron wrote: You could remove the object from the list with del myList[i] if you knew i. HOWEVER, don't do that while looping through the list! Changing a list's length will interact badly with the for loop's indexing through the list, causing the loop to mis the element following the deleted item. Jumping into a thread, I know how not to do it, but not how to do it properly? Iterating over a copy may _probably_ work: >>> t=['a', 'c', 'b', 'd'] >>> >>> for el in t[:]: del t[t.index(el)] >>> t [] However, is it really safe? Defining safe as "works reliably in every corner case for every indexable data type"? Con: suppose the data structure t is really, really big. Just deleting some items from t temporarily doubles the memory consumption. Would this work (safely) then? It does in my test cases but that of course doesn't prove it works in a general case... for item in myList: myList.remove(item) No. Same problem, The for loop iterates through the list by keeping and incrementing an internal index. Any modification of the list does not change the index correspondingly. One proper way: newList = [] for item in myList: if ... whatever... newList.append(item) myList = newList Another, using list comprehension (it's the same thing really as the above): myList = [item for item in myList if ... whatever...] For dictionaries we can just iterate over values() or items() as opposed to itervalues() or iteritems() since that's technically a copy of values or items in the dict, right? No! In fact the whole point of iteritems and itervalues and iterkeys is that they *DO NOT* make copies, so changing the dictionary out from under them is a programming error. If you use dict.items(), dict.keys() or dict.values(), then you're OK, because these methods *do* create new lists for both. Gary Herron R -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: properly delete item during "for item in..."
mk <[EMAIL PROTECTED]> wrote: > Iterating over a copy may _probably_ work: > > >>> t=['a', 'c', 'b', 'd'] > >>> > >>> for el in t[:]: > del t[t.index(el)] > > > >>> t > [] > > > However, is it really safe? Defining safe as "works reliably in every > corner case for every indexable data type"? No, because you cannot necessarily copy every indexable data type using t[:], and not all types will support index. Also it is inefficient to delete from the start of the list. If you are working with a list and deleting every object: del t[:] will suffice. If you don't want to delete everything then you could do: for index, el in enumerate(reversed(t)): if not wewant(el): del t[index] but the pythonic way is just: t[:] = [ el for el in t if wewant(el) ] or if you just want the filtered list and don't care about updating the original: t = [ el for el in t if wewant(el) ] > Con: suppose the data structure t is really, really big. Just deleting > some items from t temporarily doubles the memory consumption. No it doesn't. Copying a list doesn't copy any of the elements in the list, it just copies the references to those element. -- http://mail.python.org/mailman/listinfo/python-list
Re: properly delete item during "for item in..."
On Jul 17, 9:57 am, mk <[EMAIL PROTECTED]> wrote: > Gary Herron wrote: > > You could remove the object from the list with > > del myList[i] > > if you knew i. HOWEVER, don't do that while looping through the list! > > Changing a list's length will interact badly with the for loop's > > indexing through the list, causing the loop to mis the element following > > the deleted item. > > Jumping into a thread, I know how not to do it, but not how to do it > properly? > > Iterating over a copy may _probably_ work: > > >>> t=['a', 'c', 'b', 'd'] > >>> > >>> for el in t[:]: > del t[t.index(el)] > > >>> t > [] > > However, is it really safe? Defining safe as "works reliably in every > corner case for every indexable data type"? > > Con: suppose the data structure t is really, really big. Just deleting > some items from t temporarily doubles the memory consumption. Would this work (safely) then? It does in my test cases but that of course doesn't prove it works in a general case... for item in myList: myList.remove(item) For dictionaries we can just iterate over values() or items() as opposed to itervalues() or iteritems() since that's technically a copy of values or items in the dict, right? R -- http://mail.python.org/mailman/listinfo/python-list
Re: properly delete item during "for item in..."
Gary Herron wrote: You could remove the object from the list with del myList[i] if you knew i. HOWEVER, don't do that while looping through the list! Changing a list's length will interact badly with the for loop's indexing through the list, causing the loop to mis the element following the deleted item. Jumping into a thread, I know how not to do it, but not how to do it properly? Iterating over a copy may _probably_ work: >>> t=['a', 'c', 'b', 'd'] >>> >>> for el in t[:]: del t[t.index(el)] >>> t [] However, is it really safe? Defining safe as "works reliably in every corner case for every indexable data type"? Con: suppose the data structure t is really, really big. Just deleting some items from t temporarily doubles the memory consumption. -- http://mail.python.org/mailman/listinfo/python-list
Re: properly delete item during "for item in..."
Ratko wrote: Say you have something like this: for item in myList: del item Would this actually delete the item from the list or just decrement the reference counter because the item in myList is not associated with name "item" anymore (but still is with myList[itemIndex])? In other words, is "item" a temporary reference to myList[itemIndex] or is it actually that reference that myList has stored? The 'del' statement does not delete an object, it deletes a reference to an object. In this case, the variable item is deleted from the scope, and the referred-to object will have its reference counter decremented by 1. (But, as you surmise, not to zero, because the list will still reference it.) You could remove the object from the list with del myList[i] if you knew i. HOWEVER, don't do that while looping through the list! Changing a list's length will interact badly with the for loop's indexing through the list, causing the loop to mis the element following the deleted item. Gary Herron I am not sure if this question even makes any sense anymore. I've been using python for years and never had any problems (and I don't now either) but now that I had to revisit c++/STL, I had to deal about these issues and was wondering how python does it. Thanks, Ratko -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: properly delete item during "for item in..."
On Thu, 17 Jul 2008 09:27:27 -0700, Ratko wrote: > for item in myList: >del item > > Would this actually delete the item from the list or just decrement > the reference counter because the item in myList is not associated > with name "item" anymore (but still is with myList[itemIndex])? In > other words, is "item" a temporary reference to myList[itemIndex] or > is it actually that reference that myList has stored? The latter. Names are always bound to objects, you can't bind a name to another name or reference in Python. Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
properly delete item during "for item in..."
Say you have something like this: for item in myList: del item Would this actually delete the item from the list or just decrement the reference counter because the item in myList is not associated with name "item" anymore (but still is with myList[itemIndex])? In other words, is "item" a temporary reference to myList[itemIndex] or is it actually that reference that myList has stored? I am not sure if this question even makes any sense anymore. I've been using python for years and never had any problems (and I don't now either) but now that I had to revisit c++/STL, I had to deal about these issues and was wondering how python does it. Thanks, Ratko -- http://mail.python.org/mailman/listinfo/python-list
