Re: Optimisation Hints (dict processing and strings)

[Kent Johnson]

OPQ wrote:
for (2):
for k in hash.keys()[:]: # Note : Their may be a lot of keys here
  if len(hash[k])<2:
     del hash[k]


- use the dict.iter* methods to prevent building a list in memory. You 
shouldn't use these values directly to delete the entry as this could 
break the iterator:

for key in [k for (k, v) in hash.iteritems () if len (v) < 2]:
    del hash (key)

I gonna try, but think that would be overkill: a whole list has to be
computed !
Yes, but it is smaller than the list returned by hash.keys(), so it should be a win over what you 
were doing originally. Plus it avoids a lookup (hash[k]) which may improve the speed also.

BTW I have long assumed that iterating key, value pairs of a dict using iteritems() is faster than 
iterating with keys() followed by a lookup, since the former method should be able to avoid actually 
hashing the key and looking it up.

I finally wrote a test, and my assumption seems to be correct; using iteritems() is about 1/3 faster 
for simple keys.

Here is a simple test:
##
d = dict((i, i) for i in range(10000))
def withItems(d):
    for k,v in d.iteritems():
        pass
def withKeys(d):
    for k in d:
        d[k]
from timeit import Timer
for fn in [withItems, withKeys]:
    name = fn.__name__
    timer = Timer('%s(d)' % name, 'from __main__ import d, %s' % name)
    print name, timer.timeit(1000)
##
I get
withItems 0.980311184801
withKeys 1.37672944466
Kent
--
http://mail.python.org/mailman/listinfo/python-list