On Nov 14, 2007 5:26 PM, Steven D'Aprano <[EMAIL PROTECTED]> wrote: > On Wed, 14 Nov 2007 18:16:25 +0100, Hrvoje Niksic wrote: > > > Aaron Watters <[EMAIL PROTECTED]> writes: > > > >> On Nov 12, 12:46 pm, "Michael Bacarella" <[EMAIL PROTECTED]> wrote: > >>> > >>> > It takes about 20 seconds for me. It's possible it's related to > >>> > int/long > >>> > unification - try using Python 2.5. If you can't switch to 2.5, try > >>> > using string keys instead of longs. > >>> > >>> Yes, this was it. It ran *very* fast on Python v2.5. > >> > >> Um. Is this the take away from this thread? Longs as dictionary keys > >> are bad? Only for older versions of Python? > > > > It sounds like Python 2.4 (and previous versions) had a bug when > > populating large dicts on 64-bit architectures. > > No, I found very similar behaviour with Python 2.5. > > > >> Someone please summarize. > > > > Yes, that would be good. > > > On systems with multiple CPUs or 64-bit systems, or both, creating and/or > deleting a multi-megabyte dictionary in recent versions of Python (2.3, > 2.4, 2.5 at least) takes a LONG time, of the order of 30+ minutes, > compared to seconds if the system only has a single CPU. Turning garbage > collection off doesn't help. > >
I can't duplicate this in a dual CPU (64 bit, but running in 32 bit mode with a 32 bit OS) system. I added keys to a dict until I ran out of memory (a bit over 22 million keys) and deleting the dict took about 8 seconds (with a stopwatch, so not very precise, but obviously less than 30 minutes). >>> d = {} >>> idx = 0 >>> while idx < 1e10: ... d[idx] = idx ... idx += 1 ... Traceback (most recent call last): File "<stdin>", line 2, in <module> MemoryError >>> len(d) 22369622 >>> del d -- http://mail.python.org/mailman/listinfo/python-list