Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Martin v. Löwis
The default should be what we've had though. The new settings cause a lot more collisions and resizes. Raymond, can you kindly point to an application that demonstrates this claim (in particular the a lot more part, which I'd translate to more than 20% more). I'm fine with reverting changes,

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Mark Shannon
Martin v. Löwis wrote: The default should be what we've had though. The new settings cause a lot more collisions and resizes. Raymond, can you kindly point to an application that demonstrates this claim (in particular the a lot more part, which I'd translate to more than 20% more). It is

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Antoine Pitrou
On Mon, 18 Jun 2012 15:28:24 +0100 Mark Shannon m...@hotpy.org wrote: But do they? The results of benchmarking would seem to suggest (at least on my test machine) that overly-sparse dicts are slower. Possibly due to increased cache misses. Or, at least, they are not faster. See the synthetic

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Maciej Fijalkowski
On Mon, Jun 18, 2012 at 5:04 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 18 Jun 2012 15:28:24 +0100 Mark Shannon m...@hotpy.org wrote: But do they? The results of benchmarking would seem to suggest (at least on my test machine) that overly-sparse dicts are slower. Possibly due

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Antoine Pitrou
On Mon, 18 Jun 2012 21:31:27 +0200 Maciej Fijalkowski fij...@gmail.com wrote: On Mon, Jun 18, 2012 at 5:04 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 18 Jun 2012 15:28:24 +0100 Mark Shannon m...@hotpy.org wrote: But do they? The results of benchmarking would seem to suggest

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Raymond Hettinger
On Jun 18, 2012, at 12:35 PM, Antoine Pitrou wrote: You are right. I was thinking 50 nanoseconds (which for a - relatively high-end - 3GHz CPU puts us at 150 cycles). The last guidance I read from Intel said that a cache miss was roughly as expensive as a floating-point divide. When a

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Steven D'Aprano
Raymond Hettinger wrote: Sorry I can make a more detailed post right now. I'll make time in the next couple of weeks to post some code and timings that document the collision counts, total memory size, and its affect on various dict use cases. Is there some way to instrument dictionary

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Antoine Pitrou
On Mon, 18 Jun 2012 13:46:25 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: On Jun 18, 2012, at 12:35 PM, Antoine Pitrou wrote: You are right. I was thinking 50 nanoseconds (which for a - relatively high-end - 3GHz CPU puts us at 150 cycles). The last guidance I read from

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Martin v. Löwis
On 18.06.2012 23:08, Steven D'Aprano wrote: Raymond Hettinger wrote: Sorry I can make a more detailed post right now. I'll make time in the next couple of weeks to post some code and timings that document the collision counts, total memory size, and its affect on various dict use cases.

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-14 Thread Mark Shannon
Raymond Hettinger wrote: On Jun 13, 2012, at 2:37 PM, Mark Shannon wrote: I think that for combined tables a growth factor of x2 is best, but I don't have any hard evidence to back that up. I believe that change should be reverted. You've undone work that was based on extensive testing

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-14 Thread Kristján Valur Jónsson
FYI, I had to visit those parameters for my PS3 port and cut down on the bombastic memory footprint of 2.7 dicts. Some the supposedly tunable parameters aren´t tunable at all. See http://blog.ccpgames.com/kristjan/2012/04/25/optimizing-the-dict/ Speed and memory are most often conflicting

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-14 Thread Raymond Hettinger
On Jun 14, 2012, at 4:32 AM, Kristján Valur Jónsson wrote: I would like to two or more compile time settings to choose from: Memory optimal, speed optimal, (and mix). A compile time option would be nice. The default should be what we've had though. The new settings cause a lot more

[Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-13 Thread Mark Shannon
Raymond Hettinger wrote: On Jun 13, 2012, at 10:35 AM, Eli Bendersky wrote: Did you mean to send this to the list, Raymond? Yes. I wanted to find-out whether someone approved changing all the dict tunable parameters. I thought those weren't supposed to have changed. PEP 412 notes that

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-13 Thread Raymond Hettinger
On Jun 13, 2012, at 2:37 PM, Mark Shannon wrote: I think that for combined tables a growth factor of x2 is best, but I don't have any hard evidence to back that up. I believe that change should be reverted. You've undone work that was based on extensive testing and timings of many python

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-13 Thread Terry Reedy
On 6/13/2012 9:15 PM, Raymond Hettinger wrote: On Jun 13, 2012, at 2:37 PM, Mark Shannon wrote: I think that for combined tables a growth factor of x2 is best, but I don't have any hard evidence to back that up. I believe that change should be reverted. You've undone work that was based on