Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-26 Thread Kristján Valur Jónsson
[mailto:ncogh...@gmail.com] Sent: 24. apríl 2012 11:42 To: Kristján Valur Jónsson Cc: R. David Murray; Antoine Pitrou; python-dev@python.org Subject: Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903) On Tue, Apr 24, 2012 at 8:24 PM, Kristján Valur Jónsson krist

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-26 Thread Kristján Valur Jónsson
-Original Message- From: Martin v. Löwis [mailto:mar...@v.loewis.de] This is easy in a debug build, using sys.getobjects(). In a release build, you can use pympler: start = pympler.muppy.get_size(pympler.muppy.get_objects()) run_complicated_tests() end =

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Kristján Valur Jónsson
-Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of mar...@v.loewis.de Sent: 24. apríl 2012 17:44 To: python-dev@python.org Subject: Re: [Python-Dev] cpython: Implement PEP 412: Key

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Mark Shannon
: Implement PEP 412: Key-sharing dictionaries (closes #13903) Benchmarks should measure memory usage too, of course. Sadly that is not possible in standard cPython. It's actually very easy in standard CPython, using sys.getsizeof. Yes, you can query each python object about how big it thinks

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Kristján Valur Jónsson
-Original Message- Take a look at the benchmark suite at http://hg.python.org/benchmarks/ The test runner has an -m option that profiles memory usage, you could take a look at how that is implemented Yes, out of process monitoring of memory as reported by the OS. We do gather

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Martin v. Löwis
Benchmarks should measure memory usage too, of course. Sadly that is not possible in standard cPython. It's actually very easy in standard CPython, using sys.getsizeof. Yes, you can query each python object about how big it thinks it is. What I'm speaking of is more like: start_allocs,

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread Kristján Valur Jónsson
PEP 412: Key-sharing dictionaries (closes #13903) On Mon, 23 Apr 2012 22:22:18 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 23 Apr 2012 17:24:57 +0200 benjamin.peterson python-check...@python.org wrote: http://hg.python.org/cpython/rev/6e5855854a2e changeset: 76485

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread Antoine Pitrou
On Tue, 24 Apr 2012 10:24:16 + Kristján Valur Jónsson krist...@ccpgames.com wrote: Btw, this is of great interest to me at the moment, our Shanghai engineers are screaming at the memory waste incurred by dictionaries. A 10 item dictionary consumes 1/2k on 32 bits, did you know this?

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread Nick Coghlan
On Tue, Apr 24, 2012 at 8:24 PM, Kristján Valur Jónsson krist...@ccpgames.com wrote: Perhaps I should write about this on my blog.  Updating the memory allocation macro layer in cPython for embedding is something I'd be inclined to contribute, but it will involve a large amount of

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread martin
Benchmarks should measure memory usage too, of course. Sadly that is not possible in standard cPython. It's actually very easy in standard CPython, using sys.getsizeof. Btw, this is of great interest to me at the moment, our Shanghai engineers are screaming at the memory waste incurred by

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-23 Thread Antoine Pitrou
On Mon, 23 Apr 2012 17:24:57 +0200 benjamin.peterson python-check...@python.org wrote: http://hg.python.org/cpython/rev/6e5855854a2e changeset: 76485:6e5855854a2e user:Benjamin Peterson benja...@python.org date:Mon Apr 23 11:24:50 2012 -0400 summary: Implement PEP 412:

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-23 Thread R. David Murray
On Mon, 23 Apr 2012 22:22:18 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 23 Apr 2012 17:24:57 +0200 benjamin.peterson python-check...@python.org wrote: http://hg.python.org/cpython/rev/6e5855854a2e changeset: 76485:6e5855854a2e user:Benjamin Peterson