Welcome to *pymemtrace*, a collection of tools for monitoring and analysing Python's memory usage including that of Python Extensions. These tools work at various levels, at different granularities and with different runtime costs so you can choose one appropriate to the problem at hand. They range from periodically measuring total memory usage down to reporting every malloc and free call per Python line or C function call.
The tools are: * *process* is a very lightweight way of logging the total memory usage at regular time intervals. It can plot memory over time with plotting programs such as gnuplot. Some examples https://pymemtrace.readthedocs.io/en/latest/examples/process.html * *cPyMemTrace* is a memory tracer written in C that can report total memory usage for every function call/return for both C and Python sections. Some examples: https://pymemtrace.readthedocs.io/en/latest/examples/c_py_mem_trace.html and a technical note on cPyMemTrace: https://pymemtrace.readthedocs.io/en/latest/tech_notes/cPyMemTrace.html. * *DTrace*: There are a number of D scripts that can trace the low level ``malloc()`` and ``free()`` system calls and report how much memory was allocated and by whom. Some DTrace examples: https://pymemtrace.readthedocs.io/en/latest/examples/dtrace.html and a technical note on DTrace https://pymemtrace.readthedocs.io/en/latest/tech_notes/dtrace.html * *trace_malloc* is a convenience wrapper around the Python standard library tracemalloc module. This can take memory snapshots before and after code blocks and show the change in memory caused by that code by module and line compensating for the cost of tracemalloc itself. Some trace_malloc examples https://pymemtrace.readthedocs.io/en/latest/examples/trace_malloc.html * *debug_malloc_stats* is a wrapper around the sys._debugmallocstats function that can take snapshots of memory before and after code execution and report the significant differences in the Python small object allocator. Some debug_malloc_stats examples https://pymemtrace.readthedocs.io/en/latest/examples/debug_malloc_stats.html Within the documentation is also some (rather limited, for the moment) advice on tracking down memory leaks with Python extensions. On PyPi: https://pypi.org/project/pymemtrace/ Project: https://github.com/paulross/pymemtrace Documentation: https://pymemtrace.readthedocs.io/en/latest/index.html I hope that you find this useful as I do! Paul Ross. _______________________________________________ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
