Nathaniel Smith added the comment:
> Note: I don't understand what are domain, tag (address of the memory block?)
> or quantity (size of the memory block?).
Yes, the idea is basically the same as now, except with an extra const char *
specifying the "domain", so that one could keep separate track of different
kinds of allocations. So PyMem_Malloc would just call PyMem_RecordAlloc("heap",
ptr, size) (or act equivalently to something that called that, etc.), but
something like PyCuda might do PyMem_RecordAlloc("gpu", ptr, size) to track
allocations in GPU memory. All the tracing stuff in tracemalloc would be
awesome to have for GPU allocations, and it would hardly require any changes to
enable it. Ditto for other leakable resources like file descriptors or shmem
segments.
> I don't plan to store new information in tracemalloc. tracemalloc design is
> "simple": it stores a trace for each memory block. A memory block is
> identified by its address (void* pointer) and a trace is a tuple (size,
> traceback).
> Extending a trace would increase the memory footprint overhead of tracemalloc.
I think the extra footprint would be tiny? Logically, you'd index traces by
(domain, pointer) instead of (pointer), but since we expect that the number of
distinct domains is small (like... 1, in the common case) then the natural
implementation strategy would be to replace the current {pointer: trace}
mapping with a mapping {domain: {pointer: trace}}. So the total overhead would
be basically the overhead of allocating a single additional dict.
That said, I think having the domain argument *in the public hookable API* is
useful even if the tracemalloc hooks themselves decide to ignore it or just
ignore all calls that don't refer to the "heap" domain -- because it's
extremely cheap to put this in the API now while we're adding it, and it leaves
the door open for someone else to come along later and make an awesome gpu heap
tracker or whatever without having to first argue with python-dev and wait for
the cpython release cycle.
----------
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue26530>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
