On 19 Oct 2013 03:57, "Charles-François Natali" <cf.nat...@gmail.com> wrote:
>
> Hi,
>
> I'm happy to see this move forward!
Speaking of which... Charles-François, would you be willing to act as
BDFL-Delegate for this PEP? This will be a very useful new analysis tool,
and between yourself and Victor it looks like you'll be able to come up
with a solid API.
I just suggested that approach to Guido and he also liked the idea :)
Cheers,
Nick.
>
> > API
> > ===
> >
> > Main Functions
> > --------------
> >
> > ``clear_traces()`` function:
> >
> > Clear traces and statistics on Python memory allocations, and reset
> > the ``get_traced_memory()`` counter.
>
> That's nitpicking, but how about just ``reset()`` (I'm probably biased
> by oprofile's opcontrol --reset)?
>
> > ``get_stats()`` function:
> >
> > Get statistics on traced Python memory blocks as a dictionary
> > ``{filename (str): {line_number (int): stats}}`` where *stats* in a
> > ``(size: int, count: int)`` tuple, *filename* and *line_number* can
> > be ``None``.
>
> It's probably obvious, but you might want to say once what *size* and
> *count* represent (and the unit for *size*).
>
> > ``get_tracemalloc_memory()`` function:
> >
> > Get the memory usage in bytes of the ``tracemalloc`` module as a
> > tuple: ``(size: int, free: int)``.
> >
> > * *size*: total size of bytes allocated by the module,
> > including *free* bytes
> > * *free*: number of free bytes available to store data
>
> What's *free* exactly? I assume it's linked to the internal storage
> area used by tracemalloc itself, but that's not clear at all.
>
> Also, is the tracemalloc overhead included in the above stats (I'm
> mainly thinking about get_stats() and get_traced_memory()?
> If yes, I find it somewhat confusing: for example, AFAICT, valgrind's
> memcheck doesn't report the memory overhead, although it can be quite
> large, simply because it's not interesting.
>
> > Trace Functions
> > ---------------
> >
> > ``get_traceback_limit()`` function:
> >
> > Get the maximum number of frames stored in the traceback of a trace
> > of a memory block.
> >
> > Use the ``set_traceback_limit()`` function to change the limit.
>
> I didn't see anywhere the default value for this setting: it would be
> nice to write it somewhere, and also explain the rationale (memory/CPU
> overhead...).
>
> > ``get_object_address(obj)`` function:
> >
> > Get the address of the main memory block of the specified Python
object.
> >
> > A Python object can be composed by multiple memory blocks, the
> > function only returns the address of the main memory block.
>
> IOW, this should return the same as id() on CPython? If yes, it could
> be an interesting note.
>
> > ``get_object_trace(obj)`` function:
> >
> > Get the trace of a Python object *obj* as a ``(size: int,
> > traceback)`` tuple where *traceback* is a tuple of ``(filename: str,
> > lineno: int)`` tuples, *filename* and *lineno* can be ``None``.
>
> I find the "trace" word confusing, so it might be interesting to add a
> note somewhere explaining what it is ("callstack leading to the object
> allocation", or whatever).
>
> Also, this function leaves me a mixed feeling: it's called
> get_object_trace(), but you also return the object size - well, a
> vague estimate thereof. I wonder if the size really belongs here,
> especially if the information returned isn't really accurate: it will
> be for an integer, but not for e.g. a list, right? How about just
> using sys.getsizeof(), which would give a more accurate result?
>
> > ``get_trace(address)`` function:
> >
> > Get the trace of a memory block as a ``(size: int, traceback)``
> > tuple where *traceback* is a tuple of ``(filename: str, lineno:
> > int)`` tuples, *filename* and *lineno* can be ``None``.
> >
> > Return ``None`` if the ``tracemalloc`` module did not trace the
> > allocation of the memory block.
> >
> > See also ``get_object_trace()``, ``get_stats()`` and
> > ``get_traces()`` functions.
>
> Do you have example use cases where you want to work with a raw addresses?
>
> > Filter
> > ------
> >
> > ``Filter(include: bool, pattern: str, lineno: int=None, traceback:
> > bool=False)`` class:
> >
> > Filter to select which memory allocations are traced. Filters can be
> > used to reduce the memory usage of the ``tracemalloc`` module, which
> > can be read using the ``get_tracemalloc_memory()`` function.
> >
> > ``match(filename: str, lineno: int)`` method:
> >
> > Return ``True`` if the filter matchs the filename and line number,
> > ``False`` otherwise.
> >
> > ``match_filename(filename: str)`` method:
> >
> > Return ``True`` if the filter matchs the filename, ``False``
otherwise.
> >
> > ``match_lineno(lineno: int)`` method:
> >
> > Return ``True`` if the filter matchs the line number, ``False``
> > otherwise.
> >
> > ``match_traceback(traceback)`` method:
> >
> > Return ``True`` if the filter matchs the *traceback*, ``False``
> > otherwise.
> >
> > *traceback* is a tuple of ``(filename: str, lineno: int)`` tuples.
>
> Are those ``match`` methods really necessary for the end user, i.e.
> are they worth being exposed as part of the public API?
>
> > StatsDiff
> > ---------
> >
> > ``StatsDiff(differences, old_stats, new_stats)`` class:
> >
> > Differences between two ``GroupedStats`` instances.
> >
> > The ``GroupedStats.compare_to()`` method creates a ``StatsDiff``
> > instance.
> >
> > ``sort()`` method:
> >
> > Sort the ``differences`` list from the biggest difference to the
> > smallest difference. Sort by ``abs(size_diff)``, *size*,
> > ``abs(count_diff)``, *count* and then by *key*.
> >
> > ``differences`` attribute:
> >
> > Differences between ``old_stats`` and ``new_stats`` as a list of
> > ``(size_diff, size, count_diff, count, key)`` tuples. *size_diff*,
> > *size*, *count_diff* and *count* are ``int``. The key type depends
> > on the ``GroupedStats.group_by`` attribute of ``new_stats``: see the
> > ``Snapshot.top_by()`` method.
> >
> > ``old_stats`` attribute:
> >
> > Old ``GroupedStats`` instance, can be ``None``.
> >
> > ``new_stats`` attribute:
> >
> > New ``GroupedStats`` instance.
>
> Why keep references to ``old_stats`` and ``new_stats``?
> datetime.timedelta doesn't keep references to the date objects it was
> computed from.
>
> Also, if you sort the difference by default (which is a sensible
> choice), then the StatsDiff becomes pretty much useless, since you
> would just keep its ``differences`` attribute (sorted).
>
> > Snapshot
> > --------
> >
> > ``Snapshot(timestamp: datetime.datetime, traces: dict=None, stats:
> > dict=None)`` class:
> >
> > Snapshot of traces and statistics on memory blocks allocated by
Python.
>
>
> I'm confused.
> Why are get_trace(), get_object_trace(), get_stats() etc not methods
> of a Snapshot object?
> Is it because you don't store all the necessary information in a
> snapshot, or are they just some sort of shorthands, like:
> stats = get_stats()
> vs
> snapshot = Snapshot.create()
> stats = snapshot.stats
>
> > ``write(filename)`` method:
> >
> > Write the snapshot into a file.
>
> I assume it's in a serialized form, only readable by Snapshort.load() ?
> BTW, it's a nitpick and debatable, but write()/read() or load()/dump()
> would be more consistent (see e.g. pickle's load/dump).
>
> > Metric
> > ------
> >
> > ``Metric(name: str, value: int, format: str)`` class:
> >
> > Value of a metric when a snapshot is created.
>
> Alright, what's a metric again ;-) ?
>
> I don't know if it's customary, but having short examples would IMO be
nice.
>
> cf
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
