[issue21952] fnmatch.py can appear in tracemalloc diffs

[STINNER Victor]

STINNER Victor  added the comment:

Exposing internal caches in tracemalloc is not a bug but a deliberate choice. 
It's easy to write your own filters to ignore selected files (or even specific 
functions if you can locate them by line nubmers).

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21952] fnmatch.py can appear in tracemalloc diffs

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In 3.5+ lru_cache() is implemented in C. Is this issue still actual?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21952] fnmatch.py can appear in tracemalloc diffs

[STINNER Victor]

STINNER Victor added the comment:

> When using some filters and comparing snapshots, fnmatch will appear in the 
> statistic diffs, e.g.:

During the design of the tracemalloc module (PEP 454), it was decided to not 
filter traces in the C module, but filter traces on a snapshot object. The 
reason is to keep the C module simple. To debug, you can analyze snapshots on a 
fast machine.

Can you try to filter traces?

snapshot = snapshot.filter_traces(Filter(False, tracemalloc.__file__))

In you case, the top frame is in the fnmatch module, so you probably need to 
trace more than one thread (ex: tracemalloc.start(10)) and apply the filter on 
all frames, not only on the top frame:

snapshot = snapshot.filter_traces(Filter(False, tracemalloc.__file__, 
all_frames=True))

You can also ignore fnmatch & sre_compile modules (blacklist, exclusive 
filters), or only keep traces of the directory /home/antoine/llvmpy/llvm/ 
(whitelist, an inclusive filter).

> Perhaps it would be nice to manually cache the compiled re pattern, instead 
> of going through fnmatch's lru_cache thing which seems to make reports less 
> robust.

Ah, interesting point. Would you be interested to work on that?

I tried to use tracemalloc to detect memory leaks in test.regrtest, but it's 
very difficult to get a reliable output. See the issue #19816.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21952] fnmatch.py can appear in tracemalloc diffs

[Antoine Pitrou]

New submission from Antoine Pitrou:

When using some filters and comparing snapshots, fnmatch will appear in the 
statistic diffs, e.g.:

/home/antoine/34/lib/python3.4/fnmatch.py:70: size=824 B (+64 B), count=2 (+1), 
average=412 B
eeleak.py:37: size=2512 B (+0 B), count=3 (+0), average=837 B
/home/antoine/llvmpy/llvm/ee.py:109: size=976 B (+0 B), count=2 (+0), 
average=488 B
/home/antoine/llvmpy/llvm/core.py:401: size=904 B (+0 B), count=2 (+0), 
average=452 B
/home/antoine/34/lib/python3.4/sre_compile.py:488: size=856 B (+0 B), count=1 
(+0), average=856 B


Perhaps it would be nice to manually cache the compiled re pattern, instead of 
going through fnmatch's lru_cache thing which seems to make reports less robust.

--
components: Library (Lib)
messages: 222692
nosy: haypo, pitrou
priority: normal
severity: normal
status: open
title: fnmatch.py can appear in tracemalloc diffs
type: behavior
versions: Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com