[Python-ideas] Additional LRU cache introspection facilities

2021-01-12 Thread Steven D'Aprano
Is anyone else interested in additional introspection facilities for the functools.lru_cache? You can view the cache hit and miss statistics using the cache_info() method, but you can't see what values actually are in the cache. That would be useful to me. I propose a method: @functools.l

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Serhiy Storchaka
12.01.21 12:02, Steven D'Aprano пише: > Is anyone else interested in additional introspection facilities for the > functools.lru_cache? > > You can view the cache hit and miss statistics using the cache_info() > method, but you can't see what values actually are in the cache. That > would be us

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Steven D'Aprano
On Tue, Jan 12, 2021 at 04:32:14PM +0200, Serhiy Storchaka wrote: > 12.01.21 12:02, Steven D'Aprano пише: > > I propose a method: > > > > @functools.lru_cache() > > def function(arg): > > ... > > > > function.cache_export() > > > > that returns a dictionary {arg: value} repr

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Paul Moore
On Tue, 12 Jan 2021 at 17:04, Steven D'Aprano wrote: > My use-case is debugging functions that are using an LRU cache, > specifically complex recursive functions. I have some functions where: > > f(N) > > ends up calling itself many times, but not in any obvious pattern: > > f(N-1), f(N-2

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Christopher Barker
Is the implementation of lru_cache too opaque to poke into it without an existing method? Or write a quick monkey-patch? Sorry for not checking myself, but the ability to do that kind of thing is one of the great things about a dynamic open source language. -CHB On Tue, Jan 12, 2021 at 9:04 AM S

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Paul Moore
On Tue, 12 Jan 2021 at 17:16, Christopher Barker wrote: > > Is the implementation of lru_cache too opaque to poke into it without an > existing method? Or write a quick monkey-patch? > > Sorry for not checking myself, but the ability to do that kind of thing is > one of the great things about a

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Sebastian Kreft
Actually you can get the cache data from the python version, for that you need to force the use of the python version of functools def get_cache(f): freevars = f.__code__.co_freevars index = freevars.index('cache') return f.__closure__[index].cell_contents import importlib.abc class

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Paul Moore
On Tue, 12 Jan 2021 at 19:28, Sebastian Kreft wrote: > > Actually you can get the cache data from the python version, for that you > need to force the use of the python version of functools "as private as you can get" != "private" :-) I had a feeling there would be a way to do it, but wasn't bot

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Serhiy Storchaka
12.01.21 18:57, Steven D'Aprano пише: > Can you explain further why the cached function needs additional > syncronisation overhead? The cache uses a double-linked list to track what item is the oldest and changes it every time you call the cached function (move the found or new item to the beginn

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Serhiy Storchaka
12.01.21 21:28, Sebastian Kreft пише: > class ForcePythonLruCache(importlib.abc.MetaPathFinder): >     def find_spec(self, fullname, path, target=None): >         if fullname == '_functools': >             raise ImportError('_functools not available') You can just set sys.modules['_functools'] = N

[Python-ideas] Re: Additional LRU cache introspection facilities

2021-01-12 Thread Raymond Hettinger
> I propose a method: > ... > returns a dictionary {arg: value} representing the cache. > It wouldn't be the cache itself, just a shallow copy > of the cache data I recommend against going down this path. It exposes (and potentially locks in) implementation details such as how we distinguish po