[issue36485] Add a way to clear all caches

[Brett Cannon]

Brett Cannon  added the comment:

RE: "And do you think polling for a new magic attribute is the right 
approach?": my thinking behind that idea is that by standardizing the function 
name it's easy to tell if there's a cache, but you can also do away with the 
registration with a 3 lines of code. To me, the priority is clearing caches on 
a per-module basici and having a clear-all mechanism can be beneficial, not the 
other way around.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Ma Lin]

Ma Lin  added the comment:

> My initial idea was to add a lightweight module cachesreg with two functions: 
> register() and clear_caches().

If it only has two functions, it could be a sub-module sys.cachesreg

Or a lifecycle module, as the name, dedicated to such kind of functions. 
Register callback functions for memory low, poweroff system, etc.
I don't want lifecycle module, just provide a possibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Anders Hovmöller]

Anders Hovmöller  added the comment:

I think this is a great idea. We would have needed this many times for tests 
over the years.

--
nosy: +Anders.Hovmöller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +12571

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

My initial idea was to add a lightweight module cachesreg with two functions: 
register() and clear_caches(). PR 12639 implements it.

But I like Brett's idea more, because it is simpler. The only disadvantage of 
it is that if you make a typo in __clearcache__, this function will be silently 
ignored.

I thought also about different levels of cachesreg.cachesreg.register() could 
take two arguments -- the level and the clearing function. Then 
cachesreg.clear_caches() could allow to clear only caches of specified level 
and smaller/larger.

Both PRs add clearing callbacks only for modules which already cleared in 
regrtests. There are more caches, and with implementing any of these ideas it 
will be easier to add clearing caches in other modules.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Ma Lin]

Ma Lin  added the comment:

I suggest the documentation be written in more detail.

For example, in __clearcache__'s section, state explicitly that this magic 
function is for module-level cache, and it will be invoked by 
sys.clear_caches().

Maybe also introduce the background: some caches may grow unlimitedly, 
sys.clear_caches() gives the user a chance to empty them.

--
nosy: +Ma Lin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Michael Foord]

Michael Foord  added the comment:

Tests codify knowledge about the system under test, so it doesn't matter that 
the test suite has to know how to clear caches. It's specifically a good thing 
that the test writer knows which caches exist and need clearing, and how to do 
it. The harder thing mighty be determining what scope to do the clearing (per 
test, class or module) bit unittest exposes hooks for fixtures at those points 
for anything that needs doing automatically.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Michael Foord]

Michael Foord  added the comment:

> On 30 Mar 2019, at 23:48, Michael Foord  wrote:
> 
> 
> Michael Foord  added the comment:
> 
> An auto-magic cache clearing mechanism is really tempting. I tend to agree 
> with Raymond though, if code needs and progress a cache clearing mechanism it 
> should be treated and accessible. 

* exposes (not progress)
* tested  (not treated)

Sorry. 
> 
> They're are probably some problematic caches still within unittest however. 
> Do test results still keep alive all tracebacks until test reporting?
> 
> --
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Michael Foord]

Michael Foord  added the comment:

An auto-magic cache clearing mechanism is really tempting. I tend to agree with 
Raymond though, if code needs and progress a cache clearing mechanism it should 
be treated and accessible. 

They're are probably some problematic caches still within unittest however. Do 
test results still keep alive all tracebacks until test reporting?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Not sure that I agree there is a testing need to clear all caches regardless of 
what they do. Test code should explicitly state whether it relies on a 
particular cache being cleared at some particular point in time.

Also, the concept of "need to clear all caches" isn't well-formed.  Would you 
want sys.intern caches to be cleared? What about the internal caches in SQLite? 

And do you think polling for a new magic attribute is the right approach?  We 
would get looser coupling and better control by letting modules register 
themselves as needed.

--- re.py ---

sys.register_cache_clear_function(callback=purge, doc='pattern cache and re 
cache')

--- ipaddress.py --

sys.register(IPv4Address.is_private.is_getter.cache_clear, doc='check for 
private networks)

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +12563
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36485] Add a way to clear all caches

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Some modules have caches. There is a need to clear all tests before running 
tests. Brett proposed to add in all modules with caches a function with the 
standardized name which is responsible for clearing module related caches. [1]

The proposed PR adds a new function clear_caches() in the sys module. It 
iterates all imported modules and calls function __clearcache__() if it is 
defined.

def clear_caches():
for mod in reversed(list(sys.modules.values())):
if hasattr(mod, '__clearcache__'):
mod.__clearcache__()

clear_caches() will be used in test.regrtest and can be used in user code. The 
PR defines also function __clearcache__ for modules which are cleared manually 
in the current code.

This is a preliminary implementation, bikeshedding is welcome.

[1] https://mail.python.org/pipermail/python-ideas/2019-March/056165.html

--
components: Library (Lib), Tests
messages: 339190
nosy: brett.cannon, ezio.melotti, michael.foord, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add a way to clear all caches
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com