Re: [Python-Dev] Lazy initialization of module global state

[Nick Coghlan]

On 8 September 2017 at 08:50, Neil Schemenauer  wrote:
> This should be pretty safe to do and should give a significant
> benefit in startup time and memory usage.

And given the extended pyc header being proposed in PEP 552, we'd be
able to include flags in the header to indicate that the pyc file was
compiled with this alternative behaviour. While that wouldn't be
needed for import, it would be useful for anyone trying to confirm
that the pyc file actually matches the corresponding source file.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Lazy initialization of module global state

[Neil Schemenauer]

This is an idea that came out of the lazy module loading (via AST
analysis), posted to python-ideas.  The essential idea is to split
the marshal data stored in the .pyc into smaller pieces and only
load the parts as they are accessed.  E.g. use a __getattr__ hook on
the module to unmarshal+exec the code.

I have a very early prototype:

https://github.com/warsaw/lazyimport/blob/master/lazy_compile.py

Would work like a "compile_all.py" tool.  It writes standard .pyc
files right now.  It is not there yet but I should use the AST
analysis, like the lazy module load stuff, to determine if things
have potential side-effects on module import.  Those things will get
loaded eagerly, like they do now.

Initially I was thinking of class definitions and functions but now
I realize any global state could get this treatment.  E.g. if you
have a large dictionary global, don't unmarshal it until someone
accesses the module attribute.

This should be pretty safe to do and should give a significant
benefit in startup time and memory usage.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com