On 2009-02-11 15:30, sjbrown wrote:
Can someone describe the details of how Python loads modules into memory? I assume once the .py file is compiled to .pyc that it is mmap'ed in. But that assumption is very naive. Maybe it uses an anonymous mapping? Maybe it does other special magic? This is all very alien to me, so if someone could explain it in terms that a person who never usually worries about memory could understand, that would be much appreciated.
Python will read the .py file and compile into bytecode. This bytecode will be written out to a .pyc file as a caching mechanism; if the .pyc exists and has a newer timestamp than the .py file, the .py file will not be read or compiled. The bytecode will simply be read from the .pyc file.
The .pyc file is not really a map of a memory structure, per se. It is simply a disk representation of the bytecode. This bytecode is *executed* by the Python VM to populate a module's namespace (which is just a dict) in memory. I believe it is then discarded.
Follow up: is this process different if the modules are loaded from a zipfile?
Only in that the .py or .pyc will be extracted from the zipfile instead of being read directly from disk.
-- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
