If you want to proceed in this direction, it would be better to do some more research into current CPU architectures and then build a VM optimized byte code storage object, which is well aligned, fits into today's caches and improves locality.
freeze.py could then write out this format as well, so that the object can directly point to the structure and have the OS deal with memory mapping and sharing byte code across processes. I've had some good performance increases when I used the above approach in the mxTextTools tagging engine, which is a low-level VM for tagging and searching in text. If compilers are made aware of the structure the VM will use, they may also be able to apply additional optimizations for faster byte code access, prefetching, etc. On 22.06.2020 09:15, Inada Naoki wrote: > On Mon, Jun 22, 2020 at 12:00 AM Guido van Rossum <gu...@python.org> wrote: >> >> >> I believe this was what Greg Stein's idea here was about. (As well as >> Jonathan Fine's in this thread?) But the current use of code objects makes >> this hard. Perhaps the code objects could have a memoryview object to hold >> the bytecode instead of a bytes object. >> > > memoryview is heavy object. Using memoryview instead of bytes object > will increase memory usage. > I think lightweight bytes-like object is better. My rough idea is: > > * New code and pyc format > * pyc has "rodata" segment > * It can be copied into single memory block, or can be mmapped. > * co_code should be aligned at least 2 bytes. > * code.co_code can point to memory block in "rodata". > * docstring, signature, and lnotab can be lazy load from "rodata". > * signature is serialized in JSON like format. > * Allow multiple modules in single file > * Reduce fileno when using mmap > * Merge more constants > * New Python object: PyROData. > * It is like read-only bytearray. > * But body may be mmap-ped, instead of malloc-ed > * code objects owns reference of PyROData. > * When PyROData is deallocated, it munmap or free "rodata" segment. > > Regards, > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jun 22 2020) >>> Python Projects, Coaching and Support ... https://www.egenix.com/ >>> Python Product Development ... https://consulting.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/ _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LIS4FGWJ4RH4VYP2HICPPAWS2VVZZMZ7/ Code of Conduct: http://python.org/psf/codeofconduct/
