[issue5557] Byte-code compilation uses excessive memory

Georg Brandl added the comment: Closing, as without a specific issue to fix it is unlikely that this will change. -- nosy: +georg.brandl resolution: -> wont fix status: pending -> closed ___ Python tracker ___

[issue5557] Byte-code compilation uses excessive memory

Changes by Serhiy Storchaka : -- status: open -> pending ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue5557] Byte-code compilation uses excessive memory

Changes by Zhiping Deng : -- nosy: +Zhiping.Deng ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue5557] Byte-code compilation uses excessive memory

Changes by Collin Winter : -- nosy: +collinwinter ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue5557] Byte-code compilation uses excessive memory

Antoine Pitrou added the comment: If you want editable data, you could use json instead of pickle. The simplejson library has very fast encoding/decoding (faster than cPickle according to its author). -- ___ Python tracker

[issue5557] Byte-code compilation uses excessive memory

Tom Goddard added the comment: I agree that having such large Python code files is a rare circumstance and optimizing the byte-code compiler for that should be a low priority. Thanks for the cpickle suggestion. The Chimera session file Python code is mostly large nested dictionaries and sequen

[issue5557] Byte-code compilation uses excessive memory

Antoine Pitrou added the comment: When compiling a source file to bytecode, Python first builds a syntax tree in memory. It is very likely that the memory consumption you observe is due to the size of the syntax tree. It is also unlikely that someone else than you will want to modifying the pars

[issue5557] Byte-code compilation uses excessive memory

STINNER Victor added the comment: Python uses inefficent memory structure for integers. You should use a 3rd part library like numpy to manipulate large integer vectors. -- nosy: +haypo ___ Python tracker

[issue5557] Byte-code compilation uses excessive memory

Martin v. Löwis added the comment: It might be possible to make it more efficient. However, the primary purpose of source code is to support hand-written code, and such code should never run into such problems. So lowering the priority. If you want this resolved, it might be best if you provide

[issue5557] Byte-code compilation uses excessive memory

New submission from Tom Goddard : Bytecode compiling large Python files uses an unexpectedly large amount of memory. For example, compiling a file containing a list of 5 million integers uses about 2 Gbytes of memory while the Python file size is about 40 Mbytes. The memory used is 50 times the