New submission from Collin Winter <coll...@gmail.com>: This patch simplifies cPickle's complicated internal buffering system. The new version uses a single buffer closer to the Pickler object, flushing to a file object only when necessary. This avoids the overhead of several indirection layers for what are frequently very small writes.
Benchmarked against virgin trunk r71058 using "perf.py -r -b pickle,pickle_list,pickle_dict": pickle: Min: 2.225 -> 1.962: 13.37% faster Avg: 2.254 -> 1.994: 13.03% faster Significant (t=70.763434, a=0.95) pickle_dict: Min: 2.097 -> 1.418: 47.83% faster Avg: 2.136 -> 1.446: 47.75% faster Significant (t=214.900146, a=0.95) pickle_list: Min: 1.128 -> 0.777: 45.22% faster Avg: 1.152 -> 0.807: 42.65% faster Significant (t=169.789433, a=0.95) A similar patch for unpickling will follow. As issue 5670 and 5671 are committed, I'll update this patch to support them. This patch passes all the tests added in issue 5665. I would recommend reviewing that patch first. I'll port to py3k once this is reviewed for trunk. This patch is available on Rietveld at http://codereview.appspot.com/33070. ---------- components: Extension Modules files: cpickle_writes.patch keywords: needs review, patch messages: 85349 nosy: alexandre.vassalotti, collinwinter severity: normal stage: patch review status: open title: Speed up cPickle's pickling generally type: performance versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file13603/cpickle_writes.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue5683> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
