[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Antoine Pitrou pit...@free.fr added the comment: The patch was finally committed in r84653. Thanks to everyone who participated in this. -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Antoine Pitrou pit...@free.fr added the comment: For the record, here are the unladen swallow benchmark results against stock py3k: ### pickle ### Min: 1.644222 - 0.812691: 2.0232x faster Avg: 1.652311 - 0.814994: 2.0274x faster Significant (t=297.660908) Stddev: 0.00594 - 0.00207: 2.8732x smaller ### unpickle ### Min: 2.802217 - 0.751013: 3.7312x faster Avg: 2.807074 - 0.752646: 3.7296x faster Significant (t=980.311525) Stddev: 0.00446 - 0.00145: 3.0831x smaller ### pickle_dict ### Min: 0.744259 - 0.543992: 1.3681x faster Avg: 0.747806 - 0.545883: 1.3699x faster Significant (t=114.070170) Stddev: 0.00266 - 0.00293: 1.1014x larger ### pickle_list ### Min: 2.058899 - 1.212566: 1.6980x faster Avg: 2.066486 - 1.216711: 1.6984x faster Significant (t=269.964154) Stddev: 0.00534 - 0.00459: 1.1635x smaller ### unpickle_list ### Min: 1.458531 - 0.313154: 4.6576x faster Avg: 1.464023 - 0.319126: 4.5876x faster Significant (t=425.745063) Stddev: 0.00476 - 0.00367: 1.2976x smaller -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Antoine Pitrou pit...@free.fr added the comment: The patch doesn't apply cleanly anymore. Furthermore, I discovered some additional issues: - load, dump, loads and dumps from the _pickle module were never used because they were shadowed by the same functions in pickle.py - once the C functions above are enabled, they produce some bugs I'm working on an updated patch, fixing the aforementioned bugs and adding a buffer size limit. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Alexandre Vassalotti alexan...@peadrop.com added the comment: Antoine, I fixed these issues in the latest patch posted on Rietveld. Also, Skip added the buffer limit in Unladen Swallow (see msg112956). We just need to merge that. -- Added file: http://bugs.python.org/file18777/issue1694050_19001.diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Antoine Pitrou pit...@free.fr added the comment:
Here is a patch. Benchmark numbers:
* dumps():
./python -m timeit -s import pickle, io; d={(x, 'a'): x for x in
range(1)} pickle.dumps(d)
- before: 100 loops, best of 3: 7.47 msec per loop
- after: 100 loops, best of 3: 2.45 msec per loop
* loads():
./python -m timeit -s import pickle, io; d={(x, 'a'): x for x in
range(1)}; d=pickle.dumps(d) pickle.loads(d)
- before: 100 loops, best of 3: 12.1 msec per loop
- after: 100 loops, best of 3: 2.62 msec per loop
* dump():
./python -m timeit -s import pickle, io; d={(x, 'a'): x for x in
range(1)} pickle.dump(d, io.BytesIO())
- before: 100 loops, best of 3: 13.2 msec per loop
- after: 100 loops, best of 3: 2.54 msec per loop
* load():
./python -m timeit -s import pickle, io; d={(x, 'a'): x for x in
range(1)}; d=pickle.dumps(d) pickle.load(io.BytesIO(d))
- before: 100 loops, best of 3: 12.7 msec per loop
- after: 100 loops, best of 3: 11.6 msec per loop
As you can see, load() doesn't really benefit from the buffering improvements.
The three methods see quite massive speedups.
--
Added file: http://bugs.python.org/file18778/pickle_optimizations4.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9410
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Antoine Pitrou pit...@free.fr added the comment: Gosh. My patch is based on an outdated patch :( -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Antoine Pitrou pit...@free.fr added the comment: Ok, this patch merges my changes with Alexandre's previous patch. Performance is similar as the previous posted patch. -- Added file: http://bugs.python.org/file18779/pickle_optimizations5.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Alexandre Vassalotti alexan...@peadrop.com added the comment: Ah, that's right Skip. You did fixed it in Unladen Swallow's trunk. I will take a look at your solution. http://code.google.com/p/unladen-swallow/source/diff?spec=svn1167r=1038format=sidepath=/trunk/Modules/cPickle.cold_path=/trunk/Modules/cPickle.cold=988 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9410] Add Unladen Swallow's optimizations to Python 3's pickle.
Changes by Georg Brandl ge...@python.org: -- title: Add Unladden Swallow's optimizations to Python 3's pickle. - Add Unladen Swallow's optimizations to Python 3's pickle. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
