New submission from Sergey Fedoseev <fedoseev.ser...@gmail.com>:
Currently BytesIO.writelines() uses BytesIO.write() which returns number of
written bytes as Python ints, but these ints are not used by
BytesIO.writelines(), so avoiding creation of these ints can improve
performance of BytesIO.writelines() especially for a large number of small
lines.
Benchmarks:
python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s
"from io import BytesIO; lines = [b'']*10000;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 153 us
+- 3 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 126 us +- 4 us
Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 153 us +- 3
us -> [/home/sergey/tmp/cpython-venv/bin/python] 126 us +- 4 us: 1.22x faster
(-18%)
python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s
"from io import BytesIO; lines = [b'b']*10000;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 164 us
+- 2 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 142 us +- 1 us
Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 164 us +- 2
us -> [/home/sergey/tmp/cpython-venv/bin/python] 142 us +- 1 us: 1.16x faster
(-13%)
python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s
"from io import BytesIO; lines = [b'b'*10]*10000;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 387 us
+- 6 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 365 us +- 8 us
Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 387 us +- 6
us -> [/home/sergey/tmp/cpython-venv/bin/python] 365 us +- 8 us: 1.06x faster
(-5%)
python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s
"from io import BytesIO; lines = [b'b'*100]*10000;"
"BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 319 us
+- 5 us
/home/sergey/tmp/cpython-venv/bin/python: ..................... 305 us +- 16 us
Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 319 us +- 5
us -> [/home/sergey/tmp/cpython-venv/bin/python] 305 us +- 16 us: 1.04x faster
(-4%)
python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s
"from io import BytesIO; lines = [b'b'*1000]*10000;"
"BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 1.13 ms
+- 0.02 ms
/home/sergey/tmp/cpython-venv/bin/python: ..................... 988 us +- 88 us
Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 1.13 ms +-
0.02 ms -> [/home/sergey/tmp/cpython-venv/bin/python] 988 us +- 88 us: 1.14x
faster (-12%)
python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s
"from io import BytesIO; lines = [b'b'*10000]*10000;"
"BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: ..................... 38.7 ms
+- 0.1 ms
/home/sergey/tmp/cpython-venv/bin/python: ..................... 38.2 ms +- 0.1
ms
Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 38.7 ms +-
0.1 ms -> [/home/sergey/tmp/cpython-venv/bin/python] 38.2 ms +- 0.1 ms: 1.01x
faster (-1%)
----------
components: Extension Modules
messages: 324011
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: improve performance of BytesIO.writelines() by avoiding creation of
unused PyLongs
type: performance
versions: Python 3.8
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue34488>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
