New submission from Antoine Pitrou <[EMAIL PROTECTED]>: In py3k, buffered binary IO can be quadratic when e.g. reading a whole file. This is a small test on 50KB, 100KB and 200KB files:
-> py3k with buffering: ./python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()" 1000 loops, best of 3: 286 usec per loop ./python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()" 1000 loops, best of 3: 1.07 msec per loop ./python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()" 100 loops, best of 3: 4.85 msec per loop -> py3k without buffering (just the raw FileIO layer): ./python -m timeit -s "f = open('50KB', 'rb', buffering=0)" "f.seek(0); f.read()" 10000 loops, best of 3: 46 usec per loop ./python -m timeit -s "f = open('100KB', 'rb', buffering=0)" "f.seek(0); f.read()" 10000 loops, best of 3: 88.7 usec per loop ./python -m timeit -s "f = open('200KB', 'rb', buffering=0)" "f.seek(0); f.read()" 10000 loops, best of 3: 156 usec per loop -> for comparison, Python 2.5: python -m timeit -s "f = open('50KB', 'rb')" "f.seek(0); f.read()" 10000 loops, best of 3: 34.4 usec per loop python -m timeit -s "f = open('100KB', 'rb')" "f.seek(0); f.read()" 10000 loops, best of 3: 62.3 usec per loop python -m timeit -s "f = open('200KB', 'rb')" "f.seek(0); f.read()" 10000 loops, best of 3: 119 usec per loop I'm posting this issue as a reminder, but perhaps someone is already working on this, or the goal is to translate it to C ultimately? ---------- components: Library (Lib) messages: 64788 nosy: pitrou severity: normal status: open title: binary buffered reading is quadratic type: performance versions: Python 3.0 __________________________________ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2523> __________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com