Charles-François Natali added the comment: > However, the reason I'm keen on iterdir_stat() is that I'm seeing it speed up > os.walk() by a factor of 10 in my recent tests (note that I've made local > mods, so these results aren't reproducible for others yet). This is doing a > walk on a dir tree with 7800 files and 155 dirs: > > Using fast _betterwalk > Priming the system's cache... > Benchmarking walks on C:\Work\betterwalk\benchtree, repeat 1/3... > Benchmarking walks on C:\Work\betterwalk\benchtree, repeat 2/3... > Benchmarking walks on C:\Work\betterwalk\benchtree, repeat 3/3... > os.walk took 0.178s, BetterWalk took 0.017s -- 10.5x as fast > > Sometimes Windows will go into this "I'm really caching stat results good" > mode -- I don't know what heuristic determines this -- and then I'm seeing a > 40x speed increase. And no, you didn't read that wrong. :-)
I/O benchmarks shouldn't use timeit or repeated calls: after the first run, most of your data is in cache, so subsequent runs are meaningless. I don't know about Windows, but on Linux you should do something like: # echo 3 > /proc/sys/vm/drop_caches to start out clean. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue11406> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
