[issue15200] Faster os.walk

Serhiy Storchaka added the comment: Timing of walk depends on how deep we dive into the directories. $ ./python -m timeit -s "from os import walk" "for x in walk('/home/serhiy/py/1/2/3/4/5/6/7/8/9/cpython/'): pass" 10 loops, best of 3: 398 msec per loop $ ./python -m timeit -s "from os import

[issue15200] Faster os.walk

Antoine Pitrou added the comment: > This looks like the kind of optimization that depends hugely on what > kernel you're using. Agreed. Also, I'm worried that there might be subtle differences between walk() and fwalk() which could come and bite users if we silently redirect the former to the

[issue15200] Faster os.walk

Ross Lagerwall added the comment: This looks like the kind of optimization that depends hugely on what kernel you're using. Maybe on FreeBSD/Solaris/whatever, standard os.walk() is faster? If this micro-optimization were to be accepted, someone would have to be keen enough to test it is diffe

[issue15200] Faster os.walk

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue15200] Faster os.walk

Larry Hastings added the comment: > It doesn't have to. > Right now, it uses O(depth of the directory tree) FDs. > It can be changed to only require O(1) FDs But closing and reopening those file descriptors seems like it might slow it down; would it still be a performance win? Also, I'm not

[issue15200] Faster os.walk

Charles-François Natali added the comment: > On the other hand, fwalk also uses a lot of file descriptors. Users > with processes which were already borderline on max file descriptors > might not appreciate upgrading to find their os.walk calls suddenly > failing. It doesn't have to. Right

[issue15200] Faster os.walk

Larry Hastings added the comment: It's amusing that using fwalk and throwing away the last argument is faster than a handwritten implementation. On the other hand, fwalk also uses a lot of file descriptors. Users with processes which were already borderline on max file descriptors might not

[issue15200] Faster os.walk

Changes by Serhiy Storchaka : -- nosy: +larry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue15200] Faster os.walk

New submission from Serhiy Storchaka : Using os.fwalk (if it is available) we can make os.walk more fast. Microbenchmark: ./python -m timeit -s "from os import walk" "for x in walk('Lib'): pass" Results: Vanilla: 112 msec Patched: 90.5 msec -- components: Library (Lib) files: faster_w