Ben Hoyt added the comment:

Yeah, I very much agree with what Nick says -- we really want a way to expose 
what the platform provides. It's  less important (though still the ideal), to 
expose that in a platform-independent way. Today the only way to get access to 
opendir/readdir on Linux and FindFirst/Next on Windows is by using a bunch of 
messy (and slowish) ctypes code. And yes, os.walk() would be the main 
cross-platform abstraction built on top of this.

Charles gave this example of code that would fall over:

size = 0
for name, st in scandir(path):
    if stat.S_ISREG(st.st_mode):
        size += st.st_size

I don't see it, though. In this case you need both .st_mode and .st_size, so a 
caller would check that those are not None, like so:

size = 0
for name, st in scandir(path):
    if st.st_mode is None or st.st_size is None:
        st = os.stat(os.path.join(path, name))
    if stat.S_ISREG(st.st_mode):
        size += st.st_size

One line of extra code for the caller, but a big performance gain in most cases.

Stinner said, "But as usual, a benchmark on a real platform would be more 
convicing". Here's a start: https://github.com/benhoyt/betterwalk#benchmarks -- 
but it's not nearly as good as it gets yet, because those figures are still 
using the ctypes version. I've got a C version that's half-finished, and on 
Windows it makes os.walk() literally 10x the speed of the default version. Not 
sure about Linux/opendir/readdir yet, but I intend to do that too.

----------

_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue11406>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to