Akira Li added the comment: To see what happens at syscall level, I've run various implementations of get_tree_size() functions (see get_tree_size_listdir.diff) with strace:
get_tree_size_listdir_fd -- os.listdir(fd) + os.lstat get_tree_size -- os.scandir(path) + entry.lstat get_tree_size_listdir_stat -- os.listdir(path) + os.lstat get_tree_size_listdir -- os.listdir(path) + os.path.isdir + os.lstat Summary: - os.listdir() and os.scandir()-based variants use the same number of getdents() syscalls - and the number of openat, lstat (newfstatat) syscalls is also comparable (except for the variant that uses os.path.isdir) Log: scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_fd as f; import os; print(f(os.open("/usr/", os.O_RDONLY)))' && head -7 py.strace 5535240217 11.29user 8.14system 0:17.78elapsed 109%CPU (0avgtext+0avgdata 13460maxresident)k 0inputs+8outputs (0major+6781minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 46.51 0.075252 0 264839 newfstatat 16.88 0.027315 1 50460 getdents 11.53 0.018660 0 75621 fcntl 9.74 0.015758 0 50531 close 6.87 0.011116 0 25214 openat scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size as f; print(f("/usr/"))' && head -7 py.strace 5535240217 22.56user 8.47system 0:29.77elapsed 104%CPU (0avgtext+0avgdata 13280maxresident)k 0inputs+8outputs (0major+6306minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 62.00 0.032822 0 239644 lstat 19.97 0.010570 0 50460 getdents 8.52 0.004510 0 25215 openat 6.09 0.003224 0 25326 close 0.55 0.000292 3 95 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_stat as f; print(f("/usr/"))' && head -7 py.strace 5535240217 13.70user 6.30system 0:18.84elapsed 106%CPU (0avgtext+0avgdata 13456maxresident)k 0inputs+8outputs (0major+6769minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 64.79 0.050217 0 264849 lstat 19.37 0.015011 0 50460 getdents 8.22 0.006367 0 25215 openat 5.76 0.004465 0 25326 close 0.32 0.000247 2 114 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir as f; print(f("/usr/"))' && head -7 py.strace 5535240217 19.53user 10.61system 0:28.16elapsed 107%CPU (0avgtext+0avgdata 13452maxresident)k 0inputs+8outputs (0major+6733minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 42.04 0.063050 0 265195 37259 stat 37.40 0.056086 0 265381 lstat 11.46 0.017187 0 50460 getdents 4.82 0.007232 0 25215 openat 3.43 0.005139 0 25326 close ---------- Added file: http://bugs.python.org/file37292/get_tree_size_listdir.diff _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue22524> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com