[issue28353] os.fwalk() unhandled exception when error occurs accessing symbolic link target
Samson Lee added the comment: This is definitely a bug because there is no way an unhandled exception like this is acceptable behaviour. The behaviour is that the fwalk iteration will stop and there is no way to recover / continue. Also fwalk takes an onerror callback function that won't be called with this error. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28353] os.fwalk() unhandled exception when error occurs accessing symbolic link target
Samson Lee added the comment: Sorry, the target file needs to be in a directory you don't have permission to access: $ mkdir handsoff $ sudo chown root:root handsoff $ sudo chmod 700 handsoff $ ln -s handsoff/anything blah -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28353] os.fwalk() unhandled exception when error occurs accessing symbolic link target
Samson Lee added the comment: Similar to http://bugs.python.org/issue25860 -- nosy: +serhiy.storchaka ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28353] os.fwalk() unhandled exception when error occurs accessing symbolic link target
New submission from Samson Lee: The bug is os.fwalk() crashes with unhandled exception when there is an error accessing symbolic link targets. To reproduce the bug, create a symbolic link that targets a file that you do not have permission to access: $ touch handsoff $ sudo chown root:root handsoff $ sudo chmod 700 handsoff $ ln -s handsoff blah Now, os.fwalk() fails: >>> for root, dirs, files, fd in os.fwalk('.'): ... print(root, dirs, files) ... Traceback (most recent call last): File "test_fwalk_permission_error.py", line 3, in for root, dirs, files, fd in os.fwalk('.'): File "/usr/lib64/python3.5/os.py", line 520, in fwalk yield from _fwalk(topfd, top, topdown, onerror, follow_symlinks) File "/usr/lib64/python3.5/os.py", line 537, in _fwalk if st.S_ISDIR(stat(name, dir_fd=topfd).st_mode): PermissionError: [Errno 13] Permission denied: 'blah' The cause of the problem is in this part of os.py: for name in names: try: # Here, we don't use AT_SYMLINK_NOFOLLOW to be consistent with # walk() which reports symlinks to directories as directories. # We do however check for symlinks before recursing into # a subdirectory. if st.S_ISDIR(stat(name, dir_fd=topfd).st_mode): dirs.append(name) else: nondirs.append(name) except FileNotFoundError: try: # Add dangling symlinks, ignore disappeared files if st.S_ISLNK(stat(name, dir_fd=topfd, follow_symlinks=False) .st_mode): nondirs.append(name) except FileNotFoundError: continue To fix it, simply replace FileNotFoundError with more general OSError. Cheers -- components: Library (Lib) messages: 278016 nosy: Samson Lee priority: normal severity: normal status: open title: os.fwalk() unhandled exception when error occurs accessing symbolic link target versions: Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25860] os.fwalk() silently skips remaining directories when error occurs
Changes by Samson Lee <sammo2...@gmail.com>: -- nosy: +hynek, neologix ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25860> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25860] os.fwalk() silently skips remaining directories when error occurs
Changes by Samson Lee <sammo2...@gmail.com>: -- nosy: +benhoyt, haypo ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25860> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25860] os.fwalk() silently skips remaining directories when error occurs
New submission from Samson Lee: I noticed os.fwalk() produced different results as os.walk(). It turned out that os.fwalk() skips all remaining directories when OSError occurs (e.g. due to PermissionError). To reproduce the bug, first create a test directory structure: $ mkdir 1; touch 1/a $ mkdir 2; touch 2/b $ mkdir 3; touch 3/c $ mkdir 4; touch 4/c At this stage, everything is okay, both os.fwalk() os.walk() produce the same results: >>> import os >>> for root, dirs, files in os.walk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] ./2 [] ['b'] ./3 [] ['c'] ./4 [] ['d'] >>> for root, dirs, files, fd in os.fwalk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] ./2 [] ['b'] ./3 [] ['c'] ./4 [] ['d'] To introduce an error, force a PermissionError on one of the directories: $ sudo chown root:root 2 $ sudo chmod 700 2 Now, the os.fwalk() results are different (trust me, os.walk() is still okay): >>> for root, dirs, files, fd in os.fwalk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] So it seems that os.fwalk skips remaining directories after an error occurs. The cause of the problem is in this part of os.py: for name in dirs: try: orig_st = stat(name, dir_fd=topfd, follow_symlinks=follow_symlinks) dirfd = open(name, O_RDONLY, dir_fd=topfd) except OSError as err: if onerror is not None: onerror(err) break To fix it, simply replace break with continue. Patch attached. Cheers -- components: Library (Lib) files: fwalk-silently-skips-dirs-when-error-occurs.patch keywords: patch messages: 256377 nosy: Samson Lee priority: normal severity: normal status: open title: os.fwalk() silently skips remaining directories when error occurs type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file41304/fwalk-silently-skips-dirs-when-error-occurs.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25860> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com