[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> not a bug
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Eryk Sun]

Eryk Sun added the comment:

Yes, I think this issue should be closed. But for the record I'd like to note a 
not uncommon case in which listdir() raise FileNotFoundError on Windows.

According to MS-FSA [1], if a request to open a directory resolves to a file, 
the operation should fail with STATUS_NOT_A_DIRECTORY (see 2.1.5.1, phase 7). 
That's the scenario I was discussing in previous messages. However, I neglected 
to discuss what happens when an intermediate path component is not a directory. 
In theory this case should also fail with STATUS_NOT_A_DIRECTORY (see 2.1.5.1, 
phase 6). However, in practice MS file systems instead return 
STATUS_OBJECT_PATH_NOT_FOUND, which becomes ERROR_PATH_NOT_FOUND, for which 
Python raises FileNotFoundError.

Walking the path to find the reason for the failure shouldn't be attempted 
because it's subject to race conditions -- e.g. the file that caused the 
failure may no longer exist. 

[1]: https://msdn.microsoft.com/en-us/library/ff469536.aspx

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Terry J. Reedy]

Terry J. Reedy added the comment:

Eryk, does your last comment suggest that we close this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Eryk Sun]

Eryk Sun added the comment:

Python has no specific use for ERROR_PATH_NOT_FOUND (3) and just maps it to 
FileNotFoundError -- like the CRT maps it to ENOENT. Even if we wanted 
listdir() to specially handle this error, given that it's implemented via 
FindFirstFile, there's nothing that can be done after the fact that's not 
vulnerable to race conditions. 

Also, to reiterate this point, getting ERROR_PATH_NOT_FOUND instead of 
ERROR_DIRECTORY (267) when the target is a file seems to be pretty uncommon, 
much more so than I at first thought. I only see it with VirtualBox shared 
folders. I don't see it with Microsoft file systems such as NTFS, FAT32, or 
CDFS.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Terry J. Reedy]

Terry J. Reedy added the comment:

This is a feature-change 'enhancement' issue.  There is no violation of the doc 
at https://docs.python.org/3/library/os.html#os.listdir and as Raylu noted, the 
current behavior is accommodated in tests.

I presume that regardless of what system calls are made, FileNotFoundError and 
anything else could be caught and NotADirectoryError raised instead, with the 
message containing the Windows error message.  This would only break 
Windows-specific code that only catches FileNotFoundError and not 
NotADirectoryError.

I am not sure what our general policy is and whether this should be fixed at 
this point.  I personally like Python smoothing over OS differences when easily 
possible.

--
nosy: +terry.reedy
stage:  -> test needed
type:  -> enhancement
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Eryk Sun]

Eryk Sun added the comment:

> FindFirstFile naturally fails with ERROR_PATH_NOT_FOUND

Getting this error actually depends on the file system. I don't see it with 
NTFS, which returns STATUS_NOT_A_DIRECTORY, which gets translated to 
ERROR_DIRECTORY. On the other hand, VboxSF (VirtualBox shared folder) returns 
STATUS_OBJECT_PATH_NOT_FOUND, which gets translated to ERROR_PATH_NOT_FOUND.

Also, there are many possible errors when trying to list a path on a device 
that isn't managed by a file system (e.g. \\.\PhysicalDrive0). The device can 
return a status value that FindFirstFile doesn't special case, such as 
STATUS_UNSUCCESSFUL, STATUS_NOT_FOUND, or STATUS_OBJECT_NAME_INVALID. The 
NtOpenFile call may even succeed, but then the NtQueryDirectoryFile will 
probably fail with STATUS_INVALID_DEVICE_REQUEST.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Eryk Sun]

Eryk Sun added the comment:

Python appends "\*.*" to the path, so in the case in which the leaf element is 
a file, FindFirstFile naturally fails with ERROR_PATH_NOT_FOUND.

Python 3.5+ could maybe switch to calling CreateFile to open a handle and 
GetFileInformationByHandleEx to get the FILE_ID_BOTH_DIR_INFO (added in Vista, 
so this couldn't be backported to 2.7). It would first have to get the 
FILE_BASIC_INFO for the file attributes to ensure that the target is a 
directory. This could also support listing directories by file descriptor on 
Windows, since the CRT has O_OBTAIN_DIR (0x2000) for opening a directory.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Blame Windows for this behavior. os.listdir() is just a wrapper around few 
system calls.

It is expected that functions in the os module can raise different exceptions 
on different platforms.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[raylu]

raylu added the comment:

os.listdir should always return NotADirectoryError when run on a regular file. 
Users shouldn't have to special-case win32/FileNotFoundError.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[R. David Murray]

R. David Murray added the comment:

Could you please clarify what bug you are reporting?

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[raylu]

New submission from raylu:

According to
https://github.com/python/cpython/blob/4ed71dae9d23a412f9f73d3a0b1f4be37543e49e/Lib/test/test_unicode_file_functions.py#L106
listdir can sometimes return FileNotFoundError on Windows.

Tangentially related: https://hg.python.org/cpython/rev/385c2ec78f16

--
components: Library (Lib), Windows
messages: 286225
nosy: paul.moore, raylu, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.listdir has inconsistent behavior when run on a non-directory

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com