[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Eryk Sun]

Eryk Sun  added the comment:

> oh, I missed that a notification happens to the other process(es) in a 
> common case, a bit of retrying with backoff would actually make sense 

The other common problem with deleting an empty directory is when it's opened 
as the working directory of a process. This case fails as a sharing violation 
because the open doesn't share delete access. There's nothing reasonable to do 
about it without user interaction, which would be a complicated bit of code: 
find the process that has the directory open, display a message to the user in 
the desktop session of the process, and wait for a response. That's not a good 
candidate for the standard library.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Gregory P. Smith]

Gregory P. Smith  added the comment:

oh, I missed that a notification happens to the other process(es) in a common 
case, a bit of retrying with backoff would actually make sense there.  But I 
wouldn't let a retry run for longer than a second or three, as code tends to 
assume that rmtree is pretty quick and if something is going to close 
asynchronously, it should close quickly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Eryk Sun]

Eryk Sun  added the comment:

> Isn't this just "how windows behaves" on some filesystems with 
> little that we can do about it?

The suggestion was to include an error handler that retries unlink() and 
rmdir() -- particularly rmdir() -- a given number of times, probably with an 
exponential back off, before giving up and failing. This accounts for race 
conditions in which the delete succeeds but the file/directory can't be 
unlinked because it's currently open (with delete sharing). A lot of these 
cases are similar to ReadDirectoryChangesExW(), in which the owner of the open 
is immediately notified that the directory was deleted. If they're well 
behaved, like Explorer, they immediately close their handle to allow the 
directory to be unlinked by the system. But that may not be soon enough for the 
process that deleted the directory. The suggested retry loop would help to work 
around this race condition. A couple people were in favor of this being 
provided by the standard library, so everyone isn't forced to implement there 
own workaround for a common problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Isn't this just "how windows behaves" on some filesystems with little that we 
can do about it?

The only real action item I can see here is that if it is _reasonable_ for us 
to detect the situation and improve the error message, that'd help the users 
(and reduce bugreports like this).

Otherwise I suggest closing it as wont fix / working as the platform intends.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Eryk Sun]

Change by Eryk Sun :


--
components:  -IO

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Eryk Sun]

Eryk Sun  added the comment:

> Explorer keeps handles open to directories to get updates via
> ReadDirectoryChangesExW. It opens watched directories with 
> shared delete access, so deleting the child succeeds. But as 
> discussed above, the directory isn't unlinked from the parent
> until Explorer closes its handle.

In Windows 10, NTFS allows deleting an empty directory that's currently opened 
with shared-delete access, so this race condition will be increasingly less 
common. For example:

access = GENERIC_READ
sharing = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE
disposition = OPEN_EXISTING
flags = FILE_FLAG_BACKUP_SEMANTICS

os.mkdir('spam')
h = CreateFile('spam', access, sharing, None, disposition, flags, None)
os.rmdir('spam')

>>> print(GetFinalPathNameByHandle(h, 0))
\\?\C:\$Extend\$Deleted\004E632F70819337

FAT filesystems do not support this capability, and may never support it, so 
the problem hasn't gone away completely.

--
versions: +Python 3.10, Python 3.9 -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Ofek Lev]

Ofek Lev  added the comment:

> For convenience, a handler that retries unlink() and rmdir() could be 
> distributed with shutil. For ease of use, it could be enabled by default on 
> Windows.

Any update on that? I just spent a bunch of time debugging this on Windows.

--
nosy: +Ofekmeister

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33240] shutil.rmtree fails if inner folder is open in Windows Explorer

[Terry J. Reedy]

Change by Terry J. Reedy :


--
title: shutil.rmtree fails when the inner floder is opened in Explorer on 
Windows -> shutil.rmtree fails if inner folder is open in Windows Explorer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com