[issue41053] open() fails to read app exec links

[Eryk Sun]

Eryk Sun  added the comment:

Where the POSIX specification uses the term "symbolic link" [1], it means one 
and only one type of symlink, not multiple types of symlink with divergent 
behavior depending on the context. To be consistent, only one type of Windows 
reparse point [2] is classified as a POSIX symlink, the one that's designed to 
behave like a POSIX symlink in the kernel and API: IO_REPARSE_TAG_SYMLINK. 

It's particularly important that given os.path.islink is true, then os.readlink 
and os.symlink can create an equivalent copy. This is only implemented for 
IO_REPARSE_TAG_SYMLINK.

That said, Windows has a variety of filesystem link types, which it calls 
name-surrogate reparse points. Of this set, in most systems you're only likely 
to encounter IO_REPARSE_TAG_SYMLINK and IO_REPARSE_TAG_MOUNT_POINT. But 
Microsoft has a growing list of name-surrogate types, including: 
IO_REPARSE_TAG_IIS_CACHE, IO_REPARSE_TAG_GLOBAL_REPARSE (NPFS named-pipe 
symlink from server silo into host silo), IO_REPARSE_TAG_WCI_LINK and 
IO_REPARSE_TAG_WCI_TOMBSTONE (Windows container isolation), 
IO_REPARSE_TAG_PROJFS_TOMBSTONE (Projected filesystem tombstone, such as in VFS 
for Git), IO_REPARSE_TAG_LX_SYMLINK (WSL symlink created on a drvfs volume). In 
some cases these are used transparently behind the scenes (e.g. tombstones that 
mark deleted files), or there may be no handler for Windows callers (e.g. WSL 
symlinks are meaningless in Windows).

Note that the latter list does not include IO_REPARSE_TAG_APPEXECLINK. Even 
though "LINK" is in the name, this reparse point type is not any kind of 
filesystem link in practice since it is not handled by the I/O manager or a 
filter driver in the kernel. As discussed in my first message, all of the 
intended behavior of an app-exec link is implemented instead by user-mode API 
functions such as CreateProcessW. In that respect, an app-exec link is more 
like a shell link (i.e. a LNK file), which gets handled by ShellExecuteExW. I 
wouldn't expect the standard library to handle LNK files as symlinks.

---

The vast majority of registered reparse-point types are not link types (e.g. 
cloud files are dehydrated placeholder reparse points). The base Windows API 
has no special handling for the non-link cases. For example, MoveFileExW opens 
a non-link reparse point as a regular file. If it operated on the reparse point 
itself, like it does for a symbolic link, it would ignore the handler and 
potentially break something.

This is why we can't even rename an app-exec link, since there's no handler for 
the reparse tag:

>>> src = os.path.join(os.path.dirname(sys.executable), 'idle3.exe')
>>> dst = os.path.join(os.path.dirname(sys.executable), 'spam3.exe')
>>> try: os.rename(src, dst)
... except OSError as e: print('winerror:', e.winerror)
...
winerror: 1920

---

The standard library is not limited to just IO_REPARSE_TAG_SYMLINK links. It 
supports the broader category of Windows name-surrogate links in certain cases. 
For example, os.lstat doesn't follow them; os.readlink supports symlinks and 
mountpoints [*]; os.unlink operates on symlinks and mountpoints; and 
shutil.rmtree doesn't traverse mountpoints (unlike the POSIX implementation).

Caveat emptor regarding os.readlink, however. There are significant differences 
between symlinks and mountpoints. They're designed to behave like Unix symlinks 
and bind mountpoints, which have similar behavioral differences. Also, 
mountpoints are always evaluated on the server, so a remote mountpoint *must* 
be treated as opaque data [1]:

The following reparse tags, with the exception of
IO_REPARSE_TAG_SYMLINK, are processed on the server 
and are not processed by a client after transmission 
over the wire. Clients SHOULD treat associated 
reparse data as opaque data. 

---

[1] 
https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap03.html#tag_21_03_00_75

[2] 
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41053] open() fails to read app exec links

[Kagami Sascha Rosylight]

Kagami Sascha Rosylight  added the comment:

It seems libuv and pwsh decided to detect and read them just as symlinks:

https://github.com/libuv/libuv/pull/2812
https://github.com/PowerShell/PowerShell/pull/10331

Could Python do the same?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41053] open() fails to read app exec links

[Eryk Sun]

Eryk Sun  added the comment:

By design, appexec links (i.e. app execution aliases) cannot be followed 
automatically. There is no handler for them in the kernel. WinAPI CreateFileW 
fails with ERROR_CANT_ACCESS_FILE (1920), and the underlying NT status value is 
STATUS_IO_REPARSE_TAG_NOT_HANDLED (0xC279). 

Since 3.8, os.stat handles ERROR_CANT_ACCESS_FILE in all cases by trying to 
return the result for the reparse point instead. This at least allows getting 
the st_file_attributes and st_reparse_tag values. For example:

>>> s = os.stat(sys.executable)
>>> s.st_file_attributes & stat.FILE_ATTRIBUTE_REPARSE_POINT
1024
>>> s.st_reparse_tag == stat.IO_REPARSE_TAG_APPEXECLINK
True 

CreateProcessW follows app-exec links manually by reading the reparse point. 
But it's not that simple. The link target under "%ProgramFiles%\WindowsApps" 
isn't unconditionally executable by standard users. In other words, unless a 
particular condition is met, trying to execute the target file fails with 
access denied. Execute access depends on a conditional access-control entry 
(conditional ACEs are supported in the kernel since Windows 8) that grants 
access if the user's access token contains a "WIN://SYSAPPID" attribute that 
identifies the package. Here's the SDDL definition of this ACE for the app 
distribution of Python 3.9:

(XA;ID;0x1200a9;;;BU;(WIN://SYSAPPID Contains 
"PYTHONSOFTWAREFOUNDATION.PYTHON.3.9_QBZ5N2KFRA8P0")

"XA" is an access-allowed callback (conditional) ACE
"ID" means the ACE is inherited from the parent directory
"BU" is the security principal BUILTIN\Users (local group)
Access Mask 0x1200a9:
FILE_GENERIC_READ | FILE_GENERIC_EXECUTE:
SYNCHRONIZE
READ_CONTROL
FILE_READ_ATTRIBUTES
FILE_EXECUTE
FILE_READ_EA
FILE_READ_DATA

If the app is installed for the user, CreateProcessW handles the access denied 
result by creating and impersonating a custom access token to execute the app, 
which includes the required WIN://SYSAPPID security attribute. You can attach a 
debugger to see the security attributes added to the app token:

0:003> !token

[...]

Security Attributes Information:
 00 Attribute Name: WIN://SYSAPPID
Value Type  : TOKEN_SECURITY_ATTRIBUTE_TYPE_STRING
Value[0]: 
PythonSoftwareFoundation.Python.3.9_3.9.179.0_x64__qbz5n2kfra8p0
Value[1]: Python
Value[2]: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0

[...]

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41053] open() fails to read app exec links

[Kagami Sascha Rosylight]

New submission from Kagami Sascha Rosylight :

After installing Python from Microsoft Store, this fails:

```
>>> open('C:\\Users\\Kagami\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\\python.exe')
Traceback (most recent call last):
  File "", line 1, in 
OSError: [Errno 22] Invalid argument: 
'C:\\Users\\Kagami\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\\python.exe'
```

This causes virtualenv to fail on it:

```
INFO: Traceback (most recent call last):
INFO:   File 
"C:/Users/Kagami/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs\third_party\python\virtualenv\virtualenv.py",
 line 2349, in 
INFO: main()
INFO:   File 
"C:/Users/Kagami/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs\third_party\python\virtualenv\virtualenv.py",
 line 703, in main
INFO: create_environment(home_dir,
INFO:   File 
"C:/Users/Kagami/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs\third_party\python\virtualenv\virtualenv.py",
 line 925, in create_environment
INFO: py_executable = os.path.abspath(install_python(
INFO:   File 
"C:/Users/Kagami/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs\third_party\python\virtualenv\virtualenv.py",
 line 1239, in install_python
INFO: shutil.copyfile(executable, py_executable)
INFO:   File "C:\Program 
Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.1008.0_x64__qbz5n2kfra8p0\lib\shutil.py",
 line 261, in copyfile
INFO: with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
INFO: OSError: [Errno 22] Invalid argument: 
'C:\\Users\\Kagami\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\\python.exe'
```

--
components: Windows
messages: 371939
nosy: paul.moore, saschanaz, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: open() fails to read app exec links
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com