Eryk Sun <eryk...@gmail.com> added the comment:
> The persistent mode sounds just like Python shared memory also works > on Linux (where I can have these /dev/shm/* files even after the > Python process ends) but I think on Windows, Python is not using > the persistent mode and thus the shared memory goes away, in contrast > to how it works on Linux. Unix has the notion that everything is a file as a central organizing concept. As such, Linux opts to implement POSIX shm_open() [1] with a tmpfs [2] filesystem that's mounted on /dev/shm. tmpfs filesystems exist in virtual memory. They're not persistent in the sense of being created on a physical disk that provides persistent storage. Windows has the notion that everything is an object as a central organizing concept. Internally, the system has a "\" root object directory, which contains other object directories and object symbolic links (unrelated to filesystem symlinks). Each object directory, including the root directory, contains named kernel objects. Named device objects are normally created in r"\Device", such as r"\Device\HarddiskVolume2", and global symbolic links to devices are created in r"\GLOBAL??", such as r"\GLOBAL??\C:" -> r"\Device\HarddiskVolume2" for the "C:" drive. The root registry key object is r"\REGISTRY", which contains dynamic key objects such as r"\REGISTRY\MACHINE" (referenced via the pseudohandle HKEY_LOCAL_MACHINE) and "\REGISTRY\USER" (referenced via the pseudohandle HKEY_USERS), which in turn contain other keys such as "\REGISTRY\MACHINE\SOFTWARE" on which registry hives are mounted. For naming global kernel objects and session 0 objects, the Windows API internally uses the directory r"\BaseNamedObjects". For interactive Windows sessions, it uses r"\Sessions\<session number>\BaseNamedObjects" and, for app containers, subdirectories of r"\Sessions\<session number>\AppContainerNamedObjects". To explicitly name an object in the global directory, use the path r"Global\<object name>". The "Global" prefix is implemented as an object symbolic link to r"\BaseNamedObjects". Of course, that's an internal implementation detail; the API just refers to the r"Global\\" prefix. Naming a kernel object in r"\BaseNamedObjects" is nearly equivalent to creating a file in a tmpfs filesystem in Linux, with one major difference. Objects are reference counted, with a handle reference count and a pointer reference count. Opening a handle increments both counters, but kernel code can use just a pointer reference instead of opening a handle. By default, objects are temporary. As such, when their pointer reference count is decremented to 0, they get unlinked from the object namespace, if they're named objects, and deallocated. It's possible to create a permanent object in the object namespace, either initially via the OBJ_PERMANENT attribute [3], or later on via NtMakePermanentObject(handle). However, making an object permanent is considered a super privilege in Windows, so privileged in fact that the Windows API doesn't support this capability, at least not as far as I know. By default, SeCreatePermanentPrivilege is only granted to the SYSTEM account. Also, creating a named section object (i.e. file mapping object) in a session-restricted directory such as r"\BaseNamedObjects" requires SeCreateGlobalPrivilege, which is only granted by default to administrators and system service accounts. Unix and Linux are less concerned about creating 'permanent' files and global shared memory in virtual filesystems such as tmpfs. The lifetime while the system is running is left up to the creator, which has to manually remove the file, such as via shm_unlink() [4]. Stale files could accumulate in "/dev/shm" when processes crash, which is why a separate resource tracker is required, implementing what the Windows kernel provides by default. --- [1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html [2] https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt [3] https://docs.microsoft.com/en-us/windows/win32/api/ntdef/ns-ntdef-_object_attributes [4] https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_unlink.html ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue46888> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
