[issue40882] memory leak in multiprocessing.shared_memory.SharedMemory in Windows

[Eryk Sun]

New submission from Eryk Sun :

mmap.mmap in Windows doesn't support an exist_ok parameter and doesn't 
correctly handle the combination fileno=-1, length=0, and tagname with an 
existing file mapping. SharedMemory has to work around these limitations. 

Part of the workaround for the create=False case requires mapping a view via 
MapViewOfFile in order to get the size from VirtualQuerySize, since mmap.mmap 
requires it (needlessly if implemented right) when fileno=-1. This mapped view 
never gets unmapped, which means the shared memory will never be freed until 
the termination of all processes that have opened it with create=False. Also, 
at least in a 32-bit process, this wastes precious address space.

_winapi.UnmapViewOfFile needs to be implemented. Then the temporary view can be 
unmapped as follows:

self._name = name
h_map = _winapi.OpenFileMapping(_winapi.FILE_MAP_READ, False, name)
try:
p_buf = _winapi.MapViewOfFile(h_map, _winapi.FILE_MAP_READ, 0, 0, 0)
finally:
_winapi.CloseHandle(h_map)
try:
size = _winapi.VirtualQuerySize(p_buf)
finally:
_winapi.UnmapViewOfFile(p_buf)
self._mmap = mmap.mmap(-1, size, tagname=name)

[1]: 
https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-unmapviewoffile

--
components: Library (Lib), Windows
messages: 370794
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: memory leak in multiprocessing.shared_memory.SharedMemory in Windows
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40882] memory leak in multiprocessing.shared_memory.SharedMemory in Windows

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 5.0 -> 6.0
pull_requests: +19897
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/20684

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40882] memory leak in multiprocessing.shared_memory.SharedMemory in Windows

[Eryk Sun]

Eryk Sun  added the comment:

Thanks for working on the PR, Zackery. Would you be interested in working on 
improvements to mmap for 3.10? With support in mmap, the Windows-specific 
initialization of SharedMemory could be as simple as the following:

# Windows Named Shared Memory

while True:
tagname = _make_filename() if name is None else name
try:
self._mmap = mmap.mmap(-1, size if create else 0,
tagname, create=create)
break
except FileExistsError:
if name is not None:
raise

self._name = tagname
self._size = len(self._mmap)

The new mmap `create` parameter would default to None, which uses the current 
behavior that allows either opening or creating a file mapping with no sanity 
checking (e.g. a valid fd gets passed in, but it opens an unrelated file 
mapping via tagname). If `create` is true, and there's an existing file mapping 
named `tagname`, then raise FileExistsError. If `create` is false, call 
OpenFileMappingW instead of CreateFileMappingW. In this case, `fileno` must be 
-1, `length` is allowed to be 0, and `tagname` must be a non-empty string. If 
`length` is 0, map the entire file mapping and get the size via VirtualQuery: 
RegionSize.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40882] memory leak in multiprocessing.shared_memory.SharedMemory in Windows

[Eryk Sun]

Change by Eryk Sun :


--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com