On 2/16/21, rhyslloyd1 via python-win32 <python-win32@python.org> wrote: > > I am trying to open a memory mapped file using Python. I originally used the > "mmap" module from Python however I had issues with it because I had to use > a fixed size for the file even though my goal was to open an existing file > of which the size would be unknown.
I'm sorry to say that the only reliable solution is to directly call WinAPI OpenFileMappingW() and MapViewOfFile() using a foreign-function interface (FFI) package such as ctypes or CFFI. I can write up an example if you need help. In theory, shared_memory.SharedMemory() in the multiprocessing package could support this case. But in practice it's still based on the mmap module. It tries to improve the situation by opening and mapping an existing mapping via WinAPI OpenFileMappingW() [1] and MapViewOfFile() [2] in order to query the region size via VirtualQuery(), but this isn't reliable because it returns the size rounded up to the nearest page boundary. Calling mmap.mmap(-1, size, tagname=name) with this queried size may fail the internal MapViewOfFile() call with ERROR_ACCESS_DENIED (5) if the size is bigger than the maximum size of the file mapping. (The underlying status from the NT system call more clearly describes the failure as STATUS_INVALID_VIEW_SIZE.) The problem with the standard library mmap module and PyWin32's mmapfile is that they're designed to open a mapping only via CreateFileMappingW() [3]. When this function is called with hFile as INVALID_HANDLE_VALUE (i.e. use the system paging file) and the maximum size as 0 (i.e. dwMaximumSizeHigh=0, dwMaximumSizeLow=0), it always fails with ERROR_INVALID_PARAMETER (87). It does not check whether lpName is the name of an existing mapping before validating the parameters and failing. Hypothetically, it could have been written to check for this case, since the file handle is ignored anyway for an existing mapping. But it is what it is, and has been so for about 30 years. A high-level library function needs to be aware of this limitation and handle it appropriately by implementing an open disposition (e.g. create-new, open-always, open-existing) that can dictate the use of OpenFileMappingW() in addition to or instead of CreateFileMapping(). --- [1] https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-openfilemappinga [2] https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-mapviewoffile [3] https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-createfilemappingw _______________________________________________ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32