New submission from Eryk Sun <eryk...@gmail.com>:
In Windows, mmap.resize() unmaps the view of the section [1], closes the
section handle, resizes the file (without error checking), creates a new
section (without checking for ERROR_ALREADY_EXISTS), and maps a new view. This
code has several problems.
Case 1
If it tries to shrink a file that has existing section references, SetEndOfFile
fails with ERROR_USER_MAPPED_FILE. Since the error isn't checked, the code
proceeds to silently map a smaller view without changing the file size, which
contradicts the documented behavior. For example:
>>> f = open('C:/Temp/spam.txt', 'wb+')
>>> f.truncate(8192)
8192
>>> m1 = mmap.mmap(f.fileno(), 0)
>>> m2 = mmap.mmap(f.fileno(), 0)
>>> m1.resize(4096)
>>> os.fstat(f.fileno()).st_size
8192
In this case, it should set dwErrCode = ERROR_USER_MAPPED_FILE; set new_size =
self->size; remap the file; and fail the resize() call. The mmap object should
remain open.
Case 2
Growing the file may succeed, but if it's a named section (i.e. self->tagname
is set) with multiple handle references, then CreateFileMapping just reopens
the existing section by name. Or there could be a race condition with another
section that gets created with the same name in the window between closing the
current section, resizing the file, and creating a new section. Generally
mapping a new view will fail in this case. In particular, if the section isn't
large enough, then the NtMapViewOfSection system call fails with
STATUS_INVALID_VIEW_SIZE, which WinAPI MapViewOfFile translates to
ERROR_ACCESS_DENIED. For example:
>>> f = open('C:/Temp/spam.txt', 'wb+')
>>> f.truncate(8192)
8192
>>> m1 = mmap.mmap(f.fileno(), 0, 'spam')
>>> m2 = mmap.mmap(-1, 8192, 'spam')
>>> m1.resize(16384)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
PermissionError: [WinError 5] Access is denied
>>> m1[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: mmap closed or invalid
>>> os.fstat(f.fileno()).st_size
16384
If CreateFileMapping succeeds with the last error set to ERROR_ALREADY_EXISTS,
then it either opened the previous section or there was a race condition in
which another section was created with the same name. This case should be
handled by closing the section and failing the call. It should not proceed to
MapViewOfFile, which may 'succeed' unreliably.
Additionally, there could be a LBYL check at the outset to avoid having to
close the section in the case of a named section that's opened multiple times.
If self->tagname is set, get the handle count of the section via NtQueryObject:
ObjectBasicInformation [2]. If the handle count is not exactly 1, fail the
resize() call, but don't close the mmap object.
Case 3
resize() is broken for sections that are backed by the system paging file. For
example:
>>> m = mmap.mmap(-1, 4096)
>>> m.resize(8192)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [WinError 87] The parameter is incorrect
>>> m[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: mmap closed or invalid
Since the handle value is INVALID_HANDLE_VALUE in this case, the calls to
resize the 'file' via SetFilePointer and SetEndOfFile fail with
ERROR_INVALID_HANDLE (due to an object type mismatch because handle value -1 is
the current process). There's no error checking, so it proceeds to call
CreateFileMapping with dwMaximumSizeHigh and dwMaximumSizeLow as 0, which fails
immediately as an invalid parameter. Creating a section that's backed by the
system paging file requires a non-zero size.
The view of a section can shrink, but the section itself cannot grow. Generally
resize() in this case would have to create a copy as follows: close the
section; create another section with the new size; map the new view; copy the
contents of the old view to the new view; and unmap the old view. Note that
creating a new named section may open the previous section, or open an
unrelated section if there's a race condition, so ERROR_ALREADY_EXISTS has to
be handled.
[1]: "section" is the native system name of a "file mapping".
[2]:
https://docs.microsoft.com/en-us/windows/win32/api/winternl/nf-winternl-ntqueryobject
----------
components: IO, Library (Lib), Windows
messages: 371050
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: muultiple problems with mmap.resize() in Windows
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue40915>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
