[issue22084] Mutating while iterating
New submission from Aaron Brady: Hi, I asked about the inconsistency of the "RuntimeError" being raised when mutating a container while iterating over it here [1], "set and dict iteration" on Aug 16, 2012. [1] http://www.gossamer-threads.com/lists/python/python/1004659 I posted a patch on the ML but never submitted it. People's reaction seemed ambivalent. Now I have an idea for a different implementation. I'd like to take another shot at it. It's one of the worst silent errors, since there's an error in the *iterator* when we call a *set* method. We're going to add something to make it safer, at least in the sense of getting a clear failure, if the programmer does something that's always been ill-advised. We have a number of options for the implementation. We still have the option to introduce "IterationError", possibly a subclass of "RuntimeError". These options are still applicable: 1) Collection of iterators . Invalidate all "open" iterators on mutating . a) Linked list .. i) Uncounted references .. ii) Counted references .. iii) Weak references . b) Weak set 2) Version index / timestamp / "memo" . Iterators check whether the container has been mutated since they were created . a) No overflow - Python longs .. i) Reset index if no iterators left . b) Overflow - C ints / shorts (silent error) 3) Iterator count . Raise exception on mutation, not iteration The new option is: 2d) Use a dedicated empty *object* for a timestamp or "memo". A new memo is created on every mutation. Before advancing, the iterator checks whether the current memo is a different object than it was when it was created. Costs: The existing silent error is fairly rare. The container gains a pointer to its current memo. The iterator loses the cached length but gains a pointer to a memo. The memos are blank objects: a "Py ssize t" and a pointer with certain flags at time of writing. Speed is the same: comparing the lengths is replaced with comparing the memos. Some caveats: The memory manager is used to obtain perpetually unique IDs. A unique algorithm could be used instead of the memory manager, though the memo needs to contain a reference count more or less regardless. There can at most be one memo per iterator. The approach is outlined in pseudocode here [2]. Implementation could be optimized slightly by only creating new memos if iterators have been opened, shown here [3]. [2] http://home.comcast.net/~castironpi-misc/irc-0168%20mutating%20while%20iterating%20markup.html [3] http://home.comcast.net/~castironpi-misc/irc-0168%20mutating%20while%20iterating%202%20markup.html -- components: Library (Lib) messages: 224071 nosy: castironpi priority: normal severity: normal status: open title: Mutating while iterating type: behavior versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker <http://bugs.python.org/issue22084> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4708] os.pipe should return inheritable descriptors (Windows)
Aaron Brady added the comment: This is currently accomplished in 'multiprocessing.forking' with a 'duplicate' function. Use (line #213): rfd, wfd = os.pipe() # get handle for read end of the pipe and make it inheritable rhandle = duplicate(msvcrt.get_osfhandle(rfd), inheritable=True) Definition (line #192). Should it be included in the public interface and documented, or perhaps a public entry point to it made? ___ Python tracker <http://bugs.python.org/issue4708> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4708] os.pipe should return inheritable descriptors (Windows)
New submission from Aaron Brady :
os.pipe should return inheritable descriptors on Windows.
Patch below, test attached. New pipe() returns descriptors, which
cannot be inherited. However, their permissions are set correctly, so
msvcrt.get_osfhandle and msvcrt.open_osfhandle can be used to obtain an
inheritable handle.
Docs should contain a note to the effect. 'On Windows, use
msvcrt.get_osfhandle to obtain a handle to the descriptor which can be
inherited. In a subprocess, use msvcrt.open_osfhandle to obtain a new
corresponding descriptor.'
--- posixmodule_orig.c 2008-12-20 20:01:38.296875000 -0600
+++ posixmodule_new.c 2008-12-20 20:01:54.359375000 -0600
@@ -6481,8 +6481,12 @@
HANDLE read, write;
int read_fd, write_fd;
BOOL ok;
+ SECURITY_ATTRIBUTES sAttribs;
Py_BEGIN_ALLOW_THREADS
- ok = CreatePipe(&read, &write, NULL, 0);
+ sAttribs.nLength = sizeof( sAttribs );
+ sAttribs.lpSecurityDescriptor = NULL;
+ sAttribs.bInheritHandle = TRUE;
+ ok = CreatePipe(&read, &write, &sAttribs, 0);
Py_END_ALLOW_THREADS
if (!ok)
return win32_error("CreatePipe", NULL);
--
components: Library (Lib), Windows
files: os_pipe_test.py
messages: 78136
nosy: castironpi
severity: normal
status: open
title: os.pipe should return inheritable descriptors (Windows)
type: behavior
versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1
Added file: http://bugs.python.org/file12408/os_pipe_test.py
___
Python tracker
<http://bugs.python.org/issue4708>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4376] Nested ctypes 'BigEndianStructure' fails
New submission from Aaron Brady <[EMAIL PROTECTED]>: Nested 'BigEndianStructure' fails in 2.5 and 2.6.: TypeError: This type does not support other endian Example and traceback in attached file. -- assignee: theller components: ctypes files: ng36.py messages: 76171 nosy: castironpi, theller severity: normal status: open title: Nested ctypes 'BigEndianStructure' fails type: compile error versions: Python 2.5, Python 2.6 Added file: http://bugs.python.org/file12091/ng36.py ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4376> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4092] inspect.getargvalues return type not ArgInfo
New submission from Aaron Brady <[EMAIL PROTECTED]>: Python 2.6 (r26:66721, Oct 2 2008, 11:35:03) [MSC v.1500 32 bit (Intel)] on win 32 Type "help", "copyright", "credits" or "license" for more information. >>> import inspect >>> type( inspect.getargvalues( inspect.currentframe() ) ) Docs say: inspect.getargvalues(frame) ... Changed in version 2.6: Returns a named tuple ArgInfo(args, varargs, keywords, locals). The code defines an ArgInfo type, but doesn't instantiate it in the return, as shown here: return args, varargs, varkw, frame.f_locals -- components: Library (Lib) messages: 74595 nosy: castironpi severity: normal status: open title: inspect.getargvalues return type not ArgInfo type: behavior versions: Python 2.6 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4092> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
