Re: [Python-Dev] Release or not release the GIL
On Fri, 1 Feb 2013 15:25:27 -0800 "Gregory P. Smith" wrote: > On Fri, Feb 1, 2013 at 7:22 AM, Antoine Pitrou wrote: > > > Le Fri, 1 Feb 2013 15:18:39 +0100, > > "Amaury Forgeot d'Arc" a écrit : > > > 2013/2/1 Charles-François Natali > > > > > > > >> dup2(oldfd, newfd) closes oldfd. > > > > > > > > > > No, it doesn't close oldfd. > > > > > > > > > > It may close newfd if it was already open. > > > > > > > > (I guess that's what he meant). > > > > > > > > Anyway, only dup2() should probably release the GIL. > > > > > > > > One reasonable heuristic is to check the man page: if the syscall > > > > can return EINTR, then the GIL should be released. > > > > > > > > > Should the call be retried in the EINTR case? > > > (After a PyErr_CheckSignals) > > > > I don't think we want to retry low-level system calls (but I'm not sure > > we're currently consistent in that regard). > > > > I think this is what you meant but to be clear: Anywhere we're using them > within a library for a good purpose, we do need to retry. If we're merely > exposing them via the os module such as os.dup, its up to the caller to > deal with the retry. Indeed, that's what I meant. Sorry for not being very clear. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
On Fri, Feb 1, 2013 at 7:22 AM, Antoine Pitrou wrote: > Le Fri, 1 Feb 2013 15:18:39 +0100, > "Amaury Forgeot d'Arc" a écrit : > > 2013/2/1 Charles-François Natali > > > > > >> dup2(oldfd, newfd) closes oldfd. > > > > > > > > No, it doesn't close oldfd. > > > > > > > > It may close newfd if it was already open. > > > > > > (I guess that's what he meant). > > > > > > Anyway, only dup2() should probably release the GIL. > > > > > > One reasonable heuristic is to check the man page: if the syscall > > > can return EINTR, then the GIL should be released. > > > > > > Should the call be retried in the EINTR case? > > (After a PyErr_CheckSignals) > > I don't think we want to retry low-level system calls (but I'm not sure > we're currently consistent in that regard). > I think this is what you meant but to be clear: Anywhere we're using them within a library for a good purpose, we do need to retry. If we're merely exposing them via the os module such as os.dup, its up to the caller to deal with the retry. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
Amaury Forgeot d'Arc wrote: One reasonable heuristic is to check the man page: if the syscall can return EINTR, then the GIL should be released. Hmmm, curious. According to the MacOSX man pages, both dup() and dup2() can return EINTR, but pipe() and socketpair() can't. I'm particularly surprised about dup(), as that's the one I would have thought couldn't possibly block -- it only needs to manipulate in-memory data structures. For Linux, the only one returning EINTR is dup2(). FreeBSD has none of them returning EINTR. So it looks like there's a fair bit of variation between unices here. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
Charles-François Natali wrote: Anyway, only dup2() should probably release the GIL. Depending on how your kernel is implemented, both pipe() and socketpair() could create file system entries, and therefore could block, although probably only for a very short time. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2013-01-25 - 2013-02-01)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open3848 (+11)
closed 25028 (+57)
total 28876 (+68)
Open issues with patches: 1681
Issues opened (52)
==
#17034: Use Py_CLEAR() in stringobject.c and bytesobject.c
http://bugs.python.org/issue17034 opened by serhiy.storchaka
#17035: Use new style classes in {class, static}method examples
http://bugs.python.org/issue17035 opened by berker.peksag
#17036: Implementation of the PEP 433: Easier suppression of file desc
http://bugs.python.org/issue17036 opened by haypo
#17037: Use a test.support helper to wrap the PEP 399 boilerplate
http://bugs.python.org/issue17037 opened by eric.snow
#17043: Invalid read in test_codecs
http://bugs.python.org/issue17043 opened by skrah
#17044: Implement PEP 422: Simple class initialisation hook
http://bugs.python.org/issue17044 opened by daniel.urban
#17045: Improve C-API doc for PyTypeObject
http://bugs.python.org/issue17045 opened by eric.snow
#17046: test_subprocess test_executable_without_cwd fails when run wit
http://bugs.python.org/issue17046 opened by ned.deily
#17047: Fix double double words words
http://bugs.python.org/issue17047 opened by serhiy.storchaka
#17048: calendar should understand full- vs. half-width characters
http://bugs.python.org/issue17048 opened by nneonneo
#17050: argparse.REMAINDER doesn't work as first argument
http://bugs.python.org/issue17050 opened by chris.jerdonek
#17052: unittest discovery should use self.testLoader
http://bugs.python.org/issue17052 opened by chris.jerdonek
#17053: pydoc should use inspect.signature instead of inspect.getfulla
http://bugs.python.org/issue17053 opened by ronaldoussoren
#17056: Support Visual Studio 2012
http://bugs.python.org/issue17056 opened by cowwoc
#17057: Data model documentation grammar
http://bugs.python.org/issue17057 opened by Hobson.Lane
#17060: IDLE -- jump to home should not go past the PS1 and PS2 prompt
http://bugs.python.org/issue17060 opened by rhettinger
#17061: tokenize unconditionally emits NL after comment lines & blank
http://bugs.python.org/issue17061 opened by takluyver
#17062: An os.walk inspired replacement for pkgutil.walk_packages
http://bugs.python.org/issue17062 opened by ncoghlan
#17063: assert_called_with could be more powerful if it allowed placeh
http://bugs.python.org/issue17063 opened by pitrou
#17064: Fix sporadic buildbot failures for test_mailbox
http://bugs.python.org/issue17064 opened by jkloth
#17065: Fix sporadic buildbot failures for test_winreg
http://bugs.python.org/issue17065 opened by jkloth
#17066: Fix test discovery for test_robotparser.py
http://bugs.python.org/issue17066 opened by zach.ware
#17067: Examples in documentation for 3.x in 23.1.3.4. Deferred transl
http://bugs.python.org/issue17067 opened by carsten.kl...@axn-software.de
#17068: peephole optimization for constant strings
http://bugs.python.org/issue17068 opened by nnorwitz
#17069: HTTP result code in urllib2.urlopen() file object undocumented
http://bugs.python.org/issue17069 opened by tazle
#17070: PEP 433: Use the new cloexec to improve security and avoid bug
http://bugs.python.org/issue17070 opened by haypo
#17073: Integer overflow in sqlite module
http://bugs.python.org/issue17073 opened by serhiy.storchaka
#17076: shutil.copytree failing on xattr-less filesystems (like NFS)
http://bugs.python.org/issue17076 opened by twouters
#17077: Fix test_tools hangs
http://bugs.python.org/issue17077 opened by jkloth
#17078: string.Template.safe_substitute hard-wires "braces" as {}
http://bugs.python.org/issue17078 opened by Adam.Kellas
#17079: Fix test discovery for test_ctypes.py
http://bugs.python.org/issue17079 opened by zach.ware
#17080: A better error message for float()
http://bugs.python.org/issue17080 opened by Jonathan.Livni
#17082: Fix test discovery for test_dbm*.py
http://bugs.python.org/issue17082 opened by zach.ware
#17083: can't specify newline string for readline for binary files
http://bugs.python.org/issue17083 opened by susurrus
#17084: nntplib.NNTP.xover does not always return results as documente
http://bugs.python.org/issue17084 opened by dholm
#17085: test_socket crashes the whole test suite
http://bugs.python.org/issue17085 opened by ddve...@ucar.edu
#17086: backport cross-build patches to the 2.7 branch
http://bugs.python.org/issue17086 opened by doko
#17087: Improve the repr for regular expression match objects
http://bugs.python.org/issue17087 opened by rhettinger
#17088: ElementTree incorrectly refuses to write attributes without na
http://bugs.python.org/issue17088 opened by silverbacknet
#17089: Expat parser parses strings only when XML encoding is UTF-8
http://bugs.python.org/issue17089 opened by serhiy.storchaka
#17091: thread.lock.acquire docstr
Re: [Python-Dev] Release or not release the GIL
Le Fri, 1 Feb 2013 15:18:39 +0100, "Amaury Forgeot d'Arc" a écrit : > 2013/2/1 Charles-François Natali > > > >> dup2(oldfd, newfd) closes oldfd. > > > > > > No, it doesn't close oldfd. > > > > > > It may close newfd if it was already open. > > > > (I guess that's what he meant). > > > > Anyway, only dup2() should probably release the GIL. > > > > One reasonable heuristic is to check the man page: if the syscall > > can return EINTR, then the GIL should be released. > > > Should the call be retried in the EINTR case? > (After a PyErr_CheckSignals) I don't think we want to retry low-level system calls (but I'm not sure we're currently consistent in that regard). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
2013/2/1 Charles-François Natali > >> dup2(oldfd, newfd) closes oldfd. > > > > No, it doesn't close oldfd. > > > > It may close newfd if it was already open. > > (I guess that's what he meant). > > Anyway, only dup2() should probably release the GIL. > > One reasonable heuristic is to check the man page: if the syscall can > return EINTR, then the GIL should be released. Should the call be retried in the EINTR case? (After a PyErr_CheckSignals) -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] importlib.find_loader
Bugs #17098 and #17099 filed. On Fri, Feb 1, 2013 at 1:56 AM, Thomas Heller wrote: > Am 01.02.2013 01:42, schrieb Nick Coghlan: > > Yep, looks like a bug in the bootstrapping, failing to set __loader__ >> properly. >> > > It also has the effect that reload does not work: > > > Type "help", "copyright", "credits" or "license" for more information. > >>> import imp > >>> import math > >>> imp.reload(math) > > >>> import signal > >>> imp.reload(signal) > Traceback (most recent call last): > File "", line 1, in > File "C:\Python33-64\lib\imp.py", line 252, in reload > return module.__loader__.load_module(**name) > > AttributeError: 'module' object has no attribute '__loader__' > >>> > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
>> dup2(oldfd, newfd) closes oldfd. > > No, it doesn't close oldfd. > > It may close newfd if it was already open. (I guess that's what he meant). Anyway, only dup2() should probably release the GIL. One reasonable heuristic is to check the man page: if the syscall can return EINTR, then the GIL should be released. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
> dup2(oldfd, newfd) closes oldfd. No, it doesn't close oldfd. It may close newfd if it was already open. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Release or not release the GIL
On 01/02/2013 1:44am, Guido van Rossum wrote: I'm guessing those system calls are just implemented by the kernel and cannot block for I/O, so it was deemed unnecessary to release the GIL around them. I don't mind changing that though, you can never know what happens when you make a system call. close() can block, so I assume the same is true of dup2(), since dup2(oldfd, newfd) closes oldfd. -- Richard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] issue17097:baseManager serve_client() not check EINTR when recv request
Hi All,
We create our customised manager which will fork child process with
baseManager. Because we registered SIGCHLD to reap the zombie for manager, we
found this causes baseManager raise RemoteError when called twice.(Test script
attached.)
After look at baseManager.py we found recv() in handling the request has been
interrupted by comming SIGCHLD, not retry recv(), but raise to client side
directly.
try:
methodname = obj = None
request = recv()<--this line been interrupted
by SIGCHLD
ident, methodname, args, kwds = request
obj, exposed, gettypeid = id_to_obj[ident]
if methodname not in exposed:
raise AttributeError(
'method %r of %r object is not in exposed=%r' %
(methodname, type(obj), exposed)
)
function = getattr(obj, methodname)
try:
res = function(*args, **kwds)
except Exception, e:
msg = ('#ERROR', e)
else:
typeid = gettypeid and gettypeid.get(methodname, None)
if typeid:
rident, rexposed = self.create(conn, typeid, res)
token = Token(typeid, self.address, rident)
msg = ('#PROXY', (rexposed, token))
else:
msg = ('#RETURN', res)
except AttributeError:
if methodname is None:
msg = ('#TRACEBACK', format_exc())
else:
try:
fallback_func = self.fallback_mapping[methodname]
result = fallback_func(
self, conn, ident, obj, *args, **kwds
)
msg = ('#RETURN', result)
except Exception:
msg = ('#TRACEBACK', format_exc())
except EOFError:
util.debug('got EOF -- exiting thread serving %r',
threading.current_thread().name)
sys.exit(0)
except Exception:<--does not handle IOError,INTR here should
retry recv()
msg = ('#TRACEBACK', format_exc())
REF: http://bugs.python.org/issue17097
import sys
import os
from time import sleep
import signal
from multiprocessing import Process
from multiprocessing.managers import BaseManager
import threading
class _SuperVdsmManager(BaseManager):
pass
def func():
return 'func'
class _SuperVdsm(object):
def runChild(self):
proc = Process(target=func)
proc.start()
proc.terminate()
def _zombieReaper(signum, frame):
print "sigchld!"
def startSuper():
try:
address = '/home/lvroyce.sock'
if os.path.exists(address):
os.unlink(address)
signal.signal(signal.SIGCHLD, _zombieReaper)
try:
manager = _SuperVdsmManager(address=address, authkey='abc')
manager.register('instance', callable=_SuperVdsm)
server = manager.get_server()
servThread = threading.Thread(target=server.serve_forever)
servThread.setDaemon(True)
servThread.start()
while servThread.isAlive():
servThread.join(5)
except:
raise
finally:
if os.path.exists(address):
os.unlink(address)
except Exception:
sys.exit(1)
def main():
sv = Process(target=startSuper)
sv.start()
sleep(2)
_manager = _SuperVdsmManager(address='/home/lvroyce.sock',
authkey='abc')
_manager.register('instance')
_manager.connect()
_svdsm = _manager.instance()
_svdsm.runChild()
sleep(4)
_svdsm.runChild()
if __name__ == '__main__':
main()
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
