[issue17025] reduce multiprocessing.Queue contention

2013-01-27 Thread Charles-François Natali
Charles-François Natali added the comment: > I would like to suggest again my idea of doing it in Connection instead, > with new methods (e.g. locked_send and locked_recv). Especially given > it can be useful in user code to have a thread-safe Connection (I'm in > this situati

[issue13555] cPickle MemoryError when loading large file (while pickle works)

2013-01-27 Thread Charles-François Natali
Charles-François Natali added the comment: > Charles-François, are you going to finish this before 2.7.4 RC released? The > patch should be updated to address Antoine's comments. No. I don't have access to a 64-bit box, which makes it difficult to write (I don't get compil

[issue15359] Sockets support for CAN_BCM

2013-01-26 Thread Charles-François Natali
Charles-François Natali added the comment: > I've added (some) docs and added checking of the BCM constants to the > test_socket module. This version looks good to me. I'll commit it next week (I currently don't have access to my development machine). > I would

[issue17038] multiprocessing only use one core when C module are imported

2013-01-26 Thread Charles-François Natali
Changes by Charles-François Natali : -- resolution: -> invalid stage: -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.or

[issue17038] multiprocessing only use one core when C module are imported

2013-01-26 Thread Charles-François Natali
Charles-François Natali added the comment: > Indeed some value change when I print "cpu" line from /proc/self/status but I > don't really understand what that mean... It means that the CPU affinity is changed, so the scheduler binds your process to a subset of the availa

[issue17038] multiprocessing only use one core when C module are imported

2013-01-26 Thread Charles-François Natali
Charles-François Natali added the comment: Hello, > So I'm sorry if it's not a bug... Don't be afraid, we don't byte :-) Concerning your problem, my guess would be that one of the modules you import sets the process CPU affinity (maybe as a workaround to mitigate t

[issue17025] reduce multiprocessing.Queue contention

2013-01-24 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28813/multi_queue.py ___ Python tracker <http://bugs.python.org/issue17025> ___ ___ Python-bug

[issue17025] reduce multiprocessing.Queue contention

2013-01-24 Thread Charles-François Natali
New submission from Charles-François Natali: Here's an implementation of the idea posted on python-ideas (http://mail.python.org/pipermail/python-ideas/2013-January/018846.html). The principle is really simple, we just serialize/unserialize the objects before/after holding the locks.

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-22 Thread Charles-François Natali
Charles-François Natali added the comment: > I thought SOCK_DGRAM messages just got truncated at the receiving end. You were referring to partial writes: for a datagram-oriented protocol, if the datagram can't be sent atomically (in one send()/write() call), the kernel will return EA

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-22 Thread Charles-François Natali
Charles-François Natali added the comment: > For SOCK_STREAM, yes, not for SOCK_DGRAM (or for a pipe when trying to > write more than PIPE_BUF, although I guess any sensible implementation > doesn't report the pipe write ready if there's less than PIPE_BUF > space lef

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-22 Thread Charles-François Natali
Charles-François Natali added the comment: > If only one byte is available, recv(4096) should simply return a partial > result. Of course, but how do you know if there's data left to read without calling select() again? It's much better to call read() until you get EAGAIN tha

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-22 Thread Charles-François Natali
Charles-François Natali added the comment: > It appears that Linux's "spurious readiness notifications" are a deliberate > deviation from the POSIX standard. (They are mentioned in the BUGS section > of the man page for select.) I don't think it's a del

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-21 Thread Charles-François Natali
Charles-François Natali added the comment: > I have created an iocp branch. You could probably report the fixes for spurious notifications in the default branch. -- ___ Python tracker <http://bugs.python.org/issu

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-20 Thread Charles-François Natali
Charles-François Natali added the comment: Time for a stupid question from someone who doesn't know anything about Windows: if WSAPoll() is really terminally broken, is it really worth the hassle exposing it and warping the API? AFAICT, FD_SETSIZE is already bumped to 512 on Windows

[issue16953] select module compile errors with broken poll()

2013-01-19 Thread Charles-François Natali
Charles-François Natali added the comment: Committed, thanks for the patch! -- stage: -> committed/rejected status: open -> closed versions: +Python 2.7, Python 3.2, Python 3.4 ___ Python tracker <http://bugs.python.org/i

[issue16945] rewrite CGIHTTPRequestHandler to always use subprocess

2013-01-19 Thread Charles-François Natali
Changes by Charles-François Natali : -- nosy: +v+python ___ Python tracker <http://bugs.python.org/issue16945> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue16953] select module compile errors with broken poll()

2013-01-18 Thread Charles-François Natali
Charles-François Natali added the comment: Could you provide the patch? It's trivial, but at least we'll make sure the patch fixes the problem on your platform. -- nosy: +neologix ___ Python tracker <http://bugs.python.o

[issue15448] utimes() functions fail with ENOSYS even when detected by configure

2013-01-17 Thread Charles-François Natali
Charles-François Natali added the comment: There are actually two distinct issues. For the first one, the problem is really a distribution issue: the libc is more recent than the kernel, and exports *utimes() whereas the kernel doesn't implement those syscalls, which results in ENOS

[issue16952] test_kqueue failure on NetBSD/OpenBSD

2013-01-17 Thread Charles-François Natali
Changes by Charles-François Natali : -- resolution: -> duplicate stage: patch review -> committed/rejected status: open -> closed superseder: -> Broken test_kqueue.py on OpenBSD ___ Python tracker <http://bugs.python

[issue10701] Error pickling a dict

2013-01-16 Thread Charles-François Natali
Charles-François Natali added the comment: > It's interesting. The example behaves unstable on 3.3+ with C implementation > of picle, sometimes works, sometimes fails. With Python implementation and on > 3.2 it works always. On 2.7 it fails always. That's hash randomizatio

[issue16945] rewrite CGIHTTPRequestHandler to always use subprocess

2013-01-13 Thread Charles-François Natali
Changes by Charles-François Natali : -- nosy: +orsenthil ___ Python tracker <http://bugs.python.org/issue16945> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue16952] test_kqueue failure on NetBSD/OpenBSD

2013-01-13 Thread Charles-François Natali
New submission from Charles-François Natali: test_kqueue fails on both NetBSD and OpenBSD: http://buildbot.python.org/all/builders/AMD64 NetBSD 5.1.2 [SB] 2.7/builds/206/steps/test/logs/stdio http://buildbot.python.org/all/builders/x86 OpenBSD 5.1 [SB] 2.7/builds/153/steps/test/logs/stdio

[issue16860] Use O_CLOEXEC in the tempfile module

2013-01-12 Thread Charles-François Natali
Charles-François Natali added the comment: >> I'll wait a little before removing O_NOFOLLOW > > I don't know this flag. What is its effect of the directory part of the path? > Does it change anything if the directory is a symbolic link? No, it only has an effect if the

[issue16762] test_subprocess failure on OpenBSD/NetBSD buildbots

2013-01-12 Thread Charles-François Natali
Changes by Charles-François Natali : -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.or

[issue16945] rewrite CGIHTTPRequestHandler to always use subprocess

2013-01-12 Thread Charles-François Natali
New submission from Charles-François Natali: On Unix, CGIHTTPRequestHandler.run_cgi() uses the following code to run a CGI script: """ pid = os.fork() [...] # Child try: try: o

[issue16929] poll()/epoll() are not thread-safe

2013-01-11 Thread Charles-François Natali
Changes by Charles-François Natali : -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue16929> ___ ___ Python-bugs-list mailing list Un

[issue8865] select.poll is not thread safe

2013-01-11 Thread Charles-François Natali
Charles-François Natali added the comment: This patch should be updated to also fix epoll(). Also, is it right to raise an exception in case of concurrent invocation? Here's a simple script that crashes systematically on my Linux box. -- nosy: +neologix Added file:

[issue16929] poll()/epoll() are not thread-safe

2013-01-11 Thread Charles-François Natali
Charles-François Natali added the comment: OK, I'll close as duplicate. -- dependencies: -select.poll is not thread safe resolution: -> duplicate superseder: -> select.poll is not thread safe ___ Python tracker <http://bugs.python.

[issue16929] poll()/epoll() are not thread-safe

2013-01-11 Thread Charles-François Natali
New submission from Charles-François Natali: After optimizing epoll() to use a per-instance buffer like poll() does (http://bugs.python.org/issue16876), I realized that it wasn't thread-safe, and can result in crashes: """ ./python /tmp/test.py *** glibc detecte

[issue16853] add a Selector to the select module

2013-01-09 Thread Charles-François Natali
Charles-François Natali added the comment: > - How shall we go forward? I've made a variety of small changes to the > Tulip version (selector.py) now. Can you work those into a fresh unified > patch for CPython 3.4? Yes, but I think we could wait a little, to make sure the API

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-09 Thread Charles-François Natali
Changes by Charles-François Natali : -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.or

[issue12107] TCP listening sockets created without FD_CLOEXEC flag

2013-01-09 Thread Charles-François Natali
Charles-François Natali added the comment: > "I realize this bugreport cannot fix 35 years of a bad design decision in > linux." > > Well... Ruby made a brave choice :-) Ruby (2.0?) does set > close-on-exec flag on *ALL file descriptors (except 0, 1, 2) *by >

[issue16853] add a Selector to the select module

2013-01-09 Thread Charles-François Natali
Charles-François Natali added the comment: > Please consider my patches instead; it seems our patches crossed. Merging is > now difficult because I already submitted my version to Tulip. That's fine, I don't think there's a point into maintaining a standalone patch for n

[issue16853] add a Selector to the select module

2013-01-08 Thread Charles-François Natali
Charles-François Natali added the comment: As it is now, _Key is actually an implementation detail, that's why it's not part of the methods signature. Oonly the base register()/unregister() return the key because it's an easy way for the derived classes to get the FD w

[issue16853] add a Selector to the select module

2013-01-08 Thread Charles-François Natali
Charles-François Natali added the comment: This should fix some kqueue failures (the FDs were not unregistered, which could result in spurious FDs being reported at a later time). For the "negative FDs", all spurious events are now caught and logged (I think it's better than si

[issue16836] configure script disables support for IPv6 on a system where IPv6 is disabled

2013-01-08 Thread Charles-François Natali
Charles-François Natali added the comment: Should be fixed now, thanks. -- resolution: -> fixed stage: commit review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.or

[issue16853] add a Selector to the select module

2013-01-08 Thread Charles-François Natali
Charles-François Natali added the comment: So I assume that the second failure is fixed, which means that OS-X returns a funny event (and not POLLIN/POLLHUP/POLLERR) in case of ECONNREFUSED :-( > 2 - In EventLoopTestsMixin::test_writer_callback if the writer socket isn't > non-bl

[issue13229] Improve tools for iterating over filesystem directories

2013-01-07 Thread Charles-François Natali
Charles-François Natali added the comment: Nick, I think this would be a great addition (I have often seen people trying to implement this in their own code, and I certainly did it myself). What's the status of walkdir, do you think it's mature enough to

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28619/tulip-selectors-2.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28617/selector-8.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28622/tulip-selectors-2.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28621/selector-8.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Python-bug

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28575/selector-data.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28564/selector-3.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28619/tulip-selectors-2.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28584/selector-5.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28618/tulip-selectors-2.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28618/tulip-selectors-2.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28617/selector-8.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Python-bug

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28602/selector-6.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28606/selector_poll_events.diff ___ Python tracker <http://bugs.python.org/issue16

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28615/tulip-selectors-2.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28616/selector-8.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16836] configure script disables support for IPv6 on a system where IPv6 is disabled

2013-01-07 Thread Charles-François Natali
Changes by Charles-François Natali : -- nosy: +loewis ___ Python tracker <http://bugs.python.org/issue16836> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Charles-François Natali added the comment: > The second test runs fine on Linux, and from a cursory look, I don't > see how it could fail (the socket should be reported as write ready > upon ECONNREFUSED). Hum, thinking about it, I wonder is OS-X doesn't report POLLPRI or

[issue16853] add a Selector to the select module

2013-01-07 Thread Charles-François Natali
Charles-François Natali added the comment: > Also notice the need for a third constant, SELECT_CONNECT. For details see > the class WindowsPollPollster in the Tulip code. I'll trust Richard on all Windows matter, so if you need a SELECT_CONNECT constant, I'll expose one. >

[issue16853] add a Selector to the select module

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: > If you don't want to change the API back to separate register_*() methods for > readers and writers, perhaps you can add a method that tells me, for a given > fileobj, whether it is registered, and with which poll flags > (SELECT

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: Updated to reflect Richard's comment. -- Added file: http://bugs.python.org/file28601/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/is

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28600/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28589/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28600/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : -- nosy: +sbt ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: You're right, I just saw my mistake. Sorry for the noise (the extra performance arguement is not a good enough motivation to tune this dynaically). Closing. -- resolution: -> invalid stage: needs patch -> committed/rejected s

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: Of course it does, the write ends all get filled, so the number of ready writers drops to zero... That's not all at the problem I'm talking about (I may not be clear though, a script will make it

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: > That assumes that epoll_wait() is supposed to return *all* ready fds. But > that is not possible because maxevents is finite. If you want all events > then obviously you may need to call epoll_wait() multiple times. Yes, but the proble

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: > The program does *not* demonstrate starvation because you are servicing the > resource represented by the "starved" duplicate fds before calling poll() > again. No. What the program does is the following: while all the writ

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: The implementation can't simply drain a queue, because it can be level-triggered (which is the case by default), so you want to keep events around. The kernel actually uses a red-black tree, but I didn't really check how it's done (there&

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28589/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28587/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28587/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28586/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-05 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28586/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-05 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28585/epoll_realloc.diff ___ Python tracker <http://bugs.python.org/issue16876> ___ ___ Pytho

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-05 Thread Charles-François Natali
New submission from Charles-François Natali: Currently, epoll.poll() allocates an epoll_event buffer every time/ this is bad, because it changes an O(number of ready FDs) syscall into a O(maxevents/FD_SETSIZE) complexity. The patch attached allocates a epoll events buffer per epoll instance

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-05 Thread Charles-François Natali
New submission from Charles-François Natali: In issue #16853, it was noted that many several projects don't set epoll.poll() maxevents argument, which effectively limits the number of events retuend to FD_SETSIZE-1 (set in selectmodule.c). Also, the methode documentation can confuse

[issue16853] add a Selector to the select module

2013-01-05 Thread Charles-François Natali
Charles-François Natali added the comment: I've noticed a bug present in Tulip, pyftpdlib and tornado (and this implementation, too): none of them passes the maxevents argument to epoll.poll(), and by default the poll() wrapper in the socket module uses FD_SETSIZE-1, so you'll neve

[issue16853] add a Selector to the select module

2013-01-05 Thread Charles-François Natali
Charles-François Natali added the comment: > I think that this needs extensive tests that verify the behavior of many end > cases, including under duress (e.g. when there are too many connections for > the kernel to handle). That would seem the only way to make sure that the

[issue16853] add a Selector to the select module

2013-01-05 Thread Charles-François Natali
Charles-François Natali added the comment: >> there's no SELECT_ERR anymore [...] the error will be reported when the FD >> will be read/written > > I don't think this is a good idea, particularly during this early stage. > This assumption might be valid for

[issue16853] add a Selector to the select module

2013-01-05 Thread Charles-François Natali
Charles-François Natali added the comment: Here's a new version, with the following changes: - there's no SELECT_ERR anymore: error conditions (POLLNVAL|POLLERR and also POLLHUP) are reported as SELECT_IN|SELECT_OUT, depending on the input event mask: I don't think that a sepa

[issue16853] add a Selector to the select module

2013-01-04 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28556/selector-2.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: Ross, the select() result for a large number of ready FDs was completely skewed because of a bug spotted by Antoine (complexity was much worse than it ought to be). Here are the results with the fix: """ $ ./python ~/selector_bench.py -

[issue16853] add a Selector to the select module

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: Indeed, tornado and pyftpdlib implementations are much more complex, have somewhat different APIs (i.e. only support FDs), have special cases for older python versions, lack some newer features (e.g. context manager). The current code is much close to

[issue16836] configure script disables support for IPv6 on a system where IPv6 is disabled

2013-01-04 Thread Charles-François Natali
Changes by Charles-François Natali : -- stage: needs patch -> commit review ___ Python tracker <http://bugs.python.org/issue16836> ___ ___ Python-bugs-list mai

[issue16860] Use O_CLOEXEC in the tempfile module

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: I've committed it only to default, since it's not really a bug, but rather an improvement (if we did consider this a "security" bug then it should also be backported to 2.7, 3.1, etc). I'll wait a little before removing O_NO

[issue15359] Sockets support for CAN_BCM

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: Brian, could you add tests to Lib/test/test_socket.py (look for CANTest, you should be able to complete them). -- ___ Python tracker <http://bugs.python.org/issue15

[issue16860] Use O_CLOEXEC in the tempfile module

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: Here's the patch. It also removes O_NOFOLLOW, which is basically useless (if the file is created with O_CREAT|O_EXCL, then by definition it's not a symlink). -- keywords: +needs review, patch type: -> behavior Add

[issue16850] Atomic open + close-and-exec

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: > O_CLOEXEC solves for example a race condition in tempfile._mkstemp_inner(): > > fd = _os.open(file, flags, 0o600) > _set_cloexec(fd) Hum... """ diff --git a/Lib/tempfile.py b/Lib/tempfile.py ---

[issue16850] Atomic open + close-and-exec

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: > Windows provides O_NOINHERIT (_O_NOINHERIT) flag which has a similar purpose. > >> ... and even then, many Unices don't support it. > > Yes, but I bet that more and more OSes will support it :-) For example, it > looks lik

[issue16853] add a Selector to the select module

2013-01-04 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28553/selector-1.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-04 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28557/selector_bench.py ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Pytho

[issue16853] add a Selector to the select module

2013-01-04 Thread Charles-François Natali
Charles-François Natali added the comment: Here's a new version adressing Guido's comments (except for kqueue, for which I'll add support later when I can test it). I'm also attaching a benchmark to compare the implementations: as noted by Guido, the complexity of selec

[issue16853] add a Selector to the select module

2013-01-03 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file28546/selector.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___ Python-bug

[issue16853] add a Selector to the select module

2013-01-03 Thread Charles-François Natali
Charles-François Natali added the comment: Here's a new version closer to Tulip's one. -- Added file: http://bugs.python.org/file28553/selector-1.diff ___ Python tracker <http://bugs.python.o

[issue16850] Atomic open + close-and-exec

2013-01-03 Thread Charles-François Natali
Charles-François Natali added the comment: I don't comfortable exposing this. The main reason is that this flag is really non-portable. Having open() fail at runtime because the platform doesn't support it looks really wrong to me. And silently ignore it is even worse. The 'x&

[issue16853] add a Selector to the select module

2013-01-03 Thread Charles-François Natali
Charles-François Natali added the comment: Oh, I noticed that EpollPollster never closes the underlying epoll FD: that's why Pollster objects should have a close() method (and probably support a context manager). -- ___ Python tracker

[issue16853] add a Selector to the select module

2013-01-03 Thread Charles-François Natali
Charles-François Natali added the comment: > I think you have a point. Did you know about the tulip project? > > http://code.google.com/p/tulip/source/browse/tulip/unix_events.py#76 > > It has a PollsterBase class and a SelectPollster(PollsterBase) so the idea is > to have a P

[issue16836] configure script disables support for IPv6 on a system where IPv6 is disabled

2013-01-03 Thread Charles-François Natali
Charles-François Natali added the comment: Here's a patch, tested with ipv6.disable=1. -- keywords: +patch nosy: +neologix Added file: http://bugs.python.org/file28549/ipv6config.diff ___ Python tracker <http://bugs.python.org/is

[issue16853] add a Selector to the select module

2013-01-03 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28548/selector_telnetlib.diff ___ Python tracker <http://bugs.python.org/issue16853> ___ ___

[issue16853] add a Selector to the select module

2013-01-03 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file28547/selector_multiprocessing.diff ___ Python tracker <http://bugs.python.org/issue16

[issue16853] add a Selector to the select module

2013-01-03 Thread Charles-François Natali
New submission from Charles-François Natali: Recently, the multiprocessing and telnetlib modules have been patched to use poll() instead of select() when available (mainly for the FD_SETSIZE limitation): http://bugs.python.org/issue10527 http://bugs.python.org/issue14635 This leads to code

<    4   5   6   7   8   9   10   11   12   13   >