Charles-François Natali added the comment:
> I would like to suggest again my idea of doing it in Connection instead,
> with new methods (e.g. locked_send and locked_recv). Especially given
> it can be useful in user code to have a thread-safe Connection (I'm in
> this situati
Charles-François Natali added the comment:
> Charles-François, are you going to finish this before 2.7.4 RC released? The
> patch should be updated to address Antoine's comments.
No.
I don't have access to a 64-bit box, which makes it difficult to write
(I don't get compil
Charles-François Natali added the comment:
> I've added (some) docs and added checking of the BCM constants to the
> test_socket module.
This version looks good to me.
I'll commit it next week (I currently don't have access to my
development machine).
> I would
Changes by Charles-François Natali :
--
resolution: -> invalid
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> Indeed some value change when I print "cpu" line from /proc/self/status but I
> don't really understand what that mean...
It means that the CPU affinity is changed, so the scheduler binds your
process to a subset of the availa
Charles-François Natali added the comment:
Hello,
> So I'm sorry if it's not a bug...
Don't be afraid, we don't byte :-)
Concerning your problem, my guess would be that one of the modules you import
sets the process CPU affinity (maybe as a workaround to mitigate t
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28813/multi_queue.py
___
Python tracker
<http://bugs.python.org/issue17025>
___
___
Python-bug
New submission from Charles-François Natali:
Here's an implementation of the idea posted on python-ideas
(http://mail.python.org/pipermail/python-ideas/2013-January/018846.html).
The principle is really simple, we just serialize/unserialize the objects
before/after holding the locks.
Charles-François Natali added the comment:
> I thought SOCK_DGRAM messages just got truncated at the receiving end.
You were referring to partial writes: for a datagram-oriented
protocol, if the datagram can't be sent atomically (in one
send()/write() call), the kernel will return EA
Charles-François Natali added the comment:
> For SOCK_STREAM, yes, not for SOCK_DGRAM (or for a pipe when trying to
> write more than PIPE_BUF, although I guess any sensible implementation
> doesn't report the pipe write ready if there's less than PIPE_BUF
> space lef
Charles-François Natali added the comment:
> If only one byte is available, recv(4096) should simply return a partial
> result.
Of course, but how do you know if there's data left to read without
calling select() again? It's much better to call read() until you get
EAGAIN tha
Charles-François Natali added the comment:
> It appears that Linux's "spurious readiness notifications" are a deliberate
> deviation from the POSIX standard. (They are mentioned in the BUGS section
> of the man page for select.)
I don't think it's a del
Charles-François Natali added the comment:
> I have created an iocp branch.
You could probably report the fixes for spurious notifications in the
default branch.
--
___
Python tracker
<http://bugs.python.org/issu
Charles-François Natali added the comment:
Time for a stupid question from someone who doesn't know anything about
Windows: if WSAPoll() is really terminally broken, is it really worth the
hassle exposing it and warping the API?
AFAICT, FD_SETSIZE is already bumped to 512 on Windows
Charles-François Natali added the comment:
Committed, thanks for the patch!
--
stage: -> committed/rejected
status: open -> closed
versions: +Python 2.7, Python 3.2, Python 3.4
___
Python tracker
<http://bugs.python.org/i
Changes by Charles-François Natali :
--
nosy: +v+python
___
Python tracker
<http://bugs.python.org/issue16945>
___
___
Python-bugs-list mailing list
Unsubscribe:
Charles-François Natali added the comment:
Could you provide the patch?
It's trivial, but at least we'll make sure the patch fixes the problem on your
platform.
--
nosy: +neologix
___
Python tracker
<http://bugs.python.o
Charles-François Natali added the comment:
There are actually two distinct issues.
For the first one, the problem is really a distribution issue: the libc is more
recent than the kernel, and exports *utimes() whereas the kernel doesn't
implement those syscalls, which results in ENOS
Changes by Charles-François Natali :
--
resolution: -> duplicate
stage: patch review -> committed/rejected
status: open -> closed
superseder: -> Broken test_kqueue.py on OpenBSD
___
Python tracker
<http://bugs.python
Charles-François Natali added the comment:
> It's interesting. The example behaves unstable on 3.3+ with C implementation
> of picle, sometimes works, sometimes fails. With Python implementation and on
> 3.2 it works always. On 2.7 it fails always.
That's hash randomizatio
Changes by Charles-François Natali :
--
nosy: +orsenthil
___
Python tracker
<http://bugs.python.org/issue16945>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Charles-François Natali:
test_kqueue fails on both NetBSD and OpenBSD:
http://buildbot.python.org/all/builders/AMD64 NetBSD 5.1.2 [SB]
2.7/builds/206/steps/test/logs/stdio
http://buildbot.python.org/all/builders/x86 OpenBSD 5.1 [SB]
2.7/builds/153/steps/test/logs/stdio
Charles-François Natali added the comment:
>> I'll wait a little before removing O_NOFOLLOW
>
> I don't know this flag. What is its effect of the directory part of the path?
> Does it change anything if the directory is a symbolic link?
No, it only has an effect if the
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
New submission from Charles-François Natali:
On Unix, CGIHTTPRequestHandler.run_cgi() uses the following code to run a CGI
script:
"""
pid = os.fork()
[...]
# Child
try:
try:
o
Changes by Charles-François Natali :
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue16929>
___
___
Python-bugs-list mailing list
Un
Charles-François Natali added the comment:
This patch should be updated to also fix epoll().
Also, is it right to raise an exception in case of concurrent invocation?
Here's a simple script that crashes systematically on my Linux box.
--
nosy: +neologix
Added file:
Charles-François Natali added the comment:
OK, I'll close as duplicate.
--
dependencies: -select.poll is not thread safe
resolution: -> duplicate
superseder: -> select.poll is not thread safe
___
Python tracker
<http://bugs.python.
New submission from Charles-François Natali:
After optimizing epoll() to use a per-instance buffer like poll() does
(http://bugs.python.org/issue16876), I realized that it wasn't thread-safe, and
can result in crashes:
"""
./python /tmp/test.py
*** glibc detecte
Charles-François Natali added the comment:
> - How shall we go forward? I've made a variety of small changes to the
> Tulip version (selector.py) now. Can you work those into a fresh unified
> patch for CPython 3.4?
Yes, but I think we could wait a little, to make sure the API
Changes by Charles-François Natali :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
> "I realize this bugreport cannot fix 35 years of a bad design decision in
> linux."
>
> Well... Ruby made a brave choice :-) Ruby (2.0?) does set
> close-on-exec flag on *ALL file descriptors (except 0, 1, 2) *by
>
Charles-François Natali added the comment:
> Please consider my patches instead; it seems our patches crossed. Merging is
> now difficult because I already submitted my version to Tulip.
That's fine, I don't think there's a point into maintaining a
standalone patch for n
Charles-François Natali added the comment:
As it is now, _Key is actually an implementation detail, that's why it's not
part of the methods signature. Oonly the base register()/unregister() return
the key because it's an easy way for the derived classes to get the FD w
Charles-François Natali added the comment:
This should fix some kqueue failures (the FDs were not unregistered,
which could result in spurious FDs being reported at a later time).
For the "negative FDs", all spurious events are now caught and logged
(I think it's better than si
Charles-François Natali added the comment:
Should be fixed now, thanks.
--
resolution: -> fixed
stage: commit review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Charles-François Natali added the comment:
So I assume that the second failure is fixed, which means that OS-X
returns a funny event (and not POLLIN/POLLHUP/POLLERR) in case of
ECONNREFUSED :-(
> 2 - In EventLoopTestsMixin::test_writer_callback if the writer socket isn't
> non-bl
Charles-François Natali added the comment:
Nick, I think this would be a great addition (I have often seen people trying
to implement this in their own code, and I certainly did it myself).
What's the status of walkdir, do you think it's mature enough to
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28619/tulip-selectors-2.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28617/selector-8.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28622/tulip-selectors-2.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28621/selector-8.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Python-bug
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28575/selector-data.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28564/selector-3.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28619/tulip-selectors-2.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28584/selector-5.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28618/tulip-selectors-2.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28618/tulip-selectors-2.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28617/selector-8.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Python-bug
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28602/selector-6.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28606/selector_poll_events.diff
___
Python tracker
<http://bugs.python.org/issue16
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28615/tulip-selectors-2.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28616/selector-8.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Changes by Charles-François Natali :
--
nosy: +loewis
___
Python tracker
<http://bugs.python.org/issue16836>
___
___
Python-bugs-list mailing list
Unsubscribe:
Charles-François Natali added the comment:
> The second test runs fine on Linux, and from a cursory look, I don't
> see how it could fail (the socket should be reported as write ready
> upon ECONNREFUSED).
Hum, thinking about it, I wonder is OS-X doesn't report POLLPRI or
Charles-François Natali added the comment:
> Also notice the need for a third constant, SELECT_CONNECT. For details see
> the class WindowsPollPollster in the Tulip code.
I'll trust Richard on all Windows matter, so if you need a
SELECT_CONNECT constant, I'll expose one.
>
Charles-François Natali added the comment:
> If you don't want to change the API back to separate register_*() methods for
> readers and writers, perhaps you can add a method that tells me, for a given
> fileobj, whether it is registered, and with which poll flags
> (SELECT
Charles-François Natali added the comment:
Updated to reflect Richard's comment.
--
Added file: http://bugs.python.org/file28601/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/is
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28600/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28589/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28600/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
--
nosy: +sbt
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Python-bugs-list mailing list
Unsubscribe:
Charles-François Natali added the comment:
You're right, I just saw my mistake.
Sorry for the noise (the extra performance arguement is not a good enough
motivation to tune this dynaically).
Closing.
--
resolution: -> invalid
stage: needs patch -> committed/rejected
s
Charles-François Natali added the comment:
Of course it does, the write ends all get filled, so the number of ready
writers drops to zero... That's not all at the problem I'm talking about (I may
not be clear though, a script will make it
Charles-François Natali added the comment:
> That assumes that epoll_wait() is supposed to return *all* ready fds. But
> that is not possible because maxevents is finite. If you want all events
> then obviously you may need to call epoll_wait() multiple times.
Yes, but the proble
Charles-François Natali added the comment:
> The program does *not* demonstrate starvation because you are servicing the
> resource represented by the "starved" duplicate fds before calling poll()
> again.
No.
What the program does is the following:
while all the writ
Charles-François Natali added the comment:
The implementation can't simply drain a queue, because it can be
level-triggered (which is the case by default), so you want to keep
events around. The kernel actually uses a red-black tree, but I didn't
really check how it's done (there&
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28589/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28587/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28587/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28586/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28586/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28585/epoll_realloc.diff
___
Python tracker
<http://bugs.python.org/issue16876>
___
___
Pytho
New submission from Charles-François Natali:
Currently, epoll.poll() allocates an epoll_event buffer every time/ this is
bad, because it changes an O(number of ready FDs) syscall into a
O(maxevents/FD_SETSIZE) complexity.
The patch attached allocates a epoll events buffer per epoll instance
New submission from Charles-François Natali:
In issue #16853, it was noted that many several projects don't set epoll.poll()
maxevents argument, which effectively limits the number of events retuend to
FD_SETSIZE-1 (set in selectmodule.c).
Also, the methode documentation can confuse
Charles-François Natali added the comment:
I've noticed a bug present in Tulip, pyftpdlib and tornado (and this
implementation, too): none of them passes the maxevents argument to
epoll.poll(), and by default the poll() wrapper in the socket module
uses FD_SETSIZE-1, so you'll neve
Charles-François Natali added the comment:
> I think that this needs extensive tests that verify the behavior of many end
> cases, including under duress (e.g. when there are too many connections for
> the kernel to handle). That would seem the only way to make sure that the
Charles-François Natali added the comment:
>> there's no SELECT_ERR anymore [...] the error will be reported when the FD
>> will be read/written
>
> I don't think this is a good idea, particularly during this early stage.
> This assumption might be valid for
Charles-François Natali added the comment:
Here's a new version, with the following changes:
- there's no SELECT_ERR anymore: error conditions (POLLNVAL|POLLERR and also
POLLHUP) are reported as SELECT_IN|SELECT_OUT, depending on the input event
mask: I don't think that a sepa
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28556/selector-2.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Charles-François Natali added the comment:
Ross, the select() result for a large number of ready FDs was completely skewed
because of a bug spotted by Antoine (complexity was much worse than it ought to
be).
Here are the results with the fix:
"""
$ ./python ~/selector_bench.py -
Charles-François Natali added the comment:
Indeed, tornado and pyftpdlib implementations are much more complex,
have somewhat different APIs (i.e. only support FDs), have special
cases for older python versions, lack some newer features (e.g.
context manager).
The current code is much close to
Changes by Charles-François Natali :
--
stage: needs patch -> commit review
___
Python tracker
<http://bugs.python.org/issue16836>
___
___
Python-bugs-list mai
Charles-François Natali added the comment:
I've committed it only to default, since it's not really a bug, but rather an
improvement (if we did consider this a "security" bug then it should also be
backported to 2.7, 3.1, etc).
I'll wait a little before removing O_NO
Charles-François Natali added the comment:
Brian, could you add tests to Lib/test/test_socket.py (look for CANTest, you
should be able to complete them).
--
___
Python tracker
<http://bugs.python.org/issue15
Charles-François Natali added the comment:
Here's the patch.
It also removes O_NOFOLLOW, which is basically useless (if the file is created
with O_CREAT|O_EXCL, then by definition it's not a symlink).
--
keywords: +needs review, patch
type: -> behavior
Add
Charles-François Natali added the comment:
> O_CLOEXEC solves for example a race condition in tempfile._mkstemp_inner():
>
> fd = _os.open(file, flags, 0o600)
> _set_cloexec(fd)
Hum...
"""
diff --git a/Lib/tempfile.py b/Lib/tempfile.py
---
Charles-François Natali added the comment:
> Windows provides O_NOINHERIT (_O_NOINHERIT) flag which has a similar purpose.
>
>> ... and even then, many Unices don't support it.
>
> Yes, but I bet that more and more OSes will support it :-) For example, it
> looks lik
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28553/selector-1.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28557/selector_bench.py
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Pytho
Charles-François Natali added the comment:
Here's a new version adressing Guido's comments (except for kqueue, for which
I'll add support later when I can test it).
I'm also attaching a benchmark to compare the implementations: as noted by
Guido, the complexity of selec
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file28546/selector.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Python-bug
Charles-François Natali added the comment:
Here's a new version closer to Tulip's one.
--
Added file: http://bugs.python.org/file28553/selector-1.diff
___
Python tracker
<http://bugs.python.o
Charles-François Natali added the comment:
I don't comfortable exposing this.
The main reason is that this flag is really non-portable.
Having open() fail at runtime because the platform doesn't support it looks
really wrong to me. And silently ignore it is even worse.
The 'x&
Charles-François Natali added the comment:
Oh, I noticed that EpollPollster never closes the underlying epoll FD:
that's why Pollster objects should have a close() method (and probably
support a context manager).
--
___
Python tracker
Charles-François Natali added the comment:
> I think you have a point. Did you know about the tulip project?
>
> http://code.google.com/p/tulip/source/browse/tulip/unix_events.py#76
>
> It has a PollsterBase class and a SelectPollster(PollsterBase) so the idea is
> to have a P
Charles-François Natali added the comment:
Here's a patch, tested with ipv6.disable=1.
--
keywords: +patch
nosy: +neologix
Added file: http://bugs.python.org/file28549/ipv6config.diff
___
Python tracker
<http://bugs.python.org/is
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28548/selector_telnetlib.diff
___
Python tracker
<http://bugs.python.org/issue16853>
___
___
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file28547/selector_multiprocessing.diff
___
Python tracker
<http://bugs.python.org/issue16
New submission from Charles-François Natali:
Recently, the multiprocessing and telnetlib modules have been patched to use
poll() instead of select() when available (mainly for the FD_SETSIZE
limitation):
http://bugs.python.org/issue10527
http://bugs.python.org/issue14635
This leads to code
801 - 900 of 1864 matches
Mail list logo