Changes by Jesús Cea Avión j...@jcea.es:
--
nosy: +jcea
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20505
___
___
Python-bugs-list mailing list
Roundup Robot added the comment:
New changeset 4f1df287392c by Victor Stinner in branch '3.4':
Issue #20505: Use even shorter sleep in test_timeout_rounding() to make the
http://hg.python.org/cpython/rev/4f1df287392c
New changeset 6733d9dfffbb by Victor Stinner in branch '3.4':
Issue #20505:
Roundup Robot added the comment:
New changeset 75881b85695f by Victor Stinner in branch 'default':
Issue #20505: Fix TestLoop, set the clock resolution
http://hg.python.org/cpython/rev/75881b85695f
--
___
Python tracker rep...@bugs.python.org
Charles-François Natali added the comment:
Wow, 10 messages in one night...
Could you try to gather all your finding at once, because reading so many
messages in difficult to follow?
GetQueuedCompletionStatus(1 ms)-None took 0.307 ms (monotonic: 0.000 ms)
So basically, on Windows, select(1ms)
STINNER Victor added the comment:
Victor Since the granularity is back, why not using also the resolution of
the selector in asyncio? :-)
Guido Please, no. This has to stop.
Ok, sorry. If we still have some sporadic failures on UNIX, I think I will
maybe use Charles-François's suggestion: use
STINNER Victor added the comment:
GetQueuedCompletionStatus(1 ms)-None took 0.307 ms (monotonic: 0.000 ms)
So basically, on Windows, select(1ms) can be non-blocking?
I would not say non-blocking: it's just that select(N milliseconds) waits
between N-1 and N milliseconds on Windows when HPET
STINNER Victor added the comment:
Wow, 10 messages in one night... Could you try to gather all your finding at
once, because reading so many messages in difficult to follow?
It's not easy because I collect informations from various buildbots and
different virtual machines. But I planned to
Roundup Robot added the comment:
New changeset 03cb6ddc7040 by Victor Stinner in branch 'default':
Issue #20505: Improve debug info in asyncio event loop
http://hg.python.org/cpython/rev/03cb6ddc7040
--
___
Python tracker rep...@bugs.python.org
Roundup Robot added the comment:
New changeset 9c1840e8d643 by Victor Stinner in branch 'default':
Issue #20505: Oops, only print debug info if selector.select(timeout) took less
http://hg.python.org/cpython/rev/9c1840e8d643
--
___
Python tracker
Charles-François Natali added the comment:
It's not easy because I collect informations from various buildbots and
different virtual machines. But I planned to try to summarize the overall
work done on time in asyncio, select and selectors.
Thanks for the summary.
The first problem is that
Roundup Robot added the comment:
New changeset a631b01d1715 by Victor Stinner in branch 'default':
Issue #20505: use also the monotonic time to decide if asyncio debug traces
http://hg.python.org/cpython/rev/a631b01d1715
--
___
Python tracker
STINNER Victor added the comment:
More debug traces, Windows 7 with HPET enabled:
asyncio: IocpProactor.select(10. ms) took 9.486 ms (monotonic=0.000 ms,
clock res=15.600 ms)
asyncio: IocpProactor.select(0.0010 ms) took 0.942 ms (monotonic=0.000 ms,
clock res=15.600 ms)
asyncio:
STINNER Victor added the comment:
Windows 7, HPET disabled:
asyncio: IocpProactor.select(100. ms) took 99.871 ms (monotonic=109.000 ms,
clock res=15.600 ms)
asyncio: IocpProactor.select(10. ms) took 3.779 ms (monotonic=16.000 ms,
clock res=15.600 ms)
asyncio:
STINNER Victor added the comment:
test_timeout_rounding() pass on all major buildbots. test_asyncio hangs on
fails on some buildbots, but there are dedicated issues (like #20495 and
#20600).
I'm closing this one. I reopened the rounding issue for select and kqueue,
please help me to close
Roundup Robot added the comment:
New changeset 360976a6d8b9 by Victor Stinner in branch 'default':
Issue #20505: Remove debug code
http://hg.python.org/cpython/rev/360976a6d8b9
--
___
Python tracker rep...@bugs.python.org
STINNER Victor added the comment:
Thanks again Guido and Charles-François for your help on this tricky issue.
Sorry for having flood your mail box :-)
--
resolution: - fixed
status: open - closed
___
Python tracker rep...@bugs.python.org
STINNER Victor added the comment:
Ok, I reproduced the issue on my Windows 7 after enabling HPET:
== CPython 3.4.0b3+ (default:1b96d08c3895, Feb 10 2014, 18:44:24) [MSC v.1600 32
bit (Intel)]
== Windows-7-6.1.7601-SP1 little-endian
[1/1] test_asyncio
test_timeout_rounding
STINNER Victor added the comment:
On the same Windows 7 virtual machine, but with HPET disabled, the test pass.
[1/1] test_asyncio
test_timeout_rounding (test.test_asyncio.test_events.ProactorEventLoopTests) ...
GetQueuedCompletionStatus(2 ms)-None took 4.350 ms (monotonic: 16.000 ms)
Roundup Robot added the comment:
New changeset 20dc8d6430eb by Victor Stinner in branch 'default':
Issue #20505: Use even shorter sleep in test_timeout_rounding() to make the
http://hg.python.org/cpython/rev/20dc8d6430eb
--
___
Python tracker
STINNER Victor added the comment:
Here is clock_resolution.patch: reintroduce the old granularity but only use
the resolution of the clock (expect the selector to round away from zero).
clock_resolution.patch fixes test_timeout_rounding() on my Windows 7 with HPET
enabled, even with my more
Guido van Rossum added the comment:
How do you know that the timer used by the select/poll/etc. call has the same
resolution?
The variable 'now' should probably be given a more suitable name.
Can the clock resolution be zero? If not, I recommend adjusting the comparisons
so that an event
STINNER Victor added the comment:
How do you know that the timer used by the select/poll/etc. call has the same
resolution?
If I understood correctly, there a 3 kind of clocks on Windows:
- kernel heartbeat: GetSystemTimeAdjustment() gives the resolution (a few
milliseconds)
- multimedia
STINNER Victor added the comment:
OK, since the resolution is 1 nsec on my Ubuntu
Yeah, Linux always announce 1 nanosecond, even the real resolution is not so
good:
https://lkml.org/lkml/2012/2/9/100
On Mac OS X, the resolution is also probably hardcoded to 1 nanosecond. (Python
reads the
Charles-François Natali added the comment:
How sure are you? Suppose I use poll() with a 0.5 msec timeout. This
presumably gets rounded up to 1 msec. But if the system clock has e.g. a
10
msec resolution, won't this still wait 0 msec? Or will it wait until the
next tick occurs, which could
Guido van Rossum added the comment:
OK, this sounds like rounding up is important to avoid busy-wait (but maybe
only when rounding down would give us zero), and we shouldn't have to worry
about a courser timer, it will just make us late and that's always
acceptable.
On Fri, Feb 7, 2014 at 1:13
STINNER Victor added the comment:
About Windows, it looks like select() and GetQueuedCompletionStatus() (poll
function the IocpProactor) round the timeout away from zero (the timeout is a
number of milliseconds, an integer).
Examples with GetQueuedCompletionStatus():
- 2 ms (timeout) = 15 ms
STINNER Victor added the comment:
If you want to keep the current approach, nothing prevents from using a
fixed slack value, independant of the selector (e.g. 1ms seems
reasonable).
select() and kqueue() are able to sleep less than 1 ms. Using a slack of 1 ms
would reduce the accuracy. I don't
STINNER Victor added the comment:
Let me state this last point once again: no busy loop can occur now that
timeouts are rounded up.
Agreed. The busy loop issue was solved by rounding the timeout away from zero
in select and selectors module.
Sure, some syscalls, on some OS, can sometimes
Roundup Robot added the comment:
New changeset 3b94a4ef244e by Victor Stinner in branch 'default':
Issue #20505: add debug info
http://hg.python.org/cpython/rev/3b94a4ef244e
--
nosy: +python-dev
___
Python tracker rep...@bugs.python.org
STINNER Victor added the comment:
New changeset 3b94a4ef244e by Victor Stinner in branch 'default':
Issue #20505: add debug info
Some results.
AMD64 Windows7 SP1 3.x:
== Windows-7-6.1.7601-SP1 little-endian
SelectSelector.select(156.001 ms) took 156.000 ms (granularity=15.600 ms,
Charles-François Natali added the comment:
select() and kqueue() are able to sleep less than 1 ms. Using a slack of
1 ms would reduce the accuracy. I don't see why we should limit the
accuracy. Why 1 ms? Because of poll/epoll? What about Windows and its
resolution of 15.6 ms?
Well, under 1
Roundup Robot added the comment:
New changeset d853955491a2 by Victor Stinner in branch 'default':
Issue #20505: Remove resolution and _granularity from selectors and asyncio
http://hg.python.org/cpython/rev/d853955491a2
--
___
Python tracker
STINNER Victor added the comment:
More data from buildbots (before d853955491a2).
x86 Ubuntu Shared 3.x:
== Linux-2.6.31.5-linode21-i686-with-debian-lenny-sid little-endian
EpollSelector.select(99.947 ms) took 99.291 ms (granularity=1.000 ms,
resolution=1.000 ms)
EpollSelector.select(99.928
Guido van Rossum added the comment:
Let me state this last point once again: no busy loop can occur now that
timeouts are rounded up.
How sure are you? Suppose I use poll() with a 0.5 msec timeout. This
presumably gets rounded up to 1 msec. But if the system clock has e.g. a 10
msec
Charles-François Natali added the comment:
To solve a performance issue in asyncio, I added a new resolution
attribute to selectors.BaseSelector and a new _granularity attribute to
asyncio.BaseEventLoop. If I understood correctly, Charles-François (author
and so maintainer of the new selectors
STINNER Victor added the comment:
Here is a script to measure the accuracy of asyncio: min/max difference between
the scheduled time and the real elapsed time. It's not directly related to the
attached patch, but it can help if you want to implement a different option
like
Results on my
Changes by STINNER Victor victor.stin...@gmail.com:
Added file: http://bugs.python.org/file33897/add_granularity.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20505
___
New submission from STINNER Victor:
To solve a performance issue in asyncio, I added a new resolution attribute to
selectors.BaseSelector and a new _granularity attribute to
asyncio.BaseEventLoop. If I understood correctly, Charles-François (author and
so maintainer of the new selectors
38 matches
Mail list logo
