[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-05 Thread Charles-François Natali 

New submission from Charles-François Natali:

Currently, epoll.poll() allocates an epoll_event buffer every time/ this is 
bad, because it changes an O(number of ready FDs) syscall into a 
O(maxevents/FD_SETSIZE) complexity.
The patch attached allocates a epoll events buffer per epoll instance, and it 
only gets reallocated when necessary (note that the reallocation heuristic will 
probably be improved in #16873 (and having a per instance maxevents count will 
likely be useful).

Here's a benchmark without patch:
$ ./python -m timeit -s 'import select; ep = select.epoll(); ep.register(1, 
select.EPOLLOUT)' 'ep.poll()'
10 loops, best of 3: 4.25 usec per loop

With patch:
$ ./python -m timeit -s 'import select; ep = select.epoll(); ep.register(1, 
select.EPOLLOUT)' 'ep.poll()'
10 loops, best of 3: 3.38 usec per loop

--
components: Extension Modules
files: epoll_realloc.diff
keywords: needs review, patch
messages: 179167
nosy: neologix
priority: normal
severity: normal
stage: patch review
status: open
title: epoll: reuse epoll_event buffer instead of allocating a new one at each 
poll()
type: performance
versions: Python 3.4
Added file: http://bugs.python.org/file28585/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-05 Thread Charles-François Natali 

Changes by Charles-François Natali :


Removed file: http://bugs.python.org/file28585/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-05 Thread Christian Heimes 

Changes by Christian Heimes :


--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-05 Thread Charles-François Natali 

Changes by Charles-François Natali :


Added file: http://bugs.python.org/file28586/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


Removed file: http://bugs.python.org/file28586/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


Added file: http://bugs.python.org/file28587/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


Removed file: http://bugs.python.org/file28587/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


Added file: http://bugs.python.org/file28589/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


--
nosy: +sbt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


Added file: http://bugs.python.org/file28600/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


Removed file: http://bugs.python.org/file28589/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Changes by Charles-François Natali :


Removed file: http://bugs.python.org/file28600/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-06 Thread Charles-François Natali 

Charles-François Natali added the comment:

Updated to reflect Richard's comment.

--
Added file: http://bugs.python.org/file28601/epoll_realloc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-09 Thread Roundup Robot 

Roundup Robot added the comment:

New changeset be8e6b81284e by Charles-François Natali in branch 'default':
Issue #16876: Optimize epoll.poll() by keeping a per-instance epoll events
http://hg.python.org/cpython/rev/be8e6b81284e

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-09 Thread Charles-François Natali 

Changes by Charles-François Natali :


--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16876] epoll: reuse epoll_event buffer instead of allocating a new one at each poll()

2013-01-12 Thread Roundup Robot 

Roundup Robot added the comment:

New changeset 30eb98c8afef by Charles-François Natali in branch 'default':
Issue #16876: Revert be8e6b81284e, which wasn't thread-safe: wait until a
http://hg.python.org/cpython/rev/30eb98c8afef

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com