Re: [Twisted-Python] First pre-release of Twisted 11.1.0

2011-10-26 Thread Itamar Turner-Trauring
On Wed, 2011-10-26 at 07:09 -0400, Glyph wrote: > > On Oct 21, 2011, at 10:28 AM, Thomas Hervé wrote: > > > * A special mention to the new abortConnection support on TCP and > > SSL > > connections, heroically pushed by Itamar and Jean-Paul, and the > > oldest ticket closed by this release. >

Re: [Twisted-Python] Docs HTML suggestion/request for moving between versions.

2011-10-26 Thread Michael Tharp
On 10/26/2011 11:29 AM, Phil Mayers wrote: > Obviously it's tricky to force Google to put "current" at the top! But > could we maybe have a little dropdown in the top-right that > shows the version, and lets you pick a new one and have the page reload? Other possibilities: - Add a big yellow "th

Re: [Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread Terry Jones
Hi Jason > I don't see anything in task.Cooperator to limit the # of > simultaneously-running tasks. Am I missing something? I'm not sure, but I think so. It does limit the number of simultaneously-running tasks. > I guess, technically, could write my own scheduler that limits the # of > simul

[Twisted-Python] Docs HTML suggestion/request for moving between versions.

2011-10-26 Thread Phil Mayers
Lately, I've taken to finding Twisted docs by writing: twisted.internet.task.Cooperator ...into the Chrome bar'o'magic and hitting return to get the Google hit. This is great, except that the resultant Google hits almost always point me to the 8.2.0 HTML docs. 2nd hit is (currently) 11.0 docs.

Re: [Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread Jason Rennie
On Wed, Oct 26, 2011 at 11:08 AM, Jason Rennie wrote: > First off, have you read http://jcalderone.livejournal.com/24285.html ? >> If not, do. Take note of the line >> >> work = (callable(elem, *args, **named) for elem in iterable) >> >> work is a generator. Make sure you understand every line

Re: [Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread Phil Mayers
On 26/10/11 16:08, Jason Rennie wrote: > On Wed, Oct 26, 2011 at 10:22 AM, Terry Jones > wrote: > > Sounds like you should be using a Python generator and that you're not. > > > The issue I'm seeing is that the ~million DeferredSemaphore.run() calls > are inefficient; it's

Re: [Twisted-Python] perspective broker remote call returning a dictionary

2011-10-26 Thread exarkun
On 25 Oct, 07:57 pm, txt...@gmail.com wrote: >I'm trying to return a dictionary with objects as values from a remote >call but it's not working. Can anyone tell me what I'm doing wrong or >point me at an example that works? > >dserver.py >

Re: [Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread Jason Rennie
On Wed, Oct 26, 2011 at 10:22 AM, Terry Jones wrote: > Sounds like you should be using a Python generator and that you're not. > The issue I'm seeing is that the ~million DeferredSemaphore.run() calls are inefficient; it's independent of list/generator. First off, have you read http://jcalderon

Re: [Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread Itamar Turner-Trauring
On Wed, 2011-10-26 at 10:02 -0400, Jason Rennie wrote: > The background: > > > I've been using DeferredSemaphore and DeferredList to manage the > running of tasks with a resource constraint (only so many tasks can > run at the same time). This worked great until I tried to use it to > manage mil

Re: [Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread exarkun
On 02:02 pm, jren...@gmail.com wrote: >The background: > >I've been using DeferredSemaphore and DeferredList to manage the >running of >tasks with a resource constraint (only so many tasks can run at the >same >time). This worked great until I tried to use it to manage millions of >tasks. Simpl

Re: [Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread Terry Jones
Hi Jason > - Is there a Twisted pattern for managing tasks efficiently that I be > - missing? Sounds like you should be using a Python generator and that you're not. First off, have you read http://jcalderone.livejournal.com/24285.html ? If not, do. Take note of the line work = (callable

[Twisted-Python] running 1,000,000 tasks, 40 at-a-time

2011-10-26 Thread Jason Rennie
The background: I've been using DeferredSemaphore and DeferredList to manage the running of tasks with a resource constraint (only so many tasks can run at the same time). This worked great until I tried to use it to manage millions of tasks. Simply setting them up to run (DeferredSemaphore.run(

Re: [Twisted-Python] First pre-release of Twisted 11.1.0

2011-10-26 Thread Itamar Turner-Trauring
On Wed, 2011-10-26 at 07:09 -0400, Glyph wrote: > Does anyone have an idea what this could be? I will file a ticket > soon but I'd love to have some reproduction instructions... It seems to be just a normal TLS shutdown, the second stage after a TLS-level disconnect has been both sent and receiv

Re: [Twisted-Python] First pre-release of Twisted 11.1.0

2011-10-26 Thread Glyph
On Oct 21, 2011, at 10:28 AM, Thomas Hervé wrote: > * A special mention to the new abortConnection support on TCP and SSL > connections, heroically pushed by Itamar and Jean-Paul, and the > oldest ticket closed by this release. After doing some testing of my own, I am discovering that there