On Jul 22, 11:18 am, William Stein <wst...@gmail.com> wrote:
> On Wed, Jul 22, 2009 at 11:12 AM, ghtdak<gl...@tarbox.org> wrote:
>
> > On Jul 21, 6:40 pm, "Dr. David Kirkby" <david.kir...@onetel.net>
> > wrote:
> >> ghtdak wrote:
>
> >> > This thread has gotten long and there are many subjects embedded
> >> > within.
>
> >> > One of the problems I've had with the notebook implementation is that
> >> > the sage process supporting the notebook computation blocks on the
> >> > pipe between itself and the twistd server which spawns it. This means
> >> > that one can't build an asynchronous event handler without using
> >> > threads.
>
> >> > More conventional web server architectures use a callback style for
> >> > invocation making it much easier to splice in other events for
> >> > handling by the main thread (this is the general asynchronous
> >> > programming model and the heart of how Twisted works)
>
> >> > Perhaps it is a foregone conclusion that this approach will be taken
> >> > in the rewrite. if not, I'd like to suggest that it is an important
> >> > consideration.
>
> >> > -glenn
>
> >> I don't know about how the web server is implemented. I know it did not
> >> work on my Solaris box, but that is another matter.
>
> >> But actually including Apache might be a sensible choice. A lot of
> >> people know how to administer Apache. It offers a lot of flexibility.
> >> You can for example only serve pages to particular IP addresses.
>
> >> Worth a thought anyway.
>
> > I came across this post from the Twisted folks. It looks like they do
> > WSGI and run Django quite well...
>
> >http://blog.dreid.org/2009/03/twisted-django-it-wont-burn-down-your.html
>
> > At least there are alternatives to apache which might be simpler.
>
> > The core question remains: While the presentation layer support is
> > defined by django, pyjamas etc, the integration with the underlying
> > sage process process is the issue, not the web presentation.
>
> > In the current architecture, a twistd daemon spawns a notebook server
> > which is responsible for doing "sage" stuff. twistd is fully
> > asynchronous, but the notebook process itself is a pexpect based
> > blocking process connected with pipes to twistd. As such, the block
> > on read by pexpect precludes the sage process servicing asynchronous
> > events.
>
> Here's the relevant code that you're talking about (starting at line
> 3007 of worksheet.py):
>
> try:
> done, out, new = S._so_far(wait=wait,
> alternate_prompt=SAGE_END+str(self.synchro()))
> except RuntimeError, msg:
> verbose("Computation was interrupted or failed.
> Restarting.\n%s"%msg)
> self.__comp_is_running = False
> self.start_next_comp()
> return 'w', C
>
> The Sage notebook server does a blocking read from the
> pexpect-controlled subprocess for "wait" seconds. The default value
> of wait is 0.2 seconds. So indeed there is a block, but it is never
> for more than 0.2 seconds. During that time other asynchronous
> events can't be dealt with.
I'm not sure I understand, so let me restate what I think you're
saying:
The notebook "server" is doing a blocking read with a timeout. The
server is a twistd process which multiplexes and processes in support
the browsers.
The "pexpect-controlled subprocess" is where Sage itself is.
The issue I was raising was with the latter (subprocess) although I
now think the server implementation might also be an issue.
when twisted spawns a sub-process, typically one registers a callback
to handle information from that sub-process. There should be no
reason to check for input. Generally a callback for signals is also
registered to sense the sub-process dying, the pipe being disconnected
etc. Unless I misunderstand (very possible), you're "polling" the set
of spawned Sage sub-processes for data.
While that doesn't sound good either, thats not the problem I was
describing.
My primary problem is that the Sage subprocess is blocking forever on
the other side of the pipe when its not computing... Therefore, I
can't have a Sage sub-process that I'm using in the notebook that is
also able to communicate with other processes as I can't
asynchronously receive data (or get timing interrupts). I've gotten
around this in the past by using threads as it was the only choice I
had.
-glenn
>
>
>
> > IMHO, this architecture is incorrect and limited... Perhaps this is
> > part of what is being rethought... if not, I believe it should be.
>
> > A preferable architecture is an event loop which dispatches requests
> > within the sage process. Since Sage is written in python, I would
> > suggest Twisted for this but there might be better alternatives (I'd
> > be surprised, but its possible)
>
> > Using this approach, one could easily add other elements to the core
> > event loop to support asynchronous processing (timers, communication,
> > etc) without threads which are, in this case, unnecessary. Threads
> > when necessary are bad enough, when they're introduced because of
> > unnecessary blocking, one gets all the threading nightmare without any
> > benefit. (reminds me of health care reform)
>
> > Another benefit is that since asynchronous event processing is the
> > widely accepted approach to this type of problem, there are lots of
> > libraries / packages to "make it so".
>
> > -glenn
>
> >> dave
>
> --
> William Stein
> Associate Professor of Mathematics
> University of Washingtonhttp://wstein.org
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to
sage-devel-unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---
