On Wed, 19 Aug 2009, William Stein wrote: > > 2009/8/19 Robert Bradshaw <rober...@math.washington.edu>: >> >> On Wed, 19 Aug 2009, William Stein wrote: >> >>> >>> On Wed, Aug 19, 2009 at 1:19 AM, Robert >>> Bradshaw<rober...@math.washington.edu> wrote: >>>> Multithreading will have to be introduced at one >>>> level or another to scale the notebook to more than a single >>>> processor anyways. >>>> >>>> - Robert >>> >>> Huh? Why? I don't see any need for multithreading to solve the >>> above problem, or rather I don't understand what problem you're >>> talking about. The notebook already scales to more than a single >>> processor. >> >> I am talking about the case where there are enough users that the notebook >> process itself becomes the bottleneck. It all depends on how lightweight >> the shuffling data between the underlying processes and the browser is, >> and how many concurrent users one wants to support for a single notebook. >> In the asynchronous model there is only one thread handling all of the >> connections. (Also, anything long-running, e.g. taring up all a users >> worksheets for download, needs to spawn a separate thread/process.) >> >> Of course if the whole setup is running on a single machine, it may be >> that the computational processes are always the bottleneck. > > Thanks for the clarification, which makes sense. There are other > approaches. If one had tens of thousands of simultaneous users, > instead of having multiple threads one could assign users to a > separate process (that could handle up to n users max) when they first > connect. That could scale better than SMP threads, since it is > easier to distribute the load across servers. Maybe this is just > orthogonal though.
Yep. If the worksheet data was backed by a (synchronzed) database then this would work well. > Regarding the tar example, one solution might be to run it as a > separate process, then later check if that process finished -- it is > not necessary for the notebook server process to wait for a subprocess > doing tar to finish before continuing. Another possibility if the > tar'ing happens in the same process would be to use fork. The point is that you have to do this manually, rather than just letting that thread block for a while. (I think the right asynchronous way to do it would be to set up a callback for when it's done, rather than repeatedly coming back to check on it.) > For fun, I just looked at Activity Monitor on OS X, and sorted the > tasks I'm running by number of threads. The top is Firefox and the > bottom is python. Not surprising. Also, I forgot about the GIL, which truely limits the performance benifits of threading in Python. If anything ever kills Python, I bet it'll be the GIL (but I'm hopeful that it'll get removed before it causes an untimely death...) - Robert --~--~---------~--~----~------------~-------~--~----~ To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://www.sagemath.org -~----------~----~----~----~------~----~------~--~---
