Yes, we do have built-in dynamic load balancing. Check out the following:
client.TaskClient client.StringTask client.MapTask All of this is dynamically load balanced. Here is how I would approach this: 1. Use MultiEngineClient to setup all the basic imports and variables that all the engines need. 2. Create a TaskClient and then submit many StringTasks/MapTasks to the TaskClient. The only thing to note is that currently, the TaskClient has slightly more overhead than the MultiEngineClient, so you want to make sure that each tasks lasts long enough to be worth it. If your tasks are really short and you are not seeing good speedup, you may want each StringTask/MapTask instance handle a small batch of actual tasks. But all of this should "Just Work" Right now the best documentation for this is in the doc strings: http://ipython.scipy.org/doc/nightly/html/api/generated/IPython.kernel.taskclient.html http://ipython.scipy.org/doc/nightly/html/api/generated/IPython.kernel.task.html#IPython.kernel.task.StringTask http://ipython.scipy.org/doc/nightly/html/api/generated /IPython.kernel.task.html#IPython.kernel.task.MapTask If these task types don't work for you, it is also possible to define new task types. Cheers, Brian On Thu, Mar 19, 2009 at 12:56 AM, Ondrej Certik <ond...@certik.cz> wrote: > >> Now some thoughts about load balancing: > > [...] > >> And finally I collect the results: >> >> http://github.com/certik/sympy/blob/55de47b6f0a7bee01249fc24c03e5567695c4569/t.py#L69 >> >> and report the results to the user. This should of course be > > I forgot to ask the question: > > how could this be made more efficient using some native ipython parrallel > tools? > > Ondrej > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to sympy@googlegroups.com To unsubscribe from this group, send email to sympy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sympy?hl=en -~----------~----~----~----~------~----~------~--~---
