This actually calls for a dependency definition of notes within a notebook, so the scheduler can decide which tasks to run simultaneously. I suggest a simple counter of dependency levels, which by default increases with every new note and can be decremented to allow notes to run simultaneously. Run-all then submits each level into the target interpreters for this level, awaits termination of all results, and then starts the next level's note.
On Thu, Apr 6, 2017 at 12:57 AM, moon soo Lee <m...@apache.org> wrote: > Hi, > > That's expected behavior at the moment. The reason is > > Each interpreter has it's own scheduler (either FIFO, Parallel), and > run-all just submit all paragraphs into target interpreter's scheduler. > > I think we can add feature such as run-all-sequentially. > Do you mind file a JIRA issue? > > Thanks, > moon > > On Thu, Apr 6, 2017 at 5:35 AM <murexconsult...@googlemail.com> wrote: > >> I often have notebooks that have a %sh as the 1st paragraph. This scps >> some file from another server, and then a number of spark or sparksql >> paragraphs are after that. >> >> If I click on the run-all paragraphs at the top of the notebook the 1st >> %sh paragraph kicks off as expected, but the 2nd %spark notebook starts too >> at the same time. The others go into pending state and then start once the >> spark one has completed. >> >> Is this a bug? Or am I doing something wrong? >> >> Thanks >> >>
