Re: Bi-directional sub-process communication

2015-11-24 Thread Cameron Simpson
On 24Nov2015 14:53, Ian Kelly wrote: On Mon, Nov 23, 2015 at 10:25 PM, Cameron Simpson wrote: Then #3. I would have a common function/method for submitting a request to go to the subprocess, and have that method return an Event on which to wait. Then caller then just waits for the Event and co

Re: Bi-directional sub-process communication

2015-11-24 Thread Ian Kelly
On Mon, Nov 23, 2015 at 10:25 PM, Cameron Simpson wrote: > Then #3. I would have a common function/method for submitting a request to > go to the subprocess, and have that method return an Event on which to wait. > Then caller then just waits for the Event and collects the data. Obviously, > the m

Re: Bi-directional sub-process communication

2015-11-24 Thread Cameron Simpson
On 24Nov2015 06:33, israel wrote: On 11/23/2015 20:29, Cameron Simpson wrote: On 24Nov2015 16:25, Cameron Simpson wrote: Completely untested example code: class ReturnEvent: def __init__(self): self.event = Event() With, of course: def wait(self): return self.event.wait() Of c

Re: Bi-directional sub-process communication

2015-11-24 Thread israel
On 11/23/2015 20:29, Cameron Simpson wrote: On 24Nov2015 16:25, Cameron Simpson wrote: Completely untested example code: class ReturnEvent: def __init__(self): self.event = Event() With, of course: def wait(self): return self.event.wait() Of course :-) Ah, the Event() object

Re: Bi-directional sub-process communication

2015-11-23 Thread Cameron Simpson
On 23Nov2015 14:14, Israel Brewster wrote: On Nov 23, 2015, at 1:43 PM, Chris Kaynor wrote: On Mon, Nov 23, 2015 at 2:18 PM, Israel Brewster wrote: 3. Have an event per thread. This will use less CPU than the second option, however does require more memory and OS resources, and so will no

Re: Bi-directional sub-process communication

2015-11-23 Thread Cameron Simpson
On 24Nov2015 16:25, Cameron Simpson wrote: Completely untested example code: class ReturnEvent: def __init__(self): self.event = Event() With, of course: def wait(self): return self.event.wait() Cheers, Cameron Simpson Maintainer's Motto: If we can't fix it, it ain't broke.

Re: Bi-directional sub-process communication

2015-11-23 Thread Israel Brewster
On Nov 23, 2015, at 3:05 PM, Dennis Lee Bieber wrote: > > On Mon, 23 Nov 2015 08:54:38 -0900, Israel Brewster > declaimed the following: > >> Concern: Since the master process is multi-threaded, it seems likely enough >> that multiple threads on the master side would make requests at the same

Re: Bi-directional sub-process communication

2015-11-23 Thread Israel Brewster
On Nov 23, 2015, at 1:43 PM, Chris Kaynor wrote: > > On Mon, Nov 23, 2015 at 2:18 PM, Israel Brewster > wrote: > >> Of course, that last step could be interesting - implementing the block in >> such a way as to not tie up the processor, while still getting the data "as >> soon" as it is availab

Re: Bi-directional sub-process communication

2015-11-23 Thread Chris Kaynor
On Mon, Nov 23, 2015 at 2:18 PM, Israel Brewster wrote: > Of course, that last step could be interesting - implementing the block in > such a way as to not tie up the processor, while still getting the data "as > soon" as it is available. Unless there is some sort of built-in > notification syste

Re: Bi-directional sub-process communication

2015-11-23 Thread Israel Brewster
On Nov 23, 2015, at 12:45 PM, Cameron Simpson wrote: > > On 23Nov2015 12:22, Israel Brewster wrote: >> On Nov 23, 2015, at 11:51 AM, Ian Kelly wrote: >>> Concurrency, ugh. > > I'm a big concurrency fan myself. > >>> It's probably better just to have a Condition/Event per thread and >>> have t

Re: Bi-directional sub-process communication

2015-11-23 Thread Cameron Simpson
On 23Nov2015 12:22, Israel Brewster wrote: On Nov 23, 2015, at 11:51 AM, Ian Kelly wrote: Concurrency, ugh. I'm a big concurrency fan myself. It's probably better just to have a Condition/Event per thread and have the response thread identify the correct one to notify, rather than just not

Re: Bi-directional sub-process communication

2015-11-23 Thread Israel Brewster
On Nov 23, 2015, at 11:51 AM, Ian Kelly wrote: > > On Mon, Nov 23, 2015 at 12:55 PM, Ian Kelly wrote: >> On Mon, Nov 23, 2015 at 10:54 AM, Israel Brewster >> wrote: >>> Concern: Since the master process is multi-threaded, it seems likely enough >>> that multiple threads on the master side wou

Re: Bi-directional sub-process communication

2015-11-23 Thread Ian Kelly
On Mon, Nov 23, 2015 at 12:55 PM, Ian Kelly wrote: > On Mon, Nov 23, 2015 at 10:54 AM, Israel Brewster > wrote: >> Concern: Since the master process is multi-threaded, it seems likely enough >> that multiple threads on the master side would make requests at the same >> time. I understand that

Re: Bi-directional sub-process communication

2015-11-23 Thread Ian Kelly
On Mon, Nov 23, 2015 at 10:54 AM, Israel Brewster wrote: > Concern: Since the master process is multi-threaded, it seems likely enough > that multiple threads on the master side would make requests at the same > time. I understand that the Queue class has locks that make this fine (one > thread

Bi-directional sub-process communication

2015-11-23 Thread Israel Brewster
I have a multi-threaded python app (CherryPy WebApp to be exact) that launches a child process that it then needs to communicate with bi-driectionally. To implement this, I have used a pair of Queues: a child_queue which I use for master->child communication, and a master_queue which is used for