Re: Multiprocessing on a remote host

2018-06-10 Thread Peter J. Holzer 
On 2018-03-21 09:27:37 -0400, Larry Martell wrote:
> Yeah, I saw that and I wasn't trying to reinvent the wheel. On this
> page https://docs.python.org/2/library/multiprocessing.html it says
> this:
> 
> The multiprocessing package offers both local and remote concurrency,
> effectively side-stepping the Global Interpreter Lock by using
> subprocesses instead of threads. Due to this, the multiprocessing
> module allows the programmer to fully leverage multiple processors on
> a given machine.
> 
> I took 'remote concurrency' to mean I could use it run a process on
> another host.

Yes.

> But I don't see how to do that, and I was asking if it was possible or
> am I misinterpreting the docs.

It is described on the same page, a little bit farther down:
https://docs.python.org/2/library/multiprocessing.html#using-a-remote-manager

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing on a remote host

2018-03-21 Thread Larry Martell 
On Tue, Mar 20, 2018 at 11:15 PM, Steven D'Aprano
 wrote:
> On Wed, 21 Mar 2018 02:20:16 +, Larry Martell wrote:
>
>> Is there a way to use the multiprocessing lib to run a job on a remote
>> host?
>
> Don't try to re-invent the wheel. This is a solved problem.
>
> https://stackoverflow.com/questions/1879971/what-is-the-current-choice-for-doing-rpc-in-python
>
> I've used both rpyc and pyro, they work fine and are easy to learn.

Yeah, I saw that and I wasn't trying to reinvent the wheel. On this
page https://docs.python.org/2/library/multiprocessing.html it says
this:

The multiprocessing package offers both local and remote concurrency,
effectively side-stepping the Global Interpreter Lock by using
subprocesses instead of threads. Due to this, the multiprocessing
module allows the programmer to fully leverage multiple processors on
a given machine.

I took 'remote concurrency' to mean I could use it run a process on
another host. But I don't see how to do that, and I was asking if it
was possible or am I misinterpreting the docs.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Multiprocessing on a remote host

2018-03-20 Thread Steven D'Aprano 
On Wed, 21 Mar 2018 02:20:16 +, Larry Martell wrote:

> Is there a way to use the multiprocessing lib to run a job on a remote
> host?

Don't try to re-invent the wheel. This is a solved problem.

https://stackoverflow.com/questions/1879971/what-is-the-current-choice-for-doing-rpc-in-python

I've used both rpyc and pyro, they work fine and are easy to learn.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Multiprocessing on a remote host

2018-03-20 Thread Larry Martell 
Is there a way to use the multiprocessing lib to run a job on a remote
host?
-- 
https://mail.python.org/mailman/listinfo/python-list