Re: Parallel python in the cloud

2014-05-24 Thread Robert Kern
Mutlyvac but so far, we are struggling to accomplish the same thing we had on PiCloud. I have several "pieces" of my puzzle but can't seem to be able to put it together. I've seen and tried StarCluster and also various parallel python options but all options seem challenging t

Parallel python in the cloud

2014-05-23 Thread Charles Gagnon
accomplish the same thing we had on PiCloud. I have several "pieces" of my puzzle but can't seem to be able to put it together. I've seen and tried StarCluster and also various parallel python options but all options seem challenging to put together. The goal is to mimic PiCloud

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-27 Thread Jurko Gospodnetić
Hi. On 25.11.2013. 14:42, Jurko Gospodnetić wrote: So far all tests seem to indicate that things work out fine if we install to some dummy target folder, copy the target folder to some version specific location & uninstall. That leaves us with a working Python folder sans the start menu and

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Jurko Gospodnetić
Hi. On 25.11.2013. 17:38, Terry Reedy wrote: So far all tests seem to indicate that things work out fine if we install to some dummy target folder, copy the target folder to some version specific location & uninstall. If the dummy folder had 3.3.0, you should not need to uninstall to inst

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Terry Reedy
On 11/25/2013 8:42 AM, Jurko Gospodnetić wrote: So far all tests seem to indicate that things work out fine if we install to some dummy target folder, copy the target folder to some version specific location & uninstall. If the dummy folder had 3.3.0, you should not need to uninstall to in

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Jurko Gospodnetić
Hi. On 25.11.2013. 15:15, Albert-Jan Roskam wrote: > Are you sure? http://stackoverflow.com/questions/1534210/use-different-python-version-with-virtualenv Yup, I'm pretty sure by now (based on reading the docs, not trying it out though). Virtualenv allows you to set up differen

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Chris Angelico
On Tue, Nov 26, 2013 at 1:15 AM, Albert-Jan Roskam wrote: > Below is a little terminal session. I often switch between python 3.3 and > python 2.7. My virtualenv for python 3.3 is called "python33". "workon" is a > virtualenv wrapper command. And check out the envlist in tox.ini on > http://to

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Albert-Jan Roskam
On Mon, 11/25/13, Jurko Gospodnetić wrote: Subject: Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine To: python-list@python.org Date: Monday, November 25, 2013, 2:57 PM   Hi. On 25.11.2013. 14:20, Albert-Jan

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Jurko Gospodnetić
Hi. On 25.11.2013. 14:20, Albert-Jan Roskam wrote: Check out the following packages: virtualenv, virtualenvwrapper, tox virtualenv + wrapper make it very easy to switch from one python version to another. Stricly speaking you don't need virtualenvwrapper, but it makes working with virtualenv a

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Jurko Gospodnetić
Hi. On 25.11.2013. 13:46, Ned Batchelder wrote: IIRC, Python itself doesn't read those registry entries, except when installing pre-compiled .msi or .exe kits. Once you have Python installed, you can move the directory someplace else, then install another version of Python. If you need to

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Chris Angelico
On Tue, Nov 26, 2013 at 12:42 AM, Jurko Gospodnetić wrote: > Yup, we could do that, but at first glance it really smells like an > overkill. Not to mention the potential licensing issues with Windows and an > unlimited number of Windows installations. :-) Ah, heh... didn't think of that. When I

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Jurko Gospodnetić
Hi. On 25.11.2013. 14:04, Chris Angelico wrote: Is it possible to set up virtualization to help you out? Create a virtual machine in something like VirtualBox, then clone it for every Python patch you want to support (you could have one VM that handles all the .0 releases and another that hand

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Albert-Jan Roskam
On Mon, 11/25/13, Jurko Gospodnetić wrote: Subject: Parallel Python x.y.A and x.y.B installations on a single Windows machine To: python-list@python.org Date: Monday, November 25, 2013, 1:32 PM   Hi all.   I was wondering what is the best

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Chris Angelico
On Mon, Nov 25, 2013 at 11:32 PM, Jurko Gospodnetić wrote: > Most of the advice seems to boil down to 'do not use such versions together, > use only the latest'. > > We would like to run automated tests on one of our projects (packaged as a > Python library) with different Python versions, and s

Re: Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Ned Batchelder
On Monday, November 25, 2013 7:32:30 AM UTC-5, Jurko Gospodnetić wrote: > Hi all. > >I was wondering what is the best way to install multiple Python > installations on a single Windows machine. > >Regular Windows installer works great as long as all your > installations have a separate

Parallel Python x.y.A and x.y.B installations on a single Windows machine

2013-11-25 Thread Jurko Gospodnetić
Hi all. I was wondering what is the best way to install multiple Python installations on a single Windows machine. Regular Windows installer works great as long as all your installations have a separate major.minor version identifier. However, if you want to have let's say 2.4.3 & 2.4.

Re: Parallel python + ??

2008-06-11 Thread Thor
Gerhard Häring wrote: > This is of course OS-specific. On Linux, you can parse the proc > filesystem: > > >>> open("/proc/%i/stat" % os.getpid()).read().split()[39] > > You can use the "taskset" utility to query or set CPU affinity on Linux. > It is going to be in Linux (mainly) I was thinking

Re: Parallel python + ??

2008-06-11 Thread Gerhard Häring
Thor wrote: Hi, I am running a program using Parallel Python and I wonder if there is a way/module to know in which CPU/core the process is running in. Is that possible? This is of course OS-specific. On Linux, you can parse the proc filesystem: >>> open("/proc/%i/stat"

Parallel python + ??

2008-06-11 Thread Thor
Hi, I am running a program using Parallel Python and I wonder if there is a way/module to know in which CPU/core the process is running in. Is that possible? Ángel -- http://mail.python.org/mailman/listinfo/python-list

RE: Parallel Python environments..

2007-11-07 Thread Thorsten Kampe
* bruce (Tue, 6 Nov 2007 13:43:10 -0800) > if i have python 2.4.3 installed, it gets placed in the python2.4 dir.. if i > don't do anything different, and install python 2.4.2, it too will get > placed in the python2.4 tree... which is not what i want. > > i'm running rhel4/5... So you're using r

RE: Parallel Python environments..

2007-11-06 Thread bruce
r 06, 2007 2:07 PM To: python-list@python.org Subject: Re: Parallel Python environments.. En Tue, 06 Nov 2007 18:43:10 -0300, bruce <[EMAIL PROTECTED]> escribió: > if i have python 2.4.3 installed, it gets placed in the python2.4 dir.. > if i > don't do anything different, an

Re: Parallel Python environments..

2007-11-06 Thread Gabriel Genellina
En Tue, 06 Nov 2007 18:43:10 -0300, bruce <[EMAIL PROTECTED]> escribió: > if i have python 2.4.3 installed, it gets placed in the python2.4 dir.. > if i > don't do anything different, and install python 2.4.2, it too will get > placed in the python2.4 tree... which is not what i want. Any rea

RE: Parallel Python environments..

2007-11-06 Thread bruce
: Parallel Python environments.. * bruce (Tue, 6 Nov 2007 07:13:43 -0800) > If I wanted to be able to build/test/use parallel python versions, what > would I need to do/set (paths/libs/etc...) nothing > and where would I need to place the 2nd python version, so as not to > screw up my

RE: Parallel Python environments..

2007-11-06 Thread bruce
i'm running rhel... so there isn't a python-config script as far as i know.. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of [EMAIL PROTECTED] Sent: Tuesday, November 06, 2007 8:26 AM To: python-list@python.org Subject: Re: Parallel Python en

Re: Parallel Python environments..

2007-11-06 Thread Diez B. Roggisch
bruce wrote: > Hi.. > > If I wanted to be able to build/test/use parallel python versions, what > would I need to do/set (paths/libs/etc...) and where would I need to place > the 2nd python version, so as not to screw up my initial python dev env. > > I'd like to be

Re: Parallel Python environments..

2007-11-06 Thread [EMAIL PROTECTED]
In Gentoo Linux you can select between installed python version using python-config script. -- http://mail.python.org/mailman/listinfo/python-list

Re: Parallel Python environments..

2007-11-06 Thread Thorsten Kampe
* bruce (Tue, 6 Nov 2007 07:13:43 -0800) > If I wanted to be able to build/test/use parallel python versions, what > would I need to do/set (paths/libs/etc...) nothing > and where would I need to place the 2nd python version, so as not to > screw up my initial python dev env. Anywh

Parallel Python environments..

2007-11-06 Thread bruce
Hi.. If I wanted to be able to build/test/use parallel python versions, what would I need to do/set (paths/libs/etc...) and where would I need to place the 2nd python version, so as not to screw up my initial python dev env. I'd like to be able to switch back/forth between the different ver

Re: Can Parallel Python run on a muti-CPU server ?

2007-02-09 Thread parallelpython
Hi, That is definitely possible! To achieve the best performance split your calculation either into 128 equal parts or int >>128 part of any size (then load balancing will spread workload equally). Let us know the results, if need any help with parallelization feel free to request it here: http://

Re: Can Parallel Python run on a muti-CPU server ?

2007-02-08 Thread azrael
no, not renting. i need such one at home. when you say rent it sounds like buy a hosting package. i need one to work all the time on max power. cracking md5 needs power. :-D -- http://mail.python.org/mailman/listinfo/python-list

Re: Can Parallel Python run on a muti-CPU server ?

2007-02-07 Thread Martin P. Hellwig
azrael wrote: > On Feb 7, 3:13 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> > wrote: >> Hi all, >> >> I'm interested in Parallel Python and I learned from the website of >> Parallel Python >> that it can run on SMP and clusters. But can

Re: Can Parallel Python run on a muti-CPU server ?

2007-02-07 Thread azrael
On Feb 7, 3:13 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Hi all, > > I'm interested in Parallel Python and I learned from the website of > Parallel Python > that it can run on SMP and clusters. But can it run on a our muti-CPU > server ? > We

Re: Can Parallel Python run on a muti-CPU server ?

2007-02-07 Thread Martin P. Hellwig
[EMAIL PROTECTED] wrote: > [EMAIL PROTECTED] wrote: >> Hi all, >> >> I'm interested in Parallel Python and I learned from the website of >> Parallel Python >> that it can run on SMP and clusters. But can it run on a our muti-CPU >> server ? >>

Re: Can Parallel Python run on a muti-CPU server ?

2007-02-07 Thread Nick Vatamaniuc
Nick On Feb 6, 9:13 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Hi all, > > I'm interested in Parallel Python and I learned from the website of > Parallel Python > that it can run on SMP and clusters. But can it run on a our muti-CPU > server ?

Re: Can Parallel Python run on a muti-CPU server ?

2007-02-06 Thread [EMAIL PROTECTED]
[EMAIL PROTECTED] wrote: > Hi all, > > I'm interested in Parallel Python and I learned from the website of > Parallel Python > that it can run on SMP and clusters. But can it run on a our muti-CPU > server ? > We are running an origin3800 server with 128 CPUs. > >

Can Parallel Python run on a muti-CPU server ?

2007-02-06 Thread [EMAIL PROTECTED]
Hi all, I'm interested in Parallel Python and I learned from the website of Parallel Python that it can run on SMP and clusters. But can it run on a our muti-CPU server ? We are running an origin3800 server with 128 CPUs. Thanks. -- http://mail.python.org/mailman/listinfo/python-list

Re: Parallel Python

2007-02-04 Thread parallelpython
ut is there any way to use this for a cluster of > machines over a network (not smp)? There are 2 major updates regarding Parallel Python: http:// www.parallelpython.com 1) Now (since version 1.2) parallel python software could be used for cluster-wide parallelization (or even Internet-wide).

Re: Distributed computation of jobs (was: Parallel Python)

2007-01-17 Thread Paul Boddie
A.T.Hofkamp skrev: > > Let me add a few cents to the discussion with this announcement: [Notes about exec_proxy, batchlib and rthread] I've added entries for these modules, along with py.execnet, to the parallel processing solutions page on the python.org Wiki: http://wiki.python.org/moin/Parall

Distributed computation of jobs (was: Parallel Python)

2007-01-17 Thread A.T.Hofkamp
On 2007-01-12, robert <[EMAIL PROTECTED]> wrote: >> >> [1] http://www.python.org/pypi/parallel > > I'd be interested in an overview. > For ease of use a major criterion for me would be a pure python > solution, which also does the job of starting and controlling the > other process(es) automatic

Re: Parallel Python

2007-01-13 Thread parallelpython
> Looks interesting, but is there any way to use this for a cluster of > machines over a network (not smp)? Networking capabilities will be included in the next release of Parallel Python software (http://www.parallelpython.com), which is coming soon. > Couldn't you just p

Re: Parallel Python

2007-01-12 Thread mheslep
Konrad Hinsen wrote: Perhaps we should start a > special interest group? Not so much in order to work on a single > project; I believe that at the current state of parallel computing we > still need many different approaches to be tried. But an exchange of > experience could well be useful fo

Re: Parallel Python

2007-01-12 Thread Konrad Hinsen
On Jan 12, 2007, at 15:08, Paul Boddie wrote: > It seems to me that a more useful first step would be to create an > overview of the different modules and put it on the python.org Wiki: > > http://wiki.python.org/moin/FrontPage > http://wiki.python.org/moin/UsefulModules (a reasonable entry point)

Re: Parallel Python

2007-01-12 Thread Paul Boddie
Konrad Hinsen wrote: > > That's one more project... It seems that there is significant > interest in parallel computing in Python. Perhaps we should start a > special interest group? Not so much in order to work on a single > project; I believe that at the current state of parallel computing we > s

Re: Parallel Python

2007-01-12 Thread Konrad Hinsen
On Jan 12, 2007, at 11:21, Paul Boddie wrote: > done. My own experience with regard to releasing software is that even > with an open source licence, most people are likely to ignore your > projects than to suddenly jump on board and take control, and even if My experience is exactly the same. An

Re: Parallel Python

2007-01-12 Thread Paul Boddie
robert wrote: > Paul Boddie wrote: > > > > [1] http://www.python.org/pypi/parallel > > I'd be interested in an overview. I think we've briefly discussed the above solution before, and I don't think you're too enthusiastic about anything using interprocess communication, which is what the above sol

Re: Parallel Python

2007-01-12 Thread Neal Becker
[EMAIL PROTECTED] wrote: > Has anybody tried to run parallel python applications? > It appears that if your application is computation-bound using 'thread' > or 'threading' modules will not get you any speedup. That is because > python interpreter uses GIL(Globa

Re: Parallel Python

2007-01-12 Thread robert
Paul Boddie wrote: > [EMAIL PROTECTED] wrote: >> The main difference between MPI python solutions and ppsmp is that with >> MPI you have to organize both computations >> {MPI_Comm_rank(MPI_COMM_WORLD, &id); if id==1 then ... else } and >> data distribution (MPI_Send / MPI_Recv) by yourself. Whi

Re: Parallel Python

2007-01-12 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, "Paul Boddie" <[EMAIL PROTECTED]> writes: |> [EMAIL PROTECTED] wrote: |> > |> > The main difference between MPI python solutions and ppsmp is that with |> > MPI you have to organize both computations |> > {MPI_Comm_rank(MPI_COMM_WORLD, &id); if id==1 then ... else .

Re: Parallel Python

2007-01-12 Thread Paul Boddie
[EMAIL PROTECTED] wrote: > > The main difference between MPI python solutions and ppsmp is that with > MPI you have to organize both computations > {MPI_Comm_rank(MPI_COMM_WORLD, &id); if id==1 then ... else } and > data distribution (MPI_Send / MPI_Recv) by yourself. While with ppsmp > you jus

Re: Parallel Python

2007-01-11 Thread jairodsl
t; > > > 1 file/database based; multiple batch jobs > > 2 Message Passing, IPC, RPC, ... > > 3 Object Sharing > > 4 Sharing of global data space (Threads) > > 5 Local parallelism / Vector computing, MMX, 3DNow,... > > > > There are good reasons for all of

Re: Parallel Python

2007-01-11 Thread parallelpython
re are good reasons for all of these levels. > Yet "parallel python" to me fakes to be on level 3 or 4 (or even 5 :-) ), > while its just a level 2 > system, where "passing", "remote", "inter-process" ... are the right vocables. In one of the previou

Re: Parallel Python

2007-01-11 Thread parallelpython
termined by the slowest running process (the one which shares processor with another running program). In this particular case ppsmp will outperform MPI. The third, probably less important, difference is that with MPI based parallel python code you must have MPI installed in the system. Overall pp

Re: Parallel Python

2007-01-11 Thread Konrad Hinsen
On Jan 8, 2007, at 11:33, Duncan Booth wrote: > The 'parallel python' site seems very sparse on the details of how > it is > implemented but it looks like all it is doing is spawning some > subprocesses > and using some simple ipc to pass details of the calls and re

Re: Parallel Python

2007-01-11 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, robert <[EMAIL PROTECTED]> writes: |> |> Thus there are different levels of parallelization: |> |> 1 file/database based; multiple batch jobs |> 2 Message Passing, IPC, RPC, ... |> 3 Object Sharing |> 4 Sharing of global data space (Threads) |> 5 Local parallelis

Re: Parallel Python

2007-01-11 Thread robert
t; and debug than thread code. The OP used a similar technique in his > 'parallel python' project. Thus there are different levels of parallelization: 1 file/database based; multiple batch jobs 2 Message Passing, IPC, RPC, ... 3 Object Sharing 4 Sharing of global data space (Threads)

Re: Parallel Python

2007-01-11 Thread Sergei Organov
[EMAIL PROTECTED] (Nick Maclaren) writes: [...] > I mean precisely the first. > > The C99 standard uses a bizarre consistency model, which requires serial > execution, and its consistency is defined in terms of only volatile > objects and external I/O. Any form of memory access, signalling or > wh

Re: Parallel Python

2007-01-11 Thread sturlamolden
technique in his 'parallel python' project. This does not mean that MPI is inherently slower than threads however, as there are overhead associated with thread synchronization as well. With 'shared memory' between threads, a lot more fine grained synchronization ans scheduling

Re: Parallel Python

2007-01-11 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, robert <[EMAIL PROTECTED]> writes: |> |> Most threads on this planet are not used for number crunching jobs, |> but for "organization of execution". That is true, and it is effectively what POSIX and Microsoft threads are suitable for. With reservations, even the

Re: Parallel Python

2007-01-11 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, Sergei Organov <[EMAIL PROTECTED]> writes: |> |> OK, then I don't think the POSIX threads were "perpetrated" to be idle |> most of time. Perhaps I was being unclear. I should have added "In the case where there are more threads per system than CPUs per system".

Re: Parallel Python

2007-01-11 Thread Sergei Organov
[EMAIL PROTECTED] (Nick Maclaren) writes: > In article <[EMAIL PROTECTED]>, > Sergei Organov <[EMAIL PROTECTED]> writes: > |> > |> Do you mean that POSIX threads are inherently designed and implemented > |> to stay idle most of the time?! If so, I'm afraid those guys that > |> designed POSIX thre

Re: Parallel Python

2007-01-11 Thread robert
Nick Maclaren wrote: > In article <[EMAIL PROTECTED]>, > Paul Rubin writes: > |> > |> > Yes, I know that it is a bit Irish for the best way to use a shared > |> > memory system to be to not share memory, but that's how it is. > |> > |> But I thought serious MPI implement

Re: Parallel Python

2007-01-11 Thread robert
sturlamolden wrote: > Nick Maclaren wrote: > > I wonder if too much emphasis is put on thread programming these days. > Threads may be nice for programming web servers and the like, but not > for numerical computing. Reading books about thread programming, one > can easily get the impression that

Re: Parallel Python

2007-01-10 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, "Carl J. Van Arsdall" <[EMAIL PROTECTED]> writes: |> |> Just as something to note, but many HPC applications will use a |> combination of both MPI and threading (OpenMP usually, as for the |> underlying thread implementation i don't have much to say). Its |> in

Re: Parallel Python

2007-01-10 Thread Carl J. Van Arsdall
Just as something to note, but many HPC applications will use a combination of both MPI and threading (OpenMP usually, as for the underlying thread implementation i don't have much to say). Its interesting to see on this message board this huge "anti-threading" mindset, but the HPC community s

Re: Parallel Python

2007-01-10 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, Sergei Organov <[EMAIL PROTECTED]> writes: |> |> Do you mean that POSIX threads are inherently designed and implemented |> to stay idle most of the time?! If so, I'm afraid those guys that |> designed POSIX threads won't agree with you. In particular, as far as I |

Re: Parallel Python

2007-01-10 Thread Sergei Organov
[EMAIL PROTECTED] (Nick Maclaren) writes: > In article <[EMAIL PROTECTED]>, > "sturlamolden" <[EMAIL PROTECTED]> writes: [...] > |> I wonder if too much emphasis is put on thread programming these days. > |> Threads may be nice for programming web servers and the like, but not > |> for numerical co

Re: Parallel Python

2007-01-10 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, Paul Rubin writes: |> |> > Yes, I know that it is a bit Irish for the best way to use a shared |> > memory system to be to not share memory, but that's how it is. |> |> But I thought serious MPI implementations use shared memory if they |

Re: Parallel Python

2007-01-10 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, "sturlamolden" <[EMAIL PROTECTED]> writes: |> |> In any case, this means that Python can happily keep its GIL, as the |> CPU bound 'HPC' tasks for which the GIL does matter should be done |> using multiple processes (not threads) anyway. That leaves threads as a |>

Re: Parallel Python

2007-01-10 Thread Paul Rubin
[EMAIL PROTECTED] (Nick Maclaren) writes: > Yes, I know that it is a bit Irish for the best way to use a shared > memory system to be to not share memory, but that's how it is. But I thought serious MPI implementations use shared memory if they can. That's the beauty of it, you can run your appli

Re: Parallel Python

2007-01-10 Thread sturlamolden
Nick Maclaren wrote: > as the ones that you have to play for threaded programs. Yes, I know > that it is a bit Irish for the best way to use a shared memory system > to be to not share memory, but that's how it is. Thank you for clearing that up. In any case, this means that Python can happily

Re: Parallel Python

2007-01-10 Thread Nick Maclaren
In article <[EMAIL PROTECTED]>, "sturlamolden" <[EMAIL PROTECTED]> writes: |> |> MPI is becoming the de facto standard for high-performance parallel |> computing, both on shared memory systems (SMPs) and clusters. It has been for some time, and is still gaining ground. |> Spawning |> threads or

Re: Parallel Python

2007-01-10 Thread sturlamolden
[EMAIL PROTECTED] wrote: >That's right. ppsmp starts multiple interpreters in separate > processes and organize communication between them through IPC. Thus you are basically reinventing MPI. http://mpi4py.scipy.org/ http://en.wikipedia.org/wiki/Message_Passing_Interface -- http://mail.p

Re: Parallel Python

2007-01-10 Thread sturlamolden
robert wrote: > Thats true. IPC through sockets or (somewhat faster) shared memory - cPickle > at least - is usually the maximum of such approaches. > See > http://groups.google.de/group/comp.lang.python/browse_frm/thread/f822ec289f30b26a > > For tasks really requiring threading one can consid

Re: Parallel Python

2007-01-10 Thread parallelpython
> I always thought that if you use multiple processes (e.g. os.fork) then > Python can take advantage of multiple processors. I think the GIL locks > one processor only. The problem is that one interpreted can be run on > one processor only. Am I not right? Is your ppm module runs the same > interp

Re: Parallel Python

2007-01-08 Thread Parallel Python The team
On 1/8/07, Laszlo Nagy <[EMAIL PROTECTED]> wrote: I always thought that if you use multiple processes (e.g. os.fork) then Python can take advantage of multiple processors. I think the GIL locks one processor only. The problem is that one interpreted can be run on one processor only. Am I not rig

Re: Parallel Python

2007-01-08 Thread robert
Duncan Booth wrote: > Laszlo Nagy <[EMAIL PROTECTED]> wrote: > > > The 'parallel python' site seems very sparse on the details of how it is > implemented but it looks like all it is doing is spawning some subprocesses > and using some simple ipc to pass d

Re: Parallel Python

2007-01-08 Thread Duncan Booth
Laszlo Nagy <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: >> Has anybody tried to run parallel python applications? >> It appears that if your application is computation-bound using 'thread' >> or 'threading' modules will not get you any spee

Re: Parallel Python

2007-01-08 Thread Laszlo Nagy
[EMAIL PROTECTED] wrote: > Has anybody tried to run parallel python applications? > It appears that if your application is computation-bound using 'thread' > or 'threading' modules will not get you any speedup. That is because > python interpreter uses GIL(Globa

Parallel Python

2007-01-06 Thread parallelpython
Has anybody tried to run parallel python applications? It appears that if your application is computation-bound using 'thread' or 'threading' modules will not get you any speedup. That is because python interpreter uses GIL(Global Interpreter Lock) for internal bookkeeping. T

Re: Parallel Python on PowerMac?

2004-11-29 Thread Robert Kern
Alan Kennedy wrote: Although, iff your prospective machine supports System V IPC, you might want to check out PoSH. http://poshmodule.sourceforge.net It uses inline assembly, so that's a no-go on the PPC unless someone ports the assembly code. -- Robert Kern [EMAIL PROTECTED] "In the fields of

POSH on PowerMac? (was: Parallel Python on PowerMac?)

2004-11-29 Thread Wolfgang Keller
>> Although, iff your prospective machine supports System V IPC, > > No clue whether MacOS X does so. [EMAIL PROTECTED]&! Read before you post. According to "Mac OS X for UNIX Users" there's a compatibility library included in Mac OS X that does so. Now who's got a the tools and the know-how to

Re: Parallel Python on PowerMac?

2004-11-29 Thread Wolfgang Keller
Hello, and thanks for your reply. > But I would venture to say that, *in the general case*, the "most > efficient" way to benefit from a second cpu, both in terms of coding > time and execution efficiency, is to use either jython *cough* *choke* Err, no, sorry, not for me. > Although, iff yo

Re: Parallel Python on PowerMac?

2004-11-29 Thread Alan Kennedy
[Wolfgang Keller] > as I might get a dual-G5 PowerMac someday in the not to distant > future, I was wondering what options are available for making Python > benefit from the second CPU? Running two interpreters and using Pyro > would not be the most efficient (and easiest) way, I guess? Qualifier:

Parallel Python on PowerMac?

2004-11-29 Thread Wolfgang Keller
Hello, as I might get a dual-G5 PowerMac someday in the not to distant future, I was wondering what options are available for making Python benefit from the second CPU? Running two interpreters and using Pyro would not be the most efficient (and easiest) way, I guess? TIA, Best regards Wolfgang