From: Jean-Baptiste Marquette <marqu...@iap.fr>
To: Discussion of Numerical Python <numpy-discussion@scipy.org>
Sent: Wednesday, December 7, 2011 4:23 PM
Subject: Re: [Numpy-discussion] Simple way to launch python processes?
You should consider the powerful multiprocessing package. Have a look on this
piece of code:
importglob
importos
import multiprocessing as multi
import subprocess as sub
importtime
NPROC = 4
Python = '/Library/Frameworks/EPD64.framework/Versions/Current/bin/python'
Xterm = '/usr/X11/bin/xterm '
coord = []
Size = '100x10'
XPos = 810
YPos = 170
XOffset = 0
YOffset = 0
for i in range(NPROC):
if i % 2 == 0:
coord.append(Size + '+' + str(YPos) + '+' + str(YOffset))
else:
coord.append(Size + '+' + str(XPos) + '+' + str(YOffset))
YOffset = YOffset + YPos
def CompareColourRef(Champ):
BaseChamp = os.path.basename(Champ)
NameProc = int(multi.current_process().name[-1]) - 1
print 'Processing', BaseChamp, 'on processor', NameProc+1
os.putenv('ADAM_USER', DirWrk + 'adam_' + str(NameProc+1))
Command = Xterm + '-geometry ' + '"' + coord[NameProc] + '" -T " Proc' +
str(NameProc+1) + ' ' + BaseChamp + ' ' + '" -e " ' + Python + ' ' + DirSrc + \
'CompareColourRef.py ' + BaseChamp + ' 2>&1 | tee' + DirLog + BaseChamp
+ '.log"'
Process = sub.Popen([Command], shell=True)
Process.wait()
print BaseChamp, 'processed on processor', NameProc+1
return
pool = multi.Pool(processes=NPROC)
Champs = glob.glob(DirImg + '*/*')
results = pool.map_async(CompareColourRef, Champs)
pool.close()
while results._number_left > 0:
print"Waiting for", results._number_left, 'tasks to complete'
time.sleep(15)
pool.join()
print'Process completed'
exit(0)
Cheers
Jean-Baptiste
----------------------------------------------------------------------------------------------------------------------------------
Wow. I will have to digest that, but thank you.
-- Lou Pecora, my views are my own.
________________________________
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion