Here's a further modified version. In this case, when using the pool for the first time, it uses an n of 10, instead of 100 million. Even with such a low precision, the first execution takes 1.3 seconds. It seems some significant warm up time is needed the first time a multiprocessing.Pool object is used.
See the attachment or this link for the code: http://pastie.org/2614925 On Thu, Sep 29, 2011 at 8:24 PM, Josh Ayers <josh.ay...@gmail.com> wrote: > I think the slowdown you're seeing is due to the time it takes to create > new processes. This seems to be quite a bit slower in PyPy than in > CPython. However, once the process pool is created and has been used once, > the execution time vs. process count behaves as expected. > > I attached a modified version of your code to demonstrate the behavior. It > calculates Pi once without using multiprocessing as a baseline for > comparison. Then a multiprocessing.Pool object is created with 8 processes, > and the same pool is used multiple times. On my machine, creating the 8 new > processes takes 0.60 seconds in PyPy and only 0.20 seconds in CPython. > > The pool is first used two times in a row with only a single process > active. For some reason, the second run is a factor of 2 faster than the > first. Is this just warmup of the JIT, or some other behavior? > > Next, it repeats using 2, 4, and 8 processes. This was run on a 4 core > machine, and as expected there was an improvement in run time with 2 and 4 > processes. Using 8 processes gives approximately the same run time as 4. > > The output is pasted below. I also pasted the modified code here in case > the attached file doesn't come through: http://pastie.org/2614751. For > reference, I'm running PyPy 1.6 on Windows 7. > > Sincerely, > Josh > > > C:\Users\jayers\Documents\SVN\randomStuff\pypy_comparisons>pypy-c > pi_python2_multiprocessing_pool.py > > 3.14159265359 > non parallel execution time: 1.52899980545 > pool creation time: 0.559000015259 > ==== Python Multiprocessing Pool pi = 3.14159265359 > ==== Python Multiprocessing Pool iteration count = 100000000 > ==== Python Multiprocessing Pool elapse = 3.1930000782 > ==== Python Multiprocessing Pool process count = 1 > ==== Python Multiprocessing Pool processor count = 4 > > > ==== Python Multiprocessing Pool pi = 3.14159265359 > ==== Python Multiprocessing Pool iteration count = 100000000 > ==== Python Multiprocessing Pool elapse = 1.53900003433 > ==== Python Multiprocessing Pool process count = 1 > ==== Python Multiprocessing Pool processor count = 4 > > > ==== Python Multiprocessing Pool pi = 3.14159265359 > ==== Python Multiprocessing Pool iteration count = 100000000 > ==== Python Multiprocessing Pool elapse = 0.802000045776 > ==== Python Multiprocessing Pool process count = 2 > ==== Python Multiprocessing Pool processor count = 4 > > > ==== Python Multiprocessing Pool pi = 3.14159265359 > ==== Python Multiprocessing Pool iteration count = 100000000 > ==== Python Multiprocessing Pool elapse = 0.441999912262 > ==== Python Multiprocessing Pool process count = 4 > ==== Python Multiprocessing Pool processor count = 4 > > > ==== Python Multiprocessing Pool pi = 3.14159265359 > ==== Python Multiprocessing Pool iteration count = 100000000 > ==== Python Multiprocessing Pool elapse = 0.457000017166 > ==== Python Multiprocessing Pool process count = 8 > ==== Python Multiprocessing Pool processor count = 4 > > >
pi_python2_multiprocessing_pool_v2.py
Description: Binary data
_______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
