Henry: thanks a lot, that would be very appreciated regardless of whether I end up using it in this specific project or not. Other replies below.
Antony 2013/9/19 Robert Kern <robert.k...@gmail.com> > On Thu, Sep 19, 2013 at 2:40 AM, Antony Lee <antony....@berkeley.edu> > wrote: > > > > Thanks, I didn't know that multiprocessing Managers could be used with > processes not started by multiprocessing itself... I will give them a try. > > I just need to compute FFTs, but speed is a real issue for me (I am > using the results for real-time feedback). > > I am pretty sure that the overhead of communicating a large array from one > process to another will vastly overwhelm any speed gains you get by using > pyFFTW over numpy.fft. > I would have hoped that the large arrays are simply written (from the beginning) to shared memory (what multiprocessing.sharedctypes.Array seems to do(?)) and that interprocess communication would be cheap enough (but what do I know about that). > > > > To be honest I don't know yet if the FFTs are going to be the limiting > step but I thought I may as well give pyFFTW a try and ran into that > issue... > > In that case, thinking about multiprocessing or even pyFFTW is far too > premature. Implement your code with numpy.fft and see what performance you > actually get. > There is another (and, in fact, main) reason for me to use multiprocessing: the main app runs a GUI and running the data analysis in the same process just makes it painfully slow (I have tried that). Instead, running the data analysis in a separate process keeps the GUI responsive. Now whether the data analysis process should use numpy.fft or pyFFTW is a separate question; I realize that the gains from pyFFTW may probably be negligible compared to the other costs (... including the costs of tweaking multiprocessing beyond its specifications) but I was just giving it a try when I ran into the issue and was just puzzled by the error message I had never seen before. > -- > Robert Kern > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
