2013/9/19 Robert Kern <robert.k...@gmail.com> > On Thu, Sep 19, 2013 at 5:58 PM, Antony Lee <antony....@berkeley.edu> > wrote: > > > > Henry: thanks a lot, that would be very appreciated regardless of > whether I end up using it in this specific project or not. > > Other replies below. > > > > Antony > > > > 2013/9/19 Robert Kern <robert.k...@gmail.com> > >> > >> On Thu, Sep 19, 2013 at 2:40 AM, Antony Lee <antony....@berkeley.edu> > wrote: > >> > > >> > Thanks, I didn't know that multiprocessing Managers could be used > with processes not started by multiprocessing itself... I will give them a > try. > >> > I just need to compute FFTs, but speed is a real issue for me (I am > using the results for real-time feedback). > >> > >> I am pretty sure that the overhead of communicating a large array from > one process to another will vastly overwhelm any speed gains you get by > using pyFFTW over numpy.fft. > > > > I would have hoped that the large arrays are simply written (from the > beginning) to shared memory (what multiprocessing.sharedctypes.Array seems > to do(?)) and that interprocess communication would be cheap enough (but > what do I know about that). > > It certainly won't be automatic just by passing a numpy array to the > manager. You will have to manually create the shared memory, pass its > handle to the other process, and copy into it. But even the copy of the > array may overwhelm the speed gains between PyFFTW and numpy.fft. If you > can set it up such that the subprocess owns the shared memory for both > input and output and the GUI process always writes into the input shared > array directly and reads out the output shared array, then might work out > okay. This works well when the inputs/outputs are always the same size. > The arrays would always be the same size, and there is no array copy involved, as (I think that) I can have the C dll directly write whatever data needs to be analyzed to the shared memory array -- basically what you're suggesting.
> > -- > Robert Kern > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
