On Tue, Jan 4, 2011 at 8:14 AM, Keith Goodman <kwgood...@gmail.com> wrote: > On Tue, Jan 4, 2011 at 8:06 AM, Sebastian Haase <seb.ha...@gmail.com> wrote: >> On Mon, Jan 3, 2011 at 5:32 PM, Erik Rigtorp <e...@rigtorp.com> wrote: >>> On Mon, Jan 3, 2011 at 11:26, Eric Firing <efir...@hawaii.edu> wrote: >>>> Instead of calculating statistics independently each time the window is >>>> advanced one data point, the statistics are updated. I have not done >>>> any benchmarking, but I expect this approach to be quick. >>> >>> This might accumulate numerical errors. But could be fine for many >>> applications. >>> >>>> The code is old; I have not tried to update it to take advantage of >>>> cython's advances over pyrex. If I were writing it now, I might not >>>> bother with the C level at all; it could all be done in cython, probably >>>> with no speed penalty, and maybe even with reduced overhead. >>>> >>> >>> No doubt this would be faster, I just wanted to offer a general way to >>> this in NumPy. >>> _______________________________________________ >> >> BTW, some of these operations can be done using scipy's ndimage - right ? >> Any comments ? How does the performance compare ? >> ndimage might have more options regarding edge handling, or ? > > Take a look at the moving window function in the development version > of the la package: > > https://github.com/kwgoodman/la/blob/master/la/farray/mov.py > > Many of the moving window functions offer three calculation methods: > filter (ndimage), strides (the strides trick discussed in this > thread), and loop (a simple python loop). > > For example: > >>> a = np.random.rand(500,2000) >>> timeit la.farray.mov_max(a, window=252, axis=-1, method='filter') > 1 loops, best of 3: 336 ms per loop >>> timeit la.farray.mov_max(a, window=252, axis=-1, method='strides') > 1 loops, best of 3: 609 ms per loop >>> timeit la.farray.mov_max(a, window=252, axis=-1, method='loop') > 1 loops, best of 3: 638 ms per loop > > No one method is best for all situations. That is one of the reasons I > started the Bottleneck package. I figured Cython could beat them all.
I added four new function to Bottleneck: move_min, move_max, move_nanmin, move_nanmax. They are much faster than using SciPy's ndimage.maximum_filter1d or the strides trick: >> a = np.random.rand(500,2000) >> timeit la.farray.mov_max(a, window=252, axis=-1, method='filter') # ndimage 1 loops, best of 3: 336 ms per loop >> timeit bn.move_max(a, window=252, axis=-1) # bottleneck 100 loops, best of 3: 14.1 ms per loop That looks too good to be true. Are the outputs the same? >> a1 = la.farray.mov_max(a, window=252, axis=-1, method='filter') >> a2 = bn.move_max(a, window=252, axis=-1) >> np.testing.assert_array_almost_equal(a1, a2) >> Yes. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion