+1 on the astype(int) call. +1 also on using dask. scikit-image has a couple of functions that might be useful:
- skimage.util.apply_parallel: applies a function to an input array in chunks, with user-selectable chunk size and margins. This is powered by dask. - skimage.util.view_as_windows: uses stride tricks to produce a sliding window view over an n-dimensional array. On 16 Sep 2017, 8:16 AM +1000, Chris Barker - NOAA Federal <chris.bar...@noaa.gov>, wrote: > No thoughts on optimizing memory, but that indexing error probably comes from > np.mean producing float results. An astype call shoulder that work. > > -CHB > > Sent from my iPhone > > On Sep 15, 2017, at 5:51 PM, Robert McLeod <robbmcl...@gmail.com> wrote: > > > > > > On Fri, Sep 15, 2017 at 2:37 PM, Elliot Hallmark <permafact...@gmail.com> > > > wrote: > > > > Nope. Numpy only works on in memory arrays. You can determine your own > > > > chunking strategy using hdf5, or something like dask can figure that > > > > strategy out for you. With numpy you might worry about not accidentally > > > > making duplicates or intermediate arrays, but that's the extent of > > > > memory optimization you can do in numpy itself. > > > > NumPy does have it's own memory map variant on ndarray: > > > > https://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html > > > > > > > > -- > > Robert McLeod, Ph.D. > > robbmcl...@gmail.com > > robbmcl...@protonmail.com > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
