Hi Pietro, Thanks for the suggestion, I will have a look at the documentation.
Paulo On Fri, Feb 13, 2015 at 10:09 AM, Pietro <peter.z...@gmail.com> wrote: > Dear Paulo, > > On Fri, Feb 13, 2015 at 9:57 AM, Paulo van Breugel > <p.vanbreu...@gmail.com> wrote: > > I guess this is because the calculations are done in-memory? Any way to > > avoid this memory problem when using large data sets (something like > working > > with memmap objects?) > > With memmap you still have a limits of 2Gb I guess, you should try: dask > > Dask Array implements the NumPy ndarray interface using blocked > algorithms, cutting up the large array into many small arrays. This > lets us compute on arrays larger than memory using all of our cores. > We coordinate these blocked algorithms using dask graphs. > > http://dask.readthedocs.org/en/latest/array.html > > I didn't have a chance to try it yet, but it support a numpy array > syntax, and since you are using quite basic functionalities I think > you should be able to work with it. > > All the best > > Pietro >
_______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
