Paulo van Breugel wrote: > > It may be possible to avoid this issue using numpy.tensordot, e.g. > > > > mahdist = np.sum(np.tensordot(VI, delta, (1,0)) * delta,axis=0) > > > > I don't know for sure whether this actually uses less memory. It could > > do, as it rearranges the matrices so that the sum-of-products is > > calculated using np.dot, which is a built-in function. > > > > I did some quick testing, it seems to use slightly less memory. The dask > solution seems promising, with the disadvantage that it is not widely > available / still experimental according to the website. > > For a quick solution, what about using r.tile to split the input data in > tiles and compute the mahalanobis distance per tile. Or would that come > with too much overhead or other clear disadvantages?
I'd suggest just splitting the processing into chunks in the Python script. The original version operated one cell at a time, processing the entire map at once uses too much memory, but you could operate on a row (or N rows) at a time. IOW, rather than mahdist = np.sum(np.tensordot(VI, delta, (1,0)) * delta,axis=0) use e.g.: for row in xrange(0,rows,step): end = min(rows,row+step) dchunk = delta[:,row:end] mahdist[row:end] = np.sum(np.tensordot(VI, dchunk, (1,0)) * dchunk,axis=0) -- Glynn Clements <gl...@gclements.plus.com> _______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev