Re: [Numpy-discussion] repeat an array without allocation
If b is indeed big I don't see a problem with the python loop, elegance aside; but Cython will not beat it on that front. On Mon, May 5, 2014 at 9:34 AM, srean wrote: > Great ! thanks. I should have seen that. > > Is there any way array multiplication (as opposed to matrix > multiplication) can be sped up without forming A and (A * b) explicitly. > > A = np.repeat(x, [4, 2, 1, 3], axis = 0)# A.shape == 10,10 > c = sum(b * A, axis = 1)# b.shape == 10,10 > > In my actual setting b is pretty big, so I would like to avoid creating > another array the same size. I would also like to avoid a Python loop. > > st = 0 > for (i,rep) in enumerate([4, 2, 1, 3]): > end = st + rep > c[st : end] = np.dot(b[st : end, :], a[i,:]) > st = end > > Is Cython the only way ? > > > On Mon, May 5, 2014 at 1:20 AM, Jaime Fernández del Río < > jaime.f...@gmail.com> wrote: > >> On Sun, May 4, 2014 at 9:34 PM, srean wrote: >> >>> Hi all, >>> >>> is there an efficient way to do the following without allocating A >>> where >>> >>> A = np.repeat(x, [4, 2, 1, 3], axis=0) >>> c = A.dot(b)# b.shape >>> >> >> If x is a 2D array you can call repeat **after** dot, not before, which >> will save you some memory and a few operations: >> >> >>> a = np.random.rand(4, 5) >> >>> b = np.random.rand(5, 6) >> >>> np.allclose(np.repeat(a, [4, 2, 1, 3], axis=0).dot(b), >> ... np.repeat(a.dot(b), [4, 2, 1, 3], axis=0)) >> True >> >> Similarly, if x is a 1D array, you can sum the corresponding items of b >> before calling dot: >> >> >>> a = np.random.rand(4) >> >>> b = np.random.rand(10) >> >>> idx = np.concatenate(([0], np.cumsum([4,2,1,3])[:-1])) >> >>> np.allclose(np.dot(np.repeat(a, [4,2,1,3] ,axis=0), b), >> ... np.dot(a, np.add.reduceat(b, idx))) >> ... ) >> True >> >> Jaime >> >> -- >> (\__/) >> ( O.o) >> ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes >> de dominación mundial. >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] repeat an array without allocation
Great ! thanks. I should have seen that. Is there any way array multiplication (as opposed to matrix multiplication) can be sped up without forming A and (A * b) explicitly. A = np.repeat(x, [4, 2, 1, 3], axis = 0)# A.shape == 10,10 c = sum(b * A, axis = 1)# b.shape == 10,10 In my actual setting b is pretty big, so I would like to avoid creating another array the same size. I would also like to avoid a Python loop. st = 0 for (i,rep) in enumerate([4, 2, 1, 3]): end = st + rep c[st : end] = np.dot(b[st : end, :], a[i,:]) st = end Is Cython the only way ? On Mon, May 5, 2014 at 1:20 AM, Jaime Fernández del Río < jaime.f...@gmail.com> wrote: > On Sun, May 4, 2014 at 9:34 PM, srean wrote: > >> Hi all, >> >> is there an efficient way to do the following without allocating A where >> >> A = np.repeat(x, [4, 2, 1, 3], axis=0) >> c = A.dot(b)# b.shape >> > > If x is a 2D array you can call repeat **after** dot, not before, which > will save you some memory and a few operations: > > >>> a = np.random.rand(4, 5) > >>> b = np.random.rand(5, 6) > >>> np.allclose(np.repeat(a, [4, 2, 1, 3], axis=0).dot(b), > ... np.repeat(a.dot(b), [4, 2, 1, 3], axis=0)) > True > > Similarly, if x is a 1D array, you can sum the corresponding items of b > before calling dot: > > >>> a = np.random.rand(4) > >>> b = np.random.rand(10) > >>> idx = np.concatenate(([0], np.cumsum([4,2,1,3])[:-1])) > >>> np.allclose(np.dot(np.repeat(a, [4,2,1,3] ,axis=0), b), > ... np.dot(a, np.add.reduceat(b, idx))) > ... ) > True > > Jaime > > -- > (\__/) > ( O.o) > ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes > de dominación mundial. > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] repeat an array without allocation
On Sun, May 4, 2014 at 9:34 PM, srean wrote: > Hi all, > > is there an efficient way to do the following without allocating A where > > A = np.repeat(x, [4, 2, 1, 3], axis=0) > c = A.dot(b)# b.shape > If x is a 2D array you can call repeat **after** dot, not before, which will save you some memory and a few operations: >>> a = np.random.rand(4, 5) >>> b = np.random.rand(5, 6) >>> np.allclose(np.repeat(a, [4, 2, 1, 3], axis=0).dot(b), ... np.repeat(a.dot(b), [4, 2, 1, 3], axis=0)) True Similarly, if x is a 1D array, you can sum the corresponding items of b before calling dot: >>> a = np.random.rand(4) >>> b = np.random.rand(10) >>> idx = np.concatenate(([0], np.cumsum([4,2,1,3])[:-1])) >>> np.allclose(np.dot(np.repeat(a, [4,2,1,3] ,axis=0), b), ... np.dot(a, np.add.reduceat(b, idx))) ... ) True Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] repeat an array without allocation
nope; its impossible to express A as a strided view on x, for the repeats you have. even if you had uniform repeats, it still would not work. that would make it easy to add an extra axis to x without a new allocation; but reshaping/merging that axis with axis=0 would again trigger a copy, as it would require a non-integer stride. On Mon, May 5, 2014 at 6:34 AM, srean wrote: > Hi all, > > is there an efficient way to do the following without allocating A where > > A = np.repeat(x, [4, 2, 1, 3], axis=0) > c = A.dot(b)# b.shape > > thanks > -- srean > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] repeat an array without allocation
Hi all, is there an efficient way to do the following without allocating A where A = np.repeat(x, [4, 2, 1, 3], axis=0) c = A.dot(b)# b.shape thanks -- srean ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
