Re: [Numpy-discussion] Arrays with aliased elements?
def repeat(arr, num): arr = numpy.asarray(arr) return numpy.ndarray(arr.shape+(num,), dtype=arr.dtype, buffer=arr, strides=arr.strides+(0,)) There are limits to what these sort of stride tricks can accomplish, but repeating as above, or similar, is feasible. On Jan 1, 2011, at 8:42 PM, Enzo Michelangeli wrote: Is there any way, not involving compilation of C code, to define ndarrays where some rows or columns share the same data buffers? For example, something built by a hypothetical variant of the np.repeat() function, such that, if a = array([2,3]), calling: b = np.aliasedrepeat(x, [1, 2], axis=0) would return in b: array([[2, 3], [2, 3], [2, 3]]) ...with the understanding that the three rows would actually share the same data, so setting e.g.: b[0,1] = 5 ...would change b into: array([[2, 5], [2, 5], [2, 5]]) In other words, something with a behaviour similar to a list of lists: a = [2,3] b = [a,a,a] b [[2, 3], [2, 3], [2, 3]] b[0][1] = 5 b [[2, 5], [2, 5], [2, 5]] This would save memory (and time spent in unnecessary copying) in some applications with large arrays, and would allow to cope with the current inability of weave.blitz to understand broadcasting rules, e.g. for calculating outer products (I mentioned this in a previous thread). Enzo ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Arrays with aliased elements?
On Sat, Jan 1, 2011 at 19:42, Enzo Michelangeli enzom...@gmail.com wrote: Is there any way, not involving compilation of C code, to define ndarrays where some rows or columns share the same data buffers? For example, something built by a hypothetical variant of the np.repeat() function, such that, if a = array([2,3]), calling: b = np.aliasedrepeat(x, [1, 2], axis=0) would return in b: array([[2, 3], [2, 3], [2, 3]]) ...with the understanding that the three rows would actually share the same data, so setting e.g.: b[0,1] = 5 ...would change b into: array([[2, 5], [2, 5], [2, 5]]) In other words, something with a behaviour similar to a list of lists: a = [2,3] b = [a,a,a] b [[2, 3], [2, 3], [2, 3]] b[0][1] = 5 b [[2, 5], [2, 5], [2, 5]] This would save memory (and time spent in unnecessary copying) in some applications with large arrays, and would allow to cope with the current inability of weave.blitz to understand broadcasting rules, e.g. for calculating outer products (I mentioned this in a previous thread). See numpy.lib.stride_tricks for tools to do this, specifically the as_strided() function. See numpy.broadcast_arrays() for the latter functionality. http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast_arrays.html -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Arrays with aliased elements?
Thanks. Meanwhile, I had arrived to a solution similar to the one suggested
by Zachary:
a = array([2,3])
ndarray((3,a.shape[0]), strides=(0,a.itemsize), buffer = a, offset=0,
dtype=a.dtype)
array([[2, 3],
[2, 3],
[2, 3]])
...but I'd say that numpy.broadcast_arrays is the cleanest way of obtaining
pre-broadcasted views to pass to weave.blitz(). But alas, it appears that
blitz doesn't work well with such non-contiguous views:
tsb, pivb = broadcast_arrays(tableau[:,cand:cand+1], pivot)
tableau = tableau - tsb * pivb
...works, but:
tsb, pivb = broadcast_arrays(tableau[:,cand:cand+1], pivot)
weave.blitz('tableau = tableau - tsb * pivb')
...returns wrong results. And, of course, converting them to contiguous
through the array() function defeats the intended savings in memory and CPU
cycles...
Enzo
- Original Message -
From: Robert Kern robert.k...@gmail.com
To: Discussion of Numerical Python numpy-discussion@scipy.org
Sent: Sunday, January 02, 2011 10:08 AM
Subject: Re: [Numpy-discussion] Arrays with aliased elements?
On Sat, Jan 1, 2011 at 19:42, Enzo Michelangeli enzom...@gmail.com
wrote:
Is there any way, not involving compilation of C code, to define ndarrays
where some rows or columns share the same data buffers? For example,
something built by a hypothetical variant of the np.repeat() function,
such
that, if a = array([2,3]), calling:
b = np.aliasedrepeat(x, [1, 2], axis=0)
would return in b:
array([[2, 3],
[2, 3],
[2, 3]])
...with the understanding that the three rows would actually share the
same
data, so setting e.g.:
b[0,1] = 5
...would change b into:
array([[2, 5],
[2, 5],
[2, 5]])
In other words, something with a behaviour similar to a list of lists:
a = [2,3]
b = [a,a,a]
b
[[2, 3], [2, 3], [2, 3]]
b[0][1] = 5
b
[[2, 5], [2, 5], [2, 5]]
This would save memory (and time spent in unnecessary copying) in some
applications with large arrays, and would allow to cope with the current
inability of weave.blitz to understand broadcasting rules, e.g. for
calculating outer products (I mentioned this in a previous thread).
See numpy.lib.stride_tricks for tools to do this, specifically the
as_strided() function. See numpy.broadcast_arrays() for the latter
functionality.
http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast_arrays.html
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
