[Numpy-discussion] simple vector-matrix question
Given a vector y, I want a matrix H whose rows are y - x0 y - x1 y - x2 ... where x_i are scalars Suggestion? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple vector-matrix question
On Thu, Oct 6, 2011 at 7:08 AM, Neal Becker ndbeck...@gmail.com wrote: Given a vector y, I want a matrix H whose rows are y - x0 y - x1 y - x2 ... where x_i are scalars Suggestion? In [15]: import numpy as np In [16]: y = np.array([10.0, 20.0, 30.0]) In [17]: x = np.array([0, 1, 2, 4]) In [18]: H = y - x[:, np.newaxis] In [19]: H Out[19]: array([[ 10., 20., 30.], [ 9., 19., 29.], [ 8., 18., 28.], [ 6., 16., 26.]]) Warren ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple vector-matrix question
import numpy # Say y is y = numpy.array([1,2,3]) Y = numpy.vstack([y,y,y,y]) # Y is array([[1, 2, 3], # [1, 2, 3], # [1, 2, 3], # [1, 2, 3]]) x = numpy.array([[0],[2],[4],[6]]) # a column-vector of your scalars x0, x1... Y - x Hope this is what you meant. cheers, Samuel On 06.10.2011, at 14:08, Neal Becker wrote: Given a vector y, I want a matrix H whose rows are y - x0 y - x1 y - x2 ... where x_i are scalars ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple vector-matrix question
I just learned two things: 1. np.newaxis 2. Array dimension broadcasting rocks more than you think. The x[:, np.newaxis] might not be the most intuitive solution but it's great and powerful. Intuitive would be to have x.T to transform [0,1,2,4] into [[0],[1],[2],[4]]. Thanks Warren :-) Samuel On 06.10.2011, at 14:18, Warren Weckesser wrote: On Thu, Oct 6, 2011 at 7:08 AM, Neal Becker ndbeck...@gmail.com wrote: Given a vector y, I want a matrix H whose rows are y - x0 y - x1 y - x2 ... where x_i are scalars Suggestion? In [15]: import numpy as np In [16]: y = np.array([10.0, 20.0, 30.0]) In [17]: x = np.array([0, 1, 2, 4]) In [18]: H = y - x[:, np.newaxis] In [19]: H Out[19]: array([[ 10., 20., 30.], [ 9., 19., 29.], [ 8., 18., 28.], [ 6., 16., 26.]]) Warren ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple vector-matrix question
On Thu, Oct 6, 2011 at 7:29 AM, Samuel John sc...@samueljohn.de wrote: I just learned two things: 1. np.newaxis 2. Array dimension broadcasting rocks more than you think. Yup. :) The x[:, np.newaxis] might not be the most intuitive solution but it's great and powerful. Intuitive would be to have x.T to transform [0,1,2,4] into [[0],[1],[2],[4]]. I agree, creating a new dimension by indexing with np.newaxis isn't the first thing I would guess if I didn't already know about it. An alternative is x.reshape(4,1) (or even better, x.reshape(-1,1) so it doesn't explicitly refer to the length of x). (Also, you probably noticed that transposing won't work, because x is one-dimensional. The transpose operation simply swaps dimensions, and with just one dimension there is nothing to swap; x.T is the same as x.) Warren Thanks Warren :-) Samuel On 06.10.2011, at 14:18, Warren Weckesser wrote: On Thu, Oct 6, 2011 at 7:08 AM, Neal Becker ndbeck...@gmail.com wrote: Given a vector y, I want a matrix H whose rows are y - x0 y - x1 y - x2 ... where x_i are scalars Suggestion? In [15]: import numpy as np In [16]: y = np.array([10.0, 20.0, 30.0]) In [17]: x = np.array([0, 1, 2, 4]) In [18]: H = y - x[:, np.newaxis] In [19]: H Out[19]: array([[ 10., 20., 30.], [ 9., 19., 29.], [ 8., 18., 28.], [ 6., 16., 26.]]) Warren ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] build errors
Hi again, I have built the ATLAS dynamic shared libraries and now need to tell numpy to build against them which are located in a different location to where it expects them. Do you know how I can do that? The command I am using to build numpy is: python setup.py build --fcompiler=gnu95 but this looks in /usr/lib64/atlas and I need it to look in another location (/gpfs/grace/atlas-3.8.4). Thanks in advance, Regards. -Original Message- From: numpy-discussion-boun...@scipy.org [mailto:numpy-discussion- boun...@scipy.org] On Behalf Of Miah Wadud Dr (ITCS) Sent: Thursday, October 06, 2011 11:12 AM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] build errors Hi David, Thanks for your reply. Nope, I didn't build the ATLAS libraries myself and am trying to do that now. However, whenever I try to build the shared libraries using the configure command: [root@cn130 linux]# ../configure -Fa alg -fPIC --prefix=/gpfs/grace/atlas-3.8.4 it keeps building the static version. The ATLAS documentation stated that I need to provide the above flags to build the dynamic ones but this doesn't seem to work. Any help will be greatly appreciated. Thanks in advance. Regards, Wadud. -Original Message- From: numpy-discussion-boun...@scipy.org [mailto:numpy-discussion- boun...@scipy.org] On Behalf Of David Cournapeau Sent: Tuesday, October 04, 2011 6:40 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] build errors On Tue, Oct 4, 2011 at 11:54 AM, Miah Wadud Dr (ITCS) w.m...@uea.ac.uk wrote: Hello numpy users, I am trying to build numpy 1.6.1 and am having problems. It prints the following error message: gcc -pthread -shared build/temp.linux-x86_64- 2.4/numpy/core/blasdot/_dotblas.o -L/usr/lib64/atlas -Lbuild/temp.linux- x86_64- 2.4 -lptf77blas -lptcblas -latlas -o build/lib.linux-x86_64- 2.4/numpy/core/_dotblas.so /usr/bin/ld: /usr/lib64/atlas/libptcblas.a(cblas_dptgemm.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/lib64/atlas/libptcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status /usr/bin/ld: /usr/lib64/atlas/libptcblas.a(cblas_dptgemm.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/lib64/atlas/libptcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status error: Command gcc -pthread -shared build/temp.linux-x86_64- 2.4/numpy/core/blasdot/_dotblas.o -L/usr/lib64/atlas -Lbuild/temp.linux- x86_64- 2.4 -lptf77blas -lptcblas -latlas -o build/lib.linux-x86_64- 2.4/numpy/core/_dotblas.so failed with exit status 1 Did you build Atlas by yourself ? If so, it is most likely not usable for shared libraries (mandatory for any python extension, including bumpy). You need to configure atlas with the option -Fa alg -fPIC. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] build errors
You can use the BLAS and LAPACK environment variables. export BLAS=/path/to/libatlas.so export LAPACK=/path/to/libatlas.so python setup.py build I've recently had problems with ATLAS solving equation systems incorrectly for certain inputs with no adequate explanation. Re-running the same simulation but linking to Goto2 BLAS/LAPACK instead solved the problem. Strange, yet worrisome. YMMV! Cheers Paul On Thu, Oct 6, 2011 at 3:05 PM, Miah Wadud Dr (ITCS) w.m...@uea.ac.uk wrote: Hi again, I have built the ATLAS dynamic shared libraries and now need to tell numpy to build against them which are located in a different location to where it expects them. Do you know how I can do that? The command I am using to build numpy is: python setup.py build --fcompiler=gnu95 but this looks in /usr/lib64/atlas and I need it to look in another location (/gpfs/grace/atlas-3.8.4). Thanks in advance, Regards. -Original Message- From: numpy-discussion-boun...@scipy.org [mailto:numpy-discussion- boun...@scipy.org] On Behalf Of Miah Wadud Dr (ITCS) Sent: Thursday, October 06, 2011 11:12 AM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] build errors Hi David, Thanks for your reply. Nope, I didn't build the ATLAS libraries myself and am trying to do that now. However, whenever I try to build the shared libraries using the configure command: [root@cn130 linux]# ../configure -Fa alg -fPIC --prefix=/gpfs/grace/atlas-3.8.4 it keeps building the static version. The ATLAS documentation stated that I need to provide the above flags to build the dynamic ones but this doesn't seem to work. Any help will be greatly appreciated. Thanks in advance. Regards, Wadud. -Original Message- From: numpy-discussion-boun...@scipy.org [mailto:numpy-discussion- boun...@scipy.org] On Behalf Of David Cournapeau Sent: Tuesday, October 04, 2011 6:40 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] build errors On Tue, Oct 4, 2011 at 11:54 AM, Miah Wadud Dr (ITCS) w.m...@uea.ac.uk wrote: Hello numpy users, I am trying to build numpy 1.6.1 and am having problems. It prints the following error message: gcc -pthread -shared build/temp.linux-x86_64- 2.4/numpy/core/blasdot/_dotblas.o -L/usr/lib64/atlas -Lbuild/temp.linux- x86_64- 2.4 -lptf77blas -lptcblas -latlas -o build/lib.linux-x86_64- 2.4/numpy/core/_dotblas.so /usr/bin/ld: /usr/lib64/atlas/libptcblas.a(cblas_dptgemm.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/lib64/atlas/libptcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status /usr/bin/ld: /usr/lib64/atlas/libptcblas.a(cblas_dptgemm.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/lib64/atlas/libptcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status error: Command gcc -pthread -shared build/temp.linux-x86_64- 2.4/numpy/core/blasdot/_dotblas.o -L/usr/lib64/atlas -Lbuild/temp.linux- x86_64- 2.4 -lptf77blas -lptcblas -latlas -o build/lib.linux-x86_64- 2.4/numpy/core/_dotblas.so failed with exit status 1 Did you build Atlas by yourself ? If so, it is most likely not usable for shared libraries (mandatory for any python extension, including bumpy). You need to configure atlas with the option -Fa alg -fPIC. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] nditer: possible to manually handle dimensions with different lengths?
I ended up fixing my problem by removing the 'buffering' flag and adding the 'copy' flag to each of the input arrays. I think that nested_iters might be improved by an operand axes specification for each layer of nesting like nditer uses, though I suppose that 3 layers of nesting might be confusing for users. I get an array too big error on values, groups, outs = it1.itviews when the shape of the iterator is larger than ~(4728, 125285) even if each of the arrays should only have actual size along one dimension. Code: @cython.boundscheck(False) def groupwise_accumulate(vals, group, axis ): cdef np.ndarray[double, ndim = 2] values cdef np.ndarray[long, ndim = 2] groups cdef np.ndarray[double, ndim = 2] outs cdef int size cdef long g cdef int i, j #copy so that swaping the axis doesn't mess up the original arrays vals = vals.copy() group = group.copy() #add a dimension to match up with the new dimension and swap the given axis to the end vals.shape = vals.shape + (1,) vaxes = range(vals.ndim) vaxes.append(axis) vaxes.remove(axis) vals = np.transpose(vals, vaxes) vals = vals.copy() #the output should have the same shape as the values except along the #last two axes (which are the given axis and the new axis) oshape = list(vals.shape) bins = len(np.unique(group)) oshape[-1] = 1 oshape[-2] = bins out = np.empty(tuple(oshape)) #line up grouping with the given axis gshape = [1] * (len(oshape) - 1) + [vals.shape[-1]] group.shape = gshape #nested iterator should go along the last two axes axes = range(vals.ndim) axes0 = axes[:-2] axes1 = axes[-2:] it0, it1 = np.nested_iters([vals,group, out], [axes0, axes1], op_dtypes=['float64', 'int32', 'float64'], op_flags = [['readonly', 'copy'], ['readonly','copy'], ['readwrite']], flags = ['multi_index', 'reduce_ok' ]) size = vals.shape[-1] for x in it0: values, groups, outs = it1.itviews i = 0 j = 0 while i size: g = groups[0,i] #accumulation initialization while i size and groups[0,i] == g: #groupwise accumulation i += 1 outs[j,0] = calculation() j += 1 #swap back the new axis to the original location of the given axis out.shape = out.shape[:-1] oaxes = range(vals.ndim -1) oaxes.insert(axis, out.ndim-1) oaxes = oaxes[:-1] #remove the now reduced original given axis out = np.transpose(out, oaxes) return out On Mon, Oct 3, 2011 at 2:03 PM, John Salvatier jsalv...@u.washington.eduwrote: Some observations and questions about nested_iters. Nested_iters seems to require that all input arrays have the same number of dimensions (so you will have to pad some input shapes with 1s). Is there a way to specify how the axes line are matched together like for nditer? When I try to run the following program, @cython.boundscheck(False) def vars(vals, group, axis ): cdef np.ndarray[double, ndim = 2] values cdef np.ndarray[long long, ndim = 2] groups cdef np.ndarray[double, ndim = 2] outs cdef int size cdef double value cdef int i, j cdef long long cgroup cdef double min cdef double max cdef double open oshape = list(vals.shape) bins = len(np.unique(group)) oshape = oshape+[bins] oshape[axis] = 1 out = np.empty(tuple(oshape)) axes = range(vals.ndim) axes.remove(axis) gshape = [1] * len(oshape) gshape[axis] = len(group) group.shape = gshape vals = vals[...,np.newaxis] it0, it1 = np.nested_iters([vals,group, out], [axes, [axis,len(oshape) -1]], op_dtypes=['float64', 'int64', 'float64'], flags = ['multi_index', 'buffered']) size = vals.shape[axis] for x in it0: values, groups, outs = it1.itviews j = -1 for i in range(size): if cgroup != groups[i,0]: if j != -1: outs[0,j] = garmanklass(open, values[i,0], min, max) cgroup = groups[i,0] min = inf max = -inf open = values[i,0] j += 1 min = fmin(min, values[i,0]) max = fmax(max, values[i,0]) outs[0,j+1] = garmanklass(open, values[size -1], min, max) return out I get an error File comp.pyx, line 58, in varscale.comp.vars (varscale\comp.c:1565) values, groups, outs = it1.itviews ValueError: cannot provide an iterator view when buffering is enabled Which I am not sure how to deal with. Any advice? What I am trying to do here is to do a grouped calculation (the group specified by the group argument) on the values along the given axis. I try to use nested_iter to iterate over the specified axis and a new axis (the length of the number of groups) separately so I can do my calculation. On Mon, Oct 3, 2011 at 9:03 AM, John Salvatier jsalv...@u.washington.eduwrote: Thanks mark! I think that's exactly what I'm looking for. We even had a previous discussion about this (oops!) ( http://mail.scipy.org/pipermail/numpy-discussion/2011-January/054421.html ). I didn't find any documentation, I will try to add some once I understand how it works better. John On Sat, Oct 1, 2011 at 2:53 PM, Mark Wiebe mwwi...@gmail.com wrote: On Sat, Oct 1, 2011 at
Re: [Numpy-discussion] Crash on (un-orthodox) __import__
On Thu, Oct 6, 2011 at 17:25, Paul Ivanov pivanov...@gmail.com wrote: Hi Andrea, On Tue, Oct 4, 2011 at 3:04 AM, Andrea Gavana andrea.gav...@gmail.com wrote: Hi All, I was fiddling here and there with some code doing dynamic import of stuff, and I noticed that this code: import os import sys init_name = rC:\Python27\Lib\site-packages\numpy\__init__.py directory, module_name = os.path.split(init_name) main = os.path.splitext(module_name)[0] sys.path.insert(0, os.path.normpath(directory)) # Crash here... mainmod = __import__(main) in this case, your main is '__init__' and your directory is 'C:\Python27\Lib\site-packages\numpy' which is probably not what you intended. You should make directory 'C:\Python27\Lib\site-packages' and main into 'numpy' Still, it shouldn't segfault, and it's worth figuring out why it does. gdb has been mostly unenlightening for me since gdb won't let me navigate the traceback. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Crash on (un-orthodox) __import__
Still, it shouldn't segfault, and it's worth figuring out why it does. gdb has been mostly unenlightening for me since gdb won't let me navigate the traceback. You could try faulthandler, it prints the (python) traceback after a crash: http://pypi.python.org/pypi/faulthandler/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
