Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/25/11 12:57 PM, Pauli Virtanen wrote: On Mon, 25 Apr 2011 10:16:13 -0700, Rob Beezer wrote: [clip] Many more details and complete transcripts are at: http://trac.sagemath.org/sage_trac/ticket/11248 Any thoughts or advice to help us understand this would be greatly appreciated. The Numpy routine is a very thin wrapper of LAPACK's ZGESDD, and probably cannot have any bugs of this kind, so the problem is most likely with the LAPACK and BLAS libraries you use. You will probably be able to reproduce the problem also with an equivalent Fortran/C snippet calling LAPACK directly. Problems like this in BLAS/LAPACK are somewhat difficult to track. You could try switching to another BLAS library (or, if you use ATLAS, compile it differently) and checking if the problem disappears. I was just looking up the documentation for ZGESDD and noticed that the value we have for rwork in the numpy call [1] does not match the Lapack docs. This was changed in Lapack 3.2.2 [2]. I've submitted a pull request: https://github.com/numpy/numpy/pull/78 but I have not tested the change. I doubt this fix will fix the problem on this thread, but it makes sense to make the change anyway. Thanks, Jason [1] https://github.com/numpy/numpy/blob/master/numpy/linalg/linalg.py#L1300 [2] http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=2t=1779, or http://www.netlib.org/lapack/lapack-3.2.2.html (bug 0046) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/25/11 12:57 PM, Pauli Virtanen wrote: The Numpy routine is a very thin wrapper of LAPACK's ZGESDD, and probably cannot have any bugs of this kind, As noted in my other message, I've been digging through the ZGESDD docs to understand it better. Here is the doc string for what becomes the V^T matrix: * VT (output) COMPLEX*16 array, dimension (LDVT,N) * If JOBZ = 'A' or JOBZ = 'O' and M = N, VT contains the * N-by-N unitary matrix V**H; * if JOBZ = 'S', VT contains the first min(M,N) rows of * V**H (the right singular vectors, stored rowwise); * if JOBZ = 'O' and M N, or JOBZ = 'N', VT is not referenced. Notice that VT is supposed to contain the N-by-N V**H if M=N (i.e., more rows than columns). In our problem cases, we have MN. I looked at the numpy linalg code and it doesn't seem to check to see if MN before returning the V matrix. Is this a problem? When I run an example C program that links to the OSX Accelerate framework, I get a V matrix that has a zero row at the bottom. I got these results by: 1. Download the example found at http://software.intel.com/sites/products/documentation/hpc/mkl/lapack/mkl_lapack_examples/zgesdd_ex.c.htm 2. Change the zgesdd call to have first argument A (so it mimics the default numpy call) 3. Change print_matrix( Right singular vectors (stored rowwise), m, n, vt, ldvt ); to print_matrix( Right singular vectors (stored rowwise), n, n, vt, ldvt ); (this is to print out the full n-by-n matrix, instead of just the first m rows) 4. Compile the program with gcc zgesdd_ex.c -framework Accelerate 5. Run the program: ./a.out And indeed, I get a 0 row as the last row of the V**H matrix However, when I do the SVD of this same matrix using numpy (which I *think* uses the Accelerate framework the way we compile it in Sage), I get a full V matrix: In [13]: I=1j In [14]: a=numpy.asarray([( -5.40+ I* 7.40), ( 6.00+ I* 6.38), ( 9.91+ I* 0.16), ( -5.28+ I* -4.16), ( 1.09+ I* 1.55), ( 2.60+ I* 0.07), ( 3.98+ I* -5.26), ( 2.03+ I* 1.11), ( 9.88+ I* 1.91), ( 4.92+ I* 6.31), ( -2.11+ I* 7.39), ( -9.81+ I* -8.98)],dtype=complex).reshape(3,4) In [15]: numpy.linalg.svd(a)Out[15]: (array([[ 0.54742764+0.j, 0.76302168+0.j, -0.34368721+0.j], [-0.03507684-0.15148438j, 0.27097680-0.22637514j, 0.54572628-0.74386208j], [ 0.81299016+0.12325614j, -0.52311095-0.13956616j, 0.13357577-0.1135282j ]]), array([ 21.75519279, 16.59545017, 3.97327576]), array([[ 0.23160531+0.20669796j, 0.36590896+0.3864613j , 0.24259328+0.32833854j, -0.56133932-0.37233547j], [-0.57911906+0.40329699j, 0.10921398+0.17242422j, 0.59673801-0.27492812j, 0.15998810+0.05510835j], [ 0.60420072+0.12337134j, -0.18988750+0.29722068j, 0.39210635+0.19697635j, 0.45451433+0.31015037j], [-0.08335334+0.13557583j, 0.69622499-0.25685378j, -0.15350399+0.43075518j, 0.46295826+0.02290113j]])) That seems odd. I must be missing something. It also seems odd that numpy returns V as a full 4x4 matrix even when MN. Thanks, Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/26/11 11:07 AM, Jason Grout wrote: And indeed, I get a 0 row as the last row of the V**H matrix I just double-checked things one last time and saw that I actually hadn't changed the first argument of zgesdd to A in the program that I actually ran. So with this change, I get a nonzero last row of the V**H matrix from the C call to zgesdd. So everything is consistent between the C call to zgesdd and the numpy svd call. So now my remaining question is: if the Lapack docs only say that V**H is the full n-by-n matrix if M=N, why is numpy returning it even if MN? Thanks, Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/26/11 11:12 AM, Jason Grout wrote: On 4/26/11 11:07 AM, Jason Grout wrote: And indeed, I get a 0 row as the last row of the V**H matrix I just double-checked things one last time and saw that I actually hadn't changed the first argument of zgesdd to A in the program that I actually ran. So with this change, I get a nonzero last row of the V**H matrix from the C call to zgesdd. So everything is consistent between the C call to zgesdd and the numpy svd call. So now my remaining question is: if the Lapack docs only say that V**H is the full n-by-n matrix if M=N, why is numpy returning it even if MN? One more post talking to myself... I notice that the zgesvd routine docs guarantee that the V returned is unitary, regardless of the size of A. So this might be another argument for calling zgesvd instead of zgesdd. Thanks, Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/26/11 11:22 AM, Jason Grout wrote: On 4/26/11 11:12 AM, Jason Grout wrote: On 4/26/11 11:07 AM, Jason Grout wrote: And indeed, I get a 0 row as the last row of the V**H matrix I just double-checked things one last time and saw that I actually hadn't changed the first argument of zgesdd to A in the program that I actually ran. So with this change, I get a nonzero last row of the V**H matrix from the C call to zgesdd. So everything is consistent between the C call to zgesdd and the numpy svd call. So now my remaining question is: if the Lapack docs only say that V**H is the full n-by-n matrix if M=N, why is numpy returning it even if MN? One more post talking to myself... I notice that the zgesvd routine docs guarantee that the V returned is unitary, regardless of the size of A. So this might be another argument for calling zgesvd instead of zgesdd. Okay, just one more data point. Our people that are seeing the problem with numpy returning a non-unitary V also see a non-unitary V being returned by the test C call to zgesdd. In other words, it really appears that zgesdd follows the Lapack docs, and if rowscolumns, the returned V is not necessarily unitary, but may contain a zero row. This makes numpy's assumptions in using zgesdd false. You can see this report at http://trac.sagemath.org/sage_trac/ticket/11248#comment:25 Thanks, Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
Tue, 26 Apr 2011 11:36:19 -0500, Jason Grout wrote: [clip] Okay, just one more data point. Our people that are seeing the problem with numpy returning a non-unitary V also see a non-unitary V being returned by the test C call to zgesdd. In other words, it really appears that zgesdd follows the Lapack docs, and if rowscolumns, the returned V is not necessarily unitary, but may contain a zero row. This makes numpy's assumptions in using zgesdd false. You can see this report at http://trac.sagemath.org/sage_trac/ticket/11248#comment:25 What LAPACK promises is not very clear. Earlier on the the man page: JOBZ(input) CHARACTER*1 Specifies options for computing all or part of the matrix U: = 'A': all M columns of U and all N rows of V**H are returned in the arrays U and VT; = 'S': the first min(M,N) columns of U and the first min(M,N) rows of V**H are returned in the arrays U and VT; = 'O': If M = N, the first N columns of U are over‐ written in the array A and all rows of V**H are returned in the array VT; otherwise, all columns of U are returned in the array U and the first M rows of V**H are overwritten in the array A; = 'N': no columns of U or rows of V**H are computed. Looks to me the statement should be parsed as: return_n_rows = (jobz == 'A') or (jobz == 'O' and m = n) So the current usage should be OK. (At least, as long as jobz == 'A', in the other cases, I don't think the wrapper does the right thing.) But apparently either there's a bug, or the LAPACK man page needs to be understood as you say. Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On Tue, Apr 26, 2011 at 6:36 PM, Jason Grout jason-s...@creativetrax.com wrote: On 4/26/11 11:22 AM, Jason Grout wrote: On 4/26/11 11:12 AM, Jason Grout wrote: On 4/26/11 11:07 AM, Jason Grout wrote: And indeed, I get a 0 row as the last row of the V**H matrix I just double-checked things one last time and saw that I actually hadn't changed the first argument of zgesdd to A in the program that I actually ran. So with this change, I get a nonzero last row of the V**H matrix from the C call to zgesdd. So everything is consistent between the C call to zgesdd and the numpy svd call. So now my remaining question is: if the Lapack docs only say that V**H is the full n-by-n matrix if M=N, why is numpy returning it even if MN? One more post talking to myself... I notice that the zgesvd routine docs guarantee that the V returned is unitary, regardless of the size of A. So this might be another argument for calling zgesvd instead of zgesdd. Okay, just one more data point. Our people that are seeing the problem with numpy returning a non-unitary V also see a non-unitary V being returned by the test C call to zgesdd. In other words, it really appears that zgesdd follows the Lapack docs, and if rowscolumns, the returned V is not necessarily unitary, but may contain a zero row. This makes numpy's assumptions in using zgesdd false. You can see this report at http://trac.sagemath.org/sage_trac/ticket/11248#comment:25 If it is an Accelerate bug it got fixed on OS X 10.6. Below is the output with both current master and 1.5.1. It may be worth checking this on 10.5 with the binary Numpy installer from Sourceforge to make sure it's not the Sage build process causing this somehow. Ralf import numpy import scipy.linalg A = numpy.array( [[1 - 1j, -3j, -2 + 1j, 1, -2 + 3j], ... [ 1 - 1j, -2 + 1j, 1 + 4j, 0, 2 + 1j], ... [ -1, -5 + 1j, -2 + 1j, 1 + 1j, -5 - 4j], ... [-2 + 4j, 2 - 1j, 8 - 4j, 1 - 8j, 3 - 2j]]) U, S, VH = scipy.linalg.svd(A) VH array([[-0.10378056+0.25534799j, 0.22371070-0.07765616j, 0.55736639-0.40276461j, -0.04874824-0.50177439j, 0.32025785-0.19584279j], [-0.28067375-0.0116824j , -0.49586800+0.1305711j , 0.14432786+0.08803575j, 0.21076793+0.04575091j, -0.19220732-0.73899332j], [ 0.10136055-0.1681669j , 0.13465050+0.69687595j, 0.19709231-0.03497225j, -0.42056091+0.37188099j, 0.28760069-0.14046187j], [-0.27289262+0.66077235j, 0.06177902+0.25464336j, -0.52050549+0.06307174j, 0.11847581-0.00883692j, 0.35771384-0.05719981j], [ 0.49800436+0.21785287j, -0.32014745+0.07789755j, 0.15924098-0.39775924j, 0.50925102+0.33273359j, 0.14247809+0.14849585j]]) numpy.dot(VH, VH.conj().T) array([[ 1.e+00 +1.38777878e-17j, -8.32667268e-17 +2.84494650e-16j, 1.38777878e-16 -1.45716772e-16j, -1.94289029e-16 +1.38777878e-17j, -3.46944695e-17 +2.08166817e-17j], [ -8.32667268e-17 -2.77555756e-16j, 1.e+00 +0.e+00j, -7.56339436e-16 +1.38777878e-16j, 5.55111512e-17 -1.66533454e-16j, 2.11636264e-16 +1.11022302e-16j], [ 1.38777878e-16 +1.17961196e-16j, -7.56339436e-16 -1.28369537e-16j, 1.e+00 +6.93889390e-18j, -3.88578059e-16 +4.16333634e-17j, -1.04083409e-16 +3.12250226e-17j], [ -1.94289029e-16 -1.04083409e-17j, 5.55111512e-17 +1.99493200e-16j, -3.88578059e-16 -2.08166817e-17j, 1.e+00 -6.93889390e-18j, -9.71445147e-17 -8.15320034e-17j], [ -3.46944695e-17 +2.08166817e-17j, 2.11636264e-16 -1.00613962e-16j, -1.04083409e-16 -4.16333634e-17j, -9.71445147e-17 +6.93889390e-17j, 1.e+00 +0.e+00j]]) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 04/26/2011 10:19 AM, Ralf Gommers wrote: If it is an Accelerate bug it got fixed on OS X 10.6. Below is the output with both current master and 1.5.1. It may be worth checking this on 10.5 with the binary Numpy installer from Sourceforge to make sure it's not the Sage build process causing this somehow. Thanks, Ralf. Indeed, we have no reports of problems on OSX 10.6, and it is likely there have been many tests on that platform. So a test of numpy on 10.5, independent of Sage's build process, would be useful. I don't have access to a Mac, but I posted your suggestion to the Sage ticket for this, where hopefully somebody will take it up. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] reading tiff images
Hi What is current method of using ndiimage on a Tiff file? I've seen different methods using ndimage itself, scipy.misc and Pil. Mathew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On Tue, 26 Apr 2011 10:33:07 -0500, Jason Grout wrote: [clip] I was just looking up the documentation for ZGESDD and noticed that the value we have for rwork in the numpy call [1] does not match the Lapack docs. This was changed in Lapack 3.2.2 [2]. I've submitted a pull request: https://github.com/numpy/numpy/pull/78 but I have not tested the change. I doubt this fix will fix the problem on this thread, but it makes sense to make the change anyway. I think it is worth testing if this change fixes it. Note that current LAPACK docs have the number 7 there instead of 5, so there may really be a bug here. Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] reading tiff images
On Tue, Apr 26, 2011 at 8:31 PM, Daniel Lepage dplep...@gmail.com wrote: You need PIL no matter what; scipy.misc.imread, scipy.ndimage.imread, and scikits.image.io.imread all call PIL. Scikits.image has a plugin system for IO and can use FreeImage to load images. PIL's Tiff image handling is pretty buggy (especially multi-page and 16-bit tiffs), so that may be a good option too. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] reading tiff images
On Apr 26, 2011, at 2:31 PM, Daniel Lepage wrote: You need PIL no matter what; scipy.misc.imread, scipy.ndimage.imread, and scikits.image.io.imread all call PIL. scikits.image.io also has a ctypes wrapper for the freeimage library. I prefer these (well, I wrote them), though apparently there are some 64-bit issues (crashes?). I haven't been working on a 64-bit system so I haven't been able to address them, but I will be soon. It's a very thin wrapper around a simple image IO library, so there's lots of room to add and extend as need be... All of the PIL wrappers are kluges around serious flaws in how PIL reads images, particularly non-8-bit images and in particular non- native-endian 16-bit images. Zach Theoretically there's no difference between any of them, although in actuality some use import Image and others use from PIL import Image; one of these may fail depending on how you installed PIL. (I'm not sure which is supposed to be standard - the PIL docs use both interchangeably, and I think the latest version of PIL on pypi sets it up so that both will work). I'd use whichever tool you're already importing - if you're using ndimage anyway, just use ndimage.imread rather than adding more imports. Note that using PIL directly is easy, but does require adding an extra step; OTOH, if you're familiar with PIL, you can use some of its transformations from the start, e.g. def imread(fname, mode='RGBA'): return np.asarray(Image.open(fname).convert(mode)) to ensure that you always get 4-channel images, even for images that were initially RGB or grayscale. HTH, Dan On Tue, Apr 26, 2011 at 2:00 PM, Mathew Yeates mat.yea...@gmail.com wrote: Hi What is current method of using ndiimage on a Tiff file? I've seen different methods using ndimage itself, scipy.misc and Pil. Mathew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 04/26/2011 11:25 AM, Pauli Virtanen wrote: I think it is worth testing if this change fixes it. Note that current LAPACK docs have the number 7 there instead of 5, so there may really be a bug here. It appears to me the problem is more subtle than just a strict comparison between the number of rows and the number of columns. http://trac.sagemath.org/sage_trac/attachment/ticket/11248/broken.png shows a plot of matrix sizes (number of rows on the horizontal, number of columns on vertical). This is on an OSX 10.5 machine and whereever there is a dot then the SVD seems to *always* produce a non-unitary V, and the absence of a dot indicates no failures were found. So maybe a workspace size could be the culprit? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] reading tiff images
is scikits.image.io documented anywhere? On Tue, Apr 26, 2011 at 11:45 AM, Zachary Pincus zachary.pin...@yale.edu wrote: On Apr 26, 2011, at 2:31 PM, Daniel Lepage wrote: You need PIL no matter what; scipy.misc.imread, scipy.ndimage.imread, and scikits.image.io.imread all call PIL. scikits.image.io also has a ctypes wrapper for the freeimage library. I prefer these (well, I wrote them), though apparently there are some 64-bit issues (crashes?). I haven't been working on a 64-bit system so I haven't been able to address them, but I will be soon. It's a very thin wrapper around a simple image IO library, so there's lots of room to add and extend as need be... All of the PIL wrappers are kluges around serious flaws in how PIL reads images, particularly non-8-bit images and in particular non- native-endian 16-bit images. Zach Theoretically there's no difference between any of them, although in actuality some use import Image and others use from PIL import Image; one of these may fail depending on how you installed PIL. (I'm not sure which is supposed to be standard - the PIL docs use both interchangeably, and I think the latest version of PIL on pypi sets it up so that both will work). I'd use whichever tool you're already importing - if you're using ndimage anyway, just use ndimage.imread rather than adding more imports. Note that using PIL directly is easy, but does require adding an extra step; OTOH, if you're familiar with PIL, you can use some of its transformations from the start, e.g. def imread(fname, mode='RGBA'): return np.asarray(Image.open(fname).convert(mode)) to ensure that you always get 4-channel images, even for images that were initially RGB or grayscale. HTH, Dan On Tue, Apr 26, 2011 at 2:00 PM, Mathew Yeates mat.yea...@gmail.com wrote: Hi What is current method of using ndiimage on a Tiff file? I've seen different methods using ndimage itself, scipy.misc and Pil. Mathew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] reading tiff images
On Tue, Apr 26, 2011 at 9:09 PM, Mathew Yeates mat.yea...@gmail.com wrote: is scikits.image.io documented anywhere? http://stefanv.github.com/scikits.image/api/scikits.image.io.html ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/26/11 11:49 AM, Pauli Virtanen wrote: Tue, 26 Apr 2011 11:36:19 -0500, Jason Grout wrote: [clip] Okay, just one more data point. Our people that are seeing the problem with numpy returning a non-unitary V also see a non-unitary V being returned by the test C call to zgesdd. In other words, it really appears that zgesdd follows the Lapack docs, and if rowscolumns, the returned V is not necessarily unitary, but may contain a zero row. This makes numpy's assumptions in using zgesdd false. You can see this report at http://trac.sagemath.org/sage_trac/ticket/11248#comment:25 What LAPACK promises is not very clear. Earlier on the the man page: JOBZ(input) CHARACTER*1 Specifies options for computing all or part of the matrix U: = 'A': all M columns of U and all N rows of V**H are returned in the arrays U and VT; = 'S': the first min(M,N) columns of U and the first min(M,N) rows of V**H are returned in the arrays U and VT; = 'O': If M= N, the first N columns of U are over‐ written in the array A and all rows of V**H are returned in the array VT; otherwise, all columns of U are returned in the array U and the first M rows of V**H are overwritten in the array A; = 'N': no columns of U or rows of V**H are computed. Looks to me the statement should be parsed as: return_n_rows = (jobz == 'A') or (jobz == 'O' and m= n) So the current usage should be OK. (At least, as long as jobz == 'A', in the other cases, I don't think the wrapper does the right thing.) But apparently either there's a bug, or the LAPACK man page needs to be understood as you say. Ah, you're right that it makes sense to parse their statement that way too, so I'm not so sure what Lapack really is saying anymore either. If it's parsed the way you propose (which makes sense given the JOBZ description), I think it points to a bug in the Accelerate Lapack on the affected platforms, as we get the same zero row when we call the function directly from C, without numpy, python, or Sage in the middle. The updated rwork calculation makes no difference with a 3x4 matrix (both the old calculation and the new calculation give 66 in the 3x4 case), so I don't think that is affecting anything. Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/26/11 11:49 AM, Pauli Virtanen wrote: But apparently either there's a bug, or the LAPACK man page needs to be understood as you say. I've posted a question to the Lapack forum: http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=2t=2402 Thanks, Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] reading tiff images
On Tue, Apr 26, 2011 at 20:31, Daniel Lepage dplep...@gmail.com wrote: You need PIL no matter what; scipy.misc.imread, scipy.ndimage.imread, and scikits.image.io.imread all call PIL. I believe there are two pure python readers: http://code.google.com/p/pylibtiff/ http://www.lfd.uci.edu/~gohlke/code/tifffile.py.html The first can also write TIFF, but does not handle tiled TIFFs. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On Tue, 26 Apr 2011 14:52:52 -0500, Jason Grout wrote: [clip] The updated rwork calculation makes no difference with a 3x4 matrix (both the old calculation and the new calculation give 66 in the 3x4 case), so I don't think that is affecting anything. Actually, there *is* a difference for the 3*4 matrix: old = 3*3*5 + 5*3 = 60 new = 3*3*5 + 7*3 = 66 The old calculation had 5 instead of 7 in the formula --- I don't know if it was written according to an older version of the documentation, or if was is simply a bug. Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Zero row in SVD's unitary matrix on some Mac's
On 4/26/11 3:29 PM, Pauli Virtanen wrote: On Tue, 26 Apr 2011 14:52:52 -0500, Jason Grout wrote: [clip] The updated rwork calculation makes no difference with a 3x4 matrix (both the old calculation and the new calculation give 66 in the 3x4 case), so I don't think that is affecting anything. Actually, there *is* a difference for the 3*4 matrix: old = 3*3*5 + 5*3 = 60 new = 3*3*5 + 7*3 = 66 The old calculation had 5 instead of 7 in the formula --- I don't know if it was written according to an older version of the documentation, or if was is simply a bug. I was talking about our C example program, based on the example from Intel [1]. Intel already had the 3*3*5+7*3 calculation; that's what I called the old calculation. The new calculation I was referring to was the min(M,N)*max(5*min(M,N)+7,2*max(M,N)+2*min(M,N)+1) calculation from Lapack 3.3.2. Thanks, Jason [1] http://software.intel.com/sites/products/documentation/hpc/mkl/lapack/mkl_lapack_examples/zgesdd_ex.c.htm ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Memmap with multiprocessing
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes
totonixs...@gmail.com wrote:
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is
going on in the memmap.__array_finalize__ method under Windows. Can
you file a bug?
Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
totonixs...@gmail.com wrote:
Hi all,
I'm having a error using memmap objects shared among processes created
by the multprocessing module. This error only happen in Windows with
numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac
OS X it doesn't happen. This error is demonstrated by this little
example script here https://gist.github.com/929168 , and the traceback
is bellow (between traceback tags):
traceback
Process Process-1:
Traceback (most recent call last):
File C:\Python26\Lib\multiprocessing\process.py, line 232, in _bootstrap
self.run()
File C:\Python26\Lib\multiprocessing\process.py, line 88, in run
self._target(*self._args, **self._kwargs)
File C:\Documents and Settings\phamorim\Desktop\test.py, line 7, in
print_ma
trix
print matrix
File C:\Python26\Lib\site-packages\numpy\core\numeric.py, line 1379, in
arra
y_str
return array2string(a, max_line_width, precision, suppress_small, ' ',
, s
tr)
File C:\Python26\Lib\site-packages\numpy\core\arrayprint.py, line 309, in
ar
ray2string
separator, prefix)
File C:\Python26\Lib\site-packages\numpy\core\arrayprint.py, line 189, in
_a
rray2string
data = _leading_trailing(a)
File C:\Python26\Lib\site-packages\numpy\core\arrayprint.py, line 162, in
_l
eading_trailing
min(len(a), _summaryEdgeItems))]
File C:\Python26\Lib\site-packages\numpy\core\memmap.py, line 257, in
__arra
y_finalize__
self.filename = obj.filename
AttributeError: 'memmap' object has no attribute 'filename'
Exception AttributeError: AttributeError('NoneType' object has no attribute
'te
ll',) in bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0,
0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0], dtype=int16) ignored
/traceback
I don't know if it's a bug, but I thought it's is import to report
because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Memmap with multiprocessing
On Tue, Apr 26, 2011 at 4:21 PM, Ralf Gommers
ralf.gomm...@googlemail.comwrote:
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes
totonixs...@gmail.com wrote:
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug
tracker?
I see the same thing with recent master. Something very strange is
going on in the memmap.__array_finalize__ method under Windows. Can
you file a bug?
Just a confirmation: I get the error with numpy 1.5.1, Python 2.7.1 in
Windows XP, but not in Mac OSX or Linux (Ubuntu).
Warren
Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
totonixs...@gmail.com wrote:
Hi all,
I'm having a error using memmap objects shared among processes created
by the multprocessing module. This error only happen in Windows with
numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac
OS X it doesn't happen. This error is demonstrated by this little
example script here https://gist.github.com/929168 , and the traceback
is bellow (between traceback tags):
traceback
Process Process-1:
Traceback (most recent call last):
File C:\Python26\Lib\multiprocessing\process.py, line 232, in
_bootstrap
self.run()
File C:\Python26\Lib\multiprocessing\process.py, line 88, in run
self._target(*self._args, **self._kwargs)
File C:\Documents and Settings\phamorim\Desktop\test.py, line 7, in
print_ma
trix
print matrix
File C:\Python26\Lib\site-packages\numpy\core\numeric.py, line 1379,
in
arra
y_str
return array2string(a, max_line_width, precision, suppress_small, '
',
, s
tr)
File C:\Python26\Lib\site-packages\numpy\core\arrayprint.py, line
309, in
ar
ray2string
separator, prefix)
File C:\Python26\Lib\site-packages\numpy\core\arrayprint.py, line
189, in
_a
rray2string
data = _leading_trailing(a)
File C:\Python26\Lib\site-packages\numpy\core\arrayprint.py, line
162, in
_l
eading_trailing
min(len(a), _summaryEdgeItems))]
File C:\Python26\Lib\site-packages\numpy\core\memmap.py, line 257, in
__arra
y_finalize__
self.filename = obj.filename
AttributeError: 'memmap' object has no attribute 'filename'
Exception AttributeError: AttributeError('NoneType' object has no
attribute
'te
ll',) in bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0,
0, 0,
0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0,
0, 0, 0, 0, 0, 0, 0, 0], dtype=int16) ignored
/traceback
I don't know if it's a bug, but I thought it's is import to report
because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
