Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Mark Janikas]

When I say garbage, I mean in the context of my hypothesis testing when in the 
presence of perfect multicollinearity.  I advise the user of the combination 
that leads to the problem and move on 

-Original Message-
From: numpy-discussion-boun...@scipy.org 
[mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Bruce Southey
Sent: Wednesday, August 31, 2011 11:11 AM
To: numpy-discussion@scipy.org
Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

On 08/31/2011 12:56 PM, Mark Janikas wrote:
> Right indeed... I have spent a lot of time looking at this and it seems a 
> waste of time as the results are garbage anyways when the columns are 
> collinear.  I am just going to set a threshold, check the condition number, 
> continue is satisfied, return error/warning if not now, what is too 
> large? Ill poke around.  TY!
>
> MJ
The results are not 'garbage' as if you have collinear columns as these 
have very well-known and understandable meaning. But if you don't expect 
this then you really need to examine how you are modeling or measuring 
your data because that is where the problem lies. For example, if you 
are measuring two variables then it means that those measurements are 
not independent as you are assuming.

Bruce

> -Original Message-
> From: numpy-discussion-boun...@scipy.org 
> [mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Pauli Virtanen
> Sent: Wednesday, August 31, 2011 2:00 AM
> To: numpy-discussion@scipy.org
> Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm
>
> On Tue, 30 Aug 2011 15:48:18 -0700, Mark Janikas wrote:
>> Last week I posted a question involving the identification of linear
>> dependent columns of a matrix... but now I am finding an interesting
>> result based on the linalg.inv() function... sometime I am able to
>> invert a matrix that has linear dependent columns and other times I get
>> the LinAlgError()... this suggests that there is some kind of random
>> component to the INV method.  Is this normal?
> I suspect that this is a case of floating-point rounding errors.
> Floating-point arithmetic is inexact, so even if a certain matrix
> is singular in exact arithmetic, for a computer it may still be
> invertible (by a given algorithm). This type of things are not
> unusual in floating-point computations.
>
> The matrix condition number (`np.linalg.cond`) is a better measure
> of whether a matrix is invertible or not.
>

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Bruce Southey]

On 08/31/2011 12:56 PM, Mark Janikas wrote:
> Right indeed... I have spent a lot of time looking at this and it seems a 
> waste of time as the results are garbage anyways when the columns are 
> collinear.  I am just going to set a threshold, check the condition number, 
> continue is satisfied, return error/warning if not now, what is too 
> large? Ill poke around.  TY!
>
> MJ
The results are not 'garbage' as if you have collinear columns as these 
have very well-known and understandable meaning. But if you don't expect 
this then you really need to examine how you are modeling or measuring 
your data because that is where the problem lies. For example, if you 
are measuring two variables then it means that those measurements are 
not independent as you are assuming.

Bruce

> -Original Message-
> From: numpy-discussion-boun...@scipy.org 
> [mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Pauli Virtanen
> Sent: Wednesday, August 31, 2011 2:00 AM
> To: numpy-discussion@scipy.org
> Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm
>
> On Tue, 30 Aug 2011 15:48:18 -0700, Mark Janikas wrote:
>> Last week I posted a question involving the identification of linear
>> dependent columns of a matrix... but now I am finding an interesting
>> result based on the linalg.inv() function... sometime I am able to
>> invert a matrix that has linear dependent columns and other times I get
>> the LinAlgError()... this suggests that there is some kind of random
>> component to the INV method.  Is this normal?
> I suspect that this is a case of floating-point rounding errors.
> Floating-point arithmetic is inexact, so even if a certain matrix
> is singular in exact arithmetic, for a computer it may still be
> invertible (by a given algorithm). This type of things are not
> unusual in floating-point computations.
>
> The matrix condition number (`np.linalg.cond`) is a better measure
> of whether a matrix is invertible or not.
>

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Mark Janikas]

Right indeed... I have spent a lot of time looking at this and it seems a waste 
of time as the results are garbage anyways when the columns are collinear.  I 
am just going to set a threshold, check the condition number, continue is 
satisfied, return error/warning if not now, what is too large? Ill poke 
around.  TY!

MJ 

-Original Message-
From: numpy-discussion-boun...@scipy.org 
[mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Pauli Virtanen
Sent: Wednesday, August 31, 2011 2:00 AM
To: numpy-discussion@scipy.org
Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

On Tue, 30 Aug 2011 15:48:18 -0700, Mark Janikas wrote:
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix... but now I am finding an interesting
> result based on the linalg.inv() function... sometime I am able to
> invert a matrix that has linear dependent columns and other times I get
> the LinAlgError()... this suggests that there is some kind of random
> component to the INV method.  Is this normal?

I suspect that this is a case of floating-point rounding errors.
Floating-point arithmetic is inexact, so even if a certain matrix
is singular in exact arithmetic, for a computer it may still be
invertible (by a given algorithm). This type of things are not
unusual in floating-point computations.

The matrix condition number (`np.linalg.cond`) is a better measure
of whether a matrix is invertible or not.

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Pauli Virtanen]

On Tue, 30 Aug 2011 15:48:18 -0700, Mark Janikas wrote:
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix... but now I am finding an interesting
> result based on the linalg.inv() function... sometime I am able to
> invert a matrix that has linear dependent columns and other times I get
> the LinAlgError()... this suggests that there is some kind of random
> component to the INV method.  Is this normal?

I suspect that this is a case of floating-point rounding errors.
Floating-point arithmetic is inexact, so even if a certain matrix
is singular in exact arithmetic, for a computer it may still be
invertible (by a given algorithm). This type of things are not
unusual in floating-point computations.

The matrix condition number (`np.linalg.cond`) is a better measure
of whether a matrix is invertible or not.

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Mark Janikas]

OK... so I have been using checksums to compare and it looks like I am getting 
a different value when it fails as opposed to when it passes... I.e. the input 
is NOT the same.  When I save them to npy files and run LA.inv() I get 
consistent results.  Now I have to track down in my code why the inputs are 
different Sucks, because I keep having to dive deeper (more checksums... 
yeh!).  But it is all linear algebra from the same input, so kinda weird that 
there is a diversion. Thanks for all of your help! And Ill post again when I 
find the culprit. (probably me :-))

MJ

-Original Message-
From: numpy-discussion-boun...@scipy.org 
[mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Robert Kern
Sent: Tuesday, August 30, 2011 4:42 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

On Tue, Aug 30, 2011 at 17:48, Mark Janikas  wrote:
> Hello All,
>
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix… but now I am finding an interesting result
> based on the linalg.inv() function… sometime I am able to invert a matrix
> that has linear dependent columns and other times I get the LinAlgError()…
> this suggests that there is some kind of random component to the INV
> method.  Is this normal?  Thanks much ahead of time,

We will also need to know the platform that you are on as well as the
LAPACK library that you linked numpy against. It is the behavior of
that LAPACK library that is controlling here. Standard LAPACK does
sometimes use pseudorandom numbers in certain situations, but AFAICT
it deterministically seeds the PRNG on every call, and I don't think
it does this for any subroutine involved with inversion. But if you
use an optimized LAPACK from some vendor, I don't know what they may
be doing. Some optimized LAPACK/BLAS libraries may be threaded and may
dynamically determine how to break up the problem based on load (I
don't know of any that specifically do this, but it's a possibility).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Robert Kern]

On Tue, Aug 30, 2011 at 17:48, Mark Janikas  wrote:
> Hello All,
>
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix… but now I am finding an interesting result
> based on the linalg.inv() function… sometime I am able to invert a matrix
> that has linear dependent columns and other times I get the LinAlgError()…
> this suggests that there is some kind of random component to the INV
> method.  Is this normal?  Thanks much ahead of time,

We will also need to know the platform that you are on as well as the
LAPACK library that you linked numpy against. It is the behavior of
that LAPACK library that is controlling here. Standard LAPACK does
sometimes use pseudorandom numbers in certain situations, but AFAICT
it deterministically seeds the PRNG on every call, and I don't think
it does this for any subroutine involved with inversion. But if you
use an optimized LAPACK from some vendor, I don't know what they may
be doing. Some optimized LAPACK/BLAS libraries may be threaded and may
dynamically determine how to break up the problem based on load (I
don't know of any that specifically do this, but it's a possibility).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Robert Kern]

On Tue, Aug 30, 2011 at 18:34, Mark Janikas  wrote:
> When I export to ascii I am losing precision and it getting consistency... I 
> will try a flat dump.  More to come.  TY

Might as well np.save() it to an .npy binary file and attach it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Mark Janikas]

When I export to ascii I am losing precision and it getting consistency... I 
will try a flat dump.  More to come.  TY

MJ

-Original Message-
From: numpy-discussion-boun...@scipy.org 
[mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Mark Janikas
Sent: Tuesday, August 30, 2011 4:02 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

Working on it... Give me a few minutes to get you the data.  TY!

MJ

-Original Message-
From: numpy-discussion-boun...@scipy.org 
[mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Christopher 
Jordan-Squire
Sent: Tuesday, August 30, 2011 3:57 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

Can you give an example matrix? I'm not a numerical linear algebra
expert, but I suspect that if your matrix is singular (or nearly so,
in floating point) then any inverse given will look pretty wonky. Huge
determinant, eigenvalues, operator norm, etc..

-Chris JS

On Tue, Aug 30, 2011 at 5:48 PM, Mark Janikas  wrote:
> Hello All,
>
>
>
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix. but now I am finding an interesting result
> based on the linalg.inv() function. sometime I am able to invert a matrix
> that has linear dependent columns and other times I get the LinAlgError().
> this suggests that there is some kind of random component to the INV
> method.  Is this normal?  Thanks much ahead of time,
>
>
>
> MJ
>
>
>
> Mark Janikas
>
> Product Developer
>
> ESRI, Geoprocessing
>
> 380 New York St.
>
> Redlands, CA 92373
>
> 909-793-2853 (2563)
>
> mjani...@esri.com
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Mark Janikas]

Working on it... Give me a few minutes to get you the data.  TY!

MJ

-Original Message-
From: numpy-discussion-boun...@scipy.org 
[mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Christopher 
Jordan-Squire
Sent: Tuesday, August 30, 2011 3:57 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

Can you give an example matrix? I'm not a numerical linear algebra
expert, but I suspect that if your matrix is singular (or nearly so,
in floating point) then any inverse given will look pretty wonky. Huge
determinant, eigenvalues, operator norm, etc..

-Chris JS

On Tue, Aug 30, 2011 at 5:48 PM, Mark Janikas  wrote:
> Hello All,
>
>
>
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix. but now I am finding an interesting result
> based on the linalg.inv() function. sometime I am able to invert a matrix
> that has linear dependent columns and other times I get the LinAlgError().
> this suggests that there is some kind of random component to the INV
> method.  Is this normal?  Thanks much ahead of time,
>
>
>
> MJ
>
>
>
> Mark Janikas
>
> Product Developer
>
> ESRI, Geoprocessing
>
> 380 New York St.
>
> Redlands, CA 92373
>
> 909-793-2853 (2563)
>
> mjani...@esri.com
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Christopher Jordan-Squire]

Can you give an example matrix? I'm not a numerical linear algebra
expert, but I suspect that if your matrix is singular (or nearly so,
in floating point) then any inverse given will look pretty wonky. Huge
determinant, eigenvalues, operator norm, etc..

-Chris JS

On Tue, Aug 30, 2011 at 5:48 PM, Mark Janikas  wrote:
> Hello All,
>
>
>
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix… but now I am finding an interesting result
> based on the linalg.inv() function… sometime I am able to invert a matrix
> that has linear dependent columns and other times I get the LinAlgError()…
> this suggests that there is some kind of random component to the INV
> method.  Is this normal?  Thanks much ahead of time,
>
>
>
> MJ
>
>
>
> Mark Janikas
>
> Product Developer
>
> ESRI, Geoprocessing
>
> 380 New York St.
>
> Redlands, CA 92373
>
> 909-793-2853 (2563)
>
> mjani...@esri.com
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question on LinAlg Inverse Algorithm

[Robert Kern]

On Tue, Aug 30, 2011 at 17:48, Mark Janikas  wrote:
> Hello All,
>
> Last week I posted a question involving the identification of linear
> dependent columns of a matrix… but now I am finding an interesting result
> based on the linalg.inv() function… sometime I am able to invert a matrix
> that has linear dependent columns and other times I get the LinAlgError()…
> this suggests that there is some kind of random component to the INV
> method.  Is this normal?  Thanks much ahead of time,

With exactly the same input in the same process? Can you provide that input?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Question on LinAlg Inverse Algorithm

[Mark Janikas]

Hello All,

Last week I posted a question involving the identification of linear dependent 
columns of a matrix... but now I am finding an interesting result based on the 
linalg.inv() function... sometime I am able to invert a matrix that has linear 
dependent columns and other times I get the LinAlgError()... this suggests that 
there is some kind of random component to the INV method.  Is this normal?  
Thanks much ahead of time,

MJ

Mark Janikas
Product Developer
ESRI, Geoprocessing
380 New York St.
Redlands, CA 92373
909-793-2853 (2563)
mjani...@esri.com

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion