Re: VIF
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (Donald F. Burrill) wrote: > On 31 May 2000, Vmcw wrote: > > >>It is 10. I hope, you are talking about Variance Inflation Factor. > > >>More than 10 indicates severe multicollinearity. > Thus spake Jin Singh. And someone else (was it Dave Heiser?) retorted, > sensibly I thought, > > >And where does this magic number come from? :) > To which Tom in PA replied (possibly tongue-in-cheek?), > > Neter, Wasserman, Nachtsheim, and Kutner, of course! (or is it Wasserman, > > Kutner, Neter, and Nachtsheim or one of the other 22 permutations?). > I've heard of a Wasserman (or Wassermann?) test, but didn't think it had > to do with VIF. Dunno about all those other blokes. But apart from > argument by Appeal to Irrelevant Authority at HeadQuarters, was there > actually some _reasoning_ underlying the selection of VIF = 10, or was > it just someone's arbitrary guess Here is a direct quote from Neter et al. & cite (not much further justification for VIF>10 provided): [After a description of VIF] Diagnostic Uses The largest VIF among all X variables is often used as an indicator of the severeity of mulitcollinearity. A maximum VIF value in excess of 10 is frequently taken as an indication that multicollinearity may be unduly influencing the least squares estimates. Pp 387 in Neter J, M. H. Kutner, C. J. Nachtsheim, W. Wasserman (1996) Applied Linear Statistical Models, 4th Edition, Irwin, Chicago. Neter et al. cite two books for more detailed information on VIF's and related methods: Belsley et al. (1980) Regression Diagnostics and Belsley (1991) Conditioning diagnostics: collinearity and weak data in regression. -- Eugene D. Gallagher ECOS, UMASS/Boston Sent via Deja.com http://www.deja.com/ Before you buy. === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
On Tue, 30 May 2000 22:12:11 -0400 (EDT), Donald F. Burrill wrote: >On 31 May 2000, Vmcw wrote: > >> >>It is 10. I hope, you are talking about Variance Inflation Factor. >> >>More than 10 indicates severe multicollinearity. > >> >And where does this magic number come from? :) > > One place I have seen the magic number referenced is in Besley, Kuh, and Welsh, _Regression Diagnostics_. This is an excellent reference. Also, Belsley's _Conditioning Diagnostics_. Best, Alan === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
On Wed, 31 May 2000, jineshwar singh wrote: > --- "Donald F. Burrill" <[EMAIL PROTECTED]> wrote: Yes, I knew I'd written that... It took me a while to find it, but the sole addition I could find in your post was the statement VIF=10 is based on empirical data. If this is indeed your intended comment, where can those data, or an adequate description of them, be found? (The value "10" still looks to me like an arbitrary out-of-the-blue rule-of-thumb sort of value. I bet that if we had six digits on each hand instead of five, the value used would still be reported "10", but would mean a value 20% larger. Wonder what value the Babylonians, with their hexagesimal number system, would have used?...) > > On 31 May 2000, Vmcw wrote: > > VIF=10 is based on empirical data. > > > >>It is 10. I hope, you are talking about Variance > > Inflation Factor. > > > >>More than 10 indicates severe multicollinearity. > > > > Thus spake Jin Singh. ... < snip, the rest > Donald F. Burrill [EMAIL PROTECTED] 348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED] MSC #29, Plymouth, NH 03264 603-535-2597 184 Nashua Road, Bedford, NH 03110 603-471-7128 === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
--- "T.S. Lim" <[EMAIL PROTECTED]> wrote: > In article <000701bfca86$f831b9a0$047c6395@sprint>, > [EMAIL PROTECTED] > says... > >Simple answer-- from a college level text book. Jin > >It is 10. I hope, you are talking about Variance > Inflation Factor. More > than > >10 indicates severe multicollinearity. > > > And where does this magic number come from? :) > > > >Jin > > > >Jineshwar Singh, Coordinator, IDS > >Interdisciplinary Department > >George Brown College > >St .James campus > >[EMAIL PROTECTED] > >* > >You cannot control how others act but you can > >control how you react. > >416 -415-2089 > >http://www.gbrownc.on.ca/~jsingh > > > >- Original Message - > >From: Karen Scheltema <[EMAIL PROTECTED]> > >To: <[EMAIL PROTECTED]> > >Sent: Tuesday, May 30, 2000 4:51 PM > >Subject: VIF > > > > > >> What is the usual cutoff for saying the VIF is > too high? > >> > >> Karen Scheltema, M.A., M.S. > >> Statistician > >> HealthEast > >> 1700 University Ave W > >> St. Paul, MN 55104 > >> (651) 232-5212 fax: (651) 641-0683 > > -- > T.S. Lim > [EMAIL PROTECTED] > www.Recursive-Partitioning.com > __ > Get paid to write a review! > http://recursive-partitioning.epinions.com > > > > === > This list is open to everyone. Occasionally, less > thoughtful > people send inappropriate messages. Please DO NOT > COMPLAIN TO > THE POSTMASTER about these messages because the > postmaster has no > way of controlling them, and excessive complaints > will result in > termination of the list. > > For information about this list, including > information about the > problem of inappropriate messages and information > about how to > unsubscribe, please see the web page at > http://jse.stat.ncsu.edu/ > === > > > = J. Singh I.D.S George brown College [EMAIL PROTECTED] www.gbrownc.on.ca/~jsingh __ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
--- "Donald F. Burrill" <[EMAIL PROTECTED]> wrote: > On 31 May 2000, Vmcw wrote: > VIF=10 is based on empirical data. > > >>It is 10. I hope, you are talking about Variance > Inflation Factor. > > >>More than 10 indicates severe multicollinearity. > > Thus spake Jin Singh. And someone else (was it Dave > Heiser?) retorted, > sensibly I thought, > > > >And where does this magic number come from? :) > > To which Tom in PA replied (possibly > tongue-in-cheek?), > > > Neter, Wasserman, Nachtsheim, and Kutner, of > course! (or is it Wasserman, > > Kutner, Neter, and Nachtsheim or one of the other > 22 permutations?). > > I've heard of a Wasserman (or Wassermann?) test, but > didn't think it had > to do with VIF. Dunno about all those other blokes. > But apart from > argument by Appeal to Irrelevant Authority at > HeadQuarters, was there > actually some _reasoning_ underlying the selection > of VIF = 10, or was > it just someone's arbitrary guess (like the 10 > subjects per variable one > is supposed to have before one dares essay a factor > analysis)? > -- Don. > > > Donald F. Burrill > [EMAIL PROTECTED] > 348 Hyde Hall, Plymouth State College, > [EMAIL PROTECTED] > MSC #29, Plymouth, NH 03264 > 603-535-2597 > 184 Nashua Road, Bedford, NH 03110 > 603-471-7128 > > > > === > This list is open to everyone. Occasionally, less > thoughtful > people send inappropriate messages. Please DO NOT > COMPLAIN TO > THE POSTMASTER about these messages because the > postmaster has no > way of controlling them, and excessive complaints > will result in > termination of the list. > > For information about this list, including > information about the > problem of inappropriate messages and information > about how to > unsubscribe, please see the web page at > http://jse.stat.ncsu.edu/ > === > > > = J. Singh I.D.S George brown College [EMAIL PROTECTED] www.gbrownc.on.ca/~jsingh __ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
On 31 May 2000, Vmcw wrote: > >>It is 10. I hope, you are talking about Variance Inflation Factor. > >>More than 10 indicates severe multicollinearity. Thus spake Jin Singh. And someone else (was it Dave Heiser?) retorted, sensibly I thought, > >And where does this magic number come from? :) To which Tom in PA replied (possibly tongue-in-cheek?), > Neter, Wasserman, Nachtsheim, and Kutner, of course! (or is it Wasserman, > Kutner, Neter, and Nachtsheim or one of the other 22 permutations?). I've heard of a Wasserman (or Wassermann?) test, but didn't think it had to do with VIF. Dunno about all those other blokes. But apart from argument by Appeal to Irrelevant Authority at HeadQuarters, was there actually some _reasoning_ underlying the selection of VIF = 10, or was it just someone's arbitrary guess (like the 10 subjects per variable one is supposed to have before one dares essay a factor analysis)? -- Don. Donald F. Burrill [EMAIL PROTECTED] 348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED] MSC #29, Plymouth, NH 03264 603-535-2597 184 Nashua Road, Bedford, NH 03110 603-471-7128 === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
>>It is 10. I hope, you are talking about Variance Inflation Factor. More >than >>10 indicates severe multicollinearity. > > >And where does this magic number come from? :) > > Neter, Wasserman, Nachtsheim, and Kutner, of course! (or is it Wasserman, Kutner, Neter, and Nachtsheim or one of the other 22 permutations?). Tom in PA === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
In article <000701bfca86$f831b9a0$047c6395@sprint>, [EMAIL PROTECTED] says... > >It is 10. I hope, you are talking about Variance Inflation Factor. More than >10 indicates severe multicollinearity. And where does this magic number come from? :) >Jin > >Jineshwar Singh, Coordinator, IDS >Interdisciplinary Department >George Brown College >St .James campus >[EMAIL PROTECTED] >* >You cannot control how others act but you can >control how you react. >416 -415-2089 >http://www.gbrownc.on.ca/~jsingh > >- Original Message - >From: Karen Scheltema <[EMAIL PROTECTED]> >To: <[EMAIL PROTECTED]> >Sent: Tuesday, May 30, 2000 4:51 PM >Subject: VIF > > >> What is the usual cutoff for saying the VIF is too high? >> >> Karen Scheltema, M.A., M.S. >> Statistician >> HealthEast >> 1700 University Ave W >> St. Paul, MN 55104 >> (651) 232-5212 fax: (651) 641-0683 -- T.S. Lim [EMAIL PROTECTED] www.Recursive-Partitioning.com __ Get paid to write a review! http://recursive-partitioning.epinions.com === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
Karen Scheltema wrote in message <[EMAIL PROTECTED]>... >What is the usual cutoff for saying the VIF is too high? I don't see that there can be any general criterion for saying that a VIF is too large. A large value indicates collinearity between predictor variables. In some fields, this cannot be avoided. I have one set of set for which most of the VIFs are in excess of a million. The data are from NIR spectroscopy, where this is unavoidable. If you do have large VIFs then make sure that your least-squares software uses some form of orthogonal reduction. If it uses the normal equations, and hence squares the condition number, then you could be in trouble. > >Karen Scheltema, M.A., M.S. >Statistician >HealthEast >1700 University Ave W >St. Paul, MN 55104 >(651) 232-5212 fax: (651) 641-0683 > -- Alan Miller, Retired Scientist (Statistician) CSIRO Mathematical & Information Sciences Alan.Miller -at- vic.cmis.csiro.au http://www.ozemail.com.au/~milleraj http://users.bigpond.net.au/amiller/ === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
RE: VIF
On Tue, 30 May 2000, Dale Glaser wrote: > Karen..off the top of my head, the VIF is the inverse of tolerance, > hence, if tolerance = (1 - r^2j), then VIF = 1/(1-r^2j).. Yes, Dale is correct. > ... r^2j would be the percentage of variation accounted for by the > predictors in predicting the other predictor.. e.g., the linear > combination of x1 and x2 in predicting x3; > anyway, as with any cutoff value there can be an element of > arbitrariness, Indeed. > though some have registered concern if VIF > 10.0; my personal opinion > is that the aforementioned cutoff value is way too liberal; I agree, if one is using the idea of "cutoff"; though possibly I am thinking of "conservative" rather than "liberal", since I have seen (and dealt with) VIFs exceeding several hundreds. They don't frighten me particularly, partly because by orthogonalizing they can be reduced to manageable levels. Even partly orthogonalizing can reduce VIFs to values like 2 or 1.5, at least in some circumstances. > for VIF to equal 10.0 then 1/=(1 - .9) entails a multiple R of > .9486!!!; for me it is a stretch to conceive that collinearity only > becomes problematic when R = .9486...I'll be interested to see what > others think Strictly speaking, "multicollinearity" implies R = 1.000, I believe. (I don't know why Dale calculates R; the effective information is that [with VIF = 10] R^2 = 0.9, and 10% of the original variance in the predictor remains unaccounted for. As one of our colleagues (Rich Ulrich, I think) recently remarked in another context, with R^2 values this large one may often usefully consider their complementary values (1 - R^2).) Most computer regression programs have a control based on tolerance (the reciprocal of VIF); I believe Minitab's default tolerance threshold is around 0.0001 or 0.0002, implying VIFs of 10,000 or 5,000 respectively. This of course is not to be taken as an indication of "good practice", but of where the systems analysts thought the algorithm was in danger of breaking down: "severe multicollinearity" indeed. But a lurking question, as my earlier post may have suggested, is whether the multicollinearity apparently present is inherent in the nature of the variables, or an artifact of variable construction. The latter was the case in the problem addressed in the paper on the Minitab web site. Karen's original question was: > What is the usual cutoff for saying the VIF is too high? Depends on the purpose for which you think you want a "cutoff", and whether you propose to implement it blindly and without further thought, or as a (very!) rough guideline regarding where the currents (and perhaps the undertow) may be dangerous and REQUIRE further thought; just for two examples. -- Don. Donald F. Burrill [EMAIL PROTECTED] 348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED] MSC #29, Plymouth, NH 03264 603-535-2597 184 Nashua Road, Bedford, NH 03110 603-471-7128 === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: VIF
It is 10. I hope, you are talking about Variance Inflation Factor. More than 10 indicates severe multicollinearity. Jin Jineshwar Singh, Coordinator, IDS Interdisciplinary Department George Brown College St .James campus [EMAIL PROTECTED] * You cannot control how others act but you can control how you react. 416 -415-2089 http://www.gbrownc.on.ca/~jsingh - Original Message - From: Karen Scheltema <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, May 30, 2000 4:51 PM Subject: VIF > What is the usual cutoff for saying the VIF is too high? > > Karen Scheltema, M.A., M.S. > Statistician > HealthEast > 1700 University Ave W > St. Paul, MN 55104 > (651) 232-5212 fax: (651) 641-0683 > > > Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com > > > > === > This list is open to everyone. Occasionally, less thoughtful > people send inappropriate messages. Please DO NOT COMPLAIN TO > THE POSTMASTER about these messages because the postmaster has no > way of controlling them, and excessive complaints will result in > termination of the list. > > For information about this list, including information about the > problem of inappropriate messages and information about how to > unsubscribe, please see the web page at > http://jse.stat.ncsu.edu/ > === > === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
RE: VIF
Karen..off the top of my head, the VIF is the inverse of tolerance, hence, if tolerance = (1 - r^2j), then VIF= 1/(1-r^2j)..[excuse the sloppiness of the notation, but r^2j would be the percentage of variation accounted for by the predictors in predicting the other predictor..ie., the linear combination of x1 and x2 in predicting x3]; anyway, as with any cutoff value there can be an element of arbitrariness, though some have registered concern if VIF > 10.0; my personal (possibly misinformed!) opinion is that the aforementioned cutoff value is way too liberal; for VIF to equal 10.0 then 1/=(1 - .9) entails a multiple R of .9486!!!; for me it is a stretch to conceive that collinearity only becomes problematic when R = .9486...I'll be interested to see what others think Dale Glaser, Ph.D. Senior Statistician, Pacific Science and Engineering Group Adjunct faculty/lecturer, SDSU/USD/CSPP San Diego, CA. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Karen Scheltema Sent: Tuesday, May 30, 2000 1:52 PM To: [EMAIL PROTECTED] Subject: VIF What is the usual cutoff for saying the VIF is too high? Karen Scheltema, M.A., M.S. Statistician HealthEast 1700 University Ave W St. Paul, MN 55104 (651) 232-5212 fax: (651) 641-0683 Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ === === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
VIF
What is the usual cutoff for saying the VIF is too high? Karen Scheltema, M.A., M.S. Statistician HealthEast 1700 University Ave W St. Paul, MN 55104 (651) 232-5212 fax: (651) 641-0683 Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
