THANKS Re: [MORPHMET] using regression residuals for other analyses
Dear morphometricians, many thanks to all those who either in emails directly to me or to morphmet contributed to this discussion. It was very interesting and instructive. I have a lot of sympathy for Ian's point about what to do when one uses an ANCOVA to test slopes and finds that it is significant but the difference in the fit of the a model with separate lines compared to one with parallel is tiny. I had at least a few times when that happened to me. With short trajectories (say, static allometry in mammal crania), I'd be very cautious. In my experience, when trajectories are short, one needs very but really very large samples to trust the estimates. With longer trajectories and much more allometric variance (as often in ontogenetic allometry), besides checking the angles, if one really wants to try a 'size-correction' despite all the issues raised by Dean, Joe et al., I would possibly also explore the sensitivity of the 'correction' to different choices of size value used for the correction: if the divergence of the trajectories is really small (despite significance), I would expect results to be robust to the use of the smallest, largest or average size in the samples. Again, thanks a lot for your feedback. Cheers Andrea On 24/03/2016 16:12, Ian Dworkin wrote: Dean and Andrea, I wanted to follow up on what Dean wrote regarding using residuals from a pooled within-group regression, and what I think may be important discussion that follows from it. Considerable research has gone into investigating this issue, and as Dean points out, most of the time it is best to include the additional predictor (let's just use centroid size) in the model and fit it with the shape ~ group + size + group:size term. Indeed, I think we could all find 10-15 different papers (each) that discuss this issue (and a few of them pertaining to geometric morphometrics). However, there are some common cases in geometric morphometrics that I think many of us deal with, and at least to my mind we do not have a very satisfactory guide to deal with some of them. Let's imagine a case that I think is particularly common in geometric morphometric studies, where we are examining sexual shape dimorphism, where we have sex as a categorical predictor as well as centroid size. So we might start with the model shape ~ sex + size + sex:size Geometric morphometric analyses are pretty sensitive, and at least with some systems (like fly wings) sample sizes tend to be relatively high. Frequently I have observed that the evidence is not consistent (based on Null Hypothesis Statistical Testing, NHST) with a common allometric relationship between the two sexes. Indeed since NHST (and assessment of significance) is in part a function on sample size, with large enough N, this term will be significant (even if the magnitude of effect is very small). Thus (as Dean has already clearly laid out) it may be unreasonable to use a pooled within-group regression and use the residuals (so that you can separate out allometric from non-allometric components of sexual shape dimorphism for instance). However, if you go ahead and examine the vector correlations/angle between the slopes (shape ~ size) across sexes you will observe that the vector correlation is ~1 (angle is ~0). Similarly the partial coefficient of determination (r^2) for the size:sex term is quite small relative to the partial r^2 for the marginal contributions of size and sex. Thus despite the NHST suggesting a lack of a common allometric relationship, this "deeper" examination suggests the slopes are very similar. So what do you do (again if you want to partition the allometric and non-allometric components of shape variation)? if the vector correlation is 0.99 do you decide they are effectively the same and proceed with pooled within-group regression to extract residuals? How about if the VC is 0.95? 0.9? At what point do you risk causing substantial inferential problems? Or do you alternatively not try to use a pooled within-group regression at all, and instead just predict shapes for males or females at particular centroid sizes given the full model (sex + size + sex:size), so you can get a sense of the extent of sexual shape dimorphism for comparable sizes (or whatever your goals might be). While I do not expect any hard and fast rules, I am wondering if anyone has done the relevant simulations to look at when the former (residuals from pooled within-group regression) becomes substantially problematic (in terms of magnitude of the sex:size interaction term). While I can quibble and be a pedant (who among us GMers are not!), I think the paper by Nelly, Michel and Chris is very useful (but does not get into the issue about when using residuals from pooled regression is problematic). N. A. Gidaszewski, M. Baylac, and C. P. Klingenberg, “Evolution of sexual dimorphism of wing shape in the Drosophila melanogaster subgroup.,” /BMC Evol Biol/, vol. 9, p. 110, 200
Fwd: [MORPHMET] using regression residuals for other analyses
Hi Andrea, I think the paper that Dr. Adams may be referring to is: García‐Berthou E, 2001. On the misuse of residuals in ecology: testing regression residuals vs. the analysis of covariance. Journal of Animal Ecology, 70(4): 708-711. You can access it at: http://invasiber.org/EGarcia/papers/JAE%2070,%20708-711.pdf There are also quite a few papers in the ecology literature discussing the problem in relation to body condition residuals. The general tune is exactly what Dr. Adams explained: apart from the statistical underpinnings, ANCOVA works better and provides more information than operating with the residuals. Apart from avoiding most problems related to the selection of the regression model (particularly if there are common slopes, as can be expected in many cases), the results are way more meaningful and easy to interpret and illustrate. LC On Mar 24, 2016, at 8:21 AM, Adams, Dean [EEOBS] mailto:dcad...@iastate.edu>> wrote: Hi Andrea, It is generally preferable to perform the more complex analysis with size included as a covariate. Using a sequential approach that first obtains the shape residuals and then examines patterns using these as data is not guaranteed to get to the same, or even the correct, place. And this approach can leave potentially important biology out. Consider the simplest case with shape, size, and groups (i.e., mancova). Here the full model is: shape~size+group+size:group. Say for instance, that one finds a significant interaction term. This means that the groups have different shape~size relationships (ie different allometric slopes). In this case, using residuals from a shape~size regression in a subsequent manova is not correct, as these are residuals from a common-slope model, whereas the mancova has found evidence that the groups have different slopes. Thus the residuals are not capturing what one intends. (As a side note there was a nice paper in the mid-1990s on the univariate equivalent of this, describing why anova of regression residuals is not the same as ancova). But additionally, using the sequential-analysis approach eliminates the possibility of identifying interesting interactions between effects that one had not considered. Again take this simple example. Here, performing a manova on the regression residuals is intended to evaluate differences in the mean shapes among groups. But this explicitly ignores the possibility that the groups may differ in their allometries themselves, rather than their size-adjusted least squares means. Such allometric differences represent potentially important biological information that is left unexplored using the piecewise analysis procedure. For these reasons the analysis including size as a covariate is preferred. And while it is more complicated to consider models that include interactions, and various post-hoc comparisons are required (of group means, of slopes, etc.), one ought to do so when possible, so as to properly identify where patterns of shape variation occur, and what potential factors associate with it. Dean Dr. Dean C. Adams Professor Department of Ecology, Evolution, and Organismal Biology Department of Statistics Iowa State University www.public.iastate.edu/~dcadams/<http://www.public.iastate.edu/~dcadams/> phone: 515-294-3834 -Original Message- From: andrea cardini [mailto:alcard...@gmail.com] Sent: Thursday, March 24, 2016 12:01 PM To: morphmet@morphometrics.org<mailto:morphmet@morphometrics.org> Subject: [MORPHMET] using regression residuals for other analyses Dear All, this is something that, I believe, has already come up in the past. However, I'd like to check it again. What are the issues with, say, regressing shape on size, saving residuals and using those in further analyses (e.g., other regressions or testing group differences etc.)? I suspect that all the factors (size, other predictors, groups etc.) should be incorporated in a single model and may have a partial intuition about some of the problems with rerunning, instead, analyses on residuals but I'd be very grateful to know how those with a better understanding of the methods see it. Thanks in advance. Cheers Andrea -- Dr. Andrea Cardini Researcher, Dipartimento di Scienze Chimiche e Geologiche, Università di Modena e Reggio Emilia, Via Campi, 103 - 41125 Modena - Italy tel. 0039 059 2058472 Adjunct Associate Professor, Centre for Forensic Science , The University of Western Australia, 35 Stirling Highway, Crawley WA 6009, Australia E-mail address: alcard...@gmail.com<mailto:alcard...@gmail.com>, andrea.card...@unimore.it<mailto:andrea.card...@unimore.it> WEBPAGE: https://sites.google.com/site/alcardini/home/main FREE Yellow BOOK on Geometric Morphometrics: http://www.italian-journal-of-mammalogy.it/public/journals/3/issue_241_complete_100.pdf -- MORPHMET may be accessed via its webpage at http://www.morphometrics.org -
Re: [MORPHMET] using regression residuals for other analyses
These complicated issues aside, there is a simpler reason not to use a residual from a regression as the basis for further analyses. If two characters are under natural selection based on some combination -- such as selection on a ratio between them -- then the current value of character Y is a response, not to the current value of character X, but to its value in the past. Response to selection is not instant, so we'd really want to regress Y on past values of X. How far in the past depends on information on the strength of selection. We don't yet know that, and we don't have values of X in the past. Far better to make a joint analysis of selection on both of them. Once one takes the residual one has built in the assumption that the response of one character to another is instantaneous, in effect that the selection involved is infinitely strong and the heritabilities complete. I believe that Hansen and Bartoszek have warned about this in a paper in Systematic Biology in 2012. Joe Joe Felsenstein j...@gs.washington.edu Department of Genome Sciences and Department of Biology, University of Washington, Box 355065, Seattle, WA 98195-5065 USA -- MORPHMET may be accessed via its webpage at http://www.morphometrics.org --- You received this message because you are subscribed to the Google Groups "MORPHMET" group. To unsubscribe from this group and stop receiving emails from it, send an email to morphmet+unsubscr...@morphometrics.org.
Re: [MORPHMET] using regression residuals for other analyses
Just as a quick follow up (and clarifications). I recognize that there are already a number of methods to deal with these well known issues (McCoy et al. 2006, Klingenberg 1996, Burnaby 1966, etc..) and they may be better ways still. However, since the "using residuals from a pooled within-group regression" remains so common, I thought the broader conversation may be useful. Also I did not mean that everyone will always observe very high vector correlations for the allometric slope of shape on sex (across sexes) or small partial R^2. I was trying to make a bit of a caricature of an example to make my point. Sorry for any confusion. Cheers, Ian dwor...@mcmaster.ca -- MORPHMET may be accessed via its webpage at http://www.morphometrics.org --- You received this message because you are subscribed to the Google Groups "MORPHMET" group. To unsubscribe from this group and stop receiving emails from it, send an email to morphmet+unsubscr...@morphometrics.org.
Re: [MORPHMET] using regression residuals for other analyses
Dean and Andrea, I wanted to follow up on what Dean wrote regarding using residuals from a pooled within-group regression, and what I think may be important discussion that follows from it. Considerable research has gone into investigating this issue, and as Dean points out, most of the time it is best to include the additional predictor (let's just use centroid size) in the model and fit it with the shape ~ group + size + group:size term. Indeed, I think we could all find 10-15 different papers (each) that discuss this issue (and a few of them pertaining to geometric morphometrics). However, there are some common cases in geometric morphometrics that I think many of us deal with, and at least to my mind we do not have a very satisfactory guide to deal with some of them. Let's imagine a case that I think is particularly common in geometric morphometric studies, where we are examining sexual shape dimorphism, where we have sex as a categorical predictor as well as centroid size. So we might start with the model shape ~ sex + size + sex:size Geometric morphometric analyses are pretty sensitive, and at least with some systems (like fly wings) sample sizes tend to be relatively high. Frequently I have observed that the evidence is not consistent (based on Null Hypothesis Statistical Testing, NHST) with a common allometric relationship between the two sexes. Indeed since NHST (and assessment of significance) is in part a function on sample size, with large enough N, this term will be significant (even if the magnitude of effect is very small). Thus (as Dean has already clearly laid out) it may be unreasonable to use a pooled within-group regression and use the residuals (so that you can separate out allometric from non-allometric components of sexual shape dimorphism for instance). However, if you go ahead and examine the vector correlations/angle between the slopes (shape ~ size) across sexes you will observe that the vector correlation is ~1 (angle is ~0). Similarly the partial coefficient of determination (r^2) for the size:sex term is quite small relative to the partial r^2 for the marginal contributions of size and sex. Thus despite the NHST suggesting a lack of a common allometric relationship, this "deeper" examination suggests the slopes are very similar. So what do you do (again if you want to partition the allometric and non-allometric components of shape variation)? if the vector correlation is 0.99 do you decide they are effectively the same and proceed with pooled within-group regression to extract residuals? How about if the VC is 0.95? 0.9? At what point do you risk causing substantial inferential problems? Or do you alternatively not try to use a pooled within-group regression at all, and instead just predict shapes for males or females at particular centroid sizes given the full model (sex + size + sex:size), so you can get a sense of the extent of sexual shape dimorphism for comparable sizes (or whatever your goals might be). While I do not expect any hard and fast rules, I am wondering if anyone has done the relevant simulations to look at when the former (residuals from pooled within-group regression) becomes substantially problematic (in terms of magnitude of the sex:size interaction term). While I can quibble and be a pedant (who among us GMers are not!), I think the paper by Nelly, Michel and Chris is very useful (but does not get into the issue about when using residuals from pooled regression is problematic). N. A. Gidaszewski, M. Baylac, and C. P. Klingenberg, “Evolution of sexual dimorphism of wing shape in the Drosophila melanogaster subgroup.,” *BMC Evol Biol*, vol. 9, p. 110, 2009. I hope this leads to useful discussion! Cheers Ian dwor...@mcmaster.ca -- MORPHMET may be accessed via its webpage at http://www.morphometrics.org --- You received this message because you are subscribed to the Google Groups "MORPHMET" group. To unsubscribe from this group and stop receiving emails from it, send an email to morphmet+unsubscr...@morphometrics.org.
RE: [MORPHMET] using regression residuals for other analyses
Hi Andrea, It is generally preferable to perform the more complex analysis with size included as a covariate. Using a sequential approach that first obtains the shape residuals and then examines patterns using these as data is not guaranteed to get to the same, or even the correct, place. And this approach can leave potentially important biology out. Consider the simplest case with shape, size, and groups (i.e., mancova). Here the full model is: shape~size+group+size:group. Say for instance, that one finds a significant interaction term. This means that the groups have different shape~size relationships (ie different allometric slopes). In this case, using residuals from a shape~size regression in a subsequent manova is not correct, as these are residuals from a common-slope model, whereas the mancova has found evidence that the groups have different slopes. Thus the residuals are not capturing what one intends. (As a side note there was a nice paper in the mid-1990s on the univariate equivalent of this, describing why anova of regression residuals is not the same as ancova). But additionally, using the sequential-analysis approach eliminates the possibility of identifying interesting interactions between effects that one had not considered. Again take this simple example. Here, performing a manova on the regression residuals is intended to evaluate differences in the mean shapes among groups. But this explicitly ignores the possibility that the groups may differ in their allometries themselves, rather than their size-adjusted least squares means. Such allometric differences represent potentially important biological information that is left unexplored using the piecewise analysis procedure. For these reasons the analysis including size as a covariate is preferred. And while it is more complicated to consider models that include interactions, and various post-hoc comparisons are required (of group means, of slopes, etc.), one ought to do so when possible, so as to properly identify where patterns of shape variation occur, and what potential factors associate with it. Dean Dr. Dean C. Adams Professor Department of Ecology, Evolution, and Organismal Biology Department of Statistics Iowa State University www.public.iastate.edu/~dcadams/ phone: 515-294-3834 -Original Message- From: andrea cardini [mailto:alcard...@gmail.com] Sent: Thursday, March 24, 2016 12:01 PM To: morphmet@morphometrics.org Subject: [MORPHMET] using regression residuals for other analyses Dear All, this is something that, I believe, has already come up in the past. However, I'd like to check it again. What are the issues with, say, regressing shape on size, saving residuals and using those in further analyses (e.g., other regressions or testing group differences etc.)? I suspect that all the factors (size, other predictors, groups etc.) should be incorporated in a single model and may have a partial intuition about some of the problems with rerunning, instead, analyses on residuals but I'd be very grateful to know how those with a better understanding of the methods see it. Thanks in advance. Cheers Andrea -- Dr. Andrea Cardini Researcher, Dipartimento di Scienze Chimiche e Geologiche, Università di Modena e Reggio Emilia, Via Campi, 103 - 41125 Modena - Italy tel. 0039 059 2058472 Adjunct Associate Professor, Centre for Forensic Science , The University of Western Australia, 35 Stirling Highway, Crawley WA 6009, Australia E-mail address: alcard...@gmail.com, andrea.card...@unimore.it WEBPAGE: https://sites.google.com/site/alcardini/home/main FREE Yellow BOOK on Geometric Morphometrics: http://www.italian-journal-of-mammalogy.it/public/journals/3/issue_241_complete_100.pdf -- MORPHMET may be accessed via its webpage at http://www.morphometrics.org --- You received this message because you are subscribed to the Google Groups "MORPHMET" group. To unsubscribe from this group and stop receiving emails from it, send an email to morphmet+unsubscr...@morphometrics.org. -- MORPHMET may be accessed via its webpage at http://www.morphometrics.org --- You received this message because you are subscribed to the Google Groups "MORPHMET" group. To unsubscribe from this group and stop receiving emails from it, send an email to morphmet+unsubscr...@morphometrics.org.
[MORPHMET] using regression residuals for other analyses
Dear All, this is something that, I believe, has already come up in the past. However, I'd like to check it again. What are the issues with, say, regressing shape on size, saving residuals and using those in further analyses (e.g., other regressions or testing group differences etc.)? I suspect that all the factors (size, other predictors, groups etc.) should be incorporated in a single model and may have a partial intuition about some of the problems with rerunning, instead, analyses on residuals but I'd be very grateful to know how those with a better understanding of the methods see it. Thanks in advance. Cheers Andrea -- Dr. Andrea Cardini Researcher, Dipartimento di Scienze Chimiche e Geologiche, Università di Modena e Reggio Emilia, Via Campi, 103 - 41125 Modena - Italy tel. 0039 059 2058472 Adjunct Associate Professor, Centre for Forensic Science , The University of Western Australia, 35 Stirling Highway, Crawley WA 6009, Australia E-mail address: alcard...@gmail.com, andrea.card...@unimore.it WEBPAGE: https://sites.google.com/site/alcardini/home/main FREE Yellow BOOK on Geometric Morphometrics: http://www.italian-journal-of-mammalogy.it/public/journals/3/issue_241_complete_100.pdf -- MORPHMET may be accessed via its webpage at http://www.morphometrics.org --- You received this message because you are subscribed to the Google Groups "MORPHMET" group. To unsubscribe from this group and stop receiving emails from it, send an email to morphmet+unsubscr...@morphometrics.org.