I think using the metrics provided (standard deviation units) one can assess the degree of variability and determine whether differences are large enough to be considered practically significant. I suppose Dr. Bates and others can comment further, but making decisions solely on the basis of statistical significance is not always the most reasonable method for making research decisions, especially when it is largely a function of sample size.
In my HLM to R experience, I have found the lmList function to provide the best answer to your question, although it is not a "quantitative" test like the chi-square in HLM. You can fit a linear model for each subject, use the intervals function, and plot the intercepts and the slopes for all units with 95% CIs. You can visually see whether the intercepts and slopes vary across units. I think this is a very strong method for examining variability. However, you can eliminate the random slope (or intercept, or both) and compare the models using the ANOVA command. This will give you an empirical test of the overall model fit. I was reading in Kirk (Experimental Design) last night and was reminded that examining descriptives, raw data, and visual displays are too often overlooked and dismissed, but are extremely important. Best, HCD ------ Harold C. Doran Director of Research and Evaluation New American Schools 675 N. Washington Street, Suite 220 Alexandria, Virginia 22314 703.647.1628 <http://www.edperform.net> -----Original Message----- From: Andrej Kveder [mailto:[EMAIL PROTECTED] Sent: Wednesday, July 02, 2003 5:41 AM To: Douglas Bates; J.R. Lockwood Cc: Harold Doran; R-Help Subject: RE: [R] within group variance of the coeficients in LME Firstly let me thank all for your answers and suggestions. I would like to followup on your comments. Currently I'm implementing multilevel models within a simulation run. So I was looking for an estimate weather a covariate varies across second level units or not. I was using HLM before so I knew about the test implemented in the package and tried to reporoduce it in R. However I think I can follow your logic about not using that. I would appreciate your reflection on the following. I need a quantitative figure to evaluate weather the covariate varies across second level units in the process of simulation. Of course I will be running thousands of them and would need to program the condition in code. In one of the previous questions to the group dr. Bates suggested to use the CI estmates, however he warned me about their very conservative nature (I got the same tip from the book). I thought about using the lower bound of the CI as an estimate with the rule "if above 0 then the covariate varies". Would that be a sound think to do? Do you have any other suggestions? I would really appreciate the feedback. thanks Andrej -----Original Message----- From: Douglas Bates [mailto:[EMAIL PROTECTED] Behalf Of Douglas Bates Sent: Monday, June 30, 2003 6:09 PM To: J.R. Lockwood Cc: Harold Doran; R-Help; Andrej Kveder Subject: Re: [R] within group variance of the coeficients in LME "J.R. Lockwood" <[EMAIL PROTECTED]> writes: > > > > Dear listers, > > > > I can't find the variance or se of the coefficients in a multilevel model > > using lme. > > > > The component of an lme() object called "apVar" provides the estimated > asymptotic covariance matrix of a particular transformation of the > variance components. Dr. Bates can correct me if I'm wrong but I > believe it is the matrix logarithm of Cholesky decomposition of the > covariance matrix of the random effects. I believe the details are in > the book by Pinheiro and Bates. Once you know the transformation you > can use the "apVar" elements to get estimated asympotic standard > errors for your variance components estimates using the delta method. > > J.R. Lockwood > 412-683-2300 x4941 > [EMAIL PROTECTED] > http://www.rand.org/methodology/stat/members/lockwood/ First, thanks to those who answered the question. I have been away from my email for about a week and am just now catching up on the r-help list. As I understand the original question from Andrej he wants to obtain the standard errors for coefficients in the fixed effects part of the model. Those are calculated in the summary method for lme objects and returned as the component called 'tTable'. Try library(nlme) example(lme) summary(fm2)$tTable to see the raw values. Other software for fitting mixed-effects models, such as SAS PROC MIXED and HLM, return standard errors along with the estimates of the variances and covariances of the random effects. We don't return standard errors of estimated variances because we don't think they are useful. A standard error for a parameter estimate is most useful when the distribution of the estimator is approximately symmetric, and these are not. Instead we feel that the variances and covariances should be converted to an unconstrained scale, and preferably a scale for which the log-likelihood is approximately quadratic. The apVar component that you mention is an approximate variance-covariance matrix of the variance components on an unbounded parameterization that uses the logarithm of any standard deviation and Fisher's z transformation of any correlations. If all variance-covariance matrices being estimated are 1x1 or 2x2 then this parameterization is both unbounded and unconstrained. If any are 3x3 or larger then this parameterization must be further constrained to ensure positive definiteness. Nevertheless, once we have finished the optimization we convert to this 'natural' parameterization to assess the variability of the estimates because these parameters are easily interpreted. The actual optimization of the profiled log-likelihood is done using the log-Cholesky parameterization that you mentioned because it is always unbounded and unconstrained. Interpreting elements of this parameter vector is complicated. I hope this isn't too confusing. ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help