This is all very interesting indeed. so I appreciate that the effect of a variable will depend on the presence of other variables and that the effect of this variable has a statistical meaning only in a specific model. With the particularity of inconsistency if data arise from non normal distribution as for the Cox model.
I have two comments though; the first is statistical and the second more methodological: - first, either I am strict on the inconsistency phenomenon and, given the fact that it is very unlikely that I even come close to finding all relevant variables to a model, all models I may build are necessarily biased OR I have to be less strict. How to use tests (score, Wald, LR) to compare models when, at most, only one of them is unbiased? - second (relevance first depend on the answer to comment above I guess), I study the effect of age without adustment (model 1), after adjusting for presentation variables (model 2) and after adjusting for presentation and treatment variables (model 3). I certainly do not pretend that I am truly looking at the effect of age and I know that what I call age across the different models is not the same (otherwise I would not compare it or I would hope not to find any difference). I am merely looking the effect of something that is measured by 'age' and that emcompasses imbalances at presentation, treatment, and remaining unknown counfounders (first model), imbalances in treatment and remaining unknown counfounders (second model), and imbalances in the remaining confounders (third model). The remaining unknown confounders potentially include a true effect of age, if such a thing exists, and of other variables I don't know about (which in fact are still presentation and treatment variables since there are no other possibilities!). As everything fits in a cohesive interpretation: older patients presents with worse tumors and are undertreated (as shown by descriptive statistics) AND as this is shown by the modelisation of the effect by an erosion of the effect of 'age' from model 1 to 3, I was tempted, with all due precautions, to conclude that the increased risk for older patients was due in part because they present with worse tumors, in part because the are undertreated, and in part because of something else. Would that be overinterpreting the data? Should I even drop the idea of building and presenting these models? Thanks again, David Biau. ________________________________ De : Bert Gunter <gunter.ber...@gene.com> Cc : Frank Harrell <f.harr...@vanderbilt.edu>; r help list <r-help@r-project.org> Envoyé le : Ven 13 août 2010, 18h 22min 58s Objet : Re: [R] Re : How to compare the effect of a variable across regression models? Just to amplify a bit on what Frank said... Except in special circumstances (othogonal designs, say), regression models are only "guaranteed" to produce useful predictions -- they may not tell you anything meaningful about the relative effects of the regressors because, as Frank said, that depends on both the study design AND the true effects. So, for example, typically if one removes a regressor from a model and refits, the values of the remaining coefficients will change. Moreover, it is difficult to understand even conceptually what "the effect of a variable" should mean if, as is typical, it interacts with other variables: the "effect" of the variable then depends on what the values of the other variables are. A very simple example of this that I used to use in teaching was the effect of barbituates and alcohol on sleepiness. In the absence of barbituates, the effect of a few drinks is to make you modestly sleepy; in the presence of barbituates, the effect of a few drinks is [[elided Yahoo spam]] The point is that we live in a multivariate interactive world. Terms like "the effect of a variable" derive from univariate thinking and need to be very carefully defined to make sense in such a world -- and then studies need to be carefully designed -- happenstance data are rarely sufficient -- to quantify them. .. Not what scientists like to hear, I think, but this is the reality. Further comments and criticisms welcome, of course. Cheers, Bert Gunter Genentech Nonclinical Statistics > OK, > > thank you very much for the answer.I will look into that. Hopefully I'll find > smoething that will work out. > > Best, > > David Biau. > > > > > ________________________________ > De : Frank Harrell <f.harr...@vanderbilt.edu> > > Cc : r help list <r-help@r-project.org> > Envoyé le : Ven 13 août 2010, 15h 50min 18s > Objet : Re: [R] How to compare the effect of a variable across regression > models? > > > David, > > In the Cox and many other regression models, the effect of a variable is > context-dependent. There is an identifiability problem in what you are doing, > as discussed by > > @ARTICLE{for95mod, > author = {Ford, Ian and Norrie, John and Ahmadi, Susan}, > year = 1995, > title = {Model inconsistency, illustrated by the {Cox} proportional hazards > model}, > journal = Stat in Med, > volume = 14, > pages = {735-746}, > annote = {covariable adjustment; adjusted estimates; baseline imbalances; > RCT; model misspecification; model identification} > } > > One possible remedy, which may not work for your goals, is to embed all models > in a grand model that is used for inference. > > When coefficients ARE comparable in some sense, you can use the bootstrap to >get > confidence bands for differences in regressor effects between models. > > Frank > > Frank E Harrell Jr Professor and Chairman School of Medicine > Department of Biostatistics Vanderbilt University > > On Fri, 13 Aug 2010, Biau David wrote: > >> Hello, >> >> I would like, if it is possible, to compare the effect of a variable across >> regression models. I have looked around but I haven't found anything. Maybe >> someone could help? Here is the problem: >> >> I am studying the effect of a variable (age) on an outcome (local recurrence: >> lr). I have built 3 models: >> - model 1: lr ~ age y = \beta_(a1).age >> - model 2: lr ~ age + presentation variables (X_p) y = \beta_(a2).age > + >> \BETA_(p2).X_p >> - model 3: lr ~ age + presentation variables + treatment variables( X_t) >> y = \beta_(a3).age + \BETA_(p3).X_(p) + \BETA_(t3).X_t >> >> Presentation variables include variables such as tumor grade, tumor size, >>etc... >> the physician cannot interfer with these variables. >> Treatment variables include variables such as chemotherapy, radiation, > surgical >> margins (a surrogate for adequate surgery). >> >> I have used cph for the models and restricted cubic splines (Design library) >>for >> age. I have noted that the effect of age decreases from model 1 to 3. >> >> I would like to compare the effect of age on the outcome across the different >> models. A test of \beta_(a1) = \beta_(a2) = \beta_(a3) and then two by two >> comparisons or a global trend test maybe? Is that possible? >> >> Thank you for your help, >> >> >> David Biau. >> >> >> >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > > > > [[alternative HTML version deleted]] > > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > > [[alternative HTML version deleted]]
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.