Geoff Sayer <[EMAIL PROTECTED]> wrote: > The PDF presentation reported 188 GPs not using computers in the > analysis. > > While I agree in principle that clustering should be considered with > this type of modeling the design effect in my experience with this sort of > data is limited as the "association analysis" sort of reduces its impact. In > some sense you are trying to capitalize on that to show association. However, > there are enough statistical packages around that can handle this these > days then there were back in the old days (the 90s).
This report seems to belie your experience: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=545648 - they are reporting design effects of between 6 and 45 for single variables - that is, the effective sample size is between 6 and 45 times smaller than the actual sample, and Deffs of between 2 to 5 for association intra-class correlation i.e. effective sample sizes of half to a fifth of the actual sample size. Given design effects of this magnitude, I agree that these days there is no excuse for not taking the sampling design into account when fitting statistical models to cluster-sampled data like the BEACH data. Maybe they did that, but there was no mention of it in the abstract or the presentation. > In contrast when you are estimating the population prevalence estimates > the design effects are obviously greater if you treat the study as a simple > random sample rather than a cluster sample. Yes, as the above paper reports, the design effects are rather enormous. > When using the GP as the unit of analysis you are negating clustering as > it becomes the number of events over number of trials method for each GP. > The problem then becomes that not all GPs will have the event or number of > trials because you are relying on 100 encounters only out of 5,000 > encounters (on average in a gven year). For example, each GP will have > on average 1.9 depression encounters in the 100 encounters - or 2 trials on > average, some GPs won't have any in the 100 encounters since it was a > happy week... thereby reducing the statistical power even further for > particular clinical indicators. Also, if you analyse only the proportion of patients meeting some criteria for each GP (that is, analyse only at the GP level and not the individual patient level), then you are wasting huge amounts of information and interpretation must be very circumspect because of the possibility of serious ecological fallacy, or Simpson's paradox if you like. > General practice is comprised of many many different clinical events > that are thinly distributed across a year almost Poisson in nature rather > than binomial. Possibly sparser than Poisson - I would not be surprised if many clinical events in general practice are distributed in a negative binomial or negative binomial with zero inflation manner. > This is a problem when one looks at clinical quality > indicators particularly at a GP or practice levels hence the statistical power > question, generalisability and suitability of analyses is raised. Yes, I think that there are many methodological issues to which too little attention is paid in general practice research. I am not saying that the presentation we are discussing is guilty of such lack of attention, but I can't say for certain because not enough information about the methods is reported. Anyway, this is a useful discussion which underlines the desirability of collecting as much data as possible, electronically, as everyone has been pointing out. Tim C > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] > On Behalf Of Tim Churches > Sent: Tuesday, 10 July 2007 10:49 a.m. > To: General Practice Computing Group Talk > Cc: 'General Practice Computing Group Talk' > Subject: Re: RE: [GPCG_TALK] BEACH thinks computers don't help GP > quality > > Geoff Sayer <[EMAIL PROTECTED]> wrote: > > 3. The statistical power of the study - which would require greater > > detail > > of the occurrence of events. The 1200 odd GPs (Computers: 1069 vs Non > > computers: 188) only provided 100 consecutive encounters. Some of > these > > quality indicators may not have the power to show an association if it > > > is a small one. > > > > Depression occurs at 1.9 per 100 encs (from most recent report) > thereby > > it is expected that there are 357 depression encounters in the > Non-Computer > > GP group. You then put several factors into a multivariate analysis > the > > statistical power is reduced again if there is in fact a real but > small > > difference between the two groups. > > >From my reading of the abstract and the actual presentation, my guess > is > that the analysis was done at the GP level i.e. they fitted a linear > regression model to the number (or percentage, since there were 100 > patients > per GP presumably) of patients for each GP receiving each of the > "quality > indicators" . Thus there may well be precision and power issues if only > 188 > of the GPs were not computer users. Hmmm, but hold on - they report > "adjusted odds ratios", which suggests that they fitted a logistic model > (which is a type of generalised linear regression model and thus more or > less consistent with the way the study was reported) - if so, then the > model > really does need to take account of the cluster sample design - there > will > be a lot of correlation between patients seen by the same GP and hence > the > "design effect" is likely to be large, which further reduces the > precision > of the parameter estimates (and adjusted odds ratios) from the model. It > is > not obvious that a model wh! > i! > ch accounted for the sample design was used. Or they could have used > multilevel modelling, which is probably even better, but trickier. > > Would be interesting to know what type of model was fitted to what data, > and > if sub-optimal, to repeat the analysis using statistical methods which > make > the most of the collected data while also taking into account the sample > design. > > Tim C > _______________________________________________ > Gpcg_talk mailing list > [email protected] > http://ozdocit.org/cgi-bin/mailman/listinfo/gpcg_talk > > _______________________________________________ > Gpcg_talk mailing list > [email protected] > http://ozdocit.org/cgi-bin/mailman/listinfo/gpcg_talk _______________________________________________ Gpcg_talk mailing list [email protected] http://ozdocit.org/cgi-bin/mailman/listinfo/gpcg_talk
