Dear All I wanted to thank everyone for their helpful comments. With your help, and that of Simon Wood, I now realise that the reason I have low predicted values is because I have so many zeros in my data. As the model structure I have constructed specifies that the mean must always be positive then the model over-predicts the zero counts and in order not to predict more counts that there actually are it under-estimates the non zeros counts (this underestimation can be quite large due to the high number of zeros).
So one thing I am thinking of is to try a zero-inflated model. I have looked at the COZIGAM package but you do not seem to be able use an offset with it. I was wondering if anybody knows of a package where weighted zero-inflated GAM models with an offset can be run. Many thanks, Anna Dr Anna R. Renwick Research Ecologist British Trust for Ornithology, The Nunnery, Thetford, Norfolk, IP24 2PU, UK Tel: +44 (0)1842 750050; Fax: +44 (0)1842 750030 -----Original Message----- From: r-sig-ecology-boun...@r-project.org [mailto:r-sig-ecology-boun...@r-project.org] On Behalf Of Highland Statistics Ltd. Sent: 12 December 2009 11:28 To: r-sig-ecology@r-project.org Subject: Re: [R-sig-eco] low predicted vales in GAMs (Anna Renwick) > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 11 Dec 2009 11:43:40 -0000 > From: "Anna Renwick" <anna.renw...@bto.org> > Subject: [R-sig-eco] low predicted vales in GAMs > To: <r-sig-ecology@r-project.org> > Message-ID: <bfd6df2c5ca142c58c272652fa017...@btodomain.bto.org> > Content-Type: text/plain > > Dear All > > > > I have come across a problem with the GAM models I am running. Basically the > predicted values are consistently only about 0.4 of the actual values. > > > > A bit more detail: > > MODEL: > > m4<-gam(count~s(east,north,k=10)+ez+cv01+cv03+cv04+cv05+cv07+mtemp+mtotalrai > n+ez:mtemp+ez:mtotalrain+ > > offset(log(fit.vec)), > > weights=wt, > > data=spat6, > > family=quasipoisson, > > start=rep(0,26) > > ) > > MODEL SUMMARY: > > > > Family: quasipoisson > > Link function: log > > > > Formula: > > count ~ s(east, north, k = 10) + ez + cv01 + cv03 + cv04 + cv05 + > > cv07 + mtemp + mtotalrain + ez:mtemp + ez:mtotalrain + > offset(log(fit.vec)) > > > > Parametric coefficients: > > Estimate Std. Error t value Pr(>|t|) > > (Intercept) -5.296e+00 1.846e+00 -2.869 0.004166 ** > > ezM 1.651e+00 2.102e+00 0.785 0.432397 > > ezP 7.358e+00 2.047e+00 3.595 0.000332 *** > > ezU -1.061e+02 1.064e+07 -9.97e-06 0.999992 > > cv01 7.405e-02 5.437e-03 13.620 < 2e-16 *** > > cv03 2.258e-02 5.145e-03 4.389 1.20e-05 *** > > cv04 2.878e-02 4.839e-03 5.949 3.18e-09 *** > > cv05 3.634e-02 5.326e-03 6.823 1.17e-11 *** > > cv07 2.370e-02 5.712e-03 4.149 3.48e-05 *** > > mtemp -1.838e-01 1.750e-01 -1.050 0.293900 > > mtotalrain 1.872e-02 5.072e-03 3.692 0.000229 *** > > ezM:mtemp 6.181e-02 2.204e-01 0.280 0.779197 > > ezP:mtemp -7.028e-01 2.050e-01 -3.429 0.000619 *** > > ezU:mtemp 8.697e-01 1.371e+06 6.34e-07 0.999999 > > ezM:mtotalrain -3.393e-02 5.799e-03 -5.851 5.68e-09 *** > > ezP:mtotalrain -1.901e-02 5.379e-03 -3.535 0.000417 *** > > ezU:mtotalrain 3.510e-02 4.074e+04 8.62e-07 0.999999 > > --- > > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 > > > > Approximate significance of smooth terms: > > edf Ref.df F p-value > > s(east,north) 8.736 8.736 28.88 <2e-16 *** > > --- > > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 > > > > R-sq.(adj) = 0.324 Deviance explained = -5.12e+03% > > GCV score = 39.556 Scale est. = 39.056 n = 2038 > > > > > > Count = bird counts/square > > Is this really an integer? > ez=environmental zone > > cv = habitat types > > mtemp = mean annual temperature > > mtotalrain= mean total rain/year > > > > Sample size is approximately 2000. > > > > The offset fit.vec is bird detectability and the weighting is based on the > number of squares in each area surveyed. I belief that the strange deviance > explained is due to the weighting we have added into the model. > > Why would you use a weighting factor in a Poisson/quasi-Poisson GLM/GAM? See also the weights text for the help file for glm. Not sure what it would be doing. > > > I would have assumed that the predicted values divided by the real counts > should be around 1, however they are much lower and hence the model is > consistently predicting lower counts than were observed. I was wondering if > there is anything obvious which I am missing when carrying out these models. > > you seem to have a very large overdispersion. But that is another problem. I think your number of squares should actually be used in the offset (the log obviously). Alain > > > Many thanks, > > Anna > > > > Dr Anna R. Renwick > Research Ecologist > British Trust for Ornithology, > The Nunnery, > Thetford, > Norfolk, > IP24 2PU, > UK > Tel: +44 (0)1842 750050; Fax: +44 (0)1842 750030 > > > > > [[alternative HTML version deleted]] > > > > ------------------------------ > > _______________________________________________ > R-sig-ecology mailing list > R-sig-ecology@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-ecology > > > End of R-sig-ecology Digest, Vol 21, Issue 12 > ********************************************* > > -- Dr. Alain F. Zuur First author of: 1. Analysing Ecological Data (2007). Zuur, AF, Ieno, EN and Smith, GM. Springer. 680 p. URL: www.springer.com/0-387-45967-7 2. Mixed effects models and extensions in ecology with R. (2009). Zuur, AF, Ieno, EN, Walker, N, Saveliev, AA, and Smith, GM. Springer. http://www.springer.com/life+sci/ecology/book/978-0-387-87457-9 3. A Beginner's Guide to R (2009). Zuur, AF, Ieno, EN, Meesters, EHWG. Springer http://www.springer.com/statistics/computational/book/978-0-387-93836-3 Other books: http://www.highstat.com/books.htm Statistical consultancy, courses, data analysis and software Highland Statistics Ltd. 6 Laverock road UK - AB41 6FN Newburgh Tel: 0044 1358 788177 Email: highs...@highstat.com URL: www.highstat.com URL: www.brodgar.com _______________________________________________ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology _______________________________________________ R-sig-ecology mailing list R-sig-ecology@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-ecology