Re: [R] Loess with more than 4 predictors / offsets
I apologize for clogging up inboxes, but I realized I needed to amend what I said in my last comment below: In fact, I'd like to specify that it be unconditionally linear, but with estimated coefficients, _both an intercept and a slope_. If the "offset" were only multiplied by a nonzero constant c, this would have the effect of moving the whole response surface -log(c) units parallel to the response axis in the scenario I outline below. This would effectively give me the same thing I already have. Paul Louisell 650-833-6254 [EMAIL PROTECTED] Research Associate (Statistician) Modeling & Data Analytics ARPC -Original Message- From: Louisell, Paul Sent: Tuesday, January 23, 2007 12:40 PM To: 'Prof Brian Ripley' Cc: r-help@stat.math.ethz.ch Subject: RE: [R] Loess with more than 4 predictors / offsets In response to your questions: I asked about including the offset for convenience; I currently put the offset in by subtracting it from the response, just as you suggest. The reason for including them is that I'm doing something slightly unusual with loess: I'm fitting loess to log((response+1)/offset) because the response is actually a vector of counts. This is intended to give a rough approximation to a Poisson regression; the reason for using loess is that the mean response should be approximated by a Poisson process with 4 predictor variables which can be divided into 2 pairs, each pair of which are geographic location coordinates. The two location pairs are expected to exhibit strong interaction; hence, the reason for fitting loess to all 4 predictors. I'm aware of the curse of dimensionality, but I have a very large dataset--over 600,000 observations. Since each pair of predictors represents a point on a grid, I think Euclidean distance is probably a good choice. And this brings me to the motivation for wanting to fit with 5 predictors: The offset is not _really_ an offset; it's just an approximation to what the real offset should be. Hence, I'd rather include it as a predictor than artificially force it to be included linearly with a coefficient of 1. I'm less concerned with linearity than I am with forcing the coefficient. In fact, I'd like to specify that it be unconditionally linear, but with an estimated coefficient. Thanks, Paul Louisell 650-833-6254 [EMAIL PROTECTED] Research Associate (Statistician) Modeling & Data Analytics ARPC -Original Message- From: Prof Brian Ripley [mailto:[EMAIL PROTECTED] Sent: Monday, January 22, 2007 11:01 PM To: Louisell, Paul Cc: r-help@stat.math.ethz.ch Subject: Re: [R] Loess with more than 4 predictors / offsets On Mon, 22 Jan 2007, Louisell, Paul wrote: > Hello, > > Does anyone know of an R version of loess that allows more than 4 > predictors and/or allows the specification of offsets? For that matter, > does anyone know of _any_ version of loess that does either of the > things I mention? Why would you want offsets in a regression?: just subtract them from the lhs. (R's lm has gained offsets by analogy with glm, but the S original did not have them). If you would be more comfortable working with them, it would be very easy to create a modified version that supports them. Also, have you heard of the 'curse of dimensionality'? Localization even to 4 dimensions is no longer really an appropriate term, and Euclidean distance will be the main determinant of 'local' and is quite arbitrary. > Thanks, > > Paul Louisell > 650-833-6254 > [EMAIL PROTECTED] > Research Associate (Statistician) > Modeling & Data Analytics > ARPC > > > > [[alternative HTML version deleted]] > > __ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Loess with more than 4 predictors / offsets
In response to your questions: I asked about including the offset for convenience; I currently put the offset in by subtracting it from the response, just as you suggest. The reason for including them is that I'm doing something slightly unusual with loess: I'm fitting loess to log((response+1)/offset) because the response is actually a vector of counts. This is intended to give a rough approximation to a Poisson regression; the reason for using loess is that the mean response should be approximated by a Poisson process with 4 predictor variables which can be divided into 2 pairs, each pair of which are geographic location coordinates. The two location pairs are expected to exhibit strong interaction; hence, the reason for fitting loess to all 4 predictors. I'm aware of the curse of dimensionality, but I have a very large dataset--over 600,000 observations. Since each pair of predictors represents a point on a grid, I think Euclidean distance is probably a good choice. And this brings me to the motivation for wanting to fit with 5 predictors: The offset is not _really_ an offset; it's just an approximation to what the real offset should be. Hence, I'd rather include it as a predictor than artificially force it to be included linearly with a coefficient of 1. I'm less concerned with linearity than I am with forcing the coefficient. In fact, I'd like to specify that it be unconditionally linear, but with an estimated coefficient. Thanks, Paul Louisell 650-833-6254 [EMAIL PROTECTED] Research Associate (Statistician) Modeling & Data Analytics ARPC -Original Message- From: Prof Brian Ripley [mailto:[EMAIL PROTECTED] Sent: Monday, January 22, 2007 11:01 PM To: Louisell, Paul Cc: r-help@stat.math.ethz.ch Subject: Re: [R] Loess with more than 4 predictors / offsets On Mon, 22 Jan 2007, Louisell, Paul wrote: > Hello, > > Does anyone know of an R version of loess that allows more than 4 > predictors and/or allows the specification of offsets? For that matter, > does anyone know of _any_ version of loess that does either of the > things I mention? Why would you want offsets in a regression?: just subtract them from the lhs. (R's lm has gained offsets by analogy with glm, but the S original did not have them). If you would be more comfortable working with them, it would be very easy to create a modified version that supports them. Also, have you heard of the 'curse of dimensionality'? Localization even to 4 dimensions is no longer really an appropriate term, and Euclidean distance will be the main determinant of 'local' and is quite arbitrary. > Thanks, > > Paul Louisell > 650-833-6254 > [EMAIL PROTECTED] > Research Associate (Statistician) > Modeling & Data Analytics > ARPC > > > > [[alternative HTML version deleted]] > > __ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Loess with more than 4 predictors / offsets
On Mon, 22 Jan 2007, Louisell, Paul wrote: > Hello, > > Does anyone know of an R version of loess that allows more than 4 > predictors and/or allows the specification of offsets? For that matter, > does anyone know of _any_ version of loess that does either of the > things I mention? Why would you want offsets in a regression?: just subtract them from the lhs. (R's lm has gained offsets by analogy with glm, but the S original did not have them). If you would be more comfortable working with them, it would be very easy to create a modified version that supports them. Also, have you heard of the 'curse of dimensionality'? Localization even to 4 dimensions is no longer really an appropriate term, and Euclidean distance will be the main determinant of 'local' and is quite arbitrary. > Thanks, > > Paul Louisell > 650-833-6254 > [EMAIL PROTECTED] > Research Associate (Statistician) > Modeling & Data Analytics > ARPC > > > > [[alternative HTML version deleted]] > > __ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Loess with more than 4 predictors / offsets
Hello, Does anyone know of an R version of loess that allows more than 4 predictors and/or allows the specification of offsets? For that matter, does anyone know of _any_ version of loess that does either of the things I mention? Thanks, Paul Louisell 650-833-6254 [EMAIL PROTECTED] Research Associate (Statistician) Modeling & Data Analytics ARPC [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] loess
On Thu, 4 Jan 2007, Jukka Nyblom wrote: > Hi, > > I have tried > > > for (i in 1:100) L[,i] <- loess((i = =(1:100))~I(1:100), span=.5, > degree=1)$fit > > to create a matrix which gives me the smoothing weights (correctly as > far as I have experienced), eg. > > > yhat <- loess(y~I(1:100), span=.5,degree=1)$fit > > yhat[30] > [1] -0.2131983 > > L[30,]%*%y > [,1] > [1,] -0.2131983 > > But, L[30,] has 56 nonzero coefficients, not 50 that I expect with span > = 0.5. Actually the number of nonzero elements on rows varies being 49, > 50, 55 or 56. > > Does anyone know why? loess is a complicated algorithm, and you need to study the background references in depth to fully understand it. In particular, the default is not to do direct fitting (as I guess you are assuming) but interpolation. See ?loess.control. Most descriptions, including the help page, are simplifications. > Jukka Nyblom -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] loess
Hi, I have tried > for (i in 1:100) L[,i] <- loess((i = =(1:100))~I(1:100), span=.5, degree=1)$fit to create a matrix which gives me the smoothing weights (correctly as far as I have experienced), eg. > yhat <- loess(y~I(1:100), span=.5,degree=1)$fit > yhat[30] [1] -0.2131983 > L[30,]%*%y [,1] [1,] -0.2131983 But, L[30,] has 56 nonzero coefficients, not 50 that I expect with span = 0.5. Actually the number of nonzero elements on rows varies being 49, 50, 55 or 56. Does anyone know why? Jukka Nyblom __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] loess lines in xyplot with two or more variables on the left side of a formula
Try this:
xyplot(IDUPREV + VALIDAT~ YEAR | MSA, data = df, type= c("smooth", "o"))
On 11/23/06, Chuck Cleland <[EMAIL PROTECTED]> wrote:
> Hello:
> I recall something like this being discuss recently, but I can't seem
> to locate an example in the archives. I have data like the following:
>
> df <- expand.grid(1:4, 1992:2002)
> names(df) <- c("MSA", "YEAR")
> df$IDUPREV <- runif(44)
> df$VALIDAT <- rnorm(44)
>
> I want to create an xyplot() with separate loess lines for each series
> (IDUPREV and VALIDAT) in the same panel. I'm able to plot each series
> in the same panel like this:
>
> library(lattice)
>
> xyplot(IDUPREV + VALIDAT ~ YEAR | MSA, data = df,
> panel = panel.superpose, type="l")
>
> How could I change that so that panel.loess() is applied separately to
> each series?
>
> thanks,
>
> Chuck Cleland
>
> --
> Chuck Cleland, Ph.D.
> NDRI, Inc.
> 71 West 23rd Street, 8th floor
> New York, NY 10010
> tel: (212) 845-4495 (Tu, Th)
> tel: (732) 512-0171 (M, W, F)
> fax: (917) 438-0894
>
> __
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] loess lines in xyplot with two or more variables on the left side of a formula
Hello:
I recall something like this being discuss recently, but I can't seem
to locate an example in the archives. I have data like the following:
df <- expand.grid(1:4, 1992:2002)
names(df) <- c("MSA", "YEAR")
df$IDUPREV <- runif(44)
df$VALIDAT <- rnorm(44)
I want to create an xyplot() with separate loess lines for each series
(IDUPREV and VALIDAT) in the same panel. I'm able to plot each series
in the same panel like this:
library(lattice)
xyplot(IDUPREV + VALIDAT ~ YEAR | MSA, data = df,
panel = panel.superpose, type="l")
How could I change that so that panel.loess() is applied separately to
each series?
thanks,
Chuck Cleland
--
Chuck Cleland, Ph.D.
NDRI, Inc.
71 West 23rd Street, 8th floor
New York, NY 10010
tel: (212) 845-4495 (Tu, Th)
tel: (732) 512-0171 (M, W, F)
fax: (917) 438-0894
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] loess smoothing question
If the y values are "hypergeometrically" distributed then they are counts, right? Loess is designed for continuous, reasonably symmetric data, and so is inappropriate. You should probably consider GLM for a parametric fit; or perhaps GAM for a nonparametric fit. As the data appear to have the structure of a time series, you may wish to search CRAN for a non-Gaussian time series package. I am unfamiliar with such methodology, so I have no idea what, if anything, is available for this. Better suggestion. Get help from a local statistician, at least to get you started. -- Bert Gunter Genentech Non-Clinical Statistics South San Francisco, CA "The business of the statistician is to catalyze the scientific learning process." - George E. P. Box > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Thomas L Jones > Sent: Monday, December 19, 2005 5:53 AM > To: R-project help > Subject: [R] loess smoothing question > > I am trying to smooth a dataset with evenly spaced values of x, > perhaps using loess smoothing or something similar. However, the y > values are hypergeometrically distributed; I think I want to use a > logarithmic link function. It falls under the general heading of > non-parametric regression. The problem is of interest in predicting > the demand at a voting place, in order to avoid long lines. > > Questions: Should I use loess smoothing? >Do I want a logarithmic link function? If so, >How do I tell loess to use a logarithmic link function? > > Tom, a newbie to the R project, and not really a statistician > > __ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess smoothing question
I am trying to smooth a dataset with evenly spaced values of x, perhaps using loess smoothing or something similar. However, the y values are hypergeometrically distributed; I think I want to use a logarithmic link function. It falls under the general heading of non-parametric regression. The problem is of interest in predicting the demand at a voting place, in order to avoid long lines. Questions: Should I use loess smoothing? Do I want a logarithmic link function? If so, How do I tell loess to use a logarithmic link function? Tom, a newbie to the R project, and not really a statistician __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] loess: choose span to minimize AIC?
The locfit package (which, I believe, contains an independent implementation
of loess, plus more) contains the gcvplot() and aicplot() functions that I
think can do this.
Best,
Andy
> From: Michael Friendly
>
> Thanks very much, John
>
> The formula for AICC1 was transscribed from an ambiguously
> rendered version (in the SAS documentation). This is a
> corrected version.
>
> loess.aic <- function (x) {
> if (!(inherits(x,"loess"))) stop("Error: argument must
> be a loess object")
> # extract values from loess object
> span <- x$pars$span
> n <- x$n
> traceL <- x$trace.hat
> sigma2 <- sum( x$residuals^2 ) / (n-1)
> delta1 <- x$one.delta
> delta2 <- x$two.delta
> enp <- x$enp
>
> aicc <- log(sigma2) + 1 + 2* (2*(traceL+1)) / (n-traceL-2)
> # aicc1<- n*log(sigma2) + n* (
> (delta1/(delta2*(n+enp)))/(delta1^2/delta2)-2 )
> aicc1<- n*log(sigma2) + n* (
> (delta1/delta2)*(n+enp)/(delta1^2/delta2)-2 )
> gcv <- n*sigma2 / (n-traceL)^2
>
> result <- list(span=span, aicc=aicc, aicc1=aicc1, gcv=gcv)
> return(result)
> }
>
>
> John Fox wrote:
>
> > Dear Mike,
> >
> > You could try
> >
> > bestLoess <- function(model, criterion=c("aicc", "aicc1", "gcv"),
> > spans=c(.05, .95)){
> > criterion <- match.arg(criterion)
> > f <- function(span) {
> > mod <- update(model, span=span)
> > loess.aic(mod)[[criterion]]
> > }
> > result <- optimize(f, spans)
> > list(span=result$minimum, criterion=result$objective)
> > }
> >
> > A little experimentation suggests that aicc1 doesn't seem to behave
> > reasonably.
> >
> > Regards,
> > John
> >
> >
> > John Fox
> > Department of Sociology
> > McMaster University
> > Hamilton, Ontario
> > Canada L8S 4M4
> > 905-525-9140x23604
> > http://socserv.mcmaster.ca/jfox
> >
> >
> >
> >>-Original Message-
> >>From: [EMAIL PROTECTED]
> >>[mailto:[EMAIL PROTECTED] On Behalf Of
> >>Michael Friendly
> >>Sent: Thursday, November 17, 2005 9:58 AM
> >>To: R-help@stat.math.ethz.ch
> >>Subject: [R] loess: choose span to minimize AIC?
> >>
> >>Is there an R implementation of a scheme for automatic
> >>smoothing parameter selection with loess, e.g., by minimizing
> >>one of the AIC/GCV statistics discussed by Hurvich, Simonoff
> >>& Tsai (1998)?
> >>
> >>Below is a function that calculates the relevant values of AICC,
> >>AICC1 and GCV--- I think, because I to guess from the names
> >>of the components returned in a loess object.
> >>
> >>I guess I could use optimize(), or do a simple line search on
> >>span=, but I'm not sure how to use loess.aic to write a
> >>function that would act as a wrapper for loess() and return
> >>the mimimizing loess fit for a specified criterion.
> >>
> >>loess.aic <- function (x) {
> >># extract values from loess object
> >>if (!(inherits(x,"loess"))) stop("Error: argument must
> >>be a loess object")
> >>span <- x$pars$span
> >>n <- x$n
> >>traceL <- x$trace.hat
> >>sigma2 <- sum( x$residuals^2 ) / (n-1)
> >>delta1 <- x$one.delta
> >>delta2 <- x$two.delta
> >>enp <- x$enp
> >>
> >>aicc <- log(sigma2) + 1 + 2* (2*(traceL+1)) / (n-traceL-2)
> >>aicc1<- n*log(sigma2) + n* (
> >>(delta1/(delta2*(n+enp)))/(delta1^2/delta2)-2 )
> >>gcv <- n*sigma2 / (n-traceL)^2
> >>
> >>result <- list(span=span, aicc=aicc, aicc1=aicc1, gcv=gcv)
> >>return(result)
> >>}
> >>
> >>
> >> > cars.lo <- loess(dist ~ speed, cars)
> >> >
> >> > (values <- loess.aic(cars.lo))
> >>$span
> >>[1] 0.75
> >>
> >>$aicc
> >>[1] 6.93678
> >>
> >>$aicc1
> >>[1] 167.7267
> >>
> >>$gcv
> >>[1] 5.275487
> >>
> >> >
> >>
> >>
> >>--
> >>Michael Friendly Email: friendly AT yorku DOT ca
> >>Professor, Psychology Dept.
> >>York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
> >>4700 Keele Streethttp://www.math.yorku.ca/SCS/friendly.html
> >>Toronto, ONT M3J 1P3 CANADA
> >>
> >>__
> >>R-help@stat.math.ethz.ch mailing list
> >>https://stat.ethz.ch/mailman/listinfo/r-help
> >>PLEASE do read the posting guide!
> >>http://www.R-project.org/posting-guide.html
>
> --
> Michael Friendly Email: friendly AT yorku DOT ca
> Professor, Psychology Dept.
> York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
> 4700 Keele Streethttp://www.math.yorku.ca/SCS/friendly.html
> Toronto, ONT M3J 1P3 CANADA
>
> __
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
>
>
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] loess: choose span to minimize AIC?
Thanks very much, John
The formula for AICC1 was transscribed from an ambiguously
rendered version (in the SAS documentation). This is a
corrected version.
loess.aic <- function (x) {
if (!(inherits(x,"loess"))) stop("Error: argument must be a loess
object")
# extract values from loess object
span <- x$pars$span
n <- x$n
traceL <- x$trace.hat
sigma2 <- sum( x$residuals^2 ) / (n-1)
delta1 <- x$one.delta
delta2 <- x$two.delta
enp <- x$enp
aicc <- log(sigma2) + 1 + 2* (2*(traceL+1)) / (n-traceL-2)
# aicc1<- n*log(sigma2) + n* (
(delta1/(delta2*(n+enp)))/(delta1^2/delta2)-2 )
aicc1<- n*log(sigma2) + n* (
(delta1/delta2)*(n+enp)/(delta1^2/delta2)-2 )
gcv <- n*sigma2 / (n-traceL)^2
result <- list(span=span, aicc=aicc, aicc1=aicc1, gcv=gcv)
return(result)
}
John Fox wrote:
> Dear Mike,
>
> You could try
>
> bestLoess <- function(model, criterion=c("aicc", "aicc1", "gcv"),
> spans=c(.05, .95)){
> criterion <- match.arg(criterion)
> f <- function(span) {
> mod <- update(model, span=span)
> loess.aic(mod)[[criterion]]
> }
> result <- optimize(f, spans)
> list(span=result$minimum, criterion=result$objective)
> }
>
> A little experimentation suggests that aicc1 doesn't seem to behave
> reasonably.
>
> Regards,
> John
>
>
> John Fox
> Department of Sociology
> McMaster University
> Hamilton, Ontario
> Canada L8S 4M4
> 905-525-9140x23604
> http://socserv.mcmaster.ca/jfox
>
>
>
>>-Original Message-
>>From: [EMAIL PROTECTED]
>>[mailto:[EMAIL PROTECTED] On Behalf Of
>>Michael Friendly
>>Sent: Thursday, November 17, 2005 9:58 AM
>>To: R-help@stat.math.ethz.ch
>>Subject: [R] loess: choose span to minimize AIC?
>>
>>Is there an R implementation of a scheme for automatic
>>smoothing parameter selection with loess, e.g., by minimizing
>>one of the AIC/GCV statistics discussed by Hurvich, Simonoff
>>& Tsai (1998)?
>>
>>Below is a function that calculates the relevant values of AICC,
>>AICC1 and GCV--- I think, because I to guess from the names
>>of the components returned in a loess object.
>>
>>I guess I could use optimize(), or do a simple line search on
>>span=, but I'm not sure how to use loess.aic to write a
>>function that would act as a wrapper for loess() and return
>>the mimimizing loess fit for a specified criterion.
>>
>>loess.aic <- function (x) {
>> # extract values from loess object
>> if (!(inherits(x,"loess"))) stop("Error: argument must
>>be a loess object")
>> span <- x$pars$span
>> n <- x$n
>> traceL <- x$trace.hat
>> sigma2 <- sum( x$residuals^2 ) / (n-1)
>> delta1 <- x$one.delta
>> delta2 <- x$two.delta
>> enp <- x$enp
>>
>> aicc <- log(sigma2) + 1 + 2* (2*(traceL+1)) / (n-traceL-2)
>> aicc1<- n*log(sigma2) + n* (
>>(delta1/(delta2*(n+enp)))/(delta1^2/delta2)-2 )
>> gcv <- n*sigma2 / (n-traceL)^2
>>
>> result <- list(span=span, aicc=aicc, aicc1=aicc1, gcv=gcv)
>> return(result)
>>}
>>
>>
>> > cars.lo <- loess(dist ~ speed, cars)
>> >
>> > (values <- loess.aic(cars.lo))
>>$span
>>[1] 0.75
>>
>>$aicc
>>[1] 6.93678
>>
>>$aicc1
>>[1] 167.7267
>>
>>$gcv
>>[1] 5.275487
>>
>> >
>>
>>
>>--
>>Michael Friendly Email: friendly AT yorku DOT ca
>>Professor, Psychology Dept.
>>York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
>>4700 Keele Streethttp://www.math.yorku.ca/SCS/friendly.html
>>Toronto, ONT M3J 1P3 CANADA
>>
>>__
>>R-help@stat.math.ethz.ch mailing list
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide!
>>http://www.R-project.org/posting-guide.html
--
Michael Friendly Email: friendly AT yorku DOT ca
Professor, Psychology Dept.
York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
4700 Keele Streethttp://www.math.yorku.ca/SCS/friendly.html
Toronto, ONT M3J 1P3 CANADA
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] loess: choose span to minimize AIC?
Dear Mike,
You could try
bestLoess <- function(model, criterion=c("aicc", "aicc1", "gcv"),
spans=c(.05, .95)){
criterion <- match.arg(criterion)
f <- function(span) {
mod <- update(model, span=span)
loess.aic(mod)[[criterion]]
}
result <- optimize(f, spans)
list(span=result$minimum, criterion=result$objective)
}
A little experimentation suggests that aicc1 doesn't seem to behave
reasonably.
Regards,
John
John Fox
Department of Sociology
McMaster University
Hamilton, Ontario
Canada L8S 4M4
905-525-9140x23604
http://socserv.mcmaster.ca/jfox
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of
> Michael Friendly
> Sent: Thursday, November 17, 2005 9:58 AM
> To: R-help@stat.math.ethz.ch
> Subject: [R] loess: choose span to minimize AIC?
>
> Is there an R implementation of a scheme for automatic
> smoothing parameter selection with loess, e.g., by minimizing
> one of the AIC/GCV statistics discussed by Hurvich, Simonoff
> & Tsai (1998)?
>
> Below is a function that calculates the relevant values of AICC,
> AICC1 and GCV--- I think, because I to guess from the names
> of the components returned in a loess object.
>
> I guess I could use optimize(), or do a simple line search on
> span=, but I'm not sure how to use loess.aic to write a
> function that would act as a wrapper for loess() and return
> the mimimizing loess fit for a specified criterion.
>
> loess.aic <- function (x) {
> # extract values from loess object
> if (!(inherits(x,"loess"))) stop("Error: argument must
> be a loess object")
> span <- x$pars$span
> n <- x$n
> traceL <- x$trace.hat
> sigma2 <- sum( x$residuals^2 ) / (n-1)
> delta1 <- x$one.delta
> delta2 <- x$two.delta
> enp <- x$enp
>
> aicc <- log(sigma2) + 1 + 2* (2*(traceL+1)) / (n-traceL-2)
> aicc1<- n*log(sigma2) + n* (
> (delta1/(delta2*(n+enp)))/(delta1^2/delta2)-2 )
> gcv <- n*sigma2 / (n-traceL)^2
>
> result <- list(span=span, aicc=aicc, aicc1=aicc1, gcv=gcv)
> return(result)
> }
>
>
> > cars.lo <- loess(dist ~ speed, cars)
> >
> > (values <- loess.aic(cars.lo))
> $span
> [1] 0.75
>
> $aicc
> [1] 6.93678
>
> $aicc1
> [1] 167.7267
>
> $gcv
> [1] 5.275487
>
> >
>
>
> --
> Michael Friendly Email: friendly AT yorku DOT ca
> Professor, Psychology Dept.
> York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
> 4700 Keele Streethttp://www.math.yorku.ca/SCS/friendly.html
> Toronto, ONT M3J 1P3 CANADA
>
> __
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] loess: choose span to minimize AIC?
On Thu, 17 Nov 2005, Michael Friendly wrote:
> Is there an R implementation of a scheme for automatic smoothing
> parameter selection with loess, e.g., by minimizing one of the AIC/GCV
> statistics discussed by Hurvich, Simonoff & Tsai (1998)?
If you particularly want loess smoothing then I don't know, but if
penalised spline smoothing will do then in gam() in the mgcv package does
minimize GCV.
-thomas
> Below is a function that calculates the relevant values of AICC,
> AICC1 and GCV--- I think, because I to guess from the names of the
> components returned in a loess object.
>
> I guess I could use optimize(), or do a simple line search on span=,
> but I'm not sure how to use loess.aic to write a function
> that would act as a wrapper for loess() and return the mimimizing
> loess fit for a specified criterion.
>
> loess.aic <- function (x) {
> # extract values from loess object
> if (!(inherits(x,"loess"))) stop("Error: argument must be a loess
> object")
> span <- x$pars$span
> n <- x$n
> traceL <- x$trace.hat
> sigma2 <- sum( x$residuals^2 ) / (n-1)
> delta1 <- x$one.delta
> delta2 <- x$two.delta
> enp <- x$enp
>
> aicc <- log(sigma2) + 1 + 2* (2*(traceL+1)) / (n-traceL-2)
> aicc1<- n*log(sigma2) + n* (
> (delta1/(delta2*(n+enp)))/(delta1^2/delta2)-2 )
> gcv <- n*sigma2 / (n-traceL)^2
>
> result <- list(span=span, aicc=aicc, aicc1=aicc1, gcv=gcv)
> return(result)
> }
>
>
> > cars.lo <- loess(dist ~ speed, cars)
> >
> > (values <- loess.aic(cars.lo))
> $span
> [1] 0.75
>
> $aicc
> [1] 6.93678
>
> $aicc1
> [1] 167.7267
>
> $gcv
> [1] 5.275487
>
> >
>
>
> --
> Michael Friendly Email: friendly AT yorku DOT ca
> Professor, Psychology Dept.
> York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
> 4700 Keele Streethttp://www.math.yorku.ca/SCS/friendly.html
> Toronto, ONT M3J 1P3 CANADA
>
> __
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>
Thomas Lumley Assoc. Professor, Biostatistics
[EMAIL PROTECTED] University of Washington, Seattle
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess: choose span to minimize AIC?
Is there an R implementation of a scheme for automatic smoothing
parameter selection with loess, e.g., by minimizing one of the AIC/GCV
statistics discussed by Hurvich, Simonoff & Tsai (1998)?
Below is a function that calculates the relevant values of AICC,
AICC1 and GCV--- I think, because I to guess from the names of the
components returned in a loess object.
I guess I could use optimize(), or do a simple line search on span=,
but I'm not sure how to use loess.aic to write a function
that would act as a wrapper for loess() and return the mimimizing
loess fit for a specified criterion.
loess.aic <- function (x) {
# extract values from loess object
if (!(inherits(x,"loess"))) stop("Error: argument must be a loess
object")
span <- x$pars$span
n <- x$n
traceL <- x$trace.hat
sigma2 <- sum( x$residuals^2 ) / (n-1)
delta1 <- x$one.delta
delta2 <- x$two.delta
enp <- x$enp
aicc <- log(sigma2) + 1 + 2* (2*(traceL+1)) / (n-traceL-2)
aicc1<- n*log(sigma2) + n* (
(delta1/(delta2*(n+enp)))/(delta1^2/delta2)-2 )
gcv <- n*sigma2 / (n-traceL)^2
result <- list(span=span, aicc=aicc, aicc1=aicc1, gcv=gcv)
return(result)
}
> cars.lo <- loess(dist ~ speed, cars)
>
> (values <- loess.aic(cars.lo))
$span
[1] 0.75
$aicc
[1] 6.93678
$aicc1
[1] 167.7267
$gcv
[1] 5.275487
>
--
Michael Friendly Email: friendly AT yorku DOT ca
Professor, Psychology Dept.
York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
4700 Keele Streethttp://www.math.yorku.ca/SCS/friendly.html
Toronto, ONT M3J 1P3 CANADA
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
RE: [R] loess problems
The problem is that 90% of your data sit on the boundary. Loess is a nearest neighbor smoother (using (100 x span) % of the data to estimate at each point). If you call loess() directly with span=2/3 (the default in scatter.smooth), or something smaller than about 0.91, you'll see that it has trouble. Strangely, if you set span=.8, scatter.smooth() will also complain, but not at the default span... (Re-generating the data yet again does trigger the warnings, so seems like it does catches things some of the time.) For your second example, I think loess becomes undefined when the span is set too small (and 1/n is surely too small): You are asking the algorithm to take the nearest 1/n of the data to do the smooth. You would think that should just mean _the_ nearest data point, but the problem is: > n <- 100 > 1 / n < 1 [1] TRUE so you're asking the algoithm to take fewer than 1 data point to estimate at each point. The warnings you see for that example is pointing you in the right direction. Andy > From: Jean Eid > > I have a problem either understanding what loess is doing or > that loess > has a problem itself. > > As the x-axis variables become more concentrated on a > particular point,the > estimated loess tends to zero. the examples below show what i am > talking about, why is that? my intution tells me > that it should tend to the mean of the variable which is been > smoothed. > > Here's a worked up example > > x <- c(seq(0,100), rep(100,1000)) > y <- rnorm(length(x), mean=10, sd=2) > scatter.smooth(x,y) > > > > Although it does give warnings, I don't understand why it is > giving the > estimate as zero. > > > another example would be > > x <- seq(0,100) > y <- rnorm(length(x), mean=50, sd=2) > scatter.smooth(x,y, span=1/length(x)) > > > shoudn't this give just the points at which the smoothing algorithm is > applied? > > > > thank you > > __ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > > __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess problems
I have a problem either understanding what loess is doing or that loess has a problem itself. As the x-axis variables become more concentrated on a particular point,the estimated loess tends to zero. the examples below show what i am talking about, why is that? my intution tells me that it should tend to the mean of the variable which is been smoothed. Here's a worked up example x <- c(seq(0,100), rep(100,1000)) y <- rnorm(length(x), mean=10, sd=2) scatter.smooth(x,y) Although it does give warnings, I don't understand why it is giving the estimate as zero. another example would be x <- seq(0,100) y <- rnorm(length(x), mean=50, sd=2) scatter.smooth(x,y, span=1/length(x)) shoudn't this give just the points at which the smoothing algorithm is applied? thank you __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
R: [R] loess significance
Dear Aurélie, I think that for *fixed* (i.e. assumed known) amount of smoothing, you can use a simple LRT by comparing the two candidate models. BTW, have a look to the mgcv or gam packages for a general model-based approach. - Original Message - From: Aurélie Coulon <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, October 21, 2004 8:37 AM Subject: [R] loess significance Dear list, I would like to know it is possible to test the significance of a loess ; in other words, I would like to know if the loess I got is significantly different from a linear model. Thanks. Aurélie Coulon. [[alternative HTML version deleted]] __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess significance
Dear list, I would like to know it is possible to test the significance of a loess ; in other words, I would like to know if the loess I got is significantly different from a linear model. Thanks. Aurélie Coulon. [[alternative HTML version deleted]] __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess prediction limits
Hello I am plotting a loess curve with confidence limits as below. How do I create the prediction limits? Is multiplying the standard errors by sqrt(n) appropriate? data <- mndata lo <- loess(data[[variableName]] ~ Age, data, span=1.0, control = loess.control(surface = "direct")) xPoints <- data.frame(age = seq(1,240,1)) lo1 <- predict(lo, xPoints, se = TRUE) age <- xPoints$age lines(age,lo1$fit, col=4) # now do +/- 2 std errors lo1p <- lo1$fit + 2*lo1$se.fit lo1m <- lo1$fit - 2*lo1$se.fit lines(age,lo1p, col=4) lines(age,lo1m, col=4) Thanks, David __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] loess
On Tue, 25 May 2004, Rupen Shrestha wrote: > When I was running the function "loess(y~x, span=0.0020)", I got a warning > message "k-d tree limited by memory. ncmax= 4231" > > Does that mean the function has not been computed correctly ? If it has not, > is there any way to adjust it so that it will do correctly ? It was computed a little inaccurately. You can alter many things: see ?loess.control, especially its first item. *However*, that span is so small that this makes little sense, with too few neighbours for any visually apparent smoothing. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess
Hi, When I was running the function "loess(y~x, span=0.0020)", I got a warning message "k-d tree limited by memory. ncmax= 4231" Does that mean the function has not been computed correctly ? If it has not, is there any way to adjust it so that it will do correctly ? Thanks. Rupen. *** If you fail to plan, you are planning to fail. _ Download music tracks from 95c here: __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess and as.POSIXct
Hi there fellow R-users,
I have just upgraded to R version 1.9.0 from R version 1.7.1 for Windows.
Im trying to use the loess smoother where the X-variable is an as.POSIXct
variable.
The following works fine with R1.7.1 but not with R1.9.0.
Here is the example:
dates<-c('2003-08-03','2003-08-10','2003-08-17','2003-08-24','2003-08-31','2
003-09-07','2003-09-14','2003-09-21','2003-09-28','2003-10-05','2003-10-12',
'2003-10-19','2003-10-26','2003-11-02','2003-11-09','2003-11-16','2003-11-23
','2003-11-30','2003-12-07','2003-12-14','2003-12-21','2003-12-28','2004-01-
04','2004-01-11','2004-01-18','2004-01-25','2004-02-01','2004-02-08','2004-0
2-15','2004-02-22','2004-02-29','2004-03-07','2004-03-14','2004-03-21','2004
-03-28','2004-04-04','2004-04-11','2004-04-18','2004-04-25','2004-05-02')
length(dates)
strptime(dates,format="%Y-%m-%d")
length(strptime(dates,format="%Y-%m-%d"))
my.df<-data.frame(Sales=rnorm(40),Dates=as.POSIXct(strptime(dates,format="%Y
-%m-%d")))
my.df
loess(Sales~Dates,my.df)
I get the following error with version 1.9.0
#Error: NA/NaN/Inf in foreign function call (arg 2)
#In addition: Warning messages:
#1: longer object length
#is not a multiple of shorter object length in: cl == c("Date",
"POSIXct", "POSIXlt")
#2: NAs introduced by coercion
Can anyone help???
Dr Wayne R. Jones
Senior Statistician / Research Analyst
KSS Limited
St James's Buildings
79 Oxford Street
Manchester M1 6SS
Tel: +44(0)161 609 4084
Mob: +44(0)7810 523 713
KSS Ltd
Seventh Floor St James's Buildings 79 Oxford Street Manchester M1 6SS England
Company Registration Number 2800886
Tel: +44 (0) 161 228 0040 Fax: +44 (0) 161 236 6305
mailto:[EMAIL PROTECTED]http://www.kssg.com
The information in this Internet email is confidential and m...{{dropped}}
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] loess' robustness weights in loess
On Fri, 9 Apr 2004, Rafael A. Irizarry wrote: > hi! > > i want to change the "robustness weights" used by loess. these > are described on page 316 of chambers and hastie's "statistical models in S" > book as > > r_i = B(e_i,6m) > > where B is tukey's biweight function, e_i are the residulas, and m is the > median average distance from 0 of the residuals. i want to > change 6m to, say, 3m. > > is there a way to do this? i cant figure it out from the help files. Well, they say loess in R is an interface to C/Fortran code, and not the same code as the S code described in Chambers & Hastie. I translated the C driver routines to R for some added flexibility. At a quick look, in function simpleLoess() you will find the weights in object `robust', calculated by Fortran function lowesw. You could replace that by a call to an R-level alternative, or play with the Fortran source code. You'll have to do what I did way back, and read the source code to see how it works in detail. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess' robustness weights in loess
hi! i want to change the "robustness weights" used by loess. these are described on page 316 of chambers and hastie's "statistical models in S" book as r_i = B(e_i,6m) where B is tukey's biweight function, e_i are the residulas, and m is the median average distance from 0 of the residuals. i want to change 6m to, say, 3m. is there a way to do this? i cant figure it out from the help files. thanks, rafael __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] loess parameters
You are probably running out of memory address space. Can you
1) Try this in 1.9.0 beta which gives a more informative error message,
and
2) Use traceback() and the debugging tools to locate the error more
exactly.
3) Consider using the options to loess to reduce the load. Loess is not
designed for smoothing a 2D grid and you appear only to want the fitted
values at your grid. If so, try the fitted() extractor function.
(Or the residual() extractor function if all you want are residuals.)
On Wed, 24 Mar 2004, Thomas Jagoe wrote:
> Hi,
> I have been successfully using the loess function for normalisation of a 2D
> array set.
> We have recently improved the quality criteria for the data and the numbers
> of data points has been reduced to around from around 1000 to 700.
> Previously the following would return the loess normalised values for
> array$logratio but I am now getting an error:
>
> > array <- read.table("A1.txt", header=T, sep="\t")
> > array$logratio<-array$logs555-array$logs647
> > array$logav<-(array$logs555+array$logs647)/2
> > library(modreg)
> > loess2d<-loess(logratio~x+y,data=array)
> > array$logratio2DLoeNorm <-array$logratio - predict(loess2d, array)
> Error in vector("double", length) : negative length vectors are not allowed
>
> I am assuming that this is due to a problem fitting the data at some
> locations and I have tried altering span without much success. Can anyone
> please advise ?
>
> With thanks
>
> Thomas
>
> __
> [EMAIL PROTECTED] mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>
>
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] loess parameters
Hi,
I have been successfully using the loess function for normalisation of a 2D
array set.
We have recently improved the quality criteria for the data and the numbers
of data points has been reduced to around from around 1000 to 700.
Previously the following would return the loess normalised values for
array$logratio but I am now getting an error:
> array <- read.table("A1.txt", header=T, sep="\t")
> array$logratio<-array$logs555-array$logs647
> array$logav<-(array$logs555+array$logs647)/2
> library(modreg)
> loess2d<-loess(logratio~x+y,data=array)
> array$logratio2DLoeNorm <-array$logratio - predict(loess2d, array)
Error in vector("double", length) : negative length vectors are not allowed
I am assuming that this is due to a problem fitting the data at some
locations and I have tried altering span without much success. Can anyone
please advise ?
With thanks
Thomas
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Loess
On Sun, 23 Mar 2003, Davorka Gulisija wrote: > I am using Loess.smooth (Modreg) in order to infer certain relationship I presume you mean loess.smooth in package modreg? (People do sometimes modify functions and (un-)capitalize the names.) > for the data set of ~130,000 observations with ~300 distinct values of > single predictor. I understand that fitted values (y-hat) are just 300 > Weighted LS fits in certain neighborhood of predictors. That is not what ?loess.smooth says it does, and it does what its help says not what you claim -- you seem to be confusing loess.smooth with loess. > I am bit confused about > how exactly is this neighborhood assigned . Say I choose spanning > parameter = .5, for each LS analysis 75000 observations should be used. > However, intuitively it doesn't seem right since points are not equally > distributed among predictors and there are many observations for a single > value of predictor. > > I would appreciate if someone could clear this for me. The help pages and their references will help you clear up your confusion: the source code is the ultimate authority. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
[R] Loess
Hi, I am using Loess.smooth (Modreg) in order to infer certain relationship for the data set of ~130,000 observations with ~300 distinct values of single predictor. I understand that fitted values (y-hat) are just 300 Weighted LS fits in certain neighborhood of predictors. I am bit confused about how exactly is this neighborhood assigned . Say I choose spanning parameter = .5, for each LS analysis 75000 observations should be used. However, intuitively it doesn't seem right since points are not equally distributed among predictors and there are many observations for a single value of predictor. I would appreciate if someone could clear this for me. Thank you, Davorka __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
RE: [R] loess
If your data are generated from a polynomial of degree p, then a local polynomial smoother of degree p will reproduce that polynomial *exactly*. You can find out how to change the span to loess by reading its help page. Andy > -Original Message- > From: wensui liu [mailto:[EMAIL PROTECTED]] > Sent: Thursday, February 20, 2003 10:30 AM > To: [EMAIL PROTECTED] > Subject: [R] loess > > > Dear Users, > > I tried to use loess to fit a simple local quadratic: > loess(y~x). But it returned the exact y value to me. (residuals==0) > > Is it too good to be true? How do I specify the SPAN in loess > function? > > Thanks a lot. > > > > > > [[alternate HTML version deleted]] > > __ > [EMAIL PROTECTED] mailing list > http://www.stat.math.ethz.ch/mailman/listinfo/r-help > -- __ [EMAIL PROTECTED] mailing list http://www.stat.math.ethz.ch/mailman/listinfo/r-help
[R] loess
Dear Users, I tried to use loess to fit a simple local quadratic: loess(y~x). But it returned the exact y value to me. (residuals==0) Is it too good to be true? How do I specify the SPAN in loess function? Thanks a lot. [[alternate HTML version deleted]] __ [EMAIL PROTECTED] mailing list http://www.stat.math.ethz.ch/mailman/listinfo/r-help
