[R] local regression using loess

[cindy Guo]

Hi, All,

I have a dataset with binary response ( 0 and 1) and some numerical
covariates. I know I can use logistic regression to fit the data. But I want
to consider more locally. So I am wondering how can I fit the data with
'loess' function in R? And what will be the response: 0/1 or the probability
in either group like in logistic regression?

Thank you,
Cindy

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[Bert Gunter]

-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On
Behalf Of cindy Guo
Sent: Monday, July 27, 2009 4:06 PM
To: r-help@r-project.org
Subject: [R] local regression using loess

Hi, All,

I have a dataset with binary response ( 0 and 1) and some numerical
covariates. I know I can use logistic regression to fit the data. But I want
to consider more locally. So I am wondering how can I fit the data with
'loess' function in R? And what will be the response: 0/1 or the probability
in either group like in logistic regression?

-- Neither. Loess is an algorithm that smoothly "interpolates" the data. It
makes no claim of modeling the probability for a binary response variable. 

-- Bert Gunter
Genentech Nonclinical Statistics

Thank you,
Cindy

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[cindy Guo]

Hi, Bert,

Thanks for the response. But then in this case, can I use loess to fit the
data? If yes, then how to interpret the results?

Cindy

On Mon, Jul 27, 2009 at 4:32 PM, Bert Gunter  wrote:

>  -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On
> Behalf Of cindy Guo
> Sent: Monday, July 27, 2009 4:06 PM
> To: r-help@r-project.org
> Subject: [R] local regression using loess
>
> Hi, All,
>
> I have a dataset with binary response ( 0 and 1) and some numerical
> covariates. I know I can use logistic regression to fit the data. But I
> want
> to consider more locally. So I am wondering how can I fit the data with
> 'loess' function in R? And what will be the response: 0/1 or the
> probability
> in either group like in logistic regression?
>
> -- Neither. Loess is an algorithm that smoothly "interpolates" the data. It
> makes no claim of modeling the probability for a binary response variable.
>
> -- Bert Gunter
> Genentech Nonclinical Statistics
>
> Thank you,
> Cindy
>
>[[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[Ryan]

> >
> > Hi, All,
> >
> > I have a dataset with binary response ( 0 and 1) and some numerical
> > covariates. I know I can use logistic regression to fit the data. But I
> > want
> > to consider more locally. So I am wondering how can I fit the data with
> > 'loess' function in R? And what will be the response: 0/1 or the
> > probability
> > in either group like in logistic regression?
> >
> > -- Neither. Loess is an algorithm that smoothly "interpolates" the data. It
> > makes no claim of modeling the probability for a binary response variable.
> >
> > -- Bert Gunter
> > Genentech Nonclinical Statistics
> >
> > Thank you,
> > Cindy
> >
> >[[alternative HTML version deleted]]
> >

Actually, loess is much more than an "interpolant".  I wouldn't 
even call it that.  It is a local regression  technique that comes 
with all the equipment you get in classical regression.  But it 
is meant for normal-like errors, which is not what you have.

I would recommend that you take a look at the locfit package.  
It fits local likelihood models.  I've never tried it with binary data, 
but if y is your 0/1 response and x is a covariate, you might try 
something like:

locfit(y ~ x, ..., family="binomial")

If you have a good library at your disposal, try picking up Loader's 
book "Local Regression and Likelihood".

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[cindy Guo]

Hi, Ryan,

Thank you for the information. I tried it. But there are some error
messages.

When I use fit <- locfit(Y~X1*X2,family='binomial'), the error message is
error lfproc(x, y, weights = weights, cens = cens, base = base, geth =
geth,  :
  compparcomp: parameters out of bounds

And when I use fit <- locfit(Y~X1*X2), the error message is
error lfproc(x, y, weights = weights, cens = cens, base = base, geth =
geth,  :
  newsplit: out of vertex space

This happens sometimes, not every time for different data. Do you know
what's the reason?

Thank you,

Cindy

On Mon, Jul 27, 2009 at 5:25 PM, Ryan  wrote:

> > >
> > > Hi, All,
> > >
> > > I have a dataset with binary response ( 0 and 1) and some numerical
> > > covariates. I know I can use logistic regression to fit the data. But I
> > > want
> > > to consider more locally. So I am wondering how can I fit the data with
> > > 'loess' function in R? And what will be the response: 0/1 or the
> > > probability
> > > in either group like in logistic regression?
> > >
> > > -- Neither. Loess is an algorithm that smoothly "interpolates" the
> data. It
> > > makes no claim of modeling the probability for a binary response
> variable.
> > >
> > > -- Bert Gunter
> > > Genentech Nonclinical Statistics
> > >
> > > Thank you,
> > > Cindy
> > >
> > >[[alternative HTML version deleted]]
> > >
>
> Actually, loess is much more than an "interpolant".  I wouldn't
> even call it that.  It is a local regression  technique that comes
> with all the equipment you get in classical regression.  But it
> is meant for normal-like errors, which is not what you have.
>
> I would recommend that you take a look at the locfit package.
> It fits local likelihood models.  I've never tried it with binary data,
> but if y is your 0/1 response and x is a covariate, you might try
> something like:
>
> locfit(y ~ x, ..., family="binomial")
>
> If you have a good library at your disposal, try picking up Loader's
> book "Local Regression and Likelihood".
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[Alain Zuur]

cindy Guo wrote:
> 
> Hi, All,
> 
> I have a dataset with binary response ( 0 and 1) and some numerical
> covariates. I know I can use logistic regression to fit the data. But I
> want
> to consider more locally. So I am wondering how can I fit the data with
> 'loess' function in R? And what will be the response: 0/1 or the
> probability
> in either group like in logistic regression?
> 
> Thank you,
> Cindy
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 


Why don't you fit a GAM with a logistic link function and binomial
distirbution?

Alain


-

Dr. Alain F. Zuur
First author of:

1. Analysing Ecological Data (2007).
Zuur, AF, Ieno, EN and Smith, GM. Springer. 680 p.

2. Mixed effects models and extensions in ecology with R. (2009).
Zuur, AF, Ieno, EN, Walker, N, Saveliev, AA, and Smith, GM. Springer.

3. A Beginner's Guide to R (2009).
Zuur, AF, Ieno, EN, Meesters, EHWG. Springer


Statistical consultancy, courses, data analysis and software
Highland Statistics Ltd.
6 Laverock road
UK - AB41 6FN Newburgh
Email: highs...@highstat.com
URL: www.highstat.com



-- 
View this message in context: 
http://www.nabble.com/local-regression-using-loess-tp24689834p24696908.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[Bert Gunter]

Actually, loess is much more than an "interpolant".  I wouldn't 
even call it that.  It is a local regression  technique that comes 
with all the equipment you get in classical regression.  But it 
is meant for normal-like errors, which is not what you have.

-- This is misleading. The local smoother part makes a big difference. For
example df (and standard tests, consequently)are not defined as in
conventional multiple regression (though the enp argument gives you
something like df). Also, it was specifically designed for _non_-normal
errors -- specifically, long-tailed distributions -- via use of the
"symmetric" family argument which fits via a re-descending M-estimator not
Gaussian likelihood = least squares.

But you are certainly free to characterize it as you think appropriate if
you do not think "interpolant" is reasonable...

-- Bert



I would recommend that you take a look at the locfit package.  
It fits local likelihood models.  I've never tried it with binary data, 
but if y is your 0/1 response and x is a covariate, you might try 
something like:

locfit(y ~ x, ..., family="binomial")

If you have a good library at your disposal, try picking up Loader's 
book "Local Regression and Likelihood".

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[Ryan Hafen]

Are X1 and X2 both numeric?  You might want to get them on equivalent  
scales, and also play around with the smoothing parameter.

Try something like:

fit <- locfit(Y ~ lp(X1, X2, nn=___, scale=TRUE), family="binomial")

and see what happens for different values of nn (try values between 0  
and 1 and then some larger than one).  I can't be much more help  
without data.


On Jul 27, 2009, at 9:41 PM, cindy Guo wrote:

> Hi, Ryan,
>
> Thank you for the information. I tried it. But there are some error  
> messages.
>
> When I use fit <- locfit(Y~X1*X2,family='binomial'), the error  
> message is
> error lfproc(x, y, weights = weights, cens = cens, base = base, geth  
> = geth,  :
>   compparcomp: parameters out of bounds
>
> And when I use fit <- locfit(Y~X1*X2), the error message is
> error lfproc(x, y, weights = weights, cens = cens, base = base, geth  
> = geth,  :
>   newsplit: out of vertex space
>
> This happens sometimes, not every time for different data. Do you  
> know what's the reason?
>
> Thank you,
>
> Cindy
>
> On Mon, Jul 27, 2009 at 5:25 PM, Ryan  wrote:
> > >
> > > Hi, All,
> > >
> > > I have a dataset with binary response ( 0 and 1) and some  
> numerical
> > > covariates. I know I can use logistic regression to fit the  
> data. But I
> > > want
> > > to consider more locally. So I am wondering how can I fit the  
> data with
> > > 'loess' function in R? And what will be the response: 0/1 or the
> > > probability
> > > in either group like in logistic regression?
> > >
> > > -- Neither. Loess is an algorithm that smoothly "interpolates"  
> the data. It
> > > makes no claim of modeling the probability for a binary response  
> variable.
> > >
> > > -- Bert Gunter
> > > Genentech Nonclinical Statistics
> > >
> > > Thank you,
> > > Cindy
> > >
> > >[[alternative HTML version deleted]]
> > >
>
> Actually, loess is much more than an "interpolant".  I wouldn't
> even call it that.  It is a local regression  technique that comes
> with all the equipment you get in classical regression.  But it
> is meant for normal-like errors, which is not what you have.
>
> I would recommend that you take a look at the locfit package.
> It fits local likelihood models.  I've never tried it with binary  
> data,
> but if y is your 0/1 response and x is a covariate, you might try
> something like:
>
> locfit(y ~ x, ..., family="binomial")
>
> If you have a good library at your disposal, try picking up Loader's
> book "Local Regression and Likelihood".
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[Ryan]

Bert Gunter  gene.com> writes:

> 
> 
> Actually, loess is much more than an "interpolant".  I wouldn't 
> even call it that.  It is a local regression  technique that comes 
> with all the equipment you get in classical regression.  But it 
> is meant for normal-like errors, which is not what you have.
> 


Bert - when I hear "interpolate", I think of connecting the 
data points, like using something like divided differences 
or hermite interpolation, so I thought that's what you 
meant.  Sorry for the misunderstanding.  

True that loess was designed to be robust, but when I 
said it is meant for normal-like errors, I was referring 
to loess with statistical procedures analagous to the 
classical regression setting, such as confidence intervals, 
anova, etc. (see "Locally Weighted Regression: An 
Approach to Regression Analysis by Local Fitting", 1988).

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] local regression using loess

[cindy Guo]

Bert, Ryan, Alain,

You suggestions are very helpful. Thank you. I learned a lot from the
discussion.

Cindy

On Tue, Jul 28, 2009 at 8:53 AM, Ryan  wrote:

> Bert Gunter  gene.com> writes:
>
> >
> >
> > Actually, loess is much more than an "interpolant".  I wouldn't
> > even call it that.  It is a local regression  technique that comes
> > with all the equipment you get in classical regression.  But it
> > is meant for normal-like errors, which is not what you have.
> >
>
>
> Bert - when I hear "interpolate", I think of connecting the
> data points, like using something like divided differences
> or hermite interpolation, so I thought that's what you
> meant.  Sorry for the misunderstanding.
>
> True that loess was designed to be robust, but when I
> said it is meant for normal-like errors, I was referring
> to loess with statistical procedures analagous to the
> classical regression setting, such as confidence intervals,
> anova, etc. (see "Locally Weighted Regression: An
> Approach to Regression Analysis by Local Fitting", 1988).
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.