Re: [R] Predicting responses using ace
Luis Pineda wrote: > I gave a quick read to the documentation again and noticed I misinterpreted > it. It was print.summary.areg.boot the method I was referring to (although > the summary error should still work). Sorry for the inconvenience > > Anyway, I used the print method on my |areg.boot| object and I got this: > -- > Apparent R2 on transformed Y scale: 0.798 > Bootstrap validated R2 : 0.681 > ... > Residuals on transformed scale: > Min 1Q Median 3Q Max > -1.071312e+00 -2.876245e-01 -3.010081e-02 2.123566e-01 1.867036e+00 > Mean S.D. > 1.290634e-17 4.462159e-01 > -- > I suppose thats the R^2 evaluated using the training set, but how do I > evaluate the performance of the model on a uncontaminated test set? Please read my last note. Bootstrap validated R2 is corrected for overfitting and is an estimate of the likely future R2 on a totally independent dataset. The bootstrap is more efficient than data splitting for this purpose. Frank > > > On 9/8/05, Luis Pineda <[EMAIL PROTECTED]> wrote: > >>I'm trying to run the print method, but according to the documentation it >>needs as a parameter an object created by |summary.areg.boot| . >> > > > [[alternative HTML version deleted]] > > __ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
I gave a quick read to the documentation again and noticed I misinterpreted it. It was print.summary.areg.boot the method I was referring to (although the summary error should still work). Sorry for the inconvenience Anyway, I used the print method on my |areg.boot| object and I got this: -- Apparent R2 on transformed Y scale: 0.798 Bootstrap validated R2 : 0.681 ... Residuals on transformed scale: Min 1Q Median 3Q Max -1.071312e+00 -2.876245e-01 -3.010081e-02 2.123566e-01 1.867036e+00 Mean S.D. 1.290634e-17 4.462159e-01 -- I suppose thats the R^2 evaluated using the training set, but how do I evaluate the performance of the model on a uncontaminated test set? On 9/8/05, Luis Pineda <[EMAIL PROTECTED]> wrote: > > I'm trying to run the print method, but according to the documentation it > needs as a parameter an object created by |summary.areg.boot| . > [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
I'm trying to run the print method, but according to the documentation it needs as a parameter an object created by |summary.areg.boot| . The thing is that |summary.areg.boot| gives me the following error: "Error in bootj[, 1] : incorrect number of dimensions", when I do the simple call ---> summary( ace.r) I started the debug browser to see what was going on inside and I noticed that 'bootj' is a numeric class variable with the same number of elements as the 'evaluation' parameter for |areg.boot|. What I found is that it has only one dimension and summary is asking for bootj[, 1], which is an error. Is that the intended behavior and I'm doing something wrong elsewhere, or should I try to adjust it by myself (to boot[1] for example)? In case the answer is the latter I would apretiate some insight about how to do it, 'cause I don't know how to edit the file. Thanks for your help, Luis Pineda On 9/7/05, Frank E Harrell Jr <[EMAIL PROTECTED]> wrote: > > Luis Pineda wrote: > > > 2.) I'm evaluating the model's goodness of fit using the Fraction of > > Variance Unexplained, which I'm calculating as: > > > > rsa = za - zs > > FVUa = sum(rsa*rsa)/(1*var(zs)) #1 is the size of the test set > > That is not corrected for overfitting. You need to use the print method > for the areg.boot object and note the Bootstrap validated R2 > > -- > Frank E Harrell Jr Professor and Chair School of Medicine > Department of Biostatistics Vanderbilt University > [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
Luis Pineda wrote:
> I sent this email before, but I got a r-help-bounce message and I don know
> if it got to the m-list. Sorry if you had already seen it.
>
> I'm using the areg.boot function to do an ace regression. So far I've been
> able to do some simple running tests to fit a model with some input data,
> predict the response with new data, and find the inverse transform of such
> prediction (apparently). The commands I'm using are the following:
>
> library(MASS)
> library(Hmisc)
> xyt = read.table("tra100unifxy.dat") #2x100 table of training data (expl.
> variables)
> zt = read.table("tra100unifzR.dat") #1x100 table of training data (resp.
> variables)
> zs = read.table("sim1z.dat") #2x1 table of new data to predict
> (expl. variables)
> xys = read.table("sim1xy.dat") #1x1 table of expected responses
> x = xyt[,1]
> y = xyt[,2]
> z = zt[,1]
> xynew = data.frame(x=xys[,1],y=xys[,2])
> ace.r = areg.boot(z ~ x + y, B = 100)
> f = Function(ace.r, ytype='inverse')
> za = f$z(predict(ace.r,xynew))
>
> I have a couple question:
>
> 1.) Is that the correct way of finding the inverse transform for the
> responses?
Yes
> 2.) I'm evaluating the model's goodness of fit using the Fraction of
> Variance Unexplained, which I'm calculating as:
>
> rsa = za - zs
> FVUa = sum(rsa*rsa)/(1*var(zs)) #1 is the size of the test set
That is not corrected for overfitting. You need to use the print method
for the areg.boot object and note the Bootstrap validated R2
>
> The thing is I'm not getting satisfactory results. Is there a way to improve
> the results of the regression?. At the moment I'm not too confident with the
> formula I'm using as a parameter for areg.boot, since the response variables
> were generated as a substantially more complex function than z = x+y. I
> don't get this formula thing yet and maybe I'm passing a totally unrelated
> formula to the function.
z ~ x + y tells areg.boot to fit a model f(z) = g(x) + h(y) which is
quite general, if x and y are additive.
Frank
>
> On 9/7/05, Frank E Harrell Jr <[EMAIL PROTECTED]> wrote:
>
>>Luis Pineda wrote:
>>
>>>Well, I had no idea, since I read this in the documentation:
>>>
>>>"|x| - for |transace| a numeric matrix. For |areg.boot| |x| may be a
>>>numeric matrix or a formula...||"
>>>
>>>Luis Pineda
>>
>>Sorry about that - you are right.
>>
>>Thomas - please debug the code to make areg.boot work with x = numeric
>>matrix, or correct the help file. Thanks -Frank
>>
>>--
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
I sent this email before, but I got a r-help-bounce message and I don know
if it got to the m-list. Sorry if you had already seen it.
I'm using the areg.boot function to do an ace regression. So far I've been
able to do some simple running tests to fit a model with some input data,
predict the response with new data, and find the inverse transform of such
prediction (apparently). The commands I'm using are the following:
library(MASS)
library(Hmisc)
xyt = read.table("tra100unifxy.dat") #2x100 table of training data (expl.
variables)
zt = read.table("tra100unifzR.dat") #1x100 table of training data (resp.
variables)
zs = read.table("sim1z.dat") #2x1 table of new data to predict
(expl. variables)
xys = read.table("sim1xy.dat") #1x1 table of expected responses
x = xyt[,1]
y = xyt[,2]
z = zt[,1]
xynew = data.frame(x=xys[,1],y=xys[,2])
ace.r = areg.boot(z ~ x + y, B = 100)
f = Function(ace.r, ytype='inverse')
za = f$z(predict(ace.r,xynew))
I have a couple question:
1.) Is that the correct way of finding the inverse transform for the
responses?
2.) I'm evaluating the model's goodness of fit using the Fraction of
Variance Unexplained, which I'm calculating as:
rsa = za - zs
FVUa = sum(rsa*rsa)/(1*var(zs)) #1 is the size of the test set
The thing is I'm not getting satisfactory results. Is there a way to improve
the results of the regression?. At the moment I'm not too confident with the
formula I'm using as a parameter for areg.boot, since the response variables
were generated as a substantially more complex function than z = x+y. I
don't get this formula thing yet and maybe I'm passing a totally unrelated
formula to the function.
On 9/7/05, Frank E Harrell Jr <[EMAIL PROTECTED]> wrote:
>
> Luis Pineda wrote:
> > Well, I had no idea, since I read this in the documentation:
> >
> > "|x| - for |transace| a numeric matrix. For |areg.boot| |x| may be a
> > numeric matrix or a formula...||"
> >
> > Luis Pineda
>
> Sorry about that - you are right.
>
> Thomas - please debug the code to make areg.boot work with x = numeric
> matrix, or correct the help file. Thanks -Frank
>
> --
> Frank E Harrell Jr Professor and Chair School of Medicine
> Department of Biostatistics Vanderbilt University
>
[[alternative HTML version deleted]]
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
Luis Pineda wrote: > Hello everybody, > > I'm a new user of R and I'm working right now with the ACE function > from the acepack library. I Have a question: Is there a way to predict > new responses using ACE? What I mean is doing something similar to the > following code that uses PPR (Projection Pursuit Regression): > > library(MASS) > x <- runif(20, 0, 1) > xnew <- runif(2000, 0, 1) > y <- sin(x) > a <- ppr(x, y, 2) > ynew <- predict(ppr, xnew) > > Any help would be much appretiated, Thanks in advance, > Luis Pineda > > __ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > Look at the areg.boot function in the Hmisc package, and its associated predict method. -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] Predicting responses using ace
Hello everybody, I'm a new user of R and I'm working right now with the ACE function from the acepack library. I Have a question: Is there a way to predict new responses using ACE? What I mean is doing something similar to the following code that uses PPR (Projection Pursuit Regression): library(MASS) x <- runif(20, 0, 1) xnew <- runif(2000, 0, 1) y <- sin(x) a <- ppr(x, y, 2) ynew <- predict(ppr, xnew) Any help would be much appretiated, Thanks in advance, Luis Pineda __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
