Re: [R] Predicting responses using ace
I'm trying to run the print method, but according to the documentation it needs as a parameter an object created by |summary.areg.boot| . The thing is that |summary.areg.boot| gives me the following error: Error in bootj[, 1] : incorrect number of dimensions, when I do the simple call --- summary( ace.r) I started the debug browser to see what was going on inside and I noticed that 'bootj' is a numeric class variable with the same number of elements as the 'evaluation' parameter for |areg.boot|. What I found is that it has only one dimension and summary is asking for bootj[, 1], which is an error. Is that the intended behavior and I'm doing something wrong elsewhere, or should I try to adjust it by myself (to boot[1] for example)? In case the answer is the latter I would apretiate some insight about how to do it, 'cause I don't know how to edit the file. Thanks for your help, Luis Pineda On 9/7/05, Frank E Harrell Jr [EMAIL PROTECTED] wrote: Luis Pineda wrote: 2.) I'm evaluating the model's goodness of fit using the Fraction of Variance Unexplained, which I'm calculating as: rsa = za - zs FVUa = sum(rsa*rsa)/(1*var(zs)) #1 is the size of the test set That is not corrected for overfitting. You need to use the print method for the areg.boot object and note the Bootstrap validated R2 -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
I gave a quick read to the documentation again and noticed I misinterpreted it. It was print.summary.areg.boot the method I was referring to (although the summary error should still work). Sorry for the inconvenience Anyway, I used the print method on my |areg.boot| object and I got this: -- Apparent R2 on transformed Y scale: 0.798 Bootstrap validated R2 : 0.681 ... Residuals on transformed scale: Min 1Q Median 3Q Max -1.071312e+00 -2.876245e-01 -3.010081e-02 2.123566e-01 1.867036e+00 Mean S.D. 1.290634e-17 4.462159e-01 -- I suppose thats the R^2 evaluated using the training set, but how do I evaluate the performance of the model on a uncontaminated test set? On 9/8/05, Luis Pineda [EMAIL PROTECTED] wrote: I'm trying to run the print method, but according to the documentation it needs as a parameter an object created by |summary.areg.boot| . [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
Luis Pineda wrote: I gave a quick read to the documentation again and noticed I misinterpreted it. It was print.summary.areg.boot the method I was referring to (although the summary error should still work). Sorry for the inconvenience Anyway, I used the print method on my |areg.boot| object and I got this: -- Apparent R2 on transformed Y scale: 0.798 Bootstrap validated R2 : 0.681 ... Residuals on transformed scale: Min 1Q Median 3Q Max -1.071312e+00 -2.876245e-01 -3.010081e-02 2.123566e-01 1.867036e+00 Mean S.D. 1.290634e-17 4.462159e-01 -- I suppose thats the R^2 evaluated using the training set, but how do I evaluate the performance of the model on a uncontaminated test set? Please read my last note. Bootstrap validated R2 is corrected for overfitting and is an estimate of the likely future R2 on a totally independent dataset. The bootstrap is more efficient than data splitting for this purpose. Frank On 9/8/05, Luis Pineda [EMAIL PROTECTED] wrote: I'm trying to run the print method, but according to the documentation it needs as a parameter an object created by |summary.areg.boot| . [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
Luis Pineda wrote: I sent this email before, but I got a r-help-bounce message and I don know if it got to the m-list. Sorry if you had already seen it. I'm using the areg.boot function to do an ace regression. So far I've been able to do some simple running tests to fit a model with some input data, predict the response with new data, and find the inverse transform of such prediction (apparently). The commands I'm using are the following: library(MASS) library(Hmisc) xyt = read.table(tra100unifxy.dat) #2x100 table of training data (expl. variables) zt = read.table(tra100unifzR.dat) #1x100 table of training data (resp. variables) zs = read.table(sim1z.dat) #2x1 table of new data to predict (expl. variables) xys = read.table(sim1xy.dat) #1x1 table of expected responses x = xyt[,1] y = xyt[,2] z = zt[,1] xynew = data.frame(x=xys[,1],y=xys[,2]) ace.r = areg.boot(z ~ x + y, B = 100) f = Function(ace.r, ytype='inverse') za = f$z(predict(ace.r,xynew)) I have a couple question: 1.) Is that the correct way of finding the inverse transform for the responses? Yes 2.) I'm evaluating the model's goodness of fit using the Fraction of Variance Unexplained, which I'm calculating as: rsa = za - zs FVUa = sum(rsa*rsa)/(1*var(zs)) #1 is the size of the test set That is not corrected for overfitting. You need to use the print method for the areg.boot object and note the Bootstrap validated R2 The thing is I'm not getting satisfactory results. Is there a way to improve the results of the regression?. At the moment I'm not too confident with the formula I'm using as a parameter for areg.boot, since the response variables were generated as a substantially more complex function than z = x+y. I don't get this formula thing yet and maybe I'm passing a totally unrelated formula to the function. z ~ x + y tells areg.boot to fit a model f(z) = g(x) + h(y) which is quite general, if x and y are additive. Frank On 9/7/05, Frank E Harrell Jr [EMAIL PROTECTED] wrote: Luis Pineda wrote: Well, I had no idea, since I read this in the documentation: |x| - for |transace| a numeric matrix. For |areg.boot| |x| may be a numeric matrix or a formula...|| Luis Pineda Sorry about that - you are right. Thomas - please debug the code to make areg.boot work with x = numeric matrix, or correct the help file. Thanks -Frank -- -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Predicting responses using ace
Luis Pineda wrote: Hello everybody, I'm a new user of R and I'm working right now with the ACE function from the acepack library. I Have a question: Is there a way to predict new responses using ACE? What I mean is doing something similar to the following code that uses PPR (Projection Pursuit Regression): library(MASS) x - runif(20, 0, 1) xnew - runif(2000, 0, 1) y - sin(x) a - ppr(x, y, 2) ynew - predict(ppr, xnew) Any help would be much appretiated, Thanks in advance, Luis Pineda __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Look at the areg.boot function in the Hmisc package, and its associated predict method. -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
