Re: [R] issues with calling predict.coxph.penal (survival) inside a function
Hello,
and thanks for the answer.
1) I found a work-around - in the end it is easier than thought before.
The only thing you have to do is to have the same variable name with the
new values.
So if predict(coxph.penal.fit, newdata[subset,]) does not work inside a
function, the following works:
pred_function - function(coxph_model, newdata){
#things to do before
newdata=newdata[the_subset_i_want,]
predict(coxph_model, newdata)
}
I attach a working example.
I am not sure, maybe this is even what is written in the help to
NextMethod ;)
NextMethod works by creating a special call frame for the next method.
If no new arguments are supplied, the arguments will be the same in
number, order and name as those to the current method but their values
will be promises to evaluate their name in the current method and
environment. Any named arguments matched to ... are handled specially:
they either replace existing arguments of the same name or are appended to
the argument list. They are passed on as the promise that was supplied as
an argument to the current environment. (S does this differently!) *If
they have been evaluated in the current (or a previous environment) they
remain evaluated.* (This is a complex area, and subject to change: see the
draft R Language Definition.) (Help to NextMethod )
2) Terry, I am not sure about the work-around you provided in your mail. I
want to do subsetting on newdata, not on the model.
Additionally, when trying the example you provided, I received different
results. Example is attached.
Thanks and all the best
Julian
#---
test1 - data.frame(time=c(4,3,1,1,2,2,3),
status=c(1,1,1,0,1,1,0),
x=c(0,2,1,1,1,0,0),
sex=c(0,0,0,0,1,1,1))
# Fit a stratified model
fit1 - coxph(Surv(time, status) ~ x + strata(sex), test1)
summary(fit1)
#fit stratified wih spline
fit2 - coxph(Surv(time, status) ~ pspline(x, df=2) + strata(sex), test1)
summary(fit2)
## work-around
predicting_function_which_works- function(model, newdata ){
subs -vector(mode='logical', length=nrow(newdata))
subs[3:length(subs)]- TRUE #try with first values set to false
newdata_alt-newdata
newdata-newdata[subs,]
ret-vector(mode='numeric', length=nrow(newdata_alt))
ret[!subs]- NA
ret[subs]- predict(model,newdata )
return(ret)
}
predicting_function_which_works(fit1, test1) # works
predicting_function_which_works(fit2,test1) # works
predicting_function_which_works(fit2,data.frame(time=c(4,3,1,1,2), # works
status=c(1,1,1,0,1),
x=c(0,2,1,1,2),
sex=c(1,1,0,0,1))
)
## How I understood Terry's work-around. Provides different results and
doesn't consider subset
predicting_function_2 - function(model, newdata){
subs -vector(mode='logical', length=nrow(newdata))
subs[2:length(subs)]- TRUE
newX - model.matrix(model)
newY - model$y
newfit - coxph(newY ~ newX, iter=0, init=coef(model))
newfit$var - model$var
#print(model)
#print(newfit)
#predict(newfit)
#ret=predict(newfit)
print(comparison)
print(paste( model, original prediction:, paste(predict(model),
collapse=,)))
print(paste(newfit, original prediction:, paste(predict(newfit),
collapse=,)))
ret - predict (newfit, newdata[subs,])
return(ret)
}
predicting_function_2(fit1, test1)
predicting_function_2(fit2,test1)
-Ursprüngliche Nachricht-
Von: Terry Therneau [mailto:thern...@mayo.edu]
Gesendet: Donnerstag, 14. November 2013 16:31
An: r-help@r-project.org; julian.bo...@elitepartner.de
Betreff: Re: issues with calling predict.coxph.penal (survival) inside a
function
Thanks for the reproducable example. I can confirm that it fails on my
machine using survival 2-37.5, the next soon-to-be-released version,
The issue is with NextMethod, and my assumption that the called routine
inherited everything from the parent, including the environment chain. A
simple test this AM showed me that the assumption is false. It might have
been true for Splus. Working this out may take some time -- every other
one of my wrestling matches with predict inside a function has -- and
there is a reasonable chance that it won't make this already overdue
release.
In the meantime, here is a workaround that I have sometimes used in other
situations.
Inside your function do the following: fit a new coxph model with fixed
coefficients, and do prediction on that.
myfun - function(oldfit, subset) {
newX - model.matrix(oldfit)[subset,]
newY - oldfit$y[subset]
newfit - coxph(newY ~ newX, iter=0, init=coef(oldfit))
newfit$var - oldfit$var
predict(newfit)
}
If the subset is all of a particular strata, as you indicated, then all of
the predictions will be correct. If not, then those that make use of the
the baseline hazard (type=
expect) will be incorrect but all others are ok.
Terry Therneau
On 11/14/2013 05:00
Re: [R] issues with calling predict.coxph.penal (survival) inside a function
Thanks for the reproducable example. I can confirm that it fails on my machine using
survival 2-37.5, the next soon-to-be-released version,
The issue is with NextMethod, and my assumption that the called routine inherited
everything from the parent, including the environment chain. A simple test this AM showed
me that the assumption is false. It might have been true for Splus. Working this out may
take some time -- every other one of my wrestling matches with predict inside a function
has -- and there is a reasonable chance that it won't make this already overdue release.
In the meantime, here is a workaround that I have sometimes used in other situations.
Inside your function do the following: fit a new coxph model with fixed coefficients, and
do prediction on that.
myfun - function(oldfit, subset) {
newX - model.matrix(oldfit)[subset,]
newY - oldfit$y[subset]
newfit - coxph(newY ~ newX, iter=0, init=coef(oldfit))
newfit$var - oldfit$var
predict(newfit)
}
If the subset is all of a particular strata, as you indicated, then all of the predictions
will be correct. If not, then those that make use of the the baseline hazard (type=
expect) will be incorrect but all others are ok.
Terry Therneau
On 11/14/2013 05:00 AM, r-help-requ...@r-project.org wrote:
Hello everyone,
I got an issue with calling predict.coxph.penal inside a function.
Regarding the context: My original problem is that I wrote a function that
uses predict.coxph and survfit(model) to predict
a lot of survival-curves using only the basis-curves for the strata (as
delivered by survfit(model) ) and then adapts them with
the predicted risk-scores. Because there are cases where my new data has
strata which didn't exist in the original model I exclude
them, using a Boolean vector inside the function.
I end up with a call like this: predict (coxph_model,
newdata[subscript_vector,] )
This works fine for coxph.model, but when I fit a model with a spline
(class coxph.penal), I get an error:
Error in `[.data.frame`(newdata, [subscript_vector, ) : object
'[subscript_vector ' not found
I suppose this is because of NextMethod, but I am not sure how to work
around it. I also read a little bit about all those
matching-and-frame-issues,
But must confess I am not really into it.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] issues with calling predict.coxph.penal (survival) inside a function - subset-vector not found. Because of NextMethod?
Hello everyone,
I got an issue with calling predict.coxph.penal inside a function.
Regarding the context: My original problem is that I wrote a function that
uses predict.coxph and survfit(model) to predict
a lot of survival-curves using only the basis-curves for the strata (as
delivered by survfit(model) ) and then adapts them with
the predicted risk-scores. Because there are cases where my new data has
strata which didn't exist in the original model I exclude
them, using a Boolean vector inside the function.
I end up with a call like this: predict (coxph_model,
newdata[subscript_vector,] )
This works fine for coxph.model, but when I fit a model with a spline
(class coxph.penal), I get an error:
Error in `[.data.frame`(newdata, [subscript_vector, ) : object
'[subscript_vector ' not found
I suppose this is because of NextMethod, but I am not sure how to work
around it. I also read a little bit about all those
matching-and-frame-issues,
But must confess I am not really into it.
I attach a reproducible example.
Any help or suggestions of work-arounds will be appreciated.
Thanks
Julian
version
_
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 0.1
year 2013
month 05
day16
svn rev62743
language R
version.string R version 3.0.1 (2013-05-16)
nickname Good Sport
##TEST-DATA
# Create the simplest test data set
test1 - data.frame(time=c(4,3,1,1,2,2,3),
status=c(1,1,1,0,1,1,0),
x=c(0,2,1,1,1,0,0),
sex=c(0,0,0,0,1,1,1))
# Fit a stratified model
fit1 - coxph(Surv(time, status) ~ x + strata(sex), test1)
summary(fit1)
#fit stratified wih spline
fit2 - coxph(Surv(time, status) ~ pspline(x, df=2) + strata(sex), test1)
summary(fit2)
#function to predict within
predicting_function - function(model, newdata){
subs -vector(mode='logical', length=nrow(newdata))
subs[1:length(subs)]- TRUE
ret - predict (model, newdata=newdata[subs,])
return(ret)
}
predicting_function(fit1, test1) # works
predicting_function(fit2,test1) #doesnt work - Error in
`[.data.frame`(newdata, subs, ) : object 'subs' not found
# probably because of NextMethod
#
traceback()
#12: `[.data.frame`(newdata, subs, )
#11: newdata[subs, ]
#10: is.data.frame(data)
#9: model.frame.default(data = newdata[subs, ], formula = ~pspline(x,
# df = 2) + strata(sex), na.action = function (object, ...)
# object)
#8: model.frame(data = newdata[subs, ], formula = ~pspline(x, df = 2) +
# strata(sex), na.action = function (object, ...)
# object)
#7: eval(expr, envir, enclos)
#6: eval(tcall, parent.frame())
#5: predict.coxph(model, newdata = newdata[subs, ])
#4: NextMethod(predict, object, ...)
#3: predict.coxph.penal(model, newdata = newdata[subs, ])
#2: predict(model, newdata = newdata[subs, ]) at #5
#1: predicting_function(fit2, test1)
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] issues with calling predict.coxph.penal (survival) inside a function - subset-vector not found. Because of NextMethod?
Works for me:
predicting_function(fit2,test1)
1 2 3 4 5 6 7
-1.0481141 0.1495946 0.4492597 0.4492597 0.9982492 -0.4991246 -0.4991246
Best
Simon
On 13 Nov 2013, at 15:46, julian.bo...@elitepartner.de wrote:
Hello everyone,
I got an issue with calling predict.coxph.penal inside a function.
Regarding the context: My original problem is that I wrote a function that
uses predict.coxph and survfit(model) to predict
a lot of survival-curves using only the basis-curves for the strata (as
delivered by survfit(model) ) and then adapts them with
the predicted risk-scores. Because there are cases where my new data has
strata which didn't exist in the original model I exclude
them, using a Boolean vector inside the function.
I end up with a call like this: predict (coxph_model,
newdata[subscript_vector,] )
This works fine for coxph.model, but when I fit a model with a spline
(class coxph.penal), I get an error:
Error in `[.data.frame`(newdata, [subscript_vector, ) : object
'[subscript_vector ' not found
I suppose this is because of NextMethod, but I am not sure how to work
around it. I also read a little bit about all those
matching-and-frame-issues,
But must confess I am not really into it.
I attach a reproducible example.
Any help or suggestions of work-arounds will be appreciated.
Thanks
Julian
version
_
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 0.1
year 2013
month 05
day16
svn rev62743
language R
version.string R version 3.0.1 (2013-05-16)
nickname Good Sport
##TEST-DATA
# Create the simplest test data set
test1 - data.frame(time=c(4,3,1,1,2,2,3),
status=c(1,1,1,0,1,1,0),
x=c(0,2,1,1,1,0,0),
sex=c(0,0,0,0,1,1,1))
# Fit a stratified model
fit1 - coxph(Surv(time, status) ~ x + strata(sex), test1)
summary(fit1)
#fit stratified wih spline
fit2 - coxph(Surv(time, status) ~ pspline(x, df=2) + strata(sex), test1)
summary(fit2)
#function to predict within
predicting_function - function(model, newdata){
subs -vector(mode='logical', length=nrow(newdata))
subs[1:length(subs)]- TRUE
ret - predict (model, newdata=newdata[subs,])
return(ret)
}
predicting_function(fit1, test1) # works
predicting_function(fit2,test1) #doesnt work - Error in
`[.data.frame`(newdata, subs, ) : object 'subs' not found
# probably because of NextMethod
#
traceback()
#12: `[.data.frame`(newdata, subs, )
#11: newdata[subs, ]
#10: is.data.frame(data)
#9: model.frame.default(data = newdata[subs, ], formula = ~pspline(x,
# df = 2) + strata(sex), na.action = function (object, ...)
# object)
#8: model.frame(data = newdata[subs, ], formula = ~pspline(x, df = 2) +
# strata(sex), na.action = function (object, ...)
# object)
#7: eval(expr, envir, enclos)
#6: eval(tcall, parent.frame())
#5: predict.coxph(model, newdata = newdata[subs, ])
#4: NextMethod(predict, object, ...)
#3: predict.coxph.penal(model, newdata = newdata[subs, ])
#2: predict(model, newdata = newdata[subs, ]) at #5
#1: predicting_function(fit2, test1)
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
