RE: questions about using dapply

2016-09-12 Thread xingye
ot;cr_cd", 1:12, sep = "")], Margin=2, function(y){ y %in% c(61, 99)})},schema )The error shows Error in match.fun(FUN) : argument "FUN" is missing, with no default From: felixcheun...@hotmail.com To: user@spark.apache.org; tracy.up...@gmail.com Subject: Re: questions abo

Re: questions about using dapply

2016-09-10 Thread Felix Cheung
)], MARGIN=2, FUN = function(y){ y %in% c(61, 99)})}) # dapplyCollect does not require the schema parameter _ From: xingye <tracy.up...@gmail.com<mailto:tracy.up...@gmail.com>> Sent: Friday, September 9, 2016 10:35 AM Subject: questions about using dapply To: &l

questions about using dapply

2016-09-09 Thread xingye
I have a question about using UDF in SparkR. I’m converting some R code into SparkR. • The original R code is :cols_in <- apply(df[, paste("cr_cd", 1:12, sep = "")], MARGIN = 2, FUN = "%in%", c(61, 99)) • If I use dapply and put the original apply function as a function for dapply,cols_in