Re: UDF in SparkR

2016-08-17 Thread Yann-Aël Le Borgne
r@spark.apache.org> > > > > Hi, > > Is there is any way of using UDF in SparkR ? > > Regards, > Yogesh > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > > > -- =

Re: Avoid Cartesian product in calculating a distance matrix?

2016-08-06 Thread Yann-Aël Le Borgne
peration that > just requires a much larger cluster? > > Thank you, > > Paschalis > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- = Yann-Aël Le Borgne Machine Learning Group Université Libre de Bruxelles http://mlg.ulb.ac.be http://www.ulb.ac.be/di/map/yleborgn =

Spark R 2.0 dapply very slow

2016-07-31 Thread Yann-Aël Le Borgne
tManager: Stage 64 contains a task of very large size (16411 KB). The maximum recommended task size is 100 KB.). Why is this 100KB limit so low? I am using R 3.3.0 on Mac OS 10.10.5 Any insight welcome, Best, Yann-Aël -- ============= Yann-Aël Le Borgne Machine Learning G