Re: [Scikit-learn-general] Spark-backed implementations of scikit-learn estimators

2013-11-27 Thread Olivier Grisel
013/11/27 Nick Pentreath nick.pentre...@gmail.com: CC'ing Spark Dev list I have been thinking about this for quite a while and would really love to see this happen. Most of my pipeline ends up in Scala/Spark these days - which I love, but it is partly because I am reliant on custom Hadoop

Re: Modifying RDD.scala

2013-11-27 Thread Gerard Maas
From the looks of your exception, you modified your local class, but you forgot to deploy those local changes to the cluster. This error msg: classdesc serialVersionUID = 5151096093324583655, local class serialVersionUID = 9012954318378784201 indicates that a version being de-serialized is

Re: Modifying RDD.scala

2013-11-27 Thread Zuhair Khayyat
Dear Gerard, All servers share the spark binaries through NFS; It is unlikly that other servers contains the old class. I will test later with one server and see if I got the same problem.. Regards, Zuhair Khayyat On Nov 27, 2013, at 6:29 PM, Gerard Maas gerard.m...@gmail.com wrote: From