Re: Spark ML Pipeline inaccessible types

2015-03-25 Thread zapletal-martin
Thanks Peter, I ended up doing something similar. I however consider both the approaches you mentioned bad practices which is why I was looking for a solution directly supported by the current code. I can work with that now, but it does not seem to be the proper solution. Regards, Mar

Re: Spark ML Pipeline inaccessible types

2015-03-25 Thread zapletal-martin
Sean, thanks for your response. I am familiar with NoSuchMethodException in general, but I think it is not the case this time. The code actually attempts to get parameter by name using val m = this.getClass.getMethodName (paramName). This may be a bug, but it is only a side effect caused b

Spark ML Pipeline inaccessible types

2015-03-25 Thread zapletal-martin
Hi, I have started implementing a machine learning pipeline using Spark 1.3.0 and the new pipelining API and DataFrames. I got to a point where I have my training data set prepared using a sequence of Transformers, but I am struggling to actually train a model and use it for predictions.