Re: How to train and predict in parallel via Spark MLlib?

I put a simple example here: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/1233855/3877825096667927/588180/d9d264e39a.html On Thu, Feb 18, 2016 at 6:47 AM Игорь Ляхов wrote: > Xiangrui, thnx for your answer! > Could you

Re: How to train and predict in parallel via Spark MLlib?

Xiangrui, thnx for your answer! Could you clarify some details? What do you mean "I can trigger training jobs in different threads on the driver"? I have 4-machine cluster (It will grow in future), and I wish use them in parallel for training and predicting. Do you have any example? It will be

Re: How to train and predict in parallel via Spark MLlib?

If you have a big cluster, you can trigger training jobs in different threads on the driver. Putting RDDs inside an RDD won't work. -Xiangrui On Thu, Feb 18, 2016, 4:28 AM Igor L. wrote: > Good day, Spark team! > I have to solve regression problem for different restricitons.

How to train and predict in parallel via Spark MLlib?

Good day, Spark team! I have to solve regression problem for different restricitons. There is a bunch of criteria and rules for them, I have to build model and make predictions for each, combine all and save. So, now my solution looks like: criteria2Rules: List[(String, Set[String])]