)
>
>
>
>
>
>
>
> On the other hand, if we run in the local, everything works fine.
>
>
>
> Just want to know, if there is any successful case that run machine
> learning models on the workers.
>
>
>
>
>
> Thanks,
>
> Jianhong
>
&g
guha <guha.a...@gmail.com>;
user@spark.apache.org
Subject: Re: Can't load a RandomForestClassificationModel in Spark job
Hey there,
Creating a new SparkContext on workers will not work, only the driver is
allowed to own a SparkContext. Are you trying to distribute your model to
workers so you can cr
that run machine
> learning models on the workers.
>
>
>
>
>
> Thanks,
>
> Jianhong
>
>
>
>
>
> *From:* Sumona Routh [mailto:sumos...@gmail.com]
> *Sent:* Thursday, January 12, 2017 6:20 PM
> *To:* ayan guha <guha.a...@gmail.com>; user@s
; user@spark.apache.org
Subject: Re: Can't load a RandomForestClassificationModel in Spark job
Yes, I save it to S3 in a different process. It is actually the
RandomForestClassificationModel.load method (passed an s3 path) where I run
into problems.
When you say you load it during map stages, do you mean th
Yes, I save it to S3 in a different process. It is actually the
RandomForestClassificationModel.load method (passed an s3 path) where I run
into problems.
When you say you load it during map stages, do you mean that you are able
to directly load a model from inside of a transformation? When I try
Hi
Given training and predictions are two different applications, I typically
save model objects to hdfs and load it back during prediction map stages.
Best
Ayan
On Fri, 13 Jan 2017 at 5:39 am, Sumona Routh wrote:
> Hi all,
> I've been working with Spark mllib 2.0.2