Great. Thanks a lot. On 23 Sep 2014 18:44, "Davies Liu-2 [via Apache Spark User List]" < ml-node+s1001560n14908...@n3.nabble.com> wrote:
> Right now, there is no way to access JVM in Python worker, in order > to make this happen, we need to do: > > 1. setup py4j in Python worker > 2. serialize the JVM objects and transfer to executors > 3. link the JVM objects and py4j together to get an interface > > Before these happens, maybe you could try to setup a service > for the model (such as RESTful service), access it map via RPC. > > On Tue, Sep 23, 2014 at 9:48 AM, Tamas Jambor <[hidden email] > <http://user/SendEmail.jtp?type=node&node=14908&i=0>> wrote: > > > Hi Davies, > > > > Thanks for the reply. I saw that you guys do that way in the code. Is > > there no other way? > > > > I have implemented all the predict functions in scala, so I prefer not > > to reimplement the whole thing in python. > > > > thanks, > > > > > > On Tue, Sep 23, 2014 at 5:40 PM, Davies Liu <[hidden email] > <http://user/SendEmail.jtp?type=node&node=14908&i=1>> wrote: > >> You should create a pure Python object (copy the attributes from Java > object), > >> then it could be used in map. > >> > >> Davies > >> > >> On Tue, Sep 23, 2014 at 8:48 AM, jamborta <[hidden email] > <http://user/SendEmail.jtp?type=node&node=14908&i=2>> wrote: > >>> Hi all, > >>> > >>> I have a java object that contains a ML model which I would like to > use for > >>> prediction (in python). I just want to iterate the data through a > mapper and > >>> predict for each value. Unfortunately, this fails when it tries to > serialise > >>> the object to sent it to the nodes. > >>> > >>> Is there a trick around this? Surely, this object could be picked up > by > >>> reference at the nodes. > >>> > >>> many thanks, > >>> > >>> > >>> > >>> -- > >>> View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/access-javaobject-in-rdd-map-tp14898.html > >>> Sent from the Apache Spark User List mailing list archive at > Nabble.com. > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [hidden email] > <http://user/SendEmail.jtp?type=node&node=14908&i=3> > >>> For additional commands, e-mail: [hidden email] > <http://user/SendEmail.jtp?type=node&node=14908&i=4> > >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > <http://user/SendEmail.jtp?type=node&node=14908&i=5> > For additional commands, e-mail: [hidden email] > <http://user/SendEmail.jtp?type=node&node=14908&i=6> > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/access-javaobject-in-rdd-map-mllib-tp14898p14908.html > To unsubscribe from access javaobject in rdd map (mllib), click here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=14898&code=amFtYm9ydGFAZ21haWwuY29tfDE0ODk4fC00Mjk2ODU1NTM=> > . > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/access-javaobject-in-rdd-map-mllib-tp14898p14920.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
