Its a bit tricky - if the users data is represented in a DataFrame or
Dataset then its much easier. Assuming that the function is going to be
called from the driver program (e.g. not inside of a transformation or
action) then you can use the Py4J context to make the calls. You might find
looking at wrapper.py in the ml directory to help some.
On Tue, Apr 12, 2016 at 4:30 PM, AlexG wrote:
> I have Scala Spark code for computing a matrix factorization. I'd like to
> make it possible to use this code from PySpark, so users can pass in a
> python RDD and receive back one without knowing or caring that Scala code
> is
> being called.
>
> Please point me to an example of code (e.g. somewhere in the Spark
> codebase,
> if it's clean enough) from which I can learn how to do this.
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/how-to-write-pyspark-interface-to-scala-code-tp26765.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
