[
https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162662#comment-14162662
]
Reza Farivar edited comment on SPARK-3785 at 10/7/14 10:12 PM:
---------------------------------------------------------------
Note: project Sumatra might become a part of Java 9, so we might get official
GPU support in Java some time in the future.
Sean, I agree that the memory copying is an overhead, but for the right
application it can become small enough to ignore. Also, you can apply a series
of operations on an RDD before moving it back to the CPU land. Think rdd.map(x
=> sine(x)*x).filter( _ < 100).map(x=> 1/x)... The distributed nature of the
RDD could mean we can run a whole stage in the GPU land, with each task would
run on different GPU in the cluster not needing to get back in the CPU land
until we get to a collect() or groupBy(), etc. I imagine we can have a subclass
of ShuffleMapTask that lives in the GPU land and would call a GPU kernel when
the runtask() is called.
In fact, given that we have a good number of specialized RDDs, I think we could
have specialized GPU versions of them easily (say, the CartesianRDD for
instance). Where it gets tougher is in the mappedRDD function, where you would
want to pass the arbitrary function to the GPU and hope that it runs.
was (Author: rfarivar):
I thought to add that the project Sumatra might become a part of Java 9, so we
might get official GPU support in Java some time in the future.
Sean, I agree that the memory copying is an overhead, but for the right
application it can become small enough to ignore. Also, you can apply a series
of operations on an RDD before moving it back to the CPU land. Think rdd.map(x
=> sine(x)*x).filter( _ < 100).map(x=> 1/x)... The distributed nature of the
RDD could mean we can run a whole stage in the GPU land, with each task would
run on different GPU in the cluster not needing to get back in the CPU land
until we get to a collect() or groupBy(), etc. I imagine we can have a subclass
of ShuffleMapTask that lives in the GPU land and would call a GPU kernel when
the runtask() is called.
In fact, given that we have a good number of specialized RDDs, I think we could
have specialized GPU versions of them easily (say, the CartesianRDD for
instance). Where it gets tougher is in the mappedRDD function, where you would
want to pass the arbitrary function to the GPU and hope that it runs.
> Support off-loading computations to a GPU
> -----------------------------------------
>
> Key: SPARK-3785
> URL: https://issues.apache.org/jira/browse/SPARK-3785
> Project: Spark
> Issue Type: Brainstorming
> Components: MLlib
> Reporter: Thomas Darimont
> Priority: Minor
>
> Are there any plans to adding support for off-loading computations to the
> GPU, e.g. via an open-cl binding?
> http://www.jocl.org/
> https://code.google.com/p/javacl/
> http://lwjgl.org/wiki/index.php?title=OpenCL_in_LWJGL
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
