Hi,
completely agree with Hao. In case you are using YARN try to see the EMR
documentation on how to enable GPU as resource in YARN before trying to use
that in SPARK.
This is one of the most exciting features of SPARK 3, and you can reap huge
benefits out of it :)
Regards,
Gourav Sengupta
On
(I apologize, I totally missed that this should use GPUs because of RAPIDS.
Ignore my previous. But yeah it's more a RAPIDS question.)
On Fri, Apr 9, 2021 at 12:09 PM HaoZ wrote:
> Hi Martin,
>
> I tested the local mode in Spark on Rapids Accelerator and it works fine
> for
> me.
> The only
Hi Martin,
I tested the local mode in Spark on Rapids Accelerator and it works fine for
me.
The only possible issue is the CUDA 11.2 however the supported CUDA version
as per https://nvidia.github.io/spark-rapids/docs/download.html is 11.0.
Here is a quick test using Spark local mode.
Note: When
Hey Martin,
I would encourage you to file issues in the spark-rapids repo for questions
with that plugin: https://github.com/NVIDIA/spark-rapids/issues
I'm assuming the query ran and you looked at the sql UI or the .expalin()
output and it was on cpu and not gpu? I am assuming you have the
I don't see anything in this job that would use a GPU?
On Fri, Apr 9, 2021 at 11:19 AM Martin Somers wrote:
>
> Hi Everyone !!
>
> Im trying to get on premise GPU instance of Spark 3 running on my ubuntu
> box, and I am following:
>
>
Hi Everyone !!
Im trying to get on premise GPU instance of Spark 3 running on my ubuntu
box, and I am following:
https://nvidia.github.io/spark-rapids/docs/get-started/getting-started-on-prem.html#example-join-operation
Anyone with any insight into why a spark job isnt being ran on the GPU -
