Thanks Sean. Also, I observed that lots of things are not supported in GPU by NVIDIA. E.g. nested types/decimal type/Udfs etc. So, will it use CPU automatically for running those tasks which require nested types or will it run on GPU and fail.
Thanks Rajat On Sat, Aug 13, 2022, 18:54 Sean Owen <sro...@gmail.com> wrote: > Spark does not use GPUs itself, but tasks you run on Spark can. > The only 'support' there is is for requesting GPUs as resources for tasks, > so it's just a question of resource management. That's in OSS. > > On Sat, Aug 13, 2022 at 8:16 AM rajat kumar <kumar.rajat20...@gmail.com> > wrote: > >> Hello, >> >> I have been hearing about GPU in spark3. >> >> For batch jobs , will it help to improve GPU performance. Also is GPU >> support available only on Databricks or on cloud based Spark clusters ? >> >> I am new , if anyone can share insight , it will help >> >> Thanks >> Rajat >> >
