Dear Spark Users, Has anyone had successful experience running Spark on Mesos with GPU support? We have a Mesos cluster that can see and offer nvidia GPU resources. With Spark, it seems that the GPU support with Mesos (https://github.com/apache/spark/pull/14644 <https://github.com/apache/spark/pull/14644>) has only recently been merged into Spark Master which is not found in 2.0.2 release yet. We have a custom built Spark from 2.1-rc5 which is confirmed to have the above change. However when we try to run any code from Spark on this Mesos setup, the spark program hangs and keeps saying
“WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources” We are pretty sure that the cluster has enough resources as there is nothing running on it. If we disable the GPU support in configuration and restart mesos and retry the same program, it would work. Any comment/advice on this greatly appreciated Thanks, Ji -- The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
