I've cc'd Tim and Kevin, who worked on GPU support. On Wed, Dec 28, 2016 at 11:22 AM, Ji Yan <ji...@drive.ai> wrote:
> Dear Spark Users, > > Has anyone had successful experience running Spark on Mesos with GPU > support? We have a Mesos cluster that can see and offer nvidia GPU > resources. With Spark, it seems that the GPU support with Mesos ( > https://github.com/apache/spark/pull/14644) has only recently been merged > into Spark Master which is not found in 2.0.2 release yet. We have a custom > built Spark from 2.1-rc5 which is confirmed to have the above change. > However when we try to run any code from Spark on this Mesos setup, the > spark program hangs and keeps saying > > “WARN TaskSchedulerImpl: Initial job has not accepted any resources; > check your cluster UI to ensure that workers are registered and have > sufficient resources” > > We are pretty sure that the cluster has enough resources as there is > nothing running on it. If we disable the GPU support in configuration and > restart mesos and retry the same program, it would work. > > Any comment/advice on this greatly appreciated > > Thanks, > Ji > > > The information in this email is confidential and may be legally > privileged. It is intended solely for the addressee. Access to this email > by anyone else is unauthorized. If you are not the intended recipient, any > disclosure, copying, distribution or any action taken or omitted to be > taken in reliance on it, is prohibited and may be unlawful. > -- Michael Gummelt Software Engineer Mesosphere
