You can use an attached GPU VM for DNN training, and do other processing on regular CPU nodes. You can even deallocate the GPU VM to save costs when not using it. The GPU branch has instructions how to set up such compute environment:
https://github.com/Azure/mmlspark/tree/gpu#gpu-vm-setup Cheers, Roope – Microsoft Cloud AI Team From: hosur narahari [mailto:hnr1...@gmail.com] Sent: Thursday, July 6, 2017 1:54 AM To: Gaurav1809 <gauravhpan...@gmail.com> Cc: user <user@spark.apache.org> Subject: Re: Do we anything for Deep Learning in Spark? Hi Roope, Does this mmlspark project uses GPGPU for processing and just CPU cores since DL models are computationally very intensive. Best Regards, Hari On 6 Jul 2017 9:33 a.m., "Gaurav1809" <gauravhpan...@gmail.com<mailto:gauravhpan...@gmail.com>> wrote: Thanks Roope for the inputs. On Wed, Jul 5, 2017 at 11:41 PM, Roope [via Apache Spark User List] <[hidden email]<http://user/SendEmail.jtp?type=node&node=28826&i=0>> wrote: Microsoft Machine Learning Library for Apache Spark lets you run CNTK deep learning models on Spark. https://github.com/Azure/mmlspark<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fmmlspark&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=1FqnrRsLY6JrPb%2BdkG4AlLBN%2F8f2O6w3Ehi9DYvu6lc%3D&reserved=0> The library APIs are focused on image processing scenarios, and are compatible with SparkML Pipelines. Cheers, Roope - Microsoft Cloud AI Team ________________________________ If you reply to this email, your message will be added to the discussion below: http://apache-spark-user-list.1001560.n3.nabble.com/Do-we-anything-for-Deep-Learning-in-Spark-tp28772p28824.html<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2FDo-we-anything-for-Deep-Learning-in-Spark-tp28772p28824.html&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=zFajYYH16xwzDRwgP1se9m%2BAA5fIufUS7FFxHjzTOl0%3D&reserved=0> To start a new topic under Apache Spark User List, email [hidden email]<http://user/SendEmail.jtp?type=node&node=28826&i=1> To unsubscribe from Do we anything for Deep Learning in Spark?, click here. NAML<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2Ftemplate%2FNamlServlet.jtp%3Fmacro%3Dmacro_viewer%26id%3Dinstant_html%2521nabble%253Aemail.naml%26base%3Dnabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace%26breadcrumbs%3Dnotify_subscribers%2521nabble%253Aemail.naml-instant_emails%2521nabble%253Aemail.naml-send_instant_email%2521nabble%253Aemail.naml&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=wzDT2AoJRfwqxlumkdoRlGq6b0PaeoypZtvvLgFlR1M%3D&reserved=0> ________________________________ View this message in context: Re: Do we anything for Deep Learning in Spark?<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2FDo-we-anything-for-Deep-Learning-in-Spark-tp28772p28826.html&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=dDprNB1TSm9WkAY1iuN1NjkG7vzx6JaiXB42DcXqUt4%3D&reserved=0> Sent from the Apache Spark User List mailing list archive<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2F&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=bI5METH7hpg6xCjUQF0%2BiIKY7qFy709YGC4g0n9MjW0%3D&reserved=0> at Nabble.com.
