Re: Mesos GPU Support

2016-11-07 Thread Suresh Marru 
Hi Gourav,

I am running the master inside a virtual box and running agent natively. I 
wanted the 1.0.1 since it claims the GPU support. You EC2 installation seem to 
verify that. I am using home-brew for the agent installation which is using the 
formula - https://raw.githubusercontent.com/Homebrew/install/master/install 
 

Suresh

> On Nov 7, 2016, at 11:41 PM, Shenoy, Gourav Ganesh  
> wrote:
> 
> Hi Suresh,
>  
> I have never used Tensor Flow, so not sure if I’ll be able to help there. But 
> with respect to the GPU support for Mesos, the current installation on EC2 
> (v1.0.1) does detect GPUs, and the same is listed on their dashboard (NOTE: 
> our cluster is not GPU enabled, hence mesos dashboard shows GPU=0).
>  
> Also, how are you installing Mesos on mac? Did you install any virtualization 
> software to create VMs (for master/slave)?
>  
> Thanks and Regards,
> Gourav Shenoy
>  
> From: Suresh Marru 
> Reply-To: "dev@airavata.apache.org" 
> Date: Monday, November 7, 2016 at 11:27 PM
> To: Airavata Dev 
> Subject: Mesos GPU Support
>  
> Hi All, 
>  
> I am trying some tensor flow tutorials and seem to have gone on a wrong 
> tangent. Can any of the Mesos hackers on this list help with get me get back 
> on track?
>  
> I want run through benchmarks in [1] on mac with a GT 650M card. I got the 
> bundled example working but stuck in getting the tensor flow cnn tutorial 
> example [2] working natively on my mac. I tried to deploy the mesos first so 
> I could try to run tensor flow from Marathon as described in [3]. Have any of 
> you managed to get Mesos working on a Mac? If so, can you please point me to 
> instructions which worked for you? 
>  
> I want to have Mesos recognize and use the GPU card. From [4], it looks like 
> the latest release should have GPU support natively within mesos. Have any 
> one tried it? I am working through the tutorial [5] but I am blocked since 
> mesos on mac is not working correctly in the first place. But this tutorial 
> also seem to suggest to use spark to run tensor flow jobs. Any suggestions on 
> what is the best way to run tensor flow on gpu’s and use Mesos manage these 
> jobs? I prefer to do it on a mac, but if thats a bad idea, can try these 
> steps on EC2 instead. 
>  
> Thanks,
> Suresh
>  
> [1] - https://github.com/soumith/convnet-benchmarks 
> 
> [2] - 
> https://www.tensorflow.org/versions/r0.11/tutorials/deep_cnn/index.html#convolutional-neural-networks
>  
> 
> [3] - https://github.com/tensorflow/ecosystem/tree/master/marathon 
> 
> [4] - https://issues.apache.org/jira/browse/MESOS-4424 
>  
> [5] - 
> http://schd.ws/hosted_files/mesosconna2016/b9/GPU%20Support%20in%20Mesos.pdf 
> 
>

Re: Mesos GPU Support

2016-11-07 Thread Shenoy, Gourav Ganesh 
Hi Suresh,

I have never used Tensor Flow, so not sure if I’ll be able to help there. But 
with respect to the GPU support for Mesos, the current installation on EC2 
(v1.0.1) does detect GPUs, and the same is listed on their dashboard (NOTE: our 
cluster is not GPU enabled, hence mesos dashboard shows GPU=0).

Also, how are you installing Mesos on mac? Did you install any virtualization 
software to create VMs (for master/slave)?

Thanks and Regards,
Gourav Shenoy

From: Suresh Marru 
Reply-To: "dev@airavata.apache.org" 
Date: Monday, November 7, 2016 at 11:27 PM
To: Airavata Dev 
Subject: Mesos GPU Support

Hi All,

I am trying some tensor flow tutorials and seem to have gone on a wrong 
tangent. Can any of the Mesos hackers on this list help with get me get back on 
track?

I want run through benchmarks in [1] on mac with a GT 650M card. I got the 
bundled example working but stuck in getting the tensor flow cnn tutorial 
example [2] working natively on my mac. I tried to deploy the mesos first so I 
could try to run tensor flow from Marathon as described in [3]. Have any of you 
managed to get Mesos working on a Mac? If so, can you please point me to 
instructions which worked for you?

I want to have Mesos recognize and use the GPU card. From [4], it looks like 
the latest release should have GPU support natively within mesos. Have any one 
tried it? I am working through the tutorial [5] but I am blocked since mesos on 
mac is not working correctly in the first place. But this tutorial also seem to 
suggest to use spark to run tensor flow jobs. Any suggestions on what is the 
best way to run tensor flow on gpu’s and use Mesos manage these jobs? I prefer 
to do it on a mac, but if thats a bad idea, can try these steps on EC2 instead.

Thanks,
Suresh

[1] - https://github.com/soumith/convnet-benchmarks
[2] - 
https://www.tensorflow.org/versions/r0.11/tutorials/deep_cnn/index.html#convolutional-neural-networks
[3] - https://github.com/tensorflow/ecosystem/tree/master/marathon
[4] - https://issues.apache.org/jira/browse/MESOS-4424
[5] - 
http://schd.ws/hosted_files/mesosconna2016/b9/GPU%20Support%20in%20Mesos.pdf

Mesos GPU Support

2016-11-07 Thread Suresh Marru 
Hi All,

I am trying some tensor flow tutorials and seem to have gone on a wrong 
tangent. Can any of the Mesos hackers on this list help with get me get back on 
track?

I want run through benchmarks in [1] on mac with a GT 650M card. I got the 
bundled example working but stuck in getting the tensor flow cnn tutorial 
example [2] working natively on my mac. I tried to deploy the mesos first so I 
could try to run tensor flow from Marathon as described in [3]. Have any of you 
managed to get Mesos working on a Mac? If so, can you please point me to 
instructions which worked for you? 

I want to have Mesos recognize and use the GPU card. From [4], it looks like 
the latest release should have GPU support natively within mesos. Have any one 
tried it? I am working through the tutorial [5] but I am blocked since mesos on 
mac is not working correctly in the first place. But this tutorial also seem to 
suggest to use spark to run tensor flow jobs. Any suggestions on what is the 
best way to run tensor flow on gpu’s and use Mesos manage these jobs? I prefer 
to do it on a mac, but if thats a bad idea, can try these steps on EC2 instead. 

Thanks,
Suresh

[1] - https://github.com/soumith/convnet-benchmarks 

[2] - 
https://www.tensorflow.org/versions/r0.11/tutorials/deep_cnn/index.html#convolutional-neural-networks
 

[3] - https://github.com/tensorflow/ecosystem/tree/master/marathon 

[4] - https://issues.apache.org/jira/browse/MESOS-4424 
 
[5] - 
http://schd.ws/hosted_files/mesosconna2016/b9/GPU%20Support%20in%20Mesos.pdf

Kafka Logging Conflicts

2016-11-07 Thread Ajinkya Dhamnaskar 
Hello,

We have introduced kafka logging in airavata and for this to work properly
we need to get slf4j-log4j12-1.7.10.jar and log4j-1.2.17.jar out of the
class path.

These jars are conflicting with logback.jar which is being used for kafka
logging.

Now, the concern is which one we should keep.

I am opening this thread to discuss your concerns and suggestions on the
same.

-- 
Thanks and regards,

Ajinkya Dhamnaskar
Student ID : 0003469679
Masters (CS)
+1 (812) 369- 5416