CDH5.2.3 on mesos

2014-10-28 Thread Ankur Chauhan
H, I was trying to setup mesos/hadoop with the latest CDH version (MR1) and it seems like the instructions are sort of out of date and I also tried the suggestions in https://github.com/mesos/hadoop/issues/25 https://github.com/mesos/hadoop/issues/25 but after 4 hours of flailing around I am

Re: CDH5.2.3 on mesos

2014-10-28 Thread Tom Arnfeld
Hi Ankur, There aren't any getting started resources other than the documention there as far as I know. Could you share your hadoop configuration and perhaps a description of the problems you're having? Tom. On Tue, Oct 28, 2014 at 8:53 AM, Ankur Chauhan an...@malloc64.com wrote: H, I

0.21.0-pre Spark latest

2014-10-28 Thread Tim St Clair
Folks - We have some automated tests that run the latest Mesos against the latest Spark, and we've run across a series of issues in both fine, and course grained mode that I believe stem from a series of changes in the 0.21 cycle. I'm not certain if anyone owns this integration, but we

Re: Does Mesos support Hadoop MR V2

2014-10-28 Thread Brenden Matthews
Porting YARN to run atop Mesos is quite reasonable. Some folks at eBay have started some work on this (https://github.com/mesos/myriad). If you're interested, you should check it out, and contribute to the project. On Tue, Oct 28, 2014 at 5:21 AM, Yaneeve Shekel yaneeve.she...@sizmek.com wrote:

Re: 0.21.0-pre Spark latest

2014-10-28 Thread Brenden Matthews
Since we've recently adopted Spark, I'll second Tim's comment. We had an issue with 0.20.1 that was possibly related to Spark[1], so it's important for us to get this stuff fixed in 0.21.0. Tim, can you elaborate on the issues you saw? Have you tested with my recent Spark patches[2][3]? [1]:

Re: 0.21.0-pre Spark latest

2014-10-28 Thread Tim St Clair
inline - Original Message - From: Brenden Matthews brenden.matth...@airbedandbreakfast.com To: user@mesos.apache.org Cc: mesos-devel d...@mesos.apache.org, RJ Nowling rnowl...@redhat.com, Erik Erlandson e...@redhat.com Sent: Tuesday, October 28, 2014 9:51:58 AM Subject: Re:

Re: 0.21.0-pre Spark latest

2014-10-28 Thread Timothy Chen
Hi Tim, Thanks for doing the integration tests, that's something that I wanted to do but never got to yet. I have great interest ensuring spark and mesos work, and I know Brenden as well does. I have been tracking these spark mesos problems with spark jira and labeling them mesos. Can you

Re: 0.21.0-pre Spark latest

2014-10-28 Thread Timothy Chen
Hi RJ, I see, are you or the team on working on this problem already? If not I'd like to take a look as well. Tim On Tue, Oct 28, 2014 at 8:47 AM, RJ Nowling rnowl...@redhat.com wrote: Hi Tim, The integration test is simply to open the spark shell (Spark 1.1.0) using mesos 0.21 in

Re: Multiple schedulers on one machine?

2014-10-28 Thread Vinod Kone
Sorry for the delay in response. It looks like master received the registration request from the scheduler at 20:13, at which point it sent a registration ack back to the scheduler. What is not clear is why no registration requests were sent/received from 19:47 when the framework apparently

Re: Running mesos-slave in Docker

2014-10-28 Thread Alessandro Siragusa
Hi Tim, Yes you are right, I was running the last automated build from the master version, with v0.7.1 everything works fine. I am running Marathon, Mesos master and slave and zookeeper in a container in CoreOS. My plan is to create the needed unit files to run it in CoreOS and to write a

MPI on Mesos

2014-10-28 Thread Stratos Dimopoulos
; status=255 I1028 22:15:04.774554 4859 sched.cpp:747] Stopping framework '20141028-203440-1257767434-5050-3638-0006' 2014-10-28 22:15:04,795:4819(0x7fd7b1422700):ZOO_INFO@zookeeper_close@2505: Closing zookeeper sessionId=0x14959388d4e0020 And also in *executor stdout* I get: sh -c 'mpd