Blatant product plug: The easiest way to run hdfs-mesos on Mesos (using Marathon) is to launch a DCOS (EE) cluster. https://mesosphere.com/product/ You might also want to look at the custom DCOS config in https://github.com/mesosphere/hdfs/tree/master/example-conf/mesosphere-dcos
For basic instructions on running an app in Marathon, see https://mesosphere.github.io/marathon/docs/application-basics.html See the marathon.json in DCOS Universe: https://github.com/mesosphere/universe/blob/version-1.x/repo/packages/H/hdfs/0/marathon.json Just replace any of the {{moustache}} variables with values like the defaults in https://github.com/mesosphere/universe/blob/version-1.x/repo/packages/H/hdfs/0/config.json You can also simplify the command to "cd hdfs-mesos* && ./bin/hdfs-mesos" On Mon, Jul 6, 2015 at 2:19 PM, Kk Bk <kkbr...@gmail.com> wrote: > Adam > > I would like to choose option 2. Cab you provide pointers as to how to run > hdfs-mesos using marathon ? > > -Bhargav > > On Sun, Jul 5, 2015 at 10:53 PM, Adam Bordelon <a...@mesosphere.io> wrote: > >> Kk, >> >> There are two options for running the hdfs framework on Mesos. >> - If you already have the hadoop/hdfs binaries on all your nodes, you can >> follow the instructions in >> https://github.com/mesosphere/hdfs#if-you-have-hadoop-pre-installed-in-your-cluster >> to tell the scheduler to use the preinstalled NN/DN binaries. >> - Otherwise, you can run the hdfs framework scheduler `bin/hdfs-mesos` >> on any node that can reach the Mesos master and slaves, and it can serve >> out the binaries itself. Note that this node may not necessarily be the >> same node on which either of the namenodes end up running. Some choose to >> run the hdfs-mesos scheduler on a Mesos master node, but you can achieve >> framework scheduler HA if you run it via another framework like Marathon >> that can restart the scheduler (elsewhere) if it or its node dies. See >> example (templatized) Marathon json in >> https://github.com/mesosphere/universe/tree/version-1.x/repo/packages/H/hdfs/0 >> >> On Fri, Jul 3, 2015 at 11:31 AM, Kk Bk <kkbr...@gmail.com> wrote: >> >>> Thanks guys for the response. >>> >>> 1) I use trusty. Seems like CDH4 does not have support for Trusty. >>> >>> 2) Followed instructions as per link https://github.com/mesosphere/hdfs. >>> Able to build "hdfs-mesos-*.tgz" >>> >>> Should i copy this file to all nodes (i have multi-node mesos cluster) >>> or just the master node of mesos where i plan to keep the namenode for >>> hadoop >>> >>> >>> >>> >>> >>> On Fri, Jul 3, 2015 at 8:34 AM, Tom Arnfeld <t...@duedil.com> wrote: >>> >>>> It might be worth taking a look at the install documentation on the >>>> Hadoop on Mesos product here; https://github.com/mesos/hadoop >>>> >>>> For our installations I don't think we really do much more than >>>> installing the apt packages you mentioned and then installing the >>>> hadoop-mesos jars.. plus adding the appropriate configuration. >>>> >>>> On Friday, Jul 3, 2015 at 3:52 pm, Kk Bk <kkbr...@gmail.com>, wrote: >>>> >>>>> I am trying to install Hadoop on Mesos on ubuntu servers, So followed >>>>> instruction as per link >>>>> https://open.mesosphere.com/tutorials/run-hadoop-on-mesos/#step-2. >>>>> >>>>> Step-2 of link says to install HDFS using as per link >>>>> http://www.cloudera.com/content/cloudera/en/documentation/cdh4/latest/CDH4-Installation-Guide/cdh4ig_topic_4_4.html >>>>> . >>>>> >>>>> Question: Is it sufficient to run following commands >>>>> >>>>> 1) On Namenode: sudo apt-get install hadoop-hdfs-namenode >>>>> 2) On Datanode: sudo apt-get install hadoop-0.20-mapreduce-tasktracker >>>>> hadoop-hdfs-datanode >>>>> >>>>> Or just follow the instructions on the mesosphere link that installs >>>>> HDFS ? >>>>> >>>>> >>>>> >>>>> >>>>> >>> >> >
