Hey Craig. You are in luck! There is currently a patch under review ( https://reviews.apache.org/r/9174/) that adds support to hadoop-2.0.0-mr1-cdh4.1.2.
I will try to get it committed to trunk by EOW, but in the meanwhile you can download the patch from above and test it out. Let me know how it goes. @vinodkone On Fri, Mar 1, 2013 at 9:22 AM, Craig Vanderborgh < [email protected]> wrote: > Thanks for the reply, Vinod.. > > We are running 0.9.0-incubating downloaded from here: > > > http://www.apache.org/dyn/closer.cgi/incubator/mesos/mesos-0.9.0-incubating/ > > The Hadoop I'm trying to run is v0.20.205 that's included with this Mesos > distribution. But we WANT to run hadoop-2.0.0-mr1-cdh4.1.2. I've got a > patch for this, but not sure exactly what I should use as the starting > point. Can you give me a few pointers on how to proceed? We want the > hadoop-2.0.0-mr-cdh4.1.2 version, and we'll build/run anything required to > get it. > > Craig > > On Fri, Mar 1, 2013 at 9:16 AM, Vinod Kone <[email protected]> wrote: > > > Hey Craig, > > > > Which version of Mesos are you running? We recently (~1 month) revamped > > our Hadoop port and unfortunately didn't update the wiki. > > > > @vinodkone > > Sent from my mobile > > > > On Mar 1, 2013, at 7:48 AM, Craig Vanderborgh < > [email protected]> > > wrote: > > > > > I'm getting close. I've got the namenode formatted and HDFS running. > I > > > started hdfs with $HADOOP_HOME/bin/start-dfs.sh. Basic HDFS operations > > > seem to be working. Now it's time to start the jobtracker - I have > been > > > following the instructions here: > > > > > > https://github.com/mesos/mesos/wiki/Running-Hadoop-on-Mesos > > > > > > The instructions say "Launch a JobTracker with bin/hadoop jobtracker > (do > > > not use bin/start-mapred.sh)... > > > > > > So here's what happens: > > > > > > [craigv@sleuth-mesos-01 bin]$ hadoop jobtracker > > > 13/03/01 00:23:02 INFO mapred.JobTracker: STARTUP_MSG: > > > /************************************************************ > > > STARTUP_MSG: Starting JobTracker > > > STARTUP_MSG: host = sleuth-mesos-01.lab.webapps.rr.com/71.74.48.186 > > > STARTUP_MSG: args = [] > > > STARTUP_MSG: version = 0.20.205.1 > > > STARTUP_MSG: build = -r ; compiled by 'craigv' on Thu Feb 28 > 22:59:28 > > > UTC 2013 > > > ************************************************************/ > > > 13/03/01 00:23:02 INFO impl.MetricsConfig: loaded properties from > > > hadoop-metrics2.properties > > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > > MetricsSystem,sub=Stats registered. > > > 13/03/01 00:23:02 INFO impl.MetricsSystemImpl: Scheduled snapshot > period > > at > > > 10 second(s). > > > 13/03/01 00:23:02 INFO impl.MetricsSystemImpl: JobTracker metrics > system > > > started > > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > > QueueMetrics,q=default registered. > > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source ugi > > > registered. > > > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > > > Updating the current master key for generating delegation tokens > > > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > > > Starting expired delegation token remover thread, > > > tokenRemoverScanInterval=60 min(s) > > > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > > > Updating the current master key for generating delegation tokens > > > 13/03/01 00:23:02 INFO mapred.JobTracker: Scheduler configured with > > > (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, > limitMaxMemForMapTasks, > > > limitMaxMemForReduceTasks) (-1, -1, -1, -1) > > > 13/03/01 00:23:02 INFO util.HostsFileReader: Refreshing hosts > > > (include/exclude) list > > > 13/03/01 00:23:02 INFO mapred.JobTracker: Starting jobtracker with > owner > > as > > > craigv > > > 13/03/01 00:23:02 INFO ipc.Server: Starting SocketReader > > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > > RpcDetailedActivityForPort54311 registered. > > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > > RpcActivityForPort54311 registered. > > > 13/03/01 00:23:02 INFO mortbay.log: Logging to > > > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > > > org.mortbay.log.Slf4jLog > > > 13/03/01 00:23:02 INFO http.HttpServer: Added global filtersafety > > > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > > > 13/03/01 00:23:02 INFO http.HttpServer: Port returned by > > > webServer.getConnectors()[0].getLocalPort() before open() is -1. > Opening > > > the listener on 50030 > > > 13/03/01 00:23:02 INFO http.HttpServer: listener.getLocalPort() > returned > > > 50030 webServer.getConnectors()[0].getLocalPort() returned 50030 > > > 13/03/01 00:23:02 INFO http.HttpServer: Jetty bound to port 50030 > > > 13/03/01 00:23:02 INFO mortbay.log: jetty-6.1.26 > > > 13/03/01 00:23:02 INFO mortbay.log: Started > > > [email protected]:50030 > > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source jvm > > > registered. > > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > > JobTrackerMetrics registered. > > > 13/03/01 00:23:02 INFO mapred.JobTracker: JobTracker up at: 54311 > > > 13/03/01 00:23:02 INFO mapred.JobTracker: JobTracker webserver: 50030 > > > 13/03/01 00:23:02 INFO mapred.JobTracker: Cleaning up the system > > directory > > > 13/03/01 00:23:02 INFO mapred.JobTracker: History server being > > initialized > > > in embedded mode > > > 13/03/01 00:23:02 INFO mapred.JobHistoryServer: Started job history > > server > > > at: localhost:50030 > > > 13/03/01 00:23:02 INFO mapred.JobTracker: Job History Server web > address: > > > localhost:50030 > > > 13/03/01 00:23:02 INFO mapred.CompletedJobStatusStore: Completed job > > store > > > is inactive > > > 13/03/01 00:23:03 INFO mapred.MesosScheduler: Starting MesosScheduler > > > java: ./common/try.hpp:77: T Try<T>::get() const [with T = > > > mesos::internal::MasterDetector*]: Assertion `state == SOME' failed. > > > Aborted > > > > > > Close but no cigar. The Mesos code in this proximity: > > > > > > src/common/try.hpp > > > > > > Try<T>& operator = (const Try<T>& that) > > > { > > > if (this != &that) { > > > state = that.state; > > > if (that.t != NULL) { > > > t = new T(*that.t); > > > } else { > > > t = NULL; > > > } > > > message = that.message; > > > } > > > > > > return *this; > > > } > > > > > > bool isSome() const { return state == SOME; } > > > bool isError() const { return state == ERROR; } > > > > > > T get() const { assert(state == SOME); return *t; } > > > > > > std::string error() const { assert(state == ERROR); return message; } > > > > > > What's wrong here? Obviously it can't start MesosScheduler, but why? > > > > > > Here is my mapred-site.xml: > > > > > > <?xml version="1.0"?> > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > > > > > <!-- Put site-specific property overrides in this file. --> > > > > > > <configuration> > > > <property> > > > <name>mapred.job.tracker</name> > > > <value>localhost:54311</value> > > > </property> --> > > > <property> > > > <name>mapred.jobtracker.taskScheduler</name> > > > <value>org.apache.hadoop.mapred.MesosScheduler</value> > > > </property> > > > <property> > > > <name>mapred.mesos.master</name> > > > <value>mesos://sleuth-mesos-01:5050</value> > > > </property> > > > </configuration> > > > > > > THANKS! > > > Craig > > >
