Hey Craig, Which version of Mesos are you running? We recently (~1 month) revamped our Hadoop port and unfortunately didn't update the wiki.
@vinodkone Sent from my mobile On Mar 1, 2013, at 7:48 AM, Craig Vanderborgh <[email protected]> wrote: > I'm getting close. I've got the namenode formatted and HDFS running. I > started hdfs with $HADOOP_HOME/bin/start-dfs.sh. Basic HDFS operations > seem to be working. Now it's time to start the jobtracker - I have been > following the instructions here: > > https://github.com/mesos/mesos/wiki/Running-Hadoop-on-Mesos > > The instructions say "Launch a JobTracker with bin/hadoop jobtracker (do > not use bin/start-mapred.sh)... > > So here's what happens: > > [craigv@sleuth-mesos-01 bin]$ hadoop jobtracker > 13/03/01 00:23:02 INFO mapred.JobTracker: STARTUP_MSG: > /************************************************************ > STARTUP_MSG: Starting JobTracker > STARTUP_MSG: host = sleuth-mesos-01.lab.webapps.rr.com/71.74.48.186 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 0.20.205.1 > STARTUP_MSG: build = -r ; compiled by 'craigv' on Thu Feb 28 22:59:28 > UTC 2013 > ************************************************************/ > 13/03/01 00:23:02 INFO impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > MetricsSystem,sub=Stats registered. > 13/03/01 00:23:02 INFO impl.MetricsSystemImpl: Scheduled snapshot period at > 10 second(s). > 13/03/01 00:23:02 INFO impl.MetricsSystemImpl: JobTracker metrics system > started > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > QueueMetrics,q=default registered. > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source ugi > registered. > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > Starting expired delegation token remover thread, > tokenRemoverScanInterval=60 min(s) > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 13/03/01 00:23:02 INFO mapred.JobTracker: Scheduler configured with > (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, > limitMaxMemForReduceTasks) (-1, -1, -1, -1) > 13/03/01 00:23:02 INFO util.HostsFileReader: Refreshing hosts > (include/exclude) list > 13/03/01 00:23:02 INFO mapred.JobTracker: Starting jobtracker with owner as > craigv > 13/03/01 00:23:02 INFO ipc.Server: Starting SocketReader > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > RpcDetailedActivityForPort54311 registered. > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > RpcActivityForPort54311 registered. > 13/03/01 00:23:02 INFO mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 13/03/01 00:23:02 INFO http.HttpServer: Added global filtersafety > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > 13/03/01 00:23:02 INFO http.HttpServer: Port returned by > webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening > the listener on 50030 > 13/03/01 00:23:02 INFO http.HttpServer: listener.getLocalPort() returned > 50030 webServer.getConnectors()[0].getLocalPort() returned 50030 > 13/03/01 00:23:02 INFO http.HttpServer: Jetty bound to port 50030 > 13/03/01 00:23:02 INFO mortbay.log: jetty-6.1.26 > 13/03/01 00:23:02 INFO mortbay.log: Started > [email protected]:50030 > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source jvm > registered. > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > JobTrackerMetrics registered. > 13/03/01 00:23:02 INFO mapred.JobTracker: JobTracker up at: 54311 > 13/03/01 00:23:02 INFO mapred.JobTracker: JobTracker webserver: 50030 > 13/03/01 00:23:02 INFO mapred.JobTracker: Cleaning up the system directory > 13/03/01 00:23:02 INFO mapred.JobTracker: History server being initialized > in embedded mode > 13/03/01 00:23:02 INFO mapred.JobHistoryServer: Started job history server > at: localhost:50030 > 13/03/01 00:23:02 INFO mapred.JobTracker: Job History Server web address: > localhost:50030 > 13/03/01 00:23:02 INFO mapred.CompletedJobStatusStore: Completed job store > is inactive > 13/03/01 00:23:03 INFO mapred.MesosScheduler: Starting MesosScheduler > java: ./common/try.hpp:77: T Try<T>::get() const [with T = > mesos::internal::MasterDetector*]: Assertion `state == SOME' failed. > Aborted > > Close but no cigar. The Mesos code in this proximity: > > src/common/try.hpp > > Try<T>& operator = (const Try<T>& that) > { > if (this != &that) { > state = that.state; > if (that.t != NULL) { > t = new T(*that.t); > } else { > t = NULL; > } > message = that.message; > } > > return *this; > } > > bool isSome() const { return state == SOME; } > bool isError() const { return state == ERROR; } > > T get() const { assert(state == SOME); return *t; } > > std::string error() const { assert(state == ERROR); return message; } > > What's wrong here? Obviously it can't start MesosScheduler, but why? > > Here is my mapred-site.xml: > > <?xml version="1.0"?> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > <!-- Put site-specific property overrides in this file. --> > > <configuration> > <property> > <name>mapred.job.tracker</name> > <value>localhost:54311</value> > </property> --> > <property> > <name>mapred.jobtracker.taskScheduler</name> > <value>org.apache.hadoop.mapred.MesosScheduler</value> > </property> > <property> > <name>mapred.mesos.master</name> > <value>mesos://sleuth-mesos-01:5050</value> > </property> > </configuration> > > THANKS! > Craig
