Aha. It looks a little weird, but it makes sense. Thanks.
On Friday, 13 December 2013 22:55:41 UTC+1, Matei Zaharia wrote: > > BTW the reason for this is that Spark needs to run on non-Mesos backends > too, so the mesos:// tells it to pass the rest of the URL to Mesos. > > Matei > > On Dec 13, 2013, at 2:01 AM, [email protected] <javascript:> wrote: > > Hi, > > I'm running Spark 0.8.1 on top of Mesos 0.14.1 > Mesos is configured to register against an existing zookeeper cluster. > So, the slaves are started with : "--master 'zk://<zk-ip>/mesos_1'" > > When I launch a spark streaming job over Mesos, I pass > "zk://<zk-ip>/mesos_1" to the StreamingContext constructor. > Spark replies with a warning : > > --- > > *Master does not match expected format, parsing as Mesos URL.* > *Deprecation warning: Mesos URLs not preceded by mesos:// are deprecated > and will no longer be supported in Spark 0.9.* > --- > > Although it works. > It contacts zookeeper to fetch the mesos master and contacts it. > > Also, when I look at the code in SparkContext.scala : > case _ => > if (MESOS_REGEX.findFirstIn(master).isEmpty) { > logWarning("Master %s does not match expected format, parsing as > Mesos URL".format(master)) > logWarning("Deprecation warning: Mesos URLs not preceded by > mesos:// are deprecated " + > "and will no longer be supported in Spark 0.9.") > } > MesosNativeLibrary.load() > val scheduler = new ClusterScheduler(this) > val coarseGrained = System.getProperty("spark.mesos.coarse", > "false").toBoolean > val masterWithoutProtocol = master.replaceFirst("^mesos://", "") > // Strip initial mesos:// > val backend = if (coarseGrained) { > new CoarseMesosSchedulerBackend(scheduler, this, > masterWithoutProtocol, appName) > } else { > new MesosSchedulerBackend(scheduler, this, > masterWithoutProtocol, appName) > } > scheduler.initialize(backend) > scheduler > } > > It actually assumes there is "mesos://" in there. > How should it be configured then when running Mesos on zookeeper? > > Greets, > Bart > > > > -- > You received this message because you are subscribed to the Google Groups > "Spark Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Spark Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
