You can certainly start jobs without Chronos, but to automatically restart finished jobs or to run jobs at specific times or periods, you'll want something like Chronos.
dean Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com On Fri, Jul 24, 2015 at 5:08 PM, boci <boci.b...@gmail.com> wrote: > Thanks, > Mesos will show spark is driver is running, but what happened if my batch > job finished? How can I reschedule without chronos ? Can I submit a job > without start it? > > Thanks > > b0c1 > > > ---------------------------------------------------------------------------------------------------------------------------------- > Skype: boci13, Hangout: boci.b...@gmail.com > > On Fri, Jul 24, 2015 at 11:52 PM, Dean Wampler <deanwamp...@gmail.com> > wrote: > >> When running Spark in Mesos cluster mode, the driver program runs in one >> of the cluster nodes, like the other Spark processes that are spawned. You >> won't need a special node for this purpose. I'm not very familiar with >> Chronos, but its UI or the regular Mesos UI should show you where the >> driver is running, then you can use the Spark web UI on that machine to see >> what the Spark job is doing. >> >> dean >> >> Dean Wampler, Ph.D. >> Author: Programming Scala, 2nd Edition >> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) >> Typesafe <http://typesafe.com> >> @deanwampler <http://twitter.com/deanwampler> >> http://polyglotprogramming.com >> >> On Fri, Jul 24, 2015 at 4:47 PM, boci <boci.b...@gmail.com> wrote: >> >>> Thanks, but something is not clear... >>> I have the mesos cluster. >>> - I want to submit my application and scheduled with chronos. >>> - For cluster mode I need a dispatcher, this is another container >>> (machine in the real world)? What will this do? It's needed when I using >>> chronos? >>> - How can I access to my spark job from chronos? >>> >>> I think submit in client mode is not fit to my condition, that's right? >>> >>> Thanks >>> b0c1 >>> >>> >>> ---------------------------------------------------------------------------------------------------------------------------------- >>> Skype: boci13, Hangout: boci.b...@gmail.com >>> >>> On Wed, Jul 22, 2015 at 4:51 PM, Dean Wampler <deanwamp...@gmail.com> >>> wrote: >>> >>>> This page, http://spark.apache.org/docs/latest/running-on-mesos.html, >>>> covers many of these questions. If you submit a job with the option >>>> "--supervise", it will be restarted if it fails. >>>> >>>> You can use Chronos for scheduling. You can create a single streaming >>>> job with a 10 minute batch interval, if that works for your every 10-min. >>>> need. >>>> >>>> Dean Wampler, Ph.D. >>>> Author: Programming Scala, 2nd Edition >>>> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) >>>> Typesafe <http://typesafe.com> >>>> @deanwampler <http://twitter.com/deanwampler> >>>> http://polyglotprogramming.com >>>> >>>> On Wed, Jul 22, 2015 at 3:53 AM, boci <boci.b...@gmail.com> wrote: >>>> >>>>> Hi guys! >>>>> >>>>> I'm a new in mesos. I have two spark application (one streaming and >>>>> one batch). I want to run both app in mesos cluster. Now for testing I >>>>> want >>>>> to run in docker container so I started a simple redjack/mesos-master, but >>>>> I think a lot of think unclear for me (both mesos and spark-mesos). >>>>> >>>>> If I have a mesos cluster (for testing it will be some docker >>>>> container) i need a separate machine (container) to run my spark job? Or >>>>> can I submit the cluster and schedule (chronos or I dunno)? >>>>> How can I run the streaming job? What happened if the "controller" >>>>> died? Or if I call spark-submit with master=mesos my application started >>>>> and I can forget? How can I run in every 10 min without submit in every 10 >>>>> min? How can I run my streaming app in HA mode? >>>>> >>>>> Thanks >>>>> >>>>> b0c1 >>>>> >>>>> >>>>> ---------------------------------------------------------------------------------------------------------------------------------- >>>>> Skype: boci13, Hangout: boci.b...@gmail.com >>>>> >>>> >>>> >>> >> >
