The standalone cluster manager is fine for production. Don't use Yarn or Mesos unless you already have another need for it.
On Wed, Apr 26, 2017 at 4:53 PM, anna stax <annasta...@gmail.com> wrote: > Hi Sam, > > Thank you for the reply. > > What do you mean by > I doubt people run spark in a. Single EC2 instance, certainly not in > production I don't think > > What is wrong in having a data pipeline on EC2 that reads data from kafka, > processes using spark and outputs to cassandra? Please explain. > > Thanks > -Anna > > On Wed, Apr 26, 2017 at 2:22 PM, Sam Elamin <hussam.ela...@gmail.com> wrote: >> >> Hi Anna >> >> There are a variety of options for launching spark clusters. I doubt >> people run spark in a. Single EC2 instance, certainly not in production I >> don't think >> >> I don't have enough information of what you are trying to do but if you >> are just trying to set things up from scratch then I think you can just use >> EMR which will create a cluster for you and attach a zeppelin instance as >> well >> >> >> You can also use databricks for ease of use and very little management but >> you will pay a premium for that abstraction >> >> >> Regards >> Sam >> On Wed, 26 Apr 2017 at 22:02, anna stax <annasta...@gmail.com> wrote: >>> >>> I need to setup a spark cluster for Spark streaming and scheduled batch >>> jobs and adhoc queries. >>> Please give me some suggestions. Can this be done in standalone mode. >>> >>> Right now we have a spark cluster in standalone mode on AWS EC2 running >>> spark streaming application. Can we run spark batch jobs and zeppelin on the >>> same. Do we need a better resource manager like Mesos? >>> >>> Are there any companies or individuals that can help in setting this up? >>> >>> Thank you. >>> >>> -Anna > > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
