Re: Apache Spark data locality when integrating with Kafka

I would definitely try to avoid hosting Kafka and Spark on the same servers. Kafka and Spark will be doing alot of IO between them, so you'll want to maximize on those resources and not share them on the same server. You'll want each Kafka broker to be on a dedicated server, as well as your

Re: Apache Spark data locality when integrating with Kafka

ect: Re: Apache Spark data locality when integrating with Kafka I would definitely try to avoid hosting Kafka and Spark on the same servers. Kafka and Spark will be doing alot of IO between them, so you'll want to maximize on those resources and not share them on the same server. You'll want e

Re: Apache Spark data locality when integrating with Kafka

r . > > > > Sent from Samsung Mobile. > > > Original message > From: "Yuval.Itzchakov" <yuva...@gmail.com> > Date:07/02/2016 19:38 (GMT+05:30) > To: user@spark.apache.org > Cc: > Subject: Re: Apache Spark data locality when integrati

Re: Apache Spark data locality when integrating with Kafka

الليثي <dev.fano...@gmail.com> Date:08/02/2016 02:07 (GMT+05:30) To: Diwakar Dhanuskodi <diwakar.dhanusk...@gmail.com> Cc: "Yuval.Itzchakov" <yuva...@gmail.com>, user <user@spark.apache.org> Subject: Re: Apache Spark data locality when integrating with Kafka Diw

RE: Apache Spark data locality when integrating with Kafka

Yes . To  reduce  network  latency . Sent from Samsung Mobile. Original message From: fanooos Date:07/02/2016 09:24 (GMT+05:30) To: user@spark.apache.org Cc: Subject: Apache Spark data locality when integrating with Kafka Dears If I will use

Re: Apache Spark data locality when integrating with Kafka

spark can benefit from data locality and will try to launch tasks on the node where the kafka partition resides. however i think in production many organizations run a dedicated kafka cluster. On Sat, Feb 6, 2016 at 11:27 PM, Diwakar Dhanuskodi < diwakar.dhanusk...@gmail.com> wrote: > Yes . To