Re: Kafka 0.10 & Spark Streaming 2.0.2

> > From: Mich Talebzadeh <mich.talebza...@gmail.com> > Date: Friday, December 2, 2016 at 12:26 PM > To: Gabriel Perez <gabr...@adtheorent.com> > Cc: Jacek Laskowski <ja...@japila.pl>, user <user@spark.apache.org> > > > Subject: Re: Kafka 0.10 & Spark

Re: Kafka 0.10 & Spark Streaming 2.0.2

:26 PM To: Gabriel Perez <gabr...@adtheorent.com> Cc: Jacek Laskowski <ja...@japila.pl>, user <user@spark.apache.org> Subject: Re: Kafka 0.10 & Spark Streaming 2.0.2 in this POC of yours are you running this app with spark in Local mode by any chance? Dr Mich Ta

Re: Kafka 0.10 & Spark Streaming 2.0.2

rez <gabr...@adtheorent.com> > *Cc: *user <user@spark.apache.org> > *Subject: *Re: Kafka 0.10 & Spark Streaming 2.0.2 > > > > Hi, > > > > How many partitions does the topic have? How do you check how many > executors read from the topic? > > &

Re: Kafka 0.10 & Spark Streaming 2.0.2

Laskowski <ja...@japila.pl> Date: Friday, December 2, 2016 at 12:21 PM To: Gabriel Perez <gabr...@adtheorent.com> Cc: user <user@spark.apache.org> Subject: Re: Kafka 0.10 & Spark Streaming 2.0.2 Hi, Can you post the screenshot of the Executors and Streaming tabs? Jacek

Re: Kafka 0.10 & Spark Streaming 2.0.2

> *To: *Gabriel Perez <gabr...@adtheorent.com> > *Cc: *user <user@spark.apache.org> > *Subject: *Re: Kafka 0.10 & Spark Streaming 2.0.2 > > > > Hi, > > > > How many partitions does the topic have? How do you check how many > executors read from

Re: Kafka 0.10 & Spark Streaming 2.0.2

Friday, December 2, 2016 at 11:47 AM To: Gabriel Perez <gabr...@adtheorent.com> Cc: user <user@spark.apache.org> Subject: Re: Kafka 0.10 & Spark Streaming 2.0.2 Hi, How many partitions does the topic have? How do you check how many executors read from the topic? Jacek

Re: Kafka 0.10 & Spark Streaming 2.0.2

Hi, How many partitions does the topic have? How do you check how many executors read from the topic? Jacek On 2 Dec 2016 2:44 p.m., "gabrielperez2484" wrote: Hello, I am trying to perform a POC between Kafka 0.10 and Spark 2.0.2. Currently I am running into an

Kafka 0.10 & Spark Streaming 2.0.2

Hello, I am trying to perform a POC between Kafka 0.10 and Spark 2.0.2. Currently I am running into an issue, where only one executor ("kafka consumer") is reading from the topic. Which is causing performance to be really poor. I have tried adding "--num-executors 8" both in the script to execute