Re: Error when Spark streaming consumes from Kafka

2015-02-02 Thread Dibyendu Bhattacharya
>>> http://mail-archives.apache.org/mod_mbox/kafka-users/201312.mbox/%3CCAFbh0Q38qQ0aAg_cj=jzk-kbi8xwf+1m6xlj+fzf6eetj9z...@mail.gmail.com%3E >>> >>> On Mon, Feb 2, 2015 at 1:07 PM, Greg Temchenko >>> wrote: >>> >>>> Hi, >>>> >>

Re: Error when Spark streaming consumes from Kafka

2015-02-02 Thread Neelesh
>>> I filed an issue in jira: >>> https://issues.apache.org/jira/browse/SPARK-5505 >>> >>> Greg >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Error-whe

Re: Error when Spark streaming consumes from Kafka

2015-02-02 Thread Dibyendu Bhattacharya
2015 at 1:07 PM, Greg Temchenko wrote: > >> Hi, >> >> This seems not fixed yet. >> I filed an issue in jira: >> https://issues.apache.org/jira/browse/SPARK-5505 >> >> Greg >> >> >> >> -- >> View this message in context: >&

Re: Error when Spark streaming consumes from Kafka

2015-02-02 Thread Tathagata Das
at 1:07 PM, Greg Temchenko wrote: > Hi, > > This seems not fixed yet. > I filed an issue in jira: https://issues.apache.org/jira/browse/SPARK-5505 > > Greg > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Error-when

Re: Error when Spark streaming consumes from Kafka

2015-02-02 Thread Greg Temchenko
Hi, This seems not fixed yet. I filed an issue in jira: https://issues.apache.org/jira/browse/SPARK-5505 Greg -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Error-when-Spark-streaming-consumes-from-Kafka-tp19570p21471.html Sent from the Apache Spark User

Re: Error when Spark streaming consumes from Kafka

2014-11-23 Thread Bill Jay
Hi Dibyendu, Thank you for answer. I will try the Spark-Kafka consumer. Bill On Sat, Nov 22, 2014 at 9:15 PM, Dibyendu Bhattacharya < dibyendu.bhattach...@gmail.com> wrote: > I believe this is something to do with how Kafka High Level API manages > consumers within a Consumer group and how it r

Re: Error when Spark streaming consumes from Kafka

2014-11-22 Thread Dibyendu Bhattacharya
I believe this is something to do with how Kafka High Level API manages consumers within a Consumer group and how it re-balance during failure. You can find some mention in this Kafka wiki. https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Client+Re-Design Due to various issues in Kafka

Error when Spark streaming consumes from Kafka

2014-11-22 Thread Bill Jay
Hi all, I am using Spark to consume from Kafka. However, after the job has run for several hours, I saw the following failure of an executor: kafka.common.ConsumerRebalanceFailedException: group-1416624735998_ip-172-31-5-242.ec2.internal-1416648124230-547d2c31 can't rebalance after 4 retries