Re: Message getting lost in Kafka + Spark Streaming

2017-05-31 Thread Vikash Pareek
l have less final events. > > -Original Message- > From: Vikash Pareek [mailto:vikash.par...@infoobjects.com] > Sent: Tuesday, May 30, 2017 4:00 PM > To: user@spark.apache.org > Subject: Message getting lost in Kafka + Spark Streaming > > I am facing an issue related

RE: Message getting lost in Kafka + Spark Streaming

2017-05-31 Thread Sidney Feiner
vents. -Original Message- From: Vikash Pareek [mailto:vikash.par...@infoobjects.com] Sent: Tuesday, May 30, 2017 4:00 PM To: user@spark.apache.org Subject: Message getting lost in Kafka + Spark Streaming I am facing an issue related to spark streaming with kafka, my use case is as follow: 1.

Re: Message getting lost in Kafka + Spark Streaming

2017-05-30 Thread Cody Koeninger
KeyedMessage[String, > String](props.getProperty("outTopicHarmonized"), > null, row.toString()) > producer.send(keyedMessage) > > } > //hack, should be done with the flush > Thread.sleep(1000) >

Message getting lost in Kafka + Spark Streaming

2017-05-30 Thread Vikash Pareek
ucer.send(keyedMessage) } //hack, should be done with the flush Thread.sleep(1000) producer.close() } * We explicitely added sleep(1000) for testing purpose. But this is also not solving the problem :( Any suggestion would be appreciat