Re: Write JavaDStream to Kafka (how?)

2017-02-14 Thread Cody Koeninger
It looks like you're creating a kafka producer on the driver, and attempting to write the string representation of stringIntegerJavaPairRDD. Instead, you probably want to be calling stringIntegerJavaPairRDD.foreachPartition, so that producing to kafka is being done on the executor. Read

Write JavaDStream to Kafka (how?)

2017-02-10 Thread Gutwein, Sebastian
Hi, I'am new to Spark-Streaming and want to run some end-to-end-tests with Spark and Kafka. My program is running but at the kafka topic nothing arrives. Can someone please help me? Where is my mistake, has someone a runnig example of writing a DStream to Kafka 0.10.1.0? The program looks