Re: Spark streaming action running the same work in parallel

I was able to get it working. Instead of using customers.flatMap to return alerts. I had to use the following: customers.foreachRDD(new FunctionJavaPairRDDlt;String, Iterablelt;QueueEvent, Void() { @Override public Void call(final JavaPairRDDString, Iterablelt;QueueEvent

Spark streaming action running the same work in parallel

Hi, I'm running a unit test that keeps failing to work with the code I wrote in Spark. Here is the output logs from my test that I ran that gets the customers from incoming events in the JSON called QueueEvent and I am trying to convert the incoming events for each customer to an alert. From

KafkaUtils and specifying a specific partition

Hi, How do you use KafkaUtils to specify a specific partition? I'm writing customer Marathon jobs where a customer is given 1 partition in a topic in Kafka. The job will get the partition from our database for that customer and use that to get the messages for that customer. I misinterpreted