Is there a way to do this that preserves exactly once semantics for the
write to Kafka?
On Tue, Sep 2, 2014 at 12:30 PM, Tim Smith <secs...@gmail.com> wrote:
> I'd be interested in finding the answer too. Right now, I do:
>
> val kafkaOutMsgs = kafkInMessages.map(x=>myFunc(x._2,someParam))
> kafkaOutMsgs.foreachRDD((rdd,time) => { rdd.foreach(rec => {
> writer.output(rec) }) } ) //where writer.ouput is a method that takes a
> string and writer is an instance of a producer class.
>
>
>
>
>
> On Tue, Sep 2, 2014 at 10:12 AM, Massimiliano Tomassi <
> max.toma...@gmail.com> wrote:
>
>> Hello all,
>> after having applied several transformations to a DStream I'd like to
>> publish all the elements in all the resulting RDDs to Kafka. What the best
>> way to do that would be? Just using DStream.foreach and then RDD.foreach ?
>> Is there any other built in utility for this use case?
>>
>> Thanks a lot,
>> Max
>>
>> --
>> ------------------------------------------------
>> Massimiliano Tomassi
>> ------------------------------------------------
>> e-mail: max.toma...@gmail.com
>> ------------------------------------------------
>>
>
>
