Re: Write to cassandra...each individual statement

All you'd need to do is *transform* the rdd before writing it, e.g. using the .map function. On Thu, Aug 13, 2015 at 11:30 AM, Priya Ch learnings.chitt...@gmail.com wrote: Hi All, I have a question in writing rdd to cassandra. Instead of writing entire rdd to cassandra, i want to write

Write to cassandra...each individual statement

Hi All, I have a question in writing rdd to cassandra. Instead of writing entire rdd to cassandra, i want to write individual statement into cassandra beacuse there is a need to perform to ETL on each message ( which requires checking with the DB). How could i insert statements individually?

Re: Write to cassandra...each individual statement

Hi Philip, I have the following requirement - I read the streams of data from various partitions of kafka topic. And then I union the dstreams and apply hash partitioner so messages of same key would go into single partition of an rdd, which is ofcourse handled by a single thread. This way we

Re: Write to cassandra...each individual statement

So you need some state between messages in a partition. You can use mapPartitions or foreachPartition, which allow you to write code to process an entire partition. On Thu, Aug 13, 2015 at 11:48 AM, Priya Ch learnings.chitt...@gmail.com wrote: Hi Philip, I have the following requirement - I