RE: DStream Transformation to save JSON in Cassandra 2.1

2015-10-06 Thread Prateek .
238331780492) | Some(0.5235250642853548) I am not able to figure out how to map the Dstream[Coordinate] to columns in schema . Thank You Prateek -Original Message- From: Jean-Baptiste Onofré [mailto:j...@nanthrax.net] Sent: Monday, October 05, 2015 7:58 PM To: user@spark.apache.org Subject: Re

Re: DStream Transformation to save JSON in Cassandra 2.1

2015-10-05 Thread Ashish Soni
try this You can use dstream.map to conver it to JavaDstream with only the data you are interested probably return an Pojo of your JSON and then call foreachRDD and inside that call below line javaFunctions(rdd).writerBuilder("table", "keyspace", mapToRow(Class.class)).saveToCassandra(); On

Re: DStream Transformation to save JSON in Cassandra 2.1

2015-10-05 Thread Jean-Baptiste Onofré
Hi Prateek, I see two ways: - using Cassandra CQL to adapt the RDD in the DStream to Cassandra - using a Cassandra converter You have a couple of code snippet in the examples. Let me know if you need a code sample. Regards JB On 10/05/2015 04:14 PM, Prateek . wrote: Hi, I am beginner in

DStream Transformation to save JSON in Cassandra 2.1

2015-10-05 Thread Prateek .
Hi, I am beginner in Spark , this is sample data I get from Kafka stream: {"id": "9f5ccb3d5f4f421392fb98978a6b368f","coordinate":{"ax":"1.20","ay":"3.80","az":"9.90","oa":"8.03","ob":"8.8","og":"9.97"}} val lines = KafkaUtils.createStream(ssc, zkQuorum, group, topicMap).map(_._2) val jsonf