Re: Spark-Streaming: output to cassandra

2014-12-05 Thread Jay Vyas
Services Pvt. Ltd. Mob #: (+91) - 9836112841. From: Gerard Maas gerard.m...@gmail.com Sent: Thursday, December 4, 2014 10:22 PM To: Akhil Das Cc: Sarosh, M.; user@spark.apache.org Subject: Re: Spark-Streaming: output to cassandra I guess he's already doing so, given the 'saveToCassandra

Re: Spark-Streaming: output to cassandra

2014-12-05 Thread Helena Edelson
You can just do You can just do something like this, the Spark Cassandra Connector handles the rest KafkaUtils.createStream[String, String, StringDecoder, StringDecoder]( ssc, kafkaParams, Map(KafkaTopicRaw - 10), StorageLevel.DISK_ONLY_2) .map { case (_, line) = line.split(,)}

Re: Spark-Streaming: output to cassandra

2014-12-05 Thread m.sarosh
@spark.apache.org Subject: Re: Spark-Streaming: output to cassandra You can just do You can just do something like this, the Spark Cassandra Connector handles the rest KafkaUtils.createStream[String, String, StringDecoder, StringDecoder]( ssc, kafkaParams, Map(KafkaTopicRaw - 10

Re: Spark-Streaming: output to cassandra

2014-12-05 Thread Helena Edelson
. Thanks and Regards, Md. Aiman Sarosh. Accenture Services Pvt. Ltd. Mob #: (+91) - 9836112841. From: Helena Edelson helena.edel...@datastax.com Sent: Friday, December 5, 2014 6:26 PM To: Sarosh, M. Cc: user@spark.apache.org Subject: Re: Spark-Streaming: output to cassandra You

Re: Spark-Streaming: output to cassandra

2014-12-05 Thread m.sarosh
@spark.apache.org Subject: Re: Spark-Streaming: output to cassandra You can just do You can just do something like this, the Spark Cassandra Connector handles the rest KafkaUtils.createStream[String, String, StringDecoder, StringDecoder]( ssc, kafkaParams, Map(KafkaTopicRaw - 10

Re: Spark-Streaming: output to cassandra

2014-12-04 Thread Akhil Das
You can use the datastax's Cassandra connector. https://github.com/datastax/spark-cassandra-connector/blob/master/doc/5_saving.md Thanks Best Regards On Thu, Dec 4, 2014 at 8:21 PM, m.sar...@accenture.com wrote: Hi, I have written the code below which is streaming data from kafka, and

Re: Spark-Streaming: output to cassandra

2014-12-04 Thread Gerard Maas
I guess he's already doing so, given the 'saveToCassandra' usage. What I don't understand is the question how do I specify a batch. That doesn't make much sense to me. Could you explain further? -kr, Gerard. On Thu, Dec 4, 2014 at 5:36 PM, Akhil Das ak...@sigmoidanalytics.com wrote: You can

Re: Spark-Streaming: output to cassandra

2014-12-04 Thread m.sarosh
Sent: Thursday, December 4, 2014 10:22 PM To: Akhil Das Cc: Sarosh, M.; user@spark.apache.org Subject: Re: Spark-Streaming: output to cassandra I guess he's already doing so, given the 'saveToCassandra' usage. What I don't understand is the question how do I specify a batch. That doesn't make much

Re: Spark-Streaming: output to cassandra

2014-12-04 Thread Akhil Das
. Mob #: (+91) - 9836112841. -- *From:* Gerard Maas gerard.m...@gmail.com *Sent:* Thursday, December 4, 2014 10:22 PM *To:* Akhil Das *Cc:* Sarosh, M.; user@spark.apache.org *Subject:* Re: Spark-Streaming: output to cassandra I guess he's already doing so, given