How to insert df in HBASE

2016-01-06 Thread Sadaf
HI, I need to insert a Dataframe in to hbase using scala code. Can anyone guide me how to achieve this? Any help would be much appreciated. Thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-insert-df-in-HBASE-tp25891.html Sent from the

Streaming of COAP Resources

2015-10-16 Thread Sadaf
I am currently working on IOT Coap protocol.I accessed server on local host through copper firefox plugin. Then i Added resouce having "GET" functionality in server. After that i made its client as a streaming source. Here is the code of client streaming class customReceiver(test:String) extends

Twitter Streming using Twitter Public Streaming API and Apache Spark

2015-09-14 Thread Sadaf
Hi, I wanna fetch PUBLIC tweets (not particular to any account) containing any particular HASHTAG (#) (i.e "CocaCola" in my case) from twitter. I made an APP on twitter to get the credentials, and then used Twitter Public Streaming API. Below is the piece of code. { val config = new

Re: ERROR ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver

2015-08-11 Thread Sadaf Khan
sends back the message that it has been stopped. This is being (probably incorrectly) logged with ERROR level. On Sun, Aug 9, 2015 at 12:52 AM, Sadaf sa...@platalytics.com wrote: Hi When i tried to stop spark streaming using ssc.stop(false,true) It gives the following error. ERROR

ERROR ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver

2015-08-09 Thread Sadaf
Hi When i tried to stop spark streaming using ssc.stop(false,true) It gives the following error. ERROR ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver 15/08/07 13:41:11 WARN ReceiverSupervisorImpl: Stopped executor without error 15/08/07 13:41:20 WARN WriteAheadLogManager :

how to stop twitter-spark streaming

2015-08-06 Thread Sadaf
Hi All, i am working with spark streaming and twitter's user api. i used this code to stop streaming ssc.addStreamingListener(new StreamingListener{ var count=1 override def onBatchCompleted(batchCompleted: StreamingListenerBatchCompleted) { count += 1

Spark Streaming - CheckPointing issue

2015-08-05 Thread Sadaf
Hi i've done the twitter streaming using twitter's streaming user api and spark streaming. this runs successfully on my local machine. but when i run this program on cluster in local mode. it just run successfully for the very first time. later on it gives the following exception. Exception in

Twitter live Streaming

2015-08-04 Thread Sadaf
Hi Is there any way to get all old tweets since when the account was created using spark streaming and twitters api? Currently my connector is showing those tweets that get posted after the program runs. I've done this task using spark streaming and a custom receiver using twitter user api.

Twitter Connector-Spark Streaming

2015-07-30 Thread Sadaf
Hi. I am writing twitter connector using spark streaming. but it fetched the random tweets. Is there any way to receive the tweets of a particular account? I made an app on twitter and used the credentials as given below. def managingCredentials(): Option[twitter4j.auth.Authorization]= {

Error in starting Spark Streaming Context

2015-07-29 Thread Sadaf
Hi I am new to Spark Streaming and writing a code for twitter connector. when i run this code more than one time, it gives the following exception. I have to create a new hdfs directory for checkpointing each time to make it run successfully and moreover it doesn't get stopped. ERROR

Re: error in twitter streaming

2015-07-29 Thread Sadaf
thanks for the suggestion Akashsihag. i've tried this solution and unfortunately it is also giving the same error. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/error-in-twitter-streaming-tp24030p24056.html Sent from the Apache Spark User List mailing

Spark Streaming

2015-07-29 Thread Sadaf
Hi, I am new to Spark Streaming and writing a code for twitter connector. I am facing the following exception. ERROR StreamingContext: Error starting the context, marking it as stopped org.apache.spark.SparkException: org.apache.spark.streaming.dstream.WindowedDStream@532d0784 has not been

Checkpoint issue in spark streaming

2015-07-28 Thread Sadaf
Hi all. I am writing a twitter connector using spark streaming. i have written the following code to maintain checkpoint. val ssc=StreamingContext.getOrCreate(hdfs://192.168.23.109:9000/home/cloud9/twitterCheckpoint,()= { managingContext() }) def managingContext():StreamingContext = {