Hi Sadaf,
Im currently struggling with Twitter Streaming as well. I cant get it working
using the simple setup bellow. I use spark 1.2 and I replaced twitter4j v3 with
v4. Am I doing something wrong? How are you doing this?
twitter4j.conf.Configuration conf = new twitter4j.conf.ConfigurationBuilder()
.setOAuthConsumerKey("")
.setOAuthConsumerSecret("*")
.setOAuthAccessToken("*")
.setOAuthAccessTokenSecret("**").build();
TwitterFactory tf =new TwitterFactory(conf);
Authorization a = new OAuthAuthorization(conf);
Authorization a2 = tf.getInstance(a).getAuthorization();
SparkConf sparkConf = new
SparkConf().setAppName("TwitterStreamJob");
JavaStreamingContext jssc = new
JavaStreamingContext(sparkConf,Durations.seconds(30));
String[] filter = new String[]{"football"};
JavaReceiverInputDStream receiverStream =
TwitterUtils.createStream(jssc, a2, filter);
JavaDStream tweets= receiverStream.map(new Function() {
@Override
public String call(Status tweet) throws Exception {
return tweet.getUser().getName() +"_" +
tweet.getText() +"_" + tweet.getCreatedAt().getTime();
}
});
tweets.foreachRDD(new Function, Void>() {
@Override
public Void call(JavaRDD arg0) throws Exception
{
arg0.saveAsTextFile("hdfs://myhost/results/twitter_" +
UUID.randomUUID().toString());
return null;
}
});
jssc.start();
jssc.awaitTermination();
-Ursprüngliche Nachricht-
Von: Sadaf [mailto:sa...@platalytics.com]
Gesendet: Dienstag, 4. August 2015 10:29
An: user@spark.apache.org
Betreff: Twitter live Streaming
Hi
Is there any way to get all old tweets since when the account was created using
spark streaming and twitters api? Currently my connector is showing those
tweets that get posted after the program runs. I've done this task using spark
streaming and a custom receiver using "twitter user api".
Thanks in anticipation.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Twitter-live-Streaming-tp24124.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional
commands, e-mail: user-h...@spark.apache.org
[http://www.ti8m.ch/fileadmin/daten/ti8m/Bilder/footer/Footer_Paymit_klein.jpg]<https://www.ti8m.ch/competences/garage.html>
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
