Re: Spark twitter streaming in Java

[Yogs]

Hi Soni,

I think you need to start the JavaStreamingContext. Add something like this
at the end of your program :

jssc.start();
jssc.awaitTermination(6);
jssc.stop();

- Yogesh

On Thu, Nov 19, 2015 at 12:34 PM, Soni spark 
wrote:

> Dear Friends,
>
> I am struggling with spark twitter streaming. I am not getting any data.
> Please correct below code if you found any mistakes.
>
> import org.apache.spark.*;
> import org.apache.spark.api.java.
> function.*;
> import org.apache.spark.streaming.*;
> import org.apache.spark.streaming.api.java.*;
> import org.apache.spark.streaming.twitter.*;
> import twitter4j.GeoLocation;
> import twitter4j.Status;
> import java.util.Arrays;
> import scala.Tuple2;
>
> public class SparkTwitterStreaming {
>
> public static void main(String[] args) {
>
> final String consumerKey = "XXX";
> final String consumerSecret = "XX";
> final String accessToken = "XX";
> final String accessTokenSecret = "XXX";
> SparkConf conf = new
> SparkConf().setMaster("local[2]").setAppName("SparkTwitterStreaming");
> JavaStreamingContext jssc = new JavaStreamingContext(conf, new
> Duration(6));
> System.setProperty("twitter4j.oauth.consumerKey", consumerKey);
> System.setProperty("twitter4j.oauth.consumerSecret", consumerSecret);
> System.setProperty("twitter4j.oauth.accessToken", accessToken);
> System.setProperty("twitter4j.oauth.accessTokenSecret",
> accessTokenSecret);
> String[] filters = new String[] {"Narendra Modi"};
> JavaReceiverInputDStream twitterStream =
> TwitterUtils.createStream(jssc,filters);
>
> // Without filter: Output text of all tweets
> JavaDStream statuses = twitterStream.map(
> new Function() {
> public String call(Status status) { return
> status.getText(); }
> }
> );
> statuses.print();
> statuses.dstream().saveAsTextFiles("/home/apache/tweets", "txt");
>
>   }
>
> }
>
>

Spark twitter streaming in Java

[Soni spark]

Dear Friends,

I am struggling with spark twitter streaming. I am not getting any data.
Please correct below code if you found any mistakes.

import org.apache.spark.*;
import org.apache.spark.api.java.
function.*;
import org.apache.spark.streaming.*;
import org.apache.spark.streaming.api.java.*;
import org.apache.spark.streaming.twitter.*;
import twitter4j.GeoLocation;
import twitter4j.Status;
import java.util.Arrays;
import scala.Tuple2;

public class SparkTwitterStreaming {

public static void main(String[] args) {

final String consumerKey = "XXX";
final String consumerSecret = "XX";
final String accessToken = "XX";
final String accessTokenSecret = "XXX";
SparkConf conf = new
SparkConf().setMaster("local[2]").setAppName("SparkTwitterStreaming");
JavaStreamingContext jssc = new JavaStreamingContext(conf, new
Duration(6));
System.setProperty("twitter4j.oauth.consumerKey", consumerKey);
System.setProperty("twitter4j.oauth.consumerSecret", consumerSecret);
System.setProperty("twitter4j.oauth.accessToken", accessToken);
System.setProperty("twitter4j.oauth.accessTokenSecret",
accessTokenSecret);
String[] filters = new String[] {"Narendra Modi"};
JavaReceiverInputDStream twitterStream =
TwitterUtils.createStream(jssc,filters);

// Without filter: Output text of all tweets
JavaDStream statuses = twitterStream.map(
new Function() {
public String call(Status status) { return
status.getText(); }
}
);
statuses.print();
statuses.dstream().saveAsTextFiles("/home/apache/tweets", "txt");

  }

}