I use twitter data with spark streaming to experiment with twitter data.
Basic stuff
val ssc = new StreamingContext(sparkConf, Seconds(2))
val tweets = TwitterUtils.createStream(ssc, None)
val statuses = tweets.map(status => status.getText())
statuses.print()
Another alternative is to use Apache flume to get the twitter data and
store it as log files in hdfs.
[image: Inline images 1]
I notice that these log files are stored as binary log files.
I presume the log files can be read and converted to json through another
process or used with machine learning language.
I know this question may not be directly relevant but what are the main
approaches, one real time analysis of twitter using spark streaming and the
other store data in hdfs and use later.?
Thanks
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
