Hi,
I'm pulling some twitter data and trying to save the data into
persistent table.This is the code written.
case class Tweet(createdAt:Long, text:String)
twt.map(status=>
Tweet(status.getCreatedAt().getTime()/1000, status.getText())
).foreachRDD(rdd=>
rdd.toDF().saveAsTable("stream",Sa
Have you looked at the examples ?
examples/src/main/java/org/apache/spark/examples/streaming/JavaTwitterHashTagJoinSentiments.java
examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala
examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdHLL.sca
Hi,
Im new to Spark. I need to pull Twitter data based on some keyword
using scala and store them in spark sql table. Can anyone help me out to
work on this?
Im using scala 1.6.0
Thanks and regrads,
siva
Siva:
How did you build your project ?
You can use mvn denpendency:tree to find out the version
of spark-streaming-twitter
or take a look at ./external/twitter/pom.xml
ls ~/.m2/repository/org/apache/spark/spark-streaming-twitter_2.10/
1.4.0-SNAPSHOT/ 1.5.0-SNAPSHOT/ 1.5.2
Did you try using Twitter4J? http://twitter4j.org/en/index.html
On Fri, Feb 5, 2016 at 5:20 AM, siva kumar wrote:
> Hi ,
> Im trying to pull data from twitter and do some processing out of
> it. I found this code
>
> import org.apache.spark.streaming._import
> org.apache.spark.streami
Hi ,
Im trying to pull data from twitter and do some processing out of
it. I found this code
import org.apache.spark.streaming._import
org.apache.spark.streaming.twitter._import
org.apache.spark.storage.StorageLevelimport scala.io.Sourceimport
scala.collection.mutable.HashMapimport java.