will be added to the discussion
below:
http://apache-spark-user-list.1001560.n3.nabble.com/Re-Persist-streams-to-text-files-tp19449.html
To start a new topic under Apache Spark User List, email
ml-node+s1001560n1...@n3.nabble.com
To unsubscribe from Apache Spark User List, click here
http
Prathap (WT01 - BAS)
Cc: u...@spark.incubator.apache.org
Subject: Re: Persist streams to text files
Here's a quick version to store (append) in your local machine
val tweets = TwitterUtils.createStream(ssc, None)
val hashTags = tweets.flatMap(status = status.getText.split(
).filter
To have a single text file output for each batch you can repartition it to
1 and then call the saveAsTextFiles
stream.repartition(1).saveAsTextFiles(location)
On 21 Nov 2014 11:28, jishnu.prat...@wipro.com wrote:
Hi I am also having similar problem.. any fix suggested..
*Originally Posted
: u...@spark.incubator.apache.org
Subject: Re: Persist streams to text files
To have a single text file output for each batch you can repartition it to 1
and then call the saveAsTextFiles
stream.repartition(1).saveAsTextFiles(location)
On 21 Nov 2014 11:28,
jishnu.prat
...@spark.incubator.apache.org
*Subject:* Re: Persist streams to text files
To have a single text file output for each batch you can repartition it to
1 and then call the saveAsTextFiles
stream.repartition(1).saveAsTextFiles(location)
On 21 Nov 2014 11:28, jishnu.prat...@wipro.com wrote:
Hi I
