if you want to use true Spark Streaming (not the same as Hadoop
Streaming/Piping, as Mayur pointed out), you can use the DStream.union()
method as described in the following docs:
http://spark.apache.org/docs/0.9.1/streaming-custom-receivers.html
http://spark.apache.org/docs/0.9.1/streaming-progra
File as a stream?
I think you are confusing Spark Streaming with buffer reader. Spark
streaming is meant to process batches of data (files, packets, messages) as
they come in, infact utilizing time of packet reception as a way to create
windows etc.
In your case you are better off reading the file
Hi all,
Is it possible to read and process multiple streams with spark. I have
eeg(brain waves) csv file with 23 columns Each column is one stream(wave) and
each column has one million values.
I know one way to do it is to take transpose of the file and then give it to
spark and each mapper w
