Here's two ways of doing that:
Without the filter function :
JavaPairDStreamString, String foo =
ssc.String, String, SequenceFileInputFormatfileStream(/tmp/foo);
With the filter function:
JavaPairInputDStreamLongWritable, Text foo = ssc.fileStream(/tmp/foo,
LongWritable.class,
Hi I am trying to find correct way to use Spark Streaming API
streamingContext.fileStream(String,ClassK,ClassV,ClassF)
I tried to find example but could not find it anywhere in either Spark
documentation. I have to stream files in hdfs which is of custom hadoop
format.
