I'm a newbie with Spark,,, a simple question
val errorLines = lines.filter(_.contains(h))
val mapErrorLines = errorLines.map(line = (key, line))
val grouping = errorLinesValue.groupByKeyAndWindow(Seconds(8), Seconds(4))
I get something like:
604: ---
605:
You can create a DStream that contains the count, transforming the grouped
windowed RDD, like this:
val errorCount = grouping.map{case (k,v) = v.size }
If you need to preserve the key:
val errorCount = grouping.map{case (k,v) = (k,v.size) }
or you if you don't care about the content of the
What I would like to do it's to count the number of elements and if
it's greater than a number, I have to iterate all them and store them
in mysql or another system. So, I need to count them and preserve the
values because saving in other system.
I know about this map(line = (key, line)), it was
Basically what I want to do it'd be something like..
val errorLines = lines.filter(_.contains(h))
val mapErrorLines = errorLines.map(line = (key, line))
val grouping = errorLinesValue.groupByKeyAndWindow(Seconds(8), Seconds(4))
if (errorLinesValue.getValue().size() X){
//iterate values and
