Hi Spark ! I found out why my RDD's werent coming through in my spark stream.
It turns out you need the onStart() needs to return , it seems - i.e. you need to launch the worker part of your start process in a thread. For example.... def onStartMock():Unit ={ val future = new Thread(new Runnable() { def run() { for(x <- 1 until 1000000000) { val newMem = Runtime.getRuntime.freeMemory()/12188091; if(newMem != lastMem){ System.out.println(" in thread : " + newMem); } lastMem=newMem; store(mockStatus); } }}); Hope that helps somebody in the same situation. FYI Its in the docs :) * {{{ * class MyReceiver(storageLevel: StorageLevel) extends NetworkReceiver[String](storageLevel) { * def onStart() { * // Setup stuff (start threads, open sockets, etc.) to start receiving data. * // Must start new thread to receive data, as onStart() must be non-blocking. * * // Call store(...) in those threads to store received data into Spark's memory. * * // Call stop(...), restart(...) or reportError(...) on any thread based on how * // different errors needs to be handled. * * // See corresponding method documentation for more details * } * * def onStop() { * // Cleanup stuff (stop threads, close sockets, etc.) to stop receiving data. * } * } * }}}