Hi
I am studing the structure of the Spark Streaming(my spark version is
0.9.0). I have a question about the SocketReceiver.In the onStart function:
-------------------------------------------------------
protected def onStart() {
logInfo("Connecting to " + host + ":" + port)
val socket = new Socket(host, port)
logInfo("Connected to " + host + ":" + port)
blockGenerator.start()
val iterator = bytesToObjects(socket.getInputStream())
while(iterator.hasNext) {
val obj = iterator.next
blockGenerator += obj
}
}
---------------------------------------------------------
Here the Socket client is created and read data iteratively. My question
is the onStart function is only called once by the super class
NetworkReceiver, and correspondingly read data one time.
When the socket server send data again, how does the SocketReceiver read
the input data,I can find any src hint about the process.
In my opinion, the Socket instance should read the data cyclically as
following:
-------------------------------------------------------------
InputStream is = socket.getInputStream()
while(theEndflag){
if(is.avariable <= 0){
val iterator = bytesToObjects(is)
while(iterator.hasNext) {
val obj = iterator.next
blockGenerator += obj
}
}
}
//theEndflag is the end flag of the loop,shoud be set to false when needed.
-----------------------------------------------------------
I know it may not be the right thought,however i am really curious about
the Socket read process because it hard to understand.
Any suggestions will be appreciated.
