Hello guys,
I'm working on visualization based on Zeppelin that displays data coming
from Kafka. I'm wondering if it's possible to make my plots update in real
time as data keep coming in from Kafka.
For example, consider a simple program shown below:
%spark
import _root_.kafka.serializer.DefaultDecoder
import _root_.kafka.serializer.StringDecoder
import org.apache.spark.streaming.kafka.KafkaUtils
import org.apache.spark.storage.StorageLevel
import org.apache.spark.streaming._
val ssc = new StreamingContext(sc, Seconds(2))
val lines = KafkaUtils.createStream(ssc, zkQuorum, groupId,
Map("test-topic" -> 1))
val words = lines.map(x => x._1 + x._2)
words.print()
ssc.start()
When I execute the Zeppelin cell containing the above code, it would only
print out contents of words variable once and never update it again.
I have to re-execute the cell to see an update. How do I make words update
automatically so that I can use it later to generate plots that update
automatically as well?
Thank you,
--
Chaoran Yu
University of California at Berkeley | May 2014
B.S. Computer Science and Engineering
Phone: (510) 542-7749
