So if you want to build your own from the ground up, then yes you could go the d3js route. Like Feynman also responded you could use something like Spark Notebook or Zeppelin to create some charts as well. It really depends on your intended audience and ultimate goal. If you just want some counters and graphs without any interactivity it shouldn't be too difficult.
Another option, if you’re willing to use a hosted service, would be something like MS Power BI. I’ve used this to publish data and have realtime dashboards and reports fed by Spark. From: Shashi Vishwakarma Date: Friday, September 11, 2015 at 11:56 AM To: "user@spark.apache.org<mailto:user@spark.apache.org>" Subject: Realtime Data Visualization Tool for Spark Hi I have got streaming data which needs to be processed and send for visualization. I am planning to use spark streaming for this but little bit confused in choosing visualization tool. I read somewhere that D3.js can be used but i wanted know which is best tool for visualization while dealing with streaming application.(something that can be easily integrated) If someone has any link which can tell about D3.js(or any other visualization tool) and Spark streaming application integration then please share . That would be great help. Thanks and Regards Shashi