You could have multiple notes sharing interpreter - could you have one note for setup to call streamingcontext.start and another note on a schedule just to run the select sql statement?
_____________________________ From: kant kodali <[email protected]<mailto:[email protected]>> Sent: Tuesday, April 25, 2017 8:12 AM Subject: Re: How to create a real time dashboards from spark using web socket? To: <[email protected]<mailto:[email protected]>> yeah This creates a problem when dealing with spark streaming because we can't call streamingcontext.start() multiple times since it will result in ILLEGAL State Exception. anyways still looking for ways to update this dashboard/UI/Graph through web socket? On Tue, Apr 25, 2017 at 7:03 AM, DuyHai Doan <[email protected]<mailto:[email protected]>> wrote: Yes, the scheduler applies to the entire note, not just paragraph Le 25 avr. 2017 02:39, "kant kodali" <[email protected]<mailto:[email protected]>> a écrit : Also it doesn't look like we can run a scheduler on one paragraph? we have to run the scheduler for the entire notebook always? On Mon, Apr 24, 2017 at 1:06 PM, kant kodali <[email protected]<mailto:[email protected]>> wrote: Hi All, I currently do the following val jsonDStream = getJsonDStream() jsonDStream.foreachRDD{rdd => val jsonDF = spark.read.json(rdd) jsonDF.createOrReplaceTempView("dataframe") } client.startStream() %spark.sql select * from dataframe I can see the data and everytime I click a run button I can see the updates as well however is there anyway to update this dashboard/UI/Graph through web socket? Don't want to do polling. Thanks!
