Is this a Spark Streaming job? Regards,
Sivakumaran S > @Sivakumaran when you say create a web socket object in your spark code I > assume you meant a spark "task" opening websocket > connection from one of the worker machines to some node.js server in that > case the websocket connection terminates after the spark > task is completed right ? and when new data comes in a new task gets created > and opens a new websocket connection again…is that how it should be > On 25-Aug-2016, at 7:08 AM, kant kodali <kanth...@gmail.com> wrote: > > @Sivakumaran when you say create a web socket object in your spark code I > assume you meant a spark "task" opening websocket connection from one of the > worker machines to some node.js server in that case the websocket connection > terminates after the spark task is completed right ? and when new data comes > in a new task gets created and opens a new websocket connection again…is that > how it should be? > > > > > > On Wed, Aug 24, 2016 10:38 PM, Sivakumaran S siva.kuma...@me.com > <mailto:siva.kuma...@me.com> wrote: > You create a websocket object in your spark code and write your data to the > socket. You create a websocket object in your dashboard code and receive the > data in realtime and update the dashboard. You can use Node.js in your > dashboard (socket.io <http://socket.io/>). I am sure there are other ways too. > > Does that help? > > Sivakumaran S > >> On 25-Aug-2016, at 6:30 AM, kant kodali <kanth...@gmail.com >> <mailto:kanth...@gmail.com>> wrote: >> >> so I would need to open a websocket connection from spark worker machine to >> where? >> >> >> >> >> >> On Wed, Aug 24, 2016 8:51 PM, Kevin Mellott kevin.r.mell...@gmail.com >> <mailto:kevin.r.mell...@gmail.com> wrote: >> In the diagram you referenced, a real-time dashboard can be created using >> WebSockets. This technology essentially allows your web page to keep an >> active line of communication between the client and server, in which case >> you can detect and display new information without requiring any user input >> of page refreshes. The link below contains additional information on this >> concept, as well as links to several different implementations (based on >> your programming language preferences). >> >> https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API >> <https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API> >> >> Hope this helps! >> - Kevin >> >> On Wed, Aug 24, 2016 at 3:52 PM, kant kodali <kanth...@gmail.com >> <mailto:kanth...@gmail.com>> wrote: >> >> ---------- Forwarded message ---------- >> From: kant kodali <kanth...@gmail.com <mailto:kanth...@gmail.com>> >> Date: Wed, Aug 24, 2016 at 1:49 PM >> Subject: quick question >> To: d...@spark.apache.org <mailto:d...@spark.apache.org>, >> us...@spark.apache.org <mailto:us...@spark.apache.org> >> >> >> <attachment-1.png> >> >> In this picture what does "Dashboards" really mean? is there a open source >> project which can allow me to push the results back to Dashboards such that >> Dashboards are always in sync with real time updates? (a push based solution >> is better than poll but i am open to whatever is possible given the above >> picture) >> >
