Re: Does Zeppelin 0.7.1 work with Spark Structured Streaming 2.1.1?

2017-05-19 Thread kant kodali
Look for something that is more like in this video where the graphs automatically update themselves. Is that possible in Zeppelin? https://www.youtube.com/watch?v=IJmFTXvUZgY You can watch it from 9:20 On Fri, May 19, 2017 at 9:21 AM, kant kodali <kanth...@gmail.com> wrote: > Hi A

How to print data to console in structured streaming using Spark 2.1.0 and Zeppelin 0.7.1?

2017-05-16 Thread kant kodali
Hi All, I have the following code. val ds = sparkSession.readStream() .format("kafka") .option("kafka.bootstrap.servers",bootstrapServers)) .option("subscribe", topicName) .option("checkpointLocation", hdfsCheckPointDir)

Re: How to create a real time dashboards from spark using web socket?

2017-04-25 Thread kant kodali
Cheung <felixcheun...@hotmail.com> wrote: > You could have multiple notes sharing interpreter - could you have one > note for setup to call streamingcontext.start and another note on a > schedule just to run the select sql statement? > > _____ >

Re: How to create a real time dashboards from spark using web socket?

2017-04-25 Thread kant kodali
Doan <doanduy...@gmail.com> wrote: > Yes, the scheduler applies to the entire note, not just paragraph > Le 25 avr. 2017 02:39, "kant kodali" <kanth...@gmail.com> a écrit : > >> Also it doesn't look like we can run a scheduler on one paragraph? we >&g

How to create a real time dashboards from spark using web socket?

2017-04-24 Thread kant kodali
Hi All, I currently do the following val jsonDStream = getJsonDStream() jsonDStream.foreachRDD{rdd => val jsonDF = spark.read.json(rdd) jsonDF.createOrReplaceTempView("dataframe") } client.startStream() %spark.sql select * from dataframe I can see the data and everytime I click a run

Re: Zeppelin Spark Streaming Twitter Stuck

2017-04-24 Thread kant kodali
@Chaoran Yu Yeah I don't think its dependency issue. you wouldn't be able to call methods if you are missing dependencies. I am also in a similar boat though I am trying to get Streaming and Zeppelin to work except I have my own indirect receiver (not the direct stream). That twitter example is

Re: Zeppelin Spark Streaming Twitter Stuck

2017-04-24 Thread kant kodali
ndf On Sun, Apr 23, 2017 at 12:11 PM, kant kodali <kanth...@gmail.com> wrote: > @Chaoran Yu Yeah I don't think its dependency issue. you wouldn't be able > to call methods if you are missing dependencies. > > I am also in a similar boat though I am trying to get Streaming and &

Re: java.lang.NullpointerException

2017-04-22 Thread kant kodali
FYI: I am using Spark Standalone mode On Sat, Apr 22, 2017 at 10:57 PM, kant kodali <kanth...@gmail.com> wrote: > Hi All, > > I get the below stack trace when I am using Zeppelin. If I don't use > Zeppelin all my client Jobs are running fine. I am using spark 2.1.0 > > I

java.lang.NullpointerException

2017-04-22 Thread kant kodali
Hi All, I get the below stack trace when I am using Zeppelin. If I don't use Zeppelin all my client Jobs are running fine. I am using spark 2.1.0 I am not sure why Zeppelin is unable to create a SparkContext ? but then it says it created SparkSession so it doesn't seem to make a lot of sense.

Re: How to use web sockets with Zeppellin

2016-09-08 Thread kant kodali
@Corneau The links you have pointed out doesnt show how I can retrieve messages through websocket and tie to chars. I would stress again and say how do i tie all this up to charts? On Thu, Sep 8, 2016 at 12:17 AM, kant kodali <kanth...@gmail.com> wrote: > Hi Corneau, > &g

Re: How to use web sockets with Zeppellin

2016-09-08 Thread kant kodali
l > > You will find the list of existing interpreters in the menu. > > If you do need to create one, then you can refer to this guide > https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/development/ > writingzeppelininterpreter.html > > On Thu, Sep 8, 2016 at 3:56 PM, kant k

Re: How to use web sockets with Zeppellin

2016-09-08 Thread kant kodali
> triigger a [s-c] message from backend, a frontend event handler may also > required. > > > 在 2016年9月8日,11:39,kant kodali <kanth...@gmail.com> 写道: > > ok here is a concrete example. I am still trying to connect the dots..If I > were to split Zeppelin as both front end

Re: How to use web sockets with Zeppellin

2016-09-07 Thread kant kodali
used > Spark Streaming here: > https://www.zeppelinhub.com/viewer/showcases/Streaming > > On Thu, Sep 8, 2016 at 3:30 AM, kant kodali <kanth...@gmail.com> wrote: > >> Hi! Thanks for this but I still see the big picture. I have not used >> Angular before nor I am a fro

Re: How to use web sockets with Zeppellin

2016-09-07 Thread kant kodali
5:05 AM, spacewalk...@163.com spacewalk...@163.com wrote: @kant kodali change   zeppelin-web\src\components\baseUrl\baseUrl.service.js like this: this.getWebsocketUrl = function() {    var wsProtocol = location.protocol === 'https:' ? 'wss:' : 'ws:';    return wsProtocol + ‘//your_ip_address

How to use web sockets with Zeppellin

2016-09-07 Thread kant kodali
HI Guys, I have a server which can handle websocket connections. Now I would like to see if there is a way to tell apache zeppellin to create to websocket connect to my server and display the whatever it received? Thanks, Kant

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-25 Thread kant kodali
, Aug 25, 2016 at 11:54 AM, kant kodali < kanth...@gmail.com > wrote: yeah In a real time dashboard we need less than a second latency or even millisecond. On Wed, Aug 24, 2016 7:42 PM, Vinay Shukla vinayshu...@gmail.com wrote: Kant, Moon referred to the attached feature. You can have the

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
Aug 25, 2016 at 11:19 AM, kant kodali < kanth...@gmail.com > wrote: I dont know how angular works moreover I don't understand why we should be tied to certain framework? On Wed, Aug 24, 2016 6:53 PM, Corneau Damien cornead...@gmail.com wrote: Wouldn't angular bind work in that case? On Thu,

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
e: Try find small 'clock' icon on any particular Note next to 'Remove' button. This scheduler feature will help you auto refresh Note. Thanks, moon On Wed, Aug 24, 2016 at 11:31 AM kant kodali < kanth...@gmail.com > wrote: at very least can we auto refreshing on the Zeppelin dashboard on a

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval? On Wed, Aug 24, 2016 10:56 AM, kant kodali kanth...@gmail.com wrote: I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of the box

is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
Hi Guys, is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best