Re: Correct way to use spark streaming with apache zeppelin

Hi Storing states/intermediate data in realtime processing depends on how much throughput/latency your application requires. There are lot of technologies that help you build this realtime datastore. Some examples include HBase, Memsql, etc or in some cases an RDBMS like MySQL itself. This is a

Re: Correct way to use spark streaming with apache zeppelin

Thanks all for actively sharing your experience. @Chris: using something like Redis is something I am trying to figure out. I have a lots of transactions, so I couldn't trigger update event for every single transaction. I'm looking at Spark Streaming because it provide batch processing (e.g I

Re: Correct way to use spark streaming with apache zeppelin

Cool! Thanks for sharing. -- Chris Miller On Sun, Mar 13, 2016 at 12:53 AM, Todd Nist wrote: > Below is a link to an example which Silvio Fiorito put together > demonstrating how to link Zeppelin with Spark Stream for real-time charts. > I think the original thread was

Re: Correct way to use spark streaming with apache zeppelin

Certainly the only graphs that I can produce are from the SQL queries on base tables. That basically means that data has to be stored in a permanent tables so temporary tables in Spark cannot be used (?), Additionally it only seems to work on SQL only (and I have not seen any presentation using

Re: Correct way to use spark streaming with apache zeppelin

I'm pretty new to all of this stuff, so bare with me. Zeppelin isn't really intended for realtime dashboards as far as I know. Its reporting features (tables, graphs, etc.) are more for displaying the results from the output of something. As far as I know, there isn't really anything to "watch" a

Re: Correct way to use spark streaming with apache zeppelin

Thanks Chris and Mich for replying. Sorry for not explaining my problem clearly. Yes i am talking about a flexibke dashboard when mention Zeppelin. Here is the problem i am having: I am running a comercial website where we selle many products and we have many branchs in many place. We have a

Re: Correct way to use spark streaming with apache zeppelin

What exactly are you trying to do? Zeppelin is for interactive analysis of a dataset. What do you mean "realtime analytics" -- do you mean build a report or dashboard that automatically updates as new data comes in? -- Chris Miller On Sat, Mar 12, 2016 at 3:13 PM, trung kien

Re: Correct way to use spark streaming with apache zeppelin

Hi, I use Zeppelin as well and in the notebook mode you can do analytics much like what you do in Spark-shell. You can store your intermediate data in Parquet if you wish and then analyse data the way you like. What is your use case here? Zeppelin as I use it is a web UI to your spark-shell,

Correct way to use spark streaming with apache zeppelin

Hi all, I've just viewed some Zeppenlin's videos. The intergration between Zeppenlin and Spark is really amazing and i want to use it for my application. In my app, i will have a Spark streaming app to do some basic realtime aggregation ( intermediate data). Then i want to use Zeppenlin to do