Hi Palle, I think there is currently no way of sending the data from a streaming Flink job into Zeppelin. What rate / amount of data do you expect to send every 10 seconds to the visualization tool?
People have used Flink -> ES -> Kibana for this purpose in the past [1], but I think you can not send millions of records per second into ES. Something like 1000 - 5000 elements / second should easily work for a small ES setup. [1] https://www.elastic.co/blog/building-real-time-dashboard-applications-with-apache-flink-elasticsearch-and-kibana On Mon, May 30, 2016 at 10:00 AM, Palle <pa...@sport.dk> wrote: > OK, I found product that seems to be what I am looking for: Apache > Zeppelin. I will have a look into that one. If anyone can point me to an > example (Git) outputting data from Flink to the Zeppelin Notebook I would > be happy. > > > ----- Original meddelelse ----- > > Fra: Palle <pa...@sport.dk> > > Til: user@flink.apache.org > > Dato: Man, 30. maj 2016 08:20 > > Emne: Re: Visualize result of Flink job > > > > I know exactly what to visualize. As I wrote, it is the latest > > result of the Flink job I would like to visualize. There is no need > > to use elastic to find it first. > > > > The data I have is of such a nature that they every 10 seconds > > could be written into a file, meaning that the file at all times > > would contain the most recent results (at latest 10 seconds old). I > > am not interested in the history, and therefore I should think > > elastic is not the best fit. So my question is if anyone knows of a > > component (Apache or other) that can make the visualization a little > > nicer than just the file :-) > > > > > > ----- Original meddelelse ----- > > > Fra: Kanstantsin Kamkou <kkam...@gmail.com> > > > Til: user@flink.apache.org > > > Dato: Søn, 29. maj 2016 22:42 > > > Emne: Re: Visualize result of Flink job > > > > > > > I am thinking it may not be the best fit, because Elastic is by > > > nature a search engine that is good for trending and stuff like > > that > > > - not entire replacement of the current view. > > > Why u think that the elasticsearch is not the right tool? To > > > visualise > > > something u have to find what to visualise first, right? > > > > > > > > > On Sun, May 29, 2016 at 8:20 PM, Palle <pa...@sport.dk> wrote: > > > > Hi there > > > > > > > > I am using Flink to analyse a lot of incoming data. Every 10 > > > seconds it makes sense to present the analysis so far as some form > > > of visualization. Every 10 seconds I therefore will replace the > > > current contents of the visualization/presentation with the > > analysis > > > result of the most recent 10 seconds. > > > > > > > > I was first thinking of using ElasticSearch/Kibana for this > > > because I know it should be easy to set up, but I am thinking it > > may > > > not be the best fit, because Elastic is by nature a search engine > > > that is good for trending and stuff like that - not entire > > > replacement of the current view. And therefore I may also > > experience > > > difficulties implementing the view in Elastic. > > > > > > > > Does anyone know of any other visualization tools that work well > > > with Flink? ...where it is easy to export the result of a Flink > > job > > > to a user interface (web). > > > > > > > > Thanks > > > > Palle > > > >
