This is an automated email from the ASF dual-hosted git repository. fhueske pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit 89bff8715428521d722d700b6044b2e1de44d4d8 Author: Fabian Hueske <fhue...@apache.org> AuthorDate: Mon Mar 11 14:15:36 2019 +0100 Rebuild website. --- content/blog/feed.xml | 135 ++++++++ content/blog/index.html | 38 ++- content/blog/page2/index.html | 40 ++- content/blog/page3/index.html | 40 ++- content/blog/page4/index.html | 40 ++- content/blog/page5/index.html | 40 ++- content/blog/page6/index.html | 43 +-- content/blog/page7/index.html | 43 ++- content/blog/page8/index.html | 25 ++ .../features/2019/03/11/prometheus-monitoring.html | 339 +++++++++++++++++++++ .../prometheus.png | Bin 0 -> 56449 bytes .../prometheusalerts.png | Bin 0 -> 80256 bytes .../prometheusexamplejob.png | Bin 0 -> 60456 bytes content/index.html | 6 +- content/zh/index.html | 6 +- 15 files changed, 681 insertions(+), 114 deletions(-) diff --git a/content/blog/feed.xml b/content/blog/feed.xml index 545dce9..1381bb5 100644 --- a/content/blog/feed.xml +++ b/content/blog/feed.xml @@ -7,6 +7,141 @@ <atom:link href="https://flink.apache.org/blog/feed.xml" rel="self" type="application/rss+xml" /> <item> +<title>Flink and Prometheus: Cloud-native monitoring of streaming applications</title> +<description><p>This blog post describes how developers can leverage Apache Flink’s built-in <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/monitoring/metrics.html">metrics system</a> together with <a href="https://prometheus.io/">Prometheus</a> to observe and monitor streaming applications in an effective way. This is a follow-up post from my <a href="https://flink-forward.org/">Flink Forward</a&g [...] + +<h2 id="why-prometheus">Why Prometheus?</h2> + +<p>Prometheus is a metrics-based monitoring system that was originally created in 2012. The system is completely open-source (under the Apache License 2) with a vibrant community behind it and it has graduated from the Cloud Native Foundation last year – a sign of maturity, stability and production-readiness. As we mentioned, the system is based on metrics and it is designed to measure the overall health, behavior and performance of a service. Prometheus features a multi-dimensiona [...] + +<ul> + <li> + <p><strong>Metrics:</strong> Prometheus defines metrics as floats of information that change in time. These time series have millisecond precision.</p> + </li> + <li> + <p><strong>Labels</strong> are the key-value pairs associated with time series that support Prometheus’ flexible and powerful data model – in contrast to hierarchical data structures that one might experience with traditional metrics systems.</p> + </li> + <li> + <p><strong>Scrape:</strong> Prometheus is a pull-based system and fetches (“scrapes”) metrics data from specified sources that expose HTTP endpoints with a text-based format.</p> + </li> + <li> + <p><strong>PromQL</strong> is Prometheus’ <a href="https://prometheus.io/docs/prometheus/latest/querying/basics/">query language</a>. It can be used for both building dashboards and setting up alert rules that will trigger when specific conditions are met.</p> + </li> +</ul> + +<p>When considering metrics and monitoring systems for your Flink jobs, there are many <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/monitoring/metrics.html">options</a>. Flink offers native support for exposing data to Prometheus via the <code>PrometheusReporter</code> configuration. Setting up this integration is very easy.</p> + +<p>Prometheus is a great choice as usually Flink jobs are not running in isolation but in a greater context of microservices. For making metrics available to Prometheus from other parts of a larger system, there are two options: There exist <a href="https://prometheus.io/docs/instrumenting/clientlibs/">libraries for all major languages</a> to instrument other applications. Additionally, there is a wide variety of <a href="https://prometheus.io/docs/ins [...] + +<h2 id="prometheus-and-flink-in-action">Prometheus and Flink in Action</h2> + +<p>We have provided a <a href="https://github.com/mbode/flink-prometheus-example">GitHub repository</a> that demonstrates the integration described above. To have a look, clone the repository, make sure <a href="https://docs.docker.com/install/">Docker</a> is installed and run:</p> + +<div class="highlight"><pre><code>./gradlew composeUp +</code></pre></div> + +<p>This builds a Flink job using the build tool <a href="https://gradle.org/">Gradle</a> and starts up a local environment based on <a href="https://docs.docker.com/compose/">Docker Compose</a> running the job in a <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/ops/deployment/docker.html#flink-job-cluster">Flink job cluster</a> (reachable at <a href="http://localhost:8081/">http: [...] + +<center> +<img src="/img/blog/2019-03-11-prometheus-monitoring/prometheusexamplejob.png" width="600px" alt="PrometheusExampleJob in Flink Web UI" /> +<br /> +<i><small>Job graph and custom metric for example job in Flink web interface.</small></i> +</center> +<p><br /></p> + +<p>The <code>PrometheusExampleJob</code> has three operators: Random numbers up to 10,000 are generated, then a map counts the events and creates a histogram of the values passed through. Finally, the events are discarded without further output. The very simple code below is from the second operator. It illustrates how easy it is to add custom metrics relevant to your business logic into your Flink job.</p> + +<div class="highlight"><pre><code class="language-java"><span class="kd">class</span> <span class="nc">FlinkMetricsExposingMapFunction</span> <span class="kd">extends</span> <span class="n">RichMapFunction</span><span class="o">&lt;</span><span class="n">Integer</span><span class="o">,</span> &l [...] + <span class="kd">private</span> <span class="kd">transient</span> <span class="n">Counter</span> <span class="n">eventCounter</span><span class="o">;</span> + + <span class="nd">@Override</span> + <span class="kd">public</span> <span class="kt">void</span> <span class="nf">open</span><span class="o">(</span><span class="n">Configuration</span> <span class="n">parameters</span><span class="o">)</span> <span class="o">{</span> + <span class="n">eventCounter</span> <span class="o">=</span> <span class="n">getRuntimeContext</span><span class="o">().</span><span class="na">getMetricGroup</span><span class="o">().</span><span class="na">counter</span><span class="o">(</span><span class="s">&quot;events&quot;</spa [...] + <span class="o">}</span> + + <span class="nd">@Override</span> + <span class="kd">public</span> <span class="n">Integer</span> <span class="nf">map</span><span class="o">(</span><span class="n">Integer</span> <span class="n">value</span><span class="o">)</span> <span class="o">{</span> + <span class="n">eventCounter</span><span class="o">.</span><span class="na">inc</span><span class="o">();</span> + <span class="k">return</span> <span class="n">value</span><span class="o">;</span> + <span class="o">}</span> +<span class="o">}</span></code></pre></div> +<center><i><small>Excerpt from <a href="https://github.com/mbode/flink-prometheus-example/blob/master/src/main/java/com/github/mbode/flink_prometheus_example/FlinkMetricsExposingMapFunction.java">FlinkMetricsExposingMapFunction.java</a> demonstrating custom Flink metric.</small></i></center> + +<h2 id="configuring-prometheus-with-flink">Configuring Prometheus with Flink</h2> + +<p>To start monitoring Flink with Prometheus, the following steps are necessary:</p> + +<ol> + <li> + <p>Make the <code>PrometheusReporter</code> jar available to the classpath of the Flink cluster (it comes with the Flink distribution):</p> + + <div class="highlight"><pre><code> cp /opt/flink/opt/flink-metrics-prometheus-1.7.2.jar /opt/flink/lib +</code></pre></div> + </li> + <li> + <p><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/monitoring/metrics.html#reporter">Configure the reporter</a> in Flink’s <em>flink-conf.yaml</em>. All job managers and task managers will expose the metrics on the configured port.</p> + + <div class="highlight"><pre><code> metrics.reporters: prom + metrics.reporter.prom.class: org.apache.flink.metrics.prometheus.PrometheusReporter + metrics.reporter.prom.port: 9999 +</code></pre></div> + </li> + <li> + <p>Prometheus needs to know where to scrape metrics. In a static scenario, you can simply <a href="https://prometheus.io/docs/prometheus/latest/configuration/configuration/">configure Prometheus</a> in <em>prometheus.yml</em> with the following:</p> + + <div class="highlight"><pre><code> scrape_configs: + - job_name: 'flink' + static_configs: + - targets: ['job-cluster:9999', 'taskmanager1:9999', 'taskmanager2:9999'] +</code></pre></div> + + <p>In more dynamic scenarios we recommend using Prometheus’ service discovery support for different platforms such as Kubernetes, AWS EC2 and more.</p> + </li> +</ol> + +<p>Both custom metrics are now available in Prometheus:</p> + +<center> +<img src="/img/blog/2019-03-11-prometheus-monitoring/prometheus.png" width="600px" alt="Prometheus web UI with example metric" /> +<br /> +<i><small>Example metric in Prometheus web UI.</small></i> +</center> +<p><br /></p> + +<p>More technical metrics from the Flink cluster (like checkpoint sizes or duration, Kafka offsets or resource consumption) are also available. If you are interested, you can check out the HTTP endpoints exposing all Prometheus metrics for the job managers and the two task managers on <a href="http://localhost:9249/metrics">http://localhost:9249</a>, <a href="http://localhost:9250/metrics">http://localhost:9250</a> and <a href="ht [...] + +<p>To test Prometheus’ alerting feature, kill one of the Flink task managers via</p> + +<div class="highlight"><pre><code>docker kill taskmanager1 +</code></pre></div> + +<p>Our Flink job can recover from this partial failure via the mechanism of <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/state/checkpointing.html">Checkpointing</a>. Nevertheless, after roughly one minute (as configured in the alert rule) the following alert will fire:</p> + +<center> +<img src="/img/blog/2019-03-11-prometheus-monitoring/prometheusalerts.png" width="600px" alt="Prometheus web UI with example alert" /> +<br /> +<i><small>Example alert in Prometheus web UI.</small></i> +</center> +<p><br /></p> + +<p>In real-world situations alerts like this one can be routed through a component called <a href="https://prometheus.io/docs/alerting/alertmanager/">Alertmanager</a> and be grouped into notifications to systems like email, PagerDuty or Slack.</p> + +<p>Go ahead and play around with the setup, and check out the <a href="https://grafana.com/grafana">Grafana</a> instance reachable at <a href="http://localhost:3000/">http://localhost:3000</a> (credentials <em>admin:flink</em>) for visualizing Prometheus metrics. If there are any questions or problems, feel free to <a href="https://github.com/mbode/flink-prometheus-example/issues">create an issue</a>. Onc [...] + +<div class="highlight"><pre><code>./gradlew composeDown +</code></pre></div> +<p><br /></p> + +<h2 id="conclusion">Conclusion</h2> + +<p>Using Prometheus together with Flink provides an easy way for effective monitoring and alerting of your Flink jobs. Both projects have exciting and vibrant communities behind them with new developments and additions scheduled for upcoming releases. We encourage you to try the two technologies together as it has immensely improved our insights into Flink jobs running in production.</p> +</description> +<pubDate>Mon, 11 Mar 2019 13:00:00 +0100</pubDate> +<link>https://flink.apache.org/features/2019/03/11/prometheus-monitoring.html</link> +<guid isPermaLink="true">/features/2019/03/11/prometheus-monitoring.html</guid> +</item> + +<item> <title>What to expect from Flink Forward San Francisco 2019</title> <description><p>The third annual Flink Forward San Francisco is just a few weeks away! As always, Flink Forward will be the right place to meet and mingle with experienced Flink users, contributors, and committers. Attendees will hear and chat about the latest developments around Flink and learn from technical deep-dive sessions and exciting use cases that were put into production with Flink. The event will take place on April 1-2, 2019 at Hotel Nikko in San Francisco. The <a hr [...] diff --git a/content/blog/index.html b/content/blog/index.html index ca662d4..49a81c1 100644 --- a/content/blog/index.html +++ b/content/blog/index.html @@ -159,6 +159,19 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></h2> + + <p>11 Mar 2019 + Maximilian Bode, TNG Technology Consulting (<a href="https://twitter.com/mxpbode">@mxpbode</a>)</p> + + <p>This blog post describes how developers can leverage Apache Flink's built-in metrics system together with Prometheus to observe and monitor streaming applications in an effective way.</p> + + <p><a href="/features/2019/03/11/prometheus-monitoring.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></h2> <p>06 Mar 2019 @@ -289,21 +302,6 @@ Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa <hr> - <article> - <h2 class="blog-title"><a href="/news/2018/10/29/release-1.6.2.html">Apache Flink 1.6.2 Released</a></h2> - - <p>29 Oct 2018 - </p> - - <p><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.6 series.</p> - -</p> - - <p><a href="/news/2018/10/29/release-1.6.2.html">Continue reading »</a></p> - </article> - - <hr> - <!-- Pagination links --> @@ -336,6 +334,16 @@ Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html index 10064b3..e2246cb 100644 --- a/content/blog/page2/index.html +++ b/content/blog/page2/index.html @@ -159,6 +159,21 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/news/2018/10/29/release-1.6.2.html">Apache Flink 1.6.2 Released</a></h2> + + <p>29 Oct 2018 + </p> + + <p><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.6 series.</p> + +</p> + + <p><a href="/news/2018/10/29/release-1.6.2.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/news/2018/10/29/release-1.5.5.html">Apache Flink 1.5.5 Released</a></h2> <p>29 Oct 2018 @@ -293,21 +308,6 @@ <hr> - <article> - <h2 class="blog-title"><a href="/news/2018/03/08/release-1.4.2.html">Apache Flink 1.4.2 Released</a></h2> - - <p>08 Mar 2018 - </p> - - <p><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.4 series.</p> - -</p> - - <p><a href="/news/2018/03/08/release-1.4.2.html">Continue reading »</a></p> - </article> - - <hr> - <!-- Pagination links --> @@ -340,6 +340,16 @@ <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html index a79c361..83d257f 100644 --- a/content/blog/page3/index.html +++ b/content/blog/page3/index.html @@ -159,6 +159,21 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/news/2018/03/08/release-1.4.2.html">Apache Flink 1.4.2 Released</a></h2> + + <p>08 Mar 2018 + </p> + + <p><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.4 series.</p> + +</p> + + <p><a href="/news/2018/03/08/release-1.4.2.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/features/2018/03/01/end-to-end-exactly-once-apache-flink.html">An Overview of End-to-End Exactly-Once Processing in Apache Flink (with Apache Kafka, too!)</a></h2> <p>01 Mar 2018 @@ -290,21 +305,6 @@ what’s coming in Flink 1.4.0 as well as a preview of what the Flink community <hr> - <article> - <h2 class="blog-title"><a href="/news/2017/06/01/release-1.3.0.html">Apache Flink 1.3.0 Release Announcement</a></h2> - - <p>01 Jun 2017 by Robert Metzger (<a href="https://twitter.com/">@rmetzger_</a>) - </p> - - <p><p>The Apache Flink community is pleased to announce the 1.3.0 release. Over the past 4 months, the Flink community has been working hard to resolve more than 680 issues. See the <a href="/blog/release_1.3.0-changelog.html">complete changelog</a> for more detail.</p> - -</p> - - <p><a href="/news/2017/06/01/release-1.3.0.html">Continue reading »</a></p> - </article> - - <hr> - <!-- Pagination links --> @@ -337,6 +337,16 @@ what’s coming in Flink 1.4.0 as well as a preview of what the Flink community <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html index 1d9510c..f99fe8d 100644 --- a/content/blog/page4/index.html +++ b/content/blog/page4/index.html @@ -159,6 +159,21 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/news/2017/06/01/release-1.3.0.html">Apache Flink 1.3.0 Release Announcement</a></h2> + + <p>01 Jun 2017 by Robert Metzger (<a href="https://twitter.com/">@rmetzger_</a>) + </p> + + <p><p>The Apache Flink community is pleased to announce the 1.3.0 release. Over the past 4 months, the Flink community has been working hard to resolve more than 680 issues. See the <a href="/blog/release_1.3.0-changelog.html">complete changelog</a> for more detail.</p> + +</p> + + <p><a href="/news/2017/06/01/release-1.3.0.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/news/2017/05/16/official-docker-image.html">Introducing Docker Images for Apache Flink</a></h2> <p>16 May 2017 by Patrick Lucas (Data Artisans) and Ismaël Mejía (Talend) (<a href="https://twitter.com/">@iemejia</a>) @@ -286,21 +301,6 @@ <hr> - <article> - <h2 class="blog-title"><a href="/news/2016/09/05/release-1.1.2.html">Apache Flink 1.1.2 Released</a></h2> - - <p>05 Sep 2016 - </p> - - <p><p>The Apache Flink community released another bugfix version of the Apache Flink 1.1. series.</p> - -</p> - - <p><a href="/news/2016/09/05/release-1.1.2.html">Continue reading »</a></p> - </article> - - <hr> - <!-- Pagination links --> @@ -333,6 +333,16 @@ <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html index af8f781..2dd01f0 100644 --- a/content/blog/page5/index.html +++ b/content/blog/page5/index.html @@ -159,6 +159,21 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/news/2016/09/05/release-1.1.2.html">Apache Flink 1.1.2 Released</a></h2> + + <p>05 Sep 2016 + </p> + + <p><p>The Apache Flink community released another bugfix version of the Apache Flink 1.1. series.</p> + +</p> + + <p><a href="/news/2016/09/05/release-1.1.2.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/news/2016/08/24/ff16-keynotes-panels.html">Flink Forward 2016: Announcing Schedule, Keynotes, and Panel Discussion</a></h2> <p>24 Aug 2016 @@ -290,21 +305,6 @@ <hr> - <article> - <h2 class="blog-title"><a href="/news/2016/03/08/release-1.0.0.html">Announcing Apache Flink 1.0.0</a></h2> - - <p>08 Mar 2016 - </p> - - <p><p>The Apache Flink community is pleased to announce the availability of the 1.0.0 release. The community put significant effort into improving and extending Apache Flink since the last release, focusing on improving the experience of writing and executing data stream processing pipelines in production.</p> - -</p> - - <p><a href="/news/2016/03/08/release-1.0.0.html">Continue reading »</a></p> - </article> - - <hr> - <!-- Pagination links --> @@ -337,6 +337,16 @@ <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/blog/page6/index.html b/content/blog/page6/index.html index c611fba..a21d770 100644 --- a/content/blog/page6/index.html +++ b/content/blog/page6/index.html @@ -159,6 +159,21 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/news/2016/03/08/release-1.0.0.html">Announcing Apache Flink 1.0.0</a></h2> + + <p>08 Mar 2016 + </p> + + <p><p>The Apache Flink community is pleased to announce the availability of the 1.0.0 release. The community put significant effort into improving and extending Apache Flink since the last release, focusing on improving the experience of writing and executing data stream processing pipelines in production.</p> + +</p> + + <p><a href="/news/2016/03/08/release-1.0.0.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/news/2016/02/11/release-0.10.2.html">Flink 0.10.2 Released</a></h2> <p>11 Feb 2016 @@ -291,24 +306,6 @@ Apache Flink started.</p> <hr> - <article> - <h2 class="blog-title"><a href="/news/2015/08/24/introducing-flink-gelly.html">Introducing Gelly: Graph Processing with Apache Flink</a></h2> - - <p>24 Aug 2015 - </p> - - <p><p>This blog post introduces <strong>Gelly</strong>, Apache Flink’s <em>graph-processing API and library</em>. Flink’s native support -for iterations makes it a suitable platform for large-scale graph analytics. -By leveraging delta iterations, Gelly is able to map various graph processing models such as -vertex-centric or gather-sum-apply to Flink dataflows.</p> - -</p> - - <p><a href="/news/2015/08/24/introducing-flink-gelly.html">Continue reading »</a></p> - </article> - - <hr> - <!-- Pagination links --> @@ -341,6 +338,16 @@ vertex-centric or gather-sum-apply to Flink dataflows.</p> <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/blog/page7/index.html b/content/blog/page7/index.html index 86a8862..c90e385 100644 --- a/content/blog/page7/index.html +++ b/content/blog/page7/index.html @@ -159,6 +159,24 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/news/2015/08/24/introducing-flink-gelly.html">Introducing Gelly: Graph Processing with Apache Flink</a></h2> + + <p>24 Aug 2015 + </p> + + <p><p>This blog post introduces <strong>Gelly</strong>, Apache Flink’s <em>graph-processing API and library</em>. Flink’s native support +for iterations makes it a suitable platform for large-scale graph analytics. +By leveraging delta iterations, Gelly is able to map various graph processing models such as +vertex-centric or gather-sum-apply to Flink dataflows.</p> + +</p> + + <p><a href="/news/2015/08/24/introducing-flink-gelly.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/news/2015/06/24/announcing-apache-flink-0.9.0-release.html">Announcing Apache Flink 0.9.0</a></h2> <p>24 Jun 2015 @@ -300,21 +318,6 @@ and offers a new API including definition of flexible windows.</p> <hr> - <article> - <h2 class="blog-title"><a href="/news/2015/01/21/release-0.8.html">Apache Flink 0.8.0 available</a></h2> - - <p>21 Jan 2015 - </p> - - <p><p>We are pleased to announce the availability of Flink 0.8.0. This release includes new user-facing features as well as performance and bug fixes, extends the support for filesystems and introduces the Scala API and flexible windowing semantics for Flink Streaming. A total of 33 people have contributed to this release, a big thanks to all of them!</p> - -</p> - - <p><a href="/news/2015/01/21/release-0.8.html">Continue reading »</a></p> - </article> - - <hr> - <!-- Pagination links --> @@ -347,6 +350,16 @@ and offers a new API including definition of flexible windows.</p> <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/blog/page8/index.html b/content/blog/page8/index.html index e433ba1..6fa2118 100644 --- a/content/blog/page8/index.html +++ b/content/blog/page8/index.html @@ -159,6 +159,21 @@ <!-- Blog posts --> <article> + <h2 class="blog-title"><a href="/news/2015/01/21/release-0.8.html">Apache Flink 0.8.0 available</a></h2> + + <p>21 Jan 2015 + </p> + + <p><p>We are pleased to announce the availability of Flink 0.8.0. This release includes new user-facing features as well as performance and bug fixes, extends the support for filesystems and introduces the Scala API and flexible windowing semantics for Flink Streaming. A total of 33 people have contributed to this release, a big thanks to all of them!</p> + +</p> + + <p><a href="/news/2015/01/21/release-0.8.html">Continue reading »</a></p> + </article> + + <hr> + + <article> <h2 class="blog-title"><a href="/news/2015/01/06/december-in-flink.html">December 2014 in the Flink community</a></h2> <p>06 Jan 2015 @@ -283,6 +298,16 @@ academic and open source project that Flink originates from.</p> <ul id="markdown-toc"> + <li><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></li> + + + + + + + + + <li><a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></li> diff --git a/content/features/2019/03/11/prometheus-monitoring.html b/content/features/2019/03/11/prometheus-monitoring.html new file mode 100644 index 0000000..192ba92 --- /dev/null +++ b/content/features/2019/03/11/prometheus-monitoring.html @@ -0,0 +1,339 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> + <title>Apache Flink: Flink and Prometheus: Cloud-native monitoring of streaming applications</title> + <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> + <link rel="icon" href="/favicon.ico" type="image/x-icon"> + + <!-- Bootstrap --> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css"> + <link rel="stylesheet" href="/css/flink.css"> + <link rel="stylesheet" href="/css/syntax.css"> + + <!-- Blog RSS feed --> + <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" /> + + <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> + <!-- We need to load Jquery in the header for custom google analytics event tracking--> + <script src="/js/jquery.min.js"></script> + + <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> + <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> + <!--[if lt IE 9]> + <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> + <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> + <![endif]--> + </head> + <body> + + + <!-- Main content. --> + <div class="container"> + <div class="row"> + + + <div id="sidebar" class="col-sm-3"> + + +<!-- Top navbar. --> + <nav class="navbar navbar-default"> + <!-- The logo. --> + <div class="navbar-header"> + <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <div class="navbar-logo"> + <a href="/"> + <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px"> + </a> + </div> + </div><!-- /.navbar-header --> + + <!-- The navigation links. --> + <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> + <ul class="nav navbar-nav navbar-main"> + + <!-- First menu section explains visitors what Flink is --> + + <!-- What is Stream Processing? --> + <!-- + <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li> + --> + + <!-- What is Flink? --> + <li><a href="/flink-architecture.html">What is Apache Flink?</a></li> + + <!-- Use cases --> + <li><a href="/usecases.html">Use Cases</a></li> + + <!-- Powered by --> + <li><a href="/poweredby.html">Powered By</a></li> + + <!-- FAQ --> + <li><a href="/faq.html">FAQ</a></li> + + + <!-- Second menu section aims to support Flink users --> + + <!-- Downloads --> + <li><a href="/downloads.html">Downloads</a></li> + + <!-- Quickstart --> + <li> + <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/quickstart/setup_quickstart.html" target="_blank">Tutorials <small><span class="glyphicon glyphicon-new-window"></span></small></a> + </li> + + <!-- Documentation --> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a> + <ul class="dropdown-menu"> + <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7" target="_blank">1.7 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> + <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">1.8 (Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> + </ul> + </li> + + <!-- getting help --> + <li><a href="/gettinghelp.html">Getting Help</a></li> + + <!-- Blog --> + <li><a href="/blog/"><b>Flink Blog</b></a></li> + + + + <!-- Third menu section aim to support community and contributors --> + + <!-- Community --> + <li><a href="/community.html">Community & Project Info</a></li> + + <!-- Roadmap --> + <li><a href="/roadmap.html">Roadmap</a></li> + + <!-- Contribute --> + <li><a href="/how-to-contribute.html">How to Contribute</a></li> + + <!-- GitHub --> + <li> + <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a> + </li> + + + + <!-- Language Switcher --> + <li> + + + <a href="/zh/features/2019/03/11/prometheus-monitoring.html">中文版</a> + + + </li> + + </ul> + + <ul class="nav navbar-nav navbar-bottom"> + <hr /> + + <!-- Twitter --> + <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> + + <!-- Visualizer --> + <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> + + </ul> + </div><!-- /.navbar-collapse --> + </nav> + + </div> + <div class="col-sm-9"> + <div class="row-fluid"> + <div class="col-sm-12"> + <div class="row"> + <h1>Flink and Prometheus: Cloud-native monitoring of streaming applications</h1> + + <article> + <p>11 Mar 2019 Maximilian Bode, TNG Technology Consulting (<a href="https://twitter.com/mxpbode">@mxpbode</a>)</p> + +<p>This blog post describes how developers can leverage Apache Flink’s built-in <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/monitoring/metrics.html">metrics system</a> together with <a href="https://prometheus.io/">Prometheus</a> to observe and monitor streaming applications in an effective way. This is a follow-up post from my <a href="https://flink-forward.org/">Flink Forward</a> Berlin 2018 talk (<a href="https://www.slideshare.net/MaximilianBode1/monitoring-f [...] + +<h2 id="why-prometheus">Why Prometheus?</h2> + +<p>Prometheus is a metrics-based monitoring system that was originally created in 2012. The system is completely open-source (under the Apache License 2) with a vibrant community behind it and it has graduated from the Cloud Native Foundation last year – a sign of maturity, stability and production-readiness. As we mentioned, the system is based on metrics and it is designed to measure the overall health, behavior and performance of a service. Prometheus features a multi-dimensional data [...] + +<ul> + <li> + <p><strong>Metrics:</strong> Prometheus defines metrics as floats of information that change in time. These time series have millisecond precision.</p> + </li> + <li> + <p><strong>Labels</strong> are the key-value pairs associated with time series that support Prometheus’ flexible and powerful data model – in contrast to hierarchical data structures that one might experience with traditional metrics systems.</p> + </li> + <li> + <p><strong>Scrape:</strong> Prometheus is a pull-based system and fetches (“scrapes”) metrics data from specified sources that expose HTTP endpoints with a text-based format.</p> + </li> + <li> + <p><strong>PromQL</strong> is Prometheus’ <a href="https://prometheus.io/docs/prometheus/latest/querying/basics/">query language</a>. It can be used for both building dashboards and setting up alert rules that will trigger when specific conditions are met.</p> + </li> +</ul> + +<p>When considering metrics and monitoring systems for your Flink jobs, there are many <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/monitoring/metrics.html">options</a>. Flink offers native support for exposing data to Prometheus via the <code>PrometheusReporter</code> configuration. Setting up this integration is very easy.</p> + +<p>Prometheus is a great choice as usually Flink jobs are not running in isolation but in a greater context of microservices. For making metrics available to Prometheus from other parts of a larger system, there are two options: There exist <a href="https://prometheus.io/docs/instrumenting/clientlibs/">libraries for all major languages</a> to instrument other applications. Additionally, there is a wide variety of <a href="https://prometheus.io/docs/instrumenting/exporters/">exporters</a> [...] + +<h2 id="prometheus-and-flink-in-action">Prometheus and Flink in Action</h2> + +<p>We have provided a <a href="https://github.com/mbode/flink-prometheus-example">GitHub repository</a> that demonstrates the integration described above. To have a look, clone the repository, make sure <a href="https://docs.docker.com/install/">Docker</a> is installed and run:</p> + +<div class="highlight"><pre><code>./gradlew composeUp +</code></pre></div> + +<p>This builds a Flink job using the build tool <a href="https://gradle.org/">Gradle</a> and starts up a local environment based on <a href="https://docs.docker.com/compose/">Docker Compose</a> running the job in a <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/ops/deployment/docker.html#flink-job-cluster">Flink job cluster</a> (reachable at <a href="http://localhost:8081/">http://localhost:8081</a>) as well as a Prometheus instance (<a href="http://localhost:9090/" [...] + +<center> +<img src="/img/blog/2019-03-11-prometheus-monitoring/prometheusexamplejob.png" width="600px" alt="PrometheusExampleJob in Flink Web UI" /> +<br /> +<i><small>Job graph and custom metric for example job in Flink web interface.</small></i> +</center> +<p><br /></p> + +<p>The <code>PrometheusExampleJob</code> has three operators: Random numbers up to 10,000 are generated, then a map counts the events and creates a histogram of the values passed through. Finally, the events are discarded without further output. The very simple code below is from the second operator. It illustrates how easy it is to add custom metrics relevant to your business logic into your Flink job.</p> + +<div class="highlight"><pre><code class="language-java"><span class="kd">class</span> <span class="nc">FlinkMetricsExposingMapFunction</span> <span class="kd">extends</span> <span class="n">RichMapFunction</span><span class="o"><</span><span class="n">Integer</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="o">{</span> + <span class="kd">private</span> <span class="kd">transient</span> <span class="n">Counter</span> <span class="n">eventCounter</span><span class="o">;</span> + + <span class="nd">@Override</span> + <span class="kd">public</span> <span class="kt">void</span> <span class="nf">open</span><span class="o">(</span><span class="n">Configuration</span> <span class="n">parameters</span><span class="o">)</span> <span class="o">{</span> + <span class="n">eventCounter</span> <span class="o">=</span> <span class="n">getRuntimeContext</span><span class="o">().</span><span class="na">getMetricGroup</span><span class="o">().</span><span class="na">counter</span><span class="o">(</span><span class="s">"events"</span><span class="o">);</span> + <span class="o">}</span> + + <span class="nd">@Override</span> + <span class="kd">public</span> <span class="n">Integer</span> <span class="nf">map</span><span class="o">(</span><span class="n">Integer</span> <span class="n">value</span><span class="o">)</span> <span class="o">{</span> + <span class="n">eventCounter</span><span class="o">.</span><span class="na">inc</span><span class="o">();</span> + <span class="k">return</span> <span class="n">value</span><span class="o">;</span> + <span class="o">}</span> +<span class="o">}</span></code></pre></div> +<center><i><small>Excerpt from <a href="https://github.com/mbode/flink-prometheus-example/blob/master/src/main/java/com/github/mbode/flink_prometheus_example/FlinkMetricsExposingMapFunction.java">FlinkMetricsExposingMapFunction.java</a> demonstrating custom Flink metric.</small></i></center> + +<h2 id="configuring-prometheus-with-flink">Configuring Prometheus with Flink</h2> + +<p>To start monitoring Flink with Prometheus, the following steps are necessary:</p> + +<ol> + <li> + <p>Make the <code>PrometheusReporter</code> jar available to the classpath of the Flink cluster (it comes with the Flink distribution):</p> + + <div class="highlight"><pre><code> cp /opt/flink/opt/flink-metrics-prometheus-1.7.2.jar /opt/flink/lib +</code></pre></div> + </li> + <li> + <p><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/monitoring/metrics.html#reporter">Configure the reporter</a> in Flink’s <em>flink-conf.yaml</em>. All job managers and task managers will expose the metrics on the configured port.</p> + + <div class="highlight"><pre><code> metrics.reporters: prom + metrics.reporter.prom.class: org.apache.flink.metrics.prometheus.PrometheusReporter + metrics.reporter.prom.port: 9999 +</code></pre></div> + </li> + <li> + <p>Prometheus needs to know where to scrape metrics. In a static scenario, you can simply <a href="https://prometheus.io/docs/prometheus/latest/configuration/configuration/">configure Prometheus</a> in <em>prometheus.yml</em> with the following:</p> + + <div class="highlight"><pre><code> scrape_configs: + - job_name: 'flink' + static_configs: + - targets: ['job-cluster:9999', 'taskmanager1:9999', 'taskmanager2:9999'] +</code></pre></div> + + <p>In more dynamic scenarios we recommend using Prometheus’ service discovery support for different platforms such as Kubernetes, AWS EC2 and more.</p> + </li> +</ol> + +<p>Both custom metrics are now available in Prometheus:</p> + +<center> +<img src="/img/blog/2019-03-11-prometheus-monitoring/prometheus.png" width="600px" alt="Prometheus web UI with example metric" /> +<br /> +<i><small>Example metric in Prometheus web UI.</small></i> +</center> +<p><br /></p> + +<p>More technical metrics from the Flink cluster (like checkpoint sizes or duration, Kafka offsets or resource consumption) are also available. If you are interested, you can check out the HTTP endpoints exposing all Prometheus metrics for the job managers and the two task managers on <a href="http://localhost:9249/metrics">http://localhost:9249</a>, <a href="http://localhost:9250/metrics">http://localhost:9250</a> and <a href="http://localhost:9251/metrics">http://localhost:9251</a>, re [...] + +<p>To test Prometheus’ alerting feature, kill one of the Flink task managers via</p> + +<div class="highlight"><pre><code>docker kill taskmanager1 +</code></pre></div> + +<p>Our Flink job can recover from this partial failure via the mechanism of <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/state/checkpointing.html">Checkpointing</a>. Nevertheless, after roughly one minute (as configured in the alert rule) the following alert will fire:</p> + +<center> +<img src="/img/blog/2019-03-11-prometheus-monitoring/prometheusalerts.png" width="600px" alt="Prometheus web UI with example alert" /> +<br /> +<i><small>Example alert in Prometheus web UI.</small></i> +</center> +<p><br /></p> + +<p>In real-world situations alerts like this one can be routed through a component called <a href="https://prometheus.io/docs/alerting/alertmanager/">Alertmanager</a> and be grouped into notifications to systems like email, PagerDuty or Slack.</p> + +<p>Go ahead and play around with the setup, and check out the <a href="https://grafana.com/grafana">Grafana</a> instance reachable at <a href="http://localhost:3000/">http://localhost:3000</a> (credentials <em>admin:flink</em>) for visualizing Prometheus metrics. If there are any questions or problems, feel free to <a href="https://github.com/mbode/flink-prometheus-example/issues">create an issue</a>. Once finished, do not forget to tear down the setup via</p> + +<div class="highlight"><pre><code>./gradlew composeDown +</code></pre></div> +<p><br /></p> + +<h2 id="conclusion">Conclusion</h2> + +<p>Using Prometheus together with Flink provides an easy way for effective monitoring and alerting of your Flink jobs. Both projects have exciting and vibrant communities behind them with new developments and additions scheduled for upcoming releases. We encourage you to try the two technologies together as it has immensely improved our insights into Flink jobs running in production.</p> + + </article> + </div> + + <div class="row"> + <div id="disqus_thread"></div> + <script type="text/javascript"> + /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */ + var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname + + /* * * DON'T EDIT BELOW THIS LINE * * */ + (function() { + var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; + dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js'; + (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); + })(); + </script> + </div> + </div> +</div> + </div> + </div> + + <hr /> + + <div class="row"> + <div class="footer text-center col-sm-12"> + <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p> + <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p> + <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p> + </div> + </div> + </div><!-- /.container --> + + <!-- Include all compiled plugins (below), or include individual files as needed --> + <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script> + <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script> + <script src="/js/codetabs.js"></script> + <script src="/js/stickysidebar.js"></script> + + <!-- Google Analytics --> + <script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-52545728-1', 'auto'); + ga('send', 'pageview'); + </script> + </body> +</html> diff --git a/content/img/blog/2019-03-11-prometheus-monitoring/prometheus.png b/content/img/blog/2019-03-11-prometheus-monitoring/prometheus.png new file mode 100644 index 0000000..a8af061 Binary files /dev/null and b/content/img/blog/2019-03-11-prometheus-monitoring/prometheus.png differ diff --git a/content/img/blog/2019-03-11-prometheus-monitoring/prometheusalerts.png b/content/img/blog/2019-03-11-prometheus-monitoring/prometheusalerts.png new file mode 100755 index 0000000..0b0ce14 Binary files /dev/null and b/content/img/blog/2019-03-11-prometheus-monitoring/prometheusalerts.png differ diff --git a/content/img/blog/2019-03-11-prometheus-monitoring/prometheusexamplejob.png b/content/img/blog/2019-03-11-prometheus-monitoring/prometheusexamplejob.png new file mode 100755 index 0000000..f4bc7ac Binary files /dev/null and b/content/img/blog/2019-03-11-prometheus-monitoring/prometheusexamplejob.png differ diff --git a/content/index.html b/content/index.html index 7d9449f..66b15da 100644 --- a/content/index.html +++ b/content/index.html @@ -442,6 +442,9 @@ <dl> + <dt> <a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></dt> + <dd>This blog post describes how developers can leverage Apache Flink's built-in metrics system together with Prometheus to observe and monitor streaming applications in an effective way.</dd> + <dt> <a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></dt> <dd>The third annual Flink Forward conference in San Francisco is just a few weeks away. Let's see what Flink Forward SF 2019 has in store for the Apache Flink and stream processing communities. This post covers some of its highlights!</dd> @@ -457,9 +460,6 @@ <dd><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.7 series.</p> </dd> - - <dt> <a href="/news/2019/02/13/unified-batch-streaming-blink.html">Batch as a Special Case of Streaming and Alibaba's contribution of Blink</a></dt> - <dd>A few weeks ago, Alibaba contributed its Flink-fork 'Blink' back to Apache Flink. In this blog post we discuss how Blink's features will help the Flink community to make a big step towards its vision to build a truly unified system for stream and batch processing.</dd> </dl> diff --git a/content/zh/index.html b/content/zh/index.html index 4470230..394a574 100644 --- a/content/zh/index.html +++ b/content/zh/index.html @@ -441,6 +441,9 @@ <dl> + <dt> <a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></dt> + <dd>This blog post describes how developers can leverage Apache Flink's built-in metrics system together with Prometheus to observe and monitor streaming applications in an effective way.</dd> + <dt> <a href="/news/2019/03/06/ffsf-preview.html">What to expect from Flink Forward San Francisco 2019</a></dt> <dd>The third annual Flink Forward conference in San Francisco is just a few weeks away. Let's see what Flink Forward SF 2019 has in store for the Apache Flink and stream processing communities. This post covers some of its highlights!</dd> @@ -456,9 +459,6 @@ <dd><p>The Apache Flink community released the second bugfix version of the Apache Flink 1.7 series.</p> </dd> - - <dt> <a href="/news/2019/02/13/unified-batch-streaming-blink.html">Batch as a Special Case of Streaming and Alibaba's contribution of Blink</a></dt> - <dd>A few weeks ago, Alibaba contributed its Flink-fork 'Blink' back to Apache Flink. In this blog post we discuss how Blink's features will help the Flink community to make a big step towards its vision to build a truly unified system for stream and batch processing.</dd> </dl>