This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit ea7c1874b3986f58f5973117684bc27791222085
Author: Robert Metzger <rmetz...@apache.org>
AuthorDate: Wed Dec 4 16:22:59 2019 +0100
render html
---
content/blog/index.html | 36 +-
content/blog/page10/index.html | 25 ++
content/blog/page2/index.html | 36 +-
content/blog/page3/index.html | 38 +-
content/blog/page4/index.html | 42 +-
content/blog/page5/index.html | 42 +-
content/blog/page6/index.html | 40 +-
content/blog/page7/index.html | 40 +-
content/blog/page8/index.html | 40 +-
content/blog/page9/index.html | 40 +-
...-sql-blog-post-visual-primitive-avro-schema.png | Bin 0 -> 543150 bytes
.../img/blog/flink-pulsar-sql-blog-post-visual.png | Bin 0 -> 68506 bytes
content/index.html | 9 +-
.../query-pulsar-streams-using-apache-flink.html | 425 +++++++++++++++++++++
content/zh/index.html | 9 +-
15 files changed, 677 insertions(+), 145 deletions(-)
diff --git a/content/blog/index.html b/content/blog/index.html
index f3fa1a9..229b3f8 100644
--- a/content/blog/index.html
+++ b/content/blog/index.html
@@ -185,6 +185,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></h2>
+
+ <p>25 Nov 2019
+ Sijie Guo (<a href="https://twitter.com/sijieg";>@sijieg</a>) &
Markos Sfikas (<a href="https://twitter.com/MarkSfik";>@MarkSfik</a>)</p>
+
+ <p>This blog post discusses the new developments and integrations
between Apache Flink and Apache Pulsar and showcases how you can leverage
Pulsar’s built-in schema to query Pulsar streams in real time using Apache
Flink.</p>
+
+ <p><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">Continue
reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1 Released</a></h2>
<p>18 Oct 2019
@@ -310,19 +323,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/2019/05/19/state-ttl.html">State TTL in
Flink 1.8.0: How to Automatically Cleanup Application State in Apache
Flink</a></h2>
-
- <p>19 May 2019
- Fabian Hueske (<a href="https://twitter.com/fhueske";>@fhueske</a>)
& Andrey Zagrebin </p>
-
- <p>A common requirement for many stateful streaming applications is to
automatically cleanup application state for effective management of your state
size, or to control how long the application state can be accessed. State TTL
enables application state cleanup and efficient state size management in Apache
Flink</p>
-
- <p><a href="/2019/05/19/state-ttl.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -355,6 +355,16 @@
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page10/index.html b/content/blog/page10/index.html
index 31fb436..a840a9a 100644
--- a/content/blog/page10/index.html
+++ b/content/blog/page10/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2014/09/26/release-0.6.1.html">Apache Flink 0.6.1 available</a></h2>
+
+ <p>26 Sep 2014
+ </p>
+
+ <p><p>We are happy to announce the availability of Flink 0.6.1.</p>
+
+</p>
+
+ <p><a href="/news/2014/09/26/release-0.6.1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2014/08/26/release-0.6.html">Apache Flink 0.6 available</a></h2>
<p>26 Aug 2014
@@ -234,6 +249,16 @@ academic and open source project that Flink originates
from.</p>
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html
index 7023cd0..a182e6d 100644
--- a/content/blog/page2/index.html
+++ b/content/blog/page2/index.html
@@ -185,6 +185,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/2019/05/19/state-ttl.html">State TTL in
Flink 1.8.0: How to Automatically Cleanup Application State in Apache
Flink</a></h2>
+
+ <p>19 May 2019
+ Fabian Hueske (<a href="https://twitter.com/fhueske";>@fhueske</a>)
& Andrey Zagrebin </p>
+
+ <p>A common requirement for many stateful streaming applications is to
automatically cleanup application state for effective management of your state
size, or to control how long the application state can be accessed. State TTL
enables application state cleanup and efficient state size management in Apache
Flink</p>
+
+ <p><a href="/2019/05/19/state-ttl.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/2019/05/14/temporal-tables.html">Flux
capacitor, huh? Temporal Tables and Joins in Streaming SQL</a></h2>
<p>14 May 2019
@@ -313,19 +326,6 @@ for more details.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2019/02/13/unified-batch-streaming-blink.html">Batch as a Special
Case of Streaming and Alibaba's contribution of Blink</a></h2>
-
- <p>13 Feb 2019
- Stephan Ewen (<a
href="https://twitter.com/stephanewen";>@stephanewen</a>), Fabian Hueske (<a
href="https://twitter.com/fhueske";>@fhueske</a>), & Xiaowei Jiang (<a
href="https://twitter.com/XiaoweiJ";>@XiaoweiJ</a>)</p>
-
- <p>A few weeks ago, Alibaba contributed its Flink-fork 'Blink' back to
Apache Flink. In this blog post we discuss how Blink's features will help the
Flink community to make a big step towards its vision to build a truly unified
system for stream and batch processing.</p>
-
- <p><a
href="/news/2019/02/13/unified-batch-streaming-blink.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -358,6 +358,16 @@ for more details.</p>
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html
index ffd03de..fb49957 100644
--- a/content/blog/page3/index.html
+++ b/content/blog/page3/index.html
@@ -185,6 +185,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2019/02/13/unified-batch-streaming-blink.html">Batch as a Special
Case of Streaming and Alibaba's contribution of Blink</a></h2>
+
+ <p>13 Feb 2019
+ Stephan Ewen (<a
href="https://twitter.com/stephanewen";>@stephanewen</a>), Fabian Hueske (<a
href="https://twitter.com/fhueske";>@fhueske</a>), & Xiaowei Jiang (<a
href="https://twitter.com/XiaoweiJ";>@XiaoweiJ</a>)</p>
+
+ <p>A few weeks ago, Alibaba contributed its Flink-fork 'Blink' back to
Apache Flink. In this blog post we discuss how Blink's features will help the
Flink community to make a big step towards its vision to build a truly unified
system for stream and batch processing.</p>
+
+ <p><a
href="/news/2019/02/13/unified-batch-streaming-blink.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2018/12/26/release-1.5.6.html">Apache Flink 1.5.6 Released</a></h2>
<p>26 Dec 2018
@@ -321,21 +334,6 @@ Please check the <a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2018/08/09/release-1.6.0.html">Apache Flink 1.6.0 Release
Announcement</a></h2>
-
- <p>09 Aug 2018
- Till Rohrmann (<a href="https://twitter.com/stsffap";>@stsffap</a>)</p>
-
- <p><p>The Apache Flink community is proud to announce the 1.6.0 release.
Over the past 2 months, the Flink community has worked hard to resolve more
than 360 issues. Please check the <a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760";>complete
changelog</a> for more details.</p>
-
-</p>
-
- <p><a href="/news/2018/08/09/release-1.6.0.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -368,6 +366,16 @@ Please check the <a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index 0b74a51..2f54f0d 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2018/08/09/release-1.6.0.html">Apache Flink 1.6.0 Release
Announcement</a></h2>
+
+ <p>09 Aug 2018
+ Till Rohrmann (<a href="https://twitter.com/stsffap";>@stsffap</a>)</p>
+
+ <p><p>The Apache Flink community is proud to announce the 1.6.0 release.
Over the past 2 months, the Flink community has worked hard to resolve more
than 360 issues. Please check the <a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760";>complete
changelog</a> for more details.</p>
+
+</p>
+
+ <p><a href="/news/2018/08/09/release-1.6.0.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2018/07/31/release-1.5.2.html">Apache Flink 1.5.2 Released</a></h2>
<p>31 Jul 2018
@@ -313,23 +328,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2017/12/12/release-1.4.0.html">Apache Flink 1.4.0 Release
Announcement</a></h2>
-
- <p>12 Dec 2017
- Aljoscha Krettek (<a href="https://twitter.com/aljoscha";>@aljoscha</a>)
& Mike Winters (<a href="https://twitter.com/wints";>@wints</a>)</p>
-
- <p><p>The Apache Flink community is pleased to announce the 1.4.0
release. Over the past 5 months, the
-Flink community has been working hard to resolve more than 900 issues. See the
<a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340533";>complete
changelog</a>
-for more detail.</p>
-
-</p>
-
- <p><a href="/news/2017/12/12/release-1.4.0.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -362,6 +360,16 @@ for more detail.</p>
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html
index 3e6ea5a..ac68387 100644
--- a/content/blog/page5/index.html
+++ b/content/blog/page5/index.html
@@ -185,6 +185,23 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2017/12/12/release-1.4.0.html">Apache Flink 1.4.0 Release
Announcement</a></h2>
+
+ <p>12 Dec 2017
+ Aljoscha Krettek (<a href="https://twitter.com/aljoscha";>@aljoscha</a>)
& Mike Winters (<a href="https://twitter.com/wints";>@wints</a>)</p>
+
+ <p><p>The Apache Flink community is pleased to announce the 1.4.0
release. Over the past 5 months, the
+Flink community has been working hard to resolve more than 900 issues. See the
<a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340533";>complete
changelog</a>
+for more detail.</p>
+
+</p>
+
+ <p><a href="/news/2017/12/12/release-1.4.0.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2017/11/22/release-1.4-and-1.5-timeline.html">Looking Ahead to
Apache Flink 1.4.0 and 1.5.0</a></h2>
<p>22 Nov 2017
@@ -317,21 +334,6 @@ what’s coming in Flink 1.4.0 as well as a preview of what
the Flink community
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2017/03/23/release-1.1.5.html">Apache Flink 1.1.5 Released</a></h2>
-
- <p>23 Mar 2017
- </p>
-
- <p><p>The Apache Flink community released the next bugfix version of the
Apache Flink 1.1 series.</p>
-
-</p>
-
- <p><a href="/news/2017/03/23/release-1.1.5.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -364,6 +366,16 @@ what’s coming in Flink 1.4.0 as well as a preview of what
the Flink community
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page6/index.html b/content/blog/page6/index.html
index 79810a3..28f2d1c 100644
--- a/content/blog/page6/index.html
+++ b/content/blog/page6/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2017/03/23/release-1.1.5.html">Apache Flink 1.1.5 Released</a></h2>
+
+ <p>23 Mar 2017
+ </p>
+
+ <p><p>The Apache Flink community released the next bugfix version of the
Apache Flink 1.1 series.</p>
+
+</p>
+
+ <p><a href="/news/2017/03/23/release-1.1.5.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2017/02/06/release-1.2.0.html">Announcing Apache Flink
1.2.0</a></h2>
<p>06 Feb 2017 by Robert Metzger
@@ -314,21 +329,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2016/05/11/release-1.0.3.html">Flink 1.0.3 Released</a></h2>
-
- <p>11 May 2016
- </p>
-
- <p><p>Today, the Flink community released Flink version
<strong>1.0.3</strong>, the third bugfix release of the 1.0 series.</p>
-
-</p>
-
- <p><a href="/news/2016/05/11/release-1.0.3.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -361,6 +361,16 @@
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page7/index.html b/content/blog/page7/index.html
index 71b7c22..2befda9 100644
--- a/content/blog/page7/index.html
+++ b/content/blog/page7/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2016/05/11/release-1.0.3.html">Flink 1.0.3 Released</a></h2>
+
+ <p>11 May 2016
+ </p>
+
+ <p><p>Today, the Flink community released Flink version
<strong>1.0.3</strong>, the third bugfix release of the 1.0 series.</p>
+
+</p>
+
+ <p><a href="/news/2016/05/11/release-1.0.3.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2016/04/22/release-1.0.2.html">Flink 1.0.2 Released</a></h2>
<p>22 Apr 2016
@@ -312,21 +327,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2015/11/27/release-0.10.1.html">Flink 0.10.1 released</a></h2>
-
- <p>27 Nov 2015
- </p>
-
- <p><p>Today, the Flink community released the first bugfix release of
the 0.10 series of Flink.</p>
-
-</p>
-
- <p><a href="/news/2015/11/27/release-0.10.1.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -359,6 +359,16 @@
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page8/index.html b/content/blog/page8/index.html
index d0c136e..a18fbca 100644
--- a/content/blog/page8/index.html
+++ b/content/blog/page8/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2015/11/27/release-0.10.1.html">Flink 0.10.1 released</a></h2>
+
+ <p>27 Nov 2015
+ </p>
+
+ <p><p>Today, the Flink community released the first bugfix release of
the 0.10 series of Flink.</p>
+
+</p>
+
+ <p><a href="/news/2015/11/27/release-0.10.1.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2015/11/16/release-0.10.0.html">Announcing Apache Flink
0.10.0</a></h2>
<p>16 Nov 2015
@@ -327,21 +342,6 @@ release is a preview release that contains known
issues.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2015/04/07/march-in-flink.html">March 2015 in the Flink
community</a></h2>
-
- <p>07 Apr 2015
- </p>
-
- <p><p>March has been a busy month in the Flink community.</p>
-
-</p>
-
- <p><a href="/news/2015/04/07/march-in-flink.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -374,6 +374,16 @@ release is a preview release that contains known
issues.</p>
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git a/content/blog/page9/index.html b/content/blog/page9/index.html
index ca574a3..8f30b43 100644
--- a/content/blog/page9/index.html
+++ b/content/blog/page9/index.html
@@ -185,6 +185,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a
href="/news/2015/04/07/march-in-flink.html">March 2015 in the Flink
community</a></h2>
+
+ <p>07 Apr 2015
+ </p>
+
+ <p><p>March has been a busy month in the Flink community.</p>
+
+</p>
+
+ <p><a href="/news/2015/04/07/march-in-flink.html">Continue reading
»</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a
href="/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html">Peeking
into Apache Flink's Engine Room</a></h2>
<p>13 Mar 2015 by Fabian Hüske (<a
href="https://twitter.com/";>@fhueske</a>)
@@ -324,21 +339,6 @@ and offers a new API including definition of flexible
windows.</p>
<hr>
- <article>
- <h2 class="blog-title"><a
href="/news/2014/09/26/release-0.6.1.html">Apache Flink 0.6.1 available</a></h2>
-
- <p>26 Sep 2014
- </p>
-
- <p><p>We are happy to announce the availability of Flink 0.6.1.</p>
-
-</p>
-
- <p><a href="/news/2014/09/26/release-0.6.1.html">Continue reading
»</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -371,6 +371,16 @@ and offers a new API including definition of flexible
windows.</p>
<ul id="markdown-toc">
+ <li><a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></li>
diff --git
a/content/img/blog/flink-pulsar-sql-blog-post-visual-primitive-avro-schema.png
b/content/img/blog/flink-pulsar-sql-blog-post-visual-primitive-avro-schema.png
new file mode 100644
index 0000000..3e3aa5e
Binary files /dev/null and
b/content/img/blog/flink-pulsar-sql-blog-post-visual-primitive-avro-schema.png
differ
diff --git a/content/img/blog/flink-pulsar-sql-blog-post-visual.png
b/content/img/blog/flink-pulsar-sql-blog-post-visual.png
new file mode 100644
index 0000000..ed9c21f
Binary files /dev/null and
b/content/img/blog/flink-pulsar-sql-blog-post-visual.png differ
diff --git a/content/index.html b/content/index.html
index c6a82ea..d4cdd27 100644
--- a/content/index.html
+++ b/content/index.html
@@ -551,6 +551,9 @@
<dl>
+ <dt> <a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></dt>
+ <dd>This blog post discusses the new developments and integrations
between Apache Flink and Apache Pulsar and showcases how you can leverage
Pulsar’s built-in schema to query Pulsar streams in real time using Apache
Flink.</dd>
+
<dt> <a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></dt>
<dd><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.9 series.</p>
@@ -566,12 +569,6 @@
<dt> <a href="/news/2019/09/10/community-update.html">Flink Community
Update - September'19</a></dt>
<dd>This has been an exciting, fast-paced year for the Apache Flink
community. But with over 10k messages across the mailing lists, 3k Jira tickets
and 2k pull requests, it is not easy to keep up with the latest state of the
project. Plus everything happening around it. With that in mind, we want to
bring back regular community updates to the Flink blog.</dd>
-
- <dt> <a href="/news/2019/08/22/release-1.9.0.html">Apache Flink 1.9.0
Release Announcement</a></dt>
- <dd><p>The Apache Flink community is proud to announce the release of
Apache Flink
-1.9.0.</p>
-
-</dd>
</dl>
diff --git
a/content/news/2019/11/25/query-pulsar-streams-using-apache-flink.html
b/content/news/2019/11/25/query-pulsar-streams-using-apache-flink.html
new file mode 100644
index 0000000..0579b39
--- /dev/null
+++ b/content/news/2019/11/25/query-pulsar-streams-using-apache-flink.html
@@ -0,0 +1,425 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head
content must come *after* these tags -->
+ <title>Apache Flink: How to query Pulsar Streams using Apache Flink</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css";>
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml"
title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics
event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media
queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script
src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js";></script>
+ <script
src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js";></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed"
data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg"
width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse"
id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream
Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache
Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/faq.html">FAQ</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a
href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/tutorials/local_setup.html";
target="_blank">Tutorials <small><span class="glyphicon
glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown"
href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a
href="https://ci.apache.org/projects/flink/flink-docs-release-1.9";
target="_blank">1.9 (Latest stable release) <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ <li><a
href="https://ci.apache.org/projects/flink/flink-docs-master";
target="_blank">Master (Latest Snapshot) <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li class="active"><a href="/blog/"><b>Flink Blog</b></a></li>
+
+ <!-- Ecosystem -->
+ <li><a href="/ecosystem.html">Ecosystem</a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors
-->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to
Contribute</a></li>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink"; target="_blank">Flink
on GitHub <small><span class="glyphicon
glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <!-- link to the Chinese home page when current is blog page
-->
+ <a href="/zh">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink";
target="_blank">@ApacheFlink <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/"
target="_blank">Plan Visualizer <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+
+ <hr />
+
+ <li><a href="https://apache.org"; target="_blank">Apache Software
Foundation <small><span class="glyphicon
glyphicon-new-window"></span></small></a></li>
+
+ <li>
+ <style>
+ .smalllinks:link {
+ display: inline-block !important; background: none;
padding-top: 0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px;
+ }
+ </style>
+
+ <a class="smalllinks" href="https://www.apache.org/licenses/";
target="_blank">License</a> <small><span class="glyphicon
glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/security/";
target="_blank">Security</a> <small><span class="glyphicon
glyphicon-new-window"></span></small>
+
+ <a class="smalllinks"
href="https://www.apache.org/foundation/sponsorship.html";
target="_blank">Donate</a> <small><span class="glyphicon
glyphicon-new-window"></span></small>
+
+ <a class="smalllinks"
href="https://www.apache.org/foundation/thanks.html"; target="_blank">Thanks</a>
<small><span class="glyphicon glyphicon-new-window"></span></small>
+ </li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <div class="row">
+ <h1>How to query Pulsar Streams using Apache Flink</h1>
+
+ <article>
+ <p>25 Nov 2019 Sijie Guo (<a
href="https://twitter.com/sijieg";>@sijieg</a>) & Markos Sfikas (<a
href="https://twitter.com/MarkSfik";>@MarkSfik</a>)</p>
+
+<p>In a previous <a
href="https://flink.apache.org/2019/05/03/pulsar-flink.html";>story</a> on the
Flink blog, we explained the different ways that <a
href="https://flink.apache.org/";>Apache Flink</a> and <a
href="https://pulsar.apache.org/";>Apache Pulsar</a> can integrate to provide
elastic data processing at large scale. This blog post discusses the new
developments and integrations between the two frameworks and showcases how you
can leverage Pulsar’s built-in schema to query Pulsar s [...]
+
+<h1 id="a-short-intro-to-apache-pulsar">A short intro to Apache Pulsar</h1>
+
+<p>Apache Pulsar is a flexible pub/sub messaging system, backed by durable log
storage. Some of the framework’s highlights include multi-tenancy, a unified
message model, structured event streams and a cloud-native architecture that
make it a perfect fit for a wide set of use cases, ranging from billing,
payments and trading services all the way to the unification of the different
messaging architectures in an organization. If you are interested in finding
out more about Pulsar, you can [...]
+
+<h1 id="existing-pulsar--flink-integration-apache-flink-16">Existing Pulsar
& Flink integration (Apache Flink 1.6+)</h1>
+
+<p>The existing integration between Pulsar and Flink exploits Pulsar as a
message queue in a Flink application. Flink developers can utilize Pulsar as a
streaming source and streaming sink for their Flink applications by selecting a
specific Pulsar source and connecting to their desired Pulsar cluster and
topic:</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="c1">//
create and configure Pulsar consumer</span>
+<span class="n">PulsarSourceBuilder</span><span class="o"><</span><span
class="n">String</span><span class="o">></span><span
class="n">builder</span> <span class="o">=</span> <span
class="n">PulsarSourceBuilder</span>
+ <span class="o">.</span><span class="na">builder</span><span
class="o">(</span><span class="k">new</span> <span
class="nf">SimpleStringSchema</span><span class="o">())</span>
+ <span class="o">.</span><span class="na">serviceUrl</span><span
class="o">(</span><span class="n">serviceUrl</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">topic</span><span
class="o">(</span><span class="n">inputTopic</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">subsciptionName</span><span
class="o">(</span><span class="n">subscription</span><span class="o">);</span>
+<span class="n">SourceFunction</span><span class="o"><</span><span
class="n">String</span><span class="o">></span> <span class="n">src</span>
<span class="o">=</span> <span class="n">builder</span><span
class="o">.</span><span class="na">build</span><span class="o">();</span>
+<span class="c1">// ingest DataStream with Pulsar consumer</span>
+<span class="n">DataStream</span><span class="o"><</span><span
class="n">String</span><span class="o">></span> <span class="n">words</span>
<span class="o">=</span> <span class="n">env</span><span
class="o">.</span><span class="na">addSource</span><span
class="o">(</span><span class="n">src</span><span
class="o">);</span></code></pre></div>
+
+<p>Pulsar streams can then get connected to the Flink processing logic…</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="c1">//
perform computation on DataStream (here a simple WordCount)</span>
+<span class="n">DataStream</span><span class="o"><</span><span
class="n">WordWithCount</span><span class="o">></span> <span
class="n">wc</span> <span class="o">=</span> <span class="n">words</span>
+ <span class="o">.</span><span class="na">flatmap</span><span
class="o">((</span><span class="n">FlatMapFunction</span><span
class="o"><</span><span class="n">String</span><span class="o">,</span>
<span class="n">WordWithCount</span><span class="o">>)</span> <span
class="o">(</span><span class="n">word</span><span class="o">,</span> <span
class="n">collector</span><span class="o">)</span> <span class="o">-></span>
<span class="o">{</span>
+ <span class="n">collector</span><span class="o">.</span><span
class="na">collect</span><span class="o">(</span><span class="k">new</span>
<span class="nf">WordWithCount</span><span class="o">(</span><span
class="n">word</span><span class="o">,</span> <span class="mi">1</span><span
class="o">));</span>
+ <span class="o">})</span>
+
+ <span class="o">.</span><span class="na">returns</span><span
class="o">(</span><span class="n">WordWithCount</span><span
class="o">.</span><span class="na">class</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">keyBy</span><span
class="o">(</span><span class="s">"word"</span><span
class="o">)</span>
+ <span class="o">.</span><span class="na">timeWindow</span><span
class="o">(</span><span class="n">Time</span><span class="o">.</span><span
class="na">seconds</span><span class="o">(</span><span class="mi">5</span><span
class="o">))</span>
+ <span class="o">.</span><span class="na">reduce</span><span
class="o">((</span><span class="n">ReduceFunction</span><span
class="o"><</span><span class="n">WordWithCount</span><span
class="o">>)</span> <span class="o">(</span><span class="n">c1</span><span
class="o">,</span> <span class="n">c2</span><span class="o">)</span> <span
class="o">-></span>
+ <span class="k">new</span> <span class="nf">WordWithCount</span><span
class="o">(</span><span class="n">c1</span><span class="o">.</span><span
class="na">word</span><span class="o">,</span> <span class="n">c1</span><span
class="o">.</span><span class="na">count</span> <span class="o">+</span> <span
class="n">c2</span><span class="o">.</span><span class="na">count</span><span
class="o">));</span></code></pre></div>
+
+<p>…and then get emitted back to Pulsar (used now as a sink), sending one’s
computation results downstream, back to a Pulsar topic:</p>
+
+<div class="highlight"><pre><code class="language-java"><span class="c1">//
emit result via Pulsar producer </span>
+<span class="n">wc</span><span class="o">.</span><span
class="na">addSink</span><span class="o">(</span><span class="k">new</span>
<span class="n">FlinkPulsarProducer</span><span class="o"><>(</span>
+ <span class="n">serviceUrl</span><span class="o">,</span>
+ <span class="n">outputTopic</span><span class="o">,</span>
+ <span class="k">new</span> <span
class="nf">AuthentificationDisabled</span><span class="o">(),</span>
+ <span class="n">wordWithCount</span> <span class="o">-></span> <span
class="n">wordWithCount</span><span class="o">.</span><span
class="na">toString</span><span class="o">().</span><span
class="na">getBytes</span><span class="o">(</span><span
class="n">UTF_8</span><span class="o">),</span>
+ <span class="n">wordWithCount</span> <span class="o">-></span> <span
class="n">wordWithCount</span><span class="o">.</span><span
class="na">word</span><span class="o">)</span>
+<span class="o">);</span></code></pre></div>
+
+<p>Although this is a great first integration step, the existing design is not
leveraging the full power of Pulsar. Some shortcomings of the integration with
Flink 1.6.0 relate to Pulsar neither being utilized as durable storage nor
having schema integration with Flink, resulting in manual input when describing
an application’s schema registry.</p>
+
+<h1
id="pulsars-integration-with-flink-19-using-pulsar-as-a-flink-catalog">Pulsar’s
integration with Flink 1.9: Using Pulsar as a Flink catalog</h1>
+
+<p>The latest integration between <a
href="https://flink.apache.org/downloads.html#apache-flink-191";>Flink 1.9.0</a>
and Pulsar addresses most of the previously mentioned shortcomings. The <a
href="https://flink.apache.org/news/2019/02/13/unified-batch-streaming-blink.html";>contribution
of Alibaba’s Blink to the Flink repository</a> adds many enhancements and new
features to the processing framework that make the integration with Pulsar
significantly more powerful and impactful. Flink 1. [...]
+
+<h1 id="leveraging-the-flink--pulsar-schema-integration">Leveraging the Flink
<> Pulsar Schema Integration</h1>
+
+<p>Before delving into the integration details and how you can use Pulsar
schema with Flink, let us describe how schema in Pulsar works. Schema in Apache
Pulsar already co-exists and serves as the representation of the data on the
broker side of the framework, something that makes schema registry with
external systems obsolete. Additionally, the data schema in Pulsar is
associated with each topic so both producers and consumers send data with
predefined schema information, while the brok [...]
+
+<p>Below you can find an example of Pulsar’s schema on both the producer and
consumer side. On the producer side, you can specify which schema you want to
use and Pulsar then sends a POJO class without the need to perform any
serialization/deserialization. Similarly, on the consumer end, you can also
specify the data schema and upon receiving the data, Pulsar will automatically
validate the schema information, fetch the schema of the given version and then
deserialize the data back to a [...]
+
+<div class="highlight"><pre><code class="language-java"><span class="c1">//
Create producer with Struct schema and send messages</span>
+<span class="n">Producer</span><span class="o"><</span><span
class="n">User</span><span class="o">></span> <span
class="n">producer</span> <span class="o">=</span> <span
class="n">client</span><span class="o">.</span><span
class="na">newProducer</span><span class="o">(</span><span
class="n">Schema</span><span class="o">.</span><span
class="na">AVRO</span><span class="o">(</span><span class="n">User</span><span
class="o">.</span><span class="na">class</span><span class="o">)).</span [...]
+<span class="n">producer</span><span class="o">.</span><span
class="na">newMessage</span><span class="o">()</span>
+ <span class="o">.</span><span class="na">value</span><span
class="o">(</span><span class="n">User</span><span class="o">.</span><span
class="na">builder</span><span class="o">()</span>
+ <span class="o">.</span><span class="na">userName</span><span
class="o">(</span><span class="err">“</span><span class="n">pulsar</span><span
class="o">-</span><span class="n">user</span><span class="err">”</span><span
class="o">)</span>
+ <span class="o">.</span><span class="na">userId</span><span
class="o">(</span><span class="mi">1L</span><span class="o">)</span>
+ <span class="o">.</span><span class="na">build</span><span
class="o">())</span>
+ <span class="o">.</span><span class="na">send</span><span
class="o">();</span></code></pre></div>
+
+<div class="highlight"><pre><code class="language-java"><span class="c1">//
Create consumer with Struct schema and receive messages</span>
+<span class="n">Consumer</span><span class="o"><</span><span
class="n">User</span><span class="o">></span> <span
class="n">consumer</span> <span class="o">=</span> <span
class="n">client</span><span class="o">.</span><span
class="na">newCOnsumer</span><span class="o">(</span><span
class="n">Schema</span><span class="o">.</span><span
class="na">AVRO</span><span class="o">(</span><span class="n">User</span><span
class="o">.</span><span class="na">class</span><span class="o">)).</span [...]
+<span class="n">consumer</span><span class="o">.</span><span
class="na">receive</span><span class="o">();</span></code></pre></div>
+
+<p>Let’s assume we have an application that specifies a schema to the producer
and/or consumer. Upon receiving the schema information, the producer (or
consumer) — that is connected to the broker — will transfer such information so
that the broker can then perform schema registration, validations and schema
compatibility checks before returning or rejecting the schema as illustrated in
the diagram below:</p>
+
+<center>
+<img src="/img/blog/flink-pulsar-sql-blog-post-visual.png" width="600px"
alt="Pulsar Schema" />
+</center>
+<p><br /></p>
+
+<p>Not only is Pulsar able to handle and store the schema information, but is
additionally able to handle any schema evolution — where necessary. Pulsar will
effectively manage any schema evolution in the broker, keeping track of all
different versions of your schema while performing any necessary compatibility
checks.</p>
+
+<p>Moreover, when messages are published on the producer side, Pulsar will tag
each message with the schema version as part of each message’s metadata. On the
consumer side, when the message is received and the metadata is deserialized,
Pulsar will check the schema version associated with this message and will
fetch the corresponding schema information from the broker. As a result, when
Pulsar integrates with a Flink application it uses the pre-existing schema
information and maps indivi [...]
+
+<p>For the cases when Flink users do not interact with schema directly or make
use of primitive schema (for example, using a topic to store a string or long
number), Pulsar will either convert the message payload into a Flink row,
called ‘value’ or — for the cases of structured schema types, like JSON and
AVRO — Pulsar will extract the individual fields from the schema information
and will map the fields to Flink’s type system. Finally, all metadata
information associated with each mess [...]
+
+<center>
+<img
src="/img/blog/flink-pulsar-sql-blog-post-visual-primitive-avro-schema.png"
width="600px" alt="Primitive and AVRO Schema" />
+</center>
+<p><br /></p>
+
+<p>Once all the schema information is mapped to Flink’s type system, you can
start building a Pulsar source, sink or catalog in Flink based on the specified
schema information as illustrated below:</p>
+
+<h1 id="flink--pulsar-read-data-from-pulsar">Flink & Pulsar: Read data
from Pulsar</h1>
+
+<ul>
+ <li>Create a Pulsar source for streaming queries</li>
+</ul>
+
+<div class="highlight"><pre><code class="language-java"><span
class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span
class="n">StreamExecutionEnvironment</span><span class="o">.</span><span
class="na">getExecutionEnvironment</span>
+<span class="n">val</span> <span class="n">props</span> <span
class="o">=</span> <span class="k">new</span> <span
class="nf">Properties</span><span class="o">()</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"service.url"</span><span class="o">,</span> <span
class="s">"pulsar://..."</span><span class="o">)</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"admin.url"</span><span class="o">,</span> <span
class="s">"http://..."</span><span class="o">)</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"partitionDiscoveryIntervalMillis"</span><span
class="o">,</span> <span class="s">"5000"</span><span
class="o">)</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"startingOffsets"</span><span class="o">,</span> <span
class="s">"earliest"</span><span class="o">)</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"topic"</span><span class="o">,</span> <span
class="s">"test-source-topic"</span><span class="o">)</span>
+<span class="n">val</span> <span class="n">source</span> <span
class="o">=</span> <span class="k">new</span> <span
class="nf">FlinkPulsarSource</span><span class="o">(</span><span
class="n">props</span><span class="o">)</span>
+<span class="c1">// you don't need to provide a type information to
addSource since FlinkPulsarSource is ResultTypeQueryable</span>
+<span class="n">val</span> <span class="n">dataStream</span> <span
class="o">=</span> <span class="n">env</span><span class="o">.</span><span
class="na">addSource</span><span class="o">(</span><span
class="n">source</span><span class="o">)(</span><span
class="kc">null</span><span class="o">)</span>
+
+<span class="c1">// chain operations on dataStream of Row and sink the
output</span>
+<span class="c1">// end method chaining</span>
+
+<span class="n">env</span><span class="o">.</span><span
class="na">execute</span><span class="o">()</span></code></pre></div>
+
+<ul>
+ <li>Register topics in Pulsar as streaming tables</li>
+</ul>
+
+<div class="highlight"><pre><code class="language-java"><span
class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span
class="n">StreamExecutionEnvironment</span><span class="o">.</span><span
class="na">getExecutionEnvironment</span>
+<span class="n">val</span> <span class="n">tEnv</span> <span
class="o">=</span> <span class="n">StreamTableEnvironment</span><span
class="o">.</span><span class="na">create</span><span class="o">(</span><span
class="n">env</span><span class="o">)</span>
+
+<span class="n">val</span> <span class="n">prop</span> <span
class="o">=</span> <span class="k">new</span> <span
class="nf">Properties</span><span class="o">()</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"service.url"</span><span class="o">,</span> <span
class="n">serviceUrl</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"admin.url"</span><span class="o">,</span> <span
class="n">adminUrl</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"flushOnCheckpoint"</span><span class="o">,</span> <span
class="s">"true"</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"failOnWrite"</span><span class="o">,</span> <span
class="s">"true"</span><span class="o">)</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"topic"</span><span class="o">,</span> <span
class="s">"test-sink-topic"</span><span class="o">)</span>
+
+<span class="n">tEnv</span>
+ <span class="o">.</span><span class="na">connect</span><span
class="o">(</span><span class="k">new</span> <span
class="nf">Pulsar</span><span class="o">().</span><span
class="na">properties</span><span class="o">(</span><span
class="n">props</span><span class="o">))</span>
+ <span class="o">.</span><span class="na">inAppendMode</span><span
class="o">()</span>
+ <span class="o">.</span><span class="na">registerTableSource</span><span
class="o">(</span><span class="s">"sink-table"</span><span
class="o">)</span>
+
+<span class="n">val</span> <span class="n">sql</span> <span class="o">=</span>
<span class="s">"INSERT INTO sink-table ....."</span>
+<span class="n">tEnv</span><span class="o">.</span><span
class="na">sqlUpdate</span><span class="o">(</span><span
class="n">sql</span><span class="o">)</span>
+<span class="n">env</span><span class="o">.</span><span
class="na">execute</span><span class="o">()</span></code></pre></div>
+
+<h1 id="flink--pulsar-write-data-to-pulsar">Flink & Pulsar: Write data to
Pulsar</h1>
+
+<ul>
+ <li>Create a Pulsar sink for streaming queries</li>
+</ul>
+
+<div class="highlight"><pre><code class="language-java"><span
class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span
class="n">StreamExecutionEnvironment</span><span class="o">.</span><span
class="na">getExecutionEnvironment</span>
+<span class="n">val</span> <span class="n">stream</span> <span
class="o">=</span> <span class="o">.....</span>
+
+<span class="n">val</span> <span class="n">prop</span> <span
class="o">=</span> <span class="k">new</span> <span
class="nf">Properties</span><span class="o">()</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"service.url"</span><span class="o">,</span> <span
class="n">serviceUrl</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"admin.url"</span><span class="o">,</span> <span
class="n">adminUrl</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"flushOnCheckpoint"</span><span class="o">,</span> <span
class="s">"true"</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"failOnWrite"</span><span class="o">,</span> <span
class="s">"true"</span><span class="o">)</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"topic"</span><span class="o">,</span> <span
class="s">"test-sink-topic"</span><span class="o">)</span>
+
+<span class="n">stream</span><span class="o">.</span><span
class="na">addSink</span><span class="o">(</span><span class="k">new</span>
<span class="nf">FlinkPulsarSink</span><span class="o">(</span><span
class="n">prop</span><span class="o">,</span> <span
class="n">DummyTopicKeyExtractor</span><span class="o">))</span>
+<span class="n">env</span><span class="o">.</span><span
class="na">execute</span><span class="o">()</span></code></pre></div>
+
+<ul>
+ <li>Write a streaming table to Pulsar</li>
+</ul>
+
+<div class="highlight"><pre><code class="language-java"><span
class="n">val</span> <span class="n">env</span> <span class="o">=</span> <span
class="n">StreamExecutionEnvironment</span><span class="o">.</span><span
class="na">getExecutionEnvironment</span>
+<span class="n">val</span> <span class="n">tEnv</span> <span
class="o">=</span> <span class="n">StreamTableEnvironment</span><span
class="o">.</span><span class="na">create</span><span class="o">(</span><span
class="n">env</span><span class="o">)</span>
+
+<span class="n">val</span> <span class="n">prop</span> <span
class="o">=</span> <span class="k">new</span> <span
class="nf">Properties</span><span class="o">()</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"service.url"</span><span class="o">,</span> <span
class="n">serviceUrl</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"admin.url"</span><span class="o">,</span> <span
class="n">adminUrl</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"flushOnCheckpoint"</span><span class="o">,</span> <span
class="s">"true"</span><span class="o">)</span>
+<span class="n">prop</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"failOnWrite"</span><span class="o">,</span> <span
class="s">"true"</span><span class="o">)</span>
+<span class="n">props</span><span class="o">.</span><span
class="na">setProperty</span><span class="o">(</span><span
class="s">"topic"</span><span class="o">,</span> <span
class="s">"test-sink-topic"</span><span class="o">)</span>
+
+<span class="n">tEnv</span>
+ <span class="o">.</span><span class="na">connect</span><span
class="o">(</span><span class="k">new</span> <span
class="nf">Pulsar</span><span class="o">().</span><span
class="na">properties</span><span class="o">(</span><span
class="n">props</span><span class="o">))</span>
+ <span class="o">.</span><span class="na">inAppendMode</span><span
class="o">()</span>
+ <span class="o">.</span><span class="na">registerTableSource</span><span
class="o">(</span><span class="s">"sink-table"</span><span
class="o">)</span>
+
+<span class="n">val</span> <span class="n">sql</span> <span class="o">=</span>
<span class="s">"INSERT INTO sink-table ....."</span>
+<span class="n">tEnv</span><span class="o">.</span><span
class="na">sqlUpdate</span><span class="o">(</span><span
class="n">sql</span><span class="o">)</span>
+<span class="n">env</span><span class="o">.</span><span
class="na">execute</span><span class="o">()</span></code></pre></div>
+
+<p>In every instance, Flink developers only need to specify the properties of
how Flink will connect to a Pulsar cluster without worrying about any schema
registry, or serialization/deserialization actions and register the Pulsar
cluster as a source, sink or streaming table in Flink. Once all three elements
are put together, Pulsar can then be registered as a catalog in Flink,
something that drastically simplifies how you process and query data like, for
example, writing a program to que [...]
+
+<h1 id="next-steps--future-integration">Next Steps & Future
Integration</h1>
+
+<p>The goal of the integration between Pulsar and Flink is to simplify how
developers use the two frameworks to build a unified data processing stack. As
we progress from the classical Lamda architectures — where an online, speeding
layer is combined with an offline, batch layer to run data computations — Flink
and Pulsar present a great combination in providing a truly unified data
processing stack. We see Flink as a unified computation engine, handling both
online (streaming) and offli [...]
+
+<p>There is still a lot of ongoing work and effort from both communities in
getting the integration even better, such as a new source API (<a
href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface";>FLIP-27</a>)
that will allow the <a
href="http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Discussion-Flink-Pulsar-Connector-td22019.html";>contribution
of the Pulsar connectors to the Flink community</a> as well as a new
subscription type cal [...]
+
+<p>You can find a more detailed overview of the integration work between the
two communities in this <a href="https://youtu.be/3sBXXfgl5vs";>recording
video</a> from Flink Forward Europe 2019 or sign up to the <a
href="https://flink.apache.org/community.html#mailing-lists";>Flink dev mailing
list</a> for the latest contribution and integration efforts between Flink and
Pulsar.</p>
+
+ </article>
+ </div>
+
+ <div class="row">
+ <div id="disqus_thread"></div>
+ <script type="text/javascript">
+ /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE
* * */
+ var disqus_shortname = 'stratosphere-eu'; // required: replace example
with your forum shortname
+
+ /* * * DON'T EDIT BELOW THIS LINE * * */
+ (function() {
+ var dsq = document.createElement('script'); dsq.type =
'text/javascript'; dsq.async = true;
+ dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+ (document.getElementsByTagName('head')[0] ||
document.getElementsByTagName('body')[0]).appendChild(dsq);
+ })();
+ </script>
+ </div>
+ </div>
+</div>
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org";>The Apache
Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache
feather logo are either registered trademarks or trademarks of The Apache
Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a
href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as
needed -->
+ <script
src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js";></script>
+ <script
src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js";></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new
Date();a=s.createElement(o),
+
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/zh/index.html b/content/zh/index.html
index 7f731b8..8a49e36 100644
--- a/content/zh/index.html
+++ b/content/zh/index.html
@@ -536,6 +536,9 @@
<dl>
+ <dt> <a
href="/news/2019/11/25/query-pulsar-streams-using-apache-flink.html">How to
query Pulsar Streams using Apache Flink</a></dt>
+ <dd>This blog post discusses the new developments and integrations
between Apache Flink and Apache Pulsar and showcases how you can leverage
Pulsar’s built-in schema to query Pulsar streams in real time using Apache
Flink.</dd>
+
<dt> <a href="/news/2019/10/18/release-1.9.1.html">Apache Flink 1.9.1
Released</a></dt>
<dd><p>The Apache Flink community released the first bugfix version of
the Apache Flink 1.9 series.</p>
@@ -551,12 +554,6 @@
<dt> <a href="/news/2019/09/10/community-update.html">Flink Community
Update - September'19</a></dt>
<dd>This has been an exciting, fast-paced year for the Apache Flink
community. But with over 10k messages across the mailing lists, 3k Jira tickets
and 2k pull requests, it is not easy to keep up with the latest state of the
project. Plus everything happening around it. With that in mind, we want to
bring back regular community updates to the Flink blog.</dd>
-
- <dt> <a href="/news/2019/08/22/release-1.9.0.html">Apache Flink 1.9.0
Release Announcement</a></dt>
- <dd><p>The Apache Flink community is proud to announce the release of
Apache Flink
-1.9.0.</p>
-
-</dd>
</dl>
