This is an automated email from the ASF dual-hosted git repository.
jark pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit b2a204ecaee1fb2f6cda05dc22f3befc081717c6
Author: Jark Wu <j...@apache.org>
AuthorDate: Sun Aug 20 22:51:33 2023 +0800
Update Flink Roadmap
---
docs/content.zh/roadmap.md | 192 ++++---
docs/content/roadmap.md | 202 ++++----
docs/static/img/flink_feature_radar_4.svg | 739 +++++++++++++++++++++++++++
docs/static/img/flink_feature_radar_zh_4.svg | 739 +++++++++++++++++++++++++++
4 files changed, 1667 insertions(+), 205 deletions(-)
diff --git a/docs/content.zh/roadmap.md b/docs/content.zh/roadmap.md
index dcd623228..4ab70c50d 100644
--- a/docs/content.zh/roadmap.md
+++ b/docs/content.zh/roadmap.md
@@ -36,7 +36,7 @@ under the License.
路线图会不断更新。一旦达成共识，新的特性和工作都会添加到路线图中。
这里的共识是指这些特性和工作将来确定会发生，以及这些工作对于用户来说大致是什么样的。
-**Last Update:** 2022-11-14
+**Last Update:** 2023-08-XX
## 功能图谱
@@ -45,7 +45,7 @@ under the License.
。
<div class="row front-graphic">
- {{< img src="/img/flink_feature_radar_3.svg" width="700px" >}}
+ {{< img src="/img/flink_feature_radar_zh_4.svg" width="700px" >}}
</div>
#### 功能阶段
@@ -54,147 +54,143 @@ under the License.
- **Beta:** 您可以从中受益，但是您在使用之前应该仔细评估该功能。
- **Ready and Evolving:** 生产可用，但是请注意，将来在升级Flink时，可能需要对您的应用和设置进行一些调整。
- **Stable:** 可以在生产中稳定不受限制地使用。
-- **Reaching End-of-Life:** 仍然可以稳定使用，但请考虑替代方法。对于新的长期项目而言，不建议使用。
+- **Approaching End-of-Life:** 仍然可以稳定使用，但请考虑替代方法。对于新的长期项目而言，不建议使用。
- **Deprecated:** 不推荐使用，您需要开始寻找替代产品。
-### Unified Analytics: Where Batch and Streaming come Together; SQL and beyond.
-Flink is a streaming data system in its core, that executes "batch as a
special case of streaming".
-Efficient execution of batch jobs is powerful in its own right; but even more
so, batch processing
-capabilities (efficient processing of bounded streams) open the way for a
seamless unification of
-batch and streaming applications.
+## Scenarios We Focus On
-Unified streaming/batch up-levels the streaming data paradigm: It gives users
consistent semantics across
-their real-time and lag-time applications. Furthermore, streaming applications
often need to be complemented
-by batch (bounded stream) processing, for example when reprocessing data after
bugs or data quality issues,
-or when bootstrapping new applications. A unified API and system make this
much easier.
+### Batch / Streaming Unification and Mixing
-### A unified SQL Platform
+Flink is a streaming data system in its core, that executes “batch as a
special case of streaming”. Efficient execution of batch jobs is powerful in
its own right; but even more so, batch processing capabilities (efficient
processing of bounded streams) open the way for a seamless unification of batch
and streaming applications.
+Unified streaming/batch up-levels the streaming data paradigm: It gives users
consistent semantics across their real-time and lag-time applications.
Furthermore, streaming applications often need to be complemented by batch
(bounded stream) processing, for example when reprocessing data after bugs or
data quality issues, or when bootstrapping new applications. A unified API and
system make this much easier.
-The community has been building Flink to a powerful basis for a unified (batch
and streaming) SQL analytics
-platform, and is continuing to do so.
+Both DataStream API and SQL provide unified API to execute the same
application in different modes of batch and streaming. There have been some
efforts to make the unification much more seamless, such as unified Source API
([FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface))
and SinkV2 API
([FLIP-191](https://cwiki.apache.org/confluence/display/FLINK/FLIP-191%3A+Extend+unified+Sink+interface+to+support+small+file+compaction)).
Beyond unifica [...]
-SQL has very strong cross-batch-streaming semantics, allowing users to use the
same queries for ad-hoc analytics
-and as continuous queries. Flink already contains an efficient unified query
engine, and a wide set of
-integrations. With user feedback, those are continuously improved.
+- Dynamic checkpoint interval for processing bounded stream of historical data
and unbounded stream of incremental data
([FLIP-309](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=255069517)).
+- Event notification mechanism for the boundary of bounded part and unbounded
part in a stream. This can unlock many exciting features and improvements, such
as [FLINK-19830](https://issues.apache.org/jira/browse/FLINK-19830).
+- Bootstrap states using a batch job (bounded stream program) with a final
checkpoint, and continue processing with a streaming job (unbounded stream
program) from the checkpoint and state.
-**Going Beyond a SQL Stream/Batch Processing Engine**
+### Unified SQL Platform
-- To extend the capability of a pure stream processor and make Flink ready for
future use cases,
-
[FLIP-188](https://cwiki.apache.org/confluence/display/FLINK/FLIP-188%3A+Introduce+Built-in+Dynamic+Table+Storage)
- has been announced adding built in dynamic table storage.
-- The experience of updating Flink SQL based jobs has been rather cumbersome
as it could have
- lead to new job graphs making restoring from savepoints/checkpoints
impossible.
-
[FLIP-190](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191336489&src=contextnavpagetreemode)
- that already has been shipped as MVP is targeting this.
+The community has been building Flink to a powerful basis for a unified (batch
and streaming) SQL analytics platform and is continuing to do so.
-**Platform Infrastructure**
+SQL has very strong cross-batch-streaming semantics, allowing users to use the
same queries for ad-hoc analytics and as continuous queries. Flink already
contains an efficient unified query engine and a wide set of integrations. With
user feedback, those are continuously improved.
-- After
[FLIP-163](https://cwiki.apache.org/confluence/display/FLINK/FLIP-163%3A+SQL+Client+Improvements)
- the community is working again on a set of SQL Client usability improvements
-
([FLIP-189](https://cwiki.apache.org/confluence/display/FLINK/FLIP-189%3A+SQL+Client+Usability+Improvements))
- which is aiming at improving the user experience, when using the SQL client.
+#### Going Beyond a SQL Stream/Batch Processing Engine
-**Support for Common Languages, Formats, Catalogs**
+- The experience of updating Flink SQL based jobs has been rather cumbersome
as it could have led to new job graphs making restoring from
savepoints/checkpoints impossible.
[FLIP-190](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191336489)
which already has been shipped as MVP is targeting this.
+- To extend the capability of a stream-batch processing engine and make Flink
ready for the unified SQL platform, there is an ongoing effort to allow Flink
better manage data and metadata, including
[DELETE/UPDATE](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235838061),
[Call
Procedures](https://cwiki.apache.org/confluence/display/FLINK/FLIP-311%3A+Support+Call+Stored+Procedure),
[rich
DDLs](https://cwiki.apache.org/confluence/display/FLINK/FLIP-305%3A+Support+atomic
[...]
+- There are some initial discussions to support JSON data type for Flink SQL.
This can enable Flink SQL to better analyze semi-structured data and better
adapt to NoSQL databases.
-- With
[FLIP-216](https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A++Introduce+pluggable+dialect+and++decouple+Hive+connector)
- there's now the initiative to introduce pluggable dialects on the example of
the Hive connector.
- Including so many dependencies to make dialects work has lead to an overhead
for contributors
- and users.
+#### Platform Infrastructure
-Flink has a broad SQL coverage for batch (full TPC-DS support) and a
state-of-the-art set of supported
-operations in streaming. There is continuous effort to add more functions and
cover more SQL operations.
+- After
[FLIP-163](https://cwiki.apache.org/confluence/display/FLINK/FLIP-163%3A+SQL+Client+Improvements)
the community is working again on a set of SQL Client usability improvements
([FLIP-189](https://cwiki.apache.org/confluence/display/FLINK/FLIP-189%3A+SQL+Client+Usability+Improvements),
[FLIP-222](https://cwiki.apache.org/confluence/display/FLINK/FLIP-222%3A+Support+full+job+lifecycle+statements+in+SQL+client))
which is aiming at improving the user experience when using the SQL client.
+- To simplify the building of production SQL platforms with Flink, we are
improving the [SQL
Gateway](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql-gateway/overview/)
component as the service of the Flink SQL platform. There are many ongoing
exciting features around it, including supporting application mode
([FLIP-316](https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Introduce+SQL+Driver)),
JDBC driver client ([FLIP-293](https://cwiki.apache.org/con [...]
-### Deep Batch / Streaming Unification for the DataStream API
+#### Support for Common Languages
-The *DataStream API* is Flink's *physical* API, for use cases where users need
very explicit control over data
-types, streams, state, and time. This API is evolving to support efficient
batch execution on bounded data.
+- Hive syntax compatibility can help users migrate existing Hive SQL tasks to
Flink seamlessly, and it is convenient for users who are familiar with Hive
syntax to use Hive syntax to write SQL to query tables registered in Flink.
Until now, the Hive syntax compatibility has reached 94.1% which is measured
using the Hive qtest suite. Flink community is continuously improving the
compatibility and execution performance
([FLINK-29717](https://issues.apache.org/jira/browse/FLINK-29717)).
+- With
[FLIP-216](https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A++Introduce+pluggable+dialect+and+plan+for+migrating+Hive+dialect)
there’s now the initiative to introduce pluggable SQL dialects on the example
of the Hive syntax. It makes Flink easier to support other SQL dialects in the
future, for example, Spark SQL and PostgreSQL.
-DataStream API executes the same dataflow shape in batch as in streaming,
keeping the same operators.
-That way users keep the same level of control over the dataflow, and our goal
is to mix and switch between
-batch/streaming execution in the future to make it a seamless experience.
+### Towards Streaming Warehouses
-**Unified Sources and Sinks**
+Flink has become the leading technology and factual standard for stream
processing. The concept of unifying streaming and batch data processing is
gaining recognition and is being successfully implemented in more and more
companies. To further unify streaming-batch analytics, Flink has proposed the
concept of Streaming Warehouse. This new concept aims to unify not only
computation but also storage, ensuring that data flows and is processed in real
time. As a result, the data in the wareh [...]
-- The first APIs and implementations of sources were specific to either
streaming programs in the DataStream API
-
([SourceFunction](https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/SourceFunction.java)),
- or to batch programs in the DataSet API
([InputFormat](https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/InputFormat.java)).
+The Apache Flink community initiated the Flink Table Store subproject
([FLIP-188](https://cwiki.apache.org/confluence/display/FLINK/FLIP-188%3A+Introduce+Built-in+Dynamic+Table+Storage))
with the vision of streaming-batch unified storage. With the project growing
rapidly, Flink Table Store [joined the Apache
incubator](https://lists.apache.org/thread/pz5f9cvpyk4q9vltd7z088q5368v412t) as
an independent project called [Apache
Paimon](https://github.com/apache/incubator-paimon/). Apache Pai [...]
- In this effort, we are creating sources that work across batch and streaming
execution. The aim is to give
- users a consistent experience across both modes, and to allow them to easily
switch between streaming and batch
- execution for their unbounded and bounded streaming applications.
- The interface for this New Source API is done and available, and we are
working on migrating more source connectors
- to this new model, see
[FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface).
+OLAP is an important scenario after Flink streaming-batch data processing,
users need an OLAP engine to analyze data in the streaming warehouse. Flink
could execute “OLAP as a special case of batch” and the community is trying to
explore the possibility of improvement for short-lived jobs without affecting
streaming and batch processing. It is a nice-to-have feature and it will bring
great value for users in Flink becoming a unified streaming-batch-OLAP data
processing system.
-- Similar to the sources, the original sink APIs are also specific to streaming
-
([SinkFunction](https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java))
- and batch
([OutputFormat](https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/OutputFormat.java))
- APIs and execution.
+In order to build an efficient streaming warehouse, there are a lot of things
that need to be improved in Flink, for example:
+- Support rich warehouse APIs to manage data and metadata, such as: CTAS/RTAS
([FLIP-303](https://cwiki.apache.org/confluence/display/FLINK/FLIP-303%3A+Support+REPLACE+TABLE+AS+SELECT+statement)),
CALL
([FLIP-311](https://cwiki.apache.org/confluence/display/FLINK/FLIP-311%3A+Support+Call+Stored+Procedure)),
TRUNCATE
([FLIP-302](https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement+in+batch+mode)),
and so on.
+- CBO (cost-based optimizations) with statistics in streaming lakehouses for
streaming queries.
+- Make full use of the layout and indexes on streaming lakehouse to reduce
data reading and processing for streaming queries.
+- Improvements for short-lived jobs to support OLAP queries with low latency
and concurrent execution.
- We have introduced a new API for sinks that consistently handles result
writing and committing (*Transactions*)
- across batch and streaming. The first iteration of the API exists, and we
are porting sinks and refining the
- API in the process. See
[FLIP-143](https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API).
-### Applications vs. Clusters; "Flink as a Library"
+## Engine Evolution
-The goal of these efforts is to make it feel natural to deploy (long running
streaming) Flink applications.
-Instead of starting a cluster and submitting a job to that cluster, these
efforts support deploying a streaming
-job as a self contained application.
+### Disaggregated State Management
+
+One major advantage of Flink is its efficient and easy-to-use state management
mechanism. However, this mechanism has evolved a little since it was born and
is not suitable in the cloud-native era. In the past several releases, we’ve
made significant efforts to improve the procedure of state snapshotting
(FLIP-76 [unaligned
checkpoint](https://flink.apache.org/2020/10/15/from-aligned-to-unaligned-checkpoints-part-1-checkpoints-alignment-and-backpressure/),
FLIP-158 [generic incremental c [...]
+
+In the new design, DFS is played as primary storage. Checkpoints are shareable
between operators so we do not need to compute and store multiple copies of the
same state table. Queryable state APIs can be provided based on these
checkpoints. Compaction and clean-up of state files are not bounded to the same
Task manager anymore so we can do better load-balancing and avoid burst CPU and
network peaks.
+
+### Evolution of Flink APIs
+
+With Flink 2.0 approaching, the community is planning to evolve the APIs of
Apache Flink.
+- We are planning to remove some long deprecated APIs in Flink 2.0, to make
Flink move faster, including:
+ - The DataSet API, all Scala APIs, the legacy SinkV1 API, the legacy
TableSource/TableSink API
+ - Deprecated methods / fields / classes in the DataStream API, Table API and
REST API
+ - Deprecated configuration options and metrics
+- We are also planning to retire the legacy SourceFunction / SinkFunction
APIs, and Queryable State API in the long term. This may not happen shortly, as
the prerequisites for users to migrate from these APIs are not fully met at the
moment.
+- We are aware of some problems of the current DataStream API, such as the
exposing of and dependencies on the Flink internal implementations, which
requires significant changes to fix. To provide a smooth migration experience,
the community is designing a new ProcessFunction API, which aims to gradually
replace the DataStream API in the long term.
+
+### Flink as an Application
+
+The goal of these efforts is to make it feel natural to deploy (long-running
streaming) Flink applications. Instead of starting a cluster and submitting a
job to that cluster, these efforts support deploying a streaming job as a
self-contained application.
For example as a simple Kubernetes deployment; deployed and scaled like a
regular application without extra workflows.
+- There is currently a Flink Kubernetes Operator subproject being developed by
the community and has its own roadmap under the
[documentation](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/development/roadmap/).
+- Streaming query as an application. Make SQL Client/Gateway supports
submitting SQL jobs in the application mode
([FLIP-316](https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Introduce+SQL+Driver#FLIP316:IntroduceSQLDriver-Motivation)).
-- There is currently a Kubernetes Operator being developed by the community.
See
-
[FLIP-212](https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator).
### Performance
-Continuous work to keep improving performance and recovery speed.
+Continuous work to keep improving the performance of both Flink streaming and
batch processing.
+
+#### Large-Scale Streaming Jobs
+
+- Streaming Join is a headache for Flink users because of its large-scale
state. The community is putting lots of effort into further improving the
performance of streaming join, such as minibatch join, multi-way join, and
reducing duplicated states.
+- The community is also continuously improving and working on some other
joins, such as unordered async lookup join and processing-time temporal join
([FLIP-326](https://cwiki.apache.org/confluence/display/FLINK/FLIP-326%3A+Enhance+Watermark+to+Support+Processing-Time+Temporal+Join)).
They can be very efficient alternatives for streaming joins.
+- Change data capture and processing with Flink SQL is widely used, and the
community is improving cost and performance in this case, e.g. reducing
normalize and materialize state.
+
+#### Faster Batch Queries
-### Faster Checkpoints and Recovery
+The community's goal is to make Flink's performance on bounded streams (batch
use cases) competitive with that of dedicated batch processors. While Flink has
been shown to handle some batch processing use cases faster than widely-used
batch processors, there are some ongoing efforts to make sure this is the case
for broader use cases:
+The community has introduced Dynamic Partition Pruning
([DPP](https://cwiki.apache.org/confluence/display/FLINK/FLIP-248%3A+Introduce+dynamic+partition+pruning))
which aims to minimize I/O costs of the data read from the data sources. There
are some ongoing efforts to further reduce the I/O and shuffle costs, such as
Runtime Filter
([FLIP-324](https://cwiki.apache.org/confluence/display/FLINK/FLIP-324%3A+Introduce+Runtime+Filter+for+Flink+Batch+Jobs)).
+Operator Fusion CodeGen
([FLIP-315](https://cwiki.apache.org/confluence/display/FLINK/FLIP-315+Support+Operator+Fusion+Codegen+for+Flink+SQL))
improves the execution performance of a query by fusing an operator DAG into a
single optimized operator that eliminates virtual function calls and leverages
CPU registers for intermediate data.
+The community has supported some adaptive batch execution and scheduling
([FLIP-187](https://cwiki.apache.org/confluence/display/FLINK/FLIP-187%3A+Adaptive+Batch+Scheduler)).
We are trying to support broader adaptive cases, such as Adaptive Query
Execution that makes use of runtime statistics to choose the most efficient
query execution plan.
+The community has started improving scheduler and execution performance
([FLINK-25318](https://issues.apache.org/jira/browse/FLINK-25318)) for
short-lived jobs to support OLAP. Flink executes “OLAP as a special case of
batch”, we are trying to extend Flink to execute low-latency and currency
queries in Session Cluster and users can perform streaming, batch, and OLAP
data processing on the unified Flink engine.
-The community is continuously working on improving checkpointing and recovery
speed.
-Checkpoints and recovery are stable and have been a reliable workhorse for
years. We are still
-trying to make it faster, more predictable, and to remove some confusions and
inflexibility in some areas.
+### Stability
--
[FLIP-183](https://cwiki.apache.org/confluence/display/FLINK/FLIP-183%3A+Dynamic+buffer+size+adjustment)
- is targeting size of checkpoints by debloating the buffers. A first beta is
already available.
-- With
[FLIP-151](https://cwiki.apache.org/confluence/display/FLINK/FLIP-151%3A+Incremental+snapshots+for+heap-based+state+backend)
- there is an ongoing effort to implement a heap based state backend.
+The community is keeping improving the stability of jobs, by better tolerating
failures, and speeding up the recovery process.
-### Apache Flink as part of an ever evolving data ecosystem
+The instability of the environment is unavoidable. It can lead to a crash of
JobManager and TaskManager nodes, or slow data processing. The community has
introduced speculative execution
([FLIP-168](https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job),
[FLIP-245](https://cwiki.apache.org/confluence/display/FLINK/FLIP-245%3A+Source+Supports+Speculative+Execution+For+Batch+Job),
[FLIP-281](https://cwiki.apache.org/confluence/display/FLINK/FLIP- [...]
-There is almost no use case in which Apache Flink is used on its own. It has
established itself
-as part of many data related reference architectures. In fact you'll find the
squirrel logo covering
-several aspects.
+JobManager node crash is usually unacceptable for a batch job because the job
has to be re-run from the very beginning. Therefore, the community is planning
to improve the JobManager recovery process to avoid re-run finished stages.
Another planned improvement is to retain running tasks when the JobManager node
goes down unexpectedly, to further reduce the impact of the JobManager crash.
This can also benefit streaming jobs even if they have periodical
checkpointing, to avoid interruptio [...]
-The community has added a lot of connectors and formats. With the already
mentioned
-[FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface)
and
-[FLIP-143](https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API)
-a new default for connectors has been established.
+### Usability
-- There are efforts to revise the formats API with
-
[FLIP-219](https://cwiki.apache.org/confluence/display/FLINK/FLIP-219%3A+Revised+Format+API)
-- There is ongoing work on new connectors
- (e.g.
[Pinot](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177045634))
-- Connectors will be hosted in an external repository going forward. See the
- [ML thread](https://lists.apache.org/thread/8k1xonqt7hn0xldbky1cxfx3fzh6sj7h)
+Now and then we hear people say that, while Flink is powerful in
functionality, it is not that easy to master. Such voices are heard. The
community is working on several efforts to improve the usability of Flink.
+
+We are working on reducing the number of configuration options that users need
to specify, as well as making them easier to understand and tune. This includes:
+Removing options that require in-depth knowledge of Flink internals to
understand and use.
+Making Flink automatically and dynamically decide the proper behavior where
possible.
+Improving the default values of the options so that users need not to touch
them in most cases.
+Improving the definition and description of the options so that they are
easier to understand and work with when it’s necessary.
+
+We have already made some progress along this direction. Flink 1.17 requires
less than 10 configurations to achieve well enough performance on TPC-DS.
Hybrid shuffle supports dynamically switching between different shuffle modes
and decouples its memory footprint from the parallelism of the job.
+
+
+## Developer Experience
+
+### Ecosystem
+There is almost no use case in which Apache Flink is used on its own. It has
established itself as part of many data related reference architectures. In
fact you’ll find the squirrel logo covering several aspects.
+
+All connectors will be hosted in an external repository going forward and many
of them have been successfully externalized. See the [mailing list
thread](https://lists.apache.org/thread/8k1xonqt7hn0xldbky1cxfx3fzh6sj7h).
+Catalog as a first-class citizen. Flink catalog lets users issue batch and
streaming queries connecting to external systems without registering
DDLs/schemas manually. It is recommended to support Catalog in the highest
priority for connectors. The community is working on supporting more catalogs
for connectors (e.g.
[GlueCatalog](https://cwiki.apache.org/confluence/display/FLINK/FLIP-277%3A+Native+GlueCatalog+Support+in+Flink),
[SchemaRegistryCatalog](https://cwiki.apache.org/confluence/ [...]
+There is ongoing work on introducing more new connectors (e.g.
[Pinot](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177045634),
[Redshift](https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift),
[ClickHouse](https://cwiki.apache.org/confluence/display/FLINK/FLIP-202%3A+Introduce+ClickHouse+Connector))
-The Flink community has removed Gelly, it's old graph-processing library.
### Documentation
-There are various dedicated efforts to simplify the maintenance and structure
(more intuitive navigation/reading)
-of the documentation.
+There are various dedicated efforts to simplify the maintenance and structure
(more intuitive navigation/reading) of the documentation.
- Docs Tech Stack:
[FLIP-157](https://cwiki.apache.org/confluence/display/FLINK/FLIP-157+Migrate+Flink+Documentation+from+Jekyll+to+Hugo)
- General Docs Structure:
[FLIP-42](https://cwiki.apache.org/confluence/display/FLINK/FLIP-42%3A+Rework+Flink+Documentation)
- SQL Docs:
[FLIP-60](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685)
-### Flink Kubernetes Operator
-
-Flink Kubernetes Operator
项目拥有自己的[路线图](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/development/roadmap/)，您可以在它的说明文档下面看到社区即将开展的工作。
-
-### Flink Table Store
-The Flink Table Store subproject has its own roadmap under the
[documentation](https://nightlies.apache.org/flink/flink-table-store-docs-master/docs/development/roadmap/).
diff --git a/docs/content/roadmap.md b/docs/content/roadmap.md
index 7fddf534e..4ecabd7de 100644
--- a/docs/content/roadmap.md
+++ b/docs/content/roadmap.md
@@ -25,19 +25,14 @@ under the License.
# Roadmap
-**Preamble:** This roadmap means to provide user and contributors with a
high-level summary of ongoing efforts,
-grouped by the major threads to which the efforts belong. With so much that is
happening in Flink, we
-hope that this helps with understanding the direction of the project.
-The roadmap contains both efforts in early stages as well as nearly completed
-efforts, so that users may get a better impression of the overall status and
direction of those developments.
+**Preamble:** This roadmap means to provide users and contributors with a
high-level summary of ongoing efforts, grouped by the major threads to which
the efforts belong. With so much that is happening in Flink, we hope that this
helps with understanding the direction of the project. The roadmap contains
both efforts in early stages as well as nearly completed efforts, so that users
may get a better impression of the overall status and direction of those
developments.
More details and various smaller changes can be found in the
[FLIPs](https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals)
-The roadmap is continuously updated. New features and efforts should be added
to the roadmap once
-there is consensus that they will happen and what they will roughly look like
for the user.
+The roadmap is continuously updated. New features and efforts should be added
to the roadmap once there is consensus that they will happen and what they will
roughly look like for the user.
-**Last Update:** 2022-11-14
+**Last Update:** 2023-08-XX
## Feature Radar
@@ -46,7 +41,7 @@ are approaching end-of-life. For questions, please contact
the developer mailing
[d...@flink.apache.org](mailto:d...@flink.apache.org)
<div class="row front-graphic">
- {{< img src="/img/flink_feature_radar_3.svg" width="700px" >}}
+ {{< img src="/img/flink_feature_radar_4.svg" width="700px" >}}
</div>
#### Feature Stages
@@ -55,149 +50,142 @@ are approaching end-of-life. For questions, please
contact the developer mailing
- **Beta:** You can benefit from this, but you should carefully evaluate the
feature.
- **Ready and Evolving:** Ready to use in production, but be aware you may
need to make some adjustments to your application and setup in the future, when
you upgrade Flink.
- **Stable:** Unrestricted use in production
-- **Reaching End-of-Life:** Stable, still feel free to use, but think about
alternatives. Not a good match for new long-lived projects.
+- **Approaching End-of-Life:** Stable, still feel free to use, but think about
alternatives. Not a good match for new long-lived projects.
- **Deprecated:** Start looking for alternatives now
-### Unified Analytics: Where Batch and Streaming come Together; SQL and beyond.
+## Scenarios We Focus On
-Flink is a streaming data system in its core, that executes "batch as a
special case of streaming".
-Efficient execution of batch jobs is powerful in its own right; but even more
so, batch processing
-capabilities (efficient processing of bounded streams) open the way for a
seamless unification of
-batch and streaming applications.
+### Batch / Streaming Unification and Mixing
-Unified streaming/batch up-levels the streaming data paradigm: It gives users
consistent semantics across
-their real-time and lag-time applications. Furthermore, streaming applications
often need to be complemented
-by batch (bounded stream) processing, for example when reprocessing data after
bugs or data quality issues,
-or when bootstrapping new applications. A unified API and system make this
much easier.
+Flink is a streaming data system in its core, that executes “batch as a
special case of streaming”. Efficient execution of batch jobs is powerful in
its own right; but even more so, batch processing capabilities (efficient
processing of bounded streams) open the way for a seamless unification of batch
and streaming applications.
+Unified streaming/batch up-levels the streaming data paradigm: It gives users
consistent semantics across their real-time and lag-time applications.
Furthermore, streaming applications often need to be complemented by batch
(bounded stream) processing, for example when reprocessing data after bugs or
data quality issues, or when bootstrapping new applications. A unified API and
system make this much easier.
-### A unified SQL Platform
+Both DataStream API and SQL provide unified API to execute the same
application in different modes of batch and streaming. There have been some
efforts to make the unification much more seamless, such as unified Source API
([FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface))
and SinkV2 API
([FLIP-191](https://cwiki.apache.org/confluence/display/FLINK/FLIP-191%3A+Extend+unified+Sink+interface+to+support+small+file+compaction)).
Beyond unifica [...]
-The community has been building Flink to a powerful basis for a unified (batch
and streaming) SQL analytics
-platform, and is continuing to do so.
+- Dynamic checkpoint interval for processing bounded stream of historical data
and unbounded stream of incremental data
([FLIP-309](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=255069517)).
+- Event notification mechanism for the boundary of bounded part and unbounded
part in a stream. This can unlock many exciting features and improvements, such
as [FLINK-19830](https://issues.apache.org/jira/browse/FLINK-19830).
+- Bootstrap states using a batch job (bounded stream program) with a final
checkpoint, and continue processing with a streaming job (unbounded stream
program) from the checkpoint and state.
-SQL has very strong cross-batch-streaming semantics, allowing users to use the
same queries for ad-hoc analytics
-and as continuous queries. Flink already contains an efficient unified query
engine, and a wide set of
-integrations. With user feedback, those are continuously improved.
+### Unified SQL Platform
-**Going Beyond a SQL Stream/Batch Processing Engine**
+The community has been building Flink to a powerful basis for a unified (batch
and streaming) SQL analytics platform and is continuing to do so.
-- To extend the capability of a pure stream processor and make Flink ready for
future use cases,
-
[FLIP-188](https://cwiki.apache.org/confluence/display/FLINK/FLIP-188%3A+Introduce+Built-in+Dynamic+Table+Storage)
- has been announced adding built in dynamic table storage.
-- The experience of updating Flink SQL based jobs has been rather cumbersome
as it could have
- lead to new job graphs making restoring from savepoints/checkpoints
impossible.
-
[FLIP-190](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191336489&src=contextnavpagetreemode)
- that already has been shipped as MVP is targeting this.
+SQL has very strong cross-batch-streaming semantics, allowing users to use the
same queries for ad-hoc analytics and as continuous queries. Flink already
contains an efficient unified query engine and a wide set of integrations. With
user feedback, those are continuously improved.
-**Platform Infrastructure**
+#### Going Beyond a SQL Stream/Batch Processing Engine
-- After
[FLIP-163](https://cwiki.apache.org/confluence/display/FLINK/FLIP-163%3A+SQL+Client+Improvements)
- the community is working again on a set of SQL Client usability improvements
-
([FLIP-189](https://cwiki.apache.org/confluence/display/FLINK/FLIP-189%3A+SQL+Client+Usability+Improvements))
- which is aiming at improving the user experience, when using the SQL client.
+- The experience of updating Flink SQL based jobs has been rather cumbersome
as it could have led to new job graphs making restoring from
savepoints/checkpoints impossible.
[FLIP-190](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191336489)
which already has been shipped as MVP is targeting this.
+- To extend the capability of a stream-batch processing engine and make Flink
ready for the unified SQL platform, there is an ongoing effort to allow Flink
better manage data and metadata, including
[DELETE/UPDATE](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235838061),
[Call
Procedures](https://cwiki.apache.org/confluence/display/FLINK/FLIP-311%3A+Support+Call+Stored+Procedure),
[rich
DDLs](https://cwiki.apache.org/confluence/display/FLINK/FLIP-305%3A+Support+atomic
[...]
+- There are some initial discussions to support JSON data type for Flink SQL.
This can enable Flink SQL to better analyze semi-structured data and better
adapt to NoSQL databases.
-**Support for Common Languages, Formats, Catalogs**
+#### Platform Infrastructure
-- With
[FLIP-216](https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A++Introduce+pluggable+dialect+and++decouple+Hive+connector)
- there's now the initiative to introduce pluggable dialects on the example of
the Hive connector.
- Including so many dependencies to make dialects work has lead to an overhead
for contributors
- and users.
+- After
[FLIP-163](https://cwiki.apache.org/confluence/display/FLINK/FLIP-163%3A+SQL+Client+Improvements)
the community is working again on a set of SQL Client usability improvements
([FLIP-189](https://cwiki.apache.org/confluence/display/FLINK/FLIP-189%3A+SQL+Client+Usability+Improvements),
[FLIP-222](https://cwiki.apache.org/confluence/display/FLINK/FLIP-222%3A+Support+full+job+lifecycle+statements+in+SQL+client))
which is aiming at improving the user experience when using the SQL client.
+- To simplify the building of production SQL platforms with Flink, we are
improving the [SQL
Gateway](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql-gateway/overview/)
component as the service of the Flink SQL platform. There are many ongoing
exciting features around it, including supporting application mode
([FLIP-316](https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Introduce+SQL+Driver)),
JDBC driver client ([FLIP-293](https://cwiki.apache.org/con [...]
-Flink has a broad SQL coverage for batch (full TPC-DS support) and a
state-of-the-art set of supported
-operations in streaming. There is continuous effort to add more functions and
cover more SQL operations.
+#### Support for Common Languages
-### Deep Batch / Streaming Unification for the DataStream API
+- Hive syntax compatibility can help users migrate existing Hive SQL tasks to
Flink seamlessly, and it is convenient for users who are familiar with Hive
syntax to use Hive syntax to write SQL to query tables registered in Flink.
Until now, the Hive syntax compatibility has reached 94.1% which is measured
using the Hive qtest suite. Flink community is continuously improving the
compatibility and execution performance
([FLINK-29717](https://issues.apache.org/jira/browse/FLINK-29717)).
+- With
[FLIP-216](https://cwiki.apache.org/confluence/display/FLINK/FLIP-216%3A++Introduce+pluggable+dialect+and+plan+for+migrating+Hive+dialect)
there’s now the initiative to introduce pluggable SQL dialects on the example
of the Hive syntax. It makes Flink easier to support other SQL dialects in the
future, for example, Spark SQL and PostgreSQL.
-The *DataStream API* is Flink's *physical* API, for use cases where users need
very explicit control over data
-types, streams, state, and time. This API is evolving to support efficient
batch execution on bounded data.
+### Towards Streaming Warehouses
-DataStream API executes the same dataflow shape in batch as in streaming,
keeping the same operators.
-That way users keep the same level of control over the dataflow, and our goal
is to mix and switch between
-batch/streaming execution in the future to make it a seamless experience.
+Flink has become the leading technology and factual standard for stream
processing. The concept of unifying streaming and batch data processing is
gaining recognition and is being successfully implemented in more and more
companies. To further unify streaming-batch analytics, Flink has proposed the
concept of Streaming Warehouse. This new concept aims to unify not only
computation but also storage, ensuring that data flows and is processed in real
time. As a result, the data in the wareh [...]
-**Unified Sources and Sinks**
+The Apache Flink community initiated the Flink Table Store subproject
([FLIP-188](https://cwiki.apache.org/confluence/display/FLINK/FLIP-188%3A+Introduce+Built-in+Dynamic+Table+Storage))
with the vision of streaming-batch unified storage. With the project growing
rapidly, Flink Table Store [joined the Apache
incubator](https://lists.apache.org/thread/pz5f9cvpyk4q9vltd7z088q5368v412t) as
an independent project called [Apache
Paimon](https://github.com/apache/incubator-paimon/). Apache Pai [...]
-- The first APIs and implementations of sources were specific to either
streaming programs in the DataStream API
-
([SourceFunction](https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/SourceFunction.java)),
- or to batch programs in the DataSet API
([InputFormat](https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/InputFormat.java)).
+OLAP is an important scenario after Flink streaming-batch data processing,
users need an OLAP engine to analyze data in the streaming warehouse. Flink
could execute “OLAP as a special case of batch” and the community is trying to
explore the possibility of improvement for short-lived jobs without affecting
streaming and batch processing. It is a nice-to-have feature and it will bring
great value for users in Flink becoming a unified streaming-batch-OLAP data
processing system.
- In this effort, we are creating sources that work across batch and streaming
execution. The aim is to give
- users a consistent experience across both modes, and to allow them to easily
switch between streaming and batch
- execution for their unbounded and bounded streaming applications.
- The interface for this New Source API is done and available, and we are
working on migrating more source connectors
- to this new model, see
[FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface).
+In order to build an efficient streaming warehouse, there are a lot of things
that need to be improved in Flink, for example:
+- Support rich warehouse APIs to manage data and metadata, such as: CTAS/RTAS
([FLIP-303](https://cwiki.apache.org/confluence/display/FLINK/FLIP-303%3A+Support+REPLACE+TABLE+AS+SELECT+statement)),
CALL
([FLIP-311](https://cwiki.apache.org/confluence/display/FLINK/FLIP-311%3A+Support+Call+Stored+Procedure)),
TRUNCATE
([FLIP-302](https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement+in+batch+mode)),
and so on.
+- CBO (cost-based optimizations) with statistics in streaming lakehouses for
streaming queries.
+- Make full use of the layout and indexes on streaming lakehouse to reduce
data reading and processing for streaming queries.
+- Improvements for short-lived jobs to support OLAP queries with low latency
and concurrent execution.
-- Similar to the sources, the original sink APIs are also specific to streaming
-
([SinkFunction](https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java))
- and batch
([OutputFormat](https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/OutputFormat.java))
- APIs and execution.
- We have introduced a new API for sinks that consistently handles result
writing and committing (*Transactions*)
- across batch and streaming. The first iteration of the API exists, and we
are porting sinks and refining the
- API in the process. See
[FLIP-143](https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API).
+## Engine Evolution
-### Applications vs. Clusters; "Flink as a Library"
+### Disaggregated State Management
-The goal of these efforts is to make it feel natural to deploy (long running
streaming) Flink applications.
-Instead of starting a cluster and submitting a job to that cluster, these
efforts support deploying a streaming
-job as a self contained application.
+One major advantage of Flink is its efficient and easy-to-use state management
mechanism. However, this mechanism has evolved a little since it was born and
is not suitable in the cloud-native era. In the past several releases, we’ve
made significant efforts to improve the procedure of state snapshotting
(FLIP-76 [unaligned
checkpoint](https://flink.apache.org/2020/10/15/from-aligned-to-unaligned-checkpoints-part-1-checkpoints-alignment-and-backpressure/),
FLIP-158 [generic incremental c [...]
+
+In the new design, DFS is played as primary storage. Checkpoints are shareable
between operators so we do not need to compute and store multiple copies of the
same state table. Queryable state APIs can be provided based on these
checkpoints. Compaction and clean-up of state files are not bounded to the same
Task manager anymore so we can do better load-balancing and avoid burst CPU and
network peaks.
+
+### Evolution of Flink APIs
+
+With Flink 2.0 approaching, the community is planning to evolve the APIs of
Apache Flink.
+- We are planning to remove some long deprecated APIs in Flink 2.0, to make
Flink move faster, including:
+ - The DataSet API, all Scala APIs, the legacy SinkV1 API, the legacy
TableSource/TableSink API
+ - Deprecated methods / fields / classes in the DataStream API, Table API and
REST API
+ - Deprecated configuration options and metrics
+- We are also planning to retire the legacy SourceFunction / SinkFunction
APIs, and Queryable State API in the long term. This may not happen shortly, as
the prerequisites for users to migrate from these APIs are not fully met at the
moment.
+- We are aware of some problems of the current DataStream API, such as the
exposing of and dependencies on the Flink internal implementations, which
requires significant changes to fix. To provide a smooth migration experience,
the community is designing a new ProcessFunction API, which aims to gradually
replace the DataStream API in the long term.
+
+### Flink as an Application
+
+The goal of these efforts is to make it feel natural to deploy (long-running
streaming) Flink applications. Instead of starting a cluster and submitting a
job to that cluster, these efforts support deploying a streaming job as a
self-contained application.
For example as a simple Kubernetes deployment; deployed and scaled like a
regular application without extra workflows.
+- There is currently a Flink Kubernetes Operator subproject being developed by
the community and has its own roadmap under the
[documentation](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/development/roadmap/).
+- Streaming query as an application. Make SQL Client/Gateway supports
submitting SQL jobs in the application mode
([FLIP-316](https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Introduce+SQL+Driver#FLIP316:IntroduceSQLDriver-Motivation)).
-- There is currently a Kubernetes Operator being developed by the community.
See
-
[FLIP-212](https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator).
### Performance
-Continuous work to keep improving performance and recovery speed.
+Continuous work to keep improving the performance of both Flink streaming and
batch processing.
+
+#### Large-Scale Streaming Jobs
+
+- Streaming Join is a headache for Flink users because of its large-scale
state. The community is putting lots of effort into further improving the
performance of streaming join, such as minibatch join, multi-way join, and
reducing duplicated states.
+- The community is also continuously improving and working on some other
joins, such as unordered async lookup join and processing-time temporal join
([FLIP-326](https://cwiki.apache.org/confluence/display/FLINK/FLIP-326%3A+Enhance+Watermark+to+Support+Processing-Time+Temporal+Join)).
They can be very efficient alternatives for streaming joins.
+- Change data capture and processing with Flink SQL is widely used, and the
community is improving cost and performance in this case, e.g. reducing
normalize and materialize state.
+
+#### Faster Batch Queries
+
+The community's goal is to make Flink's performance on bounded streams (batch
use cases) competitive with that of dedicated batch processors. While Flink has
been shown to handle some batch processing use cases faster than widely-used
batch processors, there are some ongoing efforts to make sure this is the case
for broader use cases:
+The community has introduced Dynamic Partition Pruning
([DPP](https://cwiki.apache.org/confluence/display/FLINK/FLIP-248%3A+Introduce+dynamic+partition+pruning))
which aims to minimize I/O costs of the data read from the data sources. There
are some ongoing efforts to further reduce the I/O and shuffle costs, such as
Runtime Filter
([FLIP-324](https://cwiki.apache.org/confluence/display/FLINK/FLIP-324%3A+Introduce+Runtime+Filter+for+Flink+Batch+Jobs)).
+Operator Fusion CodeGen
([FLIP-315](https://cwiki.apache.org/confluence/display/FLINK/FLIP-315+Support+Operator+Fusion+Codegen+for+Flink+SQL))
improves the execution performance of a query by fusing an operator DAG into a
single optimized operator that eliminates virtual function calls and leverages
CPU registers for intermediate data.
+The community has supported some adaptive batch execution and scheduling
([FLIP-187](https://cwiki.apache.org/confluence/display/FLINK/FLIP-187%3A+Adaptive+Batch+Scheduler)).
We are trying to support broader adaptive cases, such as Adaptive Query
Execution that makes use of runtime statistics to choose the most efficient
query execution plan.
+The community has started improving scheduler and execution performance
([FLINK-25318](https://issues.apache.org/jira/browse/FLINK-25318)) for
short-lived jobs to support OLAP. Flink executes “OLAP as a special case of
batch”, we are trying to extend Flink to execute low-latency and currency
queries in Session Cluster and users can perform streaming, batch, and OLAP
data processing on the unified Flink engine.
-### Faster Checkpoints and Recovery
+### Stability
-The community is continuously working on improving checkpointing and recovery
speed.
-Checkpoints and recovery are stable and have been a reliable workhorse for
years. We are still
-trying to make it faster, more predictable, and to remove some confusions and
inflexibility in some areas.
+The community is keeping improving the stability of jobs, by better tolerating
failures, and speeding up the recovery process.
--
[FLIP-183](https://cwiki.apache.org/confluence/display/FLINK/FLIP-183%3A+Dynamic+buffer+size+adjustment)
- is targeting size of checkpoints by debloating the buffers. A first beta is
already available.
-- With
[FLIP-151](https://cwiki.apache.org/confluence/display/FLINK/FLIP-151%3A+Incremental+snapshots+for+heap-based+state+backend)
- there is an ongoing effort to implement a heap based state backend.
+The instability of the environment is unavoidable. It can lead to a crash of
JobManager and TaskManager nodes, or slow data processing. The community has
introduced speculative execution
([FLIP-168](https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job),
[FLIP-245](https://cwiki.apache.org/confluence/display/FLINK/FLIP-245%3A+Source+Supports+Speculative+Execution+For+Batch+Job),
[FLIP-281](https://cwiki.apache.org/confluence/display/FLINK/FLIP- [...]
-### Apache Flink as part of an ever evolving data ecosystem
+JobManager node crash is usually unacceptable for a batch job because the job
has to be re-run from the very beginning. Therefore, the community is planning
to improve the JobManager recovery process to avoid re-run finished stages.
Another planned improvement is to retain running tasks when the JobManager node
goes down unexpectedly, to further reduce the impact of the JobManager crash.
This can also benefit streaming jobs even if they have periodical
checkpointing, to avoid interruptio [...]
-There is almost no use case in which Apache Flink is used on its own. It has
established itself
-as part of many data related reference architectures. In fact you'll find the
squirrel logo covering
-several aspects.
+### Usability
-The community has added a lot of connectors and formats. With the already
mentioned
-[FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface)
and
-[FLIP-143](https://cwiki.apache.org/confluence/display/FLINK/FLIP-143%3A+Unified+Sink+API)
-a new default for connectors has been established.
+Now and then we hear people say that, while Flink is powerful in
functionality, it is not that easy to master. Such voices are heard. The
community is working on several efforts to improve the usability of Flink.
-- There are efforts to revise the formats API with
-
[FLIP-219](https://cwiki.apache.org/confluence/display/FLINK/FLIP-219%3A+Revised+Format+API)
-- There is ongoing work on new connectors
- (e.g.
[Pinot](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177045634))
-- Connectors will be hosted in an external repository going forward. See the
- [ML thread](https://lists.apache.org/thread/8k1xonqt7hn0xldbky1cxfx3fzh6sj7h)
+We are working on reducing the number of configuration options that users need
to specify, as well as making them easier to understand and tune. This includes:
+Removing options that require in-depth knowledge of Flink internals to
understand and use.
+Making Flink automatically and dynamically decide the proper behavior where
possible.
+Improving the default values of the options so that users need not to touch
them in most cases.
+Improving the definition and description of the options so that they are
easier to understand and work with when it’s necessary.
+
+We have already made some progress along this direction. Flink 1.17 requires
less than 10 configurations to achieve well enough performance on TPC-DS.
Hybrid shuffle supports dynamically switching between different shuffle modes
and decouples its memory footprint from the parallelism of the job.
+
+
+## Developer Experience
+
+### Ecosystem
+There is almost no use case in which Apache Flink is used on its own. It has
established itself as part of many data related reference architectures. In
fact you’ll find the squirrel logo covering several aspects.
+
+All connectors will be hosted in an external repository going forward and many
of them have been successfully externalized. See the [mailing list
thread](https://lists.apache.org/thread/8k1xonqt7hn0xldbky1cxfx3fzh6sj7h).
+Catalog as a first-class citizen. Flink catalog lets users issue batch and
streaming queries connecting to external systems without registering
DDLs/schemas manually. It is recommended to support Catalog in the highest
priority for connectors. The community is working on supporting more catalogs
for connectors (e.g.
[GlueCatalog](https://cwiki.apache.org/confluence/display/FLINK/FLIP-277%3A+Native+GlueCatalog+Support+in+Flink),
[SchemaRegistryCatalog](https://cwiki.apache.org/confluence/ [...]
+There is ongoing work on introducing more new connectors (e.g.
[Pinot](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177045634),
[Redshift](https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift),
[ClickHouse](https://cwiki.apache.org/confluence/display/FLINK/FLIP-202%3A+Introduce+ClickHouse+Connector))
-The Flink community has removed Gelly, it's old graph-processing library.
### Documentation
-There are various dedicated efforts to simplify the maintenance and structure
(more intuitive navigation/reading)
-of the documentation.
+There are various dedicated efforts to simplify the maintenance and structure
(more intuitive navigation/reading) of the documentation.
- Docs Tech Stack:
[FLIP-157](https://cwiki.apache.org/confluence/display/FLINK/FLIP-157+Migrate+Flink+Documentation+from+Jekyll+to+Hugo)
- General Docs Structure:
[FLIP-42](https://cwiki.apache.org/confluence/display/FLINK/FLIP-42%3A+Rework+Flink+Documentation)
- SQL Docs:
[FLIP-60](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685)
-<hr />
-
-### Flink Kubernetes Operator
-
-The Flink Kubernetes Operator subproject has its own roadmap under the
[documentation](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/development/roadmap/).
-
-### Flink Table Store
-The Flink Table Store subproject has its own roadmap under the
[documentation](https://nightlies.apache.org/flink/flink-table-store-docs-master/docs/development/roadmap/).
diff --git a/docs/static/img/flink_feature_radar_4.svg
b/docs/static/img/flink_feature_radar_4.svg
new file mode 100644
index 000000000..a784f8025
--- /dev/null
+++ b/docs/static/img/flink_feature_radar_4.svg
@@ -0,0 +1,739 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd";>
+<svg version="1.1" xmlns="http://www.w3.org/2000/svg";
xmlns:xl="http://www.w3.org/1999/xlink";
xmlns:dc="http://purl.org/dc/elements/1.1/"; viewBox="-.5 -.5 1460 1610"
width="1460" height="1610">
+ <defs/>
+ <g id="Canvas_1" fill="none" fill-opacity="1" stroke="none"
stroke-opacity="1" stroke-dasharray="none">
+ <title>Canvas 1</title>
+ <g id="Canvas_1_Layer_1">
+ <title>Layer 1</title>
+ <g id="Group_2">
+ <g id="Graphic_226">
+ <rect x=".94" y="1515.5596" width="1460" height="100"
fill="#edf6ff"/>
+ </g>
+ <g id="Graphic_225">
+ <rect x=".94" y="925.5596" width="1460" height="540" fill="#edf6ff"/>
+ </g>
+ <g id="Graphic_224">
+ <rect x="-1.94" y="0" width="1460" height="860" fill="#edf6ff"/>
+ </g>
+ <g id="Graphic_223"/>
+ <g id="Group_221">
+ <g id="Graphic_222">
+ <title>Text</title>
+ <text transform="translate(20.10211 625.499)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">MVP</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_220">
+ <line x1="785.44" y1="826.1196" x2="120.66" y2="215.39957"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_219"/>
+ <g id="Group_217">
+ <g id="Graphic_218">
+ <title>Text</title>
+ <text transform="translate(36.640195 261.49902)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Beta</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_216"/>
+ <g id="Group_214">
+ <g id="Graphic_215">
+ <title>Text</title>
+ <text transform="translate(223.2613 133.49902)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black"
x="3.8896484" y="28">Production Ready </tspan>
+ <tspan font-family="Helvetica" font-size="28" fill="black"
x="49.03418" y="62">& Evolving</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_213"/>
+ <g id="Group_211">
+ <g id="Graphic_212">
+ <title>Text</title>
+ <text transform="translate(1252.7437 110.49902)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Stable</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_210"/>
+ <g id="Group_208">
+ <g id="Graphic_209">
+ <title>Text</title>
+ <text transform="translate(1204.0611 1130.499)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Deprecated</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_207">
+ <line x1="900.3" y1="825.1396" x2="967.5461" y2="137.55957"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_206">
+ <line x1="665.94" y1="1385.5596" x2="900.3" y2="985.4996"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_205"/>
+ <g id="Group_203">
+ <g id="Graphic_204">
+ <title>Text</title>
+ <text transform="translate(457.45465 970.499)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Approaching End-of-Life</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_202"/>
+ <g id="Group_200">
+ <g id="Graphic_201">
+ <title>Text</title>
+ <text transform="translate(57.21002 1551.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">APIs</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_199"/>
+ <g id="Group_197">
+ <g id="Graphic_198">
+ <title>Text</title>
+ <text transform="translate(125.9986 1551.1191)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Languages</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_196"/>
+ <g id="Group_194">
+ <g id="Graphic_195">
+ <title>Text</title>
+ <text transform="translate(256.81695 1551.1191)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">Clients</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_193"/>
+ <g id="Group_191">
+ <g id="Graphic_192">
+ <title>Text</title>
+ <text transform="translate(557.18854 1551.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Connectors</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_190"/>
+ <g id="Group_188">
+ <g id="Graphic_189">
+ <title>Text</title>
+ <text transform="translate(691.3917 1551.1191)" fill="#a680b8">
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="0" y="21">State Backends</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_187"/>
+ <g id="Group_185">
+ <g id="Graphic_186">
+ <title>Text</title>
+ <text transform="translate(862.2554 1551.1191)" fill="fuchsia">
+ <tspan font-family="Helvetica" font-size="22" fill="fuchsia"
x="0" y="21">Libraries</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_184"/>
+ <g id="Group_182">
+ <g id="Graphic_183">
+ <title>Text</title>
+ <text transform="translate(344.39166 1551.1191)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Resource Managers</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_181"/>
+ <g id="Group_179">
+ <g id="Graphic_180">
+ <title>Text</title>
+ <text transform="translate(1012.6463 182.44043)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">DataStream (streaming)</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_178"/>
+ <g id="Group_176">
+ <g id="Graphic_177">
+ <title>Text</title>
+ <text transform="translate(672.95855 367.55957)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">DataStream (batch)</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_175"/>
+ <g id="Group_173">
+ <g id="Graphic_174">
+ <title>Text</title>
+ <text transform="translate(902.44 1061.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">DataSet</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_170">
+ <g id="Graphic_171">
+ <title>Text</title>
+ <text transform="translate(1022.3206 242.55957)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SQL & Table API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_169"/>
+ <g id="Group_167">
+ <g id="Graphic_168">
+ <title>Text</title>
+ <text transform="translate(1068.44 1051.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">Queryable State</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_166"/>
+ <g id="Group_164">
+ <g id="Graphic_165">
+ <title>Text</title>
+ <text transform="translate(251.44 241.11914)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">State Processor API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_163"/>
+ <g id="Group_161">
+ <g id="Graphic_162">
+ <title>Text</title>
+ <text transform="translate(1226.8296 242.55957)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Java 11</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_160"/>
+ <g id="Group_158">
+ <g id="Graphic_159">
+ <title>Text</title>
+ <text transform="translate(804.44 1241.1191)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Scala 2.12</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_157"/>
+ <g id="Group_152">
+ <g id="Graphic_153">
+ <title>Text</title>
+ <text transform="translate(1190.94 302.0293)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Standalone</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_151"/>
+ <g id="Group_149">
+ <g id="Graphic_150">
+ <title>Text</title>
+ <text transform="translate(1020.44 511.11914)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Yarn</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_148"/>
+ <g id="Group_146">
+ <title>群组</title>
+ <g id="Graphic_147">
+ <title>Text</title>
+ <text transform="translate(1234.44 555.5596)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Zookeeper HA</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_145"/>
+ <g id="Graphic_142"/>
+ <g id="Group_140">
+ <g id="Graphic_141">
+ <title>Text</title>
+ <text transform="translate(439.21295 429)" fill="#a680b8">
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="0" y="21">Generic Incremental</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="44.01074" y="47">Checkpoint</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_139"/>
+ <g id="Group_137">
+ <g id="Graphic_138">
+ <title>Text</title>
+ <text transform="translate(962.7183 559.11914)" fill="#a680b8">
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="0" y="21">RocksDB/FS State Back.</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_134">
+ <g id="Graphic_135">
+ <title>Text</title>
+ <text transform="translate(1269.8223 189.55957)" fill="fuchsia">
+ <tspan font-family="Helvetica" font-size="22" fill="fuchsia"
x="0" y="21">CEP</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_133"/>
+ <g id="Group_131">
+ <g id="Graphic_132">
+ <title>Text</title>
+ <text transform="translate(41.608457 681.5293)" fill="fuchsia">
+ <tspan font-family="Helvetica" font-size="22" fill="fuchsia"
x="0" y="21">Machine Learning</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_130"/>
+ <g id="Group_128">
+ <g id="Graphic_129">
+ <title>Text</title>
+ <text transform="translate(251.44 1041.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy JDBC Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_127"/>
+ <g id="Group_125">
+ <g id="Graphic_126">
+ <title>Text</title>
+ <text transform="translate(518.79254 213.6787)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Kafka, File, Pulsar</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47"> [via Unified Source API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_124"/>
+ <g id="Group_122">
+ <g id="Graphic_123">
+ <title>Text</title>
+ <text transform="translate(185.4317 1151.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Rabbit MQ Source</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_121"/>
+ <g id="Group_119">
+ <g id="Graphic_120">
+ <title>Text</title>
+ <text transform="translate(756.44 1315.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Kinesis Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_118"/>
+ <g id="Group_116">
+ <g id="Graphic_117">
+ <title>Text</title>
+ <text transform="translate(33.44 1226.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy PubSub Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_115"/>
+ <g id="Group_113">
+ <g id="Graphic_114">
+ <title>Text</title>
+ <text transform="translate(49.44 1291.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Cassandra Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_112"/>
+ <g id="Group_110">
+ <g id="Graphic_111">
+ <title>Text</title>
+ <text transform="translate(361.44 1271.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy HBase SQL Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_109"/>
+ <g id="Group_107">
+ <g id="Graphic_108">
+ <title>Text</title>
+ <text transform="translate(1182.12 342.55957)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Hive Catalog</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_106"/>
+ <g id="Group_104">
+ <g id="Graphic_105">
+ <title>Text</title>
+ <text transform="translate(582.44 1168.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Hive SQL</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_103"/>
+ <g id="Group_101">
+ <g id="Graphic_102">
+ <title>Text</title>
+ <text transform="translate(592.94 285.05957)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Kafka, File, Pulsar, Elasticsearch</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47"> [via Unified Sink API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_100"/>
+ <g id="Group_98">
+ <g id="Graphic_99">
+ <title>Text</title>
+ <text transform="translate(271.44 291.11914)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Python Table API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_97"/>
+ <g id="Group_95">
+ <g id="Graphic_96">
+ <title>Text</title>
+ <text transform="translate(381.94 352.55957)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Python</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="47">DataStream API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_94"/>
+ <g id="Group_92">
+ <g id="Graphic_93">
+ <title>Text</title>
+ <text transform="translate(1213.44 671.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">S3 FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_91"/>
+ <g id="Group_89">
+ <g id="Graphic_90">
+ <title>Text</title>
+ <text transform="translate(968.44 671.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">GCS FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_88"/>
+ <g id="Group_86">
+ <g id="Graphic_87">
+ <title>Text</title>
+ <text transform="translate(1234.44 593.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Local/NFS</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_85"/>
+ <g id="Group_83">
+ <g id="Graphic_84">
+ <title>Text</title>
+ <text transform="translate(976.44 611.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">HDFS FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_82"/>
+ <g id="Group_80">
+ <g id="Graphic_81">
+ <title>Text</title>
+ <text transform="translate(999.44 333.7798)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Azure Blob</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_79"/>
+ <g id="Group_77">
+ <g id="Graphic_78">
+ <title>Text</title>
+ <text transform="translate(999.44 443.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">AliCloud OSS</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_76">
+ <path d="M 188.44 795.5596 C 323.44 535.5596 712.0033 405.55957
1354.13 405.55957" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_75">
+ <path d="M 1360.88 405.55957 L 1351.88 410.05957 L 1354.13 405.55957
L 1351.88 401.05957 Z" fill="#b5739d"/>
+ <path d="M 1360.88 405.55957 L 1351.88 410.05957 L 1354.13 405.55957
L 1351.88 401.05957 Z" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_74">
+ <path d="M 175.94 1105.5596 C 719.2733 1092.2262 1087.3533 1201.2529
1280.18 1432.6396" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_73">
+ <path d="M 1284.5 1437.8296 L 1275.28 1433.7996 L 1280.18 1432.6396
L 1282.19 1428.0396 Z" fill="#b5739d"/>
+ <path d="M 1284.5 1437.8296 L 1275.28 1433.7996 L 1280.18 1432.6396
L 1282.19 1428.0396 Z" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_72"/>
+ <g id="Group_70">
+ <g id="Graphic_71">
+ <title>Text</title>
+ <text transform="translate(193.44 600.5596)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">SQL CLI</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_69">
+ <line x1="767.94" y1="840.8796" x2="20.94" y2="585.5596"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_68"/>
+ <g id="Group_66">
+ <g id="Graphic_67">
+ <title>Text</title>
+ <text transform="translate(12.44 23.798828)" fill="black">
+ <tspan font-family="Helvetica" font-size="38" fill="black" x="0"
y="37">New- and Stable Features</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_65"/>
+ <g id="Group_63">
+ <g id="Graphic_64">
+ <title>Text</title>
+ <text transform="translate(12.44 933.7988)" fill="black">
+ <tspan font-family="Helvetica" font-size="38" fill="black" x="0"
y="37">Features Phasing Out</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_62"/>
+ <g id="Group_60">
+ <g id="Graphic_61">
+ <title>Text</title>
+ <text transform="translate(785.44 1351.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy File Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_59"/>
+ <g id="Group_57">
+ <g id="Graphic_58">
+ <title>Text</title>
+ <text transform="translate(845.44 1396.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Kafka Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_56"/>
+ <g id="Group_54">
+ <g id="Graphic_55">
+ <title>Text</title>
+ <text transform="translate(939.397 1131.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SourceFunction</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_51">
+ <g id="Graphic_52">
+ <title>Text</title>
+ <text transform="translate(1012.6039 291.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">HybridSource</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_50"/>
+ <g id="Group_48">
+ <g id="Graphic_49">
+ <title>Text</title>
+ <text transform="translate(961.4078 1551.1191)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Schedulers</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_47"/>
+ <g id="Group_45">
+ <g id="Graphic_46">
+ <title>Text</title>
+ <text transform="translate(1192.44 721.1191)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Default Scheduler</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_44"/>
+ <g id="Group_42">
+ <g id="Graphic_43">
+ <title>Text</title>
+ <text transform="translate(545.8443 548.0596)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Adaptive Scheduler / Reactive Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_41"/>
+ <g id="Group_39">
+ <g id="Graphic_40">
+ <title>Text</title>
+ <text transform="translate(1124.44 1221.1191)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Java 8</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_38"/>
+ <g id="Group_36">
+ <g id="Graphic_37">
+ <title>Text</title>
+ <text transform="translate(1090.6724 1551.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Deployment Modes</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_35"/>
+ <g id="Group_33">
+ <g id="Graphic_34">
+ <title>Text</title>
+ <text transform="translate(970.0547 721.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Application Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_32"/>
+ <g id="Group_30">
+ <g id="Graphic_31">
+ <title>Text</title>
+ <text transform="translate(1202.734 771.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Session Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_29"/>
+ <g id="Group_27">
+ <g id="Graphic_28">
+ <title>Text</title>
+ <text transform="translate(1196.3516 1281.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Per-Job Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_26"/>
+ <g id="Group_24">
+ <g id="Graphic_25">
+ <title>Text</title>
+ <text transform="translate(602.3179 600.5596)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Adaptive Batch Job Scheduler</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_23"/>
+ <g id="Group_21">
+ <g id="Graphic_22">
+ <title>Text</title>
+ <text transform="translate(741.5733 725.33936)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">ASync API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_20"/>
+ <g id="Group_18">
+ <g id="Graphic_19">
+ <title>Text</title>
+ <text transform="translate(697.44 663.0596)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">Unified Sink API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_17"/>
+ <g id="Group_15">
+ <g id="Graphic_16">
+ <title>Text</title>
+ <text transform="translate(944.44 771.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">Unified Source API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_14"/>
+ <g id="Group_12">
+ <g id="Graphic_13">
+ <title>Text</title>
+ <text transform="translate(251.44 740.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SQL Upgrades</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_11"/>
+ <g id="Group_9">
+ <g id="Graphic_10">
+ <title>Text</title>
+ <text transform="translate(81.6664 485.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Kinesis Streams, Firehose, </tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">MongoDB, DynamoDB, Opensearch</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="73"> [via Unified Sink API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_8"/>
+ <g id="Group_6">
+ <g id="Graphic_7">
+ <title>Text</title>
+ <text transform="translate(24.44 408.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">MongoDB</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47"> [via Unified Source API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_5"/>
+ <g id="Group_3">
+ <g id="Graphic_4">
+ <title>Text</title>
+ <text transform="translate(634.44 1051.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SinkFunction</tspan>
+ </text>
+ </g>
+ </g>
+ </g>
+ <g id="Group_228">
+ <g id="Graphic_229">
+ <title>Text</title>
+ <text transform="translate(392 620)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">SQL Gateway</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_232">
+ <g id="Graphic_233">
+ <title>Text</title>
+ <text transform="translate(386 778)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">Flink JDBC Driver</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_234">
+ <g id="Graphic_235">
+ <title>Text</title>
+ <text transform="translate(1297 1546)" fill="#05445e">
+ <tspan font-family="Helvetica" font-size="22" fill="#05445e" x="0"
y="21">Shuffle</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_236">
+ <g id="Graphic_237">
+ <title>Text</title>
+ <text transform="translate(50.526855 722)" fill="#05445e">
+ <tspan font-family="Helvetica" font-size="22" fill="#05445e" x="0"
y="21">Hybrid Shuffle</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_238">
+ <g id="Graphic_239">
+ <title>Text</title>
+ <text transform="translate(497 678)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Java 17</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_240">
+ <title>k8s HA</title>
+ <g id="Graphic_241">
+ <title>Text</title>
+ <text transform="translate(1160 478)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Kubernetes-based HA</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="47">(ZK-alternative)</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_242">
+ <title>K8s</title>
+ <g id="Graphic_243">
+ <title>Text</title>
+ <text transform="translate(1227 439.5)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Kubernetes</tspan>
+ </text>
+ </g>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/docs/static/img/flink_feature_radar_zh_4.svg
b/docs/static/img/flink_feature_radar_zh_4.svg
new file mode 100644
index 000000000..a784f8025
--- /dev/null
+++ b/docs/static/img/flink_feature_radar_zh_4.svg
@@ -0,0 +1,739 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd";>
+<svg version="1.1" xmlns="http://www.w3.org/2000/svg";
xmlns:xl="http://www.w3.org/1999/xlink";
xmlns:dc="http://purl.org/dc/elements/1.1/"; viewBox="-.5 -.5 1460 1610"
width="1460" height="1610">
+ <defs/>
+ <g id="Canvas_1" fill="none" fill-opacity="1" stroke="none"
stroke-opacity="1" stroke-dasharray="none">
+ <title>Canvas 1</title>
+ <g id="Canvas_1_Layer_1">
+ <title>Layer 1</title>
+ <g id="Group_2">
+ <g id="Graphic_226">
+ <rect x=".94" y="1515.5596" width="1460" height="100"
fill="#edf6ff"/>
+ </g>
+ <g id="Graphic_225">
+ <rect x=".94" y="925.5596" width="1460" height="540" fill="#edf6ff"/>
+ </g>
+ <g id="Graphic_224">
+ <rect x="-1.94" y="0" width="1460" height="860" fill="#edf6ff"/>
+ </g>
+ <g id="Graphic_223"/>
+ <g id="Group_221">
+ <g id="Graphic_222">
+ <title>Text</title>
+ <text transform="translate(20.10211 625.499)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">MVP</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_220">
+ <line x1="785.44" y1="826.1196" x2="120.66" y2="215.39957"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_219"/>
+ <g id="Group_217">
+ <g id="Graphic_218">
+ <title>Text</title>
+ <text transform="translate(36.640195 261.49902)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Beta</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_216"/>
+ <g id="Group_214">
+ <g id="Graphic_215">
+ <title>Text</title>
+ <text transform="translate(223.2613 133.49902)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black"
x="3.8896484" y="28">Production Ready </tspan>
+ <tspan font-family="Helvetica" font-size="28" fill="black"
x="49.03418" y="62">& Evolving</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_213"/>
+ <g id="Group_211">
+ <g id="Graphic_212">
+ <title>Text</title>
+ <text transform="translate(1252.7437 110.49902)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Stable</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_210"/>
+ <g id="Group_208">
+ <g id="Graphic_209">
+ <title>Text</title>
+ <text transform="translate(1204.0611 1130.499)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Deprecated</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_207">
+ <line x1="900.3" y1="825.1396" x2="967.5461" y2="137.55957"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_206">
+ <line x1="665.94" y1="1385.5596" x2="900.3" y2="985.4996"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_205"/>
+ <g id="Group_203">
+ <g id="Graphic_204">
+ <title>Text</title>
+ <text transform="translate(457.45465 970.499)" fill="black">
+ <tspan font-family="Helvetica" font-size="28" fill="black" x="0"
y="28">Approaching End-of-Life</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_202"/>
+ <g id="Group_200">
+ <g id="Graphic_201">
+ <title>Text</title>
+ <text transform="translate(57.21002 1551.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">APIs</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_199"/>
+ <g id="Group_197">
+ <g id="Graphic_198">
+ <title>Text</title>
+ <text transform="translate(125.9986 1551.1191)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Languages</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_196"/>
+ <g id="Group_194">
+ <g id="Graphic_195">
+ <title>Text</title>
+ <text transform="translate(256.81695 1551.1191)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">Clients</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_193"/>
+ <g id="Group_191">
+ <g id="Graphic_192">
+ <title>Text</title>
+ <text transform="translate(557.18854 1551.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Connectors</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_190"/>
+ <g id="Group_188">
+ <g id="Graphic_189">
+ <title>Text</title>
+ <text transform="translate(691.3917 1551.1191)" fill="#a680b8">
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="0" y="21">State Backends</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_187"/>
+ <g id="Group_185">
+ <g id="Graphic_186">
+ <title>Text</title>
+ <text transform="translate(862.2554 1551.1191)" fill="fuchsia">
+ <tspan font-family="Helvetica" font-size="22" fill="fuchsia"
x="0" y="21">Libraries</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_184"/>
+ <g id="Group_182">
+ <g id="Graphic_183">
+ <title>Text</title>
+ <text transform="translate(344.39166 1551.1191)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Resource Managers</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_181"/>
+ <g id="Group_179">
+ <g id="Graphic_180">
+ <title>Text</title>
+ <text transform="translate(1012.6463 182.44043)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">DataStream (streaming)</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_178"/>
+ <g id="Group_176">
+ <g id="Graphic_177">
+ <title>Text</title>
+ <text transform="translate(672.95855 367.55957)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">DataStream (batch)</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_175"/>
+ <g id="Group_173">
+ <g id="Graphic_174">
+ <title>Text</title>
+ <text transform="translate(902.44 1061.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">DataSet</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_170">
+ <g id="Graphic_171">
+ <title>Text</title>
+ <text transform="translate(1022.3206 242.55957)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SQL & Table API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_169"/>
+ <g id="Group_167">
+ <g id="Graphic_168">
+ <title>Text</title>
+ <text transform="translate(1068.44 1051.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">Queryable State</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_166"/>
+ <g id="Group_164">
+ <g id="Graphic_165">
+ <title>Text</title>
+ <text transform="translate(251.44 241.11914)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">State Processor API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_163"/>
+ <g id="Group_161">
+ <g id="Graphic_162">
+ <title>Text</title>
+ <text transform="translate(1226.8296 242.55957)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Java 11</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_160"/>
+ <g id="Group_158">
+ <g id="Graphic_159">
+ <title>Text</title>
+ <text transform="translate(804.44 1241.1191)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Scala 2.12</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_157"/>
+ <g id="Group_152">
+ <g id="Graphic_153">
+ <title>Text</title>
+ <text transform="translate(1190.94 302.0293)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Standalone</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_151"/>
+ <g id="Group_149">
+ <g id="Graphic_150">
+ <title>Text</title>
+ <text transform="translate(1020.44 511.11914)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Yarn</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_148"/>
+ <g id="Group_146">
+ <title>群组</title>
+ <g id="Graphic_147">
+ <title>Text</title>
+ <text transform="translate(1234.44 555.5596)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Zookeeper HA</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_145"/>
+ <g id="Graphic_142"/>
+ <g id="Group_140">
+ <g id="Graphic_141">
+ <title>Text</title>
+ <text transform="translate(439.21295 429)" fill="#a680b8">
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="0" y="21">Generic Incremental</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="44.01074" y="47">Checkpoint</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_139"/>
+ <g id="Group_137">
+ <g id="Graphic_138">
+ <title>Text</title>
+ <text transform="translate(962.7183 559.11914)" fill="#a680b8">
+ <tspan font-family="Helvetica" font-size="22" fill="#a680b8"
x="0" y="21">RocksDB/FS State Back.</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_134">
+ <g id="Graphic_135">
+ <title>Text</title>
+ <text transform="translate(1269.8223 189.55957)" fill="fuchsia">
+ <tspan font-family="Helvetica" font-size="22" fill="fuchsia"
x="0" y="21">CEP</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_133"/>
+ <g id="Group_131">
+ <g id="Graphic_132">
+ <title>Text</title>
+ <text transform="translate(41.608457 681.5293)" fill="fuchsia">
+ <tspan font-family="Helvetica" font-size="22" fill="fuchsia"
x="0" y="21">Machine Learning</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_130"/>
+ <g id="Group_128">
+ <g id="Graphic_129">
+ <title>Text</title>
+ <text transform="translate(251.44 1041.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy JDBC Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_127"/>
+ <g id="Group_125">
+ <g id="Graphic_126">
+ <title>Text</title>
+ <text transform="translate(518.79254 213.6787)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Kafka, File, Pulsar</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47"> [via Unified Source API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_124"/>
+ <g id="Group_122">
+ <g id="Graphic_123">
+ <title>Text</title>
+ <text transform="translate(185.4317 1151.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Rabbit MQ Source</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_121"/>
+ <g id="Group_119">
+ <g id="Graphic_120">
+ <title>Text</title>
+ <text transform="translate(756.44 1315.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Kinesis Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_118"/>
+ <g id="Group_116">
+ <g id="Graphic_117">
+ <title>Text</title>
+ <text transform="translate(33.44 1226.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy PubSub Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_115"/>
+ <g id="Group_113">
+ <g id="Graphic_114">
+ <title>Text</title>
+ <text transform="translate(49.44 1291.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Cassandra Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_112"/>
+ <g id="Group_110">
+ <g id="Graphic_111">
+ <title>Text</title>
+ <text transform="translate(361.44 1271.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy HBase SQL Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_109"/>
+ <g id="Group_107">
+ <g id="Graphic_108">
+ <title>Text</title>
+ <text transform="translate(1182.12 342.55957)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Hive Catalog</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_106"/>
+ <g id="Group_104">
+ <g id="Graphic_105">
+ <title>Text</title>
+ <text transform="translate(582.44 1168.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Hive SQL</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_103"/>
+ <g id="Group_101">
+ <g id="Graphic_102">
+ <title>Text</title>
+ <text transform="translate(592.94 285.05957)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Kafka, File, Pulsar, Elasticsearch</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47"> [via Unified Sink API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_100"/>
+ <g id="Group_98">
+ <g id="Graphic_99">
+ <title>Text</title>
+ <text transform="translate(271.44 291.11914)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Python Table API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_97"/>
+ <g id="Group_95">
+ <g id="Graphic_96">
+ <title>Text</title>
+ <text transform="translate(381.94 352.55957)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Python</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="47">DataStream API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_94"/>
+ <g id="Group_92">
+ <g id="Graphic_93">
+ <title>Text</title>
+ <text transform="translate(1213.44 671.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">S3 FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_91"/>
+ <g id="Group_89">
+ <g id="Graphic_90">
+ <title>Text</title>
+ <text transform="translate(968.44 671.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">GCS FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_88"/>
+ <g id="Group_86">
+ <g id="Graphic_87">
+ <title>Text</title>
+ <text transform="translate(1234.44 593.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Local/NFS</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_85"/>
+ <g id="Group_83">
+ <g id="Graphic_84">
+ <title>Text</title>
+ <text transform="translate(976.44 611.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">HDFS FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_82"/>
+ <g id="Group_80">
+ <g id="Graphic_81">
+ <title>Text</title>
+ <text transform="translate(999.44 333.7798)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Azure Blob</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_79"/>
+ <g id="Group_77">
+ <g id="Graphic_78">
+ <title>Text</title>
+ <text transform="translate(999.44 443.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">AliCloud OSS</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">FileSystem</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_76">
+ <path d="M 188.44 795.5596 C 323.44 535.5596 712.0033 405.55957
1354.13 405.55957" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_75">
+ <path d="M 1360.88 405.55957 L 1351.88 410.05957 L 1354.13 405.55957
L 1351.88 401.05957 Z" fill="#b5739d"/>
+ <path d="M 1360.88 405.55957 L 1351.88 410.05957 L 1354.13 405.55957
L 1351.88 401.05957 Z" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_74">
+ <path d="M 175.94 1105.5596 C 719.2733 1092.2262 1087.3533 1201.2529
1280.18 1432.6396" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_73">
+ <path d="M 1284.5 1437.8296 L 1275.28 1433.7996 L 1280.18 1432.6396
L 1282.19 1428.0396 Z" fill="#b5739d"/>
+ <path d="M 1284.5 1437.8296 L 1275.28 1433.7996 L 1280.18 1432.6396
L 1282.19 1428.0396 Z" stroke="#b5739d" stroke-linecap="butt"
stroke-linejoin="round" stroke-width="9"/>
+ </g>
+ <g id="Graphic_72"/>
+ <g id="Group_70">
+ <g id="Graphic_71">
+ <title>Text</title>
+ <text transform="translate(193.44 600.5596)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">SQL CLI</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_69">
+ <line x1="767.94" y1="840.8796" x2="20.94" y2="585.5596"
stroke="black" stroke-linecap="butt" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ </g>
+ <g id="Graphic_68"/>
+ <g id="Group_66">
+ <g id="Graphic_67">
+ <title>Text</title>
+ <text transform="translate(12.44 23.798828)" fill="black">
+ <tspan font-family="Helvetica" font-size="38" fill="black" x="0"
y="37">New- and Stable Features</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_65"/>
+ <g id="Group_63">
+ <g id="Graphic_64">
+ <title>Text</title>
+ <text transform="translate(12.44 933.7988)" fill="black">
+ <tspan font-family="Helvetica" font-size="38" fill="black" x="0"
y="37">Features Phasing Out</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_62"/>
+ <g id="Group_60">
+ <g id="Graphic_61">
+ <title>Text</title>
+ <text transform="translate(785.44 1351.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy File Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_59"/>
+ <g id="Group_57">
+ <g id="Graphic_58">
+ <title>Text</title>
+ <text transform="translate(845.44 1396.1191)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Legacy Kafka Source & Sink</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_56"/>
+ <g id="Group_54">
+ <g id="Graphic_55">
+ <title>Text</title>
+ <text transform="translate(939.397 1131.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SourceFunction</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_51">
+ <g id="Graphic_52">
+ <title>Text</title>
+ <text transform="translate(1012.6039 291.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">HybridSource</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_50"/>
+ <g id="Group_48">
+ <g id="Graphic_49">
+ <title>Text</title>
+ <text transform="translate(961.4078 1551.1191)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Schedulers</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_47"/>
+ <g id="Group_45">
+ <g id="Graphic_46">
+ <title>Text</title>
+ <text transform="translate(1192.44 721.1191)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Default Scheduler</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_44"/>
+ <g id="Group_42">
+ <g id="Graphic_43">
+ <title>Text</title>
+ <text transform="translate(545.8443 548.0596)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Adaptive Scheduler / Reactive Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_41"/>
+ <g id="Group_39">
+ <g id="Graphic_40">
+ <title>Text</title>
+ <text transform="translate(1124.44 1221.1191)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Java 8</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_38"/>
+ <g id="Group_36">
+ <g id="Graphic_37">
+ <title>Text</title>
+ <text transform="translate(1090.6724 1551.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Deployment Modes</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_35"/>
+ <g id="Group_33">
+ <g id="Graphic_34">
+ <title>Text</title>
+ <text transform="translate(970.0547 721.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Application Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_32"/>
+ <g id="Group_30">
+ <g id="Graphic_31">
+ <title>Text</title>
+ <text transform="translate(1202.734 771.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Session Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_29"/>
+ <g id="Group_27">
+ <g id="Graphic_28">
+ <title>Text</title>
+ <text transform="translate(1196.3516 1281.1191)" fill="#990">
+ <tspan font-family="Helvetica" font-size="22" fill="#990" x="0"
y="21">Per-Job Mode</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_26"/>
+ <g id="Group_24">
+ <g id="Graphic_25">
+ <title>Text</title>
+ <text transform="translate(602.3179 600.5596)" fill="#4c0099">
+ <tspan font-family="Helvetica" font-size="22" fill="#4c0099"
x="0" y="21">Adaptive Batch Job Scheduler</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_23"/>
+ <g id="Group_21">
+ <g id="Graphic_22">
+ <title>Text</title>
+ <text transform="translate(741.5733 725.33936)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">ASync API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_20"/>
+ <g id="Group_18">
+ <g id="Graphic_19">
+ <title>Text</title>
+ <text transform="translate(697.44 663.0596)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">Unified Sink API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_17"/>
+ <g id="Group_15">
+ <g id="Graphic_16">
+ <title>Text</title>
+ <text transform="translate(944.44 771.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">Unified Source API</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_14"/>
+ <g id="Group_12">
+ <g id="Graphic_13">
+ <title>Text</title>
+ <text transform="translate(251.44 740.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SQL Upgrades</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_11"/>
+ <g id="Group_9">
+ <g id="Graphic_10">
+ <title>Text</title>
+ <text transform="translate(81.6664 485.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">Kinesis Streams, Firehose, </tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47">MongoDB, DynamoDB, Opensearch</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="73"> [via Unified Sink API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_8"/>
+ <g id="Group_6">
+ <g id="Graphic_7">
+ <title>Text</title>
+ <text transform="translate(24.44 408.11914)" fill="#67ab9f">
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="21">MongoDB</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#67ab9f"
x="0" y="47"> [via Unified Source API]</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Graphic_5"/>
+ <g id="Group_3">
+ <g id="Graphic_4">
+ <title>Text</title>
+ <text transform="translate(634.44 1051.1191)" fill="#33f">
+ <tspan font-family="Helvetica" font-size="22" fill="#33f" x="0"
y="21">SinkFunction</tspan>
+ </text>
+ </g>
+ </g>
+ </g>
+ <g id="Group_228">
+ <g id="Graphic_229">
+ <title>Text</title>
+ <text transform="translate(392 620)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">SQL Gateway</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_232">
+ <g id="Graphic_233">
+ <title>Text</title>
+ <text transform="translate(386 778)" fill="#c60">
+ <tspan font-family="Helvetica" font-size="22" fill="#c60" x="0"
y="21">Flink JDBC Driver</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_234">
+ <g id="Graphic_235">
+ <title>Text</title>
+ <text transform="translate(1297 1546)" fill="#05445e">
+ <tspan font-family="Helvetica" font-size="22" fill="#05445e" x="0"
y="21">Shuffle</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_236">
+ <g id="Graphic_237">
+ <title>Text</title>
+ <text transform="translate(50.526855 722)" fill="#05445e">
+ <tspan font-family="Helvetica" font-size="22" fill="#05445e" x="0"
y="21">Hybrid Shuffle</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_238">
+ <g id="Graphic_239">
+ <title>Text</title>
+ <text transform="translate(497 678)" fill="#090">
+ <tspan font-family="Helvetica" font-size="22" fill="#090" x="0"
y="21">Java 17</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_240">
+ <title>k8s HA</title>
+ <g id="Graphic_241">
+ <title>Text</title>
+ <text transform="translate(1160 478)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Kubernetes-based HA</tspan>
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="47">(ZK-alternative)</tspan>
+ </text>
+ </g>
+ </g>
+ <g id="Group_242">
+ <title>K8s</title>
+ <g id="Graphic_243">
+ <title>Text</title>
+ <text transform="translate(1227 439.5)" fill="#c00">
+ <tspan font-family="Helvetica" font-size="22" fill="#c00" x="0"
y="21">Kubernetes</tspan>
+ </text>
+ </g>
+ </g>
+ </g>
+ </g>
+</svg>
