nictownsend commented on code in PR #28863: URL: https://github.com/apache/flink/pull/28863#discussion_r3903360452
########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical Review Comment: Does this mean an output channel has an identity/key itself, or does the sub-task look at the destination to find that out? ########## docs/content/docs/concepts/glossary.md: ########## @@ -27,15 +27,17 @@ under the License. #### Flink Application -A Flink application is a Java Application that submits one or multiple [Flink -Jobs](#flink-job) from the `main()` method (or by some other means). Submitting -jobs is usually done by calling `execute()` on an execution environment. +A Flink Application is a Java or Python program, written against the DataStream API or the Table API, +that submits one or multiple [Flink Jobs](#flink-job) from the `main()` method (or by some other +means). Submitting Jobs is usually done by calling `execute()` on an execution environment. -The jobs of an application can either be submitted to a long running [Flink +The Jobs of an Application can either be submitted to a long-running [Flink Session Cluster](#flink-session-cluster), to a dedicated [Flink Application Cluster](#flink-application-cluster), or to a [Flink Job Cluster](#flink-job-cluster). +See [Flink Session Cluster](#flink-session-cluster) for comparison. Review Comment: For what comparison? Flink Session Cluster says: > A long-running [Flink Cluster](https://github.com/nicusX/flink/blob/c772d6a44e8dfcf0a849a2df20f7769adf4e3e4d/docs/content/docs/concepts/glossary.md#flink-cluster) which accepts multiple [Flink Jobs](https://github.com/nicusX/flink/blob/c772d6a44e8dfcf0a849a2df20f7769adf4e3e4d/docs/content/docs/concepts/glossary.md#flink-job) for execution. The lifetime of this Flink Cluster is not bound to the lifetime of any Flink Job. Formerly, a Flink Session Cluster was also known as a Flink Cluster in session mode. Compare to [Flink Application Cluster](https://github.com/nicusX/flink/blob/c772d6a44e8dfcf0a849a2df20f7769adf4e3e4d/docs/content/docs/concepts/glossary.md#flink-application-cluster). Which has no comparison between Application deployment methods. As a glossary, maybe there's no need to talk about comparisons? ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). Review Comment: Remote object storage tolerates the loss of a JobManager - but even storage on JM Java Heap allows TaskManager loss. ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the Review Comment: Is this trying to distinguish between: DataStream - `data.map(new MyMapFunction<String, Integer>() {...});` and `data.map(s -> 1);` TableAPI - `env.from("MyTable").select(call(MyMapFunction.class, $("myField")));` Where in DataStream you can pass it directly via anonymous / lambda - but in Table you have to define it as a class to then register/invoke? ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. Review Comment: ```suggestion Kafka Sink, for instance, only commits its Kafka transactions when the Checkpoint containing the transaction records completes. ``` ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the +[Transformations](#transformation) they implement. In the Table API and SQL, they are declared +separately as [User-Defined Functions]({{< ref "docs/dev/table/functions/udfs" >}}) (UDF) or +[Process Table Functions]({{< ref "docs/dev/table/functions/ptfs" >}}) (PTF). #### History Server The History Server is a standalone service that serves the detailed history of completed Flink -applications and jobs, using archives generated by the JobManager. Unlike the -[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store -minimal metadata for internal recovery decisions in highly-available clusters, the History Server -provides detailed archives for analysis via Web UI or REST API after the cluster has been shut down. +Applications and Jobs, using archives generated by the JobManager. Unlike the +[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store +minimal metadata for internal recovery decisions in highly-available Clusters, the History Server +provides detailed archives for analysis via Web UI or REST API after the Cluster has been shut down. #### Instance The term *instance* is used to describe a specific instance of a specific type (usually -[Operator](#operator) or [Function](#function)) during runtime. As Apache Flink is mostly written in +[Operator](#operator) or [Function](#function)) at runtime. As Apache Flink is mostly written in Java, this corresponds to the definition of *Instance* or *Object* in Java. In the context of Apache Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. #### Flink Job -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially Review Comment: ```suggestion started, stopped and resumed. Note: under some conditions Flink may restart a Job only partially ``` Perhaps use the "note" eyecatcher like for e.g in https://nightlies.apache.org/flink/flink-docs-master/docs/sql/reference/queries/hints/#join-hints <img width="880" height="479" alt="Image" src="https://github.com/user-attachments/assets/4ba7b7d3-168e-4839-99fc-4a49a0f4da62"; /> ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the +[Transformations](#transformation) they implement. In the Table API and SQL, they are declared +separately as [User-Defined Functions]({{< ref "docs/dev/table/functions/udfs" >}}) (UDF) or +[Process Table Functions]({{< ref "docs/dev/table/functions/ptfs" >}}) (PTF). #### History Server The History Server is a standalone service that serves the detailed history of completed Flink -applications and jobs, using archives generated by the JobManager. Unlike the -[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store -minimal metadata for internal recovery decisions in highly-available clusters, the History Server -provides detailed archives for analysis via Web UI or REST API after the cluster has been shut down. +Applications and Jobs, using archives generated by the JobManager. Unlike the +[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store +minimal metadata for internal recovery decisions in highly-available Clusters, the History Server +provides detailed archives for analysis via Web UI or REST API after the Cluster has been shut down. #### Instance The term *instance* is used to describe a specific instance of a specific type (usually -[Operator](#operator) or [Function](#function)) during runtime. As Apache Flink is mostly written in +[Operator](#operator) or [Function](#function)) at runtime. As Apache Flink is mostly written in Java, this corresponds to the definition of *Instance* or *Object* in Java. In the context of Apache Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. #### Flink Job -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially +(See [Restart Pipelined Region Failover Strategy]({{< ref "docs/ops/state/task_failure_recovery" >}}#restart-pipelined-region-failover-strategy)). + +A Job is submitted either by a [Flink Application](#flink-application), by calling `execute()` on an +execution environment, or as a single [Flink SQL Statement](#flink-sql-statement) or [Statement +Set](#statement-set). + +A Flink Job is the runtime representation of a [Logical Graph](#logical-graph) (also often called +*Dataflow Graph*). The Logical Graph is optimized into a [Job Graph](#job-graph), from which the +[Physical Graph](#physical-graph) that actually runs in a [Flink Cluster](#flink-cluster) is derived. #### Flink Job Cluster A Flink Job Cluster is a dedicated [Flink Cluster](#flink-cluster) that only executes a single [Flink Job](#flink-job). The lifetime of the -[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. -This deployment mode has been deprecated since Flink 1.15. +[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. +This deployment mode has been deprecated since Flink 1.15. + +#### Job Graph -#### JobGraph +Also called *JobGraph* or *Optimized Dataflow*. -see [Logical Graph](#logical-graph) +A Job Graph is the optimized representation of a [Logical Graph](#logical-graph), and the +representation that a [Flink Application](#flink-application) submits to the [Flink +Cluster](#flink-cluster). + +Producing the Job Graph is mainly a matter of chaining [Operators](#operator): consecutive +[Operators](#operator) that are not separated by a repartitioning are merged into a single +[Task](#task). The nodes of a Job Graph are therefore [Tasks](#task), each implementing one Operator +or one [Operator Chain](#operator-chain). + +The Job Graph is translated into a [Physical Graph](#physical-graph) for execution. #### Flink JobManager -The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It contains three distinct -components: Flink Resource Manager, Flink Dispatcher and one [Flink JobMaster](#flink-jobmaster) -per running [Flink Job](#flink-job). +Also called *Job Manager*. + +The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It does not process any +data itself: it translates the submitted [Job Graph](#job-graph) into a [Physical +Graph](#physical-graph), schedules the resulting [Sub-Tasks](#sub-task) on the +[TaskManagers](#flink-taskmanager), and coordinates [Checkpoints](#checkpoint) and +[Savepoints](#savepoint). It contains three distinct components: Flink Resource Manager, Flink +Dispatcher and one [Flink JobMaster](#flink-jobmaster) per running [Flink Job](#flink-job). Review Comment: Worth mentioning JobManager HA here? ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the +[Transformations](#transformation) they implement. In the Table API and SQL, they are declared +separately as [User-Defined Functions]({{< ref "docs/dev/table/functions/udfs" >}}) (UDF) or +[Process Table Functions]({{< ref "docs/dev/table/functions/ptfs" >}}) (PTF). #### History Server The History Server is a standalone service that serves the detailed history of completed Flink -applications and jobs, using archives generated by the JobManager. Unlike the -[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store -minimal metadata for internal recovery decisions in highly-available clusters, the History Server -provides detailed archives for analysis via Web UI or REST API after the cluster has been shut down. +Applications and Jobs, using archives generated by the JobManager. Unlike the +[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store +minimal metadata for internal recovery decisions in highly-available Clusters, the History Server +provides detailed archives for analysis via Web UI or REST API after the Cluster has been shut down. #### Instance The term *instance* is used to describe a specific instance of a specific type (usually -[Operator](#operator) or [Function](#function)) during runtime. As Apache Flink is mostly written in +[Operator](#operator) or [Function](#function)) at runtime. As Apache Flink is mostly written in Java, this corresponds to the definition of *Instance* or *Object* in Java. In the context of Apache Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. #### Flink Job -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially +(See [Restart Pipelined Region Failover Strategy]({{< ref "docs/ops/state/task_failure_recovery" >}}#restart-pipelined-region-failover-strategy)). + +A Job is submitted either by a [Flink Application](#flink-application), by calling `execute()` on an +execution environment, or as a single [Flink SQL Statement](#flink-sql-statement) or [Statement +Set](#statement-set). + +A Flink Job is the runtime representation of a [Logical Graph](#logical-graph) (also often called +*Dataflow Graph*). The Logical Graph is optimized into a [Job Graph](#job-graph), from which the +[Physical Graph](#physical-graph) that actually runs in a [Flink Cluster](#flink-cluster) is derived. #### Flink Job Cluster A Flink Job Cluster is a dedicated [Flink Cluster](#flink-cluster) that only executes a single [Flink Job](#flink-job). The lifetime of the -[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. -This deployment mode has been deprecated since Flink 1.15. +[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. +This deployment mode has been deprecated since Flink 1.15. + +#### Job Graph -#### JobGraph +Also called *JobGraph* or *Optimized Dataflow*. -see [Logical Graph](#logical-graph) +A Job Graph is the optimized representation of a [Logical Graph](#logical-graph), and the +representation that a [Flink Application](#flink-application) submits to the [Flink +Cluster](#flink-cluster). + +Producing the Job Graph is mainly a matter of chaining [Operators](#operator): consecutive +[Operators](#operator) that are not separated by a repartitioning are merged into a single +[Task](#task). The nodes of a Job Graph are therefore [Tasks](#task), each implementing one Operator +or one [Operator Chain](#operator-chain). + +The Job Graph is translated into a [Physical Graph](#physical-graph) for execution. #### Flink JobManager -The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It contains three distinct -components: Flink Resource Manager, Flink Dispatcher and one [Flink JobMaster](#flink-jobmaster) -per running [Flink Job](#flink-job). +Also called *Job Manager*. + +The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It does not process any +data itself: it translates the submitted [Job Graph](#job-graph) into a [Physical +Graph](#physical-graph), schedules the resulting [Sub-Tasks](#sub-task) on the +[TaskManagers](#flink-taskmanager), and coordinates [Checkpoints](#checkpoint) and +[Savepoints](#savepoint). It contains three distinct components: Flink Resource Manager, Flink +Dispatcher and one [Flink JobMaster](#flink-jobmaster) per running [Flink Job](#flink-job). + +See also [Flink Architecture: JobManager]({{< ref "docs/concepts/flink-architecture" >}}#jobmanager). #### Flink JobMaster JobMasters are one of the components running in the [JobManager](#flink-jobmanager). A JobMaster is -responsible for supervising the execution of the [Tasks](#task) of a single job. +responsible for supervising the execution of the [Sub-Tasks](#sub-task) of a single Job. It derives +the [Physical Graph](#physical-graph) from the Job's [Job Graph](#job-graph), requests the slots +needed to run it, deploys the Sub-Tasks to the [TaskManagers](#flink-taskmanager), and triggers the +Job's [Checkpoints](#checkpoint). #### JobResultStore The JobResultStore is a Flink component that persists the results of globally terminated -(i.e. finished, cancelled or failed) jobs to a filesystem, allowing the results to outlive -a finished job. Each result contains the job's identifier, final state, name, the application it -belongs to, etc. These results are then used by Flink to determine whether jobs should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Jobs to a filesystem, allowing the results to outlive +a finished Job. Each result contains the Job's identifier, final state, name, the Application it +belongs to, etc. These results are then used by Flink to determine whether Jobs should +be subject to recovery in highly-available Clusters. + +#### Key Group + +A Key Group is the atomic unit of key distribution and state assignment across parallel +[Sub-Tasks](#sub-task). Every key is mapped deterministically to a Key Group based on +`keyGroupIndex = MathUtils.murmurHash(key.hashCode()) % maxParallelism`. +This allows stateful [Operators](#operator) to rescale without rehashing individual keys. + +The total number of Key Groups is equal to the `maxParallelism` configuration, set at [Job](#flink-job) +level or overridden at [Operator](#operator) level. +A contiguous range of Key Groups is assigned to each [Sub-Task](#sub-task), and Key Groups are evenly +distributed across all [Sub-Tasks](#sub-task). #### Logical Graph -A logical graph is a directed graph where the nodes are [Operators](#operator) -and the edges define input/output-relationships of the operators and correspond -to data streams or data sets. A logical graph is created by submitting jobs -from a [Flink Application](#flink-application). +A Logical Graph is a Directed Acyclic Graph (DAG) where the nodes are [Operators](#operator) +and the edges define input/output relationships of the Operators and correspond +to data streams or data sets. A Logical Graph is created by submitting Jobs +from a [Flink Application](#flink-application). For the Table API and SQL, the Logical Graph is the +result of parsing and optimizing the [Table Program](#table-program) in the table planner. -Logical graphs are also often referred to as *dataflow graphs*. +Logical Graphs are also often referred to as *Dataflow Graphs* or, for the DataStream API, as +*StreamGraphs*. A Logical Graph is optimized into a [Job Graph](#job-graph) before execution. #### Managed State -Managed State describes application state which has been registered with the framework. For -Managed State, Apache Flink will take care about persistence and rescaling among other things. +Managed State describes Application State which has been registered with the framework. This includes +both [keyed state]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#using-keyed-state) and +non-keyed state (also known as [Operator State]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#operator-state)). +For Managed State, Apache Flink takes care of persistence and rescaling, among other things. #### Operator -Node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, which is -usually executed by a [Function](#function). Sources and Sinks are special Operators for data -ingestion and data egress. +A node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, such as a join, +an aggregation or a stateless transformation, which is usually executed by a [Function](#function). + +Sources and Sinks are special Operators for data ingestion and data egress: a Logical Graph always +begins with one or more Source Operators and ends with one or more Sink Operators. + +Note that parts of the Flink documentation and of the Web UI use the term *Operator* loosely, also +referring to a [Task](#task) or a [Sub-Task](#sub-task), leaving the precise meaning to be inferred +from the context. #### Operator Chain An Operator Chain consists of two or more consecutive [Operators](#operator) without any repartitioning in between. Operators within the same Operator Chain forward records to each other -directly without going through serialization or Flink's network stack. +directly without going through serialization or Flink's network stack, which removes the overhead of +the handover between them. + +An Operator Chain becomes a single [Task](#task) in the [Job Graph](#job-graph). Chains are +recognizable in graphical representations of the Job Graph, such as the Flink Web UI, because the +name of the Task is the composition of the names of the chained Operators. + +See also [Flink Architecture: Tasks and Operator Chains]({{< ref "docs/concepts/flink-architecture" >}}#tasks-and-operator-chains). + +#### Parallelism + +The number of parallel flows Flink uses to process the data, and therefore the way a [Flink Review Comment: `Parallel flows` is not very clear - as Dataflow has already been used earlier. Parallel execution, or number of sub-task instances seems clearer to me - the idea that a Task can be split into multiple instances, where each sub-task processes a subset of the data. ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the +[Transformations](#transformation) they implement. In the Table API and SQL, they are declared +separately as [User-Defined Functions]({{< ref "docs/dev/table/functions/udfs" >}}) (UDF) or +[Process Table Functions]({{< ref "docs/dev/table/functions/ptfs" >}}) (PTF). #### History Server The History Server is a standalone service that serves the detailed history of completed Flink -applications and jobs, using archives generated by the JobManager. Unlike the -[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store -minimal metadata for internal recovery decisions in highly-available clusters, the History Server -provides detailed archives for analysis via Web UI or REST API after the cluster has been shut down. +Applications and Jobs, using archives generated by the JobManager. Unlike the +[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store +minimal metadata for internal recovery decisions in highly-available Clusters, the History Server +provides detailed archives for analysis via Web UI or REST API after the Cluster has been shut down. #### Instance The term *instance* is used to describe a specific instance of a specific type (usually -[Operator](#operator) or [Function](#function)) during runtime. As Apache Flink is mostly written in +[Operator](#operator) or [Function](#function)) at runtime. As Apache Flink is mostly written in Java, this corresponds to the definition of *Instance* or *Object* in Java. In the context of Apache Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. #### Flink Job -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially +(See [Restart Pipelined Region Failover Strategy]({{< ref "docs/ops/state/task_failure_recovery" >}}#restart-pipelined-region-failover-strategy)). + +A Job is submitted either by a [Flink Application](#flink-application), by calling `execute()` on an +execution environment, or as a single [Flink SQL Statement](#flink-sql-statement) or [Statement +Set](#statement-set). + +A Flink Job is the runtime representation of a [Logical Graph](#logical-graph) (also often called +*Dataflow Graph*). The Logical Graph is optimized into a [Job Graph](#job-graph), from which the +[Physical Graph](#physical-graph) that actually runs in a [Flink Cluster](#flink-cluster) is derived. #### Flink Job Cluster A Flink Job Cluster is a dedicated [Flink Cluster](#flink-cluster) that only executes a single [Flink Job](#flink-job). The lifetime of the -[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. -This deployment mode has been deprecated since Flink 1.15. +[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. +This deployment mode has been deprecated since Flink 1.15. + +#### Job Graph -#### JobGraph +Also called *JobGraph* or *Optimized Dataflow*. -see [Logical Graph](#logical-graph) +A Job Graph is the optimized representation of a [Logical Graph](#logical-graph), and the +representation that a [Flink Application](#flink-application) submits to the [Flink +Cluster](#flink-cluster). + +Producing the Job Graph is mainly a matter of chaining [Operators](#operator): consecutive +[Operators](#operator) that are not separated by a repartitioning are merged into a single +[Task](#task). The nodes of a Job Graph are therefore [Tasks](#task), each implementing one Operator +or one [Operator Chain](#operator-chain). + +The Job Graph is translated into a [Physical Graph](#physical-graph) for execution. #### Flink JobManager -The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It contains three distinct -components: Flink Resource Manager, Flink Dispatcher and one [Flink JobMaster](#flink-jobmaster) -per running [Flink Job](#flink-job). +Also called *Job Manager*. + +The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It does not process any +data itself: it translates the submitted [Job Graph](#job-graph) into a [Physical +Graph](#physical-graph), schedules the resulting [Sub-Tasks](#sub-task) on the +[TaskManagers](#flink-taskmanager), and coordinates [Checkpoints](#checkpoint) and +[Savepoints](#savepoint). It contains three distinct components: Flink Resource Manager, Flink +Dispatcher and one [Flink JobMaster](#flink-jobmaster) per running [Flink Job](#flink-job). + +See also [Flink Architecture: JobManager]({{< ref "docs/concepts/flink-architecture" >}}#jobmanager). #### Flink JobMaster JobMasters are one of the components running in the [JobManager](#flink-jobmanager). A JobMaster is -responsible for supervising the execution of the [Tasks](#task) of a single job. +responsible for supervising the execution of the [Sub-Tasks](#sub-task) of a single Job. It derives +the [Physical Graph](#physical-graph) from the Job's [Job Graph](#job-graph), requests the slots +needed to run it, deploys the Sub-Tasks to the [TaskManagers](#flink-taskmanager), and triggers the +Job's [Checkpoints](#checkpoint). #### JobResultStore The JobResultStore is a Flink component that persists the results of globally terminated -(i.e. finished, cancelled or failed) jobs to a filesystem, allowing the results to outlive -a finished job. Each result contains the job's identifier, final state, name, the application it -belongs to, etc. These results are then used by Flink to determine whether jobs should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Jobs to a filesystem, allowing the results to outlive +a finished Job. Each result contains the Job's identifier, final state, name, the Application it +belongs to, etc. These results are then used by Flink to determine whether Jobs should +be subject to recovery in highly-available Clusters. + +#### Key Group + +A Key Group is the atomic unit of key distribution and state assignment across parallel +[Sub-Tasks](#sub-task). Every key is mapped deterministically to a Key Group based on +`keyGroupIndex = MathUtils.murmurHash(key.hashCode()) % maxParallelism`. +This allows stateful [Operators](#operator) to rescale without rehashing individual keys. + +The total number of Key Groups is equal to the `maxParallelism` configuration, set at [Job](#flink-job) +level or overridden at [Operator](#operator) level. +A contiguous range of Key Groups is assigned to each [Sub-Task](#sub-task), and Key Groups are evenly +distributed across all [Sub-Tasks](#sub-task). #### Logical Graph -A logical graph is a directed graph where the nodes are [Operators](#operator) -and the edges define input/output-relationships of the operators and correspond -to data streams or data sets. A logical graph is created by submitting jobs -from a [Flink Application](#flink-application). +A Logical Graph is a Directed Acyclic Graph (DAG) where the nodes are [Operators](#operator) +and the edges define input/output relationships of the Operators and correspond +to data streams or data sets. A Logical Graph is created by submitting Jobs +from a [Flink Application](#flink-application). For the Table API and SQL, the Logical Graph is the +result of parsing and optimizing the [Table Program](#table-program) in the table planner. -Logical graphs are also often referred to as *dataflow graphs*. +Logical Graphs are also often referred to as *Dataflow Graphs* or, for the DataStream API, as +*StreamGraphs*. A Logical Graph is optimized into a [Job Graph](#job-graph) before execution. #### Managed State -Managed State describes application state which has been registered with the framework. For -Managed State, Apache Flink will take care about persistence and rescaling among other things. +Managed State describes Application State which has been registered with the framework. This includes +both [keyed state]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#using-keyed-state) and +non-keyed state (also known as [Operator State]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#operator-state)). +For Managed State, Apache Flink takes care of persistence and rescaling, among other things. #### Operator -Node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, which is -usually executed by a [Function](#function). Sources and Sinks are special Operators for data -ingestion and data egress. +A node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, such as a join, +an aggregation or a stateless transformation, which is usually executed by a [Function](#function). + +Sources and Sinks are special Operators for data ingestion and data egress: a Logical Graph always +begins with one or more Source Operators and ends with one or more Sink Operators. + +Note that parts of the Flink documentation and of the Web UI use the term *Operator* loosely, also +referring to a [Task](#task) or a [Sub-Task](#sub-task), leaving the precise meaning to be inferred +from the context. #### Operator Chain An Operator Chain consists of two or more consecutive [Operators](#operator) without any repartitioning in between. Operators within the same Operator Chain forward records to each other -directly without going through serialization or Flink's network stack. +directly without going through serialization or Flink's network stack, which removes the overhead of +the handover between them. + +An Operator Chain becomes a single [Task](#task) in the [Job Graph](#job-graph). Chains are +recognizable in graphical representations of the Job Graph, such as the Flink Web UI, because the +name of the Task is the composition of the names of the chained Operators. + +See also [Flink Architecture: Tasks and Operator Chains]({{< ref "docs/concepts/flink-architecture" >}}#tasks-and-operator-chains). + +#### Parallelism + +The number of parallel flows Flink uses to process the data, and therefore the way a [Flink +Job](#flink-job) scales horizontally. The Parallelism of an [Operator](#operator) determines the +number of [Sub-Tasks](#sub-task) and of [Physical Partitions](#partition) it is executed with. + +The *Job Parallelism* is the default Parallelism of all Operators of a Job. The *Operator +Parallelism* may override it for an individual Operator. + +Parallelism is a property of the Job, independent of the number of [Flink +TaskManagers](#flink-taskmanager) in the [Flink Cluster](#flink-cluster). #### Partition -A partition is an independent subset of the overall data stream or data set. A data stream or -data set is divided into partitions by assigning each [record](#Record) to one or more partitions. -Partitions of data streams or data sets are consumed by [Tasks](#task) during runtime. A -transformation which changes the way a data stream or data set is partitioned is often called -repartitioning. +A Partition is an independent subset of the overall data stream or data set. A data stream or +data set is divided into Partitions by assigning each record to one or more Partitions. +A [Transformation](#transformation) which changes the way a data stream or data set is partitioned is +often called repartitioning. + +*Logical Partitioning* is how records and State are divided in the [Logical +Graph](#logical-graph) and the [Job Graph](#job-graph), in order to implement the semantics of an +operation. A `JOIN` or `GROUP BY` in SQL, or a `keyBy()` in the DataStream API, for example, requires the +data to be logically partitioned by a key, and the number of Logical Partitions is then the number of +distinct keys. Keyed State is isolated per Logical Partition: a [Function](#function) can only access +the State of the key of the record or timer it is currently processing. Operator State and Broadcast +State, in contrast, are not keyed. + +*Physical Partitioning* is how records and State are divided in the [Physical +Graph](#physical-graph), across the [Sub-Tasks](#sub-task) that Flink executes in parallel. Each +Sub-Task handles exactly one Physical Partition and holds only the State belonging to it, so the +number of Physical Partitions equals the [Parallelism](#parallelism) of the [Operators](#operator) +the Sub-Task implements. Physical Partitioning follows from Logical Partitioning: in a stream +partitioned by key, each Physical Partition holds a fixed subset of the keys. + +Note that the Flink documentation uses the word *partition* both for these internal Partitions and +for the partitions of an external system, such as the Kafka partitions of a source topic. The +intended meaning has to be inferred from the context. #### Physical Graph -A physical graph is the result of translating a [Logical Graph](#logical-graph) for execution in a -distributed runtime. The nodes are [Tasks](#task) and the edges indicate input/output-relationships -or [partitions](#partition) of data streams or data sets. +A Physical Graph is the result of translating a [Job Graph](#job-graph) for execution in a +distributed runtime, taking [Parallelism](#parallelism) into account. The nodes are +[Sub-Tasks](#sub-task) and the edges are the [Channels](#channel) connecting them. + +Physical Graphs are also referred to as *Parallel Dataflows* or as *ExecutionGraphs*. #### Record Records are the constituent elements of a data set or data stream. [Operators](#operator) and -[Functions](#Function) receive records as input and emit records as output. +[Functions](#function) receive records as input and emit records as output. #### (Runtime) Execution Mode -DataStream API programs can be executed in one of two execution modes: `BATCH` -or `STREAMING`. See [Execution Mode]({{< ref "/docs/dev/datastream/execution_mode" >}}) for more details. +DataStream API programs can be executed in one of two Execution Modes: `BATCH` +or `STREAMING`. See [Execution Mode]({{< ref "docs/dev/datastream/execution_mode" >}}) for more details. + +In `STREAMING` mode, Flink processes unbounded data as it arrives, uses [Watermarks](#watermark) to implement +event-time semantics, keeps State in the [State Backend](#state-backend) and relies on +[Checkpoints](#checkpoint) for fault tolerance. + +In `BATCH` mode, Flink processes a bounded data set with a known beginning and end. Operators may +consume their entire input before emitting any output, and Watermarks are not used for event-time +semantics. The configured State Backend is ignored: the input of a keyed operation is instead grouped +by key through sorting, so that Flink only has to hold the State of one key at a time, spilling to +local disk when memory is insufficient. + +Note that a bounded data set can also be processed in `STREAMING` mode, for example by setting Review Comment: Also note that it only applies to traditionally unbounded data sources ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the +[Transformations](#transformation) they implement. In the Table API and SQL, they are declared +separately as [User-Defined Functions]({{< ref "docs/dev/table/functions/udfs" >}}) (UDF) or +[Process Table Functions]({{< ref "docs/dev/table/functions/ptfs" >}}) (PTF). #### History Server The History Server is a standalone service that serves the detailed history of completed Flink -applications and jobs, using archives generated by the JobManager. Unlike the -[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store -minimal metadata for internal recovery decisions in highly-available clusters, the History Server -provides detailed archives for analysis via Web UI or REST API after the cluster has been shut down. +Applications and Jobs, using archives generated by the JobManager. Unlike the +[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store +minimal metadata for internal recovery decisions in highly-available Clusters, the History Server +provides detailed archives for analysis via Web UI or REST API after the Cluster has been shut down. #### Instance The term *instance* is used to describe a specific instance of a specific type (usually -[Operator](#operator) or [Function](#function)) during runtime. As Apache Flink is mostly written in +[Operator](#operator) or [Function](#function)) at runtime. As Apache Flink is mostly written in Java, this corresponds to the definition of *Instance* or *Object* in Java. In the context of Apache Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. #### Flink Job -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially +(See [Restart Pipelined Region Failover Strategy]({{< ref "docs/ops/state/task_failure_recovery" >}}#restart-pipelined-region-failover-strategy)). + +A Job is submitted either by a [Flink Application](#flink-application), by calling `execute()` on an +execution environment, or as a single [Flink SQL Statement](#flink-sql-statement) or [Statement +Set](#statement-set). + +A Flink Job is the runtime representation of a [Logical Graph](#logical-graph) (also often called +*Dataflow Graph*). The Logical Graph is optimized into a [Job Graph](#job-graph), from which the +[Physical Graph](#physical-graph) that actually runs in a [Flink Cluster](#flink-cluster) is derived. #### Flink Job Cluster A Flink Job Cluster is a dedicated [Flink Cluster](#flink-cluster) that only executes a single [Flink Job](#flink-job). The lifetime of the -[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. -This deployment mode has been deprecated since Flink 1.15. +[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. +This deployment mode has been deprecated since Flink 1.15. + +#### Job Graph -#### JobGraph +Also called *JobGraph* or *Optimized Dataflow*. -see [Logical Graph](#logical-graph) +A Job Graph is the optimized representation of a [Logical Graph](#logical-graph), and the +representation that a [Flink Application](#flink-application) submits to the [Flink +Cluster](#flink-cluster). + +Producing the Job Graph is mainly a matter of chaining [Operators](#operator): consecutive +[Operators](#operator) that are not separated by a repartitioning are merged into a single +[Task](#task). The nodes of a Job Graph are therefore [Tasks](#task), each implementing one Operator +or one [Operator Chain](#operator-chain). + +The Job Graph is translated into a [Physical Graph](#physical-graph) for execution. #### Flink JobManager -The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It contains three distinct -components: Flink Resource Manager, Flink Dispatcher and one [Flink JobMaster](#flink-jobmaster) -per running [Flink Job](#flink-job). +Also called *Job Manager*. + +The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It does not process any +data itself: it translates the submitted [Job Graph](#job-graph) into a [Physical +Graph](#physical-graph), schedules the resulting [Sub-Tasks](#sub-task) on the +[TaskManagers](#flink-taskmanager), and coordinates [Checkpoints](#checkpoint) and +[Savepoints](#savepoint). It contains three distinct components: Flink Resource Manager, Flink +Dispatcher and one [Flink JobMaster](#flink-jobmaster) per running [Flink Job](#flink-job). + +See also [Flink Architecture: JobManager]({{< ref "docs/concepts/flink-architecture" >}}#jobmanager). #### Flink JobMaster JobMasters are one of the components running in the [JobManager](#flink-jobmanager). A JobMaster is -responsible for supervising the execution of the [Tasks](#task) of a single job. +responsible for supervising the execution of the [Sub-Tasks](#sub-task) of a single Job. It derives +the [Physical Graph](#physical-graph) from the Job's [Job Graph](#job-graph), requests the slots +needed to run it, deploys the Sub-Tasks to the [TaskManagers](#flink-taskmanager), and triggers the +Job's [Checkpoints](#checkpoint). #### JobResultStore The JobResultStore is a Flink component that persists the results of globally terminated -(i.e. finished, cancelled or failed) jobs to a filesystem, allowing the results to outlive -a finished job. Each result contains the job's identifier, final state, name, the application it -belongs to, etc. These results are then used by Flink to determine whether jobs should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Jobs to a filesystem, allowing the results to outlive +a finished Job. Each result contains the Job's identifier, final state, name, the Application it +belongs to, etc. These results are then used by Flink to determine whether Jobs should +be subject to recovery in highly-available Clusters. + +#### Key Group + +A Key Group is the atomic unit of key distribution and state assignment across parallel +[Sub-Tasks](#sub-task). Every key is mapped deterministically to a Key Group based on +`keyGroupIndex = MathUtils.murmurHash(key.hashCode()) % maxParallelism`. +This allows stateful [Operators](#operator) to rescale without rehashing individual keys. + +The total number of Key Groups is equal to the `maxParallelism` configuration, set at [Job](#flink-job) +level or overridden at [Operator](#operator) level. +A contiguous range of Key Groups is assigned to each [Sub-Task](#sub-task), and Key Groups are evenly +distributed across all [Sub-Tasks](#sub-task). #### Logical Graph -A logical graph is a directed graph where the nodes are [Operators](#operator) -and the edges define input/output-relationships of the operators and correspond -to data streams or data sets. A logical graph is created by submitting jobs -from a [Flink Application](#flink-application). +A Logical Graph is a Directed Acyclic Graph (DAG) where the nodes are [Operators](#operator) +and the edges define input/output relationships of the Operators and correspond +to data streams or data sets. A Logical Graph is created by submitting Jobs +from a [Flink Application](#flink-application). For the Table API and SQL, the Logical Graph is the +result of parsing and optimizing the [Table Program](#table-program) in the table planner. -Logical graphs are also often referred to as *dataflow graphs*. +Logical Graphs are also often referred to as *Dataflow Graphs* or, for the DataStream API, as +*StreamGraphs*. A Logical Graph is optimized into a [Job Graph](#job-graph) before execution. #### Managed State -Managed State describes application state which has been registered with the framework. For -Managed State, Apache Flink will take care about persistence and rescaling among other things. +Managed State describes Application State which has been registered with the framework. This includes +both [keyed state]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#using-keyed-state) and +non-keyed state (also known as [Operator State]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#operator-state)). +For Managed State, Apache Flink takes care of persistence and rescaling, among other things. #### Operator -Node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, which is -usually executed by a [Function](#function). Sources and Sinks are special Operators for data -ingestion and data egress. +A node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, such as a join, +an aggregation or a stateless transformation, which is usually executed by a [Function](#function). + +Sources and Sinks are special Operators for data ingestion and data egress: a Logical Graph always +begins with one or more Source Operators and ends with one or more Sink Operators. + +Note that parts of the Flink documentation and of the Web UI use the term *Operator* loosely, also +referring to a [Task](#task) or a [Sub-Task](#sub-task), leaving the precise meaning to be inferred +from the context. #### Operator Chain An Operator Chain consists of two or more consecutive [Operators](#operator) without any repartitioning in between. Operators within the same Operator Chain forward records to each other -directly without going through serialization or Flink's network stack. +directly without going through serialization or Flink's network stack, which removes the overhead of +the handover between them. + +An Operator Chain becomes a single [Task](#task) in the [Job Graph](#job-graph). Chains are +recognizable in graphical representations of the Job Graph, such as the Flink Web UI, because the +name of the Task is the composition of the names of the chained Operators. + +See also [Flink Architecture: Tasks and Operator Chains]({{< ref "docs/concepts/flink-architecture" >}}#tasks-and-operator-chains). + +#### Parallelism + +The number of parallel flows Flink uses to process the data, and therefore the way a [Flink +Job](#flink-job) scales horizontally. The Parallelism of an [Operator](#operator) determines the +number of [Sub-Tasks](#sub-task) and of [Physical Partitions](#partition) it is executed with. + +The *Job Parallelism* is the default Parallelism of all Operators of a Job. The *Operator +Parallelism* may override it for an individual Operator. + +Parallelism is a property of the Job, independent of the number of [Flink +TaskManagers](#flink-taskmanager) in the [Flink Cluster](#flink-cluster). #### Partition -A partition is an independent subset of the overall data stream or data set. A data stream or -data set is divided into partitions by assigning each [record](#Record) to one or more partitions. -Partitions of data streams or data sets are consumed by [Tasks](#task) during runtime. A -transformation which changes the way a data stream or data set is partitioned is often called -repartitioning. +A Partition is an independent subset of the overall data stream or data set. A data stream or Review Comment: Could this reference parallelism as well? Assuming we can talk about each parallel task processing 1 or more partitions? ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. Review Comment: ```suggestion are enabled, it may also contain data in-flight between Sub-Tasks. ``` Consistency note ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` Review Comment: Is this necessary for a glossary? It seems a bit too fine-grained. ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the +[Transformations](#transformation) they implement. In the Table API and SQL, they are declared +separately as [User-Defined Functions]({{< ref "docs/dev/table/functions/udfs" >}}) (UDF) or +[Process Table Functions]({{< ref "docs/dev/table/functions/ptfs" >}}) (PTF). #### History Server The History Server is a standalone service that serves the detailed history of completed Flink -applications and jobs, using archives generated by the JobManager. Unlike the -[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store -minimal metadata for internal recovery decisions in highly-available clusters, the History Server -provides detailed archives for analysis via Web UI or REST API after the cluster has been shut down. +Applications and Jobs, using archives generated by the JobManager. Unlike the +[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store +minimal metadata for internal recovery decisions in highly-available Clusters, the History Server +provides detailed archives for analysis via Web UI or REST API after the Cluster has been shut down. #### Instance The term *instance* is used to describe a specific instance of a specific type (usually -[Operator](#operator) or [Function](#function)) during runtime. As Apache Flink is mostly written in +[Operator](#operator) or [Function](#function)) at runtime. As Apache Flink is mostly written in Java, this corresponds to the definition of *Instance* or *Object* in Java. In the context of Apache Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. #### Flink Job -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially +(See [Restart Pipelined Region Failover Strategy]({{< ref "docs/ops/state/task_failure_recovery" >}}#restart-pipelined-region-failover-strategy)). + +A Job is submitted either by a [Flink Application](#flink-application), by calling `execute()` on an +execution environment, or as a single [Flink SQL Statement](#flink-sql-statement) or [Statement +Set](#statement-set). + +A Flink Job is the runtime representation of a [Logical Graph](#logical-graph) (also often called +*Dataflow Graph*). The Logical Graph is optimized into a [Job Graph](#job-graph), from which the +[Physical Graph](#physical-graph) that actually runs in a [Flink Cluster](#flink-cluster) is derived. #### Flink Job Cluster A Flink Job Cluster is a dedicated [Flink Cluster](#flink-cluster) that only executes a single [Flink Job](#flink-job). The lifetime of the -[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. -This deployment mode has been deprecated since Flink 1.15. +[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. +This deployment mode has been deprecated since Flink 1.15. + +#### Job Graph -#### JobGraph +Also called *JobGraph* or *Optimized Dataflow*. -see [Logical Graph](#logical-graph) +A Job Graph is the optimized representation of a [Logical Graph](#logical-graph), and the +representation that a [Flink Application](#flink-application) submits to the [Flink +Cluster](#flink-cluster). + +Producing the Job Graph is mainly a matter of chaining [Operators](#operator): consecutive +[Operators](#operator) that are not separated by a repartitioning are merged into a single +[Task](#task). The nodes of a Job Graph are therefore [Tasks](#task), each implementing one Operator +or one [Operator Chain](#operator-chain). + +The Job Graph is translated into a [Physical Graph](#physical-graph) for execution. #### Flink JobManager -The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It contains three distinct -components: Flink Resource Manager, Flink Dispatcher and one [Flink JobMaster](#flink-jobmaster) -per running [Flink Job](#flink-job). +Also called *Job Manager*. + +The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It does not process any +data itself: it translates the submitted [Job Graph](#job-graph) into a [Physical +Graph](#physical-graph), schedules the resulting [Sub-Tasks](#sub-task) on the +[TaskManagers](#flink-taskmanager), and coordinates [Checkpoints](#checkpoint) and +[Savepoints](#savepoint). It contains three distinct components: Flink Resource Manager, Flink +Dispatcher and one [Flink JobMaster](#flink-jobmaster) per running [Flink Job](#flink-job). + +See also [Flink Architecture: JobManager]({{< ref "docs/concepts/flink-architecture" >}}#jobmanager). #### Flink JobMaster JobMasters are one of the components running in the [JobManager](#flink-jobmanager). A JobMaster is -responsible for supervising the execution of the [Tasks](#task) of a single job. +responsible for supervising the execution of the [Sub-Tasks](#sub-task) of a single Job. It derives +the [Physical Graph](#physical-graph) from the Job's [Job Graph](#job-graph), requests the slots +needed to run it, deploys the Sub-Tasks to the [TaskManagers](#flink-taskmanager), and triggers the +Job's [Checkpoints](#checkpoint). #### JobResultStore The JobResultStore is a Flink component that persists the results of globally terminated -(i.e. finished, cancelled or failed) jobs to a filesystem, allowing the results to outlive -a finished job. Each result contains the job's identifier, final state, name, the application it -belongs to, etc. These results are then used by Flink to determine whether jobs should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Jobs to a filesystem, allowing the results to outlive +a finished Job. Each result contains the Job's identifier, final state, name, the Application it +belongs to, etc. These results are then used by Flink to determine whether Jobs should +be subject to recovery in highly-available Clusters. Review Comment: Is there a link here to ApplicationResultStore? Or a note on how they are related? In other words - explaining the difference between the two - particularly, in the case where an application submits a single job - what's the difference (if any) between values stored in the stores? ########## docs/content/docs/concepts/glossary.md: ########## @@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the Flink Application. #### ApplicationResultStore The ApplicationResultStore is a Flink component that persists the results of terminated -(i.e. finished, cancelled or failed) applications to a filesystem, allowing the results to outlive -a terminated application. Each result contains the application's identifier, final state, name, -etc. These results are then used by Flink to determine whether applications should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Applications to a filesystem, allowing the results to outlive +a terminated Application. Each result contains the Application's identifier, final state, name, +etc. These results are then used by Flink to determine whether Applications should +be subject to recovery in highly-available Clusters. + +#### Channel + +Also called *Stream Partitions*. + +A Channel is the physical link between a [Sub-Task](#sub-task) and a downstream Sub-Task, and the +edge of a [Physical Graph](#physical-graph). Parts of the documentation refer to Channels as *Stream +Partitions*, in the sense of internal, physical Partitions. + +Channels carry data records as well as signals such as [Watermarks](#watermark), Watermark Status +updates and Checkpoint barriers. Transmission over a Channel is always unidirectional (upstream to +downstream) and asynchronous. + +A Sub-Task may have one or more input Channels and one or more output Channels. Source Sub-Tasks have +no input Channels, since they begin the graph, and Sink Sub-Tasks have no output Channels, since they +end it. + +A Sub-Task routes each record to one of its output Channels according to the [Physical +Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the DataStream API, `GROUP +BY` in SQL) routes a record to the Channel connected to the downstream Sub-Task that handles the +record's key, whereas `rebalance()` or `rescale()` may round-robin records across output Channels. + +A Channel is *local* when both Sub-Tasks run in the same [Flink +TaskManager](#flink-taskmanager), in which case records are handed over through an in-memory buffer, +or *remote* when the Sub-Tasks run in different TaskManagers, in which case the data crosses the +network. + +#### Checkpoint + +A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical point in time, taken +with a variant of the Chandy-Lamport algorithm and written to [Checkpoint +Storage](#checkpoint-storage). + +A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator). This also +includes source positions (for example Kafka partition offsets), assignment of [Source Splits](#source-split) +to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight data and buffered data +of some asynchronous Sink connectors are also part of the [Operator](#operator) [State](#managed-state) +and are saved in the Checkpoint. +When [Unaligned Checkpoints]({{< ref "docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing) +are enabled, it may also contain data in flight between Sub-Tasks. + +Checkpoints are triggered automatically and periodically while the Job is running, and are used to +recover from failures such as a TaskManager crash or a network problem: the Job restarts from the +latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously, +without blocking record processing, apart from a synchronous phase in each Sub-Task. +Transactional [Sources and Sinks](#operator) tie their transactions to the Checkpoint; the +Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint completes. + +Checkpoints are only used in the `STREAMING` [Execution Mode](#runtime-execution-mode). In `BATCH` +mode, Flink recovers instead by backtracking to previous processing stages whose intermediate results +are still available, so that potentially only the failed [Tasks](#task) and their predecessors are +restarted. As a consequence, Sinks that rely on Checkpoints to commit their transactions do not work +in `BATCH` mode unless they are implemented with the Unified Sink API, which commits once the whole +input has been processed. + +Compare to [Savepoint](#savepoint). + +Checkpoints and [Savepoints](#savepoint) are also referred to, collectively, as *State Snapshots* or +*Snapshots*. #### Checkpoint Storage -The location where the [State Backend](#state-backend) will store its snapshot during a checkpoint (Java Heap of [JobManager](#flink-jobmanager) or Filesystem). +The durable location where [Checkpoints](#checkpoint) and [Savepoints](#savepoint) are saved. It can +be either the Java Heap of the [Flink JobManager](#flink-jobmanager) or a filesystem. Production +deployments use a filesystem, typically remote object storage, since Checkpoint Storage is what makes +State survive the loss of a [TaskManager](#flink-taskmanager) or of the whole [Flink Cluster](#flink-cluster). + +The relationship between the State Backend and Checkpoint Storage changes with [Disaggregated +State]({{< ref "docs/ops/state/disaggregated_state" >}}), where remote storage becomes the primary +location of the State and the local State Backend acts as a cache, the two being synchronized +asynchronously. #### Flink Cluster A distributed system consisting of (typically) one [JobManager](#flink-jobmanager) and one or more -[Flink TaskManager](#flink-taskmanager) processes. +[Flink TaskManager](#flink-taskmanager) processes. Each of these processes runs in a separate JVM, +usually on a separate container or machine, although this is not a requirement. + +See also [Flink Architecture: Anatomy of a Flink Cluster]({{< ref "docs/concepts/flink-architecture" >}}#anatomy-of-a-flink-cluster). #### Event -An event is a statement about a change of the state of the domain modelled by the -application. Events can be input and/or output of a stream or batch processing application. -Events are special types of [records](#Record). +An Event is a statement about a change of the state of the domain modeled by the +Application. Events can be input and/or output of a stream or batch processing Application. +Events are special types of records. + +#### Execution Graph -#### ExecutionGraph +Also called *ExecutionGraph*. -see [Physical Graph](#physical-graph) +See [Physical Graph](#physical-graph) #### Function -Functions are implemented by the user and encapsulate the +Functions are implemented by the user, in Java or Python, and encapsulate the application logic of a Flink program. Most Functions are wrapped by a corresponding -[Operator](#operator). +[Operator](#operator). In the DataStream API, Functions are passed to the +[Transformations](#transformation) they implement. In the Table API and SQL, they are declared +separately as [User-Defined Functions]({{< ref "docs/dev/table/functions/udfs" >}}) (UDF) or +[Process Table Functions]({{< ref "docs/dev/table/functions/ptfs" >}}) (PTF). #### History Server The History Server is a standalone service that serves the detailed history of completed Flink -applications and jobs, using archives generated by the JobManager. Unlike the -[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store -minimal metadata for internal recovery decisions in highly-available clusters, the History Server -provides detailed archives for analysis via Web UI or REST API after the cluster has been shut down. +Applications and Jobs, using archives generated by the JobManager. Unlike the +[ApplicationResultStore](#applicationresultstore) and [JobResultStore](#jobresultstore), which store +minimal metadata for internal recovery decisions in highly-available Clusters, the History Server +provides detailed archives for analysis via Web UI or REST API after the Cluster has been shut down. #### Instance The term *instance* is used to describe a specific instance of a specific type (usually -[Operator](#operator) or [Function](#function)) during runtime. As Apache Flink is mostly written in +[Operator](#operator) or [Function](#function)) at runtime. As Apache Flink is mostly written in Java, this corresponds to the definition of *Instance* or *Object* in Java. In the context of Apache Flink, the term *parallel instance* is also frequently used to emphasize that multiple instances of the same [Operator](#operator) or [Function](#function) type are running in parallel. #### Flink Job -A Flink Job is the runtime representation of a [logical graph](#logical-graph) -(also often called dataflow graph) that is created and submitted by calling -`execute()` in a [Flink Application](#flink-application). +A Flink Job is the unit of data processing execution in Flink: a Job as a whole is submitted, +started, stopped and resumed, although under some conditions Flink may restart a Job only partially +(See [Restart Pipelined Region Failover Strategy]({{< ref "docs/ops/state/task_failure_recovery" >}}#restart-pipelined-region-failover-strategy)). + +A Job is submitted either by a [Flink Application](#flink-application), by calling `execute()` on an +execution environment, or as a single [Flink SQL Statement](#flink-sql-statement) or [Statement +Set](#statement-set). + +A Flink Job is the runtime representation of a [Logical Graph](#logical-graph) (also often called +*Dataflow Graph*). The Logical Graph is optimized into a [Job Graph](#job-graph), from which the +[Physical Graph](#physical-graph) that actually runs in a [Flink Cluster](#flink-cluster) is derived. #### Flink Job Cluster A Flink Job Cluster is a dedicated [Flink Cluster](#flink-cluster) that only executes a single [Flink Job](#flink-job). The lifetime of the -[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. -This deployment mode has been deprecated since Flink 1.15. +[Flink Cluster](#flink-cluster) is bound to the lifetime of the Flink Job. +This deployment mode has been deprecated since Flink 1.15. + +#### Job Graph -#### JobGraph +Also called *JobGraph* or *Optimized Dataflow*. -see [Logical Graph](#logical-graph) +A Job Graph is the optimized representation of a [Logical Graph](#logical-graph), and the +representation that a [Flink Application](#flink-application) submits to the [Flink +Cluster](#flink-cluster). + +Producing the Job Graph is mainly a matter of chaining [Operators](#operator): consecutive +[Operators](#operator) that are not separated by a repartitioning are merged into a single +[Task](#task). The nodes of a Job Graph are therefore [Tasks](#task), each implementing one Operator +or one [Operator Chain](#operator-chain). + +The Job Graph is translated into a [Physical Graph](#physical-graph) for execution. #### Flink JobManager -The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It contains three distinct -components: Flink Resource Manager, Flink Dispatcher and one [Flink JobMaster](#flink-jobmaster) -per running [Flink Job](#flink-job). +Also called *Job Manager*. + +The JobManager is the orchestrator of a [Flink Cluster](#flink-cluster). It does not process any +data itself: it translates the submitted [Job Graph](#job-graph) into a [Physical +Graph](#physical-graph), schedules the resulting [Sub-Tasks](#sub-task) on the +[TaskManagers](#flink-taskmanager), and coordinates [Checkpoints](#checkpoint) and +[Savepoints](#savepoint). It contains three distinct components: Flink Resource Manager, Flink +Dispatcher and one [Flink JobMaster](#flink-jobmaster) per running [Flink Job](#flink-job). + +See also [Flink Architecture: JobManager]({{< ref "docs/concepts/flink-architecture" >}}#jobmanager). #### Flink JobMaster JobMasters are one of the components running in the [JobManager](#flink-jobmanager). A JobMaster is -responsible for supervising the execution of the [Tasks](#task) of a single job. +responsible for supervising the execution of the [Sub-Tasks](#sub-task) of a single Job. It derives +the [Physical Graph](#physical-graph) from the Job's [Job Graph](#job-graph), requests the slots +needed to run it, deploys the Sub-Tasks to the [TaskManagers](#flink-taskmanager), and triggers the +Job's [Checkpoints](#checkpoint). #### JobResultStore The JobResultStore is a Flink component that persists the results of globally terminated -(i.e. finished, cancelled or failed) jobs to a filesystem, allowing the results to outlive -a finished job. Each result contains the job's identifier, final state, name, the application it -belongs to, etc. These results are then used by Flink to determine whether jobs should -be subject to recovery in highly-available clusters. +(i.e. finished, cancelled or failed) Jobs to a filesystem, allowing the results to outlive +a finished Job. Each result contains the Job's identifier, final state, name, the Application it +belongs to, etc. These results are then used by Flink to determine whether Jobs should +be subject to recovery in highly-available Clusters. + +#### Key Group + +A Key Group is the atomic unit of key distribution and state assignment across parallel +[Sub-Tasks](#sub-task). Every key is mapped deterministically to a Key Group based on +`keyGroupIndex = MathUtils.murmurHash(key.hashCode()) % maxParallelism`. +This allows stateful [Operators](#operator) to rescale without rehashing individual keys. + +The total number of Key Groups is equal to the `maxParallelism` configuration, set at [Job](#flink-job) +level or overridden at [Operator](#operator) level. +A contiguous range of Key Groups is assigned to each [Sub-Task](#sub-task), and Key Groups are evenly +distributed across all [Sub-Tasks](#sub-task). #### Logical Graph -A logical graph is a directed graph where the nodes are [Operators](#operator) -and the edges define input/output-relationships of the operators and correspond -to data streams or data sets. A logical graph is created by submitting jobs -from a [Flink Application](#flink-application). +A Logical Graph is a Directed Acyclic Graph (DAG) where the nodes are [Operators](#operator) +and the edges define input/output relationships of the Operators and correspond +to data streams or data sets. A Logical Graph is created by submitting Jobs +from a [Flink Application](#flink-application). For the Table API and SQL, the Logical Graph is the +result of parsing and optimizing the [Table Program](#table-program) in the table planner. -Logical graphs are also often referred to as *dataflow graphs*. +Logical Graphs are also often referred to as *Dataflow Graphs* or, for the DataStream API, as +*StreamGraphs*. A Logical Graph is optimized into a [Job Graph](#job-graph) before execution. #### Managed State -Managed State describes application state which has been registered with the framework. For -Managed State, Apache Flink will take care about persistence and rescaling among other things. +Managed State describes Application State which has been registered with the framework. This includes +both [keyed state]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#using-keyed-state) and +non-keyed state (also known as [Operator State]({{< ref "docs/dev/datastream/fault-tolerance/state" >}}#operator-state)). +For Managed State, Apache Flink takes care of persistence and rescaling, among other things. #### Operator -Node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, which is -usually executed by a [Function](#function). Sources and Sinks are special Operators for data -ingestion and data egress. +A node of a [Logical Graph](#logical-graph). An Operator performs a certain operation, such as a join, +an aggregation or a stateless transformation, which is usually executed by a [Function](#function). + +Sources and Sinks are special Operators for data ingestion and data egress: a Logical Graph always +begins with one or more Source Operators and ends with one or more Sink Operators. + +Note that parts of the Flink documentation and of the Web UI use the term *Operator* loosely, also +referring to a [Task](#task) or a [Sub-Task](#sub-task), leaving the precise meaning to be inferred +from the context. #### Operator Chain An Operator Chain consists of two or more consecutive [Operators](#operator) without any repartitioning in between. Operators within the same Operator Chain forward records to each other -directly without going through serialization or Flink's network stack. +directly without going through serialization or Flink's network stack, which removes the overhead of +the handover between them. + +An Operator Chain becomes a single [Task](#task) in the [Job Graph](#job-graph). Chains are +recognizable in graphical representations of the Job Graph, such as the Flink Web UI, because the +name of the Task is the composition of the names of the chained Operators. + +See also [Flink Architecture: Tasks and Operator Chains]({{< ref "docs/concepts/flink-architecture" >}}#tasks-and-operator-chains). + +#### Parallelism + +The number of parallel flows Flink uses to process the data, and therefore the way a [Flink +Job](#flink-job) scales horizontally. The Parallelism of an [Operator](#operator) determines the +number of [Sub-Tasks](#sub-task) and of [Physical Partitions](#partition) it is executed with. + +The *Job Parallelism* is the default Parallelism of all Operators of a Job. The *Operator +Parallelism* may override it for an individual Operator. + +Parallelism is a property of the Job, independent of the number of [Flink +TaskManagers](#flink-taskmanager) in the [Flink Cluster](#flink-cluster). #### Partition -A partition is an independent subset of the overall data stream or data set. A data stream or -data set is divided into partitions by assigning each [record](#Record) to one or more partitions. -Partitions of data streams or data sets are consumed by [Tasks](#task) during runtime. A -transformation which changes the way a data stream or data set is partitioned is often called -repartitioning. +A Partition is an independent subset of the overall data stream or data set. A data stream or +data set is divided into Partitions by assigning each record to one or more Partitions. +A [Transformation](#transformation) which changes the way a data stream or data set is partitioned is +often called repartitioning. + +*Logical Partitioning* is how records and State are divided in the [Logical +Graph](#logical-graph) and the [Job Graph](#job-graph), in order to implement the semantics of an +operation. A `JOIN` or `GROUP BY` in SQL, or a `keyBy()` in the DataStream API, for example, requires the +data to be logically partitioned by a key, and the number of Logical Partitions is then the number of +distinct keys. Keyed State is isolated per Logical Partition: a [Function](#function) can only access +the State of the key of the record or timer it is currently processing. Operator State and Broadcast +State, in contrast, are not keyed. + +*Physical Partitioning* is how records and State are divided in the [Physical +Graph](#physical-graph), across the [Sub-Tasks](#sub-task) that Flink executes in parallel. Each +Sub-Task handles exactly one Physical Partition and holds only the State belonging to it, so the +number of Physical Partitions equals the [Parallelism](#parallelism) of the [Operators](#operator) +the Sub-Task implements. Physical Partitioning follows from Logical Partitioning: in a stream +partitioned by key, each Physical Partition holds a fixed subset of the keys. + +Note that the Flink documentation uses the word *partition* both for these internal Partitions and +for the partitions of an external system, such as the Kafka partitions of a source topic. The +intended meaning has to be inferred from the context. #### Physical Graph -A physical graph is the result of translating a [Logical Graph](#logical-graph) for execution in a -distributed runtime. The nodes are [Tasks](#task) and the edges indicate input/output-relationships -or [partitions](#partition) of data streams or data sets. +A Physical Graph is the result of translating a [Job Graph](#job-graph) for execution in a +distributed runtime, taking [Parallelism](#parallelism) into account. The nodes are +[Sub-Tasks](#sub-task) and the edges are the [Channels](#channel) connecting them. + +Physical Graphs are also referred to as *Parallel Dataflows* or as *ExecutionGraphs*. #### Record Records are the constituent elements of a data set or data stream. [Operators](#operator) and -[Functions](#Function) receive records as input and emit records as output. +[Functions](#function) receive records as input and emit records as output. #### (Runtime) Execution Mode -DataStream API programs can be executed in one of two execution modes: `BATCH` -or `STREAMING`. See [Execution Mode]({{< ref "/docs/dev/datastream/execution_mode" >}}) for more details. +DataStream API programs can be executed in one of two Execution Modes: `BATCH` +or `STREAMING`. See [Execution Mode]({{< ref "docs/dev/datastream/execution_mode" >}}) for more details. + +In `STREAMING` mode, Flink processes unbounded data as it arrives, uses [Watermarks](#watermark) to implement +event-time semantics, keeps State in the [State Backend](#state-backend) and relies on +[Checkpoints](#checkpoint) for fault tolerance. + +In `BATCH` mode, Flink processes a bounded data set with a known beginning and end. Operators may +consume their entire input before emitting any output, and Watermarks are not used for event-time +semantics. The configured State Backend is ignored: the input of a keyed operation is instead grouped +by key through sorting, so that Flink only has to hold the State of one key at a time, spilling to +local disk when memory is insufficient. + +Note that a bounded data set can also be processed in `STREAMING` mode, for example by setting Review Comment: Worth mentioning the impact? E.g, in Kafka the job will stop once it consumes the latest offset at the time of job start? Or, mention the reasons why you would use streaming mode on a bounded dataset (e.g to follow event-time semantics). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
