[
https://issues.apache.org/jira/browse/FLINK-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241853#comment-16241853
]
ASF GitHub Bot commented on FLINK-7822:
---------------------------------------
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4966#discussion_r149337397
--- Diff: docs/dev/stream/state/queryable_state.md ---
@@ -32,38 +32,67 @@ under the License.
likely that there will be breaking API changes on the client side in the
upcoming Flink versions.
</div>
-In a nutshell, this feature allows users to query Flink's managed
partitioned state
-(see [Working with State]({{ site.baseurl }}/dev/stream/state/state.html))
from outside of
-Flink. For some scenarios, queryable state thus eliminates the need for
distributed
-operations/transactions with external systems such as key-value stores
which are often the
-bottleneck in practice.
+In a nutshell, this feature exposes Flink's managed keyed (partitioned)
state
+(see [Working with State]({{ site.baseurl }}/dev/stream/state/state.html))
to the outside world and
+allows the user to query a job's state from outside Flink. For some
scenarios, queryable state
+eliminates the need for distributed operations/transactions with external
systems such as key-value
+stores which are often the bottleneck in practice. In addition, this
feature may be particularly
+useful for debugging purposes.
<div class="alert alert-warning">
- <strong>Attention:</strong> Queryable state accesses keyed state from a
concurrent thread rather
- than synchronizing with the operator and potentially blocking its
operation. Since any state
- backend using Java heap space, e.g. MemoryStateBackend or
- FsStateBackend, does not work with copies when retrieving values but
instead directly
- references the stored values, read-modify-write patterns are unsafe and
may cause the
- queryable state server to fail due to concurrent modifications.
- The RocksDBStateBackend is safe from these issues.
+ <strong>Attention:</strong> When querying a state object, that object is
accessed from a concurrent
+ thread without any synchronization or copying. This is a design choice,
as any of the above would lead
+ to increased job latency, which we wanted to avoid. Since any state
backend using Java heap space,
+ <i>e.g.</i> <code>MemoryStateBackend</code> or
<code>FsStateBackend</code>, does not work
+ with copies when retrieving values but instead directly references the
stored values, read-modify-write
+ patterns are unsafe and may cause the queryable state server to fail due
to concurrent modifications.
+ The <code>RocksDBStateBackend</code> is safe from these issues.
</div>
+## Architecture
+
+Before showing how to use the Queryable State, it is useful to briefly
describe the entities that compose it.
+The Queryable State consists of three main entities:
+
+ 1. the `QueryableStateClient`, which (potentially) runs outside the Flink
cluster and submits the user queries,
+ 2. the `QueryableStateClientProxy`, which runs on each `TaskManager`
(*i.e.* inside the Flink cluster) and is responsible
+ for receiving the client's queries, fetching the requested state on his
behalf, and returning it to the client, and
--- End diff --
maybe "fetching the requested state from the responsible TaskManager"
> Add documentation for the new queryable state client.
> -----------------------------------------------------
>
> Key: FLINK-7822
> URL: https://issues.apache.org/jira/browse/FLINK-7822
> Project: Flink
> Issue Type: Sub-task
> Components: Documentation, Queryable State
> Affects Versions: 1.4.0
> Reporter: Kostas Kloudas
> Assignee: Kostas Kloudas
> Priority: Blocker
> Fix For: 1.4.0
>
>
> This has to include:
> 1) how to enable queryable state (jars)...
> 2) configuration parameters
> 3) new APIs
> 4) Guarantees/Semantics/Limitations (e.g. memory backend)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
