Ufuk Celebi created FLINK-3779:
----------------------------------

             Summary: Add support for queryable state
                 Key: FLINK-3779
                 URL: https://issues.apache.org/jira/browse/FLINK-3779
             Project: Flink
          Issue Type: Improvement
          Components: Distributed Runtime
            Reporter: Ufuk Celebi
            Assignee: Ufuk Celebi


Flink offers state abstractions for user functions in order to guarantee 
fault-tolerant processing of streams. Users can work with both non-partitioned 
(Checkpointed interface) and partitioned state 
(getRuntimeContext().getState(ValueStateDescriptor) and other variants).

The partitioned state interface provides access to different types of state 
that are all scoped to the key of the current input element. This type of state 
can only be used on a KeyedStream, which is created via stream.keyBy().

Currently, all of this state is internal to Flink and used in order to provide 
processing guarantees in failure cases (e.g. exactly-once processing).

The goal of Queryable State is to expose this state outside of Flink by 
supporting queries against the partitioned key value state.

This will help to eliminate the need for distributed operations/transactions 
with external systems such as key-value stores which are often the bottleneck 
in practice. Exposing the local state to the outside moves a good part of the 
database work into the stream processor, allowing both high throughput queries 
and immediate access to the computed state.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to